2023-11-27T14:51:30.553 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-27T14:51:30.554 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_choffman@teuthology --archive /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467805 --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/fs/misc}} --name choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi --verbose -- /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467805/orig.config.yaml 2023-11-27T14:51:30.611 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-27T14:51:30.697 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467805 branch: wip-choffman-62793 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/fs/misc}} email: null first_in_suite: false job_id: '7467805' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-choffman-62793 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 1c9565146c8647d4e00eb6c7ae626dc192680e7b subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.dir_split install: ceph: flavor: default sha1: 1c9565146c8647d4e00eb6c7ae626dc192680e7b kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62793 sha1: 5e4a743ad7fd1422ebd6c4dc950a21a303279207 owner: scheduled_choffman@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8430 sha1: 1c9565146c8647d4e00eb6c7ae626dc192680e7b sleep_before_teardown: 0 suite: fs suite_branch: wip-62793 suite_path: /home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa suite_relpath: qa suite_repo: https://github.com/chrisphoffman/ceph.git suite_sha1: 5e4a743ad7fd1422ebd6c4dc950a21a303279207 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi133.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi174.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi177.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/misc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 2442239d2653456406c25ae0c71b689c8f2657b6 timestamp: 2023-11-27_14:43:37 tube: smithi user: choffman verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-27T14:51:30.697 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa; will attempt to use it 2023-11-27T14:51:30.698 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks 2023-11-27T14:51:30.698 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-27T14:51:30.699 INFO:teuthology.task.internal:Checking packages... 2023-11-27T14:51:30.722 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '1c9565146c8647d4e00eb6c7ae626dc192680e7b' 2023-11-27T14:51:30.722 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-27T14:51:30.723 INFO:teuthology.packaging:ref: None 2023-11-27T14:51:30.723 INFO:teuthology.packaging:tag: None 2023-11-27T14:51:30.723 INFO:teuthology.packaging:branch: wip-choffman-62793 2023-11-27T14:51:30.723 INFO:teuthology.packaging:sha1: 1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T14:51:30.723 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-choffman-62793 2023-11-27T14:51:30.972 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-3906.g1c956514 2023-11-27T14:51:30.972 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-27T14:51:30.979 INFO:teuthology.task.internal:no buildpackages task found 2023-11-27T14:51:30.980 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-27T14:51:30.993 INFO:teuthology.task.internal:Saving configuration 2023-11-27T14:51:31.014 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-27T14:51:31.020 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-27T14:51:31.046 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467805', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-27 14:46:36.840520', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-27T14:51:31.070 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467805', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-27 14:46:36.836700', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-27T14:51:31.093 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467805', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-27 14:46:36.838604', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-27T14:51:31.093 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-27T14:51:31.123 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-11-27T14:51:31.123 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-11-27T14:51:31.123 INFO:teuthology.task.internal:roles: ubuntu@smithi177.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-11-27T14:51:31.123 INFO:teuthology.run_tasks:Running task console_log... 2023-11-27T14:51:31.229 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0c1eba8f70>, signals=[15]) 2023-11-27T14:51:31.230 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-27T14:51:31.235 INFO:teuthology.task.internal:Opening connections... 2023-11-27T14:51:31.236 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2023-11-27T14:51:31.237 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T14:51:31.305 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi174.front.sepia.ceph.com 2023-11-27T14:51:31.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T14:51:31.374 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi177.front.sepia.ceph.com 2023-11-27T14:51:31.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T14:51:31.439 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-27T14:51:31.445 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2023-11-27T14:51:31.466 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2023-11-27T14:51:31.469 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2023-11-27T14:51:31.526 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="Red Hat Enterprise Linux" 2023-11-27T14:51:31.527 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="8.6 (Ootpa)" 2023-11-27T14:51:31.527 INFO:teuthology.orchestra.run.smithi133.stdout:ID="rhel" 2023-11-27T14:51:31.527 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="fedora" 2023-11-27T14:51:31.527 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="8.6" 2023-11-27T14:51:31.527 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el8" 2023-11-27T14:51:31.527 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-27T14:51:31.527 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2023-11-27T14:51:31.527 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-27T14:51:31.527 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://www.redhat.com/" 2023-11-27T14:51:31.527 INFO:teuthology.orchestra.run.smithi133.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-27T14:51:31.527 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-27T14:51:31.527 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T14:51:31.528 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-27T14:51:31.528 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-27T14:51:31.528 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-27T14:51:31.528 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-27T14:51:31.529 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2023-11-27T14:51:31.556 DEBUG:teuthology.orchestra.run.smithi174:> uname -m 2023-11-27T14:51:31.578 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2023-11-27T14:51:31.578 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2023-11-27T14:51:31.638 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="Red Hat Enterprise Linux" 2023-11-27T14:51:31.638 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="8.6 (Ootpa)" 2023-11-27T14:51:31.638 INFO:teuthology.orchestra.run.smithi174.stdout:ID="rhel" 2023-11-27T14:51:31.639 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="fedora" 2023-11-27T14:51:31.639 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="8.6" 2023-11-27T14:51:31.639 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el8" 2023-11-27T14:51:31.639 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-27T14:51:31.639 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2023-11-27T14:51:31.639 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-27T14:51:31.639 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://www.redhat.com/" 2023-11-27T14:51:31.639 INFO:teuthology.orchestra.run.smithi174.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-27T14:51:31.640 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-27T14:51:31.640 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-27T14:51:31.640 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-27T14:51:31.640 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-27T14:51:31.640 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-27T14:51:31.640 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-27T14:51:31.640 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2023-11-27T14:51:31.664 DEBUG:teuthology.orchestra.run.smithi177:> uname -m 2023-11-27T14:51:31.685 INFO:teuthology.orchestra.run.smithi177.stdout:x86_64 2023-11-27T14:51:31.686 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release 2023-11-27T14:51:31.743 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="Red Hat Enterprise Linux" 2023-11-27T14:51:31.743 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="8.6 (Ootpa)" 2023-11-27T14:51:31.744 INFO:teuthology.orchestra.run.smithi177.stdout:ID="rhel" 2023-11-27T14:51:31.744 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE="fedora" 2023-11-27T14:51:31.744 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="8.6" 2023-11-27T14:51:31.744 INFO:teuthology.orchestra.run.smithi177.stdout:PLATFORM_ID="platform:el8" 2023-11-27T14:51:31.744 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-27T14:51:31.744 INFO:teuthology.orchestra.run.smithi177.stdout:ANSI_COLOR="0;31" 2023-11-27T14:51:31.744 INFO:teuthology.orchestra.run.smithi177.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-27T14:51:31.745 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://www.redhat.com/" 2023-11-27T14:51:31.745 INFO:teuthology.orchestra.run.smithi177.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-27T14:51:31.745 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-27T14:51:31.745 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-27T14:51:31.745 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-27T14:51:31.745 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-27T14:51:31.745 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-27T14:51:31.746 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-27T14:51:31.746 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server 2023-11-27T14:51:31.764 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-27T14:51:31.785 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-27T14:51:31.791 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-27T14:51:31.792 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2023-11-27T14:51:31.794 DEBUG:teuthology.orchestra.run.smithi174:> test '!' -e /home/ubuntu/cephtest 2023-11-27T14:51:31.796 DEBUG:teuthology.orchestra.run.smithi177:> test '!' -e /home/ubuntu/cephtest 2023-11-27T14:51:31.814 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-27T14:51:31.820 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-27T14:51:31.821 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2023-11-27T14:51:31.853 DEBUG:teuthology.orchestra.run.smithi174:> test -z $(ls -A /var/lib/ceph) 2023-11-27T14:51:31.855 DEBUG:teuthology.orchestra.run.smithi177:> test -z $(ls -A /var/lib/ceph) 2023-11-27T14:51:31.881 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-27T14:51:31.954 INFO:teuthology.run_tasks:Running task kernel... 2023-11-27T14:51:31.969 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-11-27T14:51:31.970 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2023-11-27T14:51:31.970 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2023-11-27T14:51:31.970 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-27T14:51:31.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T14:51:31.989 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-11-27T14:51:32.221 DEBUG:teuthology.task.kernel:sha1 for client.0 is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-27T14:51:32.221 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-27T14:51:32.221 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2023-11-27T14:51:32.240 INFO:teuthology.orchestra.run.smithi133.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-27T14:51:32.240 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-27T14:51:32.241 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-27T14:51:32.241 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum upgrade -y linux-firmware 2023-11-27T14:51:33.499 INFO:teuthology.orchestra.run.smithi133.stdout:Updating Subscription Management repositories. 2023-11-27T14:51:33.499 INFO:teuthology.orchestra.run.smithi133.stdout:Unable to read consumer identity 2023-11-27T14:51:34.910 INFO:teuthology.orchestra.run.smithi133.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2023-11-27T14:51:35.736 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2023-11-27T14:51:39.361 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 429 kB/s | 24 kB 00:00 2023-11-27T14:51:40.089 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2023-11-27T14:51:40.090 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2023-11-27T14:51:40.090 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2023-11-27T14:51:40.211 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.0... 2023-11-27T14:51:40.212 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-27T14:51:40.355 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/ 2023-11-27T14:51:40.355 DEBUG:teuthology.orchestra.run.smithi133:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/ --input-file=- 2023-11-27T14:51:40.358 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi133.front.sepia.ceph.com to complete... 2023-11-27T14:51:53.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27 14:51:53 URL:https://4.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/8/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [383178732/383178732] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-27T14:51:53.525 INFO:teuthology.orchestra.run.smithi133.stderr:FINISHED --2023-11-27 14:51:53-- 2023-11-27T14:51:53.526 INFO:teuthology.orchestra.run.smithi133.stderr:Total wall clock time: 13s 2023-11-27T14:51:53.526 INFO:teuthology.orchestra.run.smithi133.stderr:Downloaded: 1 files, 365M in 13s (28.5 MB/s) 2023-11-27T14:51:53.527 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.0... 2023-11-27T14:51:53.527 DEBUG:teuthology.orchestra.run.smithi133:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-27T14:51:55.102 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying... ######################################## 2023-11-27T14:51:55.159 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing... ######################################## 2023-11-27T14:51:55.247 INFO:teuthology.orchestra.run.smithi133.stdout:Updating / installing... 2023-11-27T14:52:33.479 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-11-27T14:52:35.038 INFO:teuthology.orchestra.run.smithi133.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-27T14:53:00.267 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi133.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-27T14:53:00.267 DEBUG:teuthology.orchestra.run.smithi133:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-27T14:53:00.323 INFO:teuthology.orchestra.run.smithi133.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-11-27T14:53:00.324 INFO:teuthology.orchestra.run.smithi133.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-11-27T14:53:00.324 INFO:teuthology.orchestra.run.smithi133.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-27T14:53:00.324 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-11-27T14:53:00.324 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-11-27T14:53:00.324 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-11-27T14:53:00.324 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-11-27T14:53:00.324 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-11-27T14:53:00.324 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-27T14:53:00.324 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-27T14:53:00.324 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-27T14:53:00.324 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-27T14:53:00.325 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-27T14:53:00.325 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-27T14:53:00.325 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-27T14:53:00.325 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-27T14:53:00.325 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-27T14:53:00.325 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-27T14:53:00.325 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-27T14:53:00.325 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-27T14:53:00.325 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-27T14:53:00.325 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-27T14:53:00.326 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-27T14:53:00.326 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-27T14:53:00.326 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-27T14:53:00.326 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-27T14:53:00.326 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-27T14:53:00.326 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-27T14:53:00.326 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-11-27T14:53:00.326 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-11-27T14:53:00.326 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-11-27T14:53:00.326 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-27T14:53:00.326 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-11-27T14:53:00.326 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-27T14:53:00.327 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-27T14:53:00.327 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-27T14:53:00.327 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-11-27T14:53:00.327 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-27T14:53:00.327 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-27T14:53:00.327 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-27T14:53:00.327 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-11-27T14:53:00.327 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-27T14:53:00.327 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-27T14:53:00.327 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-27T14:53:00.327 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-27T14:53:00.327 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-27T14:53:00.327 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-27T14:53:00.328 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-27T14:53:00.328 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-27T14:53:00.328 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-27T14:53:00.328 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-27T14:53:00.328 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-27T14:53:00.328 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-27T14:53:00.328 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-27T14:53:00.328 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-27T14:53:00.328 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-27T14:53:00.328 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-27T14:53:00.328 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-27T14:53:00.328 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-27T14:53:00.328 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-27T14:53:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-27T14:53:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-27T14:53:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-27T14:53:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-27T14:53:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-27T14:53:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-27T14:53:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-27T14:53:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-27T14:53:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-27T14:53:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-27T14:53:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-27T14:53:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-27T14:53:00.329 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-27T14:53:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-27T14:53:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-27T14:53:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-27T14:53:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-27T14:53:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-27T14:53:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-27T14:53:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-27T14:53:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-27T14:53:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-27T14:53:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-27T14:53:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-27T14:53:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-27T14:53:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-27T14:53:00.330 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-11-27T14:53:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-11-27T14:53:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-27T14:53:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-27T14:53:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-11-27T14:53:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-27T14:53:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-27T14:53:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-27T14:53:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-27T14:53:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-27T14:53:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-11-27T14:53:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-27T14:53:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-27T14:53:00.331 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-11-27T14:53:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-27T14:53:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-27T14:53:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-27T14:53:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-27T14:53:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-27T14:53:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-27T14:53:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-27T14:53:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-27T14:53:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-27T14:53:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-27T14:53:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-27T14:53:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-27T14:53:00.332 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-27T14:53:00.333 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-27T14:53:00.333 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-27T14:53:00.333 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-27T14:53:00.333 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-27T14:53:00.333 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-27T14:53:00.333 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-27T14:53:00.333 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-27T14:53:00.333 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-27T14:53:00.333 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-27T14:53:00.333 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-27T14:53:00.333 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-27T14:53:00.333 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-27T14:53:00.333 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-27T14:53:00.334 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-27T14:53:00.334 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-27T14:53:00.334 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-27T14:53:00.334 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-27T14:53:00.334 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-27T14:53:00.334 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-27T14:53:00.334 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-27T14:53:00.334 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-27T14:53:00.334 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-27T14:53:00.334 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-11-27T14:53:00.334 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-27T14:53:00.334 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-27T14:53:00.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-27T14:53:00.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-27T14:53:00.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-27T14:53:00.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-27T14:53:00.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-27T14:53:00.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-27T14:53:00.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-27T14:53:00.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-27T14:53:00.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-27T14:53:00.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-27T14:53:00.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-27T14:53:00.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-27T14:53:00.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-27T14:53:00.335 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-27T14:53:00.336 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-27T14:53:00.336 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-11-27T14:53:00.336 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-27T14:53:00.336 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-11-27T14:53:00.336 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-11-27T14:53:00.336 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-27T14:53:00.336 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-27T14:53:00.336 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-11-27T14:53:00.336 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-27T14:53:00.336 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-11-27T14:53:00.336 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-11-27T14:53:00.336 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-27T14:53:00.337 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-11-27T14:53:00.337 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-11-27T14:53:00.337 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-27T14:53:00.337 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-27T14:53:00.337 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-11-27T14:53:00.337 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-11-27T14:53:00.337 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-27T14:53:00.337 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-27T14:53:00.337 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-11-27T14:53:00.337 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-27T14:53:00.337 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-27T14:53:00.337 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-27T14:53:00.337 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-11-27T14:53:00.338 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-11-27T14:53:00.338 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-27T14:53:00.338 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-11-27T14:53:00.338 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-11-27T14:53:00.338 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-27T14:53:00.338 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-11-27T14:53:00.338 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-27T14:53:00.338 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-11-27T14:53:00.338 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-27T14:53:00.338 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-27T14:53:00.338 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-27T14:53:00.338 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-27T14:53:00.338 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-27T14:53:00.339 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-11-27T14:53:00.339 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-27T14:53:00.339 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-27T14:53:00.339 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-27T14:53:00.339 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-27T14:53:00.339 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-27T14:53:00.339 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-27T14:53:00.341 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-27T14:53:00.341 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-11-27T14:53:00.341 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-27T14:53:00.341 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-27T14:53:00.341 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-27T14:53:00.341 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-11-27T14:53:00.341 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-11-27T14:53:00.341 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-27T14:53:00.341 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-27T14:53:00.341 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-27T14:53:00.342 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-27T14:53:00.342 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-27T14:53:00.342 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-11-27T14:53:00.342 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-11-27T14:53:00.342 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-11-27T14:53:00.342 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-27T14:53:00.342 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-27T14:53:00.342 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-27T14:53:00.342 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-27T14:53:00.342 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-27T14:53:00.342 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-27T14:53:00.342 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-11-27T14:53:00.342 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-11-27T14:53:00.343 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-27T14:53:00.343 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-27T14:53:00.343 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-11-27T14:53:00.343 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-11-27T14:53:00.343 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-11-27T14:53:00.343 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-27T14:53:00.343 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-27T14:53:00.343 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-27T14:53:00.343 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-27T14:53:00.343 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-27T14:53:00.343 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-11-27T14:53:00.343 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-27T14:53:00.343 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-11-27T14:53:00.343 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-27T14:53:00.344 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-11-27T14:53:00.344 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-27T14:53:00.344 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-11-27T14:53:00.344 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-11-27T14:53:00.344 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-27T14:53:00.344 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-11-27T14:53:00.344 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-27T14:53:00.344 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-11-27T14:53:00.344 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-27T14:53:00.344 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-27T14:53:00.344 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-27T14:53:00.344 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-27T14:53:00.344 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-27T14:53:00.344 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-27T14:53:00.345 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-27T14:53:00.345 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-27T14:53:00.345 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-27T14:53:00.345 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-27T14:53:00.345 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-27T14:53:00.345 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-27T14:53:00.345 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-27T14:53:00.345 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-11-27T14:53:00.345 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-11-27T14:53:00.345 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-27T14:53:00.345 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-11-27T14:53:00.345 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-27T14:53:00.345 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-27T14:53:00.346 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-11-27T14:53:00.346 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-27T14:53:00.346 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-27T14:53:00.346 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-27T14:53:00.346 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-27T14:53:00.346 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-27T14:53:00.346 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-27T14:53:00.346 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-27T14:53:00.346 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-27T14:53:00.346 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-27T14:53:00.346 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-27T14:53:00.346 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-27T14:53:00.346 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-27T14:53:00.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-27T14:53:00.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-27T14:53:00.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-27T14:53:00.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-27T14:53:00.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-27T14:53:00.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-27T14:53:00.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-27T14:53:00.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-27T14:53:00.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-27T14:53:00.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-27T14:53:00.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-27T14:53:00.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-27T14:53:00.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-27T14:53:00.347 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-27T14:53:00.348 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-27T14:53:00.348 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-27T14:53:00.348 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-27T14:53:00.348 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-27T14:53:00.348 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-27T14:53:00.348 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-27T14:53:00.348 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-27T14:53:00.348 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-27T14:53:00.349 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-27T14:53:00.349 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-27T14:53:00.349 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-27T14:53:00.349 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-27T14:53:00.349 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-27T14:53:00.349 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-27T14:53:00.349 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-27T14:53:00.349 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-27T14:53:00.349 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-27T14:53:00.349 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-27T14:53:00.349 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-27T14:53:00.349 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-27T14:53:00.349 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-27T14:53:00.349 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-27T14:53:00.350 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-27T14:53:00.350 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-27T14:53:00.350 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-27T14:53:00.350 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-27T14:53:00.350 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-27T14:53:00.350 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-27T14:53:00.350 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-27T14:53:00.350 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-27T14:53:00.350 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-27T14:53:00.350 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-27T14:53:00.350 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-27T14:53:00.350 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-27T14:53:00.351 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-27T14:53:00.351 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-27T14:53:00.351 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-27T14:53:00.351 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-27T14:53:00.351 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-11-27T14:53:00.351 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-27T14:53:00.351 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-27T14:53:00.351 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-27T14:53:00.351 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-11-27T14:53:00.351 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-27T14:53:00.351 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-27T14:53:00.351 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-27T14:53:00.351 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-27T14:53:00.352 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-27T14:53:00.352 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-27T14:53:00.352 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-27T14:53:00.352 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-27T14:53:00.352 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-27T14:53:00.352 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-27T14:53:00.352 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-27T14:53:00.352 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-27T14:53:00.352 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-27T14:53:00.352 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-27T14:53:00.352 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-27T14:53:00.352 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-27T14:53:00.352 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-27T14:53:00.353 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-27T14:53:00.353 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-27T14:53:00.353 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-27T14:53:00.353 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-27T14:53:00.353 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-27T14:53:00.353 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-27T14:53:00.353 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-27T14:53:00.353 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-27T14:53:00.353 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-27T14:53:00.353 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-27T14:53:00.353 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-27T14:53:00.353 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-27T14:53:00.353 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-27T14:53:00.354 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-27T14:53:00.354 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-27T14:53:00.354 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-27T14:53:00.354 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-27T14:53:00.354 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-27T14:53:00.354 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-27T14:53:00.354 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-27T14:53:00.354 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-27T14:53:00.354 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-27T14:53:00.354 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-27T14:53:00.354 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-27T14:53:00.354 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-27T14:53:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-27T14:53:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-27T14:53:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-27T14:53:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-27T14:53:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-27T14:53:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-27T14:53:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-27T14:53:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-27T14:53:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-27T14:53:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-27T14:53:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-27T14:53:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-27T14:53:00.355 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-27T14:53:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-27T14:53:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-27T14:53:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-27T14:53:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-27T14:53:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-27T14:53:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-27T14:53:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-27T14:53:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-27T14:53:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-27T14:53:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-27T14:53:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-27T14:53:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-27T14:53:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-27T14:53:00.356 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-27T14:53:00.357 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-27T14:53:00.357 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-27T14:53:00.357 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-27T14:53:00.357 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-27T14:53:00.357 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-27T14:53:00.357 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-27T14:53:00.357 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-27T14:53:00.357 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-27T14:53:00.357 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-27T14:53:00.357 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-27T14:53:00.357 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-27T14:53:00.357 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-27T14:53:00.357 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-27T14:53:00.358 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-27T14:53:00.358 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-27T14:53:00.358 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-27T14:53:00.358 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-27T14:53:00.358 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-27T14:53:00.358 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-27T14:53:00.358 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-11-27T14:53:00.358 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-27T14:53:00.358 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-27T14:53:00.358 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-27T14:53:00.358 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-27T14:53:00.358 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-27T14:53:00.358 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-27T14:53:00.358 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-27T14:53:00.359 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-27T14:53:00.359 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-27T14:53:00.359 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-27T14:53:00.359 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-27T14:53:00.359 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-27T14:53:00.359 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-27T14:53:00.359 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-27T14:53:00.359 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-27T14:53:00.359 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-27T14:53:00.359 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-27T14:53:00.359 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-27T14:53:00.359 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-27T14:53:00.359 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-27T14:53:00.360 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-27T14:53:00.360 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-27T14:53:00.360 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-27T14:53:00.360 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-27T14:53:00.360 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-27T14:53:00.360 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-27T14:53:00.360 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-27T14:53:00.360 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-27T14:53:00.360 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-27T14:53:00.360 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-27T14:53:00.360 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-27T14:53:00.360 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-27T14:53:00.360 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-27T14:53:00.360 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-27T14:53:00.361 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-27T14:53:00.361 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-27T14:53:00.361 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-27T14:53:00.361 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-27T14:53:00.361 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-27T14:53:00.361 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-27T14:53:00.361 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-27T14:53:00.361 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-11-27T14:53:00.361 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-11-27T14:53:00.361 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-27T14:53:00.361 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-27T14:53:00.361 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-11-27T14:53:00.361 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-27T14:53:00.361 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-27T14:53:00.362 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-27T14:53:00.362 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-27T14:53:00.362 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-27T14:53:00.362 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-27T14:53:00.362 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-27T14:53:00.362 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-27T14:53:00.362 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-27T14:53:00.362 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-27T14:53:00.362 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-27T14:53:00.362 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-27T14:53:00.362 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-27T14:53:00.362 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-27T14:53:00.362 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-27T14:53:00.363 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-27T14:53:00.363 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-27T14:53:00.363 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-27T14:53:00.363 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-27T14:53:00.363 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-27T14:53:00.363 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-27T14:53:00.363 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-27T14:53:00.363 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-27T14:53:00.363 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-11-27T14:53:00.363 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-27T14:53:00.363 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-27T14:53:00.363 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-27T14:53:00.363 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-27T14:53:00.363 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-11-27T14:53:00.364 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-11-27T14:53:00.364 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-27T14:53:00.364 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-27T14:53:00.364 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-27T14:53:00.364 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-27T14:53:00.364 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-27T14:53:00.364 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-27T14:53:00.364 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-27T14:53:00.364 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-11-27T14:53:00.364 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-11-27T14:53:00.364 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-27T14:53:00.364 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-11-27T14:53:00.364 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-27T14:53:00.365 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-11-27T14:53:00.365 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-27T14:53:00.365 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-11-27T14:53:00.365 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-27T14:53:00.365 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-11-27T14:53:00.365 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-11-27T14:53:00.365 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-27T14:53:00.365 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-11-27T14:53:00.365 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-27T14:53:00.365 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-11-27T14:53:00.365 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-27T14:53:00.365 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-11-27T14:53:00.365 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-27T14:53:00.365 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-11-27T14:53:00.366 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-27T14:53:00.366 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-11-27T14:53:00.366 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-11-27T14:53:00.366 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-27T14:53:00.366 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-27T14:53:00.366 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-27T14:53:00.366 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-27T14:53:00.366 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-11-27T14:53:00.366 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-27T14:53:00.366 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-11-27T14:53:00.366 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-27T14:53:00.366 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-27T14:53:00.366 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-27T14:53:00.367 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-27T14:53:00.367 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-27T14:53:00.367 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-27T14:53:00.367 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-27T14:53:00.367 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-27T14:53:00.367 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-11-27T14:53:00.367 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-27T14:53:00.367 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-11-27T14:53:00.367 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-27T14:53:00.367 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-27T14:53:00.367 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-27T14:53:00.367 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-27T14:53:00.367 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-27T14:53:00.368 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-11-27T14:53:00.368 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-11-27T14:53:00.368 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-27T14:53:00.368 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-11-27T14:53:00.368 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-27T14:53:00.368 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-27T14:53:00.368 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-27T14:53:00.368 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-27T14:53:00.368 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-11-27T14:53:00.368 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-27T14:53:00.368 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-27T14:53:00.368 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-27T14:53:00.368 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-27T14:53:00.368 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-27T14:53:00.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-27T14:53:00.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-27T14:53:00.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-27T14:53:00.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-11-27T14:53:00.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-11-27T14:53:00.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-27T14:53:00.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-27T14:53:00.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-27T14:53:00.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-27T14:53:00.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-27T14:53:00.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-27T14:53:00.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-27T14:53:00.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-27T14:53:00.369 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-27T14:53:00.370 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-27T14:53:00.370 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-27T14:53:00.370 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-27T14:53:00.370 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-27T14:53:00.370 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-27T14:53:00.370 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-27T14:53:00.370 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-27T14:53:00.370 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-27T14:53:00.370 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-27T14:53:00.370 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-27T14:53:00.370 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-11-27T14:53:00.370 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-27T14:53:00.370 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-27T14:53:00.371 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-27T14:53:00.371 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-27T14:53:00.371 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-27T14:53:00.371 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-11-27T14:53:00.371 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-11-27T14:53:00.371 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-27T14:53:00.371 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-27T14:53:00.371 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-27T14:53:00.371 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-27T14:53:00.371 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-27T14:53:00.371 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-27T14:53:00.371 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-27T14:53:00.371 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-11-27T14:53:00.372 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-27T14:53:00.372 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-27T14:53:00.372 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-11-27T14:53:00.372 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-27T14:53:00.372 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-27T14:53:00.372 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-27T14:53:00.372 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-27T14:53:00.372 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-11-27T14:53:00.372 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-11-27T14:53:00.372 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-27T14:53:00.372 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-27T14:53:00.372 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-27T14:53:00.372 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-27T14:53:00.372 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-11-27T14:53:00.373 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-27T14:53:00.373 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-11-27T14:53:00.373 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-27T14:53:00.373 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-27T14:53:00.373 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-27T14:53:00.373 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-27T14:53:00.373 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-27T14:53:00.373 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-27T14:53:00.373 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-27T14:53:00.373 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-27T14:53:00.373 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-11-27T14:53:00.373 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-27T14:53:00.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-27T14:53:00.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-11-27T14:53:00.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-27T14:53:00.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-27T14:53:00.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-27T14:53:00.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-11-27T14:53:00.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-27T14:53:00.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-11-27T14:53:00.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-27T14:53:00.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-27T14:53:00.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-27T14:53:00.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-27T14:53:00.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-27T14:53:00.374 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-27T14:53:00.375 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-27T14:53:00.375 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-11-27T14:53:00.375 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-27T14:53:00.375 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-11-27T14:53:00.375 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-11-27T14:53:00.375 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-27T14:53:00.375 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-11-27T14:53:00.375 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-11-27T14:53:00.375 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-27T14:53:00.375 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-27T14:53:00.375 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-27T14:53:00.375 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-11-27T14:53:00.375 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-27T14:53:00.376 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-27T14:53:00.376 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-27T14:53:00.376 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-11-27T14:53:00.376 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-11-27T14:53:00.376 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-27T14:53:00.376 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-27T14:53:00.376 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-27T14:53:00.376 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-27T14:53:00.376 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-11-27T14:53:00.376 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-27T14:53:00.376 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-27T14:53:00.376 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-27T14:53:00.376 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-27T14:53:00.376 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-11-27T14:53:00.377 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-27T14:53:00.377 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-11-27T14:53:00.377 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-27T14:53:00.377 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-11-27T14:53:00.377 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-27T14:53:00.377 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-11-27T14:53:00.377 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-27T14:53:00.377 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-11-27T14:53:00.377 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-27T14:53:00.377 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-27T14:53:00.377 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-27T14:53:00.377 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-11-27T14:53:00.377 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-11-27T14:53:00.378 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-27T14:53:00.378 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-27T14:53:00.378 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-27T14:53:00.378 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-27T14:53:00.378 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-27T14:53:00.378 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-11-27T14:53:00.378 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-27T14:53:00.378 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-27T14:53:00.378 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-11-27T14:53:00.378 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-27T14:53:00.378 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-27T14:53:00.378 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-27T14:53:00.378 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-27T14:53:00.378 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-27T14:53:00.379 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-11-27T14:53:00.379 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-27T14:53:00.379 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-27T14:53:00.379 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-11-27T14:53:00.379 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-11-27T14:53:00.379 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-27T14:53:00.379 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-11-27T14:53:00.379 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-27T14:53:00.379 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-27T14:53:00.379 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-27T14:53:00.379 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-27T14:53:00.379 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-27T14:53:00.379 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-27T14:53:00.380 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-27T14:53:00.380 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-27T14:53:00.380 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-27T14:53:00.380 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-27T14:53:00.380 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-11-27T14:53:00.380 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-11-27T14:53:00.380 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-27T14:53:00.380 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-11-27T14:53:00.380 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-11-27T14:53:00.380 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-27T14:53:00.380 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-27T14:53:00.380 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-27T14:53:00.380 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-27T14:53:00.380 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-27T14:53:00.381 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-27T14:53:00.381 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-11-27T14:53:00.381 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-27T14:53:00.381 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-27T14:53:00.381 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-27T14:53:00.381 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-11-27T14:53:00.381 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-11-27T14:53:00.381 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-27T14:53:00.381 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-27T14:53:00.381 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-27T14:53:00.381 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-11-27T14:53:00.381 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-27T14:53:00.381 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-11-27T14:53:00.382 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-27T14:53:00.382 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-11-27T14:53:00.382 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-27T14:53:00.382 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-11-27T14:53:00.382 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-27T14:53:00.382 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-11-27T14:53:00.382 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-27T14:53:00.382 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-11-27T14:53:00.382 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-27T14:53:00.382 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-11-27T14:53:00.382 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-27T14:53:00.382 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-11-27T14:53:00.382 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-27T14:53:00.383 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-27T14:53:00.383 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-27T14:53:00.383 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-27T14:53:00.383 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-27T14:53:00.383 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-27T14:53:00.383 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-11-27T14:53:00.383 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-27T14:53:00.383 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-27T14:53:00.383 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-27T14:53:00.383 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-11-27T14:53:00.383 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-27T14:53:00.383 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-27T14:53:00.383 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-27T14:53:00.383 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-27T14:53:00.384 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-27T14:53:00.384 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-27T14:53:00.384 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-11-27T14:53:00.384 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-27T14:53:00.384 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-11-27T14:53:00.384 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-27T14:53:00.384 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-27T14:53:00.384 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-11-27T14:53:00.384 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-27T14:53:00.384 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-27T14:53:00.384 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-11-27T14:53:00.384 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-27T14:53:00.384 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-11-27T14:53:00.385 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-27T14:53:00.385 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-11-27T14:53:00.385 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-27T14:53:00.385 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-27T14:53:00.385 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-11-27T14:53:00.385 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-27T14:53:00.385 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-27T14:53:00.385 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-27T14:53:00.385 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-27T14:53:00.385 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-27T14:53:00.385 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-11-27T14:53:00.385 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-27T14:53:00.385 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-11-27T14:53:00.385 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-27T14:53:00.386 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-27T14:53:00.386 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-27T14:53:00.386 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-27T14:53:00.386 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-11-27T14:53:00.386 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-27T14:53:00.386 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-11-27T14:53:00.386 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-27T14:53:00.386 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-11-27T14:53:00.386 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-27T14:53:00.386 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-11-27T14:53:00.386 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-27T14:53:00.386 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-27T14:53:00.386 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-11-27T14:53:00.387 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-27T14:53:00.387 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-27T14:53:00.387 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-27T14:53:00.387 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-27T14:53:00.387 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-11-27T14:53:00.387 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-27T14:53:00.387 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-27T14:53:00.387 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-27T14:53:00.387 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-11-27T14:53:00.387 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-27T14:53:00.387 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-27T14:53:00.387 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-27T14:53:00.387 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-27T14:53:00.387 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-11-27T14:53:00.388 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-27T14:53:00.388 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-11-27T14:53:00.388 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-27T14:53:00.388 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-11-27T14:53:00.388 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-27T14:53:00.388 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-27T14:53:00.388 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-11-27T14:53:00.388 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-27T14:53:00.388 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-27T14:53:00.388 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-11-27T14:53:00.388 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-27T14:53:00.388 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-27T14:53:00.388 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-11-27T14:53:00.389 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-27T14:53:00.389 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-11-27T14:53:00.389 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-27T14:53:00.389 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-27T14:53:00.389 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-11-27T14:53:00.389 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-27T14:53:00.389 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-27T14:53:00.389 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-27T14:53:00.389 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-11-27T14:53:00.389 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-27T14:53:00.389 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-27T14:53:00.389 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-27T14:53:00.389 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-27T14:53:00.389 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-27T14:53:00.390 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-11-27T14:53:00.390 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-27T14:53:00.390 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-27T14:53:00.390 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-11-27T14:53:00.390 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-27T14:53:00.390 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-27T14:53:00.390 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-11-27T14:53:00.390 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-27T14:53:00.390 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-27T14:53:00.390 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-27T14:53:00.390 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-27T14:53:00.390 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-27T14:53:00.390 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-27T14:53:00.391 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-27T14:53:00.391 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-27T14:53:00.391 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-11-27T14:53:00.391 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-27T14:53:00.391 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-27T14:53:00.391 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-27T14:53:00.391 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-27T14:53:00.391 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-27T14:53:00.391 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-27T14:53:00.391 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-27T14:53:00.391 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-27T14:53:00.391 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-27T14:53:00.391 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-27T14:53:00.392 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-27T14:53:00.392 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-27T14:53:00.392 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-11-27T14:53:00.392 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-27T14:53:00.392 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-27T14:53:00.392 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-27T14:53:00.392 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-27T14:53:00.392 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-27T14:53:00.392 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-27T14:53:00.392 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-27T14:53:00.392 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-27T14:53:00.392 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-27T14:53:00.392 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-11-27T14:53:00.392 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-11-27T14:53:00.393 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-27T14:53:00.393 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-27T14:53:00.393 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-27T14:53:00.393 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-11-27T14:53:00.393 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-27T14:53:00.393 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-27T14:53:00.393 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-27T14:53:00.393 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-27T14:53:00.393 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-27T14:53:00.393 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-27T14:53:00.393 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-11-27T14:53:00.393 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-27T14:53:00.394 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-11-27T14:53:00.394 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-11-27T14:53:00.394 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-27T14:53:00.394 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-11-27T14:53:00.394 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-27T14:53:00.394 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-11-27T14:53:00.394 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-27T14:53:00.394 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-11-27T14:53:00.394 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-11-27T14:53:00.394 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-27T14:53:00.394 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-11-27T14:53:00.394 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-11-27T14:53:00.395 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-27T14:53:00.395 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-27T14:53:00.395 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-11-27T14:53:00.395 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-27T14:53:00.395 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-27T14:53:00.395 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-27T14:53:00.395 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-27T14:53:00.395 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-11-27T14:53:00.395 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-11-27T14:53:00.395 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-27T14:53:00.395 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-11-27T14:53:00.395 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-27T14:53:00.395 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-27T14:53:00.396 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-11-27T14:53:00.396 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-27T14:53:00.396 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-27T14:53:00.396 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-27T14:53:00.396 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-11-27T14:53:00.396 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-11-27T14:53:00.396 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-27T14:53:00.396 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-11-27T14:53:00.396 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-11-27T14:53:00.396 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-27T14:53:00.396 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-27T14:53:00.396 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-27T14:53:00.396 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-27T14:53:00.396 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-27T14:53:00.397 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-27T14:53:00.397 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-27T14:53:00.397 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-27T14:53:00.397 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-27T14:53:00.397 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-27T14:53:00.397 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-27T14:53:00.397 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-27T14:53:00.397 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-11-27T14:53:00.397 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-27T14:53:00.397 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-27T14:53:00.397 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-27T14:53:00.397 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-27T14:53:00.397 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-27T14:53:00.397 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-27T14:53:00.398 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-27T14:53:00.398 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-27T14:53:00.398 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-27T14:53:00.398 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-27T14:53:00.398 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-27T14:53:00.398 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-27T14:53:00.398 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-11-27T14:53:00.398 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-27T14:53:00.398 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-27T14:53:00.398 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-11-27T14:53:00.398 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-11-27T14:53:00.398 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-27T14:53:00.398 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-27T14:53:00.399 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-27T14:53:00.399 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-27T14:53:00.399 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-11-27T14:53:00.399 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-27T14:53:00.399 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-27T14:53:00.399 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-11-27T14:53:00.399 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-27T14:53:00.399 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-27T14:53:00.399 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-27T14:53:00.399 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-27T14:53:00.399 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-27T14:53:00.399 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-27T14:53:00.399 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-27T14:53:00.399 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-27T14:53:00.400 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-27T14:53:00.400 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-27T14:53:00.400 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-27T14:53:00.400 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-27T14:53:00.400 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-27T14:53:00.400 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-27T14:53:00.400 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-27T14:53:00.400 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-27T14:53:00.400 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-27T14:53:00.400 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-27T14:53:00.400 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-27T14:53:00.400 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-27T14:53:00.400 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-27T14:53:00.401 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-27T14:53:00.401 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-27T14:53:00.401 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-27T14:53:00.401 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-27T14:53:00.401 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-27T14:53:00.401 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-27T14:53:00.401 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-27T14:53:00.401 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-27T14:53:00.401 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-27T14:53:00.401 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-27T14:53:00.401 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-11-27T14:53:00.401 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-27T14:53:00.401 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-27T14:53:00.402 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-27T14:53:00.402 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-27T14:53:00.402 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-27T14:53:00.402 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-11-27T14:53:00.402 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-27T14:53:00.402 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-27T14:53:00.402 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-11-27T14:53:00.402 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-27T14:53:00.402 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-27T14:53:00.402 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-27T14:53:00.402 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-11-27T14:53:00.402 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-27T14:53:00.403 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-27T14:53:00.403 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-11-27T14:53:00.403 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-27T14:53:00.403 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-11-27T14:53:00.403 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-27T14:53:00.403 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-11-27T14:53:00.403 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-27T14:53:00.403 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-11-27T14:53:00.403 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-27T14:53:00.403 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-27T14:53:00.403 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-11-27T14:53:00.403 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-27T14:53:00.403 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-11-27T14:53:00.404 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-27T14:53:00.404 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-27T14:53:00.404 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-27T14:53:00.404 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-27T14:53:00.404 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-27T14:53:00.404 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-27T14:53:00.404 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-11-27T14:53:00.404 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-27T14:53:00.404 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-27T14:53:00.404 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-27T14:53:00.404 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-27T14:53:00.404 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-27T14:53:00.404 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-11-27T14:53:00.405 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-27T14:53:00.405 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-27T14:53:00.405 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-11-27T14:53:00.405 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-27T14:53:00.405 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-27T14:53:00.405 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-11-27T14:53:00.405 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-27T14:53:00.405 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-27T14:53:00.405 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-27T14:53:00.405 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-27T14:53:00.405 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-27T14:53:00.405 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-27T14:53:00.405 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-27T14:53:00.406 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-11-27T14:53:00.406 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-27T14:53:00.406 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-27T14:53:00.406 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-27T14:53:00.406 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-11-27T14:53:00.406 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-27T14:53:00.406 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-27T14:53:00.406 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-27T14:53:00.406 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-11-27T14:53:00.406 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-27T14:53:00.406 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-11-27T14:53:00.406 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-27T14:53:00.406 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-11-27T14:53:00.407 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-27T14:53:00.407 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-27T14:53:00.407 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-27T14:53:00.407 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-27T14:53:00.407 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-11-27T14:53:00.407 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-27T14:53:00.407 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-11-27T14:53:00.407 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-27T14:53:00.407 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-27T14:53:00.407 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-11-27T14:53:00.407 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-27T14:53:00.407 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-27T14:53:00.407 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-27T14:53:00.408 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-11-27T14:53:00.408 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-27T14:53:00.408 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-27T14:53:00.408 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-11-27T14:53:00.408 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-27T14:53:00.408 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-27T14:53:00.408 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-27T14:53:00.408 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-27T14:53:00.408 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-27T14:53:00.408 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-27T14:53:00.408 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-27T14:53:00.409 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-27T14:53:00.409 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-27T14:53:00.409 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-27T14:53:00.409 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-27T14:53:00.409 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-11-27T14:53:00.409 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-27T14:53:00.409 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-27T14:53:00.409 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-27T14:53:00.409 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-27T14:53:00.409 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-11-27T14:53:00.409 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-27T14:53:00.409 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-11-27T14:53:00.409 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-11-27T14:53:00.409 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-27T14:53:00.410 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-11-27T14:53:00.410 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-27T14:53:00.410 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-27T14:53:00.410 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-27T14:53:00.410 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-27T14:53:00.410 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-27T14:53:00.410 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-27T14:53:00.410 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-11-27T14:53:00.410 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-27T14:53:00.410 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-11-27T14:53:00.410 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-11-27T14:53:00.410 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-11-27T14:53:00.410 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-27T14:53:00.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-27T14:53:00.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-27T14:53:00.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-27T14:53:00.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-27T14:53:00.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-27T14:53:00.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-27T14:53:00.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-27T14:53:00.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-27T14:53:00.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-27T14:53:00.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-27T14:53:00.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-27T14:53:00.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-27T14:53:00.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-27T14:53:00.411 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-27T14:53:00.412 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-11-27T14:53:00.412 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-11-27T14:53:00.412 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-27T14:53:00.412 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-27T14:53:00.412 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-27T14:53:00.412 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-27T14:53:00.412 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-11-27T14:53:00.412 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-11-27T14:53:00.412 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-27T14:53:00.412 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-27T14:53:00.412 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-11-27T14:53:00.412 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-27T14:53:00.412 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-11-27T14:53:00.412 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-11-27T14:53:00.413 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-27T14:53:00.413 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-11-27T14:53:00.413 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-27T14:53:00.413 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-11-27T14:53:00.413 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-27T14:53:00.413 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-27T14:53:00.413 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-27T14:53:00.413 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-27T14:53:00.413 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-27T14:53:00.413 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-11-27T14:53:00.413 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-11-27T14:53:00.413 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-27T14:53:00.413 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-27T14:53:00.414 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-27T14:53:00.414 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-27T14:53:00.414 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-11-27T14:53:00.414 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-27T14:53:00.414 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-11-27T14:53:00.414 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-27T14:53:00.414 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-27T14:53:00.414 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-11-27T14:53:00.414 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-27T14:53:00.414 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-27T14:53:00.414 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-27T14:53:00.414 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-27T14:53:00.414 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-27T14:53:00.414 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-27T14:53:00.415 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-27T14:53:00.415 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-27T14:53:00.415 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-27T14:53:00.415 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-27T14:53:00.415 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-11-27T14:53:00.415 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-27T14:53:00.415 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-27T14:53:00.415 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-27T14:53:00.415 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-27T14:53:00.415 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-27T14:53:00.415 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-27T14:53:00.415 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-27T14:53:00.415 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-11-27T14:53:00.416 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-27T14:53:00.416 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-11-27T14:53:00.416 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-27T14:53:00.416 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-27T14:53:00.416 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-27T14:53:00.416 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-27T14:53:00.416 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-27T14:53:00.416 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-27T14:53:00.416 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-27T14:53:00.416 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-27T14:53:00.416 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-27T14:53:00.416 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-27T14:53:00.416 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-27T14:53:00.416 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-27T14:53:00.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-27T14:53:00.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-27T14:53:00.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-27T14:53:00.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-27T14:53:00.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-27T14:53:00.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-27T14:53:00.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-27T14:53:00.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-27T14:53:00.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-27T14:53:00.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-27T14:53:00.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-27T14:53:00.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-27T14:53:00.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-27T14:53:00.417 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-27T14:53:00.418 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-27T14:53:00.418 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-27T14:53:00.418 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-27T14:53:00.418 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-27T14:53:00.418 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-27T14:53:00.418 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-27T14:53:00.418 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-27T14:53:00.418 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-27T14:53:00.418 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-27T14:53:00.418 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-27T14:53:00.418 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-27T14:53:00.418 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-27T14:53:00.418 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-27T14:53:00.419 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-27T14:53:00.419 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-27T14:53:00.419 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-27T14:53:00.419 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-27T14:53:00.419 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-27T14:53:00.419 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-27T14:53:00.419 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-11-27T14:53:00.419 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-27T14:53:00.419 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-27T14:53:00.419 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-27T14:53:00.419 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-27T14:53:00.419 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-27T14:53:00.419 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-27T14:53:00.420 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-27T14:53:00.420 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-27T14:53:00.420 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-27T14:53:00.420 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-27T14:53:00.420 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-27T14:53:00.420 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-27T14:53:00.420 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-27T14:53:00.420 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-27T14:53:00.420 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-27T14:53:00.420 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-11-27T14:53:00.420 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-27T14:53:00.420 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-27T14:53:00.420 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-27T14:53:00.420 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-11-27T14:53:00.421 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-11-27T14:53:00.421 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-27T14:53:00.421 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-27T14:53:00.421 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-27T14:53:00.421 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-27T14:53:00.421 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-11-27T14:53:00.421 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-27T14:53:00.421 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-27T14:53:00.421 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-27T14:53:00.421 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-27T14:53:00.421 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-11-27T14:53:00.421 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-11-27T14:53:00.421 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-27T14:53:00.421 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-27T14:53:00.422 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-27T14:53:00.422 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-11-27T14:53:00.422 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-11-27T14:53:00.422 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-11-27T14:53:00.422 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-11-27T14:53:00.422 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-11-27T14:53:00.422 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-27T14:53:00.422 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-27T14:53:00.422 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-11-27T14:53:00.422 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-11-27T14:53:00.422 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-11-27T14:53:00.422 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-27T14:53:00.422 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-27T14:53:00.422 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-11-27T14:53:00.423 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-27T14:53:00.423 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-27T14:53:00.423 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-27T14:53:00.423 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-27T14:53:00.423 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-27T14:53:00.423 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-27T14:53:00.423 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-27T14:53:00.423 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-27T14:53:00.423 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-27T14:53:00.423 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-27T14:53:00.423 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-27T14:53:00.423 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-27T14:53:00.423 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-11-27T14:53:00.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-27T14:53:00.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-27T14:53:00.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-27T14:53:00.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-27T14:53:00.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-27T14:53:00.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-27T14:53:00.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-27T14:53:00.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-27T14:53:00.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-27T14:53:00.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-27T14:53:00.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-27T14:53:00.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-27T14:53:00.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-27T14:53:00.424 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-27T14:53:00.425 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-27T14:53:00.425 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-27T14:53:00.425 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-27T14:53:00.425 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-27T14:53:00.425 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-27T14:53:00.425 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-27T14:53:00.425 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-27T14:53:00.425 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-27T14:53:00.425 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-27T14:53:00.425 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-27T14:53:00.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-27T14:53:00.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-27T14:53:00.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-11-27T14:53:00.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-27T14:53:00.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-27T14:53:00.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-27T14:53:00.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-27T14:53:00.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-11-27T14:53:00.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-27T14:53:00.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-27T14:53:00.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-11-27T14:53:00.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-27T14:53:00.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-11-27T14:53:00.426 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-11-27T14:53:00.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-27T14:53:00.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-11-27T14:53:00.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-27T14:53:00.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-11-27T14:53:00.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-27T14:53:00.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-27T14:53:00.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-11-27T14:53:00.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-27T14:53:00.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-11-27T14:53:00.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-27T14:53:00.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-11-27T14:53:00.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-27T14:53:00.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-11-27T14:53:00.427 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-27T14:53:00.428 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-11-27T14:53:00.428 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-27T14:53:00.428 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-11-27T14:53:00.428 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-27T14:53:00.428 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-11-27T14:53:00.428 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-27T14:53:00.428 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-11-27T14:53:00.428 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-27T14:53:00.428 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-11-27T14:53:00.428 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-11-27T14:53:00.428 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-27T14:53:00.428 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-11-27T14:53:00.428 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-11-27T14:53:00.429 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-27T14:53:00.429 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-11-27T14:53:00.429 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-27T14:53:00.429 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-11-27T14:53:00.429 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-11-27T14:53:00.429 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-27T14:53:00.429 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-11-27T14:53:00.429 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-27T14:53:00.429 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-11-27T14:53:00.429 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-27T14:53:00.429 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-11-27T14:53:00.429 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-27T14:53:00.429 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-11-27T14:53:00.430 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-27T14:53:00.430 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-11-27T14:53:00.430 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-27T14:53:00.430 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-11-27T14:53:00.430 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-27T14:53:00.430 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-11-27T14:53:00.430 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-27T14:53:00.430 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-11-27T14:53:00.430 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-11-27T14:53:00.430 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-27T14:53:00.430 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-11-27T14:53:00.430 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-27T14:53:00.430 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-11-27T14:53:00.431 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-27T14:53:00.431 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-27T14:53:00.431 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-27T14:53:00.431 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-27T14:53:00.431 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-11-27T14:53:00.431 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-27T14:53:00.431 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-27T14:53:00.431 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-11-27T14:53:00.431 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-27T14:53:00.431 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-11-27T14:53:00.431 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-27T14:53:00.431 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-11-27T14:53:00.431 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-27T14:53:00.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-27T14:53:00.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-27T14:53:00.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-27T14:53:00.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-27T14:53:00.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-27T14:53:00.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-27T14:53:00.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-27T14:53:00.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-27T14:53:00.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-27T14:53:00.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-27T14:53:00.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-27T14:53:00.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-27T14:53:00.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-27T14:53:00.432 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-27T14:53:00.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-27T14:53:00.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-27T14:53:00.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-27T14:53:00.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-27T14:53:00.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-27T14:53:00.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-27T14:53:00.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-27T14:53:00.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-27T14:53:00.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-27T14:53:00.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-27T14:53:00.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-27T14:53:00.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-27T14:53:00.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-27T14:53:00.433 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-27T14:53:00.434 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-27T14:53:00.434 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-27T14:53:00.434 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-27T14:53:00.434 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-27T14:53:00.434 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-27T14:53:00.434 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-27T14:53:00.434 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-27T14:53:00.434 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-27T14:53:00.434 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-27T14:53:00.434 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-27T14:53:00.434 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-27T14:53:00.434 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-27T14:53:00.434 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-27T14:53:00.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-27T14:53:00.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-27T14:53:00.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-27T14:53:00.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-27T14:53:00.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-27T14:53:00.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-27T14:53:00.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-27T14:53:00.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-27T14:53:00.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-11-27T14:53:00.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-11-27T14:53:00.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-27T14:53:00.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-11-27T14:53:00.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-27T14:53:00.435 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-11-27T14:53:00.436 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-27T14:53:00.436 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-27T14:53:00.436 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-27T14:53:00.436 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-27T14:53:00.436 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-27T14:53:00.436 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-11-27T14:53:00.436 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-27T14:53:00.436 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-11-27T14:53:00.436 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-27T14:53:00.436 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-11-27T14:53:00.436 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-27T14:53:00.436 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-11-27T14:53:00.436 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-27T14:53:00.436 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-11-27T14:53:00.437 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-11-27T14:53:00.437 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-27T14:53:00.437 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-11-27T14:53:00.437 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-27T14:53:00.437 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-27T14:53:00.437 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-11-27T14:53:00.437 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-27T14:53:00.437 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-11-27T14:53:00.437 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-27T14:53:00.437 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-11-27T14:53:00.437 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-27T14:53:00.437 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-11-27T14:53:00.437 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-27T14:53:00.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-11-27T14:53:00.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-27T14:53:00.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-11-27T14:53:00.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-11-27T14:53:00.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-11-27T14:53:00.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-27T14:53:00.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-11-27T14:53:00.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-27T14:53:00.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-11-27T14:53:00.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-11-27T14:53:00.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-27T14:53:00.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-27T14:53:00.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-11-27T14:53:00.438 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-27T14:53:00.439 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-27T14:53:00.439 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-27T14:53:00.439 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-27T14:53:00.439 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-11-27T14:53:00.439 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-27T14:53:00.439 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-27T14:53:00.439 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-11-27T14:53:00.439 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-27T14:53:00.439 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-11-27T14:53:00.439 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-27T14:53:00.439 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-11-27T14:53:00.439 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-27T14:53:00.439 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-27T14:53:00.439 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-27T14:53:00.440 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-27T14:53:00.440 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-11-27T14:53:00.440 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-11-27T14:53:00.440 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-11-27T14:53:00.440 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-27T14:53:00.440 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-11-27T14:53:00.440 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-11-27T14:53:00.440 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-27T14:53:00.440 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-27T14:53:00.440 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-27T14:53:00.440 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-27T14:53:00.440 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-27T14:53:00.440 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-27T14:53:00.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-27T14:53:00.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-27T14:53:00.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-11-27T14:53:00.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-27T14:53:00.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-27T14:53:00.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-27T14:53:00.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-27T14:53:00.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-27T14:53:00.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-11-27T14:53:00.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-27T14:53:00.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-11-27T14:53:00.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-27T14:53:00.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-27T14:53:00.441 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-27T14:53:00.442 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-27T14:53:00.442 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-11-27T14:53:00.442 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-27T14:53:00.442 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-11-27T14:53:00.442 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-27T14:53:00.442 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-27T14:53:00.442 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-11-27T14:53:00.442 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-27T14:53:00.442 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-27T14:53:00.442 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-27T14:53:00.442 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-27T14:53:00.442 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-27T14:53:00.442 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-27T14:53:00.443 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-27T14:53:00.443 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-27T14:53:00.443 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-27T14:53:00.443 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-27T14:53:00.443 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-27T14:53:00.443 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-27T14:53:00.443 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-27T14:53:00.443 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-27T14:53:00.443 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-27T14:53:00.443 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-27T14:53:00.443 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-27T14:53:00.443 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-27T14:53:00.443 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-27T14:53:00.443 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-27T14:53:00.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-27T14:53:00.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-27T14:53:00.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-27T14:53:00.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-27T14:53:00.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-11-27T14:53:00.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-27T14:53:00.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-11-27T14:53:00.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-11-27T14:53:00.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-27T14:53:00.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-11-27T14:53:00.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-27T14:53:00.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-27T14:53:00.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-27T14:53:00.444 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-27T14:53:00.445 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-11-27T14:53:00.445 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-11-27T14:53:00.445 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-11-27T14:53:00.445 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-27T14:53:00.445 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-27T14:53:00.445 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-27T14:53:00.445 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-27T14:53:00.445 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-27T14:53:00.445 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-11-27T14:53:00.445 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-11-27T14:53:00.445 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-27T14:53:00.445 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-27T14:53:00.445 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-27T14:53:00.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-11-27T14:53:00.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-27T14:53:00.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-27T14:53:00.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-27T14:53:00.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-27T14:53:00.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-27T14:53:00.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-27T14:53:00.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-27T14:53:00.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-27T14:53:00.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-27T14:53:00.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-27T14:53:00.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-11-27T14:53:00.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-27T14:53:00.446 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-27T14:53:00.447 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-27T14:53:00.447 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-27T14:53:00.447 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-27T14:53:00.447 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-27T14:53:00.447 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-27T14:53:00.447 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-27T14:53:00.447 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-27T14:53:00.447 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-27T14:53:00.447 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-27T14:53:00.447 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-27T14:53:00.447 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-27T14:53:00.447 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-27T14:53:00.447 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-27T14:53:00.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-27T14:53:00.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-27T14:53:00.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-27T14:53:00.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-27T14:53:00.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-27T14:53:00.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-27T14:53:00.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-27T14:53:00.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-27T14:53:00.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-27T14:53:00.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-27T14:53:00.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-27T14:53:00.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-27T14:53:00.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-27T14:53:00.448 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-27T14:53:00.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-27T14:53:00.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-27T14:53:00.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-27T14:53:00.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-27T14:53:00.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-27T14:53:00.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-27T14:53:00.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-27T14:53:00.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-27T14:53:00.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-27T14:53:00.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-11-27T14:53:00.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-27T14:53:00.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-27T14:53:00.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-27T14:53:00.449 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-27T14:53:00.450 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-27T14:53:00.450 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-27T14:53:00.450 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-27T14:53:00.450 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-11-27T14:53:00.450 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-27T14:53:00.450 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-27T14:53:00.450 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-27T14:53:00.450 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-27T14:53:00.450 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-27T14:53:00.450 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-27T14:53:00.450 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-27T14:53:00.450 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-27T14:53:00.450 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-27T14:53:00.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-27T14:53:00.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-27T14:53:00.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-27T14:53:00.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-27T14:53:00.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-27T14:53:00.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-27T14:53:00.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-27T14:53:00.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-27T14:53:00.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-27T14:53:00.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-27T14:53:00.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-27T14:53:00.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-27T14:53:00.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-27T14:53:00.451 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-27T14:53:00.452 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-27T14:53:00.452 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-27T14:53:00.452 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-27T14:53:00.452 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-27T14:53:00.452 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-11-27T14:53:00.452 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-27T14:53:00.452 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-11-27T14:53:00.452 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-27T14:53:00.452 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-27T14:53:00.452 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-27T14:53:00.452 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-27T14:53:00.452 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-27T14:53:00.452 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-27T14:53:00.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-27T14:53:00.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-11-27T14:53:00.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-27T14:53:00.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-11-27T14:53:00.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-27T14:53:00.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-11-27T14:53:00.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-27T14:53:00.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-11-27T14:53:00.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-11-27T14:53:00.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-27T14:53:00.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-27T14:53:00.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-27T14:53:00.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-27T14:53:00.453 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-27T14:53:00.454 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-27T14:53:00.454 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-27T14:53:00.454 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-27T14:53:00.454 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-27T14:53:00.454 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-27T14:53:00.454 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-27T14:53:00.454 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-27T14:53:00.454 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-27T14:53:00.454 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-27T14:53:00.454 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-27T14:53:00.454 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-27T14:53:00.454 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-11-27T14:53:00.454 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-27T14:53:00.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-27T14:53:00.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-27T14:53:00.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-27T14:53:00.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-27T14:53:00.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-27T14:53:00.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-27T14:53:00.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-27T14:53:00.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-27T14:53:00.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-27T14:53:00.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-27T14:53:00.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-27T14:53:00.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-27T14:53:00.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-27T14:53:00.455 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-27T14:53:00.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-27T14:53:00.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-27T14:53:00.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-27T14:53:00.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-27T14:53:00.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-27T14:53:00.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-27T14:53:00.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-27T14:53:00.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-27T14:53:00.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-27T14:53:00.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-27T14:53:00.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-27T14:53:00.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-27T14:53:00.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-27T14:53:00.456 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-27T14:53:00.457 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-27T14:53:00.457 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-27T14:53:00.457 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-27T14:53:00.457 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-27T14:53:00.457 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-27T14:53:00.457 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-27T14:53:00.457 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-27T14:53:00.457 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-27T14:53:00.457 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-27T14:53:00.457 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-27T14:53:00.457 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-27T14:53:00.457 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-27T14:53:00.457 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-27T14:53:00.457 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-27T14:53:00.458 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-27T14:53:00.458 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-27T14:53:00.458 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-27T14:53:00.458 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-27T14:53:00.458 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-27T14:53:00.458 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-27T14:53:00.458 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-27T14:53:00.458 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-27T14:53:00.458 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-27T14:53:00.458 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-27T14:53:00.458 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-27T14:53:00.458 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-27T14:53:00.458 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-27T14:53:00.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-27T14:53:00.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-27T14:53:00.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-11-27T14:53:00.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-27T14:53:00.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-27T14:53:00.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-27T14:53:00.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-27T14:53:00.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-11-27T14:53:00.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-27T14:53:00.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-27T14:53:00.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-27T14:53:00.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-27T14:53:00.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-27T14:53:00.459 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-27T14:53:00.460 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-27T14:53:00.460 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-27T14:53:00.460 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-27T14:53:00.460 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-27T14:53:00.460 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-27T14:53:00.460 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-27T14:53:00.460 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-27T14:53:00.460 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-27T14:53:00.460 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-27T14:53:00.460 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-27T14:53:00.460 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-27T14:53:00.460 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-27T14:53:00.460 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-27T14:53:00.461 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-27T14:53:00.461 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-27T14:53:00.461 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-27T14:53:00.461 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-27T14:53:00.461 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-27T14:53:00.461 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-27T14:53:00.461 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-27T14:53:00.461 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-27T14:53:00.461 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-27T14:53:00.461 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-27T14:53:00.461 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-27T14:53:00.461 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-27T14:53:00.461 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-27T14:53:00.461 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-27T14:53:00.462 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-27T14:53:00.462 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-27T14:53:00.462 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-27T14:53:00.462 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-27T14:53:00.462 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-27T14:53:00.462 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-27T14:53:00.462 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-27T14:53:00.462 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-27T14:53:00.462 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-27T14:53:00.462 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-27T14:53:00.462 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-27T14:53:00.462 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-27T14:53:00.462 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-27T14:53:00.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-27T14:53:00.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-27T14:53:00.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-27T14:53:00.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-27T14:53:00.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-27T14:53:00.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-27T14:53:00.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-27T14:53:00.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-27T14:53:00.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-27T14:53:00.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-27T14:53:00.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-27T14:53:00.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-27T14:53:00.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-27T14:53:00.463 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-27T14:53:00.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-27T14:53:00.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-27T14:53:00.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-27T14:53:00.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-27T14:53:00.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-27T14:53:00.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-27T14:53:00.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-27T14:53:00.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-27T14:53:00.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-27T14:53:00.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-27T14:53:00.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-27T14:53:00.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-11-27T14:53:00.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-27T14:53:00.464 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-11-27T14:53:00.465 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-27T14:53:00.465 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-11-27T14:53:00.465 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-27T14:53:00.465 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-27T14:53:00.465 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-27T14:53:00.465 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-27T14:53:00.465 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-11-27T14:53:00.465 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-27T14:53:00.465 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-11-27T14:53:00.465 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-27T14:53:00.465 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-27T14:53:00.465 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-27T14:53:00.465 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-11-27T14:53:00.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-27T14:53:00.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-27T14:53:00.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-27T14:53:00.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-27T14:53:00.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-27T14:53:00.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-27T14:53:00.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-27T14:53:00.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-27T14:53:00.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-27T14:53:00.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-27T14:53:00.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-27T14:53:00.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-27T14:53:00.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-27T14:53:00.466 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-27T14:53:00.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-27T14:53:00.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-27T14:53:00.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-27T14:53:00.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-27T14:53:00.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-27T14:53:00.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-27T14:53:00.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-27T14:53:00.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-27T14:53:00.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-27T14:53:00.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-27T14:53:00.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-27T14:53:00.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-27T14:53:00.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-27T14:53:00.467 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-27T14:53:00.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-27T14:53:00.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-27T14:53:00.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-27T14:53:00.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-27T14:53:00.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-27T14:53:00.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-27T14:53:00.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-27T14:53:00.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-27T14:53:00.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-27T14:53:00.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-27T14:53:00.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-27T14:53:00.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-27T14:53:00.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-27T14:53:00.468 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-27T14:53:00.469 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-27T14:53:00.469 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-27T14:53:00.469 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-11-27T14:53:00.469 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-27T14:53:00.469 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-27T14:53:00.469 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-11-27T14:53:00.469 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-11-27T14:53:00.469 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-27T14:53:00.469 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-27T14:53:00.469 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-27T14:53:00.469 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-11-27T14:53:00.469 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-27T14:53:00.469 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-11-27T14:53:00.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-27T14:53:00.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-27T14:53:00.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-11-27T14:53:00.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-27T14:53:00.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-11-27T14:53:00.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-27T14:53:00.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-27T14:53:00.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-27T14:53:00.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-27T14:53:00.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-27T14:53:00.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-11-27T14:53:00.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-27T14:53:00.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-11-27T14:53:00.470 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-11-27T14:53:00.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-11-27T14:53:00.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-27T14:53:00.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-11-27T14:53:00.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-11-27T14:53:00.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-11-27T14:53:00.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-11-27T14:53:00.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-27T14:53:00.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-27T14:53:00.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-27T14:53:00.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-27T14:53:00.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-27T14:53:00.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-27T14:53:00.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-27T14:53:00.471 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-27T14:53:00.472 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-27T14:53:00.472 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-27T14:53:00.472 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-27T14:53:00.472 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-27T14:53:00.472 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-27T14:53:00.472 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-27T14:53:00.473 DEBUG:teuthology.orchestra.run.smithi133:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-27T14:53:00.500 INFO:teuthology.orchestra.run.smithi133.stdout:postinstall scriptlet (using /bin/sh): 2023-11-27T14:53:00.500 INFO:teuthology.orchestra.run.smithi133.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-11-27T14:53:00.500 INFO:teuthology.orchestra.run.smithi133.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-11-27T14:53:00.500 INFO:teuthology.orchestra.run.smithi133.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-27T14:53:00.500 INFO:teuthology.orchestra.run.smithi133.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-11-27T14:53:00.500 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-27T14:53:00.500 INFO:teuthology.orchestra.run.smithi133.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-27T14:53:00.500 INFO:teuthology.orchestra.run.smithi133.stdout:fi 2023-11-27T14:53:00.501 INFO:teuthology.orchestra.run.smithi133.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-27T14:53:00.501 INFO:teuthology.orchestra.run.smithi133.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-27T14:53:00.501 INFO:teuthology.orchestra.run.smithi133.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-11-27T14:53:00.501 INFO:teuthology.orchestra.run.smithi133.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-27T14:53:00.501 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-11-27T14:53:00.501 INFO:teuthology.orchestra.run.smithi133.stdout:fi 2023-11-27T14:53:00.501 INFO:teuthology.orchestra.run.smithi133.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-27T14:53:00.501 INFO:teuthology.orchestra.run.smithi133.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-27T14:53:00.501 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-11-27T14:53:00.501 INFO:teuthology.orchestra.run.smithi133.stdout:fi 2023-11-27T14:53:00.502 DEBUG:teuthology.orchestra.run.smithi133:> sudo rpm -qi grub2-tools 2023-11-27T14:53:00.583 INFO:teuthology.orchestra.run.smithi133.stdout:Name : grub2-tools 2023-11-27T14:53:00.583 INFO:teuthology.orchestra.run.smithi133.stdout:Epoch : 1 2023-11-27T14:53:00.583 INFO:teuthology.orchestra.run.smithi133.stdout:Version : 2.02 2023-11-27T14:53:00.583 INFO:teuthology.orchestra.run.smithi133.stdout:Release : 123.el8 2023-11-27T14:53:00.583 INFO:teuthology.orchestra.run.smithi133.stdout:Architecture: x86_64 2023-11-27T14:53:00.583 INFO:teuthology.orchestra.run.smithi133.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-11-27T14:53:00.583 INFO:teuthology.orchestra.run.smithi133.stdout:Group : System Environment/Base 2023-11-27T14:53:00.583 INFO:teuthology.orchestra.run.smithi133.stdout:Size : 9265450 2023-11-27T14:53:00.583 INFO:teuthology.orchestra.run.smithi133.stdout:License : GPLv3+ 2023-11-27T14:53:00.584 INFO:teuthology.orchestra.run.smithi133.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-11-27T14:53:00.584 INFO:teuthology.orchestra.run.smithi133.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-11-27T14:53:00.584 INFO:teuthology.orchestra.run.smithi133.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-11-27T14:53:00.584 INFO:teuthology.orchestra.run.smithi133.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-11-27T14:53:00.584 INFO:teuthology.orchestra.run.smithi133.stdout:Relocations : (not relocatable) 2023-11-27T14:53:00.584 INFO:teuthology.orchestra.run.smithi133.stdout:Packager : Red Hat, Inc. 2023-11-27T14:53:00.584 INFO:teuthology.orchestra.run.smithi133.stdout:Vendor : Red Hat, Inc. 2023-11-27T14:53:00.584 INFO:teuthology.orchestra.run.smithi133.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-27T14:53:00.584 INFO:teuthology.orchestra.run.smithi133.stdout:Summary : Support tools for GRUB. 2023-11-27T14:53:00.584 INFO:teuthology.orchestra.run.smithi133.stdout:Description : 2023-11-27T14:53:00.584 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T14:53:00.584 INFO:teuthology.orchestra.run.smithi133.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-27T14:53:00.584 INFO:teuthology.orchestra.run.smithi133.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-27T14:53:00.584 INFO:teuthology.orchestra.run.smithi133.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-27T14:53:00.585 INFO:teuthology.orchestra.run.smithi133.stdout:hardware devices. 2023-11-27T14:53:00.585 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T14:53:00.585 INFO:teuthology.orchestra.run.smithi133.stdout:This subpackage provides tools for support of all platforms. 2023-11-27T14:53:00.586 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-27T14:53:00.586 INFO:teuthology.task.kernel:Updating grub on smithi133 to boot 6.6.0-g8247bbb5b470 2023-11-27T14:53:00.586 DEBUG:teuthology.orchestra.run.smithi133:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-27T14:53:02.874 INFO:teuthology.orchestra.run.smithi133.stderr:Generating grub configuration file ... 2023-11-27T14:53:04.779 INFO:teuthology.orchestra.run.smithi133.stderr:done 2023-11-27T14:53:04.781 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2023-11-27T14:53:04.804 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.bD4iEl5KqI 2023-11-27T14:53:04.805 DEBUG:teuthology.orchestra.run.smithi133:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.bD4iEl5KqI 2023-11-27T14:53:04.882 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.bD4iEl5KqI 2023-11-27T14:53:05.025 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.bD4iEl5KqI is 6KB 2023-11-27T14:53:05.074 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.bD4iEl5KqI 2023-11-27T14:53:05.090 DEBUG:teuthology.orchestra.run.smithi133:> sudo /bin/ls /boot/loader/entries || true 2023-11-27T14:53:05.163 INFO:teuthology.orchestra.run.smithi133.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-11-27T14:53:05.164 INFO:teuthology.orchestra.run.smithi133.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-11-27T14:53:05.164 INFO:teuthology.orchestra.run.smithi133.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.6.0-g8247bbb5b470.conf 2023-11-27T14:53:05.165 DEBUG:teuthology.orchestra.run.smithi133:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.6.0-g8247bbb5b470 2023-11-27T14:53:05.331 DEBUG:teuthology.orchestra.run.smithi133:> sudo shutdown -r now 2023-11-27T14:53:35.334 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-11-27T14:53:35.335 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-11-27T14:53:35.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T14:54:10.022 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2023-11-27T14:54:19.023 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-11-27T14:54:19.024 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T14:54:37.410 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2023-11-27T14:54:49.411 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-11-27T14:54:49.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T14:54:50.750 DEBUG:teuthology.orchestra.run.smithi133:> true 2023-11-27T14:54:51.294 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-11-27T14:54:51.294 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-g8247bbb5b470"... 2023-11-27T14:54:51.294 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2023-11-27T14:54:51.336 INFO:teuthology.orchestra.run.smithi133.stdout:6.6.0-g8247bbb5b470 2023-11-27T14:54:51.336 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-27T14:54:51.337 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-27T14:54:51.337 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-11-27T14:54:52.338 DEBUG:teuthology.parallel:result is None 2023-11-27T14:54:52.339 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-27T14:54:52.345 INFO:teuthology.task.internal:Creating test directory... 2023-11-27T14:54:52.346 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-27T14:54:52.351 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-27T14:54:52.354 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-27T14:54:52.391 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-27T14:54:52.399 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-27T14:54:52.405 INFO:teuthology.task.internal:Creating archive directory... 2023-11-27T14:54:52.406 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-27T14:54:52.435 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-27T14:54:52.437 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-27T14:54:52.495 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-27T14:54:52.502 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-27T14:54:52.502 DEBUG:teuthology.orchestra.run.smithi174:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-27T14:54:52.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T14:54:52.523 DEBUG:teuthology.orchestra.run.smithi177:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-27T14:54:52.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T14:54:52.541 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-27T14:54:52.546 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-27T14:54:52.566 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-27T14:54:52.605 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-27T14:54:52.620 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-27T14:54:52.623 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-27T14:54:52.639 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-27T14:54:52.764 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-27T14:54:52.802 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-27T14:54:52.805 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-27T14:54:52.812 INFO:teuthology.task.internal:Configuring sudo... 2023-11-27T14:54:52.812 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-27T14:54:52.817 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-27T14:54:52.819 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-27T14:54:52.882 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-27T14:54:52.891 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-27T14:54:52.891 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-27T14:54:52.895 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-27T14:54:52.898 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-27T14:54:52.934 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-27T14:54:53.047 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-27T14:54:53.122 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-27T14:54:53.192 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-27T14:54:53.260 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-11-27T14:54:53.260 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-27T14:54:53.331 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-27T14:54:53.379 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-27T14:54:53.431 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-27T14:54:53.517 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-27T14:54:53.566 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-27T14:54:53.566 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-27T14:54:53.643 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-27T14:54:53.689 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-27T14:54:53.749 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-27T14:54:53.836 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-27T14:54:53.882 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-27T14:54:53.882 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-27T14:54:53.953 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2023-11-27T14:54:53.958 DEBUG:teuthology.orchestra.run.smithi174:> sudo service rsyslog restart 2023-11-27T14:54:53.961 DEBUG:teuthology.orchestra.run.smithi177:> sudo service rsyslog restart 2023-11-27T14:54:54.030 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-27T14:54:54.062 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-27T14:54:54.072 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-27T14:54:54.417 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-27T14:54:54.427 INFO:teuthology.task.internal:Starting timer... 2023-11-27T14:54:54.427 INFO:teuthology.run_tasks:Running task pcp... 2023-11-27T14:54:54.440 INFO:teuthology.run_tasks:Running task selinux... 2023-11-27T14:54:54.449 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-11-27T14:54:54.450 DEBUG:teuthology.orchestra.run.smithi133:> sudo service auditd rotate 2023-11-27T14:54:54.608 INFO:teuthology.orchestra.run.smithi133.stdout:Rotating logs: 2023-11-27T14:54:54.612 DEBUG:teuthology.orchestra.run.smithi174:> sudo service auditd rotate 2023-11-27T14:54:54.673 INFO:teuthology.orchestra.run.smithi174.stdout:Rotating logs: 2023-11-27T14:54:54.675 DEBUG:teuthology.orchestra.run.smithi177:> sudo service auditd rotate 2023-11-27T14:54:54.743 INFO:teuthology.orchestra.run.smithi177.stdout:Rotating logs: 2023-11-27T14:54:54.745 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-27T14:54:54.745 DEBUG:teuthology.orchestra.run.smithi133:> /usr/sbin/getenforce 2023-11-27T14:54:54.806 INFO:teuthology.orchestra.run.smithi133.stdout:Permissive 2023-11-27T14:54:54.806 DEBUG:teuthology.orchestra.run.smithi174:> /usr/sbin/getenforce 2023-11-27T14:54:54.833 INFO:teuthology.orchestra.run.smithi174.stdout:Permissive 2023-11-27T14:54:54.833 DEBUG:teuthology.orchestra.run.smithi177:> /usr/sbin/getenforce 2023-11-27T14:54:54.862 INFO:teuthology.orchestra.run.smithi177.stdout:Permissive 2023-11-27T14:54:54.863 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi133.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi174.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi177.front.sepia.ceph.com': 'permissive'} 2023-11-27T14:54:54.863 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-27T14:54:54.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T14:54:54.934 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-27T14:54:54.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T14:54:54.963 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-27T14:54:54.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T14:54:54.990 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-27T14:54:54.990 DEBUG:teuthology.orchestra.run.smithi133:> sudo /usr/sbin/setenforce permissive 2023-11-27T14:54:55.068 DEBUG:teuthology.orchestra.run.smithi174:> sudo /usr/sbin/setenforce permissive 2023-11-27T14:54:55.101 DEBUG:teuthology.orchestra.run.smithi177:> sudo /usr/sbin/setenforce permissive 2023-11-27T14:54:55.129 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-27T14:54:55.139 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-27T14:54:55.139 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-27T14:54:55.156 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-27T14:54:55.157 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi133.front.sepia.ceph.com,smithi174.front.sepia.ceph.com,smithi177.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-27T15:08:50.135 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi133.front.sepia.ceph.com'), Remote(name='ubuntu@smithi174.front.sepia.ceph.com'), Remote(name='ubuntu@smithi177.front.sepia.ceph.com')] 2023-11-27T15:08:50.136 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-11-27T15:08:50.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T15:08:50.225 DEBUG:teuthology.orchestra.run.smithi133:> true 2023-11-27T15:08:50.336 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2023-11-27T15:08:50.336 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2023-11-27T15:08:50.336 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T15:08:50.404 DEBUG:teuthology.orchestra.run.smithi174:> true 2023-11-27T15:08:50.472 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2023-11-27T15:08:50.473 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-11-27T15:08:50.473 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-27T15:08:50.542 DEBUG:teuthology.orchestra.run.smithi177:> true 2023-11-27T15:08:50.616 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-11-27T15:08:50.616 INFO:teuthology.run_tasks:Running task clock... 2023-11-27T15:08:50.626 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-27T15:08:50.626 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-27T15:08:50.626 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-27T15:08:50.631 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-27T15:08:50.631 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-27T15:08:50.633 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-27T15:08:50.633 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-27T15:08:50.664 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-27T15:08:50.679 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-27T15:08:50.691 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-27T15:08:50.700 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-27T15:08:50.707 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-27T15:08:50.735 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-27T15:08:50.747 INFO:teuthology.orchestra.run.smithi174.stderr:sudo: ntpd: command not found 2023-11-27T15:08:50.759 INFO:teuthology.orchestra.run.smithi174.stdout:506 Cannot talk to daemon 2023-11-27T15:08:50.774 INFO:teuthology.orchestra.run.smithi177.stderr:sudo: ntpd: command not found 2023-11-27T15:08:50.774 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-27T15:08:50.786 INFO:teuthology.orchestra.run.smithi177.stdout:506 Cannot talk to daemon 2023-11-27T15:08:50.787 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-27T15:08:50.800 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-27T15:08:50.814 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-27T15:08:50.827 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: ntpd: command not found 2023-11-27T15:08:50.856 INFO:teuthology.orchestra.run.smithi133.stdout:506 Cannot talk to daemon 2023-11-27T15:08:50.866 INFO:teuthology.orchestra.run.smithi174.stderr:bash: ntpq: command not found 2023-11-27T15:08:50.869 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-27T15:08:50.869 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2023-11-27T15:08:50.869 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:08:50.869 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:08:50.869 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:08:50.870 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:08:50.885 INFO:teuthology.orchestra.run.smithi177.stderr:bash: ntpq: command not found 2023-11-27T15:08:50.889 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-27T15:08:50.889 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2023-11-27T15:08:50.889 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:08:50.889 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:08:50.889 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:08:50.889 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:08:50.890 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-27T15:08:50.924 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-27T15:08:51.024 INFO:teuthology.orchestra.run.smithi133.stderr:bash: ntpq: command not found 2023-11-27T15:08:51.032 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-27T15:08:51.032 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2023-11-27T15:08:51.032 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:08:51.032 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:08:51.032 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:08:51.032 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:08:51.033 INFO:teuthology.run_tasks:Running task pexec... 2023-11-27T15:08:51.043 INFO:teuthology.task.pexec:Executing custom commands... 2023-11-27T15:08:51.043 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi133.front.sepia.ceph.com 2023-11-27T15:08:51.043 DEBUG:teuthology.orchestra.run.smithi133:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-11-27T15:08:51.044 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi174.front.sepia.ceph.com 2023-11-27T15:08:51.044 DEBUG:teuthology.orchestra.run.smithi174:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-11-27T15:08:51.044 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi177.front.sepia.ceph.com 2023-11-27T15:08:51.044 DEBUG:teuthology.orchestra.run.smithi177:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-11-27T15:08:51.414 INFO:teuthology.orchestra.run.smithi174.stdout:Updating Subscription Management repositories. 2023-11-27T15:08:51.415 INFO:teuthology.orchestra.run.smithi177.stdout:Updating Subscription Management repositories. 2023-11-27T15:08:51.526 INFO:teuthology.orchestra.run.smithi133.stdout:Updating Subscription Management repositories. 2023-11-27T15:08:52.905 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:05:13 ago on Mon 27 Nov 2023 03:03:39 PM UTC. 2023-11-27T15:08:52.933 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:04:56 ago on Mon 27 Nov 2023 03:03:56 PM UTC. 2023-11-27T15:08:53.156 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:08 ago on Mon 27 Nov 2023 03:06:44 PM UTC. 2023-11-27T15:08:54.625 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-27T15:08:54.626 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-27T15:08:54.627 INFO:teuthology.orchestra.run.smithi174.stdout: Package Architecture Version Repository Size 2023-11-27T15:08:54.627 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-27T15:08:54.627 INFO:teuthology.orchestra.run.smithi174.stdout:Resetting modules: 2023-11-27T15:08:54.627 INFO:teuthology.orchestra.run.smithi174.stdout: container-tools 2023-11-27T15:08:54.627 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-27T15:08:54.627 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-27T15:08:54.627 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2023-11-27T15:08:54.627 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-27T15:08:54.636 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2023-11-27T15:08:54.637 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2023-11-27T15:08:54.637 INFO:teuthology.orchestra.run.smithi177.stdout: Package Architecture Version Repository Size 2023-11-27T15:08:54.637 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2023-11-27T15:08:54.637 INFO:teuthology.orchestra.run.smithi177.stdout:Resetting modules: 2023-11-27T15:08:54.637 INFO:teuthology.orchestra.run.smithi177.stdout: container-tools 2023-11-27T15:08:54.637 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-27T15:08:54.637 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2023-11-27T15:08:54.637 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2023-11-27T15:08:54.638 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-27T15:08:55.027 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2023-11-27T15:08:55.028 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2023-11-27T15:08:55.028 INFO:teuthology.orchestra.run.smithi133.stdout: Package Architecture Version Repository Size 2023-11-27T15:08:55.029 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2023-11-27T15:08:55.029 INFO:teuthology.orchestra.run.smithi133.stdout:Resetting modules: 2023-11-27T15:08:55.029 INFO:teuthology.orchestra.run.smithi133.stdout: container-tools 2023-11-27T15:08:55.029 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:08:55.029 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2023-11-27T15:08:55.029 INFO:teuthology.orchestra.run.smithi133.stdout:================================================================================ 2023-11-27T15:08:55.029 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:08:55.582 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:05:16 ago on Mon 27 Nov 2023 03:03:39 PM UTC. 2023-11-27T15:08:55.618 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:04:59 ago on Mon 27 Nov 2023 03:03:56 PM UTC. 2023-11-27T15:08:56.291 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:11 ago on Mon 27 Nov 2023 03:06:44 PM UTC. 2023-11-27T15:09:03.871 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2023-11-27T15:09:04.470 INFO:teuthology.orchestra.run.smithi133.stdout:Updating Subscription Management repositories. 2023-11-27T15:09:04.834 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2023-11-27T15:09:04.868 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-27T15:09:05.345 INFO:teuthology.orchestra.run.smithi177.stdout:Updating Subscription Management repositories. 2023-11-27T15:09:05.356 INFO:teuthology.orchestra.run.smithi174.stdout:Updating Subscription Management repositories. 2023-11-27T15:09:06.107 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:21 ago on Mon 27 Nov 2023 03:06:44 PM UTC. 2023-11-27T15:09:06.745 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:05:27 ago on Mon 27 Nov 2023 03:03:39 PM UTC. 2023-11-27T15:09:06.792 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:05:10 ago on Mon 27 Nov 2023 03:03:56 PM UTC. 2023-11-27T15:09:07.894 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2023-11-27T15:09:07.993 INFO:teuthology.orchestra.run.smithi133.stdout:============================================================================================================ 2023-11-27T15:09:07.994 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2023-11-27T15:09:07.994 INFO:teuthology.orchestra.run.smithi133.stdout:============================================================================================================ 2023-11-27T15:09:07.994 INFO:teuthology.orchestra.run.smithi133.stdout:Installing group/module packages: 2023-11-27T15:09:07.994 INFO:teuthology.orchestra.run.smithi133.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-11-27T15:09:07.994 INFO:teuthology.orchestra.run.smithi133.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-11-27T15:09:07.994 INFO:teuthology.orchestra.run.smithi133.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-11-27T15:09:07.994 INFO:teuthology.orchestra.run.smithi133.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-11-27T15:09:07.994 INFO:teuthology.orchestra.run.smithi133.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-11-27T15:09:07.994 INFO:teuthology.orchestra.run.smithi133.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-11-27T15:09:07.994 INFO:teuthology.orchestra.run.smithi133.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-11-27T15:09:07.995 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2023-11-27T15:09:07.995 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-11-27T15:09:07.995 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-11-27T15:09:07.995 INFO:teuthology.orchestra.run.smithi133.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-11-27T15:09:07.995 INFO:teuthology.orchestra.run.smithi133.stdout:Installing module profiles: 2023-11-27T15:09:07.995 INFO:teuthology.orchestra.run.smithi133.stdout: container-tools/common 2023-11-27T15:09:07.995 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:09:07.995 INFO:teuthology.orchestra.run.smithi133.stdout:Enabling module streams: 2023-11-27T15:09:07.995 INFO:teuthology.orchestra.run.smithi133.stdout: container-tools rhel8 2023-11-27T15:09:07.995 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:09:07.996 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2023-11-27T15:09:07.996 INFO:teuthology.orchestra.run.smithi133.stdout:============================================================================================================ 2023-11-27T15:09:07.996 INFO:teuthology.orchestra.run.smithi133.stdout:Install 10 Packages 2023-11-27T15:09:07.996 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:09:07.997 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 18 M 2023-11-27T15:09:07.997 INFO:teuthology.orchestra.run.smithi133.stdout:Installed size: 65 M 2023-11-27T15:09:07.997 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2023-11-27T15:09:08.144 INFO:teuthology.orchestra.run.smithi133.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 277 kB/s | 41 kB 00:00 2023-11-27T15:09:08.170 INFO:teuthology.orchestra.run.smithi133.stdout:(2/10): python3-pyxdg-0.25-16.el8.noarch.rpm 543 kB/s | 94 kB 00:00 2023-11-27T15:09:08.211 INFO:teuthology.orchestra.run.smithi133.stdout:(3/10): python3-pytoml-0.1.14-5.git7dea353.el8. 116 kB/s | 25 kB 00:00 2023-11-27T15:09:08.279 INFO:teuthology.orchestra.run.smithi133.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 368 kB/s | 49 kB 00:00 2023-11-27T15:09:08.312 INFO:teuthology.orchestra.run.smithi133.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 3.6 MB/s | 524 kB 00:00 2023-11-27T15:09:08.388 INFO:teuthology.orchestra.run.smithi133.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.3 MB/s | 149 kB 00:00 2023-11-27T15:09:08.454 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-27T15:09:08.462 INFO:teuthology.orchestra.run.smithi133.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 1.4 MB/s | 213 kB 00:00 2023-11-27T15:09:08.509 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2023-11-27T15:09:08.545 INFO:teuthology.orchestra.run.smithi174.stdout:============================================================================================================ 2023-11-27T15:09:08.545 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2023-11-27T15:09:08.545 INFO:teuthology.orchestra.run.smithi174.stdout:============================================================================================================ 2023-11-27T15:09:08.546 INFO:teuthology.orchestra.run.smithi174.stdout:Installing group/module packages: 2023-11-27T15:09:08.546 INFO:teuthology.orchestra.run.smithi174.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-11-27T15:09:08.546 INFO:teuthology.orchestra.run.smithi174.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-11-27T15:09:08.546 INFO:teuthology.orchestra.run.smithi174.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-11-27T15:09:08.546 INFO:teuthology.orchestra.run.smithi174.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-11-27T15:09:08.546 INFO:teuthology.orchestra.run.smithi174.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-11-27T15:09:08.546 INFO:teuthology.orchestra.run.smithi174.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-11-27T15:09:08.546 INFO:teuthology.orchestra.run.smithi174.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-11-27T15:09:08.546 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2023-11-27T15:09:08.546 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-11-27T15:09:08.546 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-11-27T15:09:08.546 INFO:teuthology.orchestra.run.smithi174.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-11-27T15:09:08.547 INFO:teuthology.orchestra.run.smithi174.stdout:Installing module profiles: 2023-11-27T15:09:08.547 INFO:teuthology.orchestra.run.smithi174.stdout: container-tools/common 2023-11-27T15:09:08.547 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-27T15:09:08.547 INFO:teuthology.orchestra.run.smithi174.stdout:Enabling module streams: 2023-11-27T15:09:08.547 INFO:teuthology.orchestra.run.smithi174.stdout: container-tools rhel8 2023-11-27T15:09:08.547 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-27T15:09:08.547 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-27T15:09:08.547 INFO:teuthology.orchestra.run.smithi174.stdout:============================================================================================================ 2023-11-27T15:09:08.547 INFO:teuthology.orchestra.run.smithi174.stdout:Install 10 Packages 2023-11-27T15:09:08.547 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-27T15:09:08.548 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 18 M 2023-11-27T15:09:08.548 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 65 M 2023-11-27T15:09:08.548 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-27T15:09:08.602 INFO:teuthology.orchestra.run.smithi177.stdout:============================================================================================================ 2023-11-27T15:09:08.603 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2023-11-27T15:09:08.603 INFO:teuthology.orchestra.run.smithi177.stdout:============================================================================================================ 2023-11-27T15:09:08.603 INFO:teuthology.orchestra.run.smithi177.stdout:Installing group/module packages: 2023-11-27T15:09:08.603 INFO:teuthology.orchestra.run.smithi177.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-11-27T15:09:08.603 INFO:teuthology.orchestra.run.smithi177.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-11-27T15:09:08.603 INFO:teuthology.orchestra.run.smithi177.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-11-27T15:09:08.603 INFO:teuthology.orchestra.run.smithi177.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-11-27T15:09:08.603 INFO:teuthology.orchestra.run.smithi177.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-11-27T15:09:08.603 INFO:teuthology.orchestra.run.smithi177.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-11-27T15:09:08.603 INFO:teuthology.orchestra.run.smithi177.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-11-27T15:09:08.603 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2023-11-27T15:09:08.603 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-11-27T15:09:08.604 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-11-27T15:09:08.604 INFO:teuthology.orchestra.run.smithi177.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-11-27T15:09:08.604 INFO:teuthology.orchestra.run.smithi177.stdout:Installing module profiles: 2023-11-27T15:09:08.604 INFO:teuthology.orchestra.run.smithi177.stdout: container-tools/common 2023-11-27T15:09:08.604 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-27T15:09:08.604 INFO:teuthology.orchestra.run.smithi177.stdout:Enabling module streams: 2023-11-27T15:09:08.604 INFO:teuthology.orchestra.run.smithi177.stdout: container-tools rhel8 2023-11-27T15:09:08.604 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-27T15:09:08.604 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2023-11-27T15:09:08.604 INFO:teuthology.orchestra.run.smithi177.stdout:============================================================================================================ 2023-11-27T15:09:08.604 INFO:teuthology.orchestra.run.smithi177.stdout:Install 10 Packages 2023-11-27T15:09:08.604 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-27T15:09:08.605 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 18 M 2023-11-27T15:09:08.605 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 65 M 2023-11-27T15:09:08.605 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2023-11-27T15:09:08.655 INFO:teuthology.orchestra.run.smithi133.stdout:(8/10): buildah-1.26.2-1.module+el8.6.0+16771+2 18 MB/s | 8.0 MB 00:00 2023-11-27T15:09:08.677 INFO:teuthology.orchestra.run.smithi174.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 315 kB/s | 41 kB 00:00 2023-11-27T15:09:08.736 INFO:teuthology.orchestra.run.smithi174.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 132 kB/s | 25 kB 00:00 2023-11-27T15:09:08.753 INFO:teuthology.orchestra.run.smithi174.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 457 kB/s | 94 kB 00:00 2023-11-27T15:09:08.763 INFO:teuthology.orchestra.run.smithi133.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 19 MB/s | 7.0 MB 00:00 2023-11-27T15:09:08.787 INFO:teuthology.orchestra.run.smithi177.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 224 kB/s | 41 kB 00:00 2023-11-27T15:09:08.812 INFO:teuthology.orchestra.run.smithi177.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 119 kB/s | 25 kB 00:00 2023-11-27T15:09:08.813 INFO:teuthology.orchestra.run.smithi133.stdout:(10/10): toolbox-0.0.99.3-0.6.module+el8.6.0+16 6.2 MB/s | 2.2 MB 00:00 2023-11-27T15:09:08.814 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2023-11-27T15:09:08.814 INFO:teuthology.orchestra.run.smithi133.stdout:Total 22 MB/s | 18 MB 00:00 2023-11-27T15:09:08.828 INFO:teuthology.orchestra.run.smithi174.stdout:(4/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.6 MB/s | 524 kB 00:00 2023-11-27T15:09:08.837 INFO:teuthology.orchestra.run.smithi177.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 403 kB/s | 94 kB 00:00 2023-11-27T15:09:08.853 INFO:teuthology.orchestra.run.smithi174.stdout:(5/10): udica-0.2.6-4.module+el8.6.0+16771+28df 281 kB/s | 49 kB 00:00 2023-11-27T15:09:08.888 INFO:teuthology.orchestra.run.smithi177.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 492 kB/s | 49 kB 00:00 2023-11-27T15:09:08.946 INFO:teuthology.orchestra.run.smithi174.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 1.2 MB/s | 149 kB 00:00 2023-11-27T15:09:08.963 INFO:teuthology.orchestra.run.smithi177.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 3.4 MB/s | 524 kB 00:00 2023-11-27T15:09:09.071 INFO:teuthology.orchestra.run.smithi174.stdout:(7/10): buildah-1.26.2-1.module+el8.6.0+16771+2 25 MB/s | 8.0 MB 00:00 2023-11-27T15:09:09.129 INFO:teuthology.orchestra.run.smithi174.stdout:(8/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 773 kB/s | 213 kB 00:00 2023-11-27T15:09:09.147 INFO:teuthology.orchestra.run.smithi177.stdout:(6/10): buildah-1.26.2-1.module+el8.6.0+16771+2 26 MB/s | 8.0 MB 00:00 2023-11-27T15:09:09.151 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2023-11-27T15:09:09.172 INFO:teuthology.orchestra.run.smithi177.stdout:(7/10): python3-podman-4.0.1-1.module+el8.6.0+1 526 kB/s | 149 kB 00:00 2023-11-27T15:09:09.174 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2023-11-27T15:09:09.175 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2023-11-27T15:09:09.198 INFO:teuthology.orchestra.run.smithi177.stdout:(8/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 910 kB/s | 213 kB 00:00 2023-11-27T15:09:09.221 INFO:teuthology.orchestra.run.smithi174.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 15 MB/s | 2.2 MB 00:00 2023-11-27T15:09:09.313 INFO:teuthology.orchestra.run.smithi174.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 19 MB/s | 7.0 MB 00:00 2023-11-27T15:09:09.314 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-27T15:09:09.314 INFO:teuthology.orchestra.run.smithi174.stdout:Total 24 MB/s | 18 MB 00:00 2023-11-27T15:09:09.364 INFO:teuthology.orchestra.run.smithi177.stdout:(9/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 11 MB/s | 2.2 MB 00:00 2023-11-27T15:09:09.448 INFO:teuthology.orchestra.run.smithi177.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 23 MB/s | 7.0 MB 00:00 2023-11-27T15:09:09.448 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2023-11-27T15:09:09.448 INFO:teuthology.orchestra.run.smithi177.stdout:Total 22 MB/s | 18 MB 00:00 2023-11-27T15:09:09.513 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-27T15:09:09.532 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-27T15:09:09.532 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-27T15:09:09.649 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2023-11-27T15:09:09.668 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2023-11-27T15:09:09.668 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2023-11-27T15:09:09.671 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2023-11-27T15:09:09.675 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2023-11-27T15:09:09.714 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-27T15:09:09.718 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-27T15:09:09.849 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2023-11-27T15:09:09.853 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2023-11-27T15:09:10.245 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2023-11-27T15:09:10.253 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-27T15:09:10.408 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2023-11-27T15:09:10.424 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-11-27T15:09:10.455 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-11-27T15:09:10.625 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-11-27T15:09:10.634 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-27T15:09:10.642 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-27T15:09:10.803 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-27T15:09:10.861 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-11-27T15:09:10.868 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-11-27T15:09:11.022 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-11-27T15:09:11.056 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-11-27T15:09:11.107 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-11-27T15:09:11.213 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-11-27T15:09:11.438 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-11-27T15:09:11.460 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-11-27T15:09:11.606 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-11-27T15:09:12.217 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-11-27T15:09:12.235 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-11-27T15:09:12.385 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-11-27T15:09:13.098 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-11-27T15:09:13.149 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-11-27T15:09:13.348 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-11-27T15:09:13.415 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-11-27T15:09:13.508 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-11-27T15:09:13.552 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-11-27T15:09:13.606 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-11-27T15:09:13.719 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-27T15:09:13.792 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-11-27T15:09:13.847 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-27T15:09:13.960 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-11-27T15:09:13.986 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-27T15:09:13.986 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-11-27T15:09:13.986 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-27T15:09:13.986 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-11-27T15:09:13.986 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-11-27T15:09:13.986 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-11-27T15:09:13.986 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-11-27T15:09:13.986 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-11-27T15:09:13.987 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-11-27T15:09:13.987 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-11-27T15:09:14.162 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-27T15:09:14.241 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-27T15:09:14.241 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-11-27T15:09:14.241 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-27T15:09:14.241 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-11-27T15:09:14.241 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-11-27T15:09:14.241 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-11-27T15:09:14.242 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-11-27T15:09:14.242 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-11-27T15:09:14.242 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-11-27T15:09:14.242 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-11-27T15:09:14.432 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-11-27T15:09:14.432 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-11-27T15:09:14.432 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-11-27T15:09:14.432 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-11-27T15:09:14.432 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-11-27T15:09:14.432 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-11-27T15:09:14.432 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-11-27T15:09:14.432 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-11-27T15:09:14.433 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-11-27T15:09:14.433 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-11-27T15:09:15.177 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-11-27T15:09:15.177 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:05:35 ago on Mon 27 Nov 2023 03:03:39 PM UTC. 2023-11-27T15:09:15.679 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-11-27T15:09:15.679 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:05:19 ago on Mon 27 Nov 2023 03:03:56 PM UTC. 2023-11-27T15:09:15.814 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-11-27T15:09:15.814 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:31 ago on Mon 27 Nov 2023 03:06:44 PM UTC. 2023-11-27T15:09:23.701 INFO:teuthology.orchestra.run.smithi133.stdout:Installed products updated. 2023-11-27T15:09:23.880 INFO:teuthology.orchestra.run.smithi174.stdout:Installed products updated. 2023-11-27T15:09:23.911 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:09:23.912 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2023-11-27T15:09:23.912 INFO:teuthology.orchestra.run.smithi133.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:09:23.912 INFO:teuthology.orchestra.run.smithi133.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-11-27T15:09:23.912 INFO:teuthology.orchestra.run.smithi133.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:09:23.912 INFO:teuthology.orchestra.run.smithi133.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-11-27T15:09:23.912 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-11-27T15:09:23.912 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-11-27T15:09:23.912 INFO:teuthology.orchestra.run.smithi133.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:09:23.912 INFO:teuthology.orchestra.run.smithi133.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:09:23.912 INFO:teuthology.orchestra.run.smithi133.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-11-27T15:09:23.912 INFO:teuthology.orchestra.run.smithi133.stdout: yajl-2.1.0-10.el8.x86_64 2023-11-27T15:09:23.912 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:09:23.912 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2023-11-27T15:09:24.087 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-27T15:09:24.088 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-27T15:09:24.088 INFO:teuthology.orchestra.run.smithi174.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:09:24.088 INFO:teuthology.orchestra.run.smithi174.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-11-27T15:09:24.088 INFO:teuthology.orchestra.run.smithi174.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:09:24.088 INFO:teuthology.orchestra.run.smithi174.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-11-27T15:09:24.088 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-11-27T15:09:24.088 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-11-27T15:09:24.088 INFO:teuthology.orchestra.run.smithi174.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:09:24.088 INFO:teuthology.orchestra.run.smithi174.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:09:24.088 INFO:teuthology.orchestra.run.smithi174.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-11-27T15:09:24.088 INFO:teuthology.orchestra.run.smithi174.stdout: yajl-2.1.0-10.el8.x86_64 2023-11-27T15:09:24.088 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-27T15:09:24.088 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-27T15:09:24.138 DEBUG:teuthology.parallel:result is None 2023-11-27T15:09:24.253 DEBUG:teuthology.parallel:result is None 2023-11-27T15:09:25.046 INFO:teuthology.orchestra.run.smithi177.stdout:Installed products updated. 2023-11-27T15:09:25.272 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-27T15:09:25.272 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2023-11-27T15:09:25.272 INFO:teuthology.orchestra.run.smithi177.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:09:25.272 INFO:teuthology.orchestra.run.smithi177.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-11-27T15:09:25.272 INFO:teuthology.orchestra.run.smithi177.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:09:25.273 INFO:teuthology.orchestra.run.smithi177.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-11-27T15:09:25.273 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-11-27T15:09:25.273 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-11-27T15:09:25.273 INFO:teuthology.orchestra.run.smithi177.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:09:25.273 INFO:teuthology.orchestra.run.smithi177.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-11-27T15:09:25.273 INFO:teuthology.orchestra.run.smithi177.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-11-27T15:09:25.273 INFO:teuthology.orchestra.run.smithi177.stdout: yajl-2.1.0-10.el8.x86_64 2023-11-27T15:09:25.273 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-27T15:09:25.273 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2023-11-27T15:09:25.426 DEBUG:teuthology.parallel:result is None 2023-11-27T15:09:25.426 INFO:teuthology.run_tasks:Running task install... 2023-11-27T15:09:25.434 DEBUG:teuthology.task.install:project ceph 2023-11-27T15:09:25.434 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b'}} 2023-11-27T15:09:25.434 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b'} 2023-11-27T15:09:25.435 INFO:teuthology.task.install:Using flavor: default 2023-11-27T15:09:25.440 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-27T15:09:25.440 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-27T15:09:25.440 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b', 'tag': None, 'wait_for_package': False} 2023-11-27T15:09:25.441 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:09:25.442 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b', 'tag': None, 'wait_for_package': False} 2023-11-27T15:09:25.442 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:09:25.443 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b', 'tag': None, 'wait_for_package': False} 2023-11-27T15:09:25.443 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:09:25.583 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/ 2023-11-27T15:09:25.583 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3906.g1c956514 2023-11-27T15:09:25.590 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/ 2023-11-27T15:09:25.590 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3906.g1c956514 2023-11-27T15:09:25.648 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/ 2023-11-27T15:09:25.648 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3906.g1c956514 2023-11-27T15:09:25.747 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-27T15:09:25.747 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-27T15:09:25.747 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-27T15:09:25.749 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-27T15:09:25.749 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-11-27T15:09:25.749 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-27T15:09:25.751 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-27T15:09:25.752 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-27T15:09:25.752 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-27T15:09:25.776 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-27T15:09:25.776 DEBUG:teuthology.orchestra.run.smithi174:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1c9565146c8647d4e00eb6c7ae626dc192680e7b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-27T15:09:25.779 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-27T15:09:25.779 DEBUG:teuthology.orchestra.run.smithi177:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1c9565146c8647d4e00eb6c7ae626dc192680e7b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-27T15:09:25.823 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-27T15:09:25.823 DEBUG:teuthology.orchestra.run.smithi133:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1c9565146c8647d4e00eb6c7ae626dc192680e7b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-27T15:09:25.842 DEBUG:teuthology.orchestra.run.smithi174:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-27T15:09:25.846 DEBUG:teuthology.orchestra.run.smithi177:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-27T15:09:25.919 DEBUG:teuthology.orchestra.run.smithi174:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-27T15:09:25.922 DEBUG:teuthology.orchestra.run.smithi177:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-27T15:09:25.935 DEBUG:teuthology.orchestra.run.smithi133:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-27T15:09:25.984 INFO:teuthology.orchestra.run.smithi174.stdout:check_obsoletes = 1 2023-11-27T15:09:25.986 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum clean all 2023-11-27T15:09:25.987 INFO:teuthology.orchestra.run.smithi177.stdout:check_obsoletes = 1 2023-11-27T15:09:25.988 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum clean all 2023-11-27T15:09:26.030 DEBUG:teuthology.orchestra.run.smithi133:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-27T15:09:26.098 INFO:teuthology.orchestra.run.smithi133.stdout:check_obsoletes = 1 2023-11-27T15:09:26.101 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum clean all 2023-11-27T15:09:26.377 INFO:teuthology.orchestra.run.smithi174.stdout:Updating Subscription Management repositories. 2023-11-27T15:09:26.378 INFO:teuthology.orchestra.run.smithi177.stdout:Updating Subscription Management repositories. 2023-11-27T15:09:26.547 INFO:teuthology.orchestra.run.smithi133.stdout:Updating Subscription Management repositories. 2023-11-27T15:09:27.239 INFO:teuthology.orchestra.run.smithi174.stdout:44 files removed 2023-11-27T15:09:27.275 INFO:teuthology.orchestra.run.smithi177.stdout:44 files removed 2023-11-27T15:09:27.289 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-27T15:09:27.323 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-27T15:09:27.604 INFO:teuthology.orchestra.run.smithi133.stdout:44 files removed 2023-11-27T15:09:27.641 INFO:teuthology.orchestra.run.smithi174.stdout:Updating Subscription Management repositories. 2023-11-27T15:09:27.660 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-27T15:09:27.679 INFO:teuthology.orchestra.run.smithi177.stdout:Updating Subscription Management repositories. 2023-11-27T15:09:28.108 INFO:teuthology.orchestra.run.smithi133.stdout:Updating Subscription Management repositories. 2023-11-27T15:09:29.135 INFO:teuthology.orchestra.run.smithi174.stdout:ceph packages for x86_64 213 kB/s | 80 kB 00:00 2023-11-27T15:09:29.166 INFO:teuthology.orchestra.run.smithi177.stdout:ceph packages for x86_64 206 kB/s | 80 kB 00:00 2023-11-27T15:09:29.577 INFO:teuthology.orchestra.run.smithi177.stdout:ceph noarch packages 25 kB/s | 9.4 kB 00:00 2023-11-27T15:09:29.665 INFO:teuthology.orchestra.run.smithi174.stdout:ceph noarch packages 19 kB/s | 9.4 kB 00:00 2023-11-27T15:09:29.810 INFO:teuthology.orchestra.run.smithi133.stdout:ceph packages for x86_64 157 kB/s | 80 kB 00:00 2023-11-27T15:09:29.863 INFO:teuthology.orchestra.run.smithi177.stdout:ceph source packages 5.6 kB/s | 1.5 kB 00:00 2023-11-27T15:09:29.923 INFO:teuthology.orchestra.run.smithi174.stdout:ceph source packages 6.3 kB/s | 1.5 kB 00:00 2023-11-27T15:09:30.039 INFO:teuthology.orchestra.run.smithi177.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-11-27T15:09:30.091 INFO:teuthology.orchestra.run.smithi174.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-11-27T15:09:30.131 INFO:teuthology.orchestra.run.smithi133.stdout:ceph noarch packages 33 kB/s | 9.4 kB 00:00 2023-11-27T15:09:30.590 INFO:teuthology.orchestra.run.smithi133.stdout:ceph source packages 3.4 kB/s | 1.5 kB 00:00 2023-11-27T15:09:30.741 INFO:teuthology.orchestra.run.smithi133.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-11-27T15:09:31.419 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-11-27T15:09:32.385 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 9.8 MB/s | 16 MB 00:01 2023-11-27T15:09:33.625 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 4.5 MB/s | 16 MB 00:03 2023-11-27T15:09:34.975 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 343 kB/s | 24 kB 00:00 2023-11-27T15:09:36.093 INFO:teuthology.orchestra.run.smithi177.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 51 MB/s | 53 MB 00:01 2023-11-27T15:09:36.475 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 578 kB/s | 24 kB 00:00 2023-11-27T15:09:37.125 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 437 kB/s | 24 kB 00:00 2023-11-27T15:09:37.752 INFO:teuthology.orchestra.run.smithi133.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 39 MB/s | 47 MB 00:01 2023-11-27T15:09:38.269 INFO:teuthology.orchestra.run.smithi174.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 47 MB 00:01 2023-11-27T15:09:45.138 INFO:teuthology.orchestra.run.smithi177.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 26 MB/s | 47 MB 00:01 2023-11-27T15:09:46.404 INFO:teuthology.orchestra.run.smithi174.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 52 MB/s | 53 MB 00:01 2023-11-27T15:09:46.997 INFO:teuthology.orchestra.run.smithi133.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 30 MB/s | 53 MB 00:01 2023-11-27T15:09:56.168 INFO:teuthology.orchestra.run.smithi177.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-27T15:09:56.169 INFO:teuthology.orchestra.run.smithi177.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-27T15:09:56.183 INFO:teuthology.orchestra.run.smithi177.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-27T15:09:56.184 INFO:teuthology.orchestra.run.smithi177.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-27T15:09:56.185 INFO:teuthology.orchestra.run.smithi177.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-27T15:09:56.186 INFO:teuthology.orchestra.run.smithi177.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-27T15:09:56.188 INFO:teuthology.orchestra.run.smithi177.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-27T15:09:56.191 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-27T15:09:56.192 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-27T15:09:56.322 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2023-11-27T15:09:56.334 INFO:teuthology.orchestra.run.smithi177.stdout:=========================================================================================================================== 2023-11-27T15:09:56.334 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2023-11-27T15:09:56.335 INFO:teuthology.orchestra.run.smithi177.stdout:=========================================================================================================================== 2023-11-27T15:09:56.335 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2023-11-27T15:09:56.335 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-27T15:09:56.335 INFO:teuthology.orchestra.run.smithi177.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-27T15:09:56.335 INFO:teuthology.orchestra.run.smithi177.stdout: ceph x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.4 k 2023-11-27T15:09:56.335 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base x86_64 2:18.0.0-3906.g1c956514.el8 ceph 5.3 M 2023-11-27T15:09:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 848 k 2023-11-27T15:09:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3906.g1c956514.el8 ceph 165 k 2023-11-27T15:09:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr x86_64 2:18.0.0-3906.g1c956514.el8 ceph 1.5 M 2023-11-27T15:09:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 127 k 2023-11-27T15:09:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 1.7 M 2023-11-27T15:09:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 7.4 M 2023-11-27T15:09:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 47 k 2023-11-27T15:09:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 13 M 2023-11-27T15:09:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test x86_64 2:18.0.0-3906.g1c956514.el8 ceph 41 M 2023-11-27T15:09:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 265 k 2023-11-27T15:09:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 83 k 2023-11-27T15:09:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 258 k 2023-11-27T15:09:56.336 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 18 k 2023-11-27T15:09:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-27T15:09:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-27T15:09:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-27T15:09:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 31 k 2023-11-27T15:09:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 771 k 2023-11-27T15:09:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 139 k 2023-11-27T15:09:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-27T15:09:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-27T15:09:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-27T15:09:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-27T15:09:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-27T15:09:56.337 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-27T15:09:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs x86_64 2:18.0.0-3906.g1c956514.el8 ceph 197 k 2023-11-27T15:09:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados x86_64 2:18.0.0-3906.g1c956514.el8 ceph 371 k 2023-11-27T15:09:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 364 k 2023-11-27T15:09:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 107 k 2023-11-27T15:09:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 94 k 2023-11-27T15:09:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.4 M 2023-11-27T15:09:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 180 k 2023-11-27T15:09:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-27T15:09:56.338 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrading: 2023-11-27T15:09:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-27T15:09:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: librados2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.7 M 2023-11-27T15:09:56.338 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.6 M 2023-11-27T15:09:56.339 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2023-11-27T15:09:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-27T15:09:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 22 M 2023-11-27T15:09:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 23 k 2023-11-27T15:09:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds x86_64 2:18.0.0-3906.g1c956514.el8 ceph 2.3 M 2023-11-27T15:09:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 240 k 2023-11-27T15:09:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon x86_64 2:18.0.0-3906.g1c956514.el8 ceph 4.6 M 2023-11-27T15:09:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 18 M 2023-11-27T15:09:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 15 k 2023-11-27T15:09:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux x86_64 2:18.0.0-3906.g1c956514.el8 ceph 24 k 2023-11-27T15:09:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-27T15:09:56.339 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-27T15:09:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-27T15:09:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-27T15:09:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-27T15:09:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite x86_64 2:18.0.0-3906.g1c956514.el8 ceph 137 k 2023-11-27T15:09:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-27T15:09:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-27T15:09:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-27T15:09:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-27T15:09:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-27T15:09:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-27T15:09:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 473 k 2023-11-27T15:09:56.340 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-27T15:09:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.5 M 2023-11-27T15:09:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-27T15:09:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-27T15:09:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-27T15:09:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-27T15:09:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-27T15:09:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-27T15:09:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-27T15:09:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-27T15:09:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-27T15:09:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-27T15:09:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-27T15:09:56.341 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-27T15:09:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-27T15:09:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-27T15:09:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-27T15:09:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-27T15:09:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 45 k 2023-11-27T15:09:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 119 k 2023-11-27T15:09:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-27T15:09:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-27T15:09:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-27T15:09:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-27T15:09:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-27T15:09:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-27T15:09:56.342 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-27T15:09:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-27T15:09:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-27T15:09:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-27T15:09:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-27T15:09:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-27T15:09:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-27T15:09:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-27T15:09:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-27T15:09:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-27T15:09:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-27T15:09:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-27T15:09:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-27T15:09:56.343 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-27T15:09:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-27T15:09:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-27T15:09:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-27T15:09:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-27T15:09:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-27T15:09:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-27T15:09:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-27T15:09:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-27T15:09:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-27T15:09:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-27T15:09:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-27T15:09:56.344 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-27T15:09:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-27T15:09:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-27T15:09:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-27T15:09:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-27T15:09:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-27T15:09:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-27T15:09:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-27T15:09:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-27T15:09:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-27T15:09:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-27T15:09:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-27T15:09:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-27T15:09:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-27T15:09:56.345 INFO:teuthology.orchestra.run.smithi177.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-27T15:09:56.346 INFO:teuthology.orchestra.run.smithi177.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-27T15:09:56.346 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-27T15:09:56.346 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-27T15:09:56.346 INFO:teuthology.orchestra.run.smithi177.stdout:Enabling module streams: 2023-11-27T15:09:56.346 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql 10 2023-11-27T15:09:56.346 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-27T15:09:56.346 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2023-11-27T15:09:56.346 INFO:teuthology.orchestra.run.smithi177.stdout:=========================================================================================================================== 2023-11-27T15:09:56.346 INFO:teuthology.orchestra.run.smithi177.stdout:Install 128 Packages 2023-11-27T15:09:56.346 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrade 3 Packages 2023-11-27T15:09:56.346 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-27T15:09:56.347 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 197 M 2023-11-27T15:09:56.347 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2023-11-27T15:09:56.504 INFO:teuthology.orchestra.run.smithi177.stdout:(1/131): ceph-18.0.0-3906.g1c956514.el8.x86_64. 52 kB/s | 6.4 kB 00:00 2023-11-27T15:09:56.688 INFO:teuthology.orchestra.run.smithi177.stdout:(2/131): ceph-fuse-18.0.0-3906.g1c956514.el8.x8 4.5 MB/s | 848 kB 00:00 2023-11-27T15:09:56.738 INFO:teuthology.orchestra.run.smithi177.stdout:(3/131): ceph-immutable-object-cache-18.0.0-390 3.2 MB/s | 165 kB 00:00 2023-11-27T15:09:57.012 INFO:teuthology.orchestra.run.smithi177.stdout:(4/131): ceph-mds-18.0.0-3906.g1c956514.el8.x86 8.3 MB/s | 2.3 MB 00:00 2023-11-27T15:09:57.180 INFO:teuthology.orchestra.run.smithi177.stdout:(5/131): ceph-mgr-18.0.0-3906.g1c956514.el8.x86 9.2 MB/s | 1.5 MB 00:00 2023-11-27T15:09:57.405 INFO:teuthology.orchestra.run.smithi177.stdout:(6/131): ceph-base-18.0.0-3906.g1c956514.el8.x8 5.2 MB/s | 5.3 MB 00:01 2023-11-27T15:09:57.460 INFO:teuthology.orchestra.run.smithi174.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-27T15:09:57.461 INFO:teuthology.orchestra.run.smithi174.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-27T15:09:57.474 INFO:teuthology.orchestra.run.smithi174.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-27T15:09:57.475 INFO:teuthology.orchestra.run.smithi174.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-27T15:09:57.477 INFO:teuthology.orchestra.run.smithi174.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-27T15:09:57.478 INFO:teuthology.orchestra.run.smithi174.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-27T15:09:57.480 INFO:teuthology.orchestra.run.smithi174.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-27T15:09:57.483 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-27T15:09:57.484 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-27T15:09:57.589 INFO:teuthology.orchestra.run.smithi177.stdout:(7/131): ceph-mon-18.0.0-3906.g1c956514.el8.x86 11 MB/s | 4.6 MB 00:00 2023-11-27T15:09:57.615 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2023-11-27T15:09:57.627 INFO:teuthology.orchestra.run.smithi174.stdout:=========================================================================================================================== 2023-11-27T15:09:57.627 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2023-11-27T15:09:57.627 INFO:teuthology.orchestra.run.smithi174.stdout:=========================================================================================================================== 2023-11-27T15:09:57.627 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2023-11-27T15:09:57.627 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-27T15:09:57.627 INFO:teuthology.orchestra.run.smithi174.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-27T15:09:57.627 INFO:teuthology.orchestra.run.smithi174.stdout: ceph x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.4 k 2023-11-27T15:09:57.627 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base x86_64 2:18.0.0-3906.g1c956514.el8 ceph 5.3 M 2023-11-27T15:09:57.628 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 848 k 2023-11-27T15:09:57.628 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3906.g1c956514.el8 ceph 165 k 2023-11-27T15:09:57.628 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr x86_64 2:18.0.0-3906.g1c956514.el8 ceph 1.5 M 2023-11-27T15:09:57.628 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 127 k 2023-11-27T15:09:57.628 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 1.7 M 2023-11-27T15:09:57.628 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 7.4 M 2023-11-27T15:09:57.628 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 47 k 2023-11-27T15:09:57.628 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 13 M 2023-11-27T15:09:57.628 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test x86_64 2:18.0.0-3906.g1c956514.el8 ceph 41 M 2023-11-27T15:09:57.628 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 265 k 2023-11-27T15:09:57.628 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 83 k 2023-11-27T15:09:57.629 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 258 k 2023-11-27T15:09:57.629 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 18 k 2023-11-27T15:09:57.629 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-27T15:09:57.629 INFO:teuthology.orchestra.run.smithi174.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-27T15:09:57.629 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-27T15:09:57.629 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 31 k 2023-11-27T15:09:57.629 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 771 k 2023-11-27T15:09:57.629 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 139 k 2023-11-27T15:09:57.629 INFO:teuthology.orchestra.run.smithi174.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-27T15:09:57.629 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-27T15:09:57.629 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-27T15:09:57.629 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-27T15:09:57.630 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-27T15:09:57.630 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-27T15:09:57.630 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs x86_64 2:18.0.0-3906.g1c956514.el8 ceph 197 k 2023-11-27T15:09:57.630 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados x86_64 2:18.0.0-3906.g1c956514.el8 ceph 371 k 2023-11-27T15:09:57.630 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 364 k 2023-11-27T15:09:57.630 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 107 k 2023-11-27T15:09:57.630 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 94 k 2023-11-27T15:09:57.630 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.4 M 2023-11-27T15:09:57.630 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 180 k 2023-11-27T15:09:57.630 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-27T15:09:57.630 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrading: 2023-11-27T15:09:57.630 INFO:teuthology.orchestra.run.smithi174.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-27T15:09:57.631 INFO:teuthology.orchestra.run.smithi174.stdout: librados2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.7 M 2023-11-27T15:09:57.631 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.6 M 2023-11-27T15:09:57.631 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2023-11-27T15:09:57.631 INFO:teuthology.orchestra.run.smithi174.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-27T15:09:57.631 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 22 M 2023-11-27T15:09:57.631 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 23 k 2023-11-27T15:09:57.631 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds x86_64 2:18.0.0-3906.g1c956514.el8 ceph 2.3 M 2023-11-27T15:09:57.631 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 240 k 2023-11-27T15:09:57.631 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon x86_64 2:18.0.0-3906.g1c956514.el8 ceph 4.6 M 2023-11-27T15:09:57.631 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 18 M 2023-11-27T15:09:57.631 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 15 k 2023-11-27T15:09:57.631 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux x86_64 2:18.0.0-3906.g1c956514.el8 ceph 24 k 2023-11-27T15:09:57.631 INFO:teuthology.orchestra.run.smithi174.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-27T15:09:57.631 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-27T15:09:57.632 INFO:teuthology.orchestra.run.smithi174.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-27T15:09:57.632 INFO:teuthology.orchestra.run.smithi174.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-27T15:09:57.632 INFO:teuthology.orchestra.run.smithi174.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-27T15:09:57.632 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite x86_64 2:18.0.0-3906.g1c956514.el8 ceph 137 k 2023-11-27T15:09:57.632 INFO:teuthology.orchestra.run.smithi174.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-27T15:09:57.632 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-27T15:09:57.632 INFO:teuthology.orchestra.run.smithi174.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-27T15:09:57.632 INFO:teuthology.orchestra.run.smithi174.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-27T15:09:57.632 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-27T15:09:57.632 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-27T15:09:57.632 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 473 k 2023-11-27T15:09:57.632 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-27T15:09:57.633 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.5 M 2023-11-27T15:09:57.633 INFO:teuthology.orchestra.run.smithi174.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-27T15:09:57.633 INFO:teuthology.orchestra.run.smithi174.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-27T15:09:57.633 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-27T15:09:57.633 INFO:teuthology.orchestra.run.smithi174.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-27T15:09:57.633 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-27T15:09:57.633 INFO:teuthology.orchestra.run.smithi174.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-27T15:09:57.633 INFO:teuthology.orchestra.run.smithi174.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-27T15:09:57.633 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-27T15:09:57.633 INFO:teuthology.orchestra.run.smithi174.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-27T15:09:57.633 INFO:teuthology.orchestra.run.smithi174.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-27T15:09:57.633 INFO:teuthology.orchestra.run.smithi174.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-27T15:09:57.633 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-27T15:09:57.634 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-27T15:09:57.634 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-27T15:09:57.634 INFO:teuthology.orchestra.run.smithi174.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-27T15:09:57.634 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-27T15:09:57.634 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 45 k 2023-11-27T15:09:57.634 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 119 k 2023-11-27T15:09:57.634 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-27T15:09:57.634 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-27T15:09:57.634 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-27T15:09:57.634 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-27T15:09:57.634 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-27T15:09:57.634 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-27T15:09:57.635 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-27T15:09:57.635 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-27T15:09:57.635 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-27T15:09:57.635 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-27T15:09:57.635 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-27T15:09:57.635 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-27T15:09:57.635 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-27T15:09:57.635 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-27T15:09:57.635 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-27T15:09:57.635 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-27T15:09:57.635 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-27T15:09:57.635 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-27T15:09:57.635 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-27T15:09:57.635 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-27T15:09:57.636 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-27T15:09:57.636 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-27T15:09:57.636 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-27T15:09:57.636 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-27T15:09:57.636 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-27T15:09:57.636 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-27T15:09:57.636 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-27T15:09:57.636 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-27T15:09:57.636 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-27T15:09:57.636 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-27T15:09:57.636 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-27T15:09:57.636 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-27T15:09:57.636 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-27T15:09:57.636 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-27T15:09:57.636 INFO:teuthology.orchestra.run.smithi174.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-27T15:09:57.637 INFO:teuthology.orchestra.run.smithi174.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-27T15:09:57.637 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-27T15:09:57.637 INFO:teuthology.orchestra.run.smithi174.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-27T15:09:57.637 INFO:teuthology.orchestra.run.smithi174.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-27T15:09:57.637 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-27T15:09:57.637 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-27T15:09:57.637 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-27T15:09:57.637 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-27T15:09:57.637 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-27T15:09:57.637 INFO:teuthology.orchestra.run.smithi174.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-27T15:09:57.637 INFO:teuthology.orchestra.run.smithi174.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-27T15:09:57.637 INFO:teuthology.orchestra.run.smithi174.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-27T15:09:57.637 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-27T15:09:57.638 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-27T15:09:57.638 INFO:teuthology.orchestra.run.smithi174.stdout:Enabling module streams: 2023-11-27T15:09:57.638 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql 10 2023-11-27T15:09:57.638 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-27T15:09:57.638 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2023-11-27T15:09:57.638 INFO:teuthology.orchestra.run.smithi174.stdout:=========================================================================================================================== 2023-11-27T15:09:57.638 INFO:teuthology.orchestra.run.smithi174.stdout:Install 128 Packages 2023-11-27T15:09:57.638 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrade 3 Packages 2023-11-27T15:09:57.638 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-27T15:09:57.639 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 197 M 2023-11-27T15:09:57.639 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2023-11-27T15:09:58.337 INFO:teuthology.orchestra.run.smithi174.stdout:(1/131): ceph-18.0.0-3906.g1c956514.el8.x86_64. 21 kB/s | 6.4 kB 00:00 2023-11-27T15:09:58.547 INFO:teuthology.orchestra.run.smithi133.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-27T15:09:58.548 INFO:teuthology.orchestra.run.smithi133.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-27T15:09:58.563 INFO:teuthology.orchestra.run.smithi133.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-27T15:09:58.564 INFO:teuthology.orchestra.run.smithi133.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-27T15:09:58.566 INFO:teuthology.orchestra.run.smithi133.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-27T15:09:58.567 INFO:teuthology.orchestra.run.smithi133.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-27T15:09:58.569 INFO:teuthology.orchestra.run.smithi133.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-27T15:09:58.573 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-27T15:09:58.574 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-27T15:09:58.709 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2023-11-27T15:09:58.721 INFO:teuthology.orchestra.run.smithi133.stdout:=========================================================================================================================== 2023-11-27T15:09:58.721 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2023-11-27T15:09:58.722 INFO:teuthology.orchestra.run.smithi133.stdout:=========================================================================================================================== 2023-11-27T15:09:58.722 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2023-11-27T15:09:58.722 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-27T15:09:58.722 INFO:teuthology.orchestra.run.smithi133.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-27T15:09:58.722 INFO:teuthology.orchestra.run.smithi133.stdout: ceph x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.4 k 2023-11-27T15:09:58.722 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base x86_64 2:18.0.0-3906.g1c956514.el8 ceph 5.3 M 2023-11-27T15:09:58.722 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 848 k 2023-11-27T15:09:58.723 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3906.g1c956514.el8 ceph 165 k 2023-11-27T15:09:58.723 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr x86_64 2:18.0.0-3906.g1c956514.el8 ceph 1.5 M 2023-11-27T15:09:58.723 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 127 k 2023-11-27T15:09:58.723 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 1.7 M 2023-11-27T15:09:58.723 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 7.4 M 2023-11-27T15:09:58.723 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 47 k 2023-11-27T15:09:58.723 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 13 M 2023-11-27T15:09:58.724 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test x86_64 2:18.0.0-3906.g1c956514.el8 ceph 41 M 2023-11-27T15:09:58.724 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 265 k 2023-11-27T15:09:58.724 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 83 k 2023-11-27T15:09:58.724 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 258 k 2023-11-27T15:09:58.724 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 18 k 2023-11-27T15:09:58.724 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-27T15:09:58.725 INFO:teuthology.orchestra.run.smithi133.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-27T15:09:58.725 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-27T15:09:58.725 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 31 k 2023-11-27T15:09:58.725 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 771 k 2023-11-27T15:09:58.725 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel x86_64 2:18.0.0-3906.g1c956514.el8 ceph 139 k 2023-11-27T15:09:58.725 INFO:teuthology.orchestra.run.smithi133.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-27T15:09:58.725 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-27T15:09:58.726 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-27T15:09:58.726 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-27T15:09:58.726 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-27T15:09:58.726 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-27T15:09:58.726 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs x86_64 2:18.0.0-3906.g1c956514.el8 ceph 197 k 2023-11-27T15:09:58.726 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados x86_64 2:18.0.0-3906.g1c956514.el8 ceph 371 k 2023-11-27T15:09:58.727 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 364 k 2023-11-27T15:09:58.727 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw x86_64 2:18.0.0-3906.g1c956514.el8 ceph 107 k 2023-11-27T15:09:58.727 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 94 k 2023-11-27T15:09:58.727 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.4 M 2023-11-27T15:09:58.727 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 180 k 2023-11-27T15:09:58.727 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-27T15:09:58.727 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrading: 2023-11-27T15:09:58.728 INFO:teuthology.orchestra.run.smithi133.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-27T15:09:58.728 INFO:teuthology.orchestra.run.smithi133.stdout: librados2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.7 M 2023-11-27T15:09:58.728 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 3.6 M 2023-11-27T15:09:58.728 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2023-11-27T15:09:58.728 INFO:teuthology.orchestra.run.smithi133.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-27T15:09:58.728 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 22 M 2023-11-27T15:09:58.728 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 23 k 2023-11-27T15:09:58.729 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds x86_64 2:18.0.0-3906.g1c956514.el8 ceph 2.3 M 2023-11-27T15:09:58.729 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 240 k 2023-11-27T15:09:58.729 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon x86_64 2:18.0.0-3906.g1c956514.el8 ceph 4.6 M 2023-11-27T15:09:58.729 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd x86_64 2:18.0.0-3906.g1c956514.el8 ceph 18 M 2023-11-27T15:09:58.729 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3906.g1c956514.el8 ceph-noarch 15 k 2023-11-27T15:09:58.729 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux x86_64 2:18.0.0-3906.g1c956514.el8 ceph 24 k 2023-11-27T15:09:58.729 INFO:teuthology.orchestra.run.smithi133.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-27T15:09:58.730 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-27T15:09:58.730 INFO:teuthology.orchestra.run.smithi133.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-27T15:09:58.730 INFO:teuthology.orchestra.run.smithi133.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-27T15:09:58.730 INFO:teuthology.orchestra.run.smithi133.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-27T15:09:58.730 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite x86_64 2:18.0.0-3906.g1c956514.el8 ceph 137 k 2023-11-27T15:09:58.730 INFO:teuthology.orchestra.run.smithi133.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-27T15:09:58.731 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-27T15:09:58.731 INFO:teuthology.orchestra.run.smithi133.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-27T15:09:58.731 INFO:teuthology.orchestra.run.smithi133.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-27T15:09:58.731 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-27T15:09:58.731 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-27T15:09:58.731 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 473 k 2023-11-27T15:09:58.731 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-27T15:09:58.731 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2 x86_64 2:18.0.0-3906.g1c956514.el8 ceph 6.5 M 2023-11-27T15:09:58.731 INFO:teuthology.orchestra.run.smithi133.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-27T15:09:58.731 INFO:teuthology.orchestra.run.smithi133.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-27T15:09:58.731 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-27T15:09:58.731 INFO:teuthology.orchestra.run.smithi133.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-27T15:09:58.731 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-27T15:09:58.732 INFO:teuthology.orchestra.run.smithi133.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-27T15:09:58.732 INFO:teuthology.orchestra.run.smithi133.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-27T15:09:58.732 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-27T15:09:58.732 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-27T15:09:58.732 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-27T15:09:58.732 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-27T15:09:58.732 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-27T15:09:58.732 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-27T15:09:58.732 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-27T15:09:58.732 INFO:teuthology.orchestra.run.smithi133.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-27T15:09:58.732 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-27T15:09:58.732 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse x86_64 2:18.0.0-3906.g1c956514.el8 ceph 45 k 2023-11-27T15:09:58.733 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common x86_64 2:18.0.0-3906.g1c956514.el8 ceph 119 k 2023-11-27T15:09:58.733 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-27T15:09:58.733 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-27T15:09:58.733 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-27T15:09:58.733 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-27T15:09:58.733 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-27T15:09:58.733 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-27T15:09:58.733 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-27T15:09:58.733 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-27T15:09:58.733 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-27T15:09:58.733 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-27T15:09:58.733 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-27T15:09:58.734 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-27T15:09:58.734 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-27T15:09:58.734 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-27T15:09:58.734 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-27T15:09:58.734 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-27T15:09:58.734 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-27T15:09:58.734 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-27T15:09:58.734 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-27T15:09:58.734 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-27T15:09:58.734 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-27T15:09:58.734 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-27T15:09:58.734 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-27T15:09:58.734 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-27T15:09:58.735 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-27T15:09:58.735 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-27T15:09:58.735 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-27T15:09:58.735 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-27T15:09:58.735 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-27T15:09:58.735 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-27T15:09:58.735 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-27T15:09:58.735 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-27T15:09:58.735 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-27T15:09:58.735 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-27T15:09:58.736 INFO:teuthology.orchestra.run.smithi133.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-27T15:09:58.736 INFO:teuthology.orchestra.run.smithi133.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-27T15:09:58.736 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-27T15:09:58.736 INFO:teuthology.orchestra.run.smithi133.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-27T15:09:58.736 INFO:teuthology.orchestra.run.smithi133.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-27T15:09:58.736 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-27T15:09:58.736 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-27T15:09:58.736 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-27T15:09:58.736 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-27T15:09:58.736 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-27T15:09:58.736 INFO:teuthology.orchestra.run.smithi133.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-27T15:09:58.736 INFO:teuthology.orchestra.run.smithi133.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-27T15:09:58.737 INFO:teuthology.orchestra.run.smithi133.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-27T15:09:58.737 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-27T15:09:58.737 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-27T15:09:58.737 INFO:teuthology.orchestra.run.smithi133.stdout:Enabling module streams: 2023-11-27T15:09:58.737 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql 10 2023-11-27T15:09:58.737 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:09:58.737 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2023-11-27T15:09:58.737 INFO:teuthology.orchestra.run.smithi133.stdout:=========================================================================================================================== 2023-11-27T15:09:58.737 INFO:teuthology.orchestra.run.smithi133.stdout:Install 128 Packages 2023-11-27T15:09:58.737 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrade 3 Packages 2023-11-27T15:09:58.737 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:09:58.738 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 197 M 2023-11-27T15:09:58.738 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2023-11-27T15:09:58.846 INFO:teuthology.orchestra.run.smithi174.stdout:(2/131): ceph-fuse-18.0.0-3906.g1c956514.el8.x8 1.6 MB/s | 848 kB 00:00 2023-11-27T15:09:58.896 INFO:teuthology.orchestra.run.smithi174.stdout:(3/131): ceph-immutable-object-cache-18.0.0-390 3.3 MB/s | 165 kB 00:00 2023-11-27T15:09:58.939 INFO:teuthology.orchestra.run.smithi177.stdout:(8/131): ceph-radosgw-18.0.0-3906.g1c956514.el8 9.6 MB/s | 13 MB 00:01 2023-11-27T15:09:58.997 INFO:teuthology.orchestra.run.smithi177.stdout:(9/131): ceph-selinux-18.0.0-3906.g1c956514.el8 419 kB/s | 24 kB 00:00 2023-11-27T15:09:59.347 INFO:teuthology.orchestra.run.smithi133.stdout:(1/131): ceph-18.0.0-3906.g1c956514.el8.x86_64. 20 kB/s | 6.4 kB 00:00 2023-11-27T15:09:59.379 INFO:teuthology.orchestra.run.smithi174.stdout:(4/131): ceph-mds-18.0.0-3906.g1c956514.el8.x86 4.7 MB/s | 2.3 MB 00:00 2023-11-27T15:09:59.487 INFO:teuthology.orchestra.run.smithi177.stdout:(10/131): ceph-common-18.0.0-3906.g1c956514.el8 7.0 MB/s | 22 MB 00:03 2023-11-27T15:09:59.663 INFO:teuthology.orchestra.run.smithi174.stdout:(5/131): ceph-mgr-18.0.0-3906.g1c956514.el8.x86 5.5 MB/s | 1.5 MB 00:00 2023-11-27T15:09:59.715 INFO:teuthology.orchestra.run.smithi177.stdout:(11/131): ceph-osd-18.0.0-3906.g1c956514.el8.x8 7.7 MB/s | 18 MB 00:02 2023-11-27T15:09:59.740 INFO:teuthology.orchestra.run.smithi177.stdout:(12/131): cephfs-mirror-18.0.0-3906.g1c956514.e 1.0 MB/s | 258 kB 00:00 2023-11-27T15:09:59.838 INFO:teuthology.orchestra.run.smithi133.stdout:(2/131): ceph-fuse-18.0.0-3906.g1c956514.el8.x8 1.7 MB/s | 848 kB 00:00 2023-11-27T15:09:59.857 INFO:teuthology.orchestra.run.smithi177.stdout:(13/131): libcephfs-devel-18.0.0-3906.g1c956514 219 kB/s | 31 kB 00:00 2023-11-27T15:09:59.930 INFO:teuthology.orchestra.run.smithi133.stdout:(3/131): ceph-immutable-object-cache-18.0.0-390 1.8 MB/s | 165 kB 00:00 2023-11-27T15:09:59.974 INFO:teuthology.orchestra.run.smithi177.stdout:(14/131): libcephsqlite-18.0.0-3906.g1c956514.e 1.1 MB/s | 137 kB 00:00 2023-11-27T15:10:00.016 INFO:teuthology.orchestra.run.smithi177.stdout:(15/131): libcephfs2-18.0.0-3906.g1c956514.el8. 2.7 MB/s | 771 kB 00:00 2023-11-27T15:10:00.083 INFO:teuthology.orchestra.run.smithi177.stdout:(16/131): librados-devel-18.0.0-3906.g1c956514. 1.2 MB/s | 139 kB 00:00 2023-11-27T15:10:00.438 INFO:teuthology.orchestra.run.smithi174.stdout:(6/131): ceph-mon-18.0.0-3906.g1c956514.el8.x86 5.9 MB/s | 4.6 MB 00:00 2023-11-27T15:10:00.442 INFO:teuthology.orchestra.run.smithi177.stdout:(17/131): libradosstriper1-18.0.0-3906.g1c95651 1.1 MB/s | 473 kB 00:00 2023-11-27T15:10:00.559 INFO:teuthology.orchestra.run.smithi177.stdout:(18/131): python3-ceph-argparse-18.0.0-3906.g1c 384 kB/s | 45 kB 00:00 2023-11-27T15:10:00.692 INFO:teuthology.orchestra.run.smithi177.stdout:(19/131): python3-ceph-common-18.0.0-3906.g1c95 892 kB/s | 119 kB 00:00 2023-11-27T15:10:00.730 INFO:teuthology.orchestra.run.smithi133.stdout:(4/131): ceph-mds-18.0.0-3906.g1c956514.el8.x86 2.9 MB/s | 2.3 MB 00:00 2023-11-27T15:10:00.918 INFO:teuthology.orchestra.run.smithi177.stdout:(20/131): python3-cephfs-18.0.0-3906.g1c956514. 876 kB/s | 197 kB 00:00 2023-11-27T15:10:01.176 INFO:teuthology.orchestra.run.smithi177.stdout:(21/131): python3-rados-18.0.0-3906.g1c956514.e 1.4 MB/s | 371 kB 00:00 2023-11-27T15:10:01.340 INFO:teuthology.orchestra.run.smithi133.stdout:(5/131): ceph-mgr-18.0.0-3906.g1c956514.el8.x86 2.5 MB/s | 1.5 MB 00:00 2023-11-27T15:10:01.376 INFO:teuthology.orchestra.run.smithi177.stdout:(22/131): python3-rbd-18.0.0-3906.g1c956514.el8 1.8 MB/s | 364 kB 00:00 2023-11-27T15:10:01.451 INFO:teuthology.orchestra.run.smithi177.stdout:(23/131): librgw2-18.0.0-3906.g1c956514.el8.x86 4.8 MB/s | 6.5 MB 00:01 2023-11-27T15:10:01.477 INFO:teuthology.orchestra.run.smithi177.stdout:(24/131): python3-rgw-18.0.0-3906.g1c956514.el8 1.0 MB/s | 107 kB 00:00 2023-11-27T15:10:01.514 INFO:teuthology.orchestra.run.smithi133.stdout:(6/131): ceph-base-18.0.0-3906.g1c956514.el8.x8 2.1 MB/s | 5.3 MB 00:02 2023-11-27T15:10:01.535 INFO:teuthology.orchestra.run.smithi177.stdout:(25/131): rbd-fuse-18.0.0-3906.g1c956514.el8.x8 1.1 MB/s | 94 kB 00:00 2023-11-27T15:10:01.572 INFO:teuthology.orchestra.run.smithi174.stdout:(7/131): ceph-base-18.0.0-3906.g1c956514.el8.x8 1.5 MB/s | 5.3 MB 00:03 2023-11-27T15:10:01.627 INFO:teuthology.orchestra.run.smithi177.stdout:(26/131): rbd-nbd-18.0.0-3906.g1c956514.el8.x86 1.9 MB/s | 180 kB 00:00 2023-11-27T15:10:01.728 INFO:teuthology.orchestra.run.smithi177.stdout:(27/131): ceph-grafana-dashboards-18.0.0-3906.g 231 kB/s | 23 kB 00:00 2023-11-27T15:10:01.828 INFO:teuthology.orchestra.run.smithi177.stdout:(28/131): ceph-mgr-cephadm-18.0.0-3906.g1c95651 1.2 MB/s | 127 kB 00:00 2023-11-27T15:10:02.253 INFO:teuthology.orchestra.run.smithi177.stdout:(29/131): ceph-mgr-dashboard-18.0.0-3906.g1c956 4.0 MB/s | 1.7 MB 00:00 2023-11-27T15:10:02.762 INFO:teuthology.orchestra.run.smithi177.stdout:(30/131): rbd-mirror-18.0.0-3906.g1c956514.el8. 2.7 MB/s | 3.4 MB 00:01 2023-11-27T15:10:02.822 INFO:teuthology.orchestra.run.smithi174.stdout:(8/131): ceph-osd-18.0.0-3906.g1c956514.el8.x86 7.4 MB/s | 18 MB 00:02 2023-11-27T15:10:02.862 INFO:teuthology.orchestra.run.smithi177.stdout:(31/131): ceph-mgr-modules-core-18.0.0-3906.g1c 2.4 MB/s | 240 kB 00:00 2023-11-27T15:10:02.897 INFO:teuthology.orchestra.run.smithi174.stdout:(9/131): ceph-selinux-18.0.0-3906.g1c956514.el8 326 kB/s | 24 kB 00:00 2023-11-27T15:10:02.898 INFO:teuthology.orchestra.run.smithi133.stdout:(7/131): ceph-mon-18.0.0-3906.g1c956514.el8.x86 3.0 MB/s | 4.6 MB 00:01 2023-11-27T15:10:02.938 INFO:teuthology.orchestra.run.smithi177.stdout:(32/131): ceph-mgr-rook-18.0.0-3906.g1c956514.e 633 kB/s | 47 kB 00:00 2023-11-27T15:10:03.046 INFO:teuthology.orchestra.run.smithi177.stdout:(33/131): ceph-prometheus-alerts-18.0.0-3906.g1 134 kB/s | 15 kB 00:00 2023-11-27T15:10:03.171 INFO:teuthology.orchestra.run.smithi177.stdout:(34/131): ceph-volume-18.0.0-3906.g1c956514.el8 2.1 MB/s | 265 kB 00:00 2023-11-27T15:10:03.280 INFO:teuthology.orchestra.run.smithi177.stdout:(35/131): cephadm-18.0.0-3906.g1c956514.el8.noa 771 kB/s | 83 kB 00:00 2023-11-27T15:10:03.397 INFO:teuthology.orchestra.run.smithi177.stdout:(36/131): cephfs-top-18.0.0-3906.g1c956514.el8. 154 kB/s | 18 kB 00:00 2023-11-27T15:10:03.721 INFO:teuthology.orchestra.run.smithi177.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 5.0 MB/s | 7.4 MB 00:01 2023-11-27T15:10:04.664 INFO:teuthology.orchestra.run.smithi177.stdout:(38/131): ceph-test-18.0.0-3906.g1c956514.el8.x 7.2 MB/s | 41 MB 00:05 2023-11-27T15:10:04.840 INFO:teuthology.orchestra.run.smithi177.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 53 kB/s | 59 kB 00:01 2023-11-27T15:10:05.106 INFO:teuthology.orchestra.run.smithi177.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 179 kB/s | 306 kB 00:01 2023-11-27T15:10:05.540 INFO:teuthology.orchestra.run.smithi177.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 101 kB/s | 44 kB 00:00 2023-11-27T15:10:05.749 INFO:teuthology.orchestra.run.smithi177.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 524 kB/s | 476 kB 00:00 2023-11-27T15:10:05.841 INFO:teuthology.orchestra.run.smithi177.stdout:(43/131): libunwind-1.3.1-3.el8.x86_64.rpm 64 kB/s | 75 kB 00:01 2023-11-27T15:10:05.866 INFO:teuthology.orchestra.run.smithi177.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 569 kB/s | 185 kB 00:00 2023-11-27T15:10:05.983 INFO:teuthology.orchestra.run.smithi177.stdout:(45/131): python3-cachetools-3.1.1-4.el8.noarch 141 kB/s | 33 kB 00:00 2023-11-27T15:10:06.091 INFO:teuthology.orchestra.run.smithi177.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 65 kB/s | 16 kB 00:00 2023-11-27T15:10:06.175 INFO:teuthology.orchestra.run.smithi177.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 559 kB/s | 173 kB 00:00 2023-11-27T15:10:06.231 INFO:teuthology.orchestra.run.smithi133.stdout:(8/131): ceph-radosgw-18.0.0-3906.g1c956514.el8 3.9 MB/s | 13 MB 00:03 2023-11-27T15:10:06.298 INFO:teuthology.orchestra.run.smithi133.stdout:(9/131): ceph-selinux-18.0.0-3906.g1c956514.el8 366 kB/s | 24 kB 00:00 2023-11-27T15:10:06.342 INFO:teuthology.orchestra.run.smithi177.stdout:(48/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.0 MB/s | 384 kB 00:00 2023-11-27T15:10:06.409 INFO:teuthology.orchestra.run.smithi177.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 47 kB/s | 11 kB 00:00 2023-11-27T15:10:06.468 INFO:teuthology.orchestra.run.smithi177.stdout:(50/131): python3-google-auth-1.1.1-10.el8.noar 216 kB/s | 81 kB 00:00 2023-11-27T15:10:06.576 INFO:teuthology.orchestra.run.smithi177.stdout:(51/131): python3-jaraco-functools-2.0-4.el8.no 75 kB/s | 18 kB 00:00 2023-11-27T15:10:06.702 INFO:teuthology.orchestra.run.smithi174.stdout:(10/131): ceph-test-18.0.0-3906.g1c956514.el8.x 11 MB/s | 41 MB 00:03 2023-11-27T15:10:06.740 INFO:teuthology.orchestra.run.smithi133.stdout:(10/131): ceph-osd-18.0.0-3906.g1c956514.el8.x8 3.4 MB/s | 18 MB 00:05 2023-11-27T15:10:06.755 INFO:teuthology.orchestra.run.smithi174.stdout:(11/131): cephfs-mirror-18.0.0-3906.g1c956514.e 4.8 MB/s | 258 kB 00:00 2023-11-27T15:10:06.776 INFO:teuthology.orchestra.run.smithi177.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 158 kB/s | 49 kB 00:00 2023-11-27T15:10:06.797 INFO:teuthology.orchestra.run.smithi174.stdout:(12/131): libcephfs-devel-18.0.0-3906.g1c956514 748 kB/s | 31 kB 00:00 2023-11-27T15:10:06.815 INFO:teuthology.orchestra.run.smithi133.stdout:(11/131): cephfs-mirror-18.0.0-3906.g1c956514.e 3.4 MB/s | 258 kB 00:00 2023-11-27T15:10:06.843 INFO:teuthology.orchestra.run.smithi177.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 220 kB/s | 59 kB 00:00 2023-11-27T15:10:06.865 INFO:teuthology.orchestra.run.smithi133.stdout:(12/131): libcephfs-devel-18.0.0-3906.g1c956514 622 kB/s | 31 kB 00:00 2023-11-27T15:10:06.872 INFO:teuthology.orchestra.run.smithi174.stdout:(13/131): libcephfs2-18.0.0-3906.g1c956514.el8. 10 MB/s | 771 kB 00:00 2023-11-27T15:10:06.923 INFO:teuthology.orchestra.run.smithi174.stdout:(14/131): libcephsqlite-18.0.0-3906.g1c956514.e 2.7 MB/s | 137 kB 00:00 2023-11-27T15:10:06.973 INFO:teuthology.orchestra.run.smithi174.stdout:(15/131): librados-devel-18.0.0-3906.g1c956514. 2.7 MB/s | 139 kB 00:00 2023-11-27T15:10:07.007 INFO:teuthology.orchestra.run.smithi133.stdout:(13/131): libcephfs2-18.0.0-3906.g1c956514.el8. 5.3 MB/s | 771 kB 00:00 2023-11-27T15:10:07.023 INFO:teuthology.orchestra.run.smithi174.stdout:(16/131): libradosstriper1-18.0.0-3906.g1c95651 9.3 MB/s | 473 kB 00:00 2023-11-27T15:10:07.049 INFO:teuthology.orchestra.run.smithi133.stdout:(14/131): libcephsqlite-18.0.0-3906.g1c956514.e 3.2 MB/s | 137 kB 00:00 2023-11-27T15:10:07.099 INFO:teuthology.orchestra.run.smithi133.stdout:(15/131): librados-devel-18.0.0-3906.g1c956514. 2.7 MB/s | 139 kB 00:00 2023-11-27T15:10:07.118 INFO:teuthology.orchestra.run.smithi177.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 176 kB/s | 60 kB 00:00 2023-11-27T15:10:07.152 INFO:teuthology.orchestra.run.smithi177.stdout:(55/131): python3-kubernetes-11.0.0-6.el8.noarc 1.6 MB/s | 1.2 MB 00:00 2023-11-27T15:10:07.177 INFO:teuthology.orchestra.run.smithi177.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 847 kB/s | 283 kB 00:00 2023-11-27T15:10:07.199 INFO:teuthology.orchestra.run.smithi174.stdout:(17/131): ceph-radosgw-18.0.0-3906.g1c956514.el 2.3 MB/s | 13 MB 00:05 2023-11-27T15:10:07.216 INFO:teuthology.orchestra.run.smithi133.stdout:(16/131): libradosstriper1-18.0.0-3906.g1c95651 4.0 MB/s | 473 kB 00:00 2023-11-27T15:10:07.266 INFO:teuthology.orchestra.run.smithi174.stdout:(18/131): python3-ceph-argparse-18.0.0-3906.g1c 674 kB/s | 45 kB 00:00 2023-11-27T15:10:07.358 INFO:teuthology.orchestra.run.smithi174.stdout:(19/131): python3-ceph-common-18.0.0-3906.g1c95 1.3 MB/s | 119 kB 00:00 2023-11-27T15:10:07.361 INFO:teuthology.orchestra.run.smithi177.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 64 kB/s | 16 kB 00:00 2023-11-27T15:10:07.386 INFO:teuthology.orchestra.run.smithi177.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 144 kB/s | 34 kB 00:00 2023-11-27T15:10:07.478 INFO:teuthology.orchestra.run.smithi177.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 654 kB/s | 196 kB 00:00 2023-11-27T15:10:07.533 INFO:teuthology.orchestra.run.smithi174.stdout:(20/131): python3-cephfs-18.0.0-3906.g1c956514. 1.1 MB/s | 197 kB 00:00 2023-11-27T15:10:07.612 INFO:teuthology.orchestra.run.smithi177.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 83 kB/s | 19 kB 00:00 2023-11-27T15:10:07.670 INFO:teuthology.orchestra.run.smithi177.stdout:(61/131): python3-rsa-4.9-2.el8.noarch.rpm 208 kB/s | 64 kB 00:00 2023-11-27T15:10:07.683 INFO:teuthology.orchestra.run.smithi174.stdout:(21/131): librgw2-18.0.0-3906.g1c956514.el8.x86 9.9 MB/s | 6.5 MB 00:00 2023-11-27T15:10:07.709 INFO:teuthology.orchestra.run.smithi174.stdout:(22/131): python3-rados-18.0.0-3906.g1c956514.e 2.1 MB/s | 371 kB 00:00 2023-11-27T15:10:07.712 INFO:teuthology.orchestra.run.smithi177.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 102 kB/s | 24 kB 00:00 2023-11-27T15:10:07.742 INFO:teuthology.orchestra.run.smithi174.stdout:(23/131): python3-rbd-18.0.0-3906.g1c956514.el8 6.1 MB/s | 364 kB 00:00 2023-11-27T15:10:07.784 INFO:teuthology.orchestra.run.smithi174.stdout:(24/131): python3-rgw-18.0.0-3906.g1c956514.el8 1.4 MB/s | 107 kB 00:00 2023-11-27T15:10:07.809 INFO:teuthology.orchestra.run.smithi174.stdout:(25/131): rbd-fuse-18.0.0-3906.g1c956514.el8.x8 1.4 MB/s | 94 kB 00:00 2023-11-27T15:10:07.846 INFO:teuthology.orchestra.run.smithi177.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 125 kB/s | 29 kB 00:00 2023-11-27T15:10:07.868 INFO:teuthology.orchestra.run.smithi174.stdout:(26/131): rbd-nbd-18.0.0-3906.g1c956514.el8.x86 3.0 MB/s | 180 kB 00:00 2023-11-27T15:10:07.938 INFO:teuthology.orchestra.run.smithi177.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 103 kB/s | 27 kB 00:00 2023-11-27T15:10:07.952 INFO:teuthology.orchestra.run.smithi174.stdout:(27/131): ceph-grafana-dashboards-18.0.0-3906.g 274 kB/s | 23 kB 00:00 2023-11-27T15:10:08.013 INFO:teuthology.orchestra.run.smithi177.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 850 kB/s | 255 kB 00:00 2023-11-27T15:10:08.018 INFO:teuthology.orchestra.run.smithi174.stdout:(28/131): ceph-mgr-cephadm-18.0.0-3906.g1c95651 1.9 MB/s | 127 kB 00:00 2023-11-27T15:10:08.113 INFO:teuthology.orchestra.run.smithi177.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 940 kB/s | 251 kB 00:00 2023-11-27T15:10:08.228 INFO:teuthology.orchestra.run.smithi174.stdout:(29/131): ceph-mgr-dashboard-18.0.0-3906.g1c956 8.2 MB/s | 1.7 MB 00:00 2023-11-27T15:10:08.256 INFO:teuthology.orchestra.run.smithi177.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 193 kB/s | 61 kB 00:00 2023-11-27T15:10:08.281 INFO:teuthology.orchestra.run.smithi177.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 319 kB/s | 85 kB 00:00 2023-11-27T15:10:08.283 INFO:teuthology.orchestra.run.smithi133.stdout:(17/131): librgw2-18.0.0-3906.g1c956514.el8.x86 6.1 MB/s | 6.5 MB 00:01 2023-11-27T15:10:08.306 INFO:teuthology.orchestra.run.smithi177.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-11-27T15:10:08.342 INFO:teuthology.orchestra.run.smithi133.stdout:(18/131): python3-ceph-argparse-18.0.0-3906.g1c 768 kB/s | 45 kB 00:00 2023-11-27T15:10:08.348 INFO:teuthology.orchestra.run.smithi177.stdout:(70/131): python3-zc-lockfile-2.0-2.el8.noarch. 98 kB/s | 23 kB 00:00 2023-11-27T15:10:08.434 INFO:teuthology.orchestra.run.smithi133.stdout:(19/131): python3-ceph-common-18.0.0-3906.g1c95 1.3 MB/s | 119 kB 00:00 2023-11-27T15:10:08.509 INFO:teuthology.orchestra.run.smithi133.stdout:(20/131): python3-cephfs-18.0.0-3906.g1c956514. 2.6 MB/s | 197 kB 00:00 2023-11-27T15:10:08.518 INFO:teuthology.orchestra.run.smithi177.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 220 kB/s | 47 kB 00:00 2023-11-27T15:10:08.557 INFO:teuthology.orchestra.run.smithi177.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 958 kB/s | 200 kB 00:00 2023-11-27T15:10:08.626 INFO:teuthology.orchestra.run.smithi133.stdout:(21/131): python3-rados-18.0.0-3906.g1c956514.e 3.1 MB/s | 371 kB 00:00 2023-11-27T15:10:08.666 INFO:teuthology.orchestra.run.smithi177.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 291 kB/s | 43 kB 00:00 2023-11-27T15:10:08.733 INFO:teuthology.orchestra.run.smithi177.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.2 MB/s | 220 kB 00:00 2023-11-27T15:10:08.751 INFO:teuthology.orchestra.run.smithi133.stdout:(22/131): python3-rbd-18.0.0-3906.g1c956514.el8 2.8 MB/s | 364 kB 00:00 2023-11-27T15:10:08.774 INFO:teuthology.orchestra.run.smithi177.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 443 kB/s | 48 kB 00:00 2023-11-27T15:10:08.835 INFO:teuthology.orchestra.run.smithi133.stdout:(23/131): python3-rgw-18.0.0-3906.g1c956514.el8 1.3 MB/s | 107 kB 00:00 2023-11-27T15:10:08.878 INFO:teuthology.orchestra.run.smithi174.stdout:(30/131): ceph-mgr-diskprediction-local-18.0.0- 11 MB/s | 7.4 MB 00:00 2023-11-27T15:10:08.885 INFO:teuthology.orchestra.run.smithi133.stdout:(24/131): rbd-fuse-18.0.0-3906.g1c956514.el8.x8 1.8 MB/s | 94 kB 00:00 2023-11-27T15:10:08.916 INFO:teuthology.orchestra.run.smithi177.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 474 kB/s | 87 kB 00:00 2023-11-27T15:10:08.936 INFO:teuthology.orchestra.run.smithi174.stdout:(31/131): rbd-mirror-18.0.0-3906.g1c956514.el8. 3.0 MB/s | 3.4 MB 00:01 2023-11-27T15:10:08.942 INFO:teuthology.orchestra.run.smithi177.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 255 kB/s | 43 kB 00:00 2023-11-27T15:10:08.961 INFO:teuthology.orchestra.run.smithi174.stdout:(32/131): ceph-mgr-modules-core-18.0.0-3906.g1c 2.8 MB/s | 240 kB 00:00 2023-11-27T15:10:09.003 INFO:teuthology.orchestra.run.smithi174.stdout:(33/131): ceph-mgr-rook-18.0.0-3906.g1c956514.e 710 kB/s | 47 kB 00:00 2023-11-27T15:10:09.029 INFO:teuthology.orchestra.run.smithi174.stdout:(34/131): ceph-prometheus-alerts-18.0.0-3906.g1 217 kB/s | 15 kB 00:00 2023-11-27T15:10:09.050 INFO:teuthology.orchestra.run.smithi177.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 817 kB/s | 109 kB 00:00 2023-11-27T15:10:09.062 INFO:teuthology.orchestra.run.smithi174.stdout:(35/131): ceph-volume-18.0.0-3906.g1c956514.el8 4.4 MB/s | 265 kB 00:00 2023-11-27T15:10:09.084 INFO:teuthology.orchestra.run.smithi177.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.6 MB/s | 229 kB 00:00 2023-11-27T15:10:09.104 INFO:teuthology.orchestra.run.smithi174.stdout:(36/131): cephadm-18.0.0-3906.g1c956514.el8.noa 1.1 MB/s | 83 kB 00:00 2023-11-27T15:10:09.153 INFO:teuthology.orchestra.run.smithi174.stdout:(37/131): cephfs-top-18.0.0-3906.g1c956514.el8. 200 kB/s | 18 kB 00:00 2023-11-27T15:10:09.218 INFO:teuthology.orchestra.run.smithi177.stdout:(80/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 140 kB/s | 19 kB 00:00 2023-11-27T15:10:09.259 INFO:teuthology.orchestra.run.smithi177.stdout:(81/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 2.6 MB/s | 560 kB 00:00 2023-11-27T15:10:09.280 INFO:teuthology.orchestra.run.smithi174.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 463 kB/s | 59 kB 00:00 2023-11-27T15:10:09.330 INFO:teuthology.orchestra.run.smithi174.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.5 MB/s | 75 kB 00:00 2023-11-27T15:10:09.344 INFO:teuthology.orchestra.run.smithi133.stdout:(25/131): rbd-mirror-18.0.0-3906.g1c956514.el8. 7.5 MB/s | 3.4 MB 00:00 2023-11-27T15:10:09.355 INFO:teuthology.orchestra.run.smithi174.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-11-27T15:10:09.376 INFO:teuthology.orchestra.run.smithi177.stdout:(82/131): thrift-0.13.0-2.el8.x86_64.rpm 1.5 MB/s | 1.7 MB 00:01 2023-11-27T15:10:09.394 INFO:teuthology.orchestra.run.smithi133.stdout:(26/131): rbd-nbd-18.0.0-3906.g1c956514.el8.x86 3.5 MB/s | 180 kB 00:00 2023-11-27T15:10:09.402 INFO:teuthology.orchestra.run.smithi177.stdout:(83/131): python3-oauthlib-2.1.0-1.el8.noarch.r 843 kB/s | 155 kB 00:00 2023-11-27T15:10:09.414 INFO:teuthology.orchestra.run.smithi174.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 748 kB/s | 44 kB 00:00 2023-11-27T15:10:09.435 INFO:teuthology.orchestra.run.smithi177.stdout:(84/131): python3-cryptography-3.2.1-5.el8.x86_ 3.1 MB/s | 559 kB 00:00 2023-11-27T15:10:09.440 INFO:teuthology.orchestra.run.smithi174.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 4.3 MB/s | 476 kB 00:00 2023-11-27T15:10:09.444 INFO:teuthology.orchestra.run.smithi133.stdout:(27/131): ceph-grafana-dashboards-18.0.0-3906.g 461 kB/s | 23 kB 00:00 2023-11-27T15:10:09.465 INFO:teuthology.orchestra.run.smithi174.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2023-11-27T15:10:09.490 INFO:teuthology.orchestra.run.smithi174.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 657 kB/s | 33 kB 00:00 2023-11-27T15:10:09.503 INFO:teuthology.orchestra.run.smithi133.stdout:(28/131): ceph-mgr-cephadm-18.0.0-3906.g1c95651 2.1 MB/s | 127 kB 00:00 2023-11-27T15:10:09.515 INFO:teuthology.orchestra.run.smithi174.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 324 kB/s | 16 kB 00:00 2023-11-27T15:10:09.519 INFO:teuthology.orchestra.run.smithi177.stdout:(85/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 126 kB/s | 18 kB 00:00 2023-11-27T15:10:09.564 INFO:teuthology.orchestra.run.smithi174.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.3 MB/s | 173 kB 00:00 2023-11-27T15:10:09.596 INFO:teuthology.orchestra.run.smithi177.stdout:(86/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.1 MB/s | 605 kB 00:00 2023-11-27T15:10:09.599 INFO:teuthology.orchestra.run.smithi174.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 4.5 MB/s | 384 kB 00:00 2023-11-27T15:10:09.625 INFO:teuthology.orchestra.run.smithi174.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.3 MB/s | 81 kB 00:00 2023-11-27T15:10:09.628 INFO:teuthology.orchestra.run.smithi177.stdout:(87/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.2 MB/s | 238 kB 00:00 2023-11-27T15:10:09.642 INFO:teuthology.orchestra.run.smithi174.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 265 kB/s | 11 kB 00:00 2023-11-27T15:10:09.659 INFO:teuthology.orchestra.run.smithi174.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 525 kB/s | 18 kB 00:00 2023-11-27T15:10:09.692 INFO:teuthology.orchestra.run.smithi174.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.4 MB/s | 49 kB 00:00 2023-11-27T15:10:09.712 INFO:teuthology.orchestra.run.smithi177.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 12 MB/s | 2.3 MB 00:00 2023-11-27T15:10:09.728 INFO:teuthology.orchestra.run.smithi177.stdout:(89/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 548 kB/s | 72 kB 00:00 2023-11-27T15:10:09.734 INFO:teuthology.orchestra.run.smithi174.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 13 MB/s | 1.2 MB 00:00 2023-11-27T15:10:09.745 INFO:teuthology.orchestra.run.smithi133.stdout:(29/131): ceph-mgr-dashboard-18.0.0-3906.g1c956 7.1 MB/s | 1.7 MB 00:00 2023-11-27T15:10:09.751 INFO:teuthology.orchestra.run.smithi174.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-11-27T15:10:09.768 INFO:teuthology.orchestra.run.smithi174.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.8 MB/s | 60 kB 00:00 2023-11-27T15:10:09.779 INFO:teuthology.orchestra.run.smithi177.stdout:(90/131): fuse-2.9.7-15.el8.x86_64.rpm 549 kB/s | 83 kB 00:00 2023-11-27T15:10:09.802 INFO:teuthology.orchestra.run.smithi174.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2023-11-27T15:10:09.820 INFO:teuthology.orchestra.run.smithi177.stdout:(91/131): elfutils-libelf-devel-0.186-1.el8.x86 551 kB/s | 60 kB 00:00 2023-11-27T15:10:09.827 INFO:teuthology.orchestra.run.smithi174.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 266 kB/s | 16 kB 00:00 2023-11-27T15:10:09.846 INFO:teuthology.orchestra.run.smithi177.stdout:(92/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 837 kB/s | 98 kB 00:00 2023-11-27T15:10:09.852 INFO:teuthology.orchestra.run.smithi174.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 673 kB/s | 34 kB 00:00 2023-11-27T15:10:09.878 INFO:teuthology.orchestra.run.smithi174.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2023-11-27T15:10:09.894 INFO:teuthology.orchestra.run.smithi174.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.5 MB/s | 64 kB 00:00 2023-11-27T15:10:09.920 INFO:teuthology.orchestra.run.smithi174.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 449 kB/s | 19 kB 00:00 2023-11-27T15:10:09.954 INFO:teuthology.orchestra.run.smithi133.stdout:(30/131): ceph-common-18.0.0-3906.g1c956514.el8 2.0 MB/s | 22 MB 00:10 2023-11-27T15:10:09.963 INFO:teuthology.orchestra.run.smithi177.stdout:(93/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 314 kB/s | 58 kB 00:00 2023-11-27T15:10:09.975 INFO:teuthology.orchestra.run.smithi174.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 298 kB/s | 24 kB 00:00 2023-11-27T15:10:09.988 INFO:teuthology.orchestra.run.smithi177.stdout:(94/131): libcom_err-devel-1.45.6-4.el8.x86_64. 231 kB/s | 39 kB 00:00 2023-11-27T15:10:10.003 INFO:teuthology.orchestra.run.smithi174.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 351 kB/s | 29 kB 00:00 2023-11-27T15:10:10.029 INFO:teuthology.orchestra.run.smithi174.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 518 kB/s | 27 kB 00:00 2023-11-27T15:10:10.039 INFO:teuthology.orchestra.run.smithi177.stdout:(95/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 3.3 MB/s | 644 kB 00:00 2023-11-27T15:10:10.054 INFO:teuthology.orchestra.run.smithi174.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-11-27T15:10:10.079 INFO:teuthology.orchestra.run.smithi133.stdout:(31/131): ceph-mgr-modules-core-18.0.0-3906.g1c 1.9 MB/s | 240 kB 00:00 2023-11-27T15:10:10.080 INFO:teuthology.orchestra.run.smithi174.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2023-11-27T15:10:10.105 INFO:teuthology.orchestra.run.smithi174.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-11-27T15:10:10.106 INFO:teuthology.orchestra.run.smithi177.stdout:(96/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.2 MB/s | 171 kB 00:00 2023-11-27T15:10:10.130 INFO:teuthology.orchestra.run.smithi174.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-11-27T15:10:10.162 INFO:teuthology.orchestra.run.smithi133.stdout:(32/131): ceph-mgr-rook-18.0.0-3906.g1c956514.e 571 kB/s | 47 kB 00:00 2023-11-27T15:10:10.164 INFO:teuthology.orchestra.run.smithi174.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 392 kB/s | 23 kB 00:00 2023-11-27T15:10:10.164 INFO:teuthology.orchestra.run.smithi177.stdout:(97/131): NetworkManager-initscripts-updown-1.3 782 kB/s | 138 kB 00:00 2023-11-27T15:10:10.189 INFO:teuthology.orchestra.run.smithi174.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-11-27T15:10:10.229 INFO:teuthology.orchestra.run.smithi133.stdout:(33/131): ceph-prometheus-alerts-18.0.0-3906.g1 219 kB/s | 15 kB 00:00 2023-11-27T15:10:10.239 INFO:teuthology.orchestra.run.smithi174.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 16 MB/s | 1.7 MB 00:00 2023-11-27T15:10:10.315 INFO:teuthology.orchestra.run.smithi177.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 528 kB/s | 110 kB 00:00 2023-11-27T15:10:10.338 INFO:teuthology.orchestra.run.smithi133.stdout:(34/131): ceph-volume-18.0.0-3906.g1c956514.el8 2.4 MB/s | 265 kB 00:00 2023-11-27T15:10:10.365 INFO:teuthology.orchestra.run.smithi177.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 270 kB/s | 54 kB 00:00 2023-11-27T15:10:10.390 INFO:teuthology.orchestra.run.smithi174.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 762 kB/s | 110 kB 00:00 2023-11-27T15:10:10.398 INFO:teuthology.orchestra.run.smithi177.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.0 MB/s | 259 kB 00:00 2023-11-27T15:10:10.413 INFO:teuthology.orchestra.run.smithi133.stdout:(35/131): cephadm-18.0.0-3906.g1c956514.el8.noa 1.1 MB/s | 83 kB 00:00 2023-11-27T15:10:10.489 INFO:teuthology.orchestra.run.smithi133.stdout:(36/131): cephfs-top-18.0.0-3906.g1c956514.el8. 240 kB/s | 18 kB 00:00 2023-11-27T15:10:10.548 INFO:teuthology.orchestra.run.smithi174.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 342 kB/s | 54 kB 00:00 2023-11-27T15:10:10.574 INFO:teuthology.orchestra.run.smithi177.stdout:(101/131): python3-scipy-1.0.0-21.module+el8.5. 26 MB/s | 14 MB 00:00 2023-11-27T15:10:10.599 INFO:teuthology.orchestra.run.smithi177.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 194 kB/s | 39 kB 00:00 2023-11-27T15:10:10.666 INFO:teuthology.orchestra.run.smithi177.stdout:(103/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.0 MB/s | 323 kB 00:00 2023-11-27T15:10:10.715 INFO:teuthology.orchestra.run.smithi174.stdout:(73/131): python3-scipy-1.0.0-21.module+el8.5.0 26 MB/s | 14 MB 00:00 2023-11-27T15:10:10.724 INFO:teuthology.orchestra.run.smithi177.stdout:(104/131): bison-3.0.4-10.el8.x86_64.rpm 4.5 MB/s | 688 kB 00:00 2023-11-27T15:10:10.741 INFO:teuthology.orchestra.run.smithi174.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.3 MB/s | 259 kB 00:00 2023-11-27T15:10:10.766 INFO:teuthology.orchestra.run.smithi177.stdout:(105/131): python3-jinja2-2.10.1-3.el8.noarch.r 3.1 MB/s | 538 kB 00:00 2023-11-27T15:10:10.772 INFO:teuthology.orchestra.run.smithi133.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 7.2 MB/s | 7.4 MB 00:01 2023-11-27T15:10:10.792 INFO:teuthology.orchestra.run.smithi177.stdout:(106/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2023-11-27T15:10:10.892 INFO:teuthology.orchestra.run.smithi177.stdout:(107/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2023-11-27T15:10:11.049 INFO:teuthology.orchestra.run.smithi174.stdout:(75/131): ceph-common-18.0.0-3906.g1c956514.el8 1.7 MB/s | 22 MB 00:13 2023-11-27T15:10:11.066 INFO:teuthology.orchestra.run.smithi174.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 119 kB/s | 39 kB 00:00 2023-11-27T15:10:11.075 INFO:teuthology.orchestra.run.smithi177.stdout:(108/131): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-11-27T15:10:11.091 INFO:teuthology.orchestra.run.smithi174.stdout:(77/131): socat-1.7.4.1-1.el8.x86_64.rpm 861 kB/s | 323 kB 00:00 2023-11-27T15:10:11.109 INFO:teuthology.orchestra.run.smithi177.stdout:(109/131): automake-1.16.1-7.el8.noarch.rpm 2.0 MB/s | 713 kB 00:00 2023-11-27T15:10:11.168 INFO:teuthology.orchestra.run.smithi177.stdout:(110/131): python3-jsonpatch-1.21-2.el8.noarch. 96 kB/s | 27 kB 00:00 2023-11-27T15:10:11.225 INFO:teuthology.orchestra.run.smithi174.stdout:(78/131): bison-3.0.4-10.el8.x86_64.rpm 3.8 MB/s | 688 kB 00:00 2023-11-27T15:10:11.242 INFO:teuthology.orchestra.run.smithi174.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.1 MB/s | 157 kB 00:00 2023-11-27T15:10:11.243 INFO:teuthology.orchestra.run.smithi177.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 329 kB/s | 44 kB 00:00 2023-11-27T15:10:11.285 INFO:teuthology.orchestra.run.smithi177.stdout:(112/131): flex-2.6.1-9.el8.x86_64.rpm 1.5 MB/s | 320 kB 00:00 2023-11-27T15:10:11.325 INFO:teuthology.orchestra.run.smithi174.stdout:(80/131): automake-1.16.1-7.el8.noarch.rpm 8.4 MB/s | 713 kB 00:00 2023-11-27T15:10:11.359 INFO:teuthology.orchestra.run.smithi174.stdout:(81/131): python3-jinja2-2.10.1-3.el8.noarch.rp 1.8 MB/s | 538 kB 00:00 2023-11-27T15:10:11.384 INFO:teuthology.orchestra.run.smithi174.stdout:(82/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2023-11-27T15:10:11.402 INFO:teuthology.orchestra.run.smithi177.stdout:(113/131): python3-werkzeug-0.12.2-4.el8.noarch 2.8 MB/s | 457 kB 00:00 2023-11-27T15:10:11.435 INFO:teuthology.orchestra.run.smithi177.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 1.5 MB/s | 225 kB 00:00 2023-11-27T15:10:11.464 INFO:teuthology.orchestra.run.smithi133.stdout:(38/131): ceph-test-18.0.0-3906.g1c956514.el8.x 7.9 MB/s | 41 MB 00:05 2023-11-27T15:10:11.490 INFO:teuthology.orchestra.run.smithi133.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 82 kB/s | 59 kB 00:00 2023-11-27T15:10:11.494 INFO:teuthology.orchestra.run.smithi177.stdout:(115/131): python3-numpy-1.14.3-10.el8.x86_64.r 11 MB/s | 3.7 MB 00:00 2023-11-27T15:10:11.509 INFO:teuthology.orchestra.run.smithi174.stdout:(83/131): python3-babel-2.5.1-7.el8.noarch.rpm 26 MB/s | 4.8 MB 00:00 2023-11-27T15:10:11.519 INFO:teuthology.orchestra.run.smithi177.stdout:(116/131): uuid-1.6.2-43.el8.x86_64.rpm 547 kB/s | 64 kB 00:00 2023-11-27T15:10:11.535 INFO:teuthology.orchestra.run.smithi174.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 2.1 MB/s | 320 kB 00:00 2023-11-27T15:10:11.569 INFO:teuthology.orchestra.run.smithi177.stdout:(117/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 945 kB/s | 126 kB 00:00 2023-11-27T15:10:11.595 INFO:teuthology.orchestra.run.smithi177.stdout:(118/131): python3-jsonpointer-1.10-11.el8.noar 199 kB/s | 20 kB 00:00 2023-11-27T15:10:11.595 INFO:teuthology.orchestra.run.smithi174.stdout:(85/131): python3-jsonpatch-1.21-2.el8.noarch.r 113 kB/s | 27 kB 00:00 2023-11-27T15:10:11.662 INFO:teuthology.orchestra.run.smithi177.stdout:(119/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 729 kB/s | 103 kB 00:00 2023-11-27T15:10:11.668 INFO:teuthology.orchestra.run.smithi174.stdout:(86/131): python3-prettytable-0.7.2-14.el8.noar 277 kB/s | 44 kB 00:00 2023-11-27T15:10:11.754 INFO:teuthology.orchestra.run.smithi177.stdout:(120/131): libtool-2.4.6-25.el8.x86_64.rpm 3.8 MB/s | 709 kB 00:00 2023-11-27T15:10:11.777 INFO:teuthology.orchestra.run.smithi174.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 15 MB/s | 3.7 MB 00:00 2023-11-27T15:10:11.811 INFO:teuthology.orchestra.run.smithi174.stdout:(88/131): python3-werkzeug-0.12.2-4.el8.noarch. 2.1 MB/s | 457 kB 00:00 2023-11-27T15:10:11.836 INFO:teuthology.orchestra.run.smithi174.stdout:(89/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.3 MB/s | 225 kB 00:00 2023-11-27T15:10:11.854 INFO:teuthology.orchestra.run.smithi177.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.0 MB/s | 198 kB 00:00 2023-11-27T15:10:11.920 INFO:teuthology.orchestra.run.smithi174.stdout:(90/131): uuid-1.6.2-43.el8.x86_64.rpm 450 kB/s | 64 kB 00:00 2023-11-27T15:10:11.932 INFO:teuthology.orchestra.run.smithi133.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.1 MB/s | 476 kB 00:00 2023-11-27T15:10:11.937 INFO:teuthology.orchestra.run.smithi177.stdout:(122/131): openblas-threads-0.3.15-3.el8.x86_64 14 MB/s | 4.9 MB 00:00 2023-11-27T15:10:11.957 INFO:teuthology.orchestra.run.smithi133.stdout:(41/131): libunwind-1.3.1-3.el8.x86_64.rpm 153 kB/s | 75 kB 00:00 2023-11-27T15:10:12.020 INFO:teuthology.orchestra.run.smithi174.stdout:(91/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 605 kB/s | 126 kB 00:00 2023-11-27T15:10:12.038 INFO:teuthology.orchestra.run.smithi177.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-11-27T15:10:12.040 INFO:teuthology.orchestra.run.smithi133.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 404 kB/s | 44 kB 00:00 2023-11-27T15:10:12.045 INFO:teuthology.orchestra.run.smithi174.stdout:(92/131): python3-jsonpointer-1.10-11.el8.noarc 96 kB/s | 20 kB 00:00 2023-11-27T15:10:12.066 INFO:teuthology.orchestra.run.smithi133.stdout:(43/131): gperftools-libs-2.7-9.el8.x86_64.rpm 194 kB/s | 306 kB 00:01 2023-11-27T15:10:12.070 INFO:teuthology.orchestra.run.smithi174.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 688 kB/s | 103 kB 00:00 2023-11-27T15:10:12.096 INFO:teuthology.orchestra.run.smithi177.stdout:(124/131): jq-1.6-3.el8.x86_64.rpm 835 kB/s | 202 kB 00:00 2023-11-27T15:10:12.130 INFO:teuthology.orchestra.run.smithi177.stdout:(125/131): librdkafka-0.11.4-3.el8.x86_64.rpm 1.8 MB/s | 354 kB 00:00 2023-11-27T15:10:12.174 INFO:teuthology.orchestra.run.smithi133.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 247 kB/s | 33 kB 00:00 2023-11-27T15:10:12.200 INFO:teuthology.orchestra.run.smithi133.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 122 kB/s | 16 kB 00:00 2023-11-27T15:10:12.213 INFO:teuthology.orchestra.run.smithi174.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 3.6 MB/s | 709 kB 00:00 2023-11-27T15:10:12.225 INFO:teuthology.orchestra.run.smithi133.stdout:(46/131): python3-beautifulsoup4-4.6.3-2.el8.1. 692 kB/s | 185 kB 00:00 2023-11-27T15:10:12.237 INFO:teuthology.orchestra.run.smithi174.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2023-11-27T15:10:12.264 INFO:teuthology.orchestra.run.smithi177.stdout:(126/131): postgresql-server-10.21-2.module+el8 22 MB/s | 5.1 MB 00:00 2023-11-27T15:10:12.292 INFO:teuthology.orchestra.run.smithi133.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.4 MB/s | 173 kB 00:00 2023-11-27T15:10:12.314 INFO:teuthology.orchestra.run.smithi177.stdout:(127/131): postgresql-contrib-10.21-2.module+el 3.6 MB/s | 808 kB 00:00 2023-11-27T15:10:12.342 INFO:teuthology.orchestra.run.smithi133.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 696 kB/s | 81 kB 00:00 2023-11-27T15:10:12.346 INFO:teuthology.orchestra.run.smithi174.stdout:(96/131): openblas-threads-0.3.15-3.el8.x86_64. 16 MB/s | 4.9 MB 00:00 2023-11-27T15:10:12.372 INFO:teuthology.orchestra.run.smithi177.stdout:(128/131): postgresql-10.21-2.module+el8.6.0+15 6.2 MB/s | 1.5 MB 00:00 2023-11-27T15:10:12.384 INFO:teuthology.orchestra.run.smithi133.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 121 kB/s | 11 kB 00:00 2023-11-27T15:10:12.401 INFO:teuthology.orchestra.run.smithi174.stdout:(97/131): jq-1.6-3.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2023-11-27T15:10:12.443 INFO:teuthology.orchestra.run.smithi133.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 175 kB/s | 18 kB 00:00 2023-11-27T15:10:12.505 INFO:teuthology.orchestra.run.smithi174.stdout:(98/131): openblas-0.3.15-3.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2023-11-27T15:10:12.523 INFO:teuthology.orchestra.run.smithi177.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 5.2 MB/s | 798 kB 00:00 2023-11-27T15:10:12.538 INFO:teuthology.orchestra.run.smithi174.stdout:(99/131): librdkafka-0.11.4-3.el8.x86_64.rpm 1.8 MB/s | 354 kB 00:00 2023-11-27T15:10:12.576 INFO:teuthology.orchestra.run.smithi133.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 366 kB/s | 49 kB 00:00 2023-11-27T15:10:12.619 INFO:teuthology.orchestra.run.smithi133.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 5.0 MB/s | 1.2 MB 00:00 2023-11-27T15:10:12.631 INFO:teuthology.orchestra.run.smithi177.stdout:(130/131): librados2-18.0.0-3906.g1c956514.el8. 10 MB/s | 3.7 MB 00:00 2023-11-27T15:10:12.639 INFO:teuthology.orchestra.run.smithi174.stdout:(100/131): postgresql-server-10.21-2.module+el8 21 MB/s | 5.1 MB 00:00 2023-11-27T15:10:12.685 INFO:teuthology.orchestra.run.smithi133.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 541 kB/s | 59 kB 00:00 2023-11-27T15:10:12.698 INFO:teuthology.orchestra.run.smithi174.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 9.5 MB/s | 1.5 MB 00:00 2023-11-27T15:10:12.719 INFO:teuthology.orchestra.run.smithi133.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 599 kB/s | 60 kB 00:00 2023-11-27T15:10:12.740 INFO:teuthology.orchestra.run.smithi174.stdout:(102/131): postgresql-contrib-10.21-2.module+el 3.4 MB/s | 808 kB 00:00 2023-11-27T15:10:12.765 INFO:teuthology.orchestra.run.smithi174.stdout:(103/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 372 kB/s | 47 kB 00:00 2023-11-27T15:10:12.794 INFO:teuthology.orchestra.run.smithi133.stdout:(55/131): python3-cherrypy-18.4.0-1.el8.noarch. 647 kB/s | 384 kB 00:00 2023-11-27T15:10:12.820 INFO:teuthology.orchestra.run.smithi133.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 156 kB/s | 16 kB 00:00 2023-11-27T15:10:12.861 INFO:teuthology.orchestra.run.smithi133.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.6 MB/s | 283 kB 00:00 2023-11-27T15:10:12.865 INFO:teuthology.orchestra.run.smithi174.stdout:(104/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.2 MB/s | 200 kB 00:00 2023-11-27T15:10:12.881 INFO:teuthology.orchestra.run.smithi177.stdout:(131/131): librbd1-18.0.0-3906.g1c956514.el8.x8 6.3 MB/s | 3.6 MB 00:00 2023-11-27T15:10:12.886 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2023-11-27T15:10:12.886 INFO:teuthology.orchestra.run.smithi177.stdout:Total 12 MB/s | 197 MB 00:16 2023-11-27T15:10:12.907 INFO:teuthology.orchestra.run.smithi174.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.5 MB/s | 220 kB 00:00 2023-11-27T15:10:12.928 INFO:teuthology.orchestra.run.smithi133.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 251 kB/s | 34 kB 00:00 2023-11-27T15:10:12.932 INFO:teuthology.orchestra.run.smithi174.stdout:(106/131): python3-jwt-1.6.1-2.el8.noarch.rpm 221 kB/s | 43 kB 00:00 2023-11-27T15:10:12.975 INFO:teuthology.orchestra.run.smithi174.stdout:(107/131): keyutils-libs-devel-1.5.10-9.el8.x86 441 kB/s | 48 kB 00:00 2023-11-27T15:10:12.987 INFO:teuthology.orchestra.run.smithi133.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.1 MB/s | 196 kB 00:00 2023-11-27T15:10:13.012 INFO:teuthology.orchestra.run.smithi133.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 427 kB/s | 64 kB 00:00 2023-11-27T15:10:13.033 INFO:teuthology.orchestra.run.smithi174.stdout:(108/131): libsepol-devel-2.9-3.el8.x86_64.rpm 693 kB/s | 87 kB 00:00 2023-11-27T15:10:13.046 INFO:teuthology.orchestra.run.smithi133.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 160 kB/s | 19 kB 00:00 2023-11-27T15:10:13.058 INFO:teuthology.orchestra.run.smithi174.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 1.3 MB/s | 109 kB 00:00 2023-11-27T15:10:13.084 INFO:teuthology.orchestra.run.smithi174.stdout:(110/131): python3-requests-oauthlib-1.0.0-1.el 283 kB/s | 43 kB 00:00 2023-11-27T15:10:13.088 INFO:teuthology.orchestra.run.smithi133.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2023-11-27T15:10:13.121 INFO:teuthology.orchestra.run.smithi133.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 269 kB/s | 29 kB 00:00 2023-11-27T15:10:13.159 INFO:teuthology.orchestra.run.smithi174.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.8 MB/s | 229 kB 00:00 2023-11-27T15:10:13.172 INFO:teuthology.orchestra.run.smithi133.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 219 kB/s | 27 kB 00:00 2023-11-27T15:10:13.197 INFO:teuthology.orchestra.run.smithi133.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.3 MB/s | 255 kB 00:00 2023-11-27T15:10:13.259 INFO:teuthology.orchestra.run.smithi174.stdout:(112/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 2.7 MB/s | 560 kB 00:00 2023-11-27T15:10:13.272 INFO:teuthology.orchestra.run.smithi133.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.6 MB/s | 251 kB 00:00 2023-11-27T15:10:13.298 INFO:teuthology.orchestra.run.smithi133.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 853 kB/s | 85 kB 00:00 2023-11-27T15:10:13.301 INFO:teuthology.orchestra.run.smithi174.stdout:(113/131): libaio-devel-0.3.112-1.el8.x86_64.rp 86 kB/s | 19 kB 00:00 2023-11-27T15:10:13.331 INFO:teuthology.orchestra.run.smithi133.stdout:(68/131): python3-websocket-client-0.56.0-5.el8 385 kB/s | 61 kB 00:00 2023-11-27T15:10:13.335 INFO:teuthology.orchestra.run.smithi174.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 883 kB/s | 155 kB 00:00 2023-11-27T15:10:13.356 INFO:teuthology.orchestra.run.smithi133.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-11-27T15:10:13.407 INFO:teuthology.orchestra.run.smithi133.stdout:(70/131): python3-zc-lockfile-2.0-2.el8.noarch. 171 kB/s | 23 kB 00:00 2023-11-27T15:10:13.427 INFO:teuthology.orchestra.run.smithi174.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 143 kB/s | 18 kB 00:00 2023-11-27T15:10:13.460 INFO:teuthology.orchestra.run.smithi174.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 2.7 MB/s | 559 kB 00:00 2023-11-27T15:10:13.491 INFO:teuthology.orchestra.run.smithi133.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 9.0 MB/s | 1.7 MB 00:00 2023-11-27T15:10:13.536 INFO:teuthology.orchestra.run.smithi174.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 2.9 MB/s | 605 kB 00:00 2023-11-27T15:10:13.553 INFO:teuthology.orchestra.run.smithi174.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.8 MB/s | 238 kB 00:00 2023-11-27T15:10:13.582 INFO:teuthology.orchestra.run.smithi133.stdout:(72/131): python3-pyasn1-modules-0.3.7-6.el8.no 632 kB/s | 110 kB 00:00 2023-11-27T15:10:13.666 INFO:teuthology.orchestra.run.smithi133.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 310 kB/s | 54 kB 00:00 2023-11-27T15:10:13.708 INFO:teuthology.orchestra.run.smithi133.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.0 MB/s | 259 kB 00:00 2023-11-27T15:10:13.708 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2023-11-27T15:10:13.830 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2023-11-27T15:10:13.830 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2023-11-27T15:10:13.841 INFO:teuthology.orchestra.run.smithi133.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 1.8 MB/s | 323 kB 00:00 2023-11-27T15:10:13.867 INFO:teuthology.orchestra.run.smithi133.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 245 kB/s | 39 kB 00:00 2023-11-27T15:10:13.870 INFO:teuthology.orchestra.run.smithi174.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 5.7 MB/s | 2.3 MB 00:00 2023-11-27T15:10:13.992 INFO:teuthology.orchestra.run.smithi133.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 4.5 MB/s | 688 kB 00:00 2023-11-27T15:10:14.025 INFO:teuthology.orchestra.run.smithi133.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 3.3 MB/s | 538 kB 00:00 2023-11-27T15:10:14.120 INFO:teuthology.orchestra.run.smithi174.stdout:(120/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 124 kB/s | 72 kB 00:00 2023-11-27T15:10:14.145 INFO:teuthology.orchestra.run.smithi174.stdout:(121/131): fuse-2.9.7-15.el8.x86_64.rpm 140 kB/s | 83 kB 00:00 2023-11-27T15:10:14.162 INFO:teuthology.orchestra.run.smithi174.stdout:(122/131): elfutils-libelf-devel-0.186-1.el8.x8 205 kB/s | 60 kB 00:00 2023-11-27T15:10:14.192 INFO:teuthology.orchestra.run.smithi133.stdout:(79/131): python3-scipy-1.0.0-21.module+el8.5.0 16 MB/s | 14 MB 00:00 2023-11-27T15:10:14.251 INFO:teuthology.orchestra.run.smithi133.stdout:(80/131): python3-mako-1.0.6-13.el8.noarch.rpm 607 kB/s | 157 kB 00:00 2023-11-27T15:10:14.277 INFO:teuthology.orchestra.run.smithi133.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 748 kB/s | 187 kB 00:00 2023-11-27T15:10:14.279 INFO:teuthology.orchestra.run.smithi174.stdout:(123/131): libuuid-devel-2.32.1-35.el8.x86_64.r 618 kB/s | 98 kB 00:00 2023-11-27T15:10:14.296 INFO:teuthology.orchestra.run.smithi174.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 289 kB/s | 39 kB 00:00 2023-11-27T15:10:14.313 INFO:teuthology.orchestra.run.smithi174.stdout:(125/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 345 kB/s | 58 kB 00:00 2023-11-27T15:10:14.335 INFO:teuthology.orchestra.run.smithi133.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 4.9 MB/s | 713 kB 00:00 2023-11-27T15:10:14.410 INFO:teuthology.orchestra.run.smithi133.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 199 kB/s | 27 kB 00:00 2023-11-27T15:10:14.446 INFO:teuthology.orchestra.run.smithi174.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.1 MB/s | 171 kB 00:00 2023-11-27T15:10:14.480 INFO:teuthology.orchestra.run.smithi174.stdout:(127/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 3.1 MB/s | 644 kB 00:00 2023-11-27T15:10:14.485 INFO:teuthology.orchestra.run.smithi133.stdout:(84/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-11-27T15:10:14.514 INFO:teuthology.orchestra.run.smithi174.stdout:(128/131): NetworkManager-initscripts-updown-1. 686 kB/s | 138 kB 00:00 2023-11-27T15:10:14.527 INFO:teuthology.orchestra.run.smithi133.stdout:(85/131): flex-2.6.1-9.el8.x86_64.rpm 1.6 MB/s | 320 kB 00:00 2023-11-27T15:10:14.561 INFO:teuthology.orchestra.run.smithi133.stdout:(86/131): python3-prettytable-0.7.2-14.el8.noar 292 kB/s | 44 kB 00:00 2023-11-27T15:10:14.672 INFO:teuthology.orchestra.run.smithi174.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 4.9 MB/s | 798 kB 00:00 2023-11-27T15:10:14.678 INFO:teuthology.orchestra.run.smithi133.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.9 MB/s | 225 kB 00:00 2023-11-27T15:10:14.770 INFO:teuthology.orchestra.run.smithi133.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 13 MB/s | 3.7 MB 00:00 2023-11-27T15:10:14.795 INFO:teuthology.orchestra.run.smithi133.stdout:(89/131): python3-werkzeug-0.12.2-4.el8.noarch. 1.7 MB/s | 457 kB 00:00 2023-11-27T15:10:14.806 INFO:teuthology.orchestra.run.smithi174.stdout:(130/131): librados2-18.0.0-3906.g1c956514.el8. 10 MB/s | 3.7 MB 00:00 2023-11-27T15:10:14.845 INFO:teuthology.orchestra.run.smithi133.stdout:(90/131): uuid-1.6.2-43.el8.x86_64.rpm 383 kB/s | 64 kB 00:00 2023-11-27T15:10:14.896 INFO:teuthology.orchestra.run.smithi133.stdout:(91/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.0 MB/s | 126 kB 00:00 2023-11-27T15:10:14.922 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2023-11-27T15:10:14.926 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2023-11-27T15:10:14.971 INFO:teuthology.orchestra.run.smithi133.stdout:(92/131): python3-jsonpointer-1.10-11.el8.noarc 114 kB/s | 20 kB 00:00 2023-11-27T15:10:14.988 INFO:teuthology.orchestra.run.smithi133.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 728 kB/s | 103 kB 00:00 2023-11-27T15:10:15.080 INFO:teuthology.orchestra.run.smithi133.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 3.8 MB/s | 709 kB 00:00 2023-11-27T15:10:15.147 INFO:teuthology.orchestra.run.smithi133.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 1.2 MB/s | 198 kB 00:00 2023-11-27T15:10:15.264 INFO:teuthology.orchestra.run.smithi133.stdout:(96/131): openblas-threads-0.3.15-3.el8.x86_64. 17 MB/s | 4.9 MB 00:00 2023-11-27T15:10:15.289 INFO:teuthology.orchestra.run.smithi133.stdout:(97/131): jq-1.6-3.el8.x86_64.rpm 1.4 MB/s | 202 kB 00:00 2023-11-27T15:10:15.390 INFO:teuthology.orchestra.run.smithi174.stdout:(131/131): librbd1-18.0.0-3906.g1c956514.el8.x8 3.9 MB/s | 3.6 MB 00:00 2023-11-27T15:10:15.393 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2023-11-27T15:10:15.394 INFO:teuthology.orchestra.run.smithi174.stdout:Total 11 MB/s | 197 MB 00:17 2023-11-27T15:10:15.406 INFO:teuthology.orchestra.run.smithi133.stdout:(98/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-11-27T15:10:15.459 INFO:teuthology.orchestra.run.smithi133.stdout:(99/131): librdkafka-0.11.4-3.el8.x86_64.rpm 1.8 MB/s | 354 kB 00:00 2023-11-27T15:10:15.548 INFO:teuthology.orchestra.run.smithi133.stdout:(100/131): postgresql-contrib-10.21-2.module+el 5.6 MB/s | 808 kB 00:00 2023-11-27T15:10:15.632 INFO:teuthology.orchestra.run.smithi133.stdout:(101/131): postgresql-server-10.21-2.module+el8 15 MB/s | 5.1 MB 00:00 2023-11-27T15:10:15.682 INFO:teuthology.orchestra.run.smithi133.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 6.8 MB/s | 1.5 MB 00:00 2023-11-27T15:10:15.707 INFO:teuthology.orchestra.run.smithi133.stdout:(103/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 295 kB/s | 47 kB 00:00 2023-11-27T15:10:15.782 INFO:teuthology.orchestra.run.smithi133.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 426 kB/s | 43 kB 00:00 2023-11-27T15:10:15.816 INFO:teuthology.orchestra.run.smithi133.stdout:(105/131): libselinux-devel-2.9-5.el8.x86_64.rp 1.1 MB/s | 200 kB 00:00 2023-11-27T15:10:15.896 INFO:teuthology.orchestra.run.smithi133.stdout:(106/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.1 MB/s | 220 kB 00:00 2023-11-27T15:10:15.925 INFO:teuthology.orchestra.run.smithi133.stdout:(107/131): keyutils-libs-devel-1.5.10-9.el8.x86 338 kB/s | 48 kB 00:00 2023-11-27T15:10:15.967 INFO:teuthology.orchestra.run.smithi133.stdout:(108/131): libsepol-devel-2.9-3.el8.x86_64.rpm 579 kB/s | 87 kB 00:00 2023-11-27T15:10:16.034 INFO:teuthology.orchestra.run.smithi133.stdout:(109/131): python3-requests-oauthlib-1.0.0-1.el 311 kB/s | 43 kB 00:00 2023-11-27T15:10:16.084 INFO:teuthology.orchestra.run.smithi133.stdout:(110/131): python3-pycparser-2.14-14.el8.noarch 686 kB/s | 109 kB 00:00 2023-11-27T15:10:16.151 INFO:teuthology.orchestra.run.smithi133.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.2 MB/s | 229 kB 00:00 2023-11-27T15:10:16.218 INFO:teuthology.orchestra.run.smithi133.stdout:(112/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.0 MB/s | 560 kB 00:00 2023-11-27T15:10:16.232 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2023-11-27T15:10:16.276 INFO:teuthology.orchestra.run.smithi133.stdout:(113/131): libaio-devel-0.3.112-1.el8.x86_64.rp 97 kB/s | 19 kB 00:00 2023-11-27T15:10:16.352 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2023-11-27T15:10:16.352 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2023-11-27T15:10:16.577 INFO:teuthology.orchestra.run.smithi133.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 364 kB/s | 155 kB 00:00 2023-11-27T15:10:16.594 INFO:teuthology.orchestra.run.smithi133.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 56 kB/s | 18 kB 00:00 2023-11-27T15:10:16.628 INFO:teuthology.orchestra.run.smithi133.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 1.3 MB/s | 559 kB 00:00 2023-11-27T15:10:16.762 INFO:teuthology.orchestra.run.smithi133.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.2 MB/s | 605 kB 00:00 2023-11-27T15:10:16.787 INFO:teuthology.orchestra.run.smithi133.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.2 MB/s | 238 kB 00:00 2023-11-27T15:10:16.862 INFO:teuthology.orchestra.run.smithi133.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 9.9 MB/s | 2.3 MB 00:00 2023-11-27T15:10:16.895 INFO:teuthology.orchestra.run.smithi133.stdout:(120/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 542 kB/s | 72 kB 00:00 2023-11-27T15:10:16.920 INFO:teuthology.orchestra.run.smithi133.stdout:(121/131): fuse-2.9.7-15.el8.x86_64.rpm 620 kB/s | 83 kB 00:00 2023-11-27T15:10:16.987 INFO:teuthology.orchestra.run.smithi133.stdout:(122/131): elfutils-libelf-devel-0.186-1.el8.x8 477 kB/s | 60 kB 00:00 2023-11-27T15:10:17.038 INFO:teuthology.orchestra.run.smithi133.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 494 kB/s | 58 kB 00:00 2023-11-27T15:10:17.088 INFO:teuthology.orchestra.run.smithi133.stdout:(124/131): libuuid-devel-2.32.1-35.el8.x86_64.r 510 kB/s | 98 kB 00:00 2023-11-27T15:10:17.148 INFO:teuthology.orchestra.run.smithi133.stdout:(125/131): libcom_err-devel-1.45.6-4.el8.x86_64 240 kB/s | 39 kB 00:00 2023-11-27T15:10:17.189 INFO:teuthology.orchestra.run.smithi133.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.7 MB/s | 171 kB 00:00 2023-11-27T15:10:17.222 INFO:teuthology.orchestra.run.smithi133.stdout:(127/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 3.4 MB/s | 644 kB 00:00 2023-11-27T15:10:17.248 INFO:teuthology.orchestra.run.smithi133.stdout:(128/131): NetworkManager-initscripts-updown-1. 1.4 MB/s | 138 kB 00:00 2023-11-27T15:10:17.406 INFO:teuthology.orchestra.run.smithi133.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 4.9 MB/s | 798 kB 00:00 2023-11-27T15:10:17.439 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2023-11-27T15:10:17.443 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2023-11-27T15:10:17.590 INFO:teuthology.orchestra.run.smithi133.stdout:(130/131): librados2-18.0.0-3906.g1c956514.el8. 9.1 MB/s | 3.7 MB 00:00 2023-11-27T15:10:17.698 INFO:teuthology.orchestra.run.smithi133.stdout:(131/131): librbd1-18.0.0-3906.g1c956514.el8.x8 7.5 MB/s | 3.6 MB 00:00 2023-11-27T15:10:17.703 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2023-11-27T15:10:17.703 INFO:teuthology.orchestra.run.smithi133.stdout:Total 10 MB/s | 197 MB 00:18 2023-11-27T15:10:19.381 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2023-11-27T15:10:19.516 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2023-11-27T15:10:19.517 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2023-11-27T15:10:20.363 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2023-11-27T15:10:20.584 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-27T15:10:20.784 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-27T15:10:20.978 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-27T15:10:21.061 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2023-11-27T15:10:21.064 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2023-11-27T15:10:21.084 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-27T15:10:21.328 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-27T15:10:21.752 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-27T15:10:22.236 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 5/134 2023-11-27T15:10:22.410 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2023-11-27T15:10:22.616 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 5/134 2023-11-27T15:10:22.626 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-27T15:10:22.790 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-27T15:10:22.861 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:10:22.989 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:10:23.026 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-27T15:10:23.178 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-27T15:10:23.202 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:10:23.294 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:10:23.296 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-27T15:10:23.499 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:10:23.594 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-27T15:10:23.669 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-27T15:10:23.941 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-27T15:10:23.994 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-27T15:10:24.130 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-27T15:10:24.253 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:10:24.549 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-27T15:10:24.632 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:10:24.779 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-27T15:10:24.844 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:10:24.957 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:10:24.969 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-27T15:10:25.077 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-27T15:10:25.161 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:10:25.253 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:10:25.260 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-27T15:10:25.404 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-27T15:10:25.470 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 9/134 2023-11-27T15:10:25.631 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-27T15:10:25.681 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-27T15:10:25.783 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-27T15:10:25.922 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-27T15:10:26.002 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-27T15:10:26.118 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-27T15:10:26.163 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2023-11-27T15:10:26.264 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-27T15:10:26.322 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-27T15:10:26.385 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-27T15:10:26.418 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-27T15:10:26.600 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-27T15:10:26.752 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-27T15:10:26.777 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-27T15:10:26.858 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-27T15:10:26.997 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-27T15:10:27.035 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-27T15:10:27.036 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-27T15:10:27.124 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-27T15:10:27.203 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-27T15:10:27.218 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-27T15:10:27.244 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-27T15:10:27.357 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-27T15:10:27.385 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-27T15:10:27.535 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-27T15:10:27.603 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-27T15:10:27.800 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-27T15:10:28.043 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-27T15:10:28.057 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-27T15:10:28.148 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-27T15:10:28.261 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:10:28.352 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-27T15:10:28.712 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:10:28.750 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-27T15:10:28.919 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:10:28.931 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-27T15:10:28.999 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-27T15:10:29.071 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:10:29.143 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-27T15:10:29.241 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-27T15:10:29.261 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:10:29.383 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:10:29.511 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-27T15:10:29.566 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 9/134 2023-11-27T15:10:29.675 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 26/134 2023-11-27T15:10:29.834 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-27T15:10:29.846 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-27T15:10:29.862 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 27/134 2023-11-27T15:10:30.058 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-27T15:10:30.123 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 28/134 2023-11-27T15:10:30.133 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-27T15:10:30.368 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-27T15:10:30.386 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-27T15:10:30.503 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-27T15:10:30.580 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-27T15:10:30.734 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-27T15:10:30.791 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-27T15:10:30.903 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-27T15:10:30.990 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-27T15:10:31.039 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 14/134 2023-11-27T15:10:31.194 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-27T15:10:31.244 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-27T15:10:31.303 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 15/134 2023-11-27T15:10:31.450 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 26/134 2023-11-27T15:10:31.469 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-27T15:10:31.588 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 27/134 2023-11-27T15:10:31.637 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-27T15:10:32.027 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-27T15:10:32.117 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-27T15:10:32.553 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 28/134 2023-11-27T15:10:32.716 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-27T15:10:32.894 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-27T15:10:32.916 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-27T15:10:33.153 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-27T15:10:33.203 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-27T15:10:33.457 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-27T15:10:33.498 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-27T15:10:33.581 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-27T15:10:33.634 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-27T15:10:33.752 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-27T15:10:33.786 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-27T15:10:33.997 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-27T15:10:34.237 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-27T15:10:34.478 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-27T15:10:34.609 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-27T15:10:34.656 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 36/134 2023-11-27T15:10:34.848 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 37/134 2023-11-27T15:10:34.860 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-27T15:10:34.992 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-27T15:10:35.092 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-27T15:10:35.121 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-27T15:10:35.308 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-27T15:10:35.324 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-27T15:10:35.378 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-27T15:10:35.384 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-27T15:10:35.552 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-27T15:10:35.630 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-27T15:10:35.736 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-27T15:10:35.785 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-27T15:10:35.895 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-27T15:10:35.979 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-27T15:10:36.042 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-27T15:10:36.053 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-27T15:10:36.123 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-27T15:10:36.169 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-27T15:10:36.217 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 26/134 2023-11-27T15:10:36.379 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-27T15:10:36.381 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 27/134 2023-11-27T15:10:36.398 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 36/134 2023-11-27T15:10:36.580 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 37/134 2023-11-27T15:10:36.587 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-27T15:10:36.748 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-27T15:10:36.766 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-27T15:10:36.902 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-27T15:10:36.960 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-27T15:10:37.111 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-27T15:10:37.127 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-27T15:10:37.271 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-27T15:10:37.312 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-27T15:10:37.397 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 28/134 2023-11-27T15:10:37.516 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-27T15:10:37.528 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-27T15:10:37.730 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-27T15:10:37.739 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-27T15:10:37.780 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-27T15:10:38.082 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-27T15:10:38.114 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-27T15:10:38.130 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-27T15:10:38.348 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-27T15:10:38.429 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-27T15:10:38.460 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-27T15:10:38.535 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-27T15:10:38.652 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-27T15:10:38.667 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-27T15:10:38.909 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-27T15:10:38.921 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-27T15:10:39.154 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-27T15:10:39.172 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-27T15:10:39.456 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-27T15:10:39.468 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-27T15:10:39.665 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-27T15:10:39.716 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-27T15:10:39.910 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-27T15:10:39.951 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-27T15:10:40.163 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-27T15:10:40.196 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-27T15:10:40.369 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-27T15:10:40.379 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-27T15:10:40.544 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-27T15:10:40.572 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-27T15:10:40.577 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-27T15:10:40.876 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-27T15:10:40.893 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-27T15:10:40.916 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-27T15:10:41.130 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-27T15:10:41.137 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-27T15:10:41.153 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-27T15:10:41.355 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-27T15:10:41.364 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-27T15:10:41.446 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-27T15:10:41.534 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 36/134 2023-11-27T15:10:41.590 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-27T15:10:41.717 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-27T15:10:41.747 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 37/134 2023-11-27T15:10:41.765 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-27T15:10:41.890 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-27T15:10:41.909 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-27T15:10:41.921 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-27T15:10:42.121 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-27T15:10:42.124 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-27T15:10:42.126 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-27T15:10:42.179 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-27T15:10:42.295 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-27T15:10:42.408 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-27T15:10:42.448 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-27T15:10:42.455 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-27T15:10:42.622 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-27T15:10:42.653 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-27T15:10:42.690 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-27T15:10:42.811 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-27T15:10:42.825 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-27T15:10:42.880 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-27T15:10:42.907 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-27T15:10:42.983 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-27T15:10:43.151 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-27T15:10:43.214 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-27T15:10:43.322 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-27T15:10:43.325 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-27T15:10:43.443 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-27T15:10:43.482 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-27T15:10:43.625 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-27T15:10:43.632 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-27T15:10:43.685 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-27T15:10:43.835 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-27T15:10:43.857 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-27T15:10:43.942 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-27T15:10:43.988 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-27T15:10:44.094 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-27T15:10:44.173 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-27T15:10:44.254 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-27T15:10:44.320 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-27T15:10:44.377 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-27T15:10:44.501 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-27T15:10:44.528 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-27T15:10:44.707 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-27T15:10:44.713 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-27T15:10:44.750 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-27T15:10:44.936 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-27T15:10:44.960 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-27T15:10:44.995 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-27T15:10:45.118 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-27T15:10:45.154 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-27T15:10:45.257 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-27T15:10:45.327 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-27T15:10:45.343 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-27T15:10:45.422 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-27T15:10:45.540 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-27T15:10:45.665 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-27T15:10:45.704 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-27T15:10:45.856 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-27T15:10:46.039 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-27T15:10:46.114 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-27T15:10:46.310 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-27T15:10:46.328 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-27T15:10:46.555 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-27T15:10:46.565 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-27T15:10:46.767 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-27T15:10:46.797 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-27T15:10:46.841 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-27T15:10:47.009 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-27T15:10:47.040 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-27T15:10:47.076 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-27T15:10:47.215 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-27T15:10:47.237 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-27T15:10:47.365 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-27T15:10:47.391 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-27T15:10:47.474 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-27T15:10:47.475 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-27T15:10:47.672 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-27T15:10:47.726 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-27T15:10:47.955 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-27T15:10:47.981 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-27T15:10:48.091 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-27T15:10:48.186 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-27T15:10:48.229 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-27T15:10:48.439 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-27T15:10:48.600 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-27T15:10:48.627 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-27T15:10:48.801 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-27T15:10:48.945 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-27T15:10:48.999 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-27T15:10:49.015 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-27T15:10:49.191 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-27T15:10:49.231 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-27T15:10:49.282 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-27T15:10:49.307 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-27T15:10:49.498 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 93/134 2023-11-27T15:10:49.499 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-27T15:10:49.552 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-27T15:10:49.671 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-27T15:10:49.820 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-27T15:10:49.853 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-27T15:10:50.040 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-27T15:10:50.054 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-27T15:10:50.239 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-27T15:10:50.463 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-27T15:10:50.619 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-27T15:10:50.764 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-27T15:10:50.825 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-27T15:10:51.007 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-27T15:10:51.034 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-27T15:10:51.077 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-27T15:10:51.241 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 93/134 2023-11-27T15:10:51.288 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-27T15:10:51.472 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-27T15:10:51.748 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-27T15:10:51.950 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-27T15:10:52.200 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-27T15:10:52.429 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-27T15:10:53.132 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:10:53.747 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-27T15:10:54.011 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-27T15:10:54.181 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:10:54.296 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-27T15:10:54.647 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-27T15:10:54.786 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:10:54.944 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-27T15:10:54.944 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:10:55.093 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-27T15:10:55.174 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-27T15:10:55.182 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:10:55.185 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-27T15:10:55.322 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-27T15:10:55.777 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:10:55.968 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-27T15:10:56.066 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-27T15:10:56.135 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-27T15:10:56.140 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-27T15:10:56.254 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 92/134 2023-11-27T15:10:56.432 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 93/134 2023-11-27T15:10:57.056 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-27T15:11:01.484 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:11:01.715 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:11:02.401 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 94/134 2023-11-27T15:11:03.644 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-27T15:11:03.797 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 95/134 2023-11-27T15:11:03.989 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-27T15:11:14.307 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-27T15:11:14.307 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /sys 2023-11-27T15:11:14.308 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /proc 2023-11-27T15:11:14.308 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /dev 2023-11-27T15:11:14.308 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /run 2023-11-27T15:11:14.308 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /mnt 2023-11-27T15:11:14.308 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /var/tmp 2023-11-27T15:11:14.308 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /home 2023-11-27T15:11:14.308 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /tmp 2023-11-27T15:11:14.308 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /dev 2023-11-27T15:11:14.308 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-27T15:11:14.677 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-27T15:11:16.240 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-27T15:11:16.367 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-27T15:11:16.367 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /sys 2023-11-27T15:11:16.367 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /proc 2023-11-27T15:11:16.367 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /dev 2023-11-27T15:11:16.368 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /run 2023-11-27T15:11:16.368 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /mnt 2023-11-27T15:11:16.368 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /var/tmp 2023-11-27T15:11:16.368 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /home 2023-11-27T15:11:16.368 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /tmp 2023-11-27T15:11:16.368 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /dev 2023-11-27T15:11:16.368 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-27T15:11:16.472 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-27T15:11:16.691 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-27T15:11:16.703 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-27T15:11:16.910 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-27T15:11:17.359 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-27T15:11:17.568 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-27T15:11:17.585 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-27T15:11:18.267 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-27T15:11:18.454 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-27T15:11:18.672 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-27T15:11:18.870 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-27T15:11:19.267 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-27T15:11:19.320 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-27T15:11:19.474 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-27T15:11:19.503 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-27T15:11:19.520 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-27T15:11:19.640 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 102/134 2023-11-27T15:11:19.840 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 103/134 2023-11-27T15:11:20.067 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-27T15:11:20.283 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-27T15:11:20.538 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-27T15:11:21.418 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-27T15:11:21.497 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-27T15:11:21.615 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-27T15:11:21.667 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-27T15:11:21.757 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-27T15:11:21.775 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 102/134 2023-11-27T15:11:21.978 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 103/134 2023-11-27T15:11:22.141 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 107/134 2023-11-27T15:11:22.185 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-27T15:11:22.352 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-27T15:11:22.387 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-27T15:11:22.400 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-27T15:11:22.575 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-27T15:11:22.586 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-27T15:11:22.703 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-27T15:11:23.665 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-27T15:11:23.850 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 110/134 2023-11-27T15:11:23.860 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-27T15:11:23.951 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-27T15:11:24.074 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-27T15:11:24.107 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-27T15:11:24.300 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-27T15:11:24.342 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-27T15:11:24.348 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 107/134 2023-11-27T15:11:24.470 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 96/134 2023-11-27T15:11:24.470 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /sys 2023-11-27T15:11:24.470 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /proc 2023-11-27T15:11:24.470 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /dev 2023-11-27T15:11:24.470 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /run 2023-11-27T15:11:24.470 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /mnt 2023-11-27T15:11:24.470 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /var/tmp 2023-11-27T15:11:24.470 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /home 2023-11-27T15:11:24.470 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /tmp 2023-11-27T15:11:24.470 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /dev 2023-11-27T15:11:24.470 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:11:24.508 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-27T15:11:24.614 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-27T15:11:24.650 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-27T15:11:24.793 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-27T15:11:24.804 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-27T15:11:24.853 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-27T15:11:24.869 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-27T15:11:25.066 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-27T15:11:26.066 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 110/134 2023-11-27T15:11:26.288 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-27T15:11:26.322 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-27T15:11:26.469 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-27T15:11:26.513 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-27T15:11:26.577 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 97/134 2023-11-27T15:11:26.702 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-27T15:11:26.740 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-27T15:11:27.006 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 98/134 2023-11-27T15:11:27.048 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-27T15:11:27.177 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-27T15:11:27.210 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-27T15:11:27.710 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 99/134 2023-11-27T15:11:27.901 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-27T15:11:27.944 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 100/134 2023-11-27T15:11:29.000 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-27T15:11:29.923 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 115/134 2023-11-27T15:11:30.246 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-27T15:11:30.435 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 117/134 2023-11-27T15:11:30.721 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-27T15:11:30.803 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-27T15:11:30.873 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-27T15:11:30.967 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 101/134 2023-11-27T15:11:31.062 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-27T15:11:31.123 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-27T15:11:31.125 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 102/134 2023-11-27T15:11:31.330 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 103/134 2023-11-27T15:11:31.336 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-27T15:11:31.413 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 115/134 2023-11-27T15:11:31.527 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 121/134 2023-11-27T15:11:31.542 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-27T15:11:31.600 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-27T15:11:31.716 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-27T15:11:31.791 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-27T15:11:31.800 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 104/134 2023-11-27T15:11:32.005 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-27T15:11:32.679 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-27T15:11:32.990 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-27T15:11:33.087 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 105/134 2023-11-27T15:11:33.195 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 119/134 2023-11-27T15:11:33.275 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-27T15:11:33.346 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-27T15:11:33.404 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 106/134 2023-11-27T15:11:33.554 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 121/134 2023-11-27T15:11:33.701 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-27T15:11:33.845 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 107/134 2023-11-27T15:11:33.929 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:11:34.089 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-27T15:11:34.154 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 108/134 2023-11-27T15:11:34.206 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:11:34.224 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:11:34.365 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 124/134 2023-11-27T15:11:34.392 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-27T15:11:34.420 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 109/134 2023-11-27T15:11:34.697 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 125/134 2023-11-27T15:11:35.044 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 126/134 2023-11-27T15:11:35.407 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-27T15:11:35.631 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-27T15:11:35.713 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-27T15:11:35.757 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 110/134 2023-11-27T15:11:35.898 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-27T15:11:35.910 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-27T15:11:35.964 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-27T15:11:35.970 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:11:36.026 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 111/134 2023-11-27T15:11:36.097 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-27T15:11:36.105 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-27T15:11:36.175 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-27T15:11:36.217 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:11:36.233 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-27T15:11:36.233 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:11:36.235 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:11:36.246 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3906.g1c956 112/134 2023-11-27T15:11:36.434 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-27T15:11:36.450 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 124/134 2023-11-27T15:11:36.492 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:11:36.492 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-27T15:11:36.716 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 125/134 2023-11-27T15:11:36.820 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-27T15:11:36.838 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 113/134 2023-11-27T15:11:37.000 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-27T15:11:37.063 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:11:37.063 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 126/134 2023-11-27T15:11:37.393 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-27T15:11:37.524 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-27T15:11:37.533 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-27T15:11:37.595 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:11:37.595 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 1/134 2023-11-27T15:11:37.595 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2/134 2023-11-27T15:11:37.595 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 3/134 2023-11-27T15:11:37.595 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 4/134 2023-11-27T15:11:37.595 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 5/134 2023-11-27T15:11:37.595 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:11:37.595 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:11:37.595 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:11:37.595 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 9/134 2023-11-27T15:11:37.596 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 10/134 2023-11-27T15:11:37.596 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 11/134 2023-11-27T15:11:37.596 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 12/134 2023-11-27T15:11:37.596 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 13/134 2023-11-27T15:11:37.596 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 14/134 2023-11-27T15:11:37.596 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 15/134 2023-11-27T15:11:37.596 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 16/134 2023-11-27T15:11:37.596 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 17/134 2023-11-27T15:11:37.596 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 18/134 2023-11-27T15:11:37.596 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 19/134 2023-11-27T15:11:37.596 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 20/134 2023-11-27T15:11:37.596 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 21/134 2023-11-27T15:11:37.596 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 22/134 2023-11-27T15:11:37.597 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 23/134 2023-11-27T15:11:37.597 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 24/134 2023-11-27T15:11:37.597 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 25/134 2023-11-27T15:11:37.597 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 26/134 2023-11-27T15:11:37.597 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 27/134 2023-11-27T15:11:37.597 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 28/134 2023-11-27T15:11:37.597 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 29/134 2023-11-27T15:11:37.597 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 30/134 2023-11-27T15:11:37.597 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 31/134 2023-11-27T15:11:37.597 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 32/134 2023-11-27T15:11:37.597 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 33/134 2023-11-27T15:11:37.597 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 34/134 2023-11-27T15:11:37.597 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 35/134 2023-11-27T15:11:37.598 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 36/134 2023-11-27T15:11:37.598 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 37/134 2023-11-27T15:11:37.598 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 38/134 2023-11-27T15:11:37.598 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-27T15:11:37.598 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-27T15:11:37.598 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-27T15:11:37.598 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-27T15:11:37.598 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-27T15:11:37.598 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-27T15:11:37.598 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-27T15:11:37.598 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-27T15:11:37.598 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-27T15:11:37.598 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-27T15:11:37.599 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-27T15:11:37.599 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-27T15:11:37.599 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-27T15:11:37.599 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-27T15:11:37.599 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-27T15:11:37.599 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-27T15:11:37.599 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-27T15:11:37.599 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-27T15:11:37.599 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-27T15:11:37.599 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-27T15:11:37.599 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-27T15:11:37.599 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-27T15:11:37.599 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-27T15:11:37.600 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-27T15:11:37.600 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-27T15:11:37.600 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-27T15:11:37.600 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-27T15:11:37.600 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-27T15:11:37.600 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-27T15:11:37.600 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-27T15:11:37.600 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-27T15:11:37.600 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-27T15:11:37.600 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-27T15:11:37.600 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-27T15:11:37.600 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-27T15:11:37.600 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-27T15:11:37.600 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-27T15:11:37.601 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-27T15:11:37.601 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-27T15:11:37.601 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-27T15:11:37.601 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-27T15:11:37.601 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-27T15:11:37.601 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-27T15:11:37.601 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-27T15:11:37.601 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-27T15:11:37.601 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-27T15:11:37.601 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-27T15:11:37.601 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-27T15:11:37.601 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-27T15:11:37.601 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-27T15:11:37.601 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-27T15:11:37.602 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-27T15:11:37.602 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-27T15:11:37.602 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-27T15:11:37.602 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-27T15:11:37.602 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-27T15:11:37.602 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-27T15:11:37.602 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-27T15:11:37.602 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-27T15:11:37.602 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-27T15:11:37.602 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-27T15:11:37.602 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-27T15:11:37.602 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-27T15:11:37.602 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-27T15:11:37.603 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-27T15:11:37.603 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-27T15:11:37.603 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-27T15:11:37.603 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-27T15:11:37.603 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-27T15:11:37.603 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-27T15:11:37.603 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-27T15:11:37.603 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-27T15:11:37.603 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-27T15:11:37.603 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-27T15:11:37.603 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-27T15:11:37.603 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-27T15:11:37.603 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-27T15:11:37.604 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-27T15:11:37.604 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-27T15:11:37.604 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-27T15:11:37.604 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-27T15:11:37.604 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-27T15:11:37.604 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-27T15:11:37.604 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-27T15:11:37.604 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-27T15:11:37.604 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-27T15:11:37.604 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-27T15:11:37.604 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-27T15:11:37.604 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-27T15:11:37.604 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-27T15:11:37.604 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 129/134 2023-11-27T15:11:37.605 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-27T15:11:37.605 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 131/134 2023-11-27T15:11:37.605 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:11:37.605 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-27T15:11:37.759 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-27T15:11:37.932 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-27T15:11:38.033 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-27T15:11:38.216 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-27T15:11:38.229 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-27T15:11:38.229 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:11:38.444 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:11:38.444 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-27T15:11:38.664 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-27T15:11:38.873 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:11:39.398 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:11:39.398 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 1/134 2023-11-27T15:11:39.398 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2/134 2023-11-27T15:11:39.398 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 3/134 2023-11-27T15:11:39.398 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 4/134 2023-11-27T15:11:39.399 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 5/134 2023-11-27T15:11:39.399 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:11:39.399 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:11:39.399 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:11:39.399 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 9/134 2023-11-27T15:11:39.399 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 10/134 2023-11-27T15:11:39.399 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 11/134 2023-11-27T15:11:39.399 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 12/134 2023-11-27T15:11:39.399 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 13/134 2023-11-27T15:11:39.399 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 14/134 2023-11-27T15:11:39.399 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 15/134 2023-11-27T15:11:39.399 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 16/134 2023-11-27T15:11:39.400 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 17/134 2023-11-27T15:11:39.400 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 18/134 2023-11-27T15:11:39.400 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 19/134 2023-11-27T15:11:39.400 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 20/134 2023-11-27T15:11:39.400 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 21/134 2023-11-27T15:11:39.400 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 22/134 2023-11-27T15:11:39.400 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 23/134 2023-11-27T15:11:39.400 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 24/134 2023-11-27T15:11:39.400 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 25/134 2023-11-27T15:11:39.400 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 26/134 2023-11-27T15:11:39.400 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 27/134 2023-11-27T15:11:39.400 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 28/134 2023-11-27T15:11:39.400 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 29/134 2023-11-27T15:11:39.400 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 30/134 2023-11-27T15:11:39.401 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 31/134 2023-11-27T15:11:39.401 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 32/134 2023-11-27T15:11:39.401 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 33/134 2023-11-27T15:11:39.401 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 34/134 2023-11-27T15:11:39.401 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 35/134 2023-11-27T15:11:39.401 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 36/134 2023-11-27T15:11:39.401 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 37/134 2023-11-27T15:11:39.402 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 38/134 2023-11-27T15:11:39.402 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-27T15:11:39.402 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-27T15:11:39.402 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-27T15:11:39.402 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-27T15:11:39.402 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-27T15:11:39.402 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-27T15:11:39.402 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-27T15:11:39.402 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-27T15:11:39.402 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-27T15:11:39.402 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-27T15:11:39.402 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-27T15:11:39.402 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-27T15:11:39.403 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-27T15:11:39.403 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-27T15:11:39.403 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-27T15:11:39.403 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-27T15:11:39.403 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-27T15:11:39.403 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-27T15:11:39.403 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-27T15:11:39.404 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-27T15:11:39.404 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-27T15:11:39.404 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-27T15:11:39.404 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-27T15:11:39.404 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-27T15:11:39.404 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-27T15:11:39.404 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-27T15:11:39.404 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-27T15:11:39.404 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-27T15:11:39.404 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-27T15:11:39.404 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-27T15:11:39.404 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-27T15:11:39.404 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-27T15:11:39.405 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-27T15:11:39.405 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-27T15:11:39.405 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-27T15:11:39.405 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-27T15:11:39.405 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-27T15:11:39.405 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-27T15:11:39.405 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-27T15:11:39.405 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-27T15:11:39.405 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-27T15:11:39.406 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-27T15:11:39.406 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-27T15:11:39.406 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-27T15:11:39.406 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-27T15:11:39.406 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-27T15:11:39.406 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-27T15:11:39.406 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-27T15:11:39.406 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-27T15:11:39.406 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-27T15:11:39.406 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-27T15:11:39.406 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-27T15:11:39.406 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-27T15:11:39.407 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-27T15:11:39.407 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-27T15:11:39.407 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-27T15:11:39.407 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-27T15:11:39.407 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-27T15:11:39.407 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-27T15:11:39.407 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-27T15:11:39.407 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-27T15:11:39.407 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-27T15:11:39.408 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-27T15:11:39.408 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-27T15:11:39.408 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-27T15:11:39.408 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-27T15:11:39.408 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-27T15:11:39.408 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-27T15:11:39.408 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-27T15:11:39.408 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-27T15:11:39.408 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-27T15:11:39.408 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-27T15:11:39.409 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-27T15:11:39.409 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-27T15:11:39.409 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-27T15:11:39.409 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-27T15:11:39.409 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-27T15:11:39.409 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-27T15:11:39.409 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-27T15:11:39.410 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-27T15:11:39.410 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-27T15:11:39.410 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-27T15:11:39.410 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-27T15:11:39.410 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-27T15:11:39.410 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-27T15:11:39.410 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-27T15:11:39.410 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-27T15:11:39.410 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-27T15:11:39.410 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-27T15:11:39.411 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-27T15:11:39.411 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 129/134 2023-11-27T15:11:39.411 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-27T15:11:39.411 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 131/134 2023-11-27T15:11:39.411 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:11:39.411 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-27T15:11:40.229 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:11:40.229 INFO:teuthology.orchestra.run.smithi177.stdout:Last metadata expiration check: 0:01:57 ago on Mon 27 Nov 2023 03:09:43 PM UTC. 2023-11-27T15:11:41.257 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 114/134 2023-11-27T15:11:41.601 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 115/134 2023-11-27T15:11:41.773 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-27T15:11:41.925 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:11:41.925 INFO:teuthology.orchestra.run.smithi174.stdout:Last metadata expiration check: 0:01:55 ago on Mon 27 Nov 2023 03:09:46 PM UTC. 2023-11-27T15:11:42.052 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-27T15:11:43.302 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-27T15:11:43.678 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-27T15:11:43.879 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 119/134 2023-11-27T15:11:44.078 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-27T15:11:44.286 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 121/134 2023-11-27T15:11:44.424 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-27T15:11:45.737 INFO:teuthology.orchestra.run.smithi177.stdout:Installed products updated. 2023-11-27T15:11:45.954 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-27T15:11:45.954 INFO:teuthology.orchestra.run.smithi177.stdout:Upgraded: 2023-11-27T15:11:45.954 INFO:teuthology.orchestra.run.smithi177.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-27T15:11:45.954 INFO:teuthology.orchestra.run.smithi177.stdout: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.955 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.955 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2023-11-27T15:11:45.955 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-27T15:11:45.955 INFO:teuthology.orchestra.run.smithi177.stdout: automake-1.16.1-7.el8.noarch 2023-11-27T15:11:45.955 INFO:teuthology.orchestra.run.smithi177.stdout: bison-3.0.4-10.el8.x86_64 2023-11-27T15:11:45.955 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.955 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.955 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.955 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.955 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:11:45.955 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.955 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.955 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.956 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:11:45.956 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:11:45.956 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:11:45.956 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:11:45.956 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:11:45.956 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.956 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.956 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:11:45.956 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.956 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.956 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.956 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:11:45.956 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:11:45.956 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.957 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:11:45.957 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-27T15:11:45.957 INFO:teuthology.orchestra.run.smithi177.stdout: flex-2.6.1-9.el8.x86_64 2023-11-27T15:11:45.957 INFO:teuthology.orchestra.run.smithi177.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-27T15:11:45.957 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-27T15:11:45.957 INFO:teuthology.orchestra.run.smithi177.stdout: jq-1.6-3.el8.x86_64 2023-11-27T15:11:45.957 INFO:teuthology.orchestra.run.smithi177.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-27T15:11:45.957 INFO:teuthology.orchestra.run.smithi177.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-27T15:11:45.957 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-27T15:11:45.957 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.957 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.957 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.957 INFO:teuthology.orchestra.run.smithi177.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-27T15:11:45.957 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-27T15:11:45.957 INFO:teuthology.orchestra.run.smithi177.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-27T15:11:45.958 INFO:teuthology.orchestra.run.smithi177.stdout: libpq-13.5-1.el8.x86_64 2023-11-27T15:11:45.958 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-27T15:11:45.958 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-27T15:11:45.958 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.958 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.958 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-27T15:11:45.958 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.958 INFO:teuthology.orchestra.run.smithi177.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-27T15:11:45.958 INFO:teuthology.orchestra.run.smithi177.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-27T15:11:45.958 INFO:teuthology.orchestra.run.smithi177.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-27T15:11:45.958 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-27T15:11:45.958 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-27T15:11:45.958 INFO:teuthology.orchestra.run.smithi177.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-27T15:11:45.959 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-27T15:11:45.959 INFO:teuthology.orchestra.run.smithi177.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-27T15:11:45.959 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-27T15:11:45.959 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-27T15:11:45.959 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-27T15:11:45.959 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-27T15:11:45.959 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-27T15:11:45.959 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-27T15:11:45.959 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:11:45.959 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:11:45.959 INFO:teuthology.orchestra.run.smithi177.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:11:45.959 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-27T15:11:45.959 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-27T15:11:45.960 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-27T15:11:45.960 INFO:teuthology.orchestra.run.smithi177.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-27T15:11:45.960 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-27T15:11:45.960 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.960 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.960 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.960 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-27T15:11:45.960 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-27T15:11:45.960 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-27T15:11:45.960 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-27T15:11:45.960 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-27T15:11:45.960 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-27T15:11:45.960 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-27T15:11:45.961 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-27T15:11:45.961 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-27T15:11:45.961 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-27T15:11:45.961 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-27T15:11:45.961 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-27T15:11:45.961 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-27T15:11:45.961 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-27T15:11:45.961 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-27T15:11:45.961 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-27T15:11:45.961 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-27T15:11:45.961 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-27T15:11:45.961 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-27T15:11:45.961 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-27T15:11:45.962 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-27T15:11:45.962 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-27T15:11:45.962 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend-2.6-1.el8.noarch 2023-11-27T15:11:45.962 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-27T15:11:45.962 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-27T15:11:45.962 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-27T15:11:45.962 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-27T15:11:45.962 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-27T15:11:45.962 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-27T15:11:45.962 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.962 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.962 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-27T15:11:45.962 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-27T15:11:45.963 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.963 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-27T15:11:45.963 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-27T15:11:45.963 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-27T15:11:45.963 INFO:teuthology.orchestra.run.smithi177.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-27T15:11:45.963 INFO:teuthology.orchestra.run.smithi177.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-27T15:11:45.963 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-27T15:11:45.963 INFO:teuthology.orchestra.run.smithi177.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-27T15:11:45.963 INFO:teuthology.orchestra.run.smithi177.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-27T15:11:45.963 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-27T15:11:45.963 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-27T15:11:45.963 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-27T15:11:45.963 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-27T15:11:45.963 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-27T15:11:45.964 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.964 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.964 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:45.964 INFO:teuthology.orchestra.run.smithi177.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-27T15:11:45.964 INFO:teuthology.orchestra.run.smithi177.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-27T15:11:45.964 INFO:teuthology.orchestra.run.smithi177.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-27T15:11:45.964 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-27T15:11:45.964 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-27T15:11:45.964 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-27T15:11:45.965 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-27T15:11:45.965 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2023-11-27T15:11:46.161 DEBUG:teuthology.parallel:result is None 2023-11-27T15:11:47.901 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:11:48.227 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:11:48.265 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-27T15:11:48.499 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 124/134 2023-11-27T15:11:48.707 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 125/134 2023-11-27T15:11:49.074 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 126/134 2023-11-27T15:11:49.416 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-27T15:11:49.597 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-27T15:11:49.621 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-27T15:11:49.688 INFO:teuthology.orchestra.run.smithi174.stdout:Installed products updated. 2023-11-27T15:11:49.863 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-27T15:11:49.900 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-27T15:11:49.900 INFO:teuthology.orchestra.run.smithi174.stdout:Upgraded: 2023-11-27T15:11:49.900 INFO:teuthology.orchestra.run.smithi174.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-27T15:11:49.900 INFO:teuthology.orchestra.run.smithi174.stdout: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.900 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.900 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2023-11-27T15:11:49.900 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-27T15:11:49.900 INFO:teuthology.orchestra.run.smithi174.stdout: automake-1.16.1-7.el8.noarch 2023-11-27T15:11:49.900 INFO:teuthology.orchestra.run.smithi174.stdout: bison-3.0.4-10.el8.x86_64 2023-11-27T15:11:49.901 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.901 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.901 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.901 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.901 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:11:49.901 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.901 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.901 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.901 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:11:49.901 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:11:49.901 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:11:49.902 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:11:49.902 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:11:49.902 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.902 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.902 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:11:49.902 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.902 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.902 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.902 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:11:49.902 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:11:49.902 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.902 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:11:49.903 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-27T15:11:49.903 INFO:teuthology.orchestra.run.smithi174.stdout: flex-2.6.1-9.el8.x86_64 2023-11-27T15:11:49.903 INFO:teuthology.orchestra.run.smithi174.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-27T15:11:49.903 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-27T15:11:49.903 INFO:teuthology.orchestra.run.smithi174.stdout: jq-1.6-3.el8.x86_64 2023-11-27T15:11:49.903 INFO:teuthology.orchestra.run.smithi174.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-27T15:11:49.903 INFO:teuthology.orchestra.run.smithi174.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-27T15:11:49.903 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-27T15:11:49.903 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.903 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.903 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.903 INFO:teuthology.orchestra.run.smithi174.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-27T15:11:49.904 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-27T15:11:49.904 INFO:teuthology.orchestra.run.smithi174.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-27T15:11:49.904 INFO:teuthology.orchestra.run.smithi174.stdout: libpq-13.5-1.el8.x86_64 2023-11-27T15:11:49.904 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-27T15:11:49.904 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-27T15:11:49.904 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.904 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.904 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-27T15:11:49.904 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.904 INFO:teuthology.orchestra.run.smithi174.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-27T15:11:49.904 INFO:teuthology.orchestra.run.smithi174.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-27T15:11:49.904 INFO:teuthology.orchestra.run.smithi174.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-27T15:11:49.904 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-27T15:11:49.905 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-27T15:11:49.905 INFO:teuthology.orchestra.run.smithi174.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-27T15:11:49.905 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-27T15:11:49.905 INFO:teuthology.orchestra.run.smithi174.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-27T15:11:49.905 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-27T15:11:49.905 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-27T15:11:49.905 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-27T15:11:49.905 INFO:teuthology.orchestra.run.smithi174.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-27T15:11:49.905 INFO:teuthology.orchestra.run.smithi174.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-27T15:11:49.905 INFO:teuthology.orchestra.run.smithi174.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-27T15:11:49.905 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:11:49.905 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:11:49.905 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:11:49.905 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-27T15:11:49.906 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-27T15:11:49.906 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-27T15:11:49.906 INFO:teuthology.orchestra.run.smithi174.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-27T15:11:49.906 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-27T15:11:49.906 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.906 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.906 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.906 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-27T15:11:49.906 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-27T15:11:49.906 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-27T15:11:49.906 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-27T15:11:49.906 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-27T15:11:49.907 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-27T15:11:49.907 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-27T15:11:49.907 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-27T15:11:49.907 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-27T15:11:49.907 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-27T15:11:49.907 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-27T15:11:49.907 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-27T15:11:49.907 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-27T15:11:49.907 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-27T15:11:49.907 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-27T15:11:49.907 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-27T15:11:49.907 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-27T15:11:49.907 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-27T15:11:49.908 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-27T15:11:49.908 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-27T15:11:49.908 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-27T15:11:49.908 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-27T15:11:49.908 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend-2.6-1.el8.noarch 2023-11-27T15:11:49.908 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-27T15:11:49.908 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-27T15:11:49.908 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-27T15:11:49.908 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-27T15:11:49.908 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-27T15:11:49.908 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-27T15:11:49.908 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.909 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.909 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-27T15:11:49.909 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-27T15:11:49.909 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.909 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-27T15:11:49.909 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-27T15:11:49.909 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-27T15:11:49.909 INFO:teuthology.orchestra.run.smithi174.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-27T15:11:49.909 INFO:teuthology.orchestra.run.smithi174.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-27T15:11:49.909 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-27T15:11:49.909 INFO:teuthology.orchestra.run.smithi174.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-27T15:11:49.909 INFO:teuthology.orchestra.run.smithi174.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-27T15:11:49.909 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-27T15:11:49.909 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-27T15:11:49.910 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-27T15:11:49.910 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-27T15:11:49.910 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-27T15:11:49.910 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.910 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.910 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:11:49.910 INFO:teuthology.orchestra.run.smithi174.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-27T15:11:49.910 INFO:teuthology.orchestra.run.smithi174.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-27T15:11:49.910 INFO:teuthology.orchestra.run.smithi174.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-27T15:11:49.910 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-27T15:11:49.910 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-27T15:11:49.911 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-27T15:11:49.911 INFO:teuthology.orchestra.run.smithi174.stdout: 2023-11-27T15:11:49.911 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2023-11-27T15:11:50.038 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-27T15:11:50.085 DEBUG:teuthology.parallel:result is None 2023-11-27T15:11:50.151 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-27T15:11:50.355 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-27T15:11:50.384 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-27T15:11:50.385 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:11:50.620 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:11:50.621 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-27T15:11:50.872 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-27T15:11:51.105 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:11:51.379 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:11:51.379 INFO:teuthology.orchestra.run.smithi133.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-27T15:11:51.380 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:11:51.998 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-2:18.0.0-3906.g1c956514.el8.x86_64 1/134 2023-11-27T15:11:51.999 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2/134 2023-11-27T15:11:51.999 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 3/134 2023-11-27T15:11:51.999 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 4/134 2023-11-27T15:11:51.999 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3906.g1c956 5/134 2023-11-27T15:11:51.999 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 6/134 2023-11-27T15:11:51.999 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 7/134 2023-11-27T15:11:51.999 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 8/134 2023-11-27T15:11:51.999 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 9/134 2023-11-27T15:11:51.999 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 10/134 2023-11-27T15:11:51.999 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 11/134 2023-11-27T15:11:51.999 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 12/134 2023-11-27T15:11:51.999 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 13/134 2023-11-27T15:11:51.999 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_ 14/134 2023-11-27T15:11:51.999 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 15/134 2023-11-27T15:11:52.000 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 16/134 2023-11-27T15:11:52.000 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados-devel-2:18.0.0-3906.g1c956514.el8.x86_6 17/134 2023-11-27T15:11:52.000 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86 18/134 2023-11-27T15:11:52.000 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 19/134 2023-11-27T15:11:52.000 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3906.g1c956514.el 20/134 2023-11-27T15:11:52.000 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-common-2:18.0.0-3906.g1c956514.el8. 21/134 2023-11-27T15:11:52.000 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_6 22/134 2023-11-27T15:11:52.000 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 23/134 2023-11-27T15:11:52.000 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 24/134 2023-11-27T15:11:52.000 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 25/134 2023-11-27T15:11:52.000 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 26/134 2023-11-27T15:11:52.000 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 27/134 2023-11-27T15:11:52.000 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 28/134 2023-11-27T15:11:52.000 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3906.g1c956514. 29/134 2023-11-27T15:11:52.009 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noa 30/134 2023-11-27T15:11:52.009 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.n 31/134 2023-11-27T15:11:52.009 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c9 32/134 2023-11-27T15:11:52.009 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el 33/134 2023-11-27T15:11:52.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 34/134 2023-11-27T15:11:52.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.e 35/134 2023-11-27T15:11:52.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 36/134 2023-11-27T15:11:52.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephadm-2:18.0.0-3906.g1c956514.el8.noarch 37/134 2023-11-27T15:11:52.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 38/134 2023-11-27T15:11:52.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-27T15:11:52.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-27T15:11:52.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-27T15:11:52.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-27T15:11:52.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-27T15:11:52.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-27T15:11:52.010 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-27T15:11:52.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-27T15:11:52.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-27T15:11:52.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-27T15:11:52.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-27T15:11:52.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-27T15:11:52.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-27T15:11:52.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-27T15:11:52.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-27T15:11:52.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-27T15:11:52.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-27T15:11:52.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-27T15:11:52.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-27T15:11:52.011 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-27T15:11:52.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-27T15:11:52.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-27T15:11:52.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-27T15:11:52.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-27T15:11:52.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-27T15:11:52.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-27T15:11:52.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-27T15:11:52.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-27T15:11:52.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-27T15:11:52.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-27T15:11:52.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-27T15:11:52.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-27T15:11:52.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-27T15:11:52.012 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-27T15:11:52.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-27T15:11:52.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-27T15:11:52.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-27T15:11:52.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-27T15:11:52.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-27T15:11:52.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-27T15:11:52.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-27T15:11:52.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-27T15:11:52.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-27T15:11:52.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-27T15:11:52.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-27T15:11:52.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-27T15:11:52.013 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-27T15:11:52.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-27T15:11:52.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-27T15:11:52.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-27T15:11:52.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-27T15:11:52.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-27T15:11:52.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-27T15:11:52.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-27T15:11:52.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-27T15:11:52.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-27T15:11:52.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-27T15:11:52.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-27T15:11:52.014 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-27T15:11:52.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-27T15:11:52.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-27T15:11:52.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-27T15:11:52.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-27T15:11:52.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-27T15:11:52.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-27T15:11:52.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-27T15:11:52.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-27T15:11:52.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-27T15:11:52.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-27T15:11:52.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-27T15:11:52.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-27T15:11:52.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-27T15:11:52.015 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-27T15:11:52.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-27T15:11:52.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-27T15:11:52.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-27T15:11:52.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-27T15:11:52.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-27T15:11:52.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-27T15:11:52.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-27T15:11:52.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-27T15:11:52.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-27T15:11:52.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-27T15:11:52.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-27T15:11:52.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-27T15:11:52.016 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-27T15:11:52.017 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-27T15:11:52.017 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-27T15:11:52.017 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-27T15:11:52.017 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-27T15:11:52.017 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:18.0.0-3906.g1c956514.el8.x86_64 129/134 2023-11-27T15:11:52.017 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-27T15:11:52.017 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 131/134 2023-11-27T15:11:52.017 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-27T15:11:52.017 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-27T15:11:54.775 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-27T15:11:54.775 INFO:teuthology.orchestra.run.smithi133.stdout:Last metadata expiration check: 0:02:08 ago on Mon 27 Nov 2023 03:09:46 PM UTC. 2023-11-27T15:12:01.159 INFO:teuthology.orchestra.run.smithi133.stdout:Installed products updated. 2023-11-27T15:12:01.379 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:12:01.379 INFO:teuthology.orchestra.run.smithi133.stdout:Upgraded: 2023-11-27T15:12:01.380 INFO:teuthology.orchestra.run.smithi133.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-27T15:12:01.380 INFO:teuthology.orchestra.run.smithi133.stdout: librados2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.380 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.380 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2023-11-27T15:12:01.380 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-27T15:12:01.380 INFO:teuthology.orchestra.run.smithi133.stdout: automake-1.16.1-7.el8.noarch 2023-11-27T15:12:01.380 INFO:teuthology.orchestra.run.smithi133.stdout: bison-3.0.4-10.el8.x86_64 2023-11-27T15:12:01.380 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.380 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.380 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.380 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.380 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:12:01.381 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.381 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.381 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.381 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:12:01.381 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:12:01.381 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:12:01.381 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:12:01.381 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:12:01.381 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.381 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.381 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:12:01.381 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.381 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.381 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.382 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:12:01.382 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:12:01.382 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.382 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top-2:18.0.0-3906.g1c956514.el8.noarch 2023-11-27T15:12:01.382 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-27T15:12:01.382 INFO:teuthology.orchestra.run.smithi133.stdout: flex-2.6.1-9.el8.x86_64 2023-11-27T15:12:01.382 INFO:teuthology.orchestra.run.smithi133.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-27T15:12:01.382 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-27T15:12:01.382 INFO:teuthology.orchestra.run.smithi133.stdout: jq-1.6-3.el8.x86_64 2023-11-27T15:12:01.382 INFO:teuthology.orchestra.run.smithi133.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-27T15:12:01.382 INFO:teuthology.orchestra.run.smithi133.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-27T15:12:01.383 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-27T15:12:01.383 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.383 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.383 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.383 INFO:teuthology.orchestra.run.smithi133.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-27T15:12:01.383 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-27T15:12:01.383 INFO:teuthology.orchestra.run.smithi133.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-27T15:12:01.383 INFO:teuthology.orchestra.run.smithi133.stdout: libpq-13.5-1.el8.x86_64 2023-11-27T15:12:01.383 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-27T15:12:01.383 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-27T15:12:01.383 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.383 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.383 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-27T15:12:01.384 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.384 INFO:teuthology.orchestra.run.smithi133.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-27T15:12:01.384 INFO:teuthology.orchestra.run.smithi133.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-27T15:12:01.384 INFO:teuthology.orchestra.run.smithi133.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-27T15:12:01.384 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-27T15:12:01.384 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-27T15:12:01.384 INFO:teuthology.orchestra.run.smithi133.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-27T15:12:01.384 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-27T15:12:01.384 INFO:teuthology.orchestra.run.smithi133.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-27T15:12:01.384 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-27T15:12:01.384 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-27T15:12:01.384 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-27T15:12:01.385 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-27T15:12:01.385 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-27T15:12:01.385 INFO:teuthology.orchestra.run.smithi133.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-27T15:12:01.385 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:12:01.385 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:12:01.385 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-27T15:12:01.385 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-27T15:12:01.385 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-27T15:12:01.385 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-27T15:12:01.385 INFO:teuthology.orchestra.run.smithi133.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-27T15:12:01.385 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-27T15:12:01.385 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.386 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.386 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.386 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-27T15:12:01.386 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-27T15:12:01.386 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-27T15:12:01.386 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-27T15:12:01.386 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-27T15:12:01.386 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-27T15:12:01.386 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-27T15:12:01.386 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-27T15:12:01.386 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-27T15:12:01.386 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-27T15:12:01.387 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-27T15:12:01.387 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-27T15:12:01.387 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-27T15:12:01.387 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-27T15:12:01.387 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-27T15:12:01.387 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-27T15:12:01.387 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-27T15:12:01.387 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-27T15:12:01.387 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-27T15:12:01.387 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-27T15:12:01.387 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-27T15:12:01.387 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-27T15:12:01.388 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend-2.6-1.el8.noarch 2023-11-27T15:12:01.388 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-27T15:12:01.388 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-27T15:12:01.388 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-27T15:12:01.388 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-27T15:12:01.388 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-27T15:12:01.388 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-27T15:12:01.388 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.388 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.388 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-27T15:12:01.388 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-27T15:12:01.388 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.388 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-27T15:12:01.389 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-27T15:12:01.389 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-27T15:12:01.389 INFO:teuthology.orchestra.run.smithi133.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-27T15:12:01.389 INFO:teuthology.orchestra.run.smithi133.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-27T15:12:01.389 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-27T15:12:01.389 INFO:teuthology.orchestra.run.smithi133.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-27T15:12:01.389 INFO:teuthology.orchestra.run.smithi133.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-27T15:12:01.389 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-27T15:12:01.389 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-27T15:12:01.389 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-27T15:12:01.389 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-27T15:12:01.389 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-27T15:12:01.390 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.390 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.390 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd-2:18.0.0-3906.g1c956514.el8.x86_64 2023-11-27T15:12:01.390 INFO:teuthology.orchestra.run.smithi133.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-27T15:12:01.390 INFO:teuthology.orchestra.run.smithi133.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-27T15:12:01.390 INFO:teuthology.orchestra.run.smithi133.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-27T15:12:01.390 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-27T15:12:01.390 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-27T15:12:01.390 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-27T15:12:01.391 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:12:01.391 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2023-11-27T15:12:01.675 DEBUG:teuthology.parallel:result is None 2023-11-27T15:12:01.675 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:12:01.817 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-27T15:12:01.872 INFO:teuthology.orchestra.run.smithi133.stdout:18.0.0-3906.g1c956514.el8 2023-11-27T15:12:01.872 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3906.g1c956514.el8 2023-11-27T15:12:01.872 INFO:teuthology.task.install:The correct ceph version 18.0.0-3906.g1c956514 is installed. 2023-11-27T15:12:01.873 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:12:02.020 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-27T15:12:02.048 INFO:teuthology.orchestra.run.smithi174.stdout:18.0.0-3906.g1c956514.el8 2023-11-27T15:12:02.049 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3906.g1c956514.el8 2023-11-27T15:12:02.049 INFO:teuthology.task.install:The correct ceph version 18.0.0-3906.g1c956514 is installed. 2023-11-27T15:12:02.049 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:12:02.186 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-27T15:12:02.213 INFO:teuthology.orchestra.run.smithi177.stdout:18.0.0-3906.g1c956514.el8 2023-11-27T15:12:02.214 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3906.g1c956514.el8 2023-11-27T15:12:02.214 INFO:teuthology.task.install:The correct ceph version 18.0.0-3906.g1c956514 is installed. 2023-11-27T15:12:02.214 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-27T15:12:02.214 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-11-27T15:12:02.214 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-27T15:12:02.303 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-27T15:12:02.303 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-27T15:12:02.351 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-27T15:12:02.351 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-27T15:12:02.399 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-27T15:12:02.399 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-11-27T15:12:02.399 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/daemon-helper 2023-11-27T15:12:02.467 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-27T15:12:02.574 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-27T15:12:02.574 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/daemon-helper 2023-11-27T15:12:02.606 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-27T15:12:02.677 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-27T15:12:02.677 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/daemon-helper 2023-11-27T15:12:02.710 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-27T15:12:02.780 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-27T15:12:02.780 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-11-27T15:12:02.781 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-27T15:12:02.848 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-27T15:12:02.956 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-27T15:12:02.956 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-27T15:12:02.988 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-27T15:12:03.060 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-27T15:12:03.060 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-27T15:12:03.091 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-27T15:12:03.161 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-27T15:12:03.162 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-11-27T15:12:03.162 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/stdin-killer 2023-11-27T15:12:03.229 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-27T15:12:03.339 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-27T15:12:03.339 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/stdin-killer 2023-11-27T15:12:03.370 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-27T15:12:03.441 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-27T15:12:03.441 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/stdin-killer 2023-11-27T15:12:03.473 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-27T15:12:03.544 INFO:teuthology.run_tasks:Running task cephadm... 2023-11-27T15:12:03.611 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-11-27T15:12:03.612 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:12:03.612 INFO:tasks.cephadm:Cluster fsid is 5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:12:03.612 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-11-27T15:12:03.612 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.133', 'mon.b': '172.21.15.174', 'mon.c': '172.21.15.177'} 2023-11-27T15:12:03.613 INFO:tasks.cephadm:First mon is mon.a on smithi133 2023-11-27T15:12:03.613 INFO:tasks.cephadm:First mgr is x 2023-11-27T15:12:03.613 INFO:tasks.cephadm:Normalizing hostnames... 2023-11-27T15:12:03.613 DEBUG:teuthology.orchestra.run.smithi133:> sudo hostname $(hostname -s) 2023-11-27T15:12:03.689 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname $(hostname -s) 2023-11-27T15:12:03.722 DEBUG:teuthology.orchestra.run.smithi177:> sudo hostname $(hostname -s) 2023-11-27T15:12:03.755 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-11-27T15:12:03.755 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:12:03.964 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1c9565146c8647d4e00eb6c7ae626dc192680e7b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/74215/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-3906-g1c956514', 'node_name': '172.21.2.6+braggi06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-3906.g1c956514'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-11-16 20:52:31.578811', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-choffman-62793', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-11-27T15:12:04.174 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-choffman-62793, sha1 1c9565146c8647d4e00eb6c7ae626dc192680e7b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=1c9565146c8647d4e00eb6c7ae626dc192680e7b 2023-11-27T15:12:04.174 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/x86_64/flavors/default/cephadm 2023-11-27T15:12:04.175 DEBUG:teuthology.orchestra.run.smithi133:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-11-27T15:12:05.372 INFO:teuthology.orchestra.run.smithi133.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Nov 27 15:12 /home/ubuntu/cephtest/cephadm 2023-11-27T15:12:05.372 DEBUG:teuthology.orchestra.run.smithi174:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-11-27T15:12:06.511 INFO:teuthology.orchestra.run.smithi174.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Nov 27 15:12 /home/ubuntu/cephtest/cephadm 2023-11-27T15:12:06.511 DEBUG:teuthology.orchestra.run.smithi177:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1c9565146c8647d4e00eb6c7ae626dc192680e7b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-11-27T15:12:07.346 INFO:teuthology.orchestra.run.smithi177.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Nov 27 15:12 /home/ubuntu/cephtest/cephadm 2023-11-27T15:12:07.347 DEBUG:teuthology.orchestra.run.smithi133:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-11-27T15:12:07.397 DEBUG:teuthology.orchestra.run.smithi174:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-11-27T15:12:07.419 DEBUG:teuthology.orchestra.run.smithi177:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-11-27T15:12:07.453 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b on all hosts... 2023-11-27T15:12:07.453 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b pull 2023-11-27T15:12:07.458 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b pull 2023-11-27T15:12:07.461 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b pull 2023-11-27T15:12:08.963 INFO:teuthology.orchestra.run.smithi174.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b... 2023-11-27T15:12:09.177 INFO:teuthology.orchestra.run.smithi177.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b... 2023-11-27T15:12:11.718 INFO:teuthology.orchestra.run.smithi133.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b... 2023-11-27T15:12:46.354 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2023-11-27T15:12:46.354 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_version": "ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev)", 2023-11-27T15:12:46.354 INFO:teuthology.orchestra.run.smithi174.stdout: "image_id": "ace6904b066bfa00ae45df809dc67d592e0953ee68f7eed3c36f7d4e23e52db2", 2023-11-27T15:12:46.354 INFO:teuthology.orchestra.run.smithi174.stdout: "repo_digests": [ 2023-11-27T15:12:46.354 INFO:teuthology.orchestra.run.smithi174.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cb1d293acfc56ca3305212d5789a9da86fb3fb7a4af319d5f427084581ab00ca" 2023-11-27T15:12:46.354 INFO:teuthology.orchestra.run.smithi174.stdout: ] 2023-11-27T15:12:46.354 INFO:teuthology.orchestra.run.smithi174.stdout:} 2023-11-27T15:12:48.013 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2023-11-27T15:12:48.013 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_version": "ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev)", 2023-11-27T15:12:48.013 INFO:teuthology.orchestra.run.smithi177.stdout: "image_id": "ace6904b066bfa00ae45df809dc67d592e0953ee68f7eed3c36f7d4e23e52db2", 2023-11-27T15:12:48.013 INFO:teuthology.orchestra.run.smithi177.stdout: "repo_digests": [ 2023-11-27T15:12:48.013 INFO:teuthology.orchestra.run.smithi177.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cb1d293acfc56ca3305212d5789a9da86fb3fb7a4af319d5f427084581ab00ca" 2023-11-27T15:12:48.013 INFO:teuthology.orchestra.run.smithi177.stdout: ] 2023-11-27T15:12:48.013 INFO:teuthology.orchestra.run.smithi177.stdout:} 2023-11-27T15:12:50.398 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:12:50.398 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev)", 2023-11-27T15:12:50.398 INFO:teuthology.orchestra.run.smithi133.stdout: "image_id": "ace6904b066bfa00ae45df809dc67d592e0953ee68f7eed3c36f7d4e23e52db2", 2023-11-27T15:12:50.398 INFO:teuthology.orchestra.run.smithi133.stdout: "repo_digests": [ 2023-11-27T15:12:50.398 INFO:teuthology.orchestra.run.smithi133.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:cb1d293acfc56ca3305212d5789a9da86fb3fb7a4af319d5f427084581ab00ca" 2023-11-27T15:12:50.399 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-11-27T15:12:50.399 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:12:50.423 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph 2023-11-27T15:12:50.495 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /etc/ceph 2023-11-27T15:12:50.532 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /etc/ceph 2023-11-27T15:12:50.569 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 777 /etc/ceph 2023-11-27T15:12:50.636 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 777 /etc/ceph 2023-11-27T15:12:50.665 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 777 /etc/ceph 2023-11-27T15:12:50.696 INFO:tasks.cephadm:Writing seed config... 2023-11-27T15:12:50.697 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-11-27T15:12:50.698 INFO:tasks.cephadm: override: [client] debug client = 20 2023-11-27T15:12:50.698 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-11-27T15:12:50.698 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-11-27T15:12:50.698 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-11-27T15:12:50.698 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-11-27T15:12:50.698 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-11-27T15:12:50.698 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-11-27T15:12:50.699 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-11-27T15:12:50.699 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-11-27T15:12:50.699 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-11-27T15:12:50.699 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-11-27T15:12:50.699 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-11-27T15:12:50.699 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-11-27T15:12:50.699 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-11-27T15:12:50.700 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-11-27T15:12:50.700 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-11-27T15:12:50.700 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-11-27T15:12:50.700 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-11-27T15:12:50.700 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-11-27T15:12:50.700 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-11-27T15:12:50.700 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-11-27T15:12:50.700 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-11-27T15:12:50.701 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-11-27T15:12:50.701 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-11-27T15:12:50.701 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-11-27T15:12:50.701 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-11-27T15:12:50.701 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-11-27T15:12:50.701 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-11-27T15:12:50.701 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-11-27T15:12:50.702 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-11-27T15:12:50.702 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-11-27T15:12:50.702 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-11-27T15:12:50.702 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-11-27T15:12:50.702 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-11-27T15:12:50.702 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-11-27T15:12:50.702 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-11-27T15:12:50.702 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-11-27T15:12:50.703 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-11-27T15:12:50.703 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-11-27T15:12:50.703 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-11-27T15:12:50.703 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-11-27T15:12:50.703 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-11-27T15:12:50.704 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-11-27T15:12:50.704 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-11-27T15:12:50.747 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 5227da4c-8d37-11ee-95a2-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-11-27T15:12:50.747 DEBUG:teuthology.orchestra.run.smithi133:mon.a> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mon.a.service 2023-11-27T15:12:50.791 DEBUG:teuthology.orchestra.run.smithi133:mgr.x> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mgr.x.service 2023-11-27T15:12:50.837 INFO:tasks.cephadm:Bootstrapping... 2023-11-27T15:12:50.837 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b -v bootstrap --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.133 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:12:50.860 INFO:journalctl@ceph.mon.a.smithi133.stdout:-- Logs begin at Mon 2023-11-27 14:54:33 UTC. -- 2023-11-27T15:12:50.906 INFO:journalctl@ceph.mgr.x.smithi133.stdout:-- Logs begin at Mon 2023-11-27 14:54:33 UTC. -- 2023-11-27T15:12:51.176 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2023-11-27T15:12:51.176 INFO:teuthology.orchestra.run.smithi133.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b', '-v', 'bootstrap', '--fsid', '5227da4c-8d37-11ee-95a2-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.133', '--skip-admin-label'] 2023-11-27T15:12:51.351 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stdout 4.1.1 2023-11-27T15:12:51.352 INFO:teuthology.orchestra.run.smithi133.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-11-27T15:12:51.352 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying podman|docker is present... 2023-11-27T15:12:51.523 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stdout 4.1.1 2023-11-27T15:12:51.524 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying lvm2 is present... 2023-11-27T15:12:51.524 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying time synchronization is in place... 2023-11-27T15:12:51.538 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-11-27T15:12:51.538 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-11-27T15:12:51.551 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-11-27T15:12:51.551 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2023-11-27T15:12:51.565 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout enabled 2023-11-27T15:12:51.578 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout active 2023-11-27T15:12:51.578 INFO:teuthology.orchestra.run.smithi133.stdout:Unit chronyd.service is enabled and running 2023-11-27T15:12:51.578 INFO:teuthology.orchestra.run.smithi133.stdout:Repeating the final host check... 2023-11-27T15:12:51.748 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stdout 4.1.1 2023-11-27T15:12:51.748 INFO:teuthology.orchestra.run.smithi133.stdout:podman (/bin/podman) version 4.1.1 is present 2023-11-27T15:12:51.749 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl is present 2023-11-27T15:12:51.749 INFO:teuthology.orchestra.run.smithi133.stdout:lvcreate is present 2023-11-27T15:12:51.761 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-11-27T15:12:51.761 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-11-27T15:12:51.774 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-11-27T15:12:51.774 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2023-11-27T15:12:51.788 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout enabled 2023-11-27T15:12:51.801 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout active 2023-11-27T15:12:51.801 INFO:teuthology.orchestra.run.smithi133.stdout:Unit chronyd.service is enabled and running 2023-11-27T15:12:51.801 INFO:teuthology.orchestra.run.smithi133.stdout:Host looks OK 2023-11-27T15:12:51.801 INFO:teuthology.orchestra.run.smithi133.stdout:Cluster fsid: 5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:12:51.801 INFO:teuthology.orchestra.run.smithi133.stdout:Acquiring lock 140158547405288 on /run/cephadm/5227da4c-8d37-11ee-95a2-87774f69a715.lock 2023-11-27T15:12:51.802 INFO:teuthology.orchestra.run.smithi133.stdout:Lock 140158547405288 acquired on /run/cephadm/5227da4c-8d37-11ee-95a2-87774f69a715.lock 2023-11-27T15:12:51.802 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying IP 172.21.15.133 port 3300 ... 2023-11-27T15:12:51.802 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying IP 172.21.15.133 port 6789 ... 2023-11-27T15:12:51.802 INFO:teuthology.orchestra.run.smithi133.stdout:Base mon IP(s) is [172.21.15.133:3300, 172.21.15.133:6789], mon addrv is [v2:172.21.15.133:3300,v1:172.21.15.133:6789] 2023-11-27T15:12:51.810 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-11-27T15:12:51.810 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.133 metric 100 2023-11-27T15:12:51.817 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-11-27T15:12:51.817 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-11-27T15:12:51.817 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-11-27T15:12:51.824 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-11-27T15:12:51.824 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-11-27T15:12:51.824 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-11-27T15:12:51.824 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-11-27T15:12:51.824 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout inet6 fe80::89cb:f695:7bf5:3338/64 scope link noprefixroute 2023-11-27T15:12:51.824 INFO:teuthology.orchestra.run.smithi133.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-11-27T15:12:51.825 INFO:teuthology.orchestra.run.smithi133.stdout:Mon IP `172.21.15.133` is in CIDR network `172.21.0.0/20` 2023-11-27T15:12:51.825 INFO:teuthology.orchestra.run.smithi133.stdout:Mon IP `172.21.15.133` is in CIDR network `172.21.0.0/20` 2023-11-27T15:12:51.825 INFO:teuthology.orchestra.run.smithi133.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-11-27T15:12:51.826 INFO:teuthology.orchestra.run.smithi133.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-11-27T15:12:51.827 INFO:teuthology.orchestra.run.smithi133.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b... 2023-11-27T15:12:52.605 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stdout ace6904b066bfa00ae45df809dc67d592e0953ee68f7eed3c36f7d4e23e52db2 2023-11-27T15:12:52.605 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b... 2023-11-27T15:12:52.605 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Getting image source signatures 2023-11-27T15:12:52.605 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Copying blob sha256:eb1dc24ed1b729f25bfe21673f4e5cc46a126932dc459841b1c31fbebaca605e 2023-11-27T15:12:52.605 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Copying blob sha256:837e66e3ff89804ed1bb42786f80e6c23534ff65fe48526d5007449f4e1f79b0 2023-11-27T15:12:52.605 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Copying config sha256:ace6904b066bfa00ae45df809dc67d592e0953ee68f7eed3c36f7d4e23e52db2 2023-11-27T15:12:52.606 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Writing manifest to image destination 2023-11-27T15:12:52.606 INFO:teuthology.orchestra.run.smithi133.stdout:/bin/podman: stderr Storing signatures 2023-11-27T15:12:53.483 INFO:teuthology.orchestra.run.smithi133.stdout:ceph: stdout ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev) 2023-11-27T15:12:53.484 INFO:teuthology.orchestra.run.smithi133.stdout:Ceph version: ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev) 2023-11-27T15:12:53.484 INFO:teuthology.orchestra.run.smithi133.stdout:Extracting ceph user uid/gid from container image... 2023-11-27T15:12:54.359 INFO:teuthology.orchestra.run.smithi133.stdout:stat: stdout 167 167 2023-11-27T15:12:54.359 INFO:teuthology.orchestra.run.smithi133.stdout:Creating initial keys... 2023-11-27T15:12:55.134 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph-authtool: stdout AQD2sWRl2ZXlLRAAJLhGysRyTQXXZm7pyoMQ+A== 2023-11-27T15:12:55.909 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph-authtool: stdout AQD3sWRlDXH2HhAAk/kj4KaAJBJy+8bWhQabCA== 2023-11-27T15:12:56.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph-authtool: stdout AQD4sWRlwYsJEhAAJjEHFySsVYvlA8g6cJ/mRA== 2023-11-27T15:12:56.787 INFO:teuthology.orchestra.run.smithi133.stdout:Creating initial monmap... 2023-11-27T15:12:58.973 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-11-27T15:12:58.973 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-11-27T15:12:58.973 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:12:58.973 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-11-27T15:12:58.973 INFO:teuthology.orchestra.run.smithi133.stdout:monmaptool for a [v2:172.21.15.133:3300,v1:172.21.15.133:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-11-27T15:12:58.973 INFO:teuthology.orchestra.run.smithi133.stdout:setting min_mon_release = pacific 2023-11-27T15:12:58.973 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: set fsid to 5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:12:58.973 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-11-27T15:12:58.973 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:12:58.973 INFO:teuthology.orchestra.run.smithi133.stdout:Creating mon... 2023-11-27T15:12:59.850 INFO:teuthology.orchestra.run.smithi133.stdout:create mon.a on 2023-11-27T15:13:00.066 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-11-27T15:13:00.258 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-11-27T15:13:00.468 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-5227da4c-8d37-11ee-95a2-87774f69a715.target → /etc/systemd/system/ceph-5227da4c-8d37-11ee-95a2-87774f69a715.target. 2023-11-27T15:13:00.468 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-5227da4c-8d37-11ee-95a2-87774f69a715.target → /etc/systemd/system/ceph-5227da4c-8d37-11ee-95a2-87774f69a715.target. 2023-11-27T15:13:00.753 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mon.a 2023-11-27T15:13:00.754 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mon.a.service: Unit ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mon.a.service not loaded. 2023-11-27T15:13:00.951 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5227da4c-8d37-11ee-95a2-87774f69a715.target.wants/ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mon.a.service → /etc/systemd/system/ceph-5227da4c-8d37-11ee-95a2-87774f69a715@.service. 2023-11-27T15:13:01.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:00 smithi133 systemd[1]: Starting Ceph mon.a for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:13:01.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:01 smithi133 podman[97313]: 2023-11-27T15:13:01.709 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:01 smithi133 bash[97313]: 9bcdf9d402ce6afc250eee38614d5038fefabff08058ccd9ef4f9e07ebdbf97f 2023-11-27T15:13:01.771 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-27T15:13:01.771 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout disabled 2023-11-27T15:13:01.787 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-27T15:13:01.787 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2023-11-27T15:13:01.787 INFO:teuthology.orchestra.run.smithi133.stdout:firewalld.service is not enabled 2023-11-27T15:13:01.787 INFO:teuthology.orchestra.run.smithi133.stdout:Not possible to enable service . firewalld.service is not available 2023-11-27T15:13:01.788 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mon to start... 2023-11-27T15:13:01.788 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mon... 2023-11-27T15:13:02.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:01 smithi133 ceph-mon[97353]: mkfs 5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:13:02.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:01 smithi133 ceph-mon[97353]: mon.a is new leader, mons a in quorum (ranks 0) 2023-11-27T15:13:02.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:01 smithi133 systemd[1]: Started Ceph mon.a for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:13:02.776 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout cluster: 2023-11-27T15:13:02.776 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout id: 5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:13:02.776 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-11-27T15:13:02.776 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-11-27T15:13:02.777 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout services: 2023-11-27T15:13:02.777 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.705626s) 2023-11-27T15:13:02.777 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-11-27T15:13:02.777 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-11-27T15:13:02.777 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-11-27T15:13:02.777 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout data: 2023-11-27T15:13:02.777 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-11-27T15:13:02.777 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-11-27T15:13:02.777 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-11-27T15:13:02.777 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout pgs: 2023-11-27T15:13:02.777 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-11-27T15:13:02.777 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.322+0000 7fb1a4e52700 1 Processor -- start 2023-11-27T15:13:02.777 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.323+0000 7fb1a4e52700 1 -- start start 2023-11-27T15:13:02.778 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.323+0000 7fb1a4e52700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1a0105910 0x7fb1a0105d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:02.778 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.323+0000 7fb1a4e52700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a0106260 con 0x7fb1a0105910 2023-11-27T15:13:02.778 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.324+0000 7fb19e59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1a0105910 0x7fb1a0105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:02.778 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.324+0000 7fb19e59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1a0105910 0x7fb1a0105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37420/0 (socket says 172.21.15.133:37420) 2023-11-27T15:13:02.778 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.324+0000 7fb19e59c700 1 -- 172.21.15.133:0/3813085138 learned_addr learned my addr 172.21.15.133:0/3813085138 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:02.778 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.324+0000 7fb19e59c700 1 -- 172.21.15.133:0/3813085138 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a01063a0 con 0x7fb1a0105910 2023-11-27T15:13:02.778 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.324+0000 7fb19e59c700 1 --2- 172.21.15.133:0/3813085138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1a0105910 0x7fb1a0105d20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb1900098a0 tx=0x7fb190009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f045d24d2ee95d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:02.778 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.324+0000 7fb19d59a700 1 -- 172.21.15.133:0/3813085138 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb190004160 con 0x7fb1a0105910 2023-11-27T15:13:02.779 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.325+0000 7fb19d59a700 1 -- 172.21.15.133:0/3813085138 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1900042c0 con 0x7fb1a0105910 2023-11-27T15:13:02.779 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.325+0000 7fb19d59a700 1 -- 172.21.15.133:0/3813085138 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1900045a0 con 0x7fb1a0105910 2023-11-27T15:13:02.779 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.325+0000 7fb1a4e52700 1 -- 172.21.15.133:0/3813085138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1a0105910 msgr2=0x7fb1a0105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:02.779 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.325+0000 7fb1a4e52700 1 --2- 172.21.15.133:0/3813085138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1a0105910 0x7fb1a0105d20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb1900098a0 tx=0x7fb190009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:02.779 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.325+0000 7fb1a4e52700 1 -- 172.21.15.133:0/3813085138 shutdown_connections 2023-11-27T15:13:02.779 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.325+0000 7fb1a4e52700 1 --2- 172.21.15.133:0/3813085138 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1a0105910 0x7fb1a0105d20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:02.779 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.325+0000 7fb1a4e52700 1 -- 172.21.15.133:0/3813085138 >> 172.21.15.133:0/3813085138 conn(0x7fb1a0100fa0 msgr2=0x7fb1a01033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:02.779 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.325+0000 7fb1a4e52700 1 -- 172.21.15.133:0/3813085138 shutdown_connections 2023-11-27T15:13:02.779 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.325+0000 7fb1a4e52700 1 -- 172.21.15.133:0/3813085138 wait complete. 2023-11-27T15:13:02.780 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.326+0000 7fb1a4e52700 1 Processor -- start 2023-11-27T15:13:02.780 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.327+0000 7fb1a4e52700 1 -- start start 2023-11-27T15:13:02.780 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.327+0000 7fb1a4e52700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1a0105910 0x7fb1a007bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:02.780 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.327+0000 7fb1a4e52700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a007c190 con 0x7fb1a0105910 2023-11-27T15:13:02.780 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.327+0000 7fb19e59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1a0105910 0x7fb1a007bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:02.780 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.327+0000 7fb19e59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1a0105910 0x7fb1a007bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:37434/0 (socket says 172.21.15.133:37434) 2023-11-27T15:13:02.780 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.327+0000 7fb19e59c700 1 -- 172.21.15.133:0/4270844130 learned_addr learned my addr 172.21.15.133:0/4270844130 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:02.780 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.328+0000 7fb19e59c700 1 -- 172.21.15.133:0/4270844130 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb190009550 con 0x7fb1a0105910 2023-11-27T15:13:02.780 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.328+0000 7fb19e59c700 1 --2- 172.21.15.133:0/4270844130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1a0105910 0x7fb1a007bc50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0106640 tx=0x7fb190004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:02.780 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.328+0000 7fb18f7fe700 1 -- 172.21.15.133:0/4270844130 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb190011630 con 0x7fb1a0105910 2023-11-27T15:13:02.780 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.328+0000 7fb18f7fe700 1 -- 172.21.15.133:0/4270844130 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb190011790 con 0x7fb1a0105910 2023-11-27T15:13:02.780 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.328+0000 7fb1a4e52700 1 -- 172.21.15.133:0/4270844130 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a0106e60 con 0x7fb1a0105910 2023-11-27T15:13:02.781 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.328+0000 7fb18f7fe700 1 -- 172.21.15.133:0/4270844130 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb190011a30 con 0x7fb1a0105910 2023-11-27T15:13:02.781 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.328+0000 7fb1a4e52700 1 -- 172.21.15.133:0/4270844130 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a0106cd0 con 0x7fb1a0105910 2023-11-27T15:13:02.781 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.328+0000 7fb18f7fe700 1 -- 172.21.15.133:0/4270844130 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb190022070 con 0x7fb1a0105910 2023-11-27T15:13:02.781 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.328+0000 7fb18f7fe700 1 -- 172.21.15.133:0/4270844130 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb19001c450 con 0x7fb1a0105910 2023-11-27T15:13:02.781 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.329+0000 7fb1a4e52700 1 -- 172.21.15.133:0/4270844130 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a0062120 con 0x7fb1a0105910 2023-11-27T15:13:02.781 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.332+0000 7fb18f7fe700 1 -- 172.21.15.133:0/4270844130 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fb19001b450 con 0x7fb1a0105910 2023-11-27T15:13:02.781 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.404+0000 7fb1a4e52700 1 -- 172.21.15.133:0/4270844130 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fb1a0191f40 con 0x7fb1a0105910 2023-11-27T15:13:02.781 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.404+0000 7fb18f7fe700 1 -- 172.21.15.133:0/4270844130 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fb19001b5f0 con 0x7fb1a0105910 2023-11-27T15:13:02.781 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.405+0000 7fb1a4e52700 1 -- 172.21.15.133:0/4270844130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1a0105910 msgr2=0x7fb1a007bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:02.782 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.405+0000 7fb1a4e52700 1 --2- 172.21.15.133:0/4270844130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1a0105910 0x7fb1a007bc50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0106640 tx=0x7fb190004750 comp rx=0 tx=0).stop 2023-11-27T15:13:02.782 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.405+0000 7fb1a4e52700 1 -- 172.21.15.133:0/4270844130 shutdown_connections 2023-11-27T15:13:02.782 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.405+0000 7fb1a4e52700 1 --2- 172.21.15.133:0/4270844130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1a0105910 0x7fb1a007bc50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:02.782 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.405+0000 7fb1a4e52700 1 -- 172.21.15.133:0/4270844130 >> 172.21.15.133:0/4270844130 conn(0x7fb1a0100fa0 msgr2=0x7fb1a018d370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:02.782 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.405+0000 7fb1a4e52700 1 -- 172.21.15.133:0/4270844130 shutdown_connections 2023-11-27T15:13:02.782 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:02.405+0000 7fb1a4e52700 1 -- 172.21.15.133:0/4270844130 wait complete. 2023-11-27T15:13:02.782 INFO:teuthology.orchestra.run.smithi133.stdout:mon is available 2023-11-27T15:13:02.782 INFO:teuthology.orchestra.run.smithi133.stdout:Assimilating anything we can from ceph.conf... 2023-11-27T15:13:03.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:02 smithi133 ceph-mon[97353]: mon.a is new leader, mons a in quorum (ranks 0) 2023-11-27T15:13:03.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:02 smithi133 ceph-mon[97353]: monmap e1: 1 mons at {a=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0]} removed_ranks: {} 2023-11-27T15:13:03.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:02 smithi133 ceph-mon[97353]: fsmap 2023-11-27T15:13:03.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:02 smithi133 ceph-mon[97353]: osdmap e1: 0 total, 0 up, 0 in 2023-11-27T15:13:03.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:02 smithi133 ceph-mon[97353]: mgrmap e1: no daemons active 2023-11-27T15:13:03.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:02 smithi133 ceph-mon[97353]: from='client.? 172.21.15.133:0/4270844130' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-11-27T15:13:03.952 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-11-27T15:13:03.958 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [global] 2023-11-27T15:13:03.959 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout fsid = 5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:13:03.959 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.133:3300,v1:172.21.15.133:6789] 2023-11-27T15:13:03.959 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-11-27T15:13:03.959 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-11-27T15:13:03.959 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-11-27T15:13:03.959 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-11-27T15:13:03.959 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-11-27T15:13:03.959 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [mgr] 2023-11-27T15:13:03.960 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-11-27T15:13:03.960 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-11-27T15:13:03.960 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [osd] 2023-11-27T15:13:03.960 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-11-27T15:13:03.960 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-11-27T15:13:03.960 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-11-27T15:13:03.961 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-11-27T15:13:03.961 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-11-27T15:13:03.961 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.303+0000 7f5484f6b700 1 Processor -- start 2023-11-27T15:13:03.961 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.304+0000 7f5484f6b700 1 -- start start 2023-11-27T15:13:03.961 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.304+0000 7f5484f6b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f548007bde0 0x7f548007a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:03.961 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.304+0000 7f5484f6b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548007a820 con 0x7f548007bde0 2023-11-27T15:13:03.962 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.305+0000 7f547e59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f548007bde0 0x7f548007a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:03.962 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.305+0000 7f547e59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f548007bde0 0x7f548007a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58226/0 (socket says 172.21.15.133:58226) 2023-11-27T15:13:03.962 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.305+0000 7f547e59c700 1 -- 172.21.15.133:0/3976591525 learned_addr learned my addr 172.21.15.133:0/3976591525 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:03.962 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.305+0000 7f547e59c700 1 -- 172.21.15.133:0/3976591525 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548007a960 con 0x7f548007bde0 2023-11-27T15:13:03.962 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.305+0000 7f547e59c700 1 --2- 172.21.15.133:0/3976591525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f548007bde0 0x7f548007a2e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f54700098a0 tx=0x7f5470009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e89eb7f4764fe0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:03.962 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.306+0000 7f547d59a700 1 -- 172.21.15.133:0/3976591525 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5470004160 con 0x7f548007bde0 2023-11-27T15:13:03.963 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.306+0000 7f547d59a700 1 -- 172.21.15.133:0/3976591525 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54700042c0 con 0x7f548007bde0 2023-11-27T15:13:03.963 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.306+0000 7f547d59a700 1 -- 172.21.15.133:0/3976591525 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54700045a0 con 0x7f548007bde0 2023-11-27T15:13:03.963 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.306+0000 7f5484f6b700 1 -- 172.21.15.133:0/3976591525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f548007bde0 msgr2=0x7f548007a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:03.963 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.306+0000 7f5484f6b700 1 --2- 172.21.15.133:0/3976591525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f548007bde0 0x7f548007a2e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f54700098a0 tx=0x7f5470009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:03.963 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.306+0000 7f5484f6b700 1 -- 172.21.15.133:0/3976591525 shutdown_connections 2023-11-27T15:13:03.963 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.306+0000 7f5484f6b700 1 --2- 172.21.15.133:0/3976591525 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f548007bde0 0x7f548007a2e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:03.964 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.306+0000 7f5484f6b700 1 -- 172.21.15.133:0/3976591525 >> 172.21.15.133:0/3976591525 conn(0x7f5480100fa0 msgr2=0x7f54801033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:03.964 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.306+0000 7f5484f6b700 1 -- 172.21.15.133:0/3976591525 shutdown_connections 2023-11-27T15:13:03.964 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.306+0000 7f5484f6b700 1 -- 172.21.15.133:0/3976591525 wait complete. 2023-11-27T15:13:03.964 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.307+0000 7f5484f6b700 1 Processor -- start 2023-11-27T15:13:03.964 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.308+0000 7f5484f6b700 1 -- start start 2023-11-27T15:13:03.964 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.308+0000 7f5484f6b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f548007bde0 0x7f548019afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:03.965 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.308+0000 7f5484f6b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548019b520 con 0x7f548007bde0 2023-11-27T15:13:03.965 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.308+0000 7f547e59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f548007bde0 0x7f548019afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:03.965 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.308+0000 7f547e59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f548007bde0 0x7f548019afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58236/0 (socket says 172.21.15.133:58236) 2023-11-27T15:13:03.965 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.308+0000 7f547e59c700 1 -- 172.21.15.133:0/373672864 learned_addr learned my addr 172.21.15.133:0/373672864 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:03.965 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.308+0000 7f547e59c700 1 -- 172.21.15.133:0/373672864 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5470009550 con 0x7f548007bde0 2023-11-27T15:13:03.965 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.309+0000 7f547e59c700 1 --2- 172.21.15.133:0/373672864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f548007bde0 0x7f548019afe0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5470004000 tx=0x7f5470004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:03.966 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.309+0000 7f546f7fe700 1 -- 172.21.15.133:0/373672864 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5470011630 con 0x7f548007bde0 2023-11-27T15:13:03.966 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.309+0000 7f546f7fe700 1 -- 172.21.15.133:0/373672864 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5470011790 con 0x7f548007bde0 2023-11-27T15:13:03.966 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.309+0000 7f5484f6b700 1 -- 172.21.15.133:0/373672864 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548007ae60 con 0x7f548007bde0 2023-11-27T15:13:03.966 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.309+0000 7f546f7fe700 1 -- 172.21.15.133:0/373672864 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5470011a30 con 0x7f548007bde0 2023-11-27T15:13:03.966 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.309+0000 7f5484f6b700 1 -- 172.21.15.133:0/373672864 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548007acd0 con 0x7f548007bde0 2023-11-27T15:13:03.966 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.309+0000 7f546f7fe700 1 -- 172.21.15.133:0/373672864 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5470022070 con 0x7f548007bde0 2023-11-27T15:13:03.966 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.309+0000 7f546f7fe700 1 -- 172.21.15.133:0/373672864 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f547001c450 con 0x7f548007bde0 2023-11-27T15:13:03.966 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.310+0000 7f5484f6b700 1 -- 172.21.15.133:0/373672864 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f548007aff0 con 0x7f548007bde0 2023-11-27T15:13:03.966 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.313+0000 7f546f7fe700 1 -- 172.21.15.133:0/373672864 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f547001b450 con 0x7f548007bde0 2023-11-27T15:13:03.966 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.381+0000 7f5484f6b700 1 -- 172.21.15.133:0/373672864 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f5480194e60 con 0x7f548007bde0 2023-11-27T15:13:03.967 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.386+0000 7f546f7fe700 1 -- 172.21.15.133:0/373672864 <== mon.0 v2:172.21.15.133:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f547001b5f0 con 0x7f548007bde0 2023-11-27T15:13:03.967 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.386+0000 7f546f7fe700 1 -- 172.21.15.133:0/373672864 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7f54700375d0 con 0x7f548007bde0 2023-11-27T15:13:03.967 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.387+0000 7f5484f6b700 1 -- 172.21.15.133:0/373672864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f548007bde0 msgr2=0x7f548019afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:03.967 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.387+0000 7f5484f6b700 1 --2- 172.21.15.133:0/373672864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f548007bde0 0x7f548019afe0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5470004000 tx=0x7f5470004750 comp rx=0 tx=0).stop 2023-11-27T15:13:03.967 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.387+0000 7f5484f6b700 1 -- 172.21.15.133:0/373672864 shutdown_connections 2023-11-27T15:13:03.967 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.387+0000 7f5484f6b700 1 --2- 172.21.15.133:0/373672864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f548007bde0 0x7f548019afe0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:03.967 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.387+0000 7f5484f6b700 1 -- 172.21.15.133:0/373672864 >> 172.21.15.133:0/373672864 conn(0x7f5480100fa0 msgr2=0x7f5480102800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:03.967 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.387+0000 7f5484f6b700 1 -- 172.21.15.133:0/373672864 shutdown_connections 2023-11-27T15:13:03.967 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:03.387+0000 7f5484f6b700 1 -- 172.21.15.133:0/373672864 wait complete. 2023-11-27T15:13:03.967 INFO:teuthology.orchestra.run.smithi133.stdout:Generating new minimal ceph.conf... 2023-11-27T15:13:04.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:03 smithi133 ceph-mon[97353]: from='client.? 172.21.15.133:0/373672864' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-11-27T15:13:04.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:03 smithi133 ceph-mon[97353]: from='client.? 172.21.15.133:0/373672864' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-11-27T15:13:05.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:04 smithi133 ceph-mon[97353]: from='client.? 172.21.15.133:0/1504149816' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:13:05.039 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.462+0000 7fb5b7a38700 1 Processor -- start 2023-11-27T15:13:05.039 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.462+0000 7fb5b7a38700 1 -- start start 2023-11-27T15:13:05.039 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.463+0000 7fb5b7a38700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b0105910 0x7fb5b0105d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:05.039 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.463+0000 7fb5b7a38700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b0106260 con 0x7fb5b0105910 2023-11-27T15:13:05.039 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.463+0000 7fb5b57d4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b0105910 0x7fb5b0105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:05.039 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.463+0000 7fb5b57d4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b0105910 0x7fb5b0105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58248/0 (socket says 172.21.15.133:58248) 2023-11-27T15:13:05.040 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.463+0000 7fb5b57d4700 1 -- 172.21.15.133:0/898049403 learned_addr learned my addr 172.21.15.133:0/898049403 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:05.040 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.463+0000 7fb5b57d4700 1 -- 172.21.15.133:0/898049403 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b01063a0 con 0x7fb5b0105910 2023-11-27T15:13:05.040 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.464+0000 7fb5b57d4700 1 --2- 172.21.15.133:0/898049403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b0105910 0x7fb5b0105d20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac0098a0 tx=0x7fb5ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2da425cc31f606 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:05.040 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.464+0000 7fb5a7fff700 1 -- 172.21.15.133:0/898049403 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5ac004160 con 0x7fb5b0105910 2023-11-27T15:13:05.040 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.464+0000 7fb5a7fff700 1 -- 172.21.15.133:0/898049403 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb5ac0042c0 con 0x7fb5b0105910 2023-11-27T15:13:05.040 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.464+0000 7fb5a7fff700 1 -- 172.21.15.133:0/898049403 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5ac011a00 con 0x7fb5b0105910 2023-11-27T15:13:05.040 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.464+0000 7fb5b7a38700 1 -- 172.21.15.133:0/898049403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b0105910 msgr2=0x7fb5b0105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:05.040 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.464+0000 7fb5b7a38700 1 --2- 172.21.15.133:0/898049403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b0105910 0x7fb5b0105d20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac0098a0 tx=0x7fb5ac009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:05.040 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.465+0000 7fb5b7a38700 1 -- 172.21.15.133:0/898049403 shutdown_connections 2023-11-27T15:13:05.040 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.465+0000 7fb5b7a38700 1 --2- 172.21.15.133:0/898049403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b0105910 0x7fb5b0105d20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:05.040 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.465+0000 7fb5b7a38700 1 -- 172.21.15.133:0/898049403 >> 172.21.15.133:0/898049403 conn(0x7fb5b0100fa0 msgr2=0x7fb5b01033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:05.041 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.465+0000 7fb5b7a38700 1 -- 172.21.15.133:0/898049403 shutdown_connections 2023-11-27T15:13:05.041 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.465+0000 7fb5b7a38700 1 -- 172.21.15.133:0/898049403 wait complete. 2023-11-27T15:13:05.041 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.466+0000 7fb5b7a38700 1 Processor -- start 2023-11-27T15:13:05.041 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.466+0000 7fb5b7a38700 1 -- start start 2023-11-27T15:13:05.041 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.466+0000 7fb5b7a38700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b0105910 0x7fb5b019b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:05.041 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.466+0000 7fb5b7a38700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b019bca0 con 0x7fb5b0105910 2023-11-27T15:13:05.041 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.466+0000 7fb5b57d4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b0105910 0x7fb5b019b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:05.041 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.466+0000 7fb5b57d4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b0105910 0x7fb5b019b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58260/0 (socket says 172.21.15.133:58260) 2023-11-27T15:13:05.041 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.466+0000 7fb5b57d4700 1 -- 172.21.15.133:0/1504149816 learned_addr learned my addr 172.21.15.133:0/1504149816 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:05.041 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.467+0000 7fb5b57d4700 1 -- 172.21.15.133:0/1504149816 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ac009550 con 0x7fb5b0105910 2023-11-27T15:13:05.041 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.467+0000 7fb5b57d4700 1 --2- 172.21.15.133:0/1504149816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b0105910 0x7fb5b019b760 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac006af0 tx=0x7fb5ac011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:05.041 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.467+0000 7fb5a67fc700 1 -- 172.21.15.133:0/1504149816 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5ac003a40 con 0x7fb5b0105910 2023-11-27T15:13:05.041 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.467+0000 7fb5a67fc700 1 -- 172.21.15.133:0/1504149816 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb5ac003ba0 con 0x7fb5b0105910 2023-11-27T15:13:05.041 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.467+0000 7fb5a67fc700 1 -- 172.21.15.133:0/1504149816 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5ac01a6e0 con 0x7fb5b0105910 2023-11-27T15:13:05.042 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.467+0000 7fb5b7a38700 1 -- 172.21.15.133:0/1504149816 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b0107960 con 0x7fb5b0105910 2023-11-27T15:13:05.042 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.467+0000 7fb5b7a38700 1 -- 172.21.15.133:0/1504149816 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b01077d0 con 0x7fb5b0105910 2023-11-27T15:13:05.042 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.468+0000 7fb5a67fc700 1 -- 172.21.15.133:0/1504149816 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb5ac01ab60 con 0x7fb5b0105910 2023-11-27T15:13:05.042 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.468+0000 7fb5a67fc700 1 -- 172.21.15.133:0/1504149816 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb5ac019700 con 0x7fb5b0105910 2023-11-27T15:13:05.042 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.469+0000 7fb5b7a38700 1 -- 172.21.15.133:0/1504149816 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5b0062120 con 0x7fb5b0105910 2023-11-27T15:13:05.042 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.471+0000 7fb5a67fc700 1 -- 172.21.15.133:0/1504149816 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fb5ac02dde0 con 0x7fb5b0105910 2023-11-27T15:13:05.042 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.541+0000 7fb5b7a38700 1 -- 172.21.15.133:0/1504149816 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fb5b019c1e0 con 0x7fb5b0105910 2023-11-27T15:13:05.042 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.541+0000 7fb5a67fc700 1 -- 172.21.15.133:0/1504149816 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7fb5ac024a60 con 0x7fb5b0105910 2023-11-27T15:13:05.042 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.542+0000 7fb5b7a38700 1 -- 172.21.15.133:0/1504149816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b0105910 msgr2=0x7fb5b019b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:05.042 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.542+0000 7fb5b7a38700 1 --2- 172.21.15.133:0/1504149816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b0105910 0x7fb5b019b760 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac006af0 tx=0x7fb5ac011ad0 comp rx=0 tx=0).stop 2023-11-27T15:13:05.042 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.542+0000 7fb5b7a38700 1 -- 172.21.15.133:0/1504149816 shutdown_connections 2023-11-27T15:13:05.042 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.542+0000 7fb5b7a38700 1 --2- 172.21.15.133:0/1504149816 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb5b0105910 0x7fb5b019b760 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:05.043 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.542+0000 7fb5b7a38700 1 -- 172.21.15.133:0/1504149816 >> 172.21.15.133:0/1504149816 conn(0x7fb5b0100fa0 msgr2=0x7fb5b0101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:05.043 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.542+0000 7fb5b7a38700 1 -- 172.21.15.133:0/1504149816 shutdown_connections 2023-11-27T15:13:05.043 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:04.543+0000 7fb5b7a38700 1 -- 172.21.15.133:0/1504149816 wait complete. 2023-11-27T15:13:05.043 INFO:teuthology.orchestra.run.smithi133.stdout:Restarting the monitor... 2023-11-27T15:13:05.498 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:05 smithi133 systemd[1]: Stopping Ceph mon.a for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:13:05.771 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:05 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mon-a[97329]: 2023-11-27T15:13:05.492+0000 7f9fa25eb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:13:05.771 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:05 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mon-a[97329]: 2023-11-27T15:13:05.492+0000 7f9fa25eb700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-11-27T15:13:06.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:05 smithi133 bash[97694]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mon-a 2023-11-27T15:13:06.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 systemd[1]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mon.a.service: Succeeded. 2023-11-27T15:13:06.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 systemd[1]: Stopped Ceph mon.a for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:13:06.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 systemd[1]: Starting Ceph mon.a for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:13:06.978 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 podman[97820]: 2023-11-27T15:13:06.978 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 bash[97820]: bbdb066cdec5db127b7ce1c0ead511cedbcbd4946f982ad4061fc446ba82785c 2023-11-27T15:13:06.978 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: set uid:gid to 167:167 (ceph:ceph) 2023-11-27T15:13:07.019 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev), process ceph-mon, pid 7 2023-11-27T15:13:07.019 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: pidfile_write: ignore empty --pid-file 2023-11-27T15:13:07.019 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: load: jerasure load: lrc 2023-11-27T15:13:07.019 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: RocksDB version: 7.9.2 2023-11-27T15:13:07.019 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Git sha 0 2023-11-27T15:13:07.019 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Compile date 2023-11-16 19:40:53 2023-11-27T15:13:07.019 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: DB SUMMARY 2023-11-27T15:13:07.019 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: DB Session ID: XM2DIX43CCVZUKUHNV59 2023-11-27T15:13:07.019 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: CURRENT file: CURRENT 2023-11-27T15:13:07.019 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: IDENTITY file: IDENTITY 2023-11-27T15:13:07.020 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-11-27T15:13:07.020 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-11-27T15:13:07.020 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 97738 ; 2023-11-27T15:13:07.020 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.error_if_exists: 0 2023-11-27T15:13:07.020 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.create_if_missing: 0 2023-11-27T15:13:07.020 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.paranoid_checks: 1 2023-11-27T15:13:07.020 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.flush_verify_memtable_count: 1 2023-11-27T15:13:07.020 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-11-27T15:13:07.020 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-11-27T15:13:07.020 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.env: 0x55cf22eb4720 2023-11-27T15:13:07.020 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.fs: PosixFileSystem 2023-11-27T15:13:07.020 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.info_log: 0x55cf253a13c0 2023-11-27T15:13:07.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_file_opening_threads: 16 2023-11-27T15:13:07.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.statistics: (nil) 2023-11-27T15:13:07.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.use_fsync: 0 2023-11-27T15:13:07.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_log_file_size: 0 2023-11-27T15:13:07.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-11-27T15:13:07.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.log_file_time_to_roll: 0 2023-11-27T15:13:07.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.keep_log_file_num: 1000 2023-11-27T15:13:07.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.recycle_log_file_num: 0 2023-11-27T15:13:07.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.allow_fallocate: 1 2023-11-27T15:13:07.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.allow_mmap_reads: 0 2023-11-27T15:13:07.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.allow_mmap_writes: 0 2023-11-27T15:13:07.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.use_direct_reads: 0 2023-11-27T15:13:07.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-11-27T15:13:07.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.create_missing_column_families: 0 2023-11-27T15:13:07.022 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.db_log_dir: 2023-11-27T15:13:07.022 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.wal_dir: 2023-11-27T15:13:07.022 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.table_cache_numshardbits: 6 2023-11-27T15:13:07.022 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.WAL_ttl_seconds: 0 2023-11-27T15:13:07.022 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.WAL_size_limit_MB: 0 2023-11-27T15:13:07.022 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-11-27T15:13:07.022 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-11-27T15:13:07.022 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.is_fd_close_on_exec: 1 2023-11-27T15:13:07.022 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.advise_random_on_open: 1 2023-11-27T15:13:07.022 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.db_write_buffer_size: 0 2023-11-27T15:13:07.022 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.write_buffer_manager: 0x55cf246123c0 2023-11-27T15:13:07.022 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-11-27T15:13:07.022 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-11-27T15:13:07.022 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.use_adaptive_mutex: 0 2023-11-27T15:13:07.023 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.rate_limiter: (nil) 2023-11-27T15:13:07.023 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-11-27T15:13:07.023 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.wal_recovery_mode: 2 2023-11-27T15:13:07.023 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.enable_thread_tracking: 0 2023-11-27T15:13:07.023 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.enable_pipelined_write: 0 2023-11-27T15:13:07.023 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.unordered_write: 0 2023-11-27T15:13:07.023 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-11-27T15:13:07.023 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-11-27T15:13:07.023 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-11-27T15:13:07.023 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-11-27T15:13:07.023 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.row_cache: None 2023-11-27T15:13:07.023 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.wal_filter: None 2023-11-27T15:13:07.023 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-11-27T15:13:07.023 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.allow_ingest_behind: 0 2023-11-27T15:13:07.023 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.two_write_queues: 0 2023-11-27T15:13:07.024 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.manual_wal_flush: 0 2023-11-27T15:13:07.024 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.wal_compression: 0 2023-11-27T15:13:07.024 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.atomic_flush: 0 2023-11-27T15:13:07.024 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-11-27T15:13:07.024 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.persist_stats_to_disk: 0 2023-11-27T15:13:07.024 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.write_dbid_to_manifest: 0 2023-11-27T15:13:07.024 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.log_readahead_size: 0 2023-11-27T15:13:07.024 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-11-27T15:13:07.024 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.best_efforts_recovery: 0 2023-11-27T15:13:07.024 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-11-27T15:13:07.024 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-11-27T15:13:07.024 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.allow_data_in_errors: 0 2023-11-27T15:13:07.024 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.db_host_id: __hostname__ 2023-11-27T15:13:07.025 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.enforce_single_del_contracts: true 2023-11-27T15:13:07.026 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_background_jobs: 2 2023-11-27T15:13:07.026 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_background_compactions: -1 2023-11-27T15:13:07.026 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_subcompactions: 1 2023-11-27T15:13:07.026 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-11-27T15:13:07.026 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-11-27T15:13:07.026 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.delayed_write_rate : 16777216 2023-11-27T15:13:07.026 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_total_wal_size: 0 2023-11-27T15:13:07.026 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-11-27T15:13:07.026 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.stats_dump_period_sec: 600 2023-11-27T15:13:07.026 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.stats_persist_period_sec: 600 2023-11-27T15:13:07.026 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-11-27T15:13:07.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_open_files: -1 2023-11-27T15:13:07.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.bytes_per_sync: 0 2023-11-27T15:13:07.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.wal_bytes_per_sync: 0 2023-11-27T15:13:07.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.strict_bytes_per_sync: 0 2023-11-27T15:13:07.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compaction_readahead_size: 0 2023-11-27T15:13:07.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_background_flushes: -1 2023-11-27T15:13:07.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Compression algorithms supported: 2023-11-27T15:13:07.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-11-27T15:13:07.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: kZSTD supported: 0 2023-11-27T15:13:07.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: kXpressCompression supported: 0 2023-11-27T15:13:07.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: kLZ4HCCompression supported: 1 2023-11-27T15:13:07.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: kZlibCompression supported: 1 2023-11-27T15:13:07.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: kSnappyCompression supported: 1 2023-11-27T15:13:07.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: kLZ4Compression supported: 1 2023-11-27T15:13:07.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: kBZip2Compression supported: 0 2023-11-27T15:13:07.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Fast CRC32 supported: Supported on x86 2023-11-27T15:13:07.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: DMutex implementation: pthread_mutex_t 2023-11-27T15:13:07.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-11-27T15:13:07.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-11-27T15:13:07.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-11-27T15:13:07.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.merge_operator: 2023-11-27T15:13:07.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compaction_filter: None 2023-11-27T15:13:07.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compaction_filter_factory: None 2023-11-27T15:13:07.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.sst_partitioner_factory: None 2023-11-27T15:13:07.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.memtable_factory: SkipListFactory 2023-11-27T15:13:07.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.table_factory: BlockBasedTable 2023-11-27T15:13:07.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55cf253a14e0) 2023-11-27T15:13:07.028 INFO:journalctl@ceph.mon.a.smithi133.stdout: cache_index_and_filter_blocks: 1 2023-11-27T15:13:07.029 INFO:journalctl@ceph.mon.a.smithi133.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-11-27T15:13:07.029 INFO:journalctl@ceph.mon.a.smithi133.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-11-27T15:13:07.029 INFO:journalctl@ceph.mon.a.smithi133.stdout: pin_top_level_index_and_filter: 1 2023-11-27T15:13:07.029 INFO:journalctl@ceph.mon.a.smithi133.stdout: index_type: 0 2023-11-27T15:13:07.029 INFO:journalctl@ceph.mon.a.smithi133.stdout: data_block_index_type: 0 2023-11-27T15:13:07.029 INFO:journalctl@ceph.mon.a.smithi133.stdout: index_shortening: 1 2023-11-27T15:13:07.029 INFO:journalctl@ceph.mon.a.smithi133.stdout: data_block_hash_table_util_ratio: 0.750000 2023-11-27T15:13:07.029 INFO:journalctl@ceph.mon.a.smithi133.stdout: checksum: 4 2023-11-27T15:13:07.029 INFO:journalctl@ceph.mon.a.smithi133.stdout: no_block_cache: 0 2023-11-27T15:13:07.029 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_cache: 0x55cf246b31f0 2023-11-27T15:13:07.029 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_cache_name: BinnedLRUCache 2023-11-27T15:13:07.029 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_cache_options: 2023-11-27T15:13:07.029 INFO:journalctl@ceph.mon.a.smithi133.stdout: capacity : 536870912 2023-11-27T15:13:07.029 INFO:journalctl@ceph.mon.a.smithi133.stdout: num_shard_bits : 4 2023-11-27T15:13:07.030 INFO:journalctl@ceph.mon.a.smithi133.stdout: strict_capacity_limit : 0 2023-11-27T15:13:07.030 INFO:journalctl@ceph.mon.a.smithi133.stdout: high_pri_pool_ratio: 0.000 2023-11-27T15:13:07.030 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_cache_compressed: (nil) 2023-11-27T15:13:07.030 INFO:journalctl@ceph.mon.a.smithi133.stdout: persistent_cache: (nil) 2023-11-27T15:13:07.030 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_size: 4096 2023-11-27T15:13:07.030 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_size_deviation: 10 2023-11-27T15:13:07.030 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_restart_interval: 16 2023-11-27T15:13:07.030 INFO:journalctl@ceph.mon.a.smithi133.stdout: index_block_restart_interval: 1 2023-11-27T15:13:07.030 INFO:journalctl@ceph.mon.a.smithi133.stdout: metadata_block_size: 4096 2023-11-27T15:13:07.030 INFO:journalctl@ceph.mon.a.smithi133.stdout: partition_filters: 0 2023-11-27T15:13:07.030 INFO:journalctl@ceph.mon.a.smithi133.stdout: use_delta_encoding: 1 2023-11-27T15:13:07.030 INFO:journalctl@ceph.mon.a.smithi133.stdout: filter_policy: bloomfilter 2023-11-27T15:13:07.030 INFO:journalctl@ceph.mon.a.smithi133.stdout: whole_key_filtering: 1 2023-11-27T15:13:07.030 INFO:journalctl@ceph.mon.a.smithi133.stdout: verify_compression: 0 2023-11-27T15:13:07.031 INFO:journalctl@ceph.mon.a.smithi133.stdout: read_amp_bytes_per_bit: 0 2023-11-27T15:13:07.031 INFO:journalctl@ceph.mon.a.smithi133.stdout: format_version: 5 2023-11-27T15:13:07.031 INFO:journalctl@ceph.mon.a.smithi133.stdout: enable_index_compression: 1 2023-11-27T15:13:07.031 INFO:journalctl@ceph.mon.a.smithi133.stdout: block_align: 0 2023-11-27T15:13:07.031 INFO:journalctl@ceph.mon.a.smithi133.stdout: max_auto_readahead_size: 262144 2023-11-27T15:13:07.031 INFO:journalctl@ceph.mon.a.smithi133.stdout: prepopulate_block_cache: 0 2023-11-27T15:13:07.031 INFO:journalctl@ceph.mon.a.smithi133.stdout: initial_auto_readahead_size: 8192 2023-11-27T15:13:07.031 INFO:journalctl@ceph.mon.a.smithi133.stdout: num_file_reads_for_auto_readahead: 2 2023-11-27T15:13:07.031 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.write_buffer_size: 33554432 2023-11-27T15:13:07.031 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_write_buffer_number: 2 2023-11-27T15:13:07.031 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compression: NoCompression 2023-11-27T15:13:07.031 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.bottommost_compression: Disabled 2023-11-27T15:13:07.031 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.prefix_extractor: nullptr 2023-11-27T15:13:07.031 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-11-27T15:13:07.032 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.num_levels: 7 2023-11-27T15:13:07.032 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-11-27T15:13:07.032 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-11-27T15:13:07.032 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-11-27T15:13:07.032 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-11-27T15:13:07.032 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-11-27T15:13:07.032 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-11-27T15:13:07.032 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-11-27T15:13:07.032 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-11-27T15:13:07.032 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-11-27T15:13:07.032 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-11-27T15:13:07.032 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-11-27T15:13:07.033 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-11-27T15:13:07.033 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compression_opts.window_bits: -14 2023-11-27T15:13:07.033 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compression_opts.level: 32767 2023-11-27T15:13:07.033 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compression_opts.strategy: 0 2023-11-27T15:13:07.033 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-11-27T15:13:07.033 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-11-27T15:13:07.033 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-11-27T15:13:07.033 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-11-27T15:13:07.033 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compression_opts.enabled: false 2023-11-27T15:13:07.033 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-11-27T15:13:07.033 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-11-27T15:13:07.033 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-11-27T15:13:07.033 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-11-27T15:13:07.034 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.target_file_size_base: 67108864 2023-11-27T15:13:07.034 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.target_file_size_multiplier: 1 2023-11-27T15:13:07.034 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-11-27T15:13:07.034 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-11-27T15:13:07.034 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-11-27T15:13:07.034 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-11-27T15:13:07.034 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-11-27T15:13:07.034 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-11-27T15:13:07.034 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-11-27T15:13:07.034 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-11-27T15:13:07.034 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-11-27T15:13:07.034 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-11-27T15:13:07.034 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-11-27T15:13:07.035 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-11-27T15:13:07.035 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-11-27T15:13:07.035 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.arena_block_size: 1048576 2023-11-27T15:13:07.035 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-11-27T15:13:07.035 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-11-27T15:13:07.035 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.disable_auto_compactions: 0 2023-11-27T15:13:07.035 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-11-27T15:13:07.035 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-11-27T15:13:07.035 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-11-27T15:13:07.035 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-11-27T15:13:07.035 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-11-27T15:13:07.035 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-11-27T15:13:07.036 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-11-27T15:13:07.036 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-11-27T15:13:07.036 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-11-27T15:13:07.036 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-11-27T15:13:07.036 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.table_properties_collectors: 2023-11-27T15:13:07.036 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.inplace_update_support: 0 2023-11-27T15:13:07.036 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.inplace_update_num_locks: 10000 2023-11-27T15:13:07.036 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-11-27T15:13:07.036 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-11-27T15:13:07.036 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.memtable_huge_page_size: 0 2023-11-27T15:13:07.036 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.bloom_locality: 0 2023-11-27T15:13:07.036 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.max_successive_merges: 0 2023-11-27T15:13:07.037 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.optimize_filters_for_hits: 0 2023-11-27T15:13:07.037 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.paranoid_file_checks: 0 2023-11-27T15:13:07.037 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.force_consistency_checks: 1 2023-11-27T15:13:07.037 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.report_bg_io_stats: 0 2023-11-27T15:13:07.037 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.ttl: 2592000 2023-11-27T15:13:07.037 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.periodic_compaction_seconds: 0 2023-11-27T15:13:07.037 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-11-27T15:13:07.037 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-11-27T15:13:07.037 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.enable_blob_files: false 2023-11-27T15:13:07.037 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.min_blob_size: 0 2023-11-27T15:13:07.037 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.blob_file_size: 268435456 2023-11-27T15:13:07.037 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.blob_compression_type: NoCompression 2023-11-27T15:13:07.037 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.enable_blob_garbage_collection: false 2023-11-27T15:13:07.038 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-11-27T15:13:07.038 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-11-27T15:13:07.038 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-11-27T15:13:07.038 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.blob_file_starting_level: 0 2023-11-27T15:13:07.038 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-11-27T15:13:07.038 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-11-27T15:13:07.038 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-11-27T15:13:07.038 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c9203b00-bb22-4c98-a607-df8be69bc3fd 2023-11-27T15:13:07.038 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701097986982260, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-11-27T15:13:07.038 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-11-27T15:13:07.038 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701097986983881, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93046, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 310, "table_properties": {"data_size": 91068, "index_size": 260, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13416, "raw_average_key_size": 49, "raw_value_size": 83760, "raw_average_value_size": 311, "num_data_blocks": 11, "num_entries": 269, "num_filter_entries": 269, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1701097986, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c9203b00-bb22-4c98-a607-df8be69bc3fd", "db_session_id": "XM2DIX43CCVZUKUHNV59", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-11-27T15:13:07.038 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701097986984023, "job": 1, "event": "recovery_finished"} 2023-11-27T15:13:07.039 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-11-27T15:13:07.039 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-11-27T15:13:07.039 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55cf24750000 2023-11-27T15:13:07.039 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: DB pointer 0x55cf2473c000 2023-11-27T15:13:07.039 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-11-27T15:13:07.039 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-11-27T15:13:07.039 INFO:journalctl@ceph.mon.a.smithi133.stdout: ** DB Stats ** 2023-11-27T15:13:07.039 INFO:journalctl@ceph.mon.a.smithi133.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-27T15:13:07.039 INFO:journalctl@ceph.mon.a.smithi133.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-11-27T15:13:07.039 INFO:journalctl@ceph.mon.a.smithi133.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-27T15:13:07.039 INFO:journalctl@ceph.mon.a.smithi133.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-27T15:13:07.039 INFO:journalctl@ceph.mon.a.smithi133.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-11-27T15:13:07.040 INFO:journalctl@ceph.mon.a.smithi133.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-27T15:13:07.040 INFO:journalctl@ceph.mon.a.smithi133.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-27T15:13:07.040 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2023-11-27T15:13:07.040 INFO:journalctl@ceph.mon.a.smithi133.stdout: ** Compaction Stats [default] ** 2023-11-27T15:13:07.040 INFO:journalctl@ceph.mon.a.smithi133.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-27T15:13:07.040 INFO:journalctl@ceph.mon.a.smithi133.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-11-27T15:13:07.040 INFO:journalctl@ceph.mon.a.smithi133.stdout: L0 2/0 92.70 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 89.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-27T15:13:07.040 INFO:journalctl@ceph.mon.a.smithi133.stdout: Sum 2/0 92.70 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 89.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-27T15:13:07.040 INFO:journalctl@ceph.mon.a.smithi133.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 89.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-27T15:13:07.040 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2023-11-27T15:13:07.040 INFO:journalctl@ceph.mon.a.smithi133.stdout: ** Compaction Stats [default] ** 2023-11-27T15:13:07.040 INFO:journalctl@ceph.mon.a.smithi133.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-27T15:13:07.040 INFO:journalctl@ceph.mon.a.smithi133.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-11-27T15:13:07.041 INFO:journalctl@ceph.mon.a.smithi133.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 89.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-11-27T15:13:07.041 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2023-11-27T15:13:07.041 INFO:journalctl@ceph.mon.a.smithi133.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-11-27T15:13:07.041 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2023-11-27T15:13:07.041 INFO:journalctl@ceph.mon.a.smithi133.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-27T15:13:07.041 INFO:journalctl@ceph.mon.a.smithi133.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-11-27T15:13:07.041 INFO:journalctl@ceph.mon.a.smithi133.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-11-27T15:13:07.041 INFO:journalctl@ceph.mon.a.smithi133.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-11-27T15:13:07.041 INFO:journalctl@ceph.mon.a.smithi133.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-11-27T15:13:07.041 INFO:journalctl@ceph.mon.a.smithi133.stdout: AddFile(Keys): cumulative 0, interval 0 2023-11-27T15:13:07.041 INFO:journalctl@ceph.mon.a.smithi133.stdout: Cumulative compaction: 0.00 GB write, 15.96 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-27T15:13:07.041 INFO:journalctl@ceph.mon.a.smithi133.stdout: Interval compaction: 0.00 GB write, 15.96 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-27T15:13:07.042 INFO:journalctl@ceph.mon.a.smithi133.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-11-27T15:13:07.042 INFO:journalctl@ceph.mon.a.smithi133.stdout: Block cache BinnedLRUCache@0x55cf246b31f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-11-27T15:13:07.042 INFO:journalctl@ceph.mon.a.smithi133.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-11-27T15:13:07.042 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2023-11-27T15:13:07.042 INFO:journalctl@ceph.mon.a.smithi133.stdout: ** File Read Latency Histogram By Level [default] ** 2023-11-27T15:13:07.042 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: starting mon.a rank 0 at public addrs [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] at bind addrs [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:13:07.042 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: mon.a@-1(???) e1 preinit fsid 5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:13:07.042 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: mon.a@-1(???).mds e1 new map 2023-11-27T15:13:07.042 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: mon.a@-1(???).mds e1 print_map 2023-11-27T15:13:07.042 INFO:journalctl@ceph.mon.a.smithi133.stdout: e1 2023-11-27T15:13:07.042 INFO:journalctl@ceph.mon.a.smithi133.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-11-27T15:13:07.042 INFO:journalctl@ceph.mon.a.smithi133.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-27T15:13:07.042 INFO:journalctl@ceph.mon.a.smithi133.stdout: legacy client fscid: -1 2023-11-27T15:13:07.043 INFO:journalctl@ceph.mon.a.smithi133.stdout: 2023-11-27T15:13:07.043 INFO:journalctl@ceph.mon.a.smithi133.stdout: No filesystems configured 2023-11-27T15:13:07.043 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-11-27T15:13:07.043 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:13:07.043 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:13:07.043 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:13:07.043 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-11-27T15:13:07.043 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: expand_channel_meta expand map: {default=false} 2023-11-27T15:13:07.043 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: expand_channel_meta from 'false' to 'false' 2023-11-27T15:13:07.043 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: expand_channel_meta expanded map: {default=false} 2023-11-27T15:13:07.043 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: expand_channel_meta expand map: {default=info} 2023-11-27T15:13:07.043 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: expand_channel_meta from 'info' to 'info' 2023-11-27T15:13:07.043 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: expand_channel_meta expanded map: {default=info} 2023-11-27T15:13:07.043 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: expand_channel_meta expand map: {default=daemon} 2023-11-27T15:13:07.044 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: expand_channel_meta from 'daemon' to 'daemon' 2023-11-27T15:13:07.044 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: expand_channel_meta expanded map: {default=daemon} 2023-11-27T15:13:07.044 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: expand_channel_meta expand map: {default=debug} 2023-11-27T15:13:07.044 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: expand_channel_meta from 'debug' to 'debug' 2023-11-27T15:13:07.044 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:06 smithi133 ceph-mon[97860]: expand_channel_meta expanded map: {default=debug} 2023-11-27T15:13:07.044 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:07 smithi133 systemd[1]: Started Ceph mon.a for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:13:07.044 INFO:teuthology.orchestra.run.smithi133.stdout:Setting mon public_network to 172.21.0.0/20 2023-11-27T15:13:07.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:07 smithi133 ceph-mon[97860]: mon.a is new leader, mons a in quorum (ranks 0) 2023-11-27T15:13:07.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:07 smithi133 ceph-mon[97860]: monmap e1: 1 mons at {a=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0]} removed_ranks: {} 2023-11-27T15:13:07.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:07 smithi133 ceph-mon[97860]: fsmap 2023-11-27T15:13:07.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:07 smithi133 ceph-mon[97860]: osdmap e1: 0 total, 0 up, 0 in 2023-11-27T15:13:07.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:07 smithi133 ceph-mon[97860]: mgrmap e1: no daemons active 2023-11-27T15:13:08.104 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.558+0000 7fb969659700 1 Processor -- start 2023-11-27T15:13:08.109 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.558+0000 7fb969659700 1 -- start start 2023-11-27T15:13:08.109 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.559+0000 7fb969659700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9641047f0 0x7fb964104c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:08.109 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.559+0000 7fb969659700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb964105140 con 0x7fb9641047f0 2023-11-27T15:13:08.109 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.559+0000 7fb962ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9641047f0 0x7fb964104c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:08.109 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.559+0000 7fb962ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9641047f0 0x7fb964104c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58266/0 (socket says 172.21.15.133:58266) 2023-11-27T15:13:08.109 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.559+0000 7fb962ffd700 1 -- 172.21.15.133:0/3649061701 learned_addr learned my addr 172.21.15.133:0/3649061701 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:08.109 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.560+0000 7fb962ffd700 1 -- 172.21.15.133:0/3649061701 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb964105280 con 0x7fb9641047f0 2023-11-27T15:13:08.110 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.560+0000 7fb962ffd700 1 --2- 172.21.15.133:0/3649061701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9641047f0 0x7fb964104c00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb9540098a0 tx=0x7fb954009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22267d89915e4c93 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:08.110 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.560+0000 7fb961ffb700 1 -- 172.21.15.133:0/3649061701 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb954004160 con 0x7fb9641047f0 2023-11-27T15:13:08.110 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.560+0000 7fb961ffb700 1 -- 172.21.15.133:0/3649061701 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb9540042c0 con 0x7fb9641047f0 2023-11-27T15:13:08.110 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.560+0000 7fb961ffb700 1 -- 172.21.15.133:0/3649061701 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb954011a00 con 0x7fb9641047f0 2023-11-27T15:13:08.110 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.561+0000 7fb969659700 1 -- 172.21.15.133:0/3649061701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9641047f0 msgr2=0x7fb964104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:08.110 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.561+0000 7fb969659700 1 --2- 172.21.15.133:0/3649061701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9641047f0 0x7fb964104c00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb9540098a0 tx=0x7fb954009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:08.110 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.561+0000 7fb969659700 1 -- 172.21.15.133:0/3649061701 shutdown_connections 2023-11-27T15:13:08.110 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.561+0000 7fb969659700 1 --2- 172.21.15.133:0/3649061701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9641047f0 0x7fb964104c00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:08.110 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.561+0000 7fb969659700 1 -- 172.21.15.133:0/3649061701 >> 172.21.15.133:0/3649061701 conn(0x7fb9640ffe40 msgr2=0x7fb964102270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:08.110 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.561+0000 7fb969659700 1 -- 172.21.15.133:0/3649061701 shutdown_connections 2023-11-27T15:13:08.110 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.561+0000 7fb969659700 1 -- 172.21.15.133:0/3649061701 wait complete. 2023-11-27T15:13:08.110 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.562+0000 7fb969659700 1 Processor -- start 2023-11-27T15:13:08.110 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.562+0000 7fb969659700 1 -- start start 2023-11-27T15:13:08.111 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.562+0000 7fb969659700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9641047f0 0x7fb96418fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:08.111 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.562+0000 7fb969659700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96418ff40 con 0x7fb9641047f0 2023-11-27T15:13:08.111 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.563+0000 7fb962ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9641047f0 0x7fb96418fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:08.111 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.563+0000 7fb962ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9641047f0 0x7fb96418fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58280/0 (socket says 172.21.15.133:58280) 2023-11-27T15:13:08.111 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.563+0000 7fb962ffd700 1 -- 172.21.15.133:0/2060684879 learned_addr learned my addr 172.21.15.133:0/2060684879 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:08.111 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.563+0000 7fb962ffd700 1 -- 172.21.15.133:0/2060684879 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb954009550 con 0x7fb9641047f0 2023-11-27T15:13:08.111 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.563+0000 7fb962ffd700 1 --2- 172.21.15.133:0/2060684879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9641047f0 0x7fb96418fa00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb954006af0 tx=0x7fb954011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:08.111 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.563+0000 7fb94bfff700 1 -- 172.21.15.133:0/2060684879 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb954003a40 con 0x7fb9641047f0 2023-11-27T15:13:08.111 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.563+0000 7fb94bfff700 1 -- 172.21.15.133:0/2060684879 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb954003ba0 con 0x7fb9641047f0 2023-11-27T15:13:08.111 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.563+0000 7fb969659700 1 -- 172.21.15.133:0/2060684879 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb964106840 con 0x7fb9641047f0 2023-11-27T15:13:08.111 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.563+0000 7fb94bfff700 1 -- 172.21.15.133:0/2060684879 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb95401a6e0 con 0x7fb9641047f0 2023-11-27T15:13:08.111 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.564+0000 7fb969659700 1 -- 172.21.15.133:0/2060684879 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9641066b0 con 0x7fb9641047f0 2023-11-27T15:13:08.111 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.564+0000 7fb94bfff700 1 -- 172.21.15.133:0/2060684879 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb95401ac50 con 0x7fb9641047f0 2023-11-27T15:13:08.112 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.564+0000 7fb94bfff700 1 -- 172.21.15.133:0/2060684879 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb954019700 con 0x7fb9641047f0 2023-11-27T15:13:08.112 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.565+0000 7fb969659700 1 -- 172.21.15.133:0/2060684879 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb964062120 con 0x7fb9641047f0 2023-11-27T15:13:08.112 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.568+0000 7fb94bfff700 1 -- 172.21.15.133:0/2060684879 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fb95402de30 con 0x7fb9641047f0 2023-11-27T15:13:08.112 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.637+0000 7fb969659700 1 -- 172.21.15.133:0/2060684879 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fb96418e970 con 0x7fb9641047f0 2023-11-27T15:13:08.112 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.639+0000 7fb94bfff700 1 -- 172.21.15.133:0/2060684879 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fb954024a60 con 0x7fb9641047f0 2023-11-27T15:13:08.112 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.640+0000 7fb969659700 1 -- 172.21.15.133:0/2060684879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9641047f0 msgr2=0x7fb96418fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:08.112 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.640+0000 7fb969659700 1 --2- 172.21.15.133:0/2060684879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9641047f0 0x7fb96418fa00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb954006af0 tx=0x7fb954011ad0 comp rx=0 tx=0).stop 2023-11-27T15:13:08.112 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.640+0000 7fb969659700 1 -- 172.21.15.133:0/2060684879 shutdown_connections 2023-11-27T15:13:08.112 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.640+0000 7fb969659700 1 --2- 172.21.15.133:0/2060684879 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9641047f0 0x7fb96418fa00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:08.112 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.640+0000 7fb969659700 1 -- 172.21.15.133:0/2060684879 >> 172.21.15.133:0/2060684879 conn(0x7fb9640ffe40 msgr2=0x7fb9640798c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:08.112 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.640+0000 7fb969659700 1 -- 172.21.15.133:0/2060684879 shutdown_connections 2023-11-27T15:13:08.112 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:07.640+0000 7fb969659700 1 -- 172.21.15.133:0/2060684879 wait complete. 2023-11-27T15:13:08.113 INFO:teuthology.orchestra.run.smithi133.stdout:Wrote config to /etc/ceph/ceph.conf 2023-11-27T15:13:08.113 INFO:teuthology.orchestra.run.smithi133.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:13:08.113 INFO:teuthology.orchestra.run.smithi133.stdout:Creating mgr... 2023-11-27T15:13:08.113 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying port 9283 ... 2023-11-27T15:13:08.113 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying port 8765 ... 2023-11-27T15:13:08.389 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mgr.x 2023-11-27T15:13:08.389 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mgr.x.service: Unit ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mgr.x.service not loaded. 2023-11-27T15:13:08.586 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5227da4c-8d37-11ee-95a2-87774f69a715.target.wants/ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-5227da4c-8d37-11ee-95a2-87774f69a715@.service. 2023-11-27T15:13:08.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:08 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2060684879' entity='client.admin' 2023-11-27T15:13:08.778 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:08 smithi133 systemd[1]: Starting Ceph mgr.x for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:13:09.385 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:09 smithi133 podman[98090]: 2023-11-27T15:13:09.385 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:09 smithi133 bash[98090]: c47377972a0e3df4cc85e1e8e55ff244456650e554104bc6f2b070a491ee4c13 2023-11-27T15:13:09.408 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-27T15:13:09.408 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout disabled 2023-11-27T15:13:09.425 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-27T15:13:09.425 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2023-11-27T15:13:09.425 INFO:teuthology.orchestra.run.smithi133.stdout:firewalld.service is not enabled 2023-11-27T15:13:09.425 INFO:teuthology.orchestra.run.smithi133.stdout:Not possible to enable service . firewalld.service is not available 2023-11-27T15:13:09.443 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-27T15:13:09.443 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout disabled 2023-11-27T15:13:09.459 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-27T15:13:09.460 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2023-11-27T15:13:09.460 INFO:teuthology.orchestra.run.smithi133.stdout:firewalld.service is not enabled 2023-11-27T15:13:09.460 INFO:teuthology.orchestra.run.smithi133.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-11-27T15:13:09.460 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mgr to start... 2023-11-27T15:13:09.460 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mgr... 2023-11-27T15:13:09.749 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:09 smithi133 systemd[1]: Started Ceph mgr.x for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:13:09.749 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:09 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:09.519+0000 7f62f5084200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-27T15:13:09.750 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:09 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:09.625+0000 7f62f5084200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-27T15:13:09.750 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:09 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:09.744+0000 7f62f5084200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-27T15:13:10.028 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:09 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:09.874+0000 7f62f5084200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-27T15:13:10.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:10 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2054785813' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-27T15:13:10.528 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:10 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:10.118+0000 7f62f5084200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-27T15:13:10.654 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-11-27T15:13:10.654 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:13:10.654 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsid": "5227da4c-8d37-11ee-95a2-87774f69a715", 2023-11-27T15:13:10.654 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "health": { 2023-11-27T15:13:10.654 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-27T15:13:10.654 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-27T15:13:10.654 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-27T15:13:10.654 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:13:10.654 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-27T15:13:10.654 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-27T15:13:10.654 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 0 2023-11-27T15:13:10.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:13:10.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-27T15:13:10.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "a" 2023-11-27T15:13:10.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:13:10.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-11-27T15:13:10.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-27T15:13:10.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:13:10.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-27T15:13:10.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-27T15:13:10.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:13:10.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-27T15:13:10.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:13:10.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-27T15:13:10.655 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-27T15:13:10.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-27T15:13:10.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-27T15:13:10.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-27T15:13:10.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-27T15:13:10.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:13:10.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-27T15:13:10.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-27T15:13:10.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-27T15:13:10.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-27T15:13:10.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-27T15:13:10.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-27T15:13:10.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-27T15:13:10.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-27T15:13:10.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-27T15:13:10.656 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:13:10.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-27T15:13:10.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:13:10.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-27T15:13:10.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-27T15:13:10.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:13:10.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-27T15:13:10.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": false, 2023-11-27T15:13:10.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-27T15:13:10.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-27T15:13:10.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-27T15:13:10.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-27T15:13:10.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "restful" 2023-11-27T15:13:10.657 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:13:10.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:13:10.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:13:10.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-27T15:13:10.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:13:10.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modified": "2023-11-27T15:13:01.702076+0000", 2023-11-27T15:13:10.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:13:10.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:13:10.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-27T15:13:10.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:13:10.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.048+0000 7fedde353700 1 Processor -- start 2023-11-27T15:13:10.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.049+0000 7fedde353700 1 -- start start 2023-11-27T15:13:10.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.049+0000 7fedde353700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedd8105910 0x7fedd8105d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:10.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.049+0000 7fedde353700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd8106260 con 0x7fedd8105910 2023-11-27T15:13:10.658 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.049+0000 7fedd7fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedd8105910 0x7fedd8105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:10.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.049+0000 7fedd7fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedd8105910 0x7fedd8105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58306/0 (socket says 172.21.15.133:58306) 2023-11-27T15:13:10.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.049+0000 7fedd7fff700 1 -- 172.21.15.133:0/1455504742 learned_addr learned my addr 172.21.15.133:0/1455504742 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:10.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.050+0000 7fedd7fff700 1 -- 172.21.15.133:0/1455504742 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd81063a0 con 0x7fedd8105910 2023-11-27T15:13:10.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.050+0000 7fedd7fff700 1 --2- 172.21.15.133:0/1455504742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedd8105910 0x7fedd8105d20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fedc80098a0 tx=0x7fedc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d328786dbf5682e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:10.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.050+0000 7fedd6ffd700 1 -- 172.21.15.133:0/1455504742 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fedc8004160 con 0x7fedd8105910 2023-11-27T15:13:10.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.050+0000 7fedd6ffd700 1 -- 172.21.15.133:0/1455504742 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fedc80042c0 con 0x7fedd8105910 2023-11-27T15:13:10.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.050+0000 7fedd6ffd700 1 -- 172.21.15.133:0/1455504742 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fedc8011a00 con 0x7fedd8105910 2023-11-27T15:13:10.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.051+0000 7fedde353700 1 -- 172.21.15.133:0/1455504742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedd8105910 msgr2=0x7fedd8105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:10.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.051+0000 7fedde353700 1 --2- 172.21.15.133:0/1455504742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedd8105910 0x7fedd8105d20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fedc80098a0 tx=0x7fedc8009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:10.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.051+0000 7fedde353700 1 -- 172.21.15.133:0/1455504742 shutdown_connections 2023-11-27T15:13:10.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.051+0000 7fedde353700 1 --2- 172.21.15.133:0/1455504742 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedd8105910 0x7fedd8105d20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:10.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.051+0000 7fedde353700 1 -- 172.21.15.133:0/1455504742 >> 172.21.15.133:0/1455504742 conn(0x7fedd8100fa0 msgr2=0x7fedd81033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:10.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.051+0000 7fedde353700 1 -- 172.21.15.133:0/1455504742 shutdown_connections 2023-11-27T15:13:10.659 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.051+0000 7fedde353700 1 -- 172.21.15.133:0/1455504742 wait complete. 2023-11-27T15:13:10.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.052+0000 7fedde353700 1 Processor -- start 2023-11-27T15:13:10.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.052+0000 7fedde353700 1 -- start start 2023-11-27T15:13:10.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.053+0000 7fedde353700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedd8105910 0x7fedd807bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:10.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.053+0000 7fedde353700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd807c190 con 0x7fedd8105910 2023-11-27T15:13:10.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.053+0000 7fedd7fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedd8105910 0x7fedd807bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:10.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.053+0000 7fedd7fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedd8105910 0x7fedd807bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58320/0 (socket says 172.21.15.133:58320) 2023-11-27T15:13:10.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.053+0000 7fedd7fff700 1 -- 172.21.15.133:0/2054785813 learned_addr learned my addr 172.21.15.133:0/2054785813 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:10.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.053+0000 7fedd7fff700 1 -- 172.21.15.133:0/2054785813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc8009550 con 0x7fedd8105910 2023-11-27T15:13:10.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.053+0000 7fedd7fff700 1 --2- 172.21.15.133:0/2054785813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedd8105910 0x7fedd807bc50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fedc8006af0 tx=0x7fedc8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:10.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.054+0000 7fedd57fa700 1 -- 172.21.15.133:0/2054785813 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fedc8003a40 con 0x7fedd8105910 2023-11-27T15:13:10.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.054+0000 7fedd57fa700 1 -- 172.21.15.133:0/2054785813 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fedc8003ba0 con 0x7fedd8105910 2023-11-27T15:13:10.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.054+0000 7fedde353700 1 -- 172.21.15.133:0/2054785813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd8107960 con 0x7fedd8105910 2023-11-27T15:13:10.660 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.054+0000 7fedd57fa700 1 -- 172.21.15.133:0/2054785813 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fedc801a6e0 con 0x7fedd8105910 2023-11-27T15:13:10.661 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.054+0000 7fedde353700 1 -- 172.21.15.133:0/2054785813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd81077d0 con 0x7fedd8105910 2023-11-27T15:13:10.661 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.055+0000 7fedd57fa700 1 -- 172.21.15.133:0/2054785813 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fedc8024430 con 0x7fedd8105910 2023-11-27T15:13:10.661 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.055+0000 7fedd57fa700 1 -- 172.21.15.133:0/2054785813 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fedc802d730 con 0x7fedd8105910 2023-11-27T15:13:10.661 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.055+0000 7fedde353700 1 -- 172.21.15.133:0/2054785813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd8062120 con 0x7fedd8105910 2023-11-27T15:13:10.661 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.058+0000 7fedd57fa700 1 -- 172.21.15.133:0/2054785813 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fedc801ae50 con 0x7fedd8105910 2023-11-27T15:13:10.661 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.151+0000 7fedde353700 1 -- 172.21.15.133:0/2054785813 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fedd807b0d0 con 0x7fedd8105910 2023-11-27T15:13:10.661 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.152+0000 7fedd57fa700 1 -- 172.21.15.133:0/2054785813 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fedc8024870 con 0x7fedd8105910 2023-11-27T15:13:10.661 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.153+0000 7fedde353700 1 -- 172.21.15.133:0/2054785813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedd8105910 msgr2=0x7fedd807bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:10.661 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.153+0000 7fedde353700 1 --2- 172.21.15.133:0/2054785813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedd8105910 0x7fedd807bc50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fedc8006af0 tx=0x7fedc8011ad0 comp rx=0 tx=0).stop 2023-11-27T15:13:10.661 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.153+0000 7fedde353700 1 -- 172.21.15.133:0/2054785813 shutdown_connections 2023-11-27T15:13:10.661 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.153+0000 7fedde353700 1 --2- 172.21.15.133:0/2054785813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedd8105910 0x7fedd807bc50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:10.661 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.153+0000 7fedde353700 1 -- 172.21.15.133:0/2054785813 >> 172.21.15.133:0/2054785813 conn(0x7fedd8100fa0 msgr2=0x7fedd8101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:10.661 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.153+0000 7fedde353700 1 -- 172.21.15.133:0/2054785813 shutdown_connections 2023-11-27T15:13:10.661 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:10.153+0000 7fedde353700 1 -- 172.21.15.133:0/2054785813 wait complete. 2023-11-27T15:13:10.662 INFO:teuthology.orchestra.run.smithi133.stdout:mgr not available, waiting (1/15)... 2023-11-27T15:13:10.818 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:10 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:10.530+0000 7f62f5084200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-27T15:13:10.818 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:10 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:10.649+0000 7f62f5084200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-27T15:13:11.277 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:10 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:10.812+0000 7f62f5084200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-27T15:13:11.278 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:10 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:10.937+0000 7f62f5084200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-27T15:13:11.777 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:11 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:11.475+0000 7f62f5084200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-27T15:13:12.527 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:12 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:12.249+0000 7f62f5084200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-27T15:13:12.528 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:12 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:12.459+0000 7f62f5084200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-27T15:13:13.596 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:13 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3743058057' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-27T15:13:13.597 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:13 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:13.473+0000 7f62f5084200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:13:14.027 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:13 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:13.591+0000 7f62f5084200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-27T15:13:14.028 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:13 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:13.815+0000 7f62f5084200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:13:14.462 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:14 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:14.176+0000 7f62f5084200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-27T15:13:14.777 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:14 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:14.457+0000 7f62f5084200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-27T15:13:14.778 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:14 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:14.650+0000 7f62f5084200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-27T15:13:15.278 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:14 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:14.992+0000 7f62f5084200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-27T15:13:16.278 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:15 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:15.921+0000 7f62f5084200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-27T15:13:16.278 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:16 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:16.092+0000 7f62f5084200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-27T15:13:16.765 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:16 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:16.337+0000 7f62f5084200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-27T15:13:16.765 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:16 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:16.485+0000 7f62f5084200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-27T15:13:16.766 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:16 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:16.761+0000 7f62f5084200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-27T15:13:18.278 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:17 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:17.821+0000 7f62f5084200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-27T15:13:18.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:17 smithi133 ceph-mon[97860]: Activating manager daemon x 2023-11-27T15:13:18.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:17 smithi133 ceph-mon[97860]: mgrmap e2: x(active, starting, since 0.00414226s) 2023-11-27T15:13:18.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:17 smithi133 ceph-mon[97860]: from='mgr.14100 172.21.15.133:0/4096972751' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-27T15:13:18.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:17 smithi133 ceph-mon[97860]: from='mgr.14100 172.21.15.133:0/4096972751' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-27T15:13:18.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:17 smithi133 ceph-mon[97860]: from='mgr.14100 172.21.15.133:0/4096972751' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-27T15:13:18.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:17 smithi133 ceph-mon[97860]: from='mgr.14100 172.21.15.133:0/4096972751' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:13:18.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:17 smithi133 ceph-mon[97860]: from='mgr.14100 172.21.15.133:0/4096972751' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-27T15:13:18.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:17 smithi133 ceph-mon[97860]: Manager daemon x is now available 2023-11-27T15:13:18.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:17 smithi133 ceph-mon[97860]: from='mgr.14100 172.21.15.133:0/4096972751' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-27T15:13:18.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:17 smithi133 ceph-mon[97860]: from='mgr.14100 172.21.15.133:0/4096972751' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-27T15:13:18.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:17 smithi133 ceph-mon[97860]: from='mgr.14100 172.21.15.133:0/4096972751' entity='mgr.x' 2023-11-27T15:13:18.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:17 smithi133 ceph-mon[97860]: from='mgr.14100 172.21.15.133:0/4096972751' entity='mgr.x' 2023-11-27T15:13:18.280 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:17 smithi133 ceph-mon[97860]: from='mgr.14100 172.21.15.133:0/4096972751' entity='mgr.x' 2023-11-27T15:13:20.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:19 smithi133 ceph-mon[97860]: mgrmap e3: x(active, since 1.01051s) 2023-11-27T15:13:21.253 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-11-27T15:13:21.253 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:13:21.253 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsid": "5227da4c-8d37-11ee-95a2-87774f69a715", 2023-11-27T15:13:21.253 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "health": { 2023-11-27T15:13:21.254 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-27T15:13:21.254 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-27T15:13:21.254 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-27T15:13:21.254 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:13:21.254 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-27T15:13:21.254 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-27T15:13:21.254 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 0 2023-11-27T15:13:21.254 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:13:21.254 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-27T15:13:21.254 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "a" 2023-11-27T15:13:21.254 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:13:21.254 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-11-27T15:13:21.254 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-27T15:13:21.255 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:13:21.255 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-27T15:13:21.255 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-27T15:13:21.255 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:13:21.255 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-27T15:13:21.255 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:13:21.255 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-27T15:13:21.255 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-27T15:13:21.255 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-27T15:13:21.263 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-27T15:13:21.263 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-27T15:13:21.263 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-27T15:13:21.263 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:13:21.263 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-27T15:13:21.263 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-27T15:13:21.263 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-27T15:13:21.263 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-27T15:13:21.263 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-27T15:13:21.263 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-27T15:13:21.263 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-27T15:13:21.263 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-27T15:13:21.264 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-27T15:13:21.264 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:13:21.264 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-27T15:13:21.264 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:13:21.264 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-27T15:13:21.264 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-27T15:13:21.264 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:13:21.264 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-27T15:13:21.264 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": false, 2023-11-27T15:13:21.264 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-27T15:13:21.264 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-27T15:13:21.264 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-27T15:13:21.264 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-27T15:13:21.264 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "restful" 2023-11-27T15:13:21.265 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:13:21.265 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:13:21.265 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:13:21.265 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-27T15:13:21.265 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:13:21.265 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modified": "2023-11-27T15:13:01.702076+0000", 2023-11-27T15:13:21.265 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:13:21.265 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:13:21.265 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-27T15:13:21.265 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:13:21.265 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.198+0000 7f66e33b0700 1 Processor -- start 2023-11-27T15:13:21.265 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.198+0000 7f66e33b0700 1 -- start start 2023-11-27T15:13:21.266 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.198+0000 7f66e33b0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66dc07bde0 0x7f66dc07a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:21.266 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.198+0000 7f66e33b0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc07a820 con 0x7f66dc07bde0 2023-11-27T15:13:21.266 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.199+0000 7f66e114c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66dc07bde0 0x7f66dc07a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:21.266 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.199+0000 7f66e114c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66dc07bde0 0x7f66dc07a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39918/0 (socket says 172.21.15.133:39918) 2023-11-27T15:13:21.266 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.199+0000 7f66e114c700 1 -- 172.21.15.133:0/4103135289 learned_addr learned my addr 172.21.15.133:0/4103135289 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:21.266 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.199+0000 7f66e114c700 1 -- 172.21.15.133:0/4103135289 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66dc07a960 con 0x7f66dc07bde0 2023-11-27T15:13:21.266 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.199+0000 7f66e114c700 1 --2- 172.21.15.133:0/4103135289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66dc07bde0 0x7f66dc07a2e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f66d80098a0 tx=0x7f66d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c10c2f3509cabfb server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:21.266 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.200+0000 7f66d3fff700 1 -- 172.21.15.133:0/4103135289 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66d8004160 con 0x7f66dc07bde0 2023-11-27T15:13:21.266 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.200+0000 7f66d3fff700 1 -- 172.21.15.133:0/4103135289 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f66d80042c0 con 0x7f66dc07bde0 2023-11-27T15:13:21.266 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.200+0000 7f66d3fff700 1 -- 172.21.15.133:0/4103135289 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66d8011a00 con 0x7f66dc07bde0 2023-11-27T15:13:21.266 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.201+0000 7f66e33b0700 1 -- 172.21.15.133:0/4103135289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66dc07bde0 msgr2=0x7f66dc07a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:21.266 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.201+0000 7f66e33b0700 1 --2- 172.21.15.133:0/4103135289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66dc07bde0 0x7f66dc07a2e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f66d80098a0 tx=0x7f66d8009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:21.267 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.201+0000 7f66e33b0700 1 -- 172.21.15.133:0/4103135289 shutdown_connections 2023-11-27T15:13:21.267 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.201+0000 7f66e33b0700 1 --2- 172.21.15.133:0/4103135289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66dc07bde0 0x7f66dc07a2e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:21.267 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.201+0000 7f66e33b0700 1 -- 172.21.15.133:0/4103135289 >> 172.21.15.133:0/4103135289 conn(0x7f66dc100fa0 msgr2=0x7f66dc1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:21.267 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.201+0000 7f66e33b0700 1 -- 172.21.15.133:0/4103135289 shutdown_connections 2023-11-27T15:13:21.267 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.201+0000 7f66e33b0700 1 -- 172.21.15.133:0/4103135289 wait complete. 2023-11-27T15:13:21.267 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.202+0000 7f66e33b0700 1 Processor -- start 2023-11-27T15:13:21.267 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.203+0000 7f66e33b0700 1 -- start start 2023-11-27T15:13:21.267 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.203+0000 7f66e33b0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66dc07bde0 0x7f66dc19b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:21.267 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.203+0000 7f66e33b0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc19bcf0 con 0x7f66dc07bde0 2023-11-27T15:13:21.267 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.203+0000 7f66e114c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66dc07bde0 0x7f66dc19b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:21.267 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.203+0000 7f66e114c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66dc07bde0 0x7f66dc19b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39930/0 (socket says 172.21.15.133:39930) 2023-11-27T15:13:21.267 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.203+0000 7f66e114c700 1 -- 172.21.15.133:0/3743058057 learned_addr learned my addr 172.21.15.133:0/3743058057 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:21.267 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.204+0000 7f66e114c700 1 -- 172.21.15.133:0/3743058057 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d8009550 con 0x7f66dc07bde0 2023-11-27T15:13:21.268 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.204+0000 7f66e114c700 1 --2- 172.21.15.133:0/3743058057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66dc07bde0 0x7f66dc19b7b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f66d8006af0 tx=0x7f66d8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:21.268 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.204+0000 7f66d27fc700 1 -- 172.21.15.133:0/3743058057 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66d8003a40 con 0x7f66dc07bde0 2023-11-27T15:13:21.268 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.204+0000 7f66d27fc700 1 -- 172.21.15.133:0/3743058057 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f66d8003ba0 con 0x7f66dc07bde0 2023-11-27T15:13:21.268 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.204+0000 7f66d27fc700 1 -- 172.21.15.133:0/3743058057 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66d801a700 con 0x7f66dc07bde0 2023-11-27T15:13:21.268 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.204+0000 7f66e33b0700 1 -- 172.21.15.133:0/3743058057 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66dc10be00 con 0x7f66dc07bde0 2023-11-27T15:13:21.268 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.205+0000 7f66e33b0700 1 -- 172.21.15.133:0/3743058057 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66dc10bc70 con 0x7f66dc07bde0 2023-11-27T15:13:21.268 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.205+0000 7f66d27fc700 1 -- 172.21.15.133:0/3743058057 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f66d801ac80 con 0x7f66dc07bde0 2023-11-27T15:13:21.268 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.205+0000 7f66d27fc700 1 -- 172.21.15.133:0/3743058057 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f66d8019700 con 0x7f66dc07bde0 2023-11-27T15:13:21.268 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.206+0000 7f66e33b0700 1 -- 172.21.15.133:0/3743058057 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66dc062120 con 0x7f66dc07bde0 2023-11-27T15:13:21.268 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.208+0000 7f66d27fc700 1 -- 172.21.15.133:0/3743058057 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f66d8024ad0 con 0x7f66dc07bde0 2023-11-27T15:13:21.268 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.285+0000 7f66e33b0700 1 -- 172.21.15.133:0/3743058057 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f66dc19c230 con 0x7f66dc07bde0 2023-11-27T15:13:21.268 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.286+0000 7f66d27fc700 1 -- 172.21.15.133:0/3743058057 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f66d8024c70 con 0x7f66dc07bde0 2023-11-27T15:13:21.268 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.287+0000 7f66e33b0700 1 -- 172.21.15.133:0/3743058057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66dc07bde0 msgr2=0x7f66dc19b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:21.269 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.287+0000 7f66e33b0700 1 --2- 172.21.15.133:0/3743058057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66dc07bde0 0x7f66dc19b7b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f66d8006af0 tx=0x7f66d8011ad0 comp rx=0 tx=0).stop 2023-11-27T15:13:21.269 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.287+0000 7f66e33b0700 1 -- 172.21.15.133:0/3743058057 shutdown_connections 2023-11-27T15:13:21.269 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.287+0000 7f66e33b0700 1 --2- 172.21.15.133:0/3743058057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66dc07bde0 0x7f66dc19b7b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:21.269 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.287+0000 7f66e33b0700 1 -- 172.21.15.133:0/3743058057 >> 172.21.15.133:0/3743058057 conn(0x7f66dc100fa0 msgr2=0x7f66dc101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:21.269 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.287+0000 7f66e33b0700 1 -- 172.21.15.133:0/3743058057 shutdown_connections 2023-11-27T15:13:21.269 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:13.287+0000 7f66e33b0700 1 -- 172.21.15.133:0/3743058057 wait complete. 2023-11-27T15:13:21.269 INFO:teuthology.orchestra.run.smithi133.stdout:mgr not available, waiting (2/15)... 2023-11-27T15:13:21.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:20 smithi133 ceph-mon[97860]: mgrmap e4: x(active, since 2s) 2023-11-27T15:13:24.521 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:24 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2369544278' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-11-27T15:13:24.547 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-11-27T15:13:24.547 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:13:24.548 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsid": "5227da4c-8d37-11ee-95a2-87774f69a715", 2023-11-27T15:13:24.548 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "health": { 2023-11-27T15:13:24.548 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-11-27T15:13:24.548 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-11-27T15:13:24.548 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-11-27T15:13:24.548 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:13:24.548 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-11-27T15:13:24.548 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-11-27T15:13:24.548 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 0 2023-11-27T15:13:24.548 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:13:24.548 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-11-27T15:13:24.548 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "a" 2023-11-27T15:13:24.549 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:13:24.549 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2023-11-27T15:13:24.549 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "monmap": { 2023-11-27T15:13:24.549 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:13:24.549 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-11-27T15:13:24.549 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-11-27T15:13:24.549 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:13:24.549 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-11-27T15:13:24.549 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:13:24.549 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-11-27T15:13:24.549 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-11-27T15:13:24.557 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-11-27T15:13:24.557 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-11-27T15:13:24.557 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-11-27T15:13:24.557 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-11-27T15:13:24.557 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:13:24.557 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-11-27T15:13:24.557 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-11-27T15:13:24.558 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-11-27T15:13:24.558 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-11-27T15:13:24.558 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-11-27T15:13:24.558 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-11-27T15:13:24.558 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-11-27T15:13:24.558 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-11-27T15:13:24.558 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-11-27T15:13:24.558 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:13:24.558 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-11-27T15:13:24.558 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:13:24.558 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-11-27T15:13:24.558 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-11-27T15:13:24.558 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:13:24.559 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-11-27T15:13:24.559 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": true, 2023-11-27T15:13:24.559 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-11-27T15:13:24.559 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modules": [ 2023-11-27T15:13:24.559 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "iostat", 2023-11-27T15:13:24.559 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "nfs", 2023-11-27T15:13:24.559 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "restful" 2023-11-27T15:13:24.559 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ], 2023-11-27T15:13:24.559 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:13:24.559 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:13:24.559 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-11-27T15:13:24.559 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-11-27T15:13:24.560 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "modified": "2023-11-27T15:13:01.702076+0000", 2023-11-27T15:13:24.560 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "services": {} 2023-11-27T15:13:24.560 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout }, 2023-11-27T15:13:24.560 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-11-27T15:13:24.560 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:13:24.560 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.808+0000 7f2b70a58700 1 Processor -- start 2023-11-27T15:13:24.560 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.808+0000 7f2b70a58700 1 -- start start 2023-11-27T15:13:24.560 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.808+0000 7f2b70a58700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b6c07be20 0x7f2b6c07a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:24.560 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.808+0000 7f2b70a58700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c07a860 con 0x7f2b6c07be20 2023-11-27T15:13:24.560 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.808+0000 7f2b6a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b6c07be20 0x7f2b6c07a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:24.560 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.808+0000 7f2b6a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b6c07be20 0x7f2b6c07a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49902/0 (socket says 172.21.15.133:49902) 2023-11-27T15:13:24.560 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.808+0000 7f2b6a59c700 1 -- 172.21.15.133:0/529339798 learned_addr learned my addr 172.21.15.133:0/529339798 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:24.560 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.809+0000 7f2b6a59c700 1 -- 172.21.15.133:0/529339798 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6c07a9a0 con 0x7f2b6c07be20 2023-11-27T15:13:24.560 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.809+0000 7f2b6a59c700 1 --2- 172.21.15.133:0/529339798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b6c07be20 0x7f2b6c07a320 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0098a0 tx=0x7f2b5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b9659108924cd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:24.561 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.809+0000 7f2b6959a700 1 -- 172.21.15.133:0/529339798 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b5c004160 con 0x7f2b6c07be20 2023-11-27T15:13:24.561 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.809+0000 7f2b6959a700 1 -- 172.21.15.133:0/529339798 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2b5c0042c0 con 0x7f2b6c07be20 2023-11-27T15:13:24.561 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.809+0000 7f2b6959a700 1 -- 172.21.15.133:0/529339798 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b5c011a00 con 0x7f2b6c07be20 2023-11-27T15:13:24.561 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.810+0000 7f2b70a58700 1 -- 172.21.15.133:0/529339798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b6c07be20 msgr2=0x7f2b6c07a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:24.561 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.810+0000 7f2b70a58700 1 --2- 172.21.15.133:0/529339798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b6c07be20 0x7f2b6c07a320 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0098a0 tx=0x7f2b5c009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:24.561 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.810+0000 7f2b70a58700 1 -- 172.21.15.133:0/529339798 shutdown_connections 2023-11-27T15:13:24.561 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.810+0000 7f2b70a58700 1 --2- 172.21.15.133:0/529339798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b6c07be20 0x7f2b6c07a320 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:24.561 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.810+0000 7f2b70a58700 1 -- 172.21.15.133:0/529339798 >> 172.21.15.133:0/529339798 conn(0x7f2b6c100fa0 msgr2=0x7f2b6c1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:24.561 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.810+0000 7f2b70a58700 1 -- 172.21.15.133:0/529339798 shutdown_connections 2023-11-27T15:13:24.561 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.810+0000 7f2b70a58700 1 -- 172.21.15.133:0/529339798 wait complete. 2023-11-27T15:13:24.561 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.811+0000 7f2b70a58700 1 Processor -- start 2023-11-27T15:13:24.561 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.811+0000 7f2b70a58700 1 -- start start 2023-11-27T15:13:24.562 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.812+0000 7f2b70a58700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b6c07be20 0x7f2b6c19b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:24.562 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.812+0000 7f2b70a58700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6c19bce0 con 0x7f2b6c07be20 2023-11-27T15:13:24.562 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.812+0000 7f2b6a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b6c07be20 0x7f2b6c19b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:24.562 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.812+0000 7f2b6a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b6c07be20 0x7f2b6c19b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49914/0 (socket says 172.21.15.133:49914) 2023-11-27T15:13:24.562 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.812+0000 7f2b6a59c700 1 -- 172.21.15.133:0/2369544278 learned_addr learned my addr 172.21.15.133:0/2369544278 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:24.562 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.812+0000 7f2b6a59c700 1 -- 172.21.15.133:0/2369544278 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5c009550 con 0x7f2b6c07be20 2023-11-27T15:13:24.562 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.812+0000 7f2b6a59c700 1 --2- 172.21.15.133:0/2369544278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b6c07be20 0x7f2b6c19b7a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c006af0 tx=0x7f2b5c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:24.562 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.813+0000 7f2b5b7fe700 1 -- 172.21.15.133:0/2369544278 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b5c003a40 con 0x7f2b6c07be20 2023-11-27T15:13:24.562 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.813+0000 7f2b5b7fe700 1 -- 172.21.15.133:0/2369544278 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2b5c003ba0 con 0x7f2b6c07be20 2023-11-27T15:13:24.562 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.813+0000 7f2b70a58700 1 -- 172.21.15.133:0/2369544278 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b6c10bdc0 con 0x7f2b6c07be20 2023-11-27T15:13:24.562 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.813+0000 7f2b5b7fe700 1 -- 172.21.15.133:0/2369544278 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b5c01a700 con 0x7f2b6c07be20 2023-11-27T15:13:24.562 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.813+0000 7f2b70a58700 1 -- 172.21.15.133:0/2369544278 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b6c10bc30 con 0x7f2b6c07be20 2023-11-27T15:13:24.562 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.814+0000 7f2b5b7fe700 1 -- 172.21.15.133:0/2369544278 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f2b5c024ad0 con 0x7f2b6c07be20 2023-11-27T15:13:24.563 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.814+0000 7f2b70a58700 1 -- 172.21.15.133:0/2369544278 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b6c062120 con 0x7f2b6c07be20 2023-11-27T15:13:24.563 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.814+0000 7f2b5b7fe700 1 --2- 172.21.15.133:0/2369544278 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7f2b540387d0 0x7f2b5403ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:24.563 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.814+0000 7f2b5b7fe700 1 -- 172.21.15.133:0/2369544278 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2b5c04c630 con 0x7f2b6c07be20 2023-11-27T15:13:24.563 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.814+0000 7f2b69d9b700 1 --2- 172.21.15.133:0/2369544278 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7f2b540387d0 0x7f2b5403ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:24.563 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.815+0000 7f2b69d9b700 1 --2- 172.21.15.133:0/2369544278 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7f2b540387d0 0x7f2b5403ac80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2b60006fd0 tx=0x7f2b60006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:24.563 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:23.820+0000 7f2b5b7fe700 1 -- 172.21.15.133:0/2369544278 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2b5c018bd0 con 0x7f2b6c07be20 2023-11-27T15:13:24.563 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:24.090+0000 7f2b70a58700 1 -- 172.21.15.133:0/2369544278 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2b6c19c470 con 0x7f2b6c07be20 2023-11-27T15:13:24.563 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:24.090+0000 7f2b5b7fe700 1 -- 172.21.15.133:0/2369544278 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f2b5c0187c0 con 0x7f2b6c07be20 2023-11-27T15:13:24.563 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:24.092+0000 7f2b70a58700 1 -- 172.21.15.133:0/2369544278 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7f2b540387d0 msgr2=0x7f2b5403ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:24.563 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:24.092+0000 7f2b70a58700 1 --2- 172.21.15.133:0/2369544278 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7f2b540387d0 0x7f2b5403ac80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2b60006fd0 tx=0x7f2b60006e40 comp rx=0 tx=0).stop 2023-11-27T15:13:24.563 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:24.092+0000 7f2b70a58700 1 -- 172.21.15.133:0/2369544278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b6c07be20 msgr2=0x7f2b6c19b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:24.563 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:24.092+0000 7f2b70a58700 1 --2- 172.21.15.133:0/2369544278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b6c07be20 0x7f2b6c19b7a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c006af0 tx=0x7f2b5c011ad0 comp rx=0 tx=0).stop 2023-11-27T15:13:24.563 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:24.092+0000 7f2b70a58700 1 -- 172.21.15.133:0/2369544278 shutdown_connections 2023-11-27T15:13:24.564 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:24.092+0000 7f2b70a58700 1 --2- 172.21.15.133:0/2369544278 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7f2b540387d0 0x7f2b5403ac80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:24.564 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:24.092+0000 7f2b70a58700 1 --2- 172.21.15.133:0/2369544278 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b6c07be20 0x7f2b6c19b7a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:24.564 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:24.092+0000 7f2b70a58700 1 -- 172.21.15.133:0/2369544278 >> 172.21.15.133:0/2369544278 conn(0x7f2b6c100fa0 msgr2=0x7f2b6c101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:24.564 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:24.093+0000 7f2b70a58700 1 -- 172.21.15.133:0/2369544278 shutdown_connections 2023-11-27T15:13:24.564 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:24.093+0000 7f2b70a58700 1 -- 172.21.15.133:0/2369544278 wait complete. 2023-11-27T15:13:24.564 INFO:teuthology.orchestra.run.smithi133.stdout:mgr is available 2023-11-27T15:13:25.728 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-11-27T15:13:25.728 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:25 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1962304234' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-11-27T15:13:25.735 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [global] 2023-11-27T15:13:25.736 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout fsid = 5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:13:25.736 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-11-27T15:13:25.736 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-11-27T15:13:25.736 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-11-27T15:13:25.736 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-11-27T15:13:25.736 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-11-27T15:13:25.736 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [mgr] 2023-11-27T15:13:25.736 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-11-27T15:13:25.736 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 2023-11-27T15:13:25.736 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout [osd] 2023-11-27T15:13:25.737 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-11-27T15:13:25.737 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-11-27T15:13:25.737 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-11-27T15:13:25.737 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-11-27T15:13:25.737 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-11-27T15:13:25.737 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.092+0000 7f6db53c4700 1 Processor -- start 2023-11-27T15:13:25.737 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.092+0000 7f6db53c4700 1 -- start start 2023-11-27T15:13:25.737 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.092+0000 7f6db53c4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db007bdc0 0x7f6db007a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:25.737 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.093+0000 7f6db53c4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db007a800 con 0x7f6db007bdc0 2023-11-27T15:13:25.737 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.093+0000 7f6daeffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db007bdc0 0x7f6db007a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:25.738 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.093+0000 7f6daeffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db007bdc0 0x7f6db007a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49928/0 (socket says 172.21.15.133:49928) 2023-11-27T15:13:25.738 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.093+0000 7f6daeffd700 1 -- 172.21.15.133:0/1794282795 learned_addr learned my addr 172.21.15.133:0/1794282795 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:25.738 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.093+0000 7f6daeffd700 1 -- 172.21.15.133:0/1794282795 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db007a940 con 0x7f6db007bdc0 2023-11-27T15:13:25.738 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.093+0000 7f6daeffd700 1 --2- 172.21.15.133:0/1794282795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db007bdc0 0x7f6db007a2c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6da00098a0 tx=0x7f6da0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc34b2055e571455 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:25.738 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.094+0000 7f6dadffb700 1 -- 172.21.15.133:0/1794282795 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6da0004160 con 0x7f6db007bdc0 2023-11-27T15:13:25.738 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.094+0000 7f6dadffb700 1 -- 172.21.15.133:0/1794282795 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6da00042c0 con 0x7f6db007bdc0 2023-11-27T15:13:25.738 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.094+0000 7f6dadffb700 1 -- 172.21.15.133:0/1794282795 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6da0011a00 con 0x7f6db007bdc0 2023-11-27T15:13:25.738 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.094+0000 7f6db53c4700 1 -- 172.21.15.133:0/1794282795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db007bdc0 msgr2=0x7f6db007a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:25.738 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.094+0000 7f6db53c4700 1 --2- 172.21.15.133:0/1794282795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db007bdc0 0x7f6db007a2c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6da00098a0 tx=0x7f6da0009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:25.738 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.094+0000 7f6db53c4700 1 -- 172.21.15.133:0/1794282795 shutdown_connections 2023-11-27T15:13:25.739 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.094+0000 7f6db53c4700 1 --2- 172.21.15.133:0/1794282795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db007bdc0 0x7f6db007a2c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:25.739 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.094+0000 7f6db53c4700 1 -- 172.21.15.133:0/1794282795 >> 172.21.15.133:0/1794282795 conn(0x7f6db0100fa0 msgr2=0x7f6db01033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:25.739 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.094+0000 7f6db53c4700 1 -- 172.21.15.133:0/1794282795 shutdown_connections 2023-11-27T15:13:25.739 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.095+0000 7f6db53c4700 1 -- 172.21.15.133:0/1794282795 wait complete. 2023-11-27T15:13:25.739 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.095+0000 7f6db53c4700 1 Processor -- start 2023-11-27T15:13:25.739 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.096+0000 7f6db53c4700 1 -- start start 2023-11-27T15:13:25.739 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.096+0000 7f6db53c4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db007bdc0 0x7f6db019b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:25.739 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.096+0000 7f6db53c4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db019bcf0 con 0x7f6db007bdc0 2023-11-27T15:13:25.739 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.096+0000 7f6daeffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db007bdc0 0x7f6db019b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:25.739 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.096+0000 7f6daeffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db007bdc0 0x7f6db019b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49932/0 (socket says 172.21.15.133:49932) 2023-11-27T15:13:25.740 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.096+0000 7f6daeffd700 1 -- 172.21.15.133:0/1962304234 learned_addr learned my addr 172.21.15.133:0/1962304234 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:25.740 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.096+0000 7f6daeffd700 1 -- 172.21.15.133:0/1962304234 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da0009550 con 0x7f6db007bdc0 2023-11-27T15:13:25.740 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.097+0000 7f6daeffd700 1 --2- 172.21.15.133:0/1962304234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db007bdc0 0x7f6db019b7b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6da0006af0 tx=0x7f6da0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:25.740 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.097+0000 7f6d97fff700 1 -- 172.21.15.133:0/1962304234 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6da0003a40 con 0x7f6db007bdc0 2023-11-27T15:13:25.740 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.097+0000 7f6d97fff700 1 -- 172.21.15.133:0/1962304234 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6da0003ba0 con 0x7f6db007bdc0 2023-11-27T15:13:25.740 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.097+0000 7f6db53c4700 1 -- 172.21.15.133:0/1962304234 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db010bdc0 con 0x7f6db007bdc0 2023-11-27T15:13:25.740 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.097+0000 7f6d97fff700 1 -- 172.21.15.133:0/1962304234 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6da001a700 con 0x7f6db007bdc0 2023-11-27T15:13:25.740 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.097+0000 7f6db53c4700 1 -- 172.21.15.133:0/1962304234 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db010bc30 con 0x7f6db007bdc0 2023-11-27T15:13:25.740 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.098+0000 7f6d97fff700 1 -- 172.21.15.133:0/1962304234 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f6da0024ad0 con 0x7f6db007bdc0 2023-11-27T15:13:25.740 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.098+0000 7f6db53c4700 1 -- 172.21.15.133:0/1962304234 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db0062120 con 0x7f6db007bdc0 2023-11-27T15:13:25.741 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.099+0000 7f6d97fff700 1 --2- 172.21.15.133:0/1962304234 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7f6d98038780 0x7f6d9803ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:25.741 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.099+0000 7f6d97fff700 1 -- 172.21.15.133:0/1962304234 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6da004c6b0 con 0x7f6db007bdc0 2023-11-27T15:13:25.741 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.099+0000 7f6dae7fc700 1 --2- 172.21.15.133:0/1962304234 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7f6d98038780 0x7f6d9803ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:25.741 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.100+0000 7f6dae7fc700 1 --2- 172.21.15.133:0/1962304234 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7f6d98038780 0x7f6d9803ac30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6da4006fd0 tx=0x7f6da4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:25.741 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.104+0000 7f6d97fff700 1 -- 172.21.15.133:0/1962304234 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6da0018bd0 con 0x7f6db007bdc0 2023-11-27T15:13:25.741 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.297+0000 7f6db53c4700 1 -- 172.21.15.133:0/1962304234 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f6db019c480 con 0x7f6db007bdc0 2023-11-27T15:13:25.741 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.298+0000 7f6d97fff700 1 -- 172.21.15.133:0/1962304234 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f6da00187c0 con 0x7f6db007bdc0 2023-11-27T15:13:25.741 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.301+0000 7f6db53c4700 1 -- 172.21.15.133:0/1962304234 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7f6d98038780 msgr2=0x7f6d9803ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:25.741 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.301+0000 7f6db53c4700 1 --2- 172.21.15.133:0/1962304234 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7f6d98038780 0x7f6d9803ac30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6da4006fd0 tx=0x7f6da4006e40 comp rx=0 tx=0).stop 2023-11-27T15:13:25.742 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.301+0000 7f6db53c4700 1 -- 172.21.15.133:0/1962304234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db007bdc0 msgr2=0x7f6db019b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:25.742 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.301+0000 7f6db53c4700 1 --2- 172.21.15.133:0/1962304234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db007bdc0 0x7f6db019b7b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6da0006af0 tx=0x7f6da0011ad0 comp rx=0 tx=0).stop 2023-11-27T15:13:25.742 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.301+0000 7f6db53c4700 1 -- 172.21.15.133:0/1962304234 shutdown_connections 2023-11-27T15:13:25.742 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.301+0000 7f6db53c4700 1 --2- 172.21.15.133:0/1962304234 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7f6d98038780 0x7f6d9803ac30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:25.742 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.301+0000 7f6db53c4700 1 --2- 172.21.15.133:0/1962304234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6db007bdc0 0x7f6db019b7b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:25.742 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.301+0000 7f6db53c4700 1 -- 172.21.15.133:0/1962304234 >> 172.21.15.133:0/1962304234 conn(0x7f6db0100fa0 msgr2=0x7f6db0101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:25.742 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.301+0000 7f6db53c4700 1 -- 172.21.15.133:0/1962304234 shutdown_connections 2023-11-27T15:13:25.742 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:25.301+0000 7f6db53c4700 1 -- 172.21.15.133:0/1962304234 wait complete. 2023-11-27T15:13:25.742 INFO:teuthology.orchestra.run.smithi133.stdout:Enabling cephadm module... 2023-11-27T15:13:26.804 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:26 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3056752000' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-11-27T15:13:26.804 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:26 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: ignoring --setuser ceph since I am not root 2023-11-27T15:13:26.805 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:26 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: ignoring --setgroup ceph since I am not root 2023-11-27T15:13:26.805 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:26 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:26.800+0000 7fa964a90200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-27T15:13:27.016 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.226+0000 7fe88f89a700 1 Processor -- start 2023-11-27T15:13:27.016 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.227+0000 7fe88f89a700 1 -- start start 2023-11-27T15:13:27.016 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.227+0000 7fe88f89a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe88807bde0 0x7fe88807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:27.016 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.227+0000 7fe88f89a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88807a820 con 0x7fe88807bde0 2023-11-27T15:13:27.016 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.227+0000 7fe88d636700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe88807bde0 0x7fe88807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:27.016 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.227+0000 7fe88d636700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe88807bde0 0x7fe88807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49944/0 (socket says 172.21.15.133:49944) 2023-11-27T15:13:27.016 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.227+0000 7fe88d636700 1 -- 172.21.15.133:0/1105324598 learned_addr learned my addr 172.21.15.133:0/1105324598 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:27.016 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.228+0000 7fe88d636700 1 -- 172.21.15.133:0/1105324598 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88807a960 con 0x7fe88807bde0 2023-11-27T15:13:27.016 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.228+0000 7fe88d636700 1 --2- 172.21.15.133:0/1105324598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe88807bde0 0x7fe88807a2e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe8840098a0 tx=0x7fe884009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b1244a7ec7fd95 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:27.016 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.228+0000 7fe87ffff700 1 -- 172.21.15.133:0/1105324598 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe884004160 con 0x7fe88807bde0 2023-11-27T15:13:27.017 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.228+0000 7fe87ffff700 1 -- 172.21.15.133:0/1105324598 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe8840042c0 con 0x7fe88807bde0 2023-11-27T15:13:27.017 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.228+0000 7fe87ffff700 1 -- 172.21.15.133:0/1105324598 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe884011a00 con 0x7fe88807bde0 2023-11-27T15:13:27.017 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.229+0000 7fe88f89a700 1 -- 172.21.15.133:0/1105324598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe88807bde0 msgr2=0x7fe88807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:27.017 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.229+0000 7fe88f89a700 1 --2- 172.21.15.133:0/1105324598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe88807bde0 0x7fe88807a2e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe8840098a0 tx=0x7fe884009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:27.017 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.229+0000 7fe88f89a700 1 -- 172.21.15.133:0/1105324598 shutdown_connections 2023-11-27T15:13:27.017 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.229+0000 7fe88f89a700 1 --2- 172.21.15.133:0/1105324598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe88807bde0 0x7fe88807a2e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:27.017 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.229+0000 7fe88f89a700 1 -- 172.21.15.133:0/1105324598 >> 172.21.15.133:0/1105324598 conn(0x7fe888100fa0 msgr2=0x7fe8881033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:27.017 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.229+0000 7fe88f89a700 1 -- 172.21.15.133:0/1105324598 shutdown_connections 2023-11-27T15:13:27.017 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.229+0000 7fe88f89a700 1 -- 172.21.15.133:0/1105324598 wait complete. 2023-11-27T15:13:27.017 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.230+0000 7fe88f89a700 1 Processor -- start 2023-11-27T15:13:27.017 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.231+0000 7fe88f89a700 1 -- start start 2023-11-27T15:13:27.017 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.231+0000 7fe88f89a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe88807bde0 0x7fe88819b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:27.017 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.231+0000 7fe88f89a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88819bcc0 con 0x7fe88807bde0 2023-11-27T15:13:27.018 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.231+0000 7fe88d636700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe88807bde0 0x7fe88819b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:27.018 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.231+0000 7fe88d636700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe88807bde0 0x7fe88819b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49948/0 (socket says 172.21.15.133:49948) 2023-11-27T15:13:27.018 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.231+0000 7fe88d636700 1 -- 172.21.15.133:0/3056752000 learned_addr learned my addr 172.21.15.133:0/3056752000 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:27.018 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.232+0000 7fe88d636700 1 -- 172.21.15.133:0/3056752000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe884009550 con 0x7fe88807bde0 2023-11-27T15:13:27.018 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.232+0000 7fe88d636700 1 --2- 172.21.15.133:0/3056752000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe88807bde0 0x7fe88819b780 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe884006af0 tx=0x7fe884011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:27.018 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.232+0000 7fe87e7fc700 1 -- 172.21.15.133:0/3056752000 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe884003a40 con 0x7fe88807bde0 2023-11-27T15:13:27.018 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.232+0000 7fe87e7fc700 1 -- 172.21.15.133:0/3056752000 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe884003ba0 con 0x7fe88807bde0 2023-11-27T15:13:27.018 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.232+0000 7fe87e7fc700 1 -- 172.21.15.133:0/3056752000 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe88401a700 con 0x7fe88807bde0 2023-11-27T15:13:27.018 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.232+0000 7fe88f89a700 1 -- 172.21.15.133:0/3056752000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe88810be00 con 0x7fe88807bde0 2023-11-27T15:13:27.018 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.232+0000 7fe88f89a700 1 -- 172.21.15.133:0/3056752000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88810bc70 con 0x7fe88807bde0 2023-11-27T15:13:27.018 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.233+0000 7fe87e7fc700 1 -- 172.21.15.133:0/3056752000 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7fe884024ad0 con 0x7fe88807bde0 2023-11-27T15:13:27.018 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.233+0000 7fe88f89a700 1 -- 172.21.15.133:0/3056752000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe888062120 con 0x7fe88807bde0 2023-11-27T15:13:27.019 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.234+0000 7fe87e7fc700 1 --2- 172.21.15.133:0/3056752000 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7fe8740387d0 0x7fe87403ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:27.019 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.234+0000 7fe87e7fc700 1 -- 172.21.15.133:0/3056752000 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe88404c630 con 0x7fe88807bde0 2023-11-27T15:13:27.019 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.234+0000 7fe88ce35700 1 --2- 172.21.15.133:0/3056752000 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7fe8740387d0 0x7fe87403ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:27.019 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.235+0000 7fe88ce35700 1 --2- 172.21.15.133:0/3056752000 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7fe8740387d0 0x7fe87403ac80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe878006fd0 tx=0x7fe878006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:27.019 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.239+0000 7fe87e7fc700 1 -- 172.21.15.133:0/3056752000 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe884018bd0 con 0x7fe88807bde0 2023-11-27T15:13:27.019 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.488+0000 7fe88f89a700 1 -- 172.21.15.133:0/3056752000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fe88819c450 con 0x7fe88807bde0 2023-11-27T15:13:27.019 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.541+0000 7fe87e7fc700 1 -- 172.21.15.133:0/3056752000 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fe88402d770 con 0x7fe88807bde0 2023-11-27T15:13:27.019 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.543+0000 7fe87e7fc700 1 -- 172.21.15.133:0/3056752000 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fe8840172b0 con 0x7fe88807bde0 2023-11-27T15:13:27.019 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.546+0000 7fe88f89a700 1 -- 172.21.15.133:0/3056752000 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7fe8740387d0 msgr2=0x7fe87403ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:27.019 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.546+0000 7fe88f89a700 1 --2- 172.21.15.133:0/3056752000 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7fe8740387d0 0x7fe87403ac80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe878006fd0 tx=0x7fe878006e40 comp rx=0 tx=0).stop 2023-11-27T15:13:27.019 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.546+0000 7fe88f89a700 1 -- 172.21.15.133:0/3056752000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe88807bde0 msgr2=0x7fe88819b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:27.019 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.546+0000 7fe88f89a700 1 --2- 172.21.15.133:0/3056752000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe88807bde0 0x7fe88819b780 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe884006af0 tx=0x7fe884011ad0 comp rx=0 tx=0).stop 2023-11-27T15:13:27.019 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.546+0000 7fe88f89a700 1 -- 172.21.15.133:0/3056752000 shutdown_connections 2023-11-27T15:13:27.020 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.546+0000 7fe88f89a700 1 --2- 172.21.15.133:0/3056752000 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7fe8740387d0 0x7fe87403ac80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:27.020 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.546+0000 7fe88f89a700 1 --2- 172.21.15.133:0/3056752000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe88807bde0 0x7fe88819b780 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:27.020 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.546+0000 7fe88f89a700 1 -- 172.21.15.133:0/3056752000 >> 172.21.15.133:0/3056752000 conn(0x7fe888100fa0 msgr2=0x7fe888101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:27.020 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.546+0000 7fe88f89a700 1 -- 172.21.15.133:0/3056752000 shutdown_connections 2023-11-27T15:13:27.020 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:26.546+0000 7fe88f89a700 1 -- 172.21.15.133:0/3056752000 wait complete. 2023-11-27T15:13:27.163 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:26 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:26.905+0000 7fa964a90200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-27T15:13:27.163 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:27 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:27.025+0000 7fa964a90200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-27T15:13:27.527 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:27 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:27.158+0000 7fa964a90200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-27T15:13:27.528 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:27 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:27.389+0000 7fa964a90200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-27T15:13:27.850 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:27 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3056752000' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-11-27T15:13:27.850 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:27 smithi133 ceph-mon[97860]: mgrmap e5: x(active, since 8s) 2023-11-27T15:13:28.101 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:27 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:27.845+0000 7fa964a90200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-27T15:13:28.102 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:27 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:27.969+0000 7fa964a90200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-27T15:13:28.407 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:13:28.407 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-11-27T15:13:28.408 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": true, 2023-11-27T15:13:28.408 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-11-27T15:13:28.408 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-11-27T15:13:28.408 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:13:28.408 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.556+0000 7f92618e8700 1 Processor -- start 2023-11-27T15:13:28.408 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.556+0000 7f92618e8700 1 -- start start 2023-11-27T15:13:28.408 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.556+0000 7f92618e8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f925c07bde0 0x7f925c07a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:28.408 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.556+0000 7f92618e8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c07a820 con 0x7f925c07bde0 2023-11-27T15:13:28.409 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.557+0000 7f925affd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f925c07bde0 0x7f925c07a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:28.409 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.557+0000 7f925affd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f925c07bde0 0x7f925c07a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49968/0 (socket says 172.21.15.133:49968) 2023-11-27T15:13:28.409 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.557+0000 7f925affd700 1 -- 172.21.15.133:0/3876774548 learned_addr learned my addr 172.21.15.133:0/3876774548 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:28.409 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.557+0000 7f925affd700 1 -- 172.21.15.133:0/3876774548 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925c07a960 con 0x7f925c07bde0 2023-11-27T15:13:28.409 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.557+0000 7f925affd700 1 --2- 172.21.15.133:0/3876774548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f925c07bde0 0x7f925c07a2e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f924c0098a0 tx=0x7f924c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff42b8a2edc8a016 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:28.409 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.558+0000 7f9259ffb700 1 -- 172.21.15.133:0/3876774548 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f924c004160 con 0x7f925c07bde0 2023-11-27T15:13:28.409 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.558+0000 7f9259ffb700 1 -- 172.21.15.133:0/3876774548 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f924c0042c0 con 0x7f925c07bde0 2023-11-27T15:13:28.409 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.558+0000 7f9259ffb700 1 -- 172.21.15.133:0/3876774548 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f924c011a00 con 0x7f925c07bde0 2023-11-27T15:13:28.409 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.558+0000 7f92618e8700 1 -- 172.21.15.133:0/3876774548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f925c07bde0 msgr2=0x7f925c07a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:28.410 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.558+0000 7f92618e8700 1 --2- 172.21.15.133:0/3876774548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f925c07bde0 0x7f925c07a2e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f924c0098a0 tx=0x7f924c009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:28.410 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.558+0000 7f92618e8700 1 -- 172.21.15.133:0/3876774548 shutdown_connections 2023-11-27T15:13:28.410 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.558+0000 7f92618e8700 1 --2- 172.21.15.133:0/3876774548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f925c07bde0 0x7f925c07a2e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:28.410 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.558+0000 7f92618e8700 1 -- 172.21.15.133:0/3876774548 >> 172.21.15.133:0/3876774548 conn(0x7f925c100fa0 msgr2=0x7f925c1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:28.410 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.558+0000 7f92618e8700 1 -- 172.21.15.133:0/3876774548 shutdown_connections 2023-11-27T15:13:28.410 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.559+0000 7f92618e8700 1 -- 172.21.15.133:0/3876774548 wait complete. 2023-11-27T15:13:28.410 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.559+0000 7f92618e8700 1 Processor -- start 2023-11-27T15:13:28.410 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.560+0000 7f92618e8700 1 -- start start 2023-11-27T15:13:28.410 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.560+0000 7f92618e8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f925c07bde0 0x7f925c19b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:28.410 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.560+0000 7f92618e8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c19bcf0 con 0x7f925c07bde0 2023-11-27T15:13:28.410 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.560+0000 7f925affd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f925c07bde0 0x7f925c19b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:28.410 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.560+0000 7f925affd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f925c07bde0 0x7f925c19b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49974/0 (socket says 172.21.15.133:49974) 2023-11-27T15:13:28.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.560+0000 7f925affd700 1 -- 172.21.15.133:0/326907885 learned_addr learned my addr 172.21.15.133:0/326907885 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:28.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.560+0000 7f925affd700 1 -- 172.21.15.133:0/326907885 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c009550 con 0x7f925c07bde0 2023-11-27T15:13:28.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.561+0000 7f925affd700 1 --2- 172.21.15.133:0/326907885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f925c07bde0 0x7f925c19b7b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f924c006af0 tx=0x7f924c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:28.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.561+0000 7f92608e6700 1 -- 172.21.15.133:0/326907885 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f924c003a40 con 0x7f925c07bde0 2023-11-27T15:13:28.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.561+0000 7f92608e6700 1 -- 172.21.15.133:0/326907885 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f924c003ba0 con 0x7f925c07bde0 2023-11-27T15:13:28.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.561+0000 7f92608e6700 1 -- 172.21.15.133:0/326907885 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f924c01a700 con 0x7f925c07bde0 2023-11-27T15:13:28.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.561+0000 7f92618e8700 1 -- 172.21.15.133:0/326907885 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925c10be00 con 0x7f925c07bde0 2023-11-27T15:13:28.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.561+0000 7f92618e8700 1 -- 172.21.15.133:0/326907885 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925c10bc70 con 0x7f925c07bde0 2023-11-27T15:13:28.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.562+0000 7f92608e6700 1 -- 172.21.15.133:0/326907885 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f924c024430 con 0x7f925c07bde0 2023-11-27T15:13:28.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.562+0000 7f92618e8700 1 -- 172.21.15.133:0/326907885 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f925c062120 con 0x7f925c07bde0 2023-11-27T15:13:28.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.563+0000 7f92608e6700 1 --2- 172.21.15.133:0/326907885 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7f92440387d0 0x7f924403ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:28.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.563+0000 7f92608e6700 1 -- 172.21.15.133:0/326907885 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f924c04c6d0 con 0x7f925c07bde0 2023-11-27T15:13:28.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.563+0000 7f925a7fc700 1 -- 172.21.15.133:0/326907885 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7f92440387d0 msgr2=0x7f924403ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/3149426454 2023-11-27T15:13:28.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.563+0000 7f925a7fc700 1 --2- 172.21.15.133:0/326907885 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7f92440387d0 0x7f924403ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:13:28.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.568+0000 7f92608e6700 1 -- 172.21.15.133:0/326907885 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f924c0189a0 con 0x7f925c07bde0 2023-11-27T15:13:28.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.764+0000 7f925a7fc700 1 -- 172.21.15.133:0/326907885 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7f92440387d0 msgr2=0x7f924403ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/3149426454 2023-11-27T15:13:28.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.764+0000 7f925a7fc700 1 --2- 172.21.15.133:0/326907885 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7f92440387d0 0x7f924403ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-27T15:13:28.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.902+0000 7f92618e8700 1 -- 172.21.15.133:0/326907885 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f925c19c230 con 0x7f925c07bde0 2023-11-27T15:13:28.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.902+0000 7f92608e6700 1 -- 172.21.15.133:0/326907885 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f924c00be80 con 0x7f925c07bde0 2023-11-27T15:13:28.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.905+0000 7f92618e8700 1 -- 172.21.15.133:0/326907885 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7f92440387d0 msgr2=0x7f924403ac80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:13:28.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.905+0000 7f92618e8700 1 --2- 172.21.15.133:0/326907885 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7f92440387d0 0x7f924403ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:28.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.905+0000 7f92618e8700 1 -- 172.21.15.133:0/326907885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f925c07bde0 msgr2=0x7f925c19b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:28.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.905+0000 7f92618e8700 1 --2- 172.21.15.133:0/326907885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f925c07bde0 0x7f925c19b7b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f924c006af0 tx=0x7f924c011ad0 comp rx=0 tx=0).stop 2023-11-27T15:13:28.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.905+0000 7f92618e8700 1 -- 172.21.15.133:0/326907885 shutdown_connections 2023-11-27T15:13:28.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.905+0000 7f92618e8700 1 --2- 172.21.15.133:0/326907885 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7f92440387d0 0x7f924403ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:28.413 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.905+0000 7f92618e8700 1 --2- 172.21.15.133:0/326907885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f925c07bde0 0x7f925c19b7b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:28.413 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.905+0000 7f92618e8700 1 -- 172.21.15.133:0/326907885 >> 172.21.15.133:0/326907885 conn(0x7f925c100fa0 msgr2=0x7f925c101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:28.413 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.905+0000 7f92618e8700 1 -- 172.21.15.133:0/326907885 shutdown_connections 2023-11-27T15:13:28.413 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:27.905+0000 7f92618e8700 1 -- 172.21.15.133:0/326907885 wait complete. 2023-11-27T15:13:28.413 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for the mgr to restart... 2023-11-27T15:13:28.413 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mgr epoch 5... 2023-11-27T15:13:28.528 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:28 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:28.142+0000 7fa964a90200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-27T15:13:28.528 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:28 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:28.267+0000 7fa964a90200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-27T15:13:28.847 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:28 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/326907885' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-11-27T15:13:29.277 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:28 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:28.843+0000 7fa964a90200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-27T15:13:30.028 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:29 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:29.581+0000 7fa964a90200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-27T15:13:30.028 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:29 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:29.790+0000 7fa964a90200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-27T15:13:31.027 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:30 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:30.723+0000 7fa964a90200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:13:31.028 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:30 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:30.840+0000 7fa964a90200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-27T15:13:31.423 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:31 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:31.061+0000 7fa964a90200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:13:31.701 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:31 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:31.419+0000 7fa964a90200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-27T15:13:32.027 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:31 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:31.696+0000 7fa964a90200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-27T15:13:32.028 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:31 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:31.886+0000 7fa964a90200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-27T15:13:32.527 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:32 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:32.226+0000 7fa964a90200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-27T15:13:33.527 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:33 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:33.148+0000 7fa964a90200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-27T15:13:33.528 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:33 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:33.317+0000 7fa964a90200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-27T15:13:33.984 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:33 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:33.560+0000 7fa964a90200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-27T15:13:33.984 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:33 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:33.707+0000 7fa964a90200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-27T15:13:34.277 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:33 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:33.979+0000 7fa964a90200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-27T15:13:35.527 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:35 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:35.031+0000 7fa964a90200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-27T15:13:35.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:35 smithi133 ceph-mon[97860]: Active manager daemon x restarted 2023-11-27T15:13:35.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:35 smithi133 ceph-mon[97860]: Activating manager daemon x 2023-11-27T15:13:35.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:35 smithi133 ceph-mon[97860]: osdmap e2: 0 total, 0 up, 0 in 2023-11-27T15:13:35.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:35 smithi133 ceph-mon[97860]: mgrmap e6: x(active, starting, since 0.00412688s) 2023-11-27T15:13:35.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:35 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:13:35.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:35 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-27T15:13:35.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:35 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-27T15:13:35.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:35 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-27T15:13:35.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:35 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-27T15:13:35.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:35 smithi133 ceph-mon[97860]: Manager daemon x is now available 2023-11-27T15:13:35.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:35 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' 2023-11-27T15:13:35.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:35 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' 2023-11-27T15:13:35.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:35 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:13:35.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:35 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:13:35.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:35 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:13:36.447 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:36 smithi133 ceph-mon[97860]: Found migration_current of "None". Setting to last migration. 2023-11-27T15:13:36.447 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:36 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-27T15:13:36.448 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:36 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-27T15:13:36.448 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:36 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' 2023-11-27T15:13:36.448 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:36 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' 2023-11-27T15:13:36.448 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:36 smithi133 ceph-mon[97860]: [27/Nov/2023:15:13:35] ENGINE Bus STARTING 2023-11-27T15:13:36.448 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:36 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:13:36.448 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:36 smithi133 ceph-mon[97860]: mgrmap e7: x(active, since 1.00656s) 2023-11-27T15:13:36.503 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:13:36.503 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-11-27T15:13:36.503 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "initialized": true 2023-11-27T15:13:36.503 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:13:36.503 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.943+0000 7fe1a8920700 1 Processor -- start 2023-11-27T15:13:36.504 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.943+0000 7fe1a8920700 1 -- start start 2023-11-27T15:13:36.504 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.943+0000 7fe1a8920700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1a4106400 0x7fe1a41087e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:36.504 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.943+0000 7fe1a8920700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a4079d80 con 0x7fe1a4106400 2023-11-27T15:13:36.504 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.944+0000 7fe1a259c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1a4106400 0x7fe1a41087e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:36.504 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.944+0000 7fe1a259c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1a4106400 0x7fe1a41087e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49980/0 (socket says 172.21.15.133:49980) 2023-11-27T15:13:36.504 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.944+0000 7fe1a259c700 1 -- 172.21.15.133:0/1009719577 learned_addr learned my addr 172.21.15.133:0/1009719577 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:36.504 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.944+0000 7fe1a259c700 1 -- 172.21.15.133:0/1009719577 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a4079ec0 con 0x7fe1a4106400 2023-11-27T15:13:36.504 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.944+0000 7fe1a259c700 1 --2- 172.21.15.133:0/1009719577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1a4106400 0x7fe1a41087e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe1940098a0 tx=0x7fe194009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81bcd513a1a0e2c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:36.504 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.945+0000 7fe1a159a700 1 -- 172.21.15.133:0/1009719577 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe194004160 con 0x7fe1a4106400 2023-11-27T15:13:36.504 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.945+0000 7fe1a159a700 1 -- 172.21.15.133:0/1009719577 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe1940042c0 con 0x7fe1a4106400 2023-11-27T15:13:36.504 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.945+0000 7fe1a159a700 1 -- 172.21.15.133:0/1009719577 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe194011a00 con 0x7fe1a4106400 2023-11-27T15:13:36.504 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.945+0000 7fe1a8920700 1 -- 172.21.15.133:0/1009719577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1a4106400 msgr2=0x7fe1a41087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:36.504 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.945+0000 7fe1a8920700 1 --2- 172.21.15.133:0/1009719577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1a4106400 0x7fe1a41087e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe1940098a0 tx=0x7fe194009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:36.505 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.946+0000 7fe1a8920700 1 -- 172.21.15.133:0/1009719577 shutdown_connections 2023-11-27T15:13:36.505 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.946+0000 7fe1a8920700 1 --2- 172.21.15.133:0/1009719577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1a4106400 0x7fe1a41087e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:36.505 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.946+0000 7fe1a8920700 1 -- 172.21.15.133:0/1009719577 >> 172.21.15.133:0/1009719577 conn(0x7fe1a40ffe40 msgr2=0x7fe1a4102270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:36.505 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.946+0000 7fe1a8920700 1 -- 172.21.15.133:0/1009719577 shutdown_connections 2023-11-27T15:13:36.505 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.946+0000 7fe1a8920700 1 -- 172.21.15.133:0/1009719577 wait complete. 2023-11-27T15:13:36.505 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.947+0000 7fe1a8920700 1 Processor -- start 2023-11-27T15:13:36.505 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.947+0000 7fe1a8920700 1 -- start start 2023-11-27T15:13:36.505 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.947+0000 7fe1a8920700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1a4106400 0x7fe1a419b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:36.505 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.947+0000 7fe1a8920700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a419bc70 con 0x7fe1a4106400 2023-11-27T15:13:36.505 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.947+0000 7fe1a259c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1a4106400 0x7fe1a419b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:36.505 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.947+0000 7fe1a259c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1a4106400 0x7fe1a419b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:49990/0 (socket says 172.21.15.133:49990) 2023-11-27T15:13:36.505 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.947+0000 7fe1a259c700 1 -- 172.21.15.133:0/1181357263 learned_addr learned my addr 172.21.15.133:0/1181357263 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:36.505 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.948+0000 7fe1a259c700 1 -- 172.21.15.133:0/1181357263 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe194009550 con 0x7fe1a4106400 2023-11-27T15:13:36.506 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.948+0000 7fe1a259c700 1 --2- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1a4106400 0x7fe1a419b730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe194006af0 tx=0x7fe194011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:36.506 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.948+0000 7fe1937fe700 1 -- 172.21.15.133:0/1181357263 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe194003a40 con 0x7fe1a4106400 2023-11-27T15:13:36.506 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.948+0000 7fe1937fe700 1 -- 172.21.15.133:0/1181357263 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe194003ba0 con 0x7fe1a4106400 2023-11-27T15:13:36.506 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.948+0000 7fe1a8920700 1 -- 172.21.15.133:0/1181357263 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a4109b80 con 0x7fe1a4106400 2023-11-27T15:13:36.506 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.948+0000 7fe1937fe700 1 -- 172.21.15.133:0/1181357263 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe19401a700 con 0x7fe1a4106400 2023-11-27T15:13:36.506 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.948+0000 7fe1a8920700 1 -- 172.21.15.133:0/1181357263 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a41099f0 con 0x7fe1a4106400 2023-11-27T15:13:36.506 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.950+0000 7fe1937fe700 1 -- 172.21.15.133:0/1181357263 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fe19401a980 con 0x7fe1a4106400 2023-11-27T15:13:36.506 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.950+0000 7fe1937fe700 1 --2- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7fe18c0387d0 0x7fe18c03ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:36.506 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.950+0000 7fe1937fe700 1 -- 172.21.15.133:0/1181357263 --> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe18c03b390 con 0x7fe18c0387d0 2023-11-27T15:13:36.506 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.950+0000 7fe1937fe700 1 -- 172.21.15.133:0/1181357263 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe19404c8d0 con 0x7fe1a4106400 2023-11-27T15:13:36.506 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.950+0000 7fe1a1d9b700 1 -- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7fe18c0387d0 msgr2=0x7fe18c03ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/3149426454 2023-11-27T15:13:36.506 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:28.950+0000 7fe1a1d9b700 1 --2- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7fe18c0387d0 0x7fe18c03ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:13:36.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:29.151+0000 7fe1a1d9b700 1 -- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7fe18c0387d0 msgr2=0x7fe18c03ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/3149426454 2023-11-27T15:13:36.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:29.151+0000 7fe1a1d9b700 1 --2- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7fe18c0387d0 0x7fe18c03ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-27T15:13:36.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:29.552+0000 7fe1a1d9b700 1 -- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7fe18c0387d0 msgr2=0x7fe18c03ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/3149426454 2023-11-27T15:13:36.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:29.552+0000 7fe1a1d9b700 1 --2- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7fe18c0387d0 0x7fe18c03ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-27T15:13:36.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:30.353+0000 7fe1a1d9b700 1 -- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7fe18c0387d0 msgr2=0x7fe18c03ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/3149426454 2023-11-27T15:13:36.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:30.353+0000 7fe1a1d9b700 1 --2- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7fe18c0387d0 0x7fe18c03ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-27T15:13:36.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:31.954+0000 7fe1a1d9b700 1 -- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7fe18c0387d0 msgr2=0x7fe18c03ac80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/3149426454 2023-11-27T15:13:36.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:31.955+0000 7fe1a1d9b700 1 --2- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7fe18c0387d0 0x7fe18c03ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-11-27T15:13:36.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:35.036+0000 7fe1937fe700 1 -- 172.21.15.133:0/1181357263 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7fe19402d960 con 0x7fe1a4106400 2023-11-27T15:13:36.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:35.036+0000 7fe1937fe700 1 -- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7fe18c0387d0 msgr2=0x7fe18c03ac80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:13:36.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:35.036+0000 7fe1937fe700 1 --2- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7fe18c0387d0 0x7fe18c03ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:36.507 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:36.037+0000 7fe1937fe700 1 -- 172.21.15.133:0/1181357263 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7fe194008d10 con 0x7fe1a4106400 2023-11-27T15:13:36.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:36.038+0000 7fe1937fe700 1 --2- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fe18c03bfb0 0x7fe18c03e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:36.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:36.038+0000 7fe1937fe700 1 -- 172.21.15.133:0/1181357263 --> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe18c03b390 con 0x7fe18c03bfb0 2023-11-27T15:13:36.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:36.038+0000 7fe1a1d9b700 1 --2- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fe18c03bfb0 0x7fe18c03e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:36.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:36.039+0000 7fe1a1d9b700 1 --2- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fe18c03bfb0 0x7fe18c03e390 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe198003a10 tx=0x7fe198009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:36.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:36.041+0000 7fe1937fe700 1 -- 172.21.15.133:0/1181357263 <== mgr.14118 v2:172.21.15.133:6800/2374615220 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fe18c03b390 con 0x7fe18c03bfb0 2023-11-27T15:13:36.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:36.048+0000 7fe1a8920700 1 -- 172.21.15.133:0/1181357263 --> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fe1a419bdb0 con 0x7fe18c03bfb0 2023-11-27T15:13:36.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:36.048+0000 7fe1937fe700 1 -- 172.21.15.133:0/1181357263 <== mgr.14118 v2:172.21.15.133:6800/2374615220 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fe1980181b0 con 0x7fe18c03bfb0 2023-11-27T15:13:36.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:36.048+0000 7fe1a8920700 1 -- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fe18c03bfb0 msgr2=0x7fe18c03e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:36.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:36.048+0000 7fe1a8920700 1 --2- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fe18c03bfb0 0x7fe18c03e390 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe198003a10 tx=0x7fe198009290 comp rx=0 tx=0).stop 2023-11-27T15:13:36.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:36.048+0000 7fe1a8920700 1 -- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1a4106400 msgr2=0x7fe1a419b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:36.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:36.048+0000 7fe1a8920700 1 --2- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1a4106400 0x7fe1a419b730 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe194006af0 tx=0x7fe194011ad0 comp rx=0 tx=0).stop 2023-11-27T15:13:36.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:36.049+0000 7fe1a8920700 1 -- 172.21.15.133:0/1181357263 shutdown_connections 2023-11-27T15:13:36.508 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:36.049+0000 7fe1a8920700 1 --2- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fe18c03bfb0 0x7fe18c03e390 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:36.509 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:36.049+0000 7fe1a8920700 1 --2- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1a4106400 0x7fe1a419b730 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:36.509 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:36.049+0000 7fe1a8920700 1 --2- 172.21.15.133:0/1181357263 >> [v2:172.21.15.133:6800/3149426454,v1:172.21.15.133:6801/3149426454] conn(0x7fe18c0387d0 0x7fe18c03ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:36.509 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:36.049+0000 7fe1a8920700 1 -- 172.21.15.133:0/1181357263 >> 172.21.15.133:0/1181357263 conn(0x7fe1a40ffe40 msgr2=0x7fe1a4100b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:36.509 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:36.049+0000 7fe1a8920700 1 -- 172.21.15.133:0/1181357263 shutdown_connections 2023-11-27T15:13:36.509 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:36.049+0000 7fe1a8920700 1 -- 172.21.15.133:0/1181357263 wait complete. 2023-11-27T15:13:36.509 INFO:teuthology.orchestra.run.smithi133.stdout:mgr epoch 5 is available 2023-11-27T15:13:36.509 INFO:teuthology.orchestra.run.smithi133.stdout:Setting orchestrator backend to cephadm... 2023-11-27T15:13:37.472 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:37 smithi133 ceph-mon[97860]: [27/Nov/2023:15:13:35] ENGINE Serving on https://172.21.15.133:7150 2023-11-27T15:13:37.472 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:37 smithi133 ceph-mon[97860]: [27/Nov/2023:15:13:35] ENGINE Serving on http://172.21.15.133:8765 2023-11-27T15:13:37.472 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:37 smithi133 ceph-mon[97860]: [27/Nov/2023:15:13:35] ENGINE Bus STARTED 2023-11-27T15:13:37.472 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:37 smithi133 ceph-mon[97860]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-11-27T15:13:37.472 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:37 smithi133 ceph-mon[97860]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-11-27T15:13:37.782 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.034+0000 7fc9262eb700 1 Processor -- start 2023-11-27T15:13:37.782 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.035+0000 7fc9262eb700 1 -- start start 2023-11-27T15:13:37.783 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.035+0000 7fc9262eb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9201047f0 0x7fc920104c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:37.783 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.035+0000 7fc9262eb700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc920105140 con 0x7fc9201047f0 2023-11-27T15:13:37.783 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.035+0000 7fc91ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9201047f0 0x7fc920104c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:37.783 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.035+0000 7fc91ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9201047f0 0x7fc920104c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54716/0 (socket says 172.21.15.133:54716) 2023-11-27T15:13:37.783 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.035+0000 7fc91ffff700 1 -- 172.21.15.133:0/1447372365 learned_addr learned my addr 172.21.15.133:0/1447372365 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:37.783 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.036+0000 7fc91ffff700 1 -- 172.21.15.133:0/1447372365 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc920105280 con 0x7fc9201047f0 2023-11-27T15:13:37.783 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.036+0000 7fc91ffff700 1 --2- 172.21.15.133:0/1447372365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9201047f0 0x7fc920104c00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc9100098a0 tx=0x7fc910009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c372c9ec3f571a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:37.783 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.036+0000 7fc91effd700 1 -- 172.21.15.133:0/1447372365 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc910004160 con 0x7fc9201047f0 2023-11-27T15:13:37.783 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.036+0000 7fc91effd700 1 -- 172.21.15.133:0/1447372365 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc9100042c0 con 0x7fc9201047f0 2023-11-27T15:13:37.783 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.036+0000 7fc91effd700 1 -- 172.21.15.133:0/1447372365 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc910011a00 con 0x7fc9201047f0 2023-11-27T15:13:37.783 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.037+0000 7fc9262eb700 1 -- 172.21.15.133:0/1447372365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9201047f0 msgr2=0x7fc920104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:37.783 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.037+0000 7fc9262eb700 1 --2- 172.21.15.133:0/1447372365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9201047f0 0x7fc920104c00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc9100098a0 tx=0x7fc910009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:37.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.037+0000 7fc9262eb700 1 -- 172.21.15.133:0/1447372365 shutdown_connections 2023-11-27T15:13:37.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.037+0000 7fc9262eb700 1 --2- 172.21.15.133:0/1447372365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9201047f0 0x7fc920104c00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:37.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.037+0000 7fc9262eb700 1 -- 172.21.15.133:0/1447372365 >> 172.21.15.133:0/1447372365 conn(0x7fc9200ffe40 msgr2=0x7fc920102270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:37.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.037+0000 7fc9262eb700 1 -- 172.21.15.133:0/1447372365 shutdown_connections 2023-11-27T15:13:37.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.037+0000 7fc9262eb700 1 -- 172.21.15.133:0/1447372365 wait complete. 2023-11-27T15:13:37.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.038+0000 7fc9262eb700 1 Processor -- start 2023-11-27T15:13:37.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.039+0000 7fc9262eb700 1 -- start start 2023-11-27T15:13:37.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.039+0000 7fc9262eb700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9201047f0 0x7fc92019b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:37.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.039+0000 7fc9262eb700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92019bc70 con 0x7fc9201047f0 2023-11-27T15:13:37.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.039+0000 7fc91ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9201047f0 0x7fc92019b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:37.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.039+0000 7fc91ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9201047f0 0x7fc92019b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54724/0 (socket says 172.21.15.133:54724) 2023-11-27T15:13:37.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.039+0000 7fc91ffff700 1 -- 172.21.15.133:0/3084660358 learned_addr learned my addr 172.21.15.133:0/3084660358 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:37.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.040+0000 7fc91ffff700 1 -- 172.21.15.133:0/3084660358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc910009550 con 0x7fc9201047f0 2023-11-27T15:13:37.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.040+0000 7fc91ffff700 1 --2- 172.21.15.133:0/3084660358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9201047f0 0x7fc92019b730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc910006af0 tx=0x7fc910011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:37.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.040+0000 7fc91d7fa700 1 -- 172.21.15.133:0/3084660358 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc910003a40 con 0x7fc9201047f0 2023-11-27T15:13:37.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.040+0000 7fc91d7fa700 1 -- 172.21.15.133:0/3084660358 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc910003ba0 con 0x7fc9201047f0 2023-11-27T15:13:37.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.040+0000 7fc91d7fa700 1 -- 172.21.15.133:0/3084660358 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc91001a6e0 con 0x7fc9201047f0 2023-11-27T15:13:37.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.040+0000 7fc9262eb700 1 -- 172.21.15.133:0/3084660358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc920106840 con 0x7fc9201047f0 2023-11-27T15:13:37.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.040+0000 7fc9262eb700 1 -- 172.21.15.133:0/3084660358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9201066b0 con 0x7fc9201047f0 2023-11-27T15:13:37.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.041+0000 7fc91d7fa700 1 -- 172.21.15.133:0/3084660358 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7fc910024a60 con 0x7fc9201047f0 2023-11-27T15:13:37.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.041+0000 7fc9262eb700 1 -- 172.21.15.133:0/3084660358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc920108cc0 con 0x7fc9201047f0 2023-11-27T15:13:37.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.042+0000 7fc91d7fa700 1 --2- 172.21.15.133:0/3084660358 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fc9080386a0 0x7fc90803ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:37.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.042+0000 7fc91d7fa700 1 -- 172.21.15.133:0/3084660358 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc91004c630 con 0x7fc9201047f0 2023-11-27T15:13:37.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.042+0000 7fc91f7fe700 1 --2- 172.21.15.133:0/3084660358 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fc9080386a0 0x7fc90803ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:37.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.042+0000 7fc91f7fe700 1 --2- 172.21.15.133:0/3084660358 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fc9080386a0 0x7fc90803ab50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc914006fd0 tx=0x7fc914006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:37.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.047+0000 7fc91d7fa700 1 -- 172.21.15.133:0/3084660358 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc910024430 con 0x7fc9201047f0 2023-11-27T15:13:37.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.257+0000 7fc9262eb700 1 -- 172.21.15.133:0/3084660358 --> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fc920101e80 con 0x7fc9080386a0 2023-11-27T15:13:37.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.266+0000 7fc91d7fa700 1 -- 172.21.15.133:0/3084660358 <== mgr.14118 v2:172.21.15.133:6800/2374615220 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc920101e80 con 0x7fc9080386a0 2023-11-27T15:13:37.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.269+0000 7fc9262eb700 1 -- 172.21.15.133:0/3084660358 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fc9080386a0 msgr2=0x7fc90803ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:37.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.269+0000 7fc9262eb700 1 --2- 172.21.15.133:0/3084660358 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fc9080386a0 0x7fc90803ab50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc914006fd0 tx=0x7fc914006e40 comp rx=0 tx=0).stop 2023-11-27T15:13:37.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.269+0000 7fc9262eb700 1 -- 172.21.15.133:0/3084660358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9201047f0 msgr2=0x7fc92019b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:37.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.269+0000 7fc9262eb700 1 --2- 172.21.15.133:0/3084660358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9201047f0 0x7fc92019b730 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc910006af0 tx=0x7fc910011ad0 comp rx=0 tx=0).stop 2023-11-27T15:13:37.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.269+0000 7fc9262eb700 1 -- 172.21.15.133:0/3084660358 shutdown_connections 2023-11-27T15:13:37.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.269+0000 7fc9262eb700 1 --2- 172.21.15.133:0/3084660358 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fc9080386a0 0x7fc90803ab50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:37.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.269+0000 7fc9262eb700 1 --2- 172.21.15.133:0/3084660358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9201047f0 0x7fc92019b730 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:37.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.269+0000 7fc9262eb700 1 -- 172.21.15.133:0/3084660358 >> 172.21.15.133:0/3084660358 conn(0x7fc9200ffe40 msgr2=0x7fc920079920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:37.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.269+0000 7fc9262eb700 1 -- 172.21.15.133:0/3084660358 shutdown_connections 2023-11-27T15:13:37.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:37.269+0000 7fc9262eb700 1 -- 172.21.15.133:0/3084660358 wait complete. 2023-11-27T15:13:38.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:38 smithi133 ceph-mon[97860]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:13:39.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:38 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' 2023-11-27T15:13:39.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:38 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:13:39.211 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:38 smithi133 ceph-mon[97860]: mgrmap e8: x(active, since 2s) 2023-11-27T15:13:39.216 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout value unchanged 2023-11-27T15:13:39.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.318+0000 7f5f0602c700 1 Processor -- start 2023-11-27T15:13:39.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.318+0000 7f5f0602c700 1 -- start start 2023-11-27T15:13:39.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.318+0000 7f5f0602c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0007be20 0x7f5f0007a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:39.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.318+0000 7f5f0602c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0007a860 con 0x7f5f0007be20 2023-11-27T15:13:39.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.319+0000 7f5eff7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0007be20 0x7f5f0007a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:39.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.319+0000 7f5eff7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0007be20 0x7f5f0007a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54728/0 (socket says 172.21.15.133:54728) 2023-11-27T15:13:39.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.319+0000 7f5eff7fe700 1 -- 172.21.15.133:0/74706403 learned_addr learned my addr 172.21.15.133:0/74706403 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:39.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.319+0000 7f5eff7fe700 1 -- 172.21.15.133:0/74706403 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0007a9a0 con 0x7f5f0007be20 2023-11-27T15:13:39.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.319+0000 7f5eff7fe700 1 --2- 172.21.15.133:0/74706403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0007be20 0x7f5f0007a320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00098a0 tx=0x7f5ef0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d59085b6b05997b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:39.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.319+0000 7f5efe7fc700 1 -- 172.21.15.133:0/74706403 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ef0004160 con 0x7f5f0007be20 2023-11-27T15:13:39.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.320+0000 7f5efe7fc700 1 -- 172.21.15.133:0/74706403 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5ef00042c0 con 0x7f5f0007be20 2023-11-27T15:13:39.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.320+0000 7f5efe7fc700 1 -- 172.21.15.133:0/74706403 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ef0011a00 con 0x7f5f0007be20 2023-11-27T15:13:39.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.320+0000 7f5f0602c700 1 -- 172.21.15.133:0/74706403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0007be20 msgr2=0x7f5f0007a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:39.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.320+0000 7f5f0602c700 1 --2- 172.21.15.133:0/74706403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0007be20 0x7f5f0007a320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00098a0 tx=0x7f5ef0009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:39.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.320+0000 7f5f0602c700 1 -- 172.21.15.133:0/74706403 shutdown_connections 2023-11-27T15:13:39.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.320+0000 7f5f0602c700 1 --2- 172.21.15.133:0/74706403 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0007be20 0x7f5f0007a320 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:39.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.320+0000 7f5f0602c700 1 -- 172.21.15.133:0/74706403 >> 172.21.15.133:0/74706403 conn(0x7f5f00100fa0 msgr2=0x7f5f001033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:39.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.320+0000 7f5f0602c700 1 -- 172.21.15.133:0/74706403 shutdown_connections 2023-11-27T15:13:39.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.320+0000 7f5f0602c700 1 -- 172.21.15.133:0/74706403 wait complete. 2023-11-27T15:13:39.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.321+0000 7f5f0602c700 1 Processor -- start 2023-11-27T15:13:39.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.321+0000 7f5f0602c700 1 -- start start 2023-11-27T15:13:39.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.322+0000 7f5f0602c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0007be20 0x7f5f0019b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:39.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.322+0000 7f5f0602c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0019bce0 con 0x7f5f0007be20 2023-11-27T15:13:39.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.322+0000 7f5eff7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0007be20 0x7f5f0019b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:39.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.322+0000 7f5eff7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0007be20 0x7f5f0019b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54738/0 (socket says 172.21.15.133:54738) 2023-11-27T15:13:39.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.322+0000 7f5eff7fe700 1 -- 172.21.15.133:0/3105947967 learned_addr learned my addr 172.21.15.133:0/3105947967 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:39.233 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.322+0000 7f5eff7fe700 1 -- 172.21.15.133:0/3105947967 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef0009550 con 0x7f5f0007be20 2023-11-27T15:13:39.233 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.322+0000 7f5eff7fe700 1 --2- 172.21.15.133:0/3105947967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0007be20 0x7f5f0019b7a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0006af0 tx=0x7f5ef0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:39.233 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.323+0000 7f5efcff9700 1 -- 172.21.15.133:0/3105947967 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ef0003a40 con 0x7f5f0007be20 2023-11-27T15:13:39.233 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.323+0000 7f5efcff9700 1 -- 172.21.15.133:0/3105947967 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5ef0003ba0 con 0x7f5f0007be20 2023-11-27T15:13:39.233 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.323+0000 7f5efcff9700 1 -- 172.21.15.133:0/3105947967 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ef001a700 con 0x7f5f0007be20 2023-11-27T15:13:39.233 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.323+0000 7f5f0602c700 1 -- 172.21.15.133:0/3105947967 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0010bdc0 con 0x7f5f0007be20 2023-11-27T15:13:39.233 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.323+0000 7f5f0602c700 1 -- 172.21.15.133:0/3105947967 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0010bc30 con 0x7f5f0007be20 2023-11-27T15:13:39.233 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.324+0000 7f5efcff9700 1 -- 172.21.15.133:0/3105947967 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f5ef0024ad0 con 0x7f5f0007be20 2023-11-27T15:13:39.233 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.324+0000 7f5f0602c700 1 -- 172.21.15.133:0/3105947967 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f00062120 con 0x7f5f0007be20 2023-11-27T15:13:39.233 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.324+0000 7f5efcff9700 1 --2- 172.21.15.133:0/3105947967 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f5ee8038820 0x7f5ee803acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:39.233 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.324+0000 7f5efcff9700 1 -- 172.21.15.133:0/3105947967 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5ef004c780 con 0x7f5f0007be20 2023-11-27T15:13:39.233 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.324+0000 7f5efeffd700 1 --2- 172.21.15.133:0/3105947967 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f5ee8038820 0x7f5ee803acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:39.233 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.325+0000 7f5efeffd700 1 --2- 172.21.15.133:0/3105947967 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f5ee8038820 0x7f5ee803acd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4006fd0 tx=0x7f5ef4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:39.234 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.329+0000 7f5efcff9700 1 -- 172.21.15.133:0/3105947967 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5ef0018b80 con 0x7f5f0007be20 2023-11-27T15:13:39.234 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.528+0000 7f5f0602c700 1 -- 172.21.15.133:0/3105947967 --> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f00102380 con 0x7f5ee8038820 2023-11-27T15:13:39.234 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.529+0000 7f5efcff9700 1 -- 172.21.15.133:0/3105947967 <== mgr.14118 v2:172.21.15.133:6800/2374615220 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f5f00102380 con 0x7f5ee8038820 2023-11-27T15:13:39.234 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.531+0000 7f5f0602c700 1 -- 172.21.15.133:0/3105947967 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f5ee8038820 msgr2=0x7f5ee803acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:39.234 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.531+0000 7f5f0602c700 1 --2- 172.21.15.133:0/3105947967 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f5ee8038820 0x7f5ee803acd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4006fd0 tx=0x7f5ef4006e40 comp rx=0 tx=0).stop 2023-11-27T15:13:39.234 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.531+0000 7f5f0602c700 1 -- 172.21.15.133:0/3105947967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0007be20 msgr2=0x7f5f0019b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:39.234 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.531+0000 7f5f0602c700 1 --2- 172.21.15.133:0/3105947967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0007be20 0x7f5f0019b7a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0006af0 tx=0x7f5ef0011ad0 comp rx=0 tx=0).stop 2023-11-27T15:13:39.234 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.531+0000 7f5f0602c700 1 -- 172.21.15.133:0/3105947967 shutdown_connections 2023-11-27T15:13:39.234 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.531+0000 7f5f0602c700 1 --2- 172.21.15.133:0/3105947967 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f5ee8038820 0x7f5ee803acd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:39.234 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.531+0000 7f5f0602c700 1 --2- 172.21.15.133:0/3105947967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0007be20 0x7f5f0019b7a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:39.234 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.531+0000 7f5f0602c700 1 -- 172.21.15.133:0/3105947967 >> 172.21.15.133:0/3105947967 conn(0x7f5f00100fa0 msgr2=0x7f5f00101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:39.234 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.531+0000 7f5f0602c700 1 -- 172.21.15.133:0/3105947967 shutdown_connections 2023-11-27T15:13:39.234 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:38.531+0000 7f5f0602c700 1 -- 172.21.15.133:0/3105947967 wait complete. 2023-11-27T15:13:39.235 INFO:teuthology.orchestra.run.smithi133.stdout:Generating ssh key... 2023-11-27T15:13:39.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:39 smithi133 ceph-mon[97860]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:13:40.439 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.498+0000 7fa42ddcc700 1 Processor -- start 2023-11-27T15:13:40.439 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.498+0000 7fa42ddcc700 1 -- start start 2023-11-27T15:13:40.439 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.498+0000 7fa42ddcc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42807a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:40.439 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.498+0000 7fa42ddcc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42807a840 con 0x7fa42807be00 2023-11-27T15:13:40.439 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.499+0000 7fa4277fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42807a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:40.440 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.499+0000 7fa4277fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42807a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54740/0 (socket says 172.21.15.133:54740) 2023-11-27T15:13:40.440 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.499+0000 7fa4277fe700 1 -- 172.21.15.133:0/2660080020 learned_addr learned my addr 172.21.15.133:0/2660080020 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:40.440 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.499+0000 7fa4277fe700 1 -- 172.21.15.133:0/2660080020 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42807a980 con 0x7fa42807be00 2023-11-27T15:13:40.440 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.499+0000 7fa4277fe700 1 --2- 172.21.15.133:0/2660080020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42807a300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa4180098a0 tx=0x7fa418009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91730a71699b7a0c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:40.440 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.499+0000 7fa4267fc700 1 -- 172.21.15.133:0/2660080020 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa418004160 con 0x7fa42807be00 2023-11-27T15:13:40.440 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.499+0000 7fa4267fc700 1 -- 172.21.15.133:0/2660080020 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa4180042c0 con 0x7fa42807be00 2023-11-27T15:13:40.440 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.499+0000 7fa4267fc700 1 -- 172.21.15.133:0/2660080020 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa418011a00 con 0x7fa42807be00 2023-11-27T15:13:40.440 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.500+0000 7fa42ddcc700 1 -- 172.21.15.133:0/2660080020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 msgr2=0x7fa42807a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:40.440 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.500+0000 7fa42ddcc700 1 --2- 172.21.15.133:0/2660080020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42807a300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa4180098a0 tx=0x7fa418009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:40.440 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.500+0000 7fa42ddcc700 1 -- 172.21.15.133:0/2660080020 shutdown_connections 2023-11-27T15:13:40.440 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.500+0000 7fa42ddcc700 1 --2- 172.21.15.133:0/2660080020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42807a300 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:40.441 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.500+0000 7fa42ddcc700 1 -- 172.21.15.133:0/2660080020 >> 172.21.15.133:0/2660080020 conn(0x7fa428100fa0 msgr2=0x7fa4281033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:40.441 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.500+0000 7fa42ddcc700 1 -- 172.21.15.133:0/2660080020 shutdown_connections 2023-11-27T15:13:40.441 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.500+0000 7fa42ddcc700 1 -- 172.21.15.133:0/2660080020 wait complete. 2023-11-27T15:13:40.441 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.502+0000 7fa42ddcc700 1 Processor -- start 2023-11-27T15:13:40.441 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.502+0000 7fa42ddcc700 1 -- start start 2023-11-27T15:13:40.441 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.502+0000 7fa42ddcc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42819b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:40.441 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.502+0000 7fa42ddcc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42819bd00 con 0x7fa42807be00 2023-11-27T15:13:40.441 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.502+0000 7fa4277fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42819b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:40.441 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.503+0000 7fa4277fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42819b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54750/0 (socket says 172.21.15.133:54750) 2023-11-27T15:13:40.441 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.503+0000 7fa4277fe700 1 -- 172.21.15.133:0/140573831 learned_addr learned my addr 172.21.15.133:0/140573831 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:40.441 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.503+0000 7fa4277fe700 1 -- 172.21.15.133:0/140573831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa418009550 con 0x7fa42807be00 2023-11-27T15:13:40.442 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.503+0000 7fa4277fe700 1 --2- 172.21.15.133:0/140573831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42819b7c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa418006b20 tx=0x7fa418011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:40.442 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.503+0000 7fa424ff9700 1 -- 172.21.15.133:0/140573831 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa418003b70 con 0x7fa42807be00 2023-11-27T15:13:40.442 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.503+0000 7fa424ff9700 1 -- 172.21.15.133:0/140573831 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa418003cd0 con 0x7fa42807be00 2023-11-27T15:13:40.442 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.503+0000 7fa42ddcc700 1 -- 172.21.15.133:0/140573831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42810bde0 con 0x7fa42807be00 2023-11-27T15:13:40.442 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.503+0000 7fa424ff9700 1 -- 172.21.15.133:0/140573831 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa4180247c0 con 0x7fa42807be00 2023-11-27T15:13:40.442 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.503+0000 7fa42ddcc700 1 -- 172.21.15.133:0/140573831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa42810bc50 con 0x7fa42807be00 2023-11-27T15:13:40.442 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.504+0000 7fa424ff9700 1 -- 172.21.15.133:0/140573831 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fa41801ab40 con 0x7fa42807be00 2023-11-27T15:13:40.442 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.504+0000 7fa42ddcc700 1 -- 172.21.15.133:0/140573831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa42810bf70 con 0x7fa42807be00 2023-11-27T15:13:40.442 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.505+0000 7fa424ff9700 1 --2- 172.21.15.133:0/140573831 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fa4100387d0 0x7fa41003ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:40.442 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.505+0000 7fa424ff9700 1 -- 172.21.15.133:0/140573831 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa41804ca90 con 0x7fa42807be00 2023-11-27T15:13:40.442 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.505+0000 7fa426ffd700 1 --2- 172.21.15.133:0/140573831 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fa4100387d0 0x7fa41003ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:40.443 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.506+0000 7fa426ffd700 1 --2- 172.21.15.133:0/140573831 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fa4100387d0 0x7fa41003ac80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa41c006fd0 tx=0x7fa41c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:40.443 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.510+0000 7fa424ff9700 1 -- 172.21.15.133:0/140573831 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa41801a950 con 0x7fa42807be00 2023-11-27T15:13:40.443 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:39.707+0000 7fa42ddcc700 1 -- 172.21.15.133:0/140573831 --> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fa428102380 con 0x7fa4100387d0 2023-11-27T15:13:40.443 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.041+0000 7fa424ff9700 1 -- 172.21.15.133:0/140573831 <== mgr.14118 v2:172.21.15.133:6800/2374615220 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa428102380 con 0x7fa4100387d0 2023-11-27T15:13:40.443 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.043+0000 7fa42ddcc700 1 -- 172.21.15.133:0/140573831 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fa4100387d0 msgr2=0x7fa41003ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:40.443 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.043+0000 7fa42ddcc700 1 --2- 172.21.15.133:0/140573831 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fa4100387d0 0x7fa41003ac80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa41c006fd0 tx=0x7fa41c006e40 comp rx=0 tx=0).stop 2023-11-27T15:13:40.443 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.043+0000 7fa42ddcc700 1 -- 172.21.15.133:0/140573831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 msgr2=0x7fa42819b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:40.443 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.043+0000 7fa42ddcc700 1 --2- 172.21.15.133:0/140573831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42819b7c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa418006b20 tx=0x7fa418011ad0 comp rx=0 tx=0).stop 2023-11-27T15:13:40.443 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.043+0000 7fa42ddcc700 1 -- 172.21.15.133:0/140573831 shutdown_connections 2023-11-27T15:13:40.443 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.043+0000 7fa42ddcc700 1 --2- 172.21.15.133:0/140573831 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fa4100387d0 0x7fa41003ac80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:40.443 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.043+0000 7fa42ddcc700 1 --2- 172.21.15.133:0/140573831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42819b7c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:40.444 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.043+0000 7fa42ddcc700 1 -- 172.21.15.133:0/140573831 >> 172.21.15.133:0/140573831 conn(0x7fa428100fa0 msgr2=0x7fa428101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:40.444 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.044+0000 7fa42ddcc700 1 -- 172.21.15.133:0/140573831 shutdown_connections 2023-11-27T15:13:40.444 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.044+0000 7fa42ddcc700 1 -- 172.21.15.133:0/140573831 wait complete. 2023-11-27T15:13:40.470 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: Generating public/private rsa key pair. 2023-11-27T15:13:40.470 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: Your identification has been saved in /tmp/tmpiu0jr26w/key. 2023-11-27T15:13:40.470 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: Your public key has been saved in /tmp/tmpiu0jr26w/key.pub. 2023-11-27T15:13:40.471 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: The key fingerprint is: 2023-11-27T15:13:40.471 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: SHA256:GhCOAJU7SaAZH04QZOKpSIBRP1F6ZJJjOxv/953RbY8 ceph-5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:13:40.471 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: The key's randomart image is: 2023-11-27T15:13:40.471 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: +---[RSA 3072]----+ 2023-11-27T15:13:40.471 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: |#X=.+o+ | 2023-11-27T15:13:40.471 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: |BB+=+B | 2023-11-27T15:13:40.471 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: |o=++*o. | 2023-11-27T15:13:40.471 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: |+ + ++ | 2023-11-27T15:13:40.471 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: |o . =. S | 2023-11-27T15:13:40.471 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: | . .o ..| 2023-11-27T15:13:40.471 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: | .. . +| 2023-11-27T15:13:40.472 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: | . . . =.| 2023-11-27T15:13:40.472 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: | . .. E .| 2023-11-27T15:13:40.472 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: +----[SHA256]-----+ 2023-11-27T15:13:41.347 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:41 smithi133 ceph-mon[97860]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:13:41.347 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:41 smithi133 ceph-mon[97860]: Generating ssh key... 2023-11-27T15:13:41.347 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:41 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' 2023-11-27T15:13:41.347 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:41 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' 2023-11-27T15:13:41.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvzDKejWOae18ZOHtCrzB9Hgme2PKW1Fi/b14QgvG3+r81TTqgD/eJ5T2CtyGQa/yktGAUNPWKGfEUd7LVRP9lbCuIdUhZR9cC7vykLXD37bR0Lg6MIFI6lhkHzR61vSPZ/YbG2rqnyUr6JexwgcyJWmBnxZoxez9sTKLQpyEzfePhNXo0YfI24kJObBA8DZR+sKqCmlDkwwNlQLFQQpfEcXCQN5NG+hwQIxrWM3luCikkNWbLhsgFNLbaredn4x0+MH6+AM5Hecp6cbZi+Xtz/VYn33S90jh5hUD2acROkpMds7IsaXHtj6vAXZafi1BokEUwE7CUOBH2ZTL4ML1DO8Tl+/zcbqf94SoMhgBAD7Fw5EM9Prh0vQdekuxJTi9RiiVnU/gmKJXKBVAeS7apurnA3u83jQgy+sN7QxQ1S1stkzTSQaC4iGnFsvpEAr94fc/BBtjcxpCUAZoyp9maoiKSV2baHlnFvqDrSLd9AMInXkm+KmPagv5TTUbK9oM= ceph-5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:13:41.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.974+0000 7fa430018700 1 Processor -- start 2023-11-27T15:13:41.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.974+0000 7fa430018700 1 -- start start 2023-11-27T15:13:41.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.975+0000 7fa430018700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42807a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:41.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.975+0000 7fa430018700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42807a840 con 0x7fa42807be00 2023-11-27T15:13:41.617 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.975+0000 7fa42ddb4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42807a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:41.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.975+0000 7fa42ddb4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42807a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54760/0 (socket says 172.21.15.133:54760) 2023-11-27T15:13:41.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.975+0000 7fa42ddb4700 1 -- 172.21.15.133:0/2758904638 learned_addr learned my addr 172.21.15.133:0/2758904638 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:41.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.975+0000 7fa42ddb4700 1 -- 172.21.15.133:0/2758904638 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42807a980 con 0x7fa42807be00 2023-11-27T15:13:41.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.975+0000 7fa42ddb4700 1 --2- 172.21.15.133:0/2758904638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42807a300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa4240098a0 tx=0x7fa424009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ddea3b6f02454f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:41.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.976+0000 7fa42cdb2700 1 -- 172.21.15.133:0/2758904638 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa424004160 con 0x7fa42807be00 2023-11-27T15:13:41.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.976+0000 7fa42cdb2700 1 -- 172.21.15.133:0/2758904638 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa4240042c0 con 0x7fa42807be00 2023-11-27T15:13:41.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.976+0000 7fa42cdb2700 1 -- 172.21.15.133:0/2758904638 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa424011a00 con 0x7fa42807be00 2023-11-27T15:13:41.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.976+0000 7fa430018700 1 -- 172.21.15.133:0/2758904638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 msgr2=0x7fa42807a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:41.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.976+0000 7fa430018700 1 --2- 172.21.15.133:0/2758904638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42807a300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa4240098a0 tx=0x7fa424009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:41.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.976+0000 7fa430018700 1 -- 172.21.15.133:0/2758904638 shutdown_connections 2023-11-27T15:13:41.618 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.976+0000 7fa430018700 1 --2- 172.21.15.133:0/2758904638 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42807a300 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:41.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.976+0000 7fa430018700 1 -- 172.21.15.133:0/2758904638 >> 172.21.15.133:0/2758904638 conn(0x7fa428100fa0 msgr2=0x7fa4281033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:41.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.976+0000 7fa430018700 1 -- 172.21.15.133:0/2758904638 shutdown_connections 2023-11-27T15:13:41.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.977+0000 7fa430018700 1 -- 172.21.15.133:0/2758904638 wait complete. 2023-11-27T15:13:41.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.977+0000 7fa430018700 1 Processor -- start 2023-11-27T15:13:41.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.978+0000 7fa430018700 1 -- start start 2023-11-27T15:13:41.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.978+0000 7fa430018700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42819b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:41.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.978+0000 7fa430018700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42819bd00 con 0x7fa42807be00 2023-11-27T15:13:41.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.978+0000 7fa42ddb4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42819b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:41.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.978+0000 7fa42ddb4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42819b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54772/0 (socket says 172.21.15.133:54772) 2023-11-27T15:13:41.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.978+0000 7fa42ddb4700 1 -- 172.21.15.133:0/693394566 learned_addr learned my addr 172.21.15.133:0/693394566 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:41.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.979+0000 7fa42ddb4700 1 -- 172.21.15.133:0/693394566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa424009550 con 0x7fa42807be00 2023-11-27T15:13:41.619 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.979+0000 7fa42ddb4700 1 --2- 172.21.15.133:0/693394566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42819b7c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa424006b20 tx=0x7fa424011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:41.620 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.979+0000 7fa41effd700 1 -- 172.21.15.133:0/693394566 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa424003b70 con 0x7fa42807be00 2023-11-27T15:13:41.620 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.979+0000 7fa41effd700 1 -- 172.21.15.133:0/693394566 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa424003cd0 con 0x7fa42807be00 2023-11-27T15:13:41.620 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.979+0000 7fa41effd700 1 -- 172.21.15.133:0/693394566 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa4240247c0 con 0x7fa42807be00 2023-11-27T15:13:41.620 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.979+0000 7fa430018700 1 -- 172.21.15.133:0/693394566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42810bde0 con 0x7fa42807be00 2023-11-27T15:13:41.620 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.979+0000 7fa430018700 1 -- 172.21.15.133:0/693394566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa42810bc50 con 0x7fa42807be00 2023-11-27T15:13:41.620 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.980+0000 7fa41effd700 1 -- 172.21.15.133:0/693394566 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fa42401a430 con 0x7fa42807be00 2023-11-27T15:13:41.620 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.980+0000 7fa430018700 1 -- 172.21.15.133:0/693394566 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa428062120 con 0x7fa42807be00 2023-11-27T15:13:41.620 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.981+0000 7fa41effd700 1 --2- 172.21.15.133:0/693394566 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fa4140387d0 0x7fa41403ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:41.620 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.981+0000 7fa41effd700 1 -- 172.21.15.133:0/693394566 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa42404ca90 con 0x7fa42807be00 2023-11-27T15:13:41.620 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.981+0000 7fa42d5b3700 1 --2- 172.21.15.133:0/693394566 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fa4140387d0 0x7fa41403ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:41.620 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.982+0000 7fa42d5b3700 1 --2- 172.21.15.133:0/693394566 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fa4140387d0 0x7fa41403ac80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa418006fd0 tx=0x7fa418006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:41.621 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:40.986+0000 7fa41effd700 1 -- 172.21.15.133:0/693394566 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa42401a6e0 con 0x7fa42807be00 2023-11-27T15:13:41.621 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:41.186+0000 7fa430018700 1 -- 172.21.15.133:0/693394566 --> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fa428102380 con 0x7fa4140387d0 2023-11-27T15:13:41.621 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:41.187+0000 7fa41effd700 1 -- 172.21.15.133:0/693394566 <== mgr.14118 v2:172.21.15.133:6800/2374615220 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fa428102380 con 0x7fa4140387d0 2023-11-27T15:13:41.621 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:41.189+0000 7fa430018700 1 -- 172.21.15.133:0/693394566 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fa4140387d0 msgr2=0x7fa41403ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:41.621 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:41.189+0000 7fa430018700 1 --2- 172.21.15.133:0/693394566 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fa4140387d0 0x7fa41403ac80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa418006fd0 tx=0x7fa418006e40 comp rx=0 tx=0).stop 2023-11-27T15:13:41.621 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:41.189+0000 7fa430018700 1 -- 172.21.15.133:0/693394566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 msgr2=0x7fa42819b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:41.621 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:41.189+0000 7fa430018700 1 --2- 172.21.15.133:0/693394566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42819b7c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa424006b20 tx=0x7fa424011ad0 comp rx=0 tx=0).stop 2023-11-27T15:13:41.621 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:41.189+0000 7fa430018700 1 -- 172.21.15.133:0/693394566 shutdown_connections 2023-11-27T15:13:41.621 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:41.189+0000 7fa430018700 1 --2- 172.21.15.133:0/693394566 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7fa4140387d0 0x7fa41403ac80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:41.621 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:41.189+0000 7fa430018700 1 --2- 172.21.15.133:0/693394566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa42807be00 0x7fa42819b7c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:41.621 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:41.189+0000 7fa430018700 1 -- 172.21.15.133:0/693394566 >> 172.21.15.133:0/693394566 conn(0x7fa428100fa0 msgr2=0x7fa428101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:41.621 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:41.189+0000 7fa430018700 1 -- 172.21.15.133:0/693394566 shutdown_connections 2023-11-27T15:13:41.622 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:41.189+0000 7fa430018700 1 -- 172.21.15.133:0/693394566 wait complete. 2023-11-27T15:13:41.622 INFO:teuthology.orchestra.run.smithi133.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-11-27T15:13:41.622 INFO:teuthology.orchestra.run.smithi133.stdout:Adding key to root@localhost authorized_keys... 2023-11-27T15:13:41.622 INFO:teuthology.orchestra.run.smithi133.stdout:Adding host smithi133... 2023-11-27T15:13:42.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:42 smithi133 ceph-mon[97860]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:13:43.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:43 smithi133 ceph-mon[97860]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi133", "addr": "172.21.15.133", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:13:44.372 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:44 smithi133 ceph-mon[97860]: Deploying cephadm binary to smithi133 2023-11-27T15:13:45.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout Added host 'smithi133' with addr '172.21.15.133' 2023-11-27T15:13:45.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.112+0000 7ff66cf19700 1 Processor -- start 2023-11-27T15:13:45.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.112+0000 7ff66cf19700 1 -- start start 2023-11-27T15:13:45.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.112+0000 7ff66cf19700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff66807bde0 0x7ff66807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:45.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.112+0000 7ff66cf19700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66807a820 con 0x7ff66807bde0 2023-11-27T15:13:45.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.113+0000 7ff66659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff66807bde0 0x7ff66807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:45.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.113+0000 7ff66659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff66807bde0 0x7ff66807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54788/0 (socket says 172.21.15.133:54788) 2023-11-27T15:13:45.809 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.113+0000 7ff66659c700 1 -- 172.21.15.133:0/368830829 learned_addr learned my addr 172.21.15.133:0/368830829 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:45.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.113+0000 7ff66659c700 1 -- 172.21.15.133:0/368830829 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66807a960 con 0x7ff66807bde0 2023-11-27T15:13:45.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.113+0000 7ff66659c700 1 --2- 172.21.15.133:0/368830829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff66807bde0 0x7ff66807a2e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff6580098a0 tx=0x7ff658009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf16abfb36c3a83 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:45.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.114+0000 7ff66559a700 1 -- 172.21.15.133:0/368830829 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff658004160 con 0x7ff66807bde0 2023-11-27T15:13:45.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.114+0000 7ff66559a700 1 -- 172.21.15.133:0/368830829 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff6580042c0 con 0x7ff66807bde0 2023-11-27T15:13:45.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.114+0000 7ff66559a700 1 -- 172.21.15.133:0/368830829 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff658011a00 con 0x7ff66807bde0 2023-11-27T15:13:45.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.114+0000 7ff66cf19700 1 -- 172.21.15.133:0/368830829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff66807bde0 msgr2=0x7ff66807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:45.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.114+0000 7ff66cf19700 1 --2- 172.21.15.133:0/368830829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff66807bde0 0x7ff66807a2e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff6580098a0 tx=0x7ff658009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:45.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.114+0000 7ff66cf19700 1 -- 172.21.15.133:0/368830829 shutdown_connections 2023-11-27T15:13:45.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.114+0000 7ff66cf19700 1 --2- 172.21.15.133:0/368830829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff66807bde0 0x7ff66807a2e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:45.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.114+0000 7ff66cf19700 1 -- 172.21.15.133:0/368830829 >> 172.21.15.133:0/368830829 conn(0x7ff668100fa0 msgr2=0x7ff6681033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:45.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.115+0000 7ff66cf19700 1 -- 172.21.15.133:0/368830829 shutdown_connections 2023-11-27T15:13:45.810 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.115+0000 7ff66cf19700 1 -- 172.21.15.133:0/368830829 wait complete. 2023-11-27T15:13:45.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.116+0000 7ff66cf19700 1 Processor -- start 2023-11-27T15:13:45.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.116+0000 7ff66cf19700 1 -- start start 2023-11-27T15:13:45.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.116+0000 7ff66cf19700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff66807bde0 0x7ff66819b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:45.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.116+0000 7ff66cf19700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66819bcc0 con 0x7ff66807bde0 2023-11-27T15:13:45.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.116+0000 7ff66659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff66807bde0 0x7ff66819b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:45.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.116+0000 7ff66659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff66807bde0 0x7ff66819b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54802/0 (socket says 172.21.15.133:54802) 2023-11-27T15:13:45.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.116+0000 7ff66659c700 1 -- 172.21.15.133:0/3873054831 learned_addr learned my addr 172.21.15.133:0/3873054831 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:45.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.117+0000 7ff66659c700 1 -- 172.21.15.133:0/3873054831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff658009550 con 0x7ff66807bde0 2023-11-27T15:13:45.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.117+0000 7ff66659c700 1 --2- 172.21.15.133:0/3873054831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff66807bde0 0x7ff66819b780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff658006af0 tx=0x7ff658011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:45.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.117+0000 7ff6577fe700 1 -- 172.21.15.133:0/3873054831 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff658003a40 con 0x7ff66807bde0 2023-11-27T15:13:45.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.117+0000 7ff6577fe700 1 -- 172.21.15.133:0/3873054831 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff658003ba0 con 0x7ff66807bde0 2023-11-27T15:13:45.811 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.117+0000 7ff6577fe700 1 -- 172.21.15.133:0/3873054831 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff65801a700 con 0x7ff66807bde0 2023-11-27T15:13:45.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.117+0000 7ff66cf19700 1 -- 172.21.15.133:0/3873054831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66810be00 con 0x7ff66807bde0 2023-11-27T15:13:45.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.117+0000 7ff66cf19700 1 -- 172.21.15.133:0/3873054831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66810bc70 con 0x7ff66807bde0 2023-11-27T15:13:45.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.118+0000 7ff6577fe700 1 -- 172.21.15.133:0/3873054831 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7ff658024ad0 con 0x7ff66807bde0 2023-11-27T15:13:45.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.118+0000 7ff66cf19700 1 -- 172.21.15.133:0/3873054831 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff668062120 con 0x7ff66807bde0 2023-11-27T15:13:45.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.119+0000 7ff6577fe700 1 --2- 172.21.15.133:0/3873054831 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7ff650038820 0x7ff65003acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:45.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.119+0000 7ff6577fe700 1 -- 172.21.15.133:0/3873054831 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff65804c6c0 con 0x7ff66807bde0 2023-11-27T15:13:45.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.119+0000 7ff665d9b700 1 --2- 172.21.15.133:0/3873054831 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7ff650038820 0x7ff65003acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:45.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.119+0000 7ff665d9b700 1 --2- 172.21.15.133:0/3873054831 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7ff650038820 0x7ff65003acd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff65c006fd0 tx=0x7ff65c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:45.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.124+0000 7ff6577fe700 1 -- 172.21.15.133:0/3873054831 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff658018b80 con 0x7ff66807bde0 2023-11-27T15:13:45.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:42.326+0000 7ff66cf19700 1 -- 172.21.15.133:0/3873054831 --> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi133", "addr": "172.21.15.133", "target": ["mon-mgr", ""]}) v1 -- 0x7ff668102390 con 0x7ff650038820 2023-11-27T15:13:45.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:45.277+0000 7ff6577fe700 1 -- 172.21.15.133:0/3873054831 <== mgr.14118 v2:172.21.15.133:6800/2374615220 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7ff668102390 con 0x7ff650038820 2023-11-27T15:13:45.812 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:45.280+0000 7ff66cf19700 1 -- 172.21.15.133:0/3873054831 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7ff650038820 msgr2=0x7ff65003acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:45.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:45.280+0000 7ff66cf19700 1 --2- 172.21.15.133:0/3873054831 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7ff650038820 0x7ff65003acd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff65c006fd0 tx=0x7ff65c006e40 comp rx=0 tx=0).stop 2023-11-27T15:13:45.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:45.280+0000 7ff66cf19700 1 -- 172.21.15.133:0/3873054831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff66807bde0 msgr2=0x7ff66819b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:45.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:45.280+0000 7ff66cf19700 1 --2- 172.21.15.133:0/3873054831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff66807bde0 0x7ff66819b780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff658006af0 tx=0x7ff658011ad0 comp rx=0 tx=0).stop 2023-11-27T15:13:45.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:45.280+0000 7ff66cf19700 1 -- 172.21.15.133:0/3873054831 shutdown_connections 2023-11-27T15:13:45.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:45.280+0000 7ff66cf19700 1 --2- 172.21.15.133:0/3873054831 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7ff650038820 0x7ff65003acd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:45.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:45.280+0000 7ff66cf19700 1 --2- 172.21.15.133:0/3873054831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff66807bde0 0x7ff66819b780 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:45.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:45.280+0000 7ff66cf19700 1 -- 172.21.15.133:0/3873054831 >> 172.21.15.133:0/3873054831 conn(0x7ff668100fa0 msgr2=0x7ff668101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:45.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:45.281+0000 7ff66cf19700 1 -- 172.21.15.133:0/3873054831 shutdown_connections 2023-11-27T15:13:45.813 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:45.281+0000 7ff66cf19700 1 -- 172.21.15.133:0/3873054831 wait complete. 2023-11-27T15:13:45.813 INFO:teuthology.orchestra.run.smithi133.stdout:Deploying unmanaged mon service... 2023-11-27T15:13:46.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:46 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' 2023-11-27T15:13:46.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:46 smithi133 ceph-mon[97860]: Added host smithi133 2023-11-27T15:13:46.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:46 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:13:47.190 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-11-27T15:13:47.190 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.348+0000 7f6c20918700 1 Processor -- start 2023-11-27T15:13:47.190 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.349+0000 7f6c20918700 1 -- start start 2023-11-27T15:13:47.190 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.349+0000 7f6c20918700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c1c07bde0 0x7f6c1c07a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:47.191 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.349+0000 7f6c20918700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c07a820 con 0x7f6c1c07bde0 2023-11-27T15:13:47.191 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.349+0000 7f6c1a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c1c07bde0 0x7f6c1c07a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:47.191 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.349+0000 7f6c1a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c1c07bde0 0x7f6c1c07a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52074/0 (socket says 172.21.15.133:52074) 2023-11-27T15:13:47.191 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.349+0000 7f6c1a59c700 1 -- 172.21.15.133:0/601535515 learned_addr learned my addr 172.21.15.133:0/601535515 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:47.191 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.350+0000 7f6c1a59c700 1 -- 172.21.15.133:0/601535515 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c07a960 con 0x7f6c1c07bde0 2023-11-27T15:13:47.191 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.350+0000 7f6c1a59c700 1 --2- 172.21.15.133:0/601535515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c1c07bde0 0x7f6c1c07a2e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0098a0 tx=0x7f6c0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac830b7c6cc4c5a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:47.191 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.350+0000 7f6c1959a700 1 -- 172.21.15.133:0/601535515 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c0c004160 con 0x7f6c1c07bde0 2023-11-27T15:13:47.191 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.350+0000 7f6c1959a700 1 -- 172.21.15.133:0/601535515 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6c0c0042c0 con 0x7f6c1c07bde0 2023-11-27T15:13:47.191 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.350+0000 7f6c1959a700 1 -- 172.21.15.133:0/601535515 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c0c011a00 con 0x7f6c1c07bde0 2023-11-27T15:13:47.191 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.351+0000 7f6c20918700 1 -- 172.21.15.133:0/601535515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c1c07bde0 msgr2=0x7f6c1c07a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:47.191 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.351+0000 7f6c20918700 1 --2- 172.21.15.133:0/601535515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c1c07bde0 0x7f6c1c07a2e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0098a0 tx=0x7f6c0c009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:47.192 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.351+0000 7f6c20918700 1 -- 172.21.15.133:0/601535515 shutdown_connections 2023-11-27T15:13:47.192 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.351+0000 7f6c20918700 1 --2- 172.21.15.133:0/601535515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c1c07bde0 0x7f6c1c07a2e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:47.192 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.351+0000 7f6c20918700 1 -- 172.21.15.133:0/601535515 >> 172.21.15.133:0/601535515 conn(0x7f6c1c100fa0 msgr2=0x7f6c1c1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:47.192 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.351+0000 7f6c20918700 1 -- 172.21.15.133:0/601535515 shutdown_connections 2023-11-27T15:13:47.192 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.351+0000 7f6c20918700 1 -- 172.21.15.133:0/601535515 wait complete. 2023-11-27T15:13:47.192 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.352+0000 7f6c20918700 1 Processor -- start 2023-11-27T15:13:47.192 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.352+0000 7f6c20918700 1 -- start start 2023-11-27T15:13:47.192 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.352+0000 7f6c20918700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c1c07bde0 0x7f6c1c19b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:47.192 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.352+0000 7f6c20918700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c19bcc0 con 0x7f6c1c07bde0 2023-11-27T15:13:47.192 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.352+0000 7f6c1a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c1c07bde0 0x7f6c1c19b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:47.192 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.353+0000 7f6c1a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c1c07bde0 0x7f6c1c19b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52090/0 (socket says 172.21.15.133:52090) 2023-11-27T15:13:47.192 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.353+0000 7f6c1a59c700 1 -- 172.21.15.133:0/4293011463 learned_addr learned my addr 172.21.15.133:0/4293011463 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:47.193 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.353+0000 7f6c1a59c700 1 -- 172.21.15.133:0/4293011463 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c009550 con 0x7f6c1c07bde0 2023-11-27T15:13:47.193 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.353+0000 7f6c1a59c700 1 --2- 172.21.15.133:0/4293011463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c1c07bde0 0x7f6c1c19b780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c006af0 tx=0x7f6c0c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:47.193 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.353+0000 7f6c0b7fe700 1 -- 172.21.15.133:0/4293011463 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c0c003a40 con 0x7f6c1c07bde0 2023-11-27T15:13:47.193 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.353+0000 7f6c0b7fe700 1 -- 172.21.15.133:0/4293011463 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6c0c003ba0 con 0x7f6c1c07bde0 2023-11-27T15:13:47.193 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.353+0000 7f6c0b7fe700 1 -- 172.21.15.133:0/4293011463 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c0c01a700 con 0x7f6c1c07bde0 2023-11-27T15:13:47.193 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.353+0000 7f6c20918700 1 -- 172.21.15.133:0/4293011463 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c1c10be00 con 0x7f6c1c07bde0 2023-11-27T15:13:47.193 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.354+0000 7f6c20918700 1 -- 172.21.15.133:0/4293011463 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1c10bc70 con 0x7f6c1c07bde0 2023-11-27T15:13:47.193 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.355+0000 7f6c0b7fe700 1 -- 172.21.15.133:0/4293011463 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f6c0c024ad0 con 0x7f6c1c07bde0 2023-11-27T15:13:47.193 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.355+0000 7f6c20918700 1 -- 172.21.15.133:0/4293011463 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1c062120 con 0x7f6c1c07bde0 2023-11-27T15:13:47.193 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.355+0000 7f6c0b7fe700 1 --2- 172.21.15.133:0/4293011463 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f6c04038820 0x7f6c0403acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:47.193 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.355+0000 7f6c0b7fe700 1 -- 172.21.15.133:0/4293011463 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6c0c04c6b0 con 0x7f6c1c07bde0 2023-11-27T15:13:47.194 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.355+0000 7f6c19d9b700 1 --2- 172.21.15.133:0/4293011463 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f6c04038820 0x7f6c0403acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:47.194 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.356+0000 7f6c19d9b700 1 --2- 172.21.15.133:0/4293011463 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f6c04038820 0x7f6c0403acd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6c10006fd0 tx=0x7f6c10006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:47.194 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.360+0000 7f6c0b7fe700 1 -- 172.21.15.133:0/4293011463 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6c0c024430 con 0x7f6c1c07bde0 2023-11-27T15:13:47.194 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.584+0000 7f6c20918700 1 -- 172.21.15.133:0/4293011463 --> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f6c1c102390 con 0x7f6c04038820 2023-11-27T15:13:47.194 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.592+0000 7f6c0b7fe700 1 -- 172.21.15.133:0/4293011463 <== mgr.14118 v2:172.21.15.133:6800/2374615220 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f6c1c102390 con 0x7f6c04038820 2023-11-27T15:13:47.194 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.594+0000 7f6c20918700 1 -- 172.21.15.133:0/4293011463 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f6c04038820 msgr2=0x7f6c0403acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:47.194 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.594+0000 7f6c20918700 1 --2- 172.21.15.133:0/4293011463 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f6c04038820 0x7f6c0403acd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6c10006fd0 tx=0x7f6c10006e40 comp rx=0 tx=0).stop 2023-11-27T15:13:47.194 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.594+0000 7f6c20918700 1 -- 172.21.15.133:0/4293011463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c1c07bde0 msgr2=0x7f6c1c19b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:47.194 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.594+0000 7f6c20918700 1 --2- 172.21.15.133:0/4293011463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c1c07bde0 0x7f6c1c19b780 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c006af0 tx=0x7f6c0c011ad0 comp rx=0 tx=0).stop 2023-11-27T15:13:47.194 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.594+0000 7f6c20918700 1 -- 172.21.15.133:0/4293011463 shutdown_connections 2023-11-27T15:13:47.194 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.594+0000 7f6c20918700 1 --2- 172.21.15.133:0/4293011463 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f6c04038820 0x7f6c0403acd0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:47.195 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.594+0000 7f6c20918700 1 --2- 172.21.15.133:0/4293011463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c1c07bde0 0x7f6c1c19b780 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:47.195 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.594+0000 7f6c20918700 1 -- 172.21.15.133:0/4293011463 >> 172.21.15.133:0/4293011463 conn(0x7f6c1c100fa0 msgr2=0x7f6c1c101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:47.195 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.595+0000 7f6c20918700 1 -- 172.21.15.133:0/4293011463 shutdown_connections 2023-11-27T15:13:47.195 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:46.595+0000 7f6c20918700 1 -- 172.21.15.133:0/4293011463 wait complete. 2023-11-27T15:13:47.195 INFO:teuthology.orchestra.run.smithi133.stdout:Deploying unmanaged mgr service... 2023-11-27T15:13:47.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:47 smithi133 ceph-mon[97860]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:13:47.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:47 smithi133 ceph-mon[97860]: Saving service mon spec with placement count:5 2023-11-27T15:13:47.710 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:47 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' 2023-11-27T15:13:48.783 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-11-27T15:13:48.783 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.017+0000 7f28c86db700 1 Processor -- start 2023-11-27T15:13:48.783 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.017+0000 7f28c86db700 1 -- start start 2023-11-27T15:13:48.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.017+0000 7f28c86db700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c007bde0 0x7f28c007a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:48.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.017+0000 7f28c86db700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c007a820 con 0x7f28c007bde0 2023-11-27T15:13:48.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.018+0000 7f28c6477700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c007bde0 0x7f28c007a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:48.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.018+0000 7f28c6477700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c007bde0 0x7f28c007a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52096/0 (socket says 172.21.15.133:52096) 2023-11-27T15:13:48.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.018+0000 7f28c6477700 1 -- 172.21.15.133:0/4084441825 learned_addr learned my addr 172.21.15.133:0/4084441825 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:48.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.018+0000 7f28c6477700 1 -- 172.21.15.133:0/4084441825 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c007a960 con 0x7f28c007bde0 2023-11-27T15:13:48.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.018+0000 7f28c6477700 1 --2- 172.21.15.133:0/4084441825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c007bde0 0x7f28c007a2e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0098a0 tx=0x7f28bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37705f29072e3904 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:48.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.019+0000 7f28c5475700 1 -- 172.21.15.133:0/4084441825 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28bc004160 con 0x7f28c007bde0 2023-11-27T15:13:48.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.019+0000 7f28c5475700 1 -- 172.21.15.133:0/4084441825 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f28bc0042c0 con 0x7f28c007bde0 2023-11-27T15:13:48.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.019+0000 7f28c5475700 1 -- 172.21.15.133:0/4084441825 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28bc011a00 con 0x7f28c007bde0 2023-11-27T15:13:48.784 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.019+0000 7f28c86db700 1 -- 172.21.15.133:0/4084441825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c007bde0 msgr2=0x7f28c007a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:48.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.019+0000 7f28c86db700 1 --2- 172.21.15.133:0/4084441825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c007bde0 0x7f28c007a2e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0098a0 tx=0x7f28bc009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:48.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.019+0000 7f28c86db700 1 -- 172.21.15.133:0/4084441825 shutdown_connections 2023-11-27T15:13:48.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.019+0000 7f28c86db700 1 --2- 172.21.15.133:0/4084441825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c007bde0 0x7f28c007a2e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:48.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.019+0000 7f28c86db700 1 -- 172.21.15.133:0/4084441825 >> 172.21.15.133:0/4084441825 conn(0x7f28c0100fa0 msgr2=0x7f28c01033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:48.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.019+0000 7f28c86db700 1 -- 172.21.15.133:0/4084441825 shutdown_connections 2023-11-27T15:13:48.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.019+0000 7f28c86db700 1 -- 172.21.15.133:0/4084441825 wait complete. 2023-11-27T15:13:48.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.020+0000 7f28c86db700 1 Processor -- start 2023-11-27T15:13:48.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.021+0000 7f28c86db700 1 -- start start 2023-11-27T15:13:48.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.021+0000 7f28c86db700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c007bde0 0x7f28c019b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:48.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.021+0000 7f28c86db700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c019bcc0 con 0x7f28c007bde0 2023-11-27T15:13:48.785 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.021+0000 7f28c6477700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c007bde0 0x7f28c019b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:48.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.021+0000 7f28c6477700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c007bde0 0x7f28c019b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52108/0 (socket says 172.21.15.133:52108) 2023-11-27T15:13:48.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.021+0000 7f28c6477700 1 -- 172.21.15.133:0/1882821904 learned_addr learned my addr 172.21.15.133:0/1882821904 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:48.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.021+0000 7f28c6477700 1 -- 172.21.15.133:0/1882821904 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc009550 con 0x7f28c007bde0 2023-11-27T15:13:48.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.022+0000 7f28c6477700 1 --2- 172.21.15.133:0/1882821904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c007bde0 0x7f28c019b780 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f28bc006af0 tx=0x7f28bc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:48.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.022+0000 7f28b77fe700 1 -- 172.21.15.133:0/1882821904 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28bc003a40 con 0x7f28c007bde0 2023-11-27T15:13:48.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.022+0000 7f28b77fe700 1 -- 172.21.15.133:0/1882821904 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f28bc003ba0 con 0x7f28c007bde0 2023-11-27T15:13:48.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.022+0000 7f28c86db700 1 -- 172.21.15.133:0/1882821904 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c010be00 con 0x7f28c007bde0 2023-11-27T15:13:48.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.022+0000 7f28b77fe700 1 -- 172.21.15.133:0/1882821904 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28bc01a700 con 0x7f28c007bde0 2023-11-27T15:13:48.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.022+0000 7f28c86db700 1 -- 172.21.15.133:0/1882821904 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c010bc70 con 0x7f28c007bde0 2023-11-27T15:13:48.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.023+0000 7f28b77fe700 1 -- 172.21.15.133:0/1882821904 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f28bc024430 con 0x7f28c007bde0 2023-11-27T15:13:48.786 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.023+0000 7f28c86db700 1 -- 172.21.15.133:0/1882821904 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c0062120 con 0x7f28c007bde0 2023-11-27T15:13:48.787 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.024+0000 7f28b77fe700 1 --2- 172.21.15.133:0/1882821904 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f28ac038820 0x7f28ac03acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:48.787 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.024+0000 7f28b77fe700 1 -- 172.21.15.133:0/1882821904 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f28bc04c6d0 con 0x7f28c007bde0 2023-11-27T15:13:48.787 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.024+0000 7f28c5c76700 1 --2- 172.21.15.133:0/1882821904 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f28ac038820 0x7f28ac03acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:48.787 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.024+0000 7f28c5c76700 1 --2- 172.21.15.133:0/1882821904 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f28ac038820 0x7f28ac03acd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f28b0006fd0 tx=0x7f28b0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:48.787 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.029+0000 7f28b77fe700 1 -- 172.21.15.133:0/1882821904 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f28bc018710 con 0x7f28c007bde0 2023-11-27T15:13:48.787 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.098+0000 7f28b77fe700 1 -- 172.21.15.133:0/1882821904 <== mon.0 v2:172.21.15.133:3300/0 7 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28bc04c0e0 con 0x7f28c007bde0 2023-11-27T15:13:48.787 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.261+0000 7f28c86db700 1 -- 172.21.15.133:0/1882821904 --> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f28c0102390 con 0x7f28ac038820 2023-11-27T15:13:48.787 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.265+0000 7f28b77fe700 1 -- 172.21.15.133:0/1882821904 <== mgr.14118 v2:172.21.15.133:6800/2374615220 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f28c0102390 con 0x7f28ac038820 2023-11-27T15:13:48.787 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.268+0000 7f28c86db700 1 -- 172.21.15.133:0/1882821904 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f28ac038820 msgr2=0x7f28ac03acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:48.787 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.268+0000 7f28c86db700 1 --2- 172.21.15.133:0/1882821904 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f28ac038820 0x7f28ac03acd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f28b0006fd0 tx=0x7f28b0006e40 comp rx=0 tx=0).stop 2023-11-27T15:13:48.787 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.268+0000 7f28c86db700 1 -- 172.21.15.133:0/1882821904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c007bde0 msgr2=0x7f28c019b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:48.787 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.268+0000 7f28c86db700 1 --2- 172.21.15.133:0/1882821904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c007bde0 0x7f28c019b780 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f28bc006af0 tx=0x7f28bc011ad0 comp rx=0 tx=0).stop 2023-11-27T15:13:48.788 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.268+0000 7f28c86db700 1 -- 172.21.15.133:0/1882821904 shutdown_connections 2023-11-27T15:13:48.788 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.268+0000 7f28c86db700 1 --2- 172.21.15.133:0/1882821904 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f28ac038820 0x7f28ac03acd0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:48.788 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.268+0000 7f28c86db700 1 --2- 172.21.15.133:0/1882821904 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28c007bde0 0x7f28c019b780 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:48.788 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.268+0000 7f28c86db700 1 -- 172.21.15.133:0/1882821904 >> 172.21.15.133:0/1882821904 conn(0x7f28c0100fa0 msgr2=0x7f28c0101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:48.788 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.268+0000 7f28c86db700 1 -- 172.21.15.133:0/1882821904 shutdown_connections 2023-11-27T15:13:48.788 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:48.268+0000 7f28c86db700 1 -- 172.21.15.133:0/1882821904 wait complete. 2023-11-27T15:13:49.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:49 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' 2023-11-27T15:13:49.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:49 smithi133 ceph-mon[97860]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:13:49.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:49 smithi133 ceph-mon[97860]: Saving service mgr spec with placement count:2 2023-11-27T15:13:49.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:49 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' 2023-11-27T15:13:49.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:49 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' 2023-11-27T15:13:49.958 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.317+0000 7f29a0782700 1 Processor -- start 2023-11-27T15:13:49.958 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.317+0000 7f29a0782700 1 -- start start 2023-11-27T15:13:49.958 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.317+0000 7f29a0782700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f299807bde0 0x7f299807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:49.958 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.317+0000 7f29a0782700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299807a820 con 0x7f299807bde0 2023-11-27T15:13:49.958 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.317+0000 7f299e51e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f299807bde0 0x7f299807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:49.962 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.317+0000 7f299e51e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f299807bde0 0x7f299807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52114/0 (socket says 172.21.15.133:52114) 2023-11-27T15:13:49.962 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.318+0000 7f299e51e700 1 -- 172.21.15.133:0/2374809240 learned_addr learned my addr 172.21.15.133:0/2374809240 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:49.962 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.318+0000 7f299e51e700 1 -- 172.21.15.133:0/2374809240 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299807a960 con 0x7f299807bde0 2023-11-27T15:13:49.962 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.318+0000 7f299e51e700 1 --2- 172.21.15.133:0/2374809240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f299807bde0 0x7f299807a2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f29940098a0 tx=0x7f2994009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d71d7d5845c6d22 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:49.963 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.318+0000 7f299d51c700 1 -- 172.21.15.133:0/2374809240 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2994004160 con 0x7f299807bde0 2023-11-27T15:13:49.963 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.318+0000 7f299d51c700 1 -- 172.21.15.133:0/2374809240 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29940042c0 con 0x7f299807bde0 2023-11-27T15:13:49.963 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.318+0000 7f299d51c700 1 -- 172.21.15.133:0/2374809240 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29940119b0 con 0x7f299807bde0 2023-11-27T15:13:49.963 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.319+0000 7f29a0782700 1 -- 172.21.15.133:0/2374809240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f299807bde0 msgr2=0x7f299807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:49.963 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.319+0000 7f29a0782700 1 --2- 172.21.15.133:0/2374809240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f299807bde0 0x7f299807a2e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f29940098a0 tx=0x7f2994009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:49.963 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.319+0000 7f29a0782700 1 -- 172.21.15.133:0/2374809240 shutdown_connections 2023-11-27T15:13:49.963 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.319+0000 7f29a0782700 1 --2- 172.21.15.133:0/2374809240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f299807bde0 0x7f299807a2e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:49.963 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.319+0000 7f29a0782700 1 -- 172.21.15.133:0/2374809240 >> 172.21.15.133:0/2374809240 conn(0x7f2998100fa0 msgr2=0x7f29981033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:49.963 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.319+0000 7f29a0782700 1 -- 172.21.15.133:0/2374809240 shutdown_connections 2023-11-27T15:13:49.963 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.319+0000 7f29a0782700 1 -- 172.21.15.133:0/2374809240 wait complete. 2023-11-27T15:13:49.963 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.320+0000 7f29a0782700 1 Processor -- start 2023-11-27T15:13:49.964 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.320+0000 7f29a0782700 1 -- start start 2023-11-27T15:13:49.964 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.321+0000 7f29a0782700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f299807bde0 0x7f299819b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:49.964 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.321+0000 7f29a0782700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299819bce0 con 0x7f299807bde0 2023-11-27T15:13:49.964 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.321+0000 7f299e51e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f299807bde0 0x7f299819b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:49.964 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.321+0000 7f299e51e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f299807bde0 0x7f299819b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52130/0 (socket says 172.21.15.133:52130) 2023-11-27T15:13:49.964 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.321+0000 7f299e51e700 1 -- 172.21.15.133:0/828689833 learned_addr learned my addr 172.21.15.133:0/828689833 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:49.964 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.321+0000 7f299e51e700 1 -- 172.21.15.133:0/828689833 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2994009550 con 0x7f299807bde0 2023-11-27T15:13:49.964 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.321+0000 7f299e51e700 1 --2- 172.21.15.133:0/828689833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f299807bde0 0x7f299819b7a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2994000c00 tx=0x7f2994011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:49.964 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.322+0000 7f298f7fe700 1 -- 172.21.15.133:0/828689833 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f29940039f0 con 0x7f299807bde0 2023-11-27T15:13:49.964 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.322+0000 7f298f7fe700 1 -- 172.21.15.133:0/828689833 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2994003b50 con 0x7f299807bde0 2023-11-27T15:13:49.964 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.322+0000 7f298f7fe700 1 -- 172.21.15.133:0/828689833 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f299401a780 con 0x7f299807bde0 2023-11-27T15:13:49.965 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.322+0000 7f29a0782700 1 -- 172.21.15.133:0/828689833 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f299810be50 con 0x7f299807bde0 2023-11-27T15:13:49.965 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.322+0000 7f29a0782700 1 -- 172.21.15.133:0/828689833 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f299810bcc0 con 0x7f299807bde0 2023-11-27T15:13:49.965 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.323+0000 7f298f7fe700 1 -- 172.21.15.133:0/828689833 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f2994024440 con 0x7f299807bde0 2023-11-27T15:13:49.965 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.323+0000 7f29a0782700 1 -- 172.21.15.133:0/828689833 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f299810bfe0 con 0x7f299807bde0 2023-11-27T15:13:49.965 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.323+0000 7f298f7fe700 1 --2- 172.21.15.133:0/828689833 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f2984038820 0x7f298403acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:49.965 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.323+0000 7f298f7fe700 1 -- 172.21.15.133:0/828689833 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f299404c7b0 con 0x7f299807bde0 2023-11-27T15:13:49.965 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.324+0000 7f299dd1d700 1 --2- 172.21.15.133:0/828689833 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f2984038820 0x7f298403acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:49.965 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.324+0000 7f299dd1d700 1 --2- 172.21.15.133:0/828689833 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f2984038820 0x7f298403acd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2988006fd0 tx=0x7f2988006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:49.965 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.330+0000 7f298f7fe700 1 -- 172.21.15.133:0/828689833 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f29940246f0 con 0x7f299807bde0 2023-11-27T15:13:49.965 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.533+0000 7f29a0782700 1 -- 172.21.15.133:0/828689833 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f2998062120 con 0x7f299807bde0 2023-11-27T15:13:49.966 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.536+0000 7f298f7fe700 1 -- 172.21.15.133:0/828689833 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f2994018350 con 0x7f299807bde0 2023-11-27T15:13:49.966 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.539+0000 7f29a0782700 1 -- 172.21.15.133:0/828689833 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f2984038820 msgr2=0x7f298403acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:49.966 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.539+0000 7f29a0782700 1 --2- 172.21.15.133:0/828689833 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f2984038820 0x7f298403acd0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2988006fd0 tx=0x7f2988006e40 comp rx=0 tx=0).stop 2023-11-27T15:13:49.966 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.539+0000 7f29a0782700 1 -- 172.21.15.133:0/828689833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f299807bde0 msgr2=0x7f299819b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:49.966 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.539+0000 7f29a0782700 1 --2- 172.21.15.133:0/828689833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f299807bde0 0x7f299819b7a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2994000c00 tx=0x7f2994011a30 comp rx=0 tx=0).stop 2023-11-27T15:13:49.966 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.539+0000 7f29a0782700 1 -- 172.21.15.133:0/828689833 shutdown_connections 2023-11-27T15:13:49.966 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.539+0000 7f29a0782700 1 --2- 172.21.15.133:0/828689833 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f2984038820 0x7f298403acd0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:49.966 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.539+0000 7f29a0782700 1 --2- 172.21.15.133:0/828689833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f299807bde0 0x7f299819b7a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:49.966 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.539+0000 7f29a0782700 1 -- 172.21.15.133:0/828689833 >> 172.21.15.133:0/828689833 conn(0x7f2998100fa0 msgr2=0x7f2998101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:49.966 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.539+0000 7f29a0782700 1 -- 172.21.15.133:0/828689833 shutdown_connections 2023-11-27T15:13:49.967 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:49.539+0000 7f29a0782700 1 -- 172.21.15.133:0/828689833 wait complete. 2023-11-27T15:13:51.020 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:50 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/828689833' entity='client.admin' 2023-11-27T15:13:51.343 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.490+0000 7f35e47ff700 1 Processor -- start 2023-11-27T15:13:51.349 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.491+0000 7f35e47ff700 1 -- start start 2023-11-27T15:13:51.349 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.491+0000 7f35e47ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35dc105910 0x7f35dc105d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:51.349 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.491+0000 7f35e47ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc106260 con 0x7f35dc105910 2023-11-27T15:13:51.349 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.491+0000 7f35e259b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35dc105910 0x7f35dc105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:51.349 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.491+0000 7f35e259b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35dc105910 0x7f35dc105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52140/0 (socket says 172.21.15.133:52140) 2023-11-27T15:13:51.350 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.491+0000 7f35e259b700 1 -- 172.21.15.133:0/2568895926 learned_addr learned my addr 172.21.15.133:0/2568895926 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:51.350 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.492+0000 7f35e259b700 1 -- 172.21.15.133:0/2568895926 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc1063a0 con 0x7f35dc105910 2023-11-27T15:13:51.350 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.492+0000 7f35e259b700 1 --2- 172.21.15.133:0/2568895926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35dc105910 0x7f35dc105d20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f35d80098a0 tx=0x7f35d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901cd239e7ead222 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:51.350 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.492+0000 7f35e1599700 1 -- 172.21.15.133:0/2568895926 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35d8004160 con 0x7f35dc105910 2023-11-27T15:13:51.350 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.492+0000 7f35e1599700 1 -- 172.21.15.133:0/2568895926 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35d80042c0 con 0x7f35dc105910 2023-11-27T15:13:51.350 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.492+0000 7f35e1599700 1 -- 172.21.15.133:0/2568895926 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35d80119b0 con 0x7f35dc105910 2023-11-27T15:13:51.350 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.493+0000 7f35e47ff700 1 -- 172.21.15.133:0/2568895926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35dc105910 msgr2=0x7f35dc105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:51.351 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.493+0000 7f35e47ff700 1 --2- 172.21.15.133:0/2568895926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35dc105910 0x7f35dc105d20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f35d80098a0 tx=0x7f35d8009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:51.351 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.493+0000 7f35e47ff700 1 -- 172.21.15.133:0/2568895926 shutdown_connections 2023-11-27T15:13:51.351 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.493+0000 7f35e47ff700 1 --2- 172.21.15.133:0/2568895926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35dc105910 0x7f35dc105d20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:51.351 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.493+0000 7f35e47ff700 1 -- 172.21.15.133:0/2568895926 >> 172.21.15.133:0/2568895926 conn(0x7f35dc100fa0 msgr2=0x7f35dc1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:51.351 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.493+0000 7f35e47ff700 1 -- 172.21.15.133:0/2568895926 shutdown_connections 2023-11-27T15:13:51.351 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.493+0000 7f35e47ff700 1 -- 172.21.15.133:0/2568895926 wait complete. 2023-11-27T15:13:51.351 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.494+0000 7f35e47ff700 1 Processor -- start 2023-11-27T15:13:51.352 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.495+0000 7f35e47ff700 1 -- start start 2023-11-27T15:13:51.352 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.495+0000 7f35e47ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35dc105910 0x7f35dc19b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:51.352 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.495+0000 7f35e47ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc19bc90 con 0x7f35dc105910 2023-11-27T15:13:51.352 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.495+0000 7f35e259b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35dc105910 0x7f35dc19b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:51.352 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.495+0000 7f35e259b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35dc105910 0x7f35dc19b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52152/0 (socket says 172.21.15.133:52152) 2023-11-27T15:13:51.352 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.495+0000 7f35e259b700 1 -- 172.21.15.133:0/729129961 learned_addr learned my addr 172.21.15.133:0/729129961 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:51.353 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.496+0000 7f35e259b700 1 -- 172.21.15.133:0/729129961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d8009550 con 0x7f35dc105910 2023-11-27T15:13:51.353 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.496+0000 7f35e259b700 1 --2- 172.21.15.133:0/729129961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35dc105910 0x7f35dc19b750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f35d8011980 tx=0x7f35d8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:51.353 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.496+0000 7f35d37fe700 1 -- 172.21.15.133:0/729129961 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35d8003a60 con 0x7f35dc105910 2023-11-27T15:13:51.353 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.496+0000 7f35d37fe700 1 -- 172.21.15.133:0/729129961 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35d8003bc0 con 0x7f35dc105910 2023-11-27T15:13:51.353 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.496+0000 7f35d37fe700 1 -- 172.21.15.133:0/729129961 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35d801a670 con 0x7f35dc105910 2023-11-27T15:13:51.353 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.496+0000 7f35e47ff700 1 -- 172.21.15.133:0/729129961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35dc1079a0 con 0x7f35dc105910 2023-11-27T15:13:51.353 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.496+0000 7f35e47ff700 1 -- 172.21.15.133:0/729129961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35dc107810 con 0x7f35dc105910 2023-11-27T15:13:51.354 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.497+0000 7f35e47ff700 1 -- 172.21.15.133:0/729129961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35dc109de0 con 0x7f35dc105910 2023-11-27T15:13:51.354 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.497+0000 7f35d37fe700 1 -- 172.21.15.133:0/729129961 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f35d8024a00 con 0x7f35dc105910 2023-11-27T15:13:51.354 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.498+0000 7f35d37fe700 1 --2- 172.21.15.133:0/729129961 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f35c8038820 0x7f35c803acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:51.354 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.498+0000 7f35d37fe700 1 -- 172.21.15.133:0/729129961 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f35d804c8a0 con 0x7f35dc105910 2023-11-27T15:13:51.354 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.498+0000 7f35e1d9a700 1 --2- 172.21.15.133:0/729129961 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f35c8038820 0x7f35c803acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:51.354 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.499+0000 7f35e1d9a700 1 --2- 172.21.15.133:0/729129961 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f35c8038820 0x7f35c803acd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f35cc006fd0 tx=0x7f35cc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:51.355 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.503+0000 7f35d37fe700 1 -- 172.21.15.133:0/729129961 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f35d8024440 con 0x7f35dc105910 2023-11-27T15:13:51.355 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.709+0000 7f35e47ff700 1 -- 172.21.15.133:0/729129961 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f35dc19f3d0 con 0x7f35dc105910 2023-11-27T15:13:51.355 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.716+0000 7f35d37fe700 1 -- 172.21.15.133:0/729129961 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f35d8018350 con 0x7f35dc105910 2023-11-27T15:13:51.355 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.718+0000 7f35e47ff700 1 -- 172.21.15.133:0/729129961 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f35c8038820 msgr2=0x7f35c803acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:51.355 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.718+0000 7f35e47ff700 1 --2- 172.21.15.133:0/729129961 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f35c8038820 0x7f35c803acd0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f35cc006fd0 tx=0x7f35cc006e40 comp rx=0 tx=0).stop 2023-11-27T15:13:51.355 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.718+0000 7f35e47ff700 1 -- 172.21.15.133:0/729129961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35dc105910 msgr2=0x7f35dc19b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:51.355 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.718+0000 7f35e47ff700 1 --2- 172.21.15.133:0/729129961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35dc105910 0x7f35dc19b750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f35d8011980 tx=0x7f35d8011a60 comp rx=0 tx=0).stop 2023-11-27T15:13:51.356 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.718+0000 7f35e47ff700 1 -- 172.21.15.133:0/729129961 shutdown_connections 2023-11-27T15:13:51.356 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.718+0000 7f35e47ff700 1 --2- 172.21.15.133:0/729129961 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f35c8038820 0x7f35c803acd0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:51.356 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.718+0000 7f35e47ff700 1 --2- 172.21.15.133:0/729129961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35dc105910 0x7f35dc19b750 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:51.356 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.718+0000 7f35e47ff700 1 -- 172.21.15.133:0/729129961 >> 172.21.15.133:0/729129961 conn(0x7f35dc100fa0 msgr2=0x7f35dc101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:51.356 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.718+0000 7f35e47ff700 1 -- 172.21.15.133:0/729129961 shutdown_connections 2023-11-27T15:13:51.356 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:50.718+0000 7f35e47ff700 1 -- 172.21.15.133:0/729129961 wait complete. 2023-11-27T15:13:51.356 INFO:teuthology.orchestra.run.smithi133.stdout:Enabling the dashboard module... 2023-11-27T15:13:52.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:51 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/729129961' entity='client.admin' 2023-11-27T15:13:52.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:51 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' 2023-11-27T15:13:53.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:53 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3754852782' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-11-27T15:13:53.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:53 smithi133 ceph-mon[97860]: from='mgr.14118 172.21.15.133:0/528833833' entity='mgr.x' 2023-11-27T15:13:53.278 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:53 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: ignoring --setuser ceph since I am not root 2023-11-27T15:13:53.278 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:53 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: ignoring --setgroup ceph since I am not root 2023-11-27T15:13:53.434 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.909+0000 7f345c525700 1 Processor -- start 2023-11-27T15:13:53.434 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.909+0000 7f345c525700 1 -- start start 2023-11-27T15:13:53.434 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.910+0000 7f345c525700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3454107af0 0x7f3454107f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:53.434 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.910+0000 7f345c525700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3454108440 con 0x7f3454107af0 2023-11-27T15:13:53.434 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.910+0000 7f345a2c1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3454107af0 0x7f3454107f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:53.434 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.910+0000 7f345a2c1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3454107af0 0x7f3454107f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52154/0 (socket says 172.21.15.133:52154) 2023-11-27T15:13:53.434 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.910+0000 7f345a2c1700 1 -- 172.21.15.133:0/3568043796 learned_addr learned my addr 172.21.15.133:0/3568043796 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:53.434 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.910+0000 7f345a2c1700 1 -- 172.21.15.133:0/3568043796 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3454108580 con 0x7f3454107af0 2023-11-27T15:13:53.434 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.911+0000 7f345a2c1700 1 --2- 172.21.15.133:0/3568043796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3454107af0 0x7f3454107f00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f34500098a0 tx=0x7f3450009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7b5116d6249484 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:53.434 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.911+0000 7f34592bf700 1 -- 172.21.15.133:0/3568043796 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3450004160 con 0x7f3454107af0 2023-11-27T15:13:53.435 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.911+0000 7f34592bf700 1 -- 172.21.15.133:0/3568043796 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34500042c0 con 0x7f3454107af0 2023-11-27T15:13:53.435 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.911+0000 7f34592bf700 1 -- 172.21.15.133:0/3568043796 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34500119b0 con 0x7f3454107af0 2023-11-27T15:13:53.435 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.911+0000 7f345c525700 1 -- 172.21.15.133:0/3568043796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3454107af0 msgr2=0x7f3454107f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:53.435 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.911+0000 7f345c525700 1 --2- 172.21.15.133:0/3568043796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3454107af0 0x7f3454107f00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f34500098a0 tx=0x7f3450009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:53.435 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.912+0000 7f345c525700 1 -- 172.21.15.133:0/3568043796 shutdown_connections 2023-11-27T15:13:53.435 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.912+0000 7f345c525700 1 --2- 172.21.15.133:0/3568043796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3454107af0 0x7f3454107f00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:53.435 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.912+0000 7f345c525700 1 -- 172.21.15.133:0/3568043796 >> 172.21.15.133:0/3568043796 conn(0x7f345407afb0 msgr2=0x7f345407b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:53.435 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.912+0000 7f345c525700 1 -- 172.21.15.133:0/3568043796 shutdown_connections 2023-11-27T15:13:53.435 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.912+0000 7f345c525700 1 -- 172.21.15.133:0/3568043796 wait complete. 2023-11-27T15:13:53.435 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.913+0000 7f345c525700 1 Processor -- start 2023-11-27T15:13:53.435 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.913+0000 7f345c525700 1 -- start start 2023-11-27T15:13:53.435 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.914+0000 7f345c525700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3454107af0 0x7f345419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:53.436 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.914+0000 7f345c525700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345419bca0 con 0x7f3454107af0 2023-11-27T15:13:53.436 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.914+0000 7f345a2c1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3454107af0 0x7f345419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:53.436 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.914+0000 7f345a2c1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3454107af0 0x7f345419b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52166/0 (socket says 172.21.15.133:52166) 2023-11-27T15:13:53.436 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.914+0000 7f345a2c1700 1 -- 172.21.15.133:0/3754852782 learned_addr learned my addr 172.21.15.133:0/3754852782 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:53.436 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.914+0000 7f345a2c1700 1 -- 172.21.15.133:0/3754852782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3450009550 con 0x7f3454107af0 2023-11-27T15:13:53.436 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.915+0000 7f345a2c1700 1 --2- 172.21.15.133:0/3754852782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3454107af0 0x7f345419b760 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3450011980 tx=0x7f3450011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:53.436 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.915+0000 7f344b7fe700 1 -- 172.21.15.133:0/3754852782 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3450003a60 con 0x7f3454107af0 2023-11-27T15:13:53.436 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.915+0000 7f344b7fe700 1 -- 172.21.15.133:0/3754852782 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3450003bc0 con 0x7f3454107af0 2023-11-27T15:13:53.436 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.915+0000 7f344b7fe700 1 -- 172.21.15.133:0/3754852782 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f345001a670 con 0x7f3454107af0 2023-11-27T15:13:53.436 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.915+0000 7f345c525700 1 -- 172.21.15.133:0/3754852782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3454109b80 con 0x7f3454107af0 2023-11-27T15:13:53.436 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.915+0000 7f345c525700 1 -- 172.21.15.133:0/3754852782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34541099f0 con 0x7f3454107af0 2023-11-27T15:13:53.436 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.916+0000 7f345c525700 1 -- 172.21.15.133:0/3754852782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f345410bfc0 con 0x7f3454107af0 2023-11-27T15:13:53.437 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.917+0000 7f344b7fe700 1 -- 172.21.15.133:0/3754852782 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f3450024a00 con 0x7f3454107af0 2023-11-27T15:13:53.437 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.917+0000 7f344b7fe700 1 --2- 172.21.15.133:0/3754852782 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f34400387d0 0x7f344003ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:53.437 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.917+0000 7f344b7fe700 1 -- 172.21.15.133:0/3754852782 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f345004c8e0 con 0x7f3454107af0 2023-11-27T15:13:53.437 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.917+0000 7f3459ac0700 1 --2- 172.21.15.133:0/3754852782 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f34400387d0 0x7f344003ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:53.437 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.918+0000 7f3459ac0700 1 --2- 172.21.15.133:0/3754852782 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f34400387d0 0x7f344003ac80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3444006fd0 tx=0x7f3444006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:53.437 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:51.922+0000 7f344b7fe700 1 -- 172.21.15.133:0/3754852782 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f34500189a0 con 0x7f3454107af0 2023-11-27T15:13:53.437 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:52.223+0000 7f345c525700 1 -- 172.21.15.133:0/3754852782 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f345419f370 con 0x7f3454107af0 2023-11-27T15:13:53.437 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:53.009+0000 7f344b7fe700 1 -- 172.21.15.133:0/3754852782 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f345002d910 con 0x7f3454107af0 2023-11-27T15:13:53.437 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:53.010+0000 7f344b7fe700 1 -- 172.21.15.133:0/3754852782 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f345004c270 con 0x7f3454107af0 2023-11-27T15:13:53.437 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:53.014+0000 7f345c525700 1 -- 172.21.15.133:0/3754852782 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f34400387d0 msgr2=0x7f344003ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:53.437 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:53.014+0000 7f345c525700 1 --2- 172.21.15.133:0/3754852782 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f34400387d0 0x7f344003ac80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3444006fd0 tx=0x7f3444006e40 comp rx=0 tx=0).stop 2023-11-27T15:13:53.437 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:53.014+0000 7f345c525700 1 -- 172.21.15.133:0/3754852782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3454107af0 msgr2=0x7f345419b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:53.438 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:53.014+0000 7f345c525700 1 --2- 172.21.15.133:0/3754852782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3454107af0 0x7f345419b760 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3450011980 tx=0x7f3450011a60 comp rx=0 tx=0).stop 2023-11-27T15:13:53.438 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:53.014+0000 7f345c525700 1 -- 172.21.15.133:0/3754852782 shutdown_connections 2023-11-27T15:13:53.438 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:53.014+0000 7f345c525700 1 --2- 172.21.15.133:0/3754852782 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f34400387d0 0x7f344003ac80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:53.438 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:53.014+0000 7f345c525700 1 --2- 172.21.15.133:0/3754852782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3454107af0 0x7f345419b760 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:53.438 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:53.014+0000 7f345c525700 1 -- 172.21.15.133:0/3754852782 >> 172.21.15.133:0/3754852782 conn(0x7f345407afb0 msgr2=0x7f34541056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:53.438 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:53.014+0000 7f345c525700 1 -- 172.21.15.133:0/3754852782 shutdown_connections 2023-11-27T15:13:53.438 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:53.014+0000 7f345c525700 1 -- 172.21.15.133:0/3754852782 wait complete. 2023-11-27T15:13:53.706 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:53 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:53.335+0000 7f98cbc66200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-27T15:13:53.706 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:53 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:53.449+0000 7f98cbc66200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-27T15:13:53.706 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:53 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:53.573+0000 7f98cbc66200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-27T15:13:53.706 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:53 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:53.701+0000 7f98cbc66200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-27T15:13:54.014 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:53 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:53.940+0000 7f98cbc66200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-27T15:13:54.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:54 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3754852782' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-11-27T15:13:54.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:54 smithi133 ceph-mon[97860]: mgrmap e9: x(active, since 17s) 2023-11-27T15:13:54.580 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:54 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:54.360+0000 7f98cbc66200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-27T15:13:54.580 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:54 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:54.486+0000 7f98cbc66200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-27T15:13:54.928 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:54 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:54.667+0000 7f98cbc66200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-27T15:13:54.929 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:54 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:54.796+0000 7f98cbc66200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-27T15:13:55.211 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:13:55.216 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-11-27T15:13:55.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "available": true, 2023-11-27T15:13:55.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-11-27T15:13:55.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-11-27T15:13:55.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:13:55.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.007+0000 7ff34abc7700 1 Processor -- start 2023-11-27T15:13:55.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.007+0000 7ff34abc7700 1 -- start start 2023-11-27T15:13:55.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.008+0000 7ff34abc7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34407be00 0x7ff34407a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:55.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.008+0000 7ff34abc7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34407a840 con 0x7ff34407be00 2023-11-27T15:13:55.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.009+0000 7ff348963700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34407be00 0x7ff34407a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:55.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.009+0000 7ff348963700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34407be00 0x7ff34407a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35840/0 (socket says 172.21.15.133:35840) 2023-11-27T15:13:55.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.009+0000 7ff348963700 1 -- 172.21.15.133:0/4104420355 learned_addr learned my addr 172.21.15.133:0/4104420355 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:55.217 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.010+0000 7ff348963700 1 -- 172.21.15.133:0/4104420355 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34407a980 con 0x7ff34407be00 2023-11-27T15:13:55.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.010+0000 7ff348963700 1 --2- 172.21.15.133:0/4104420355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34407be00 0x7ff34407a300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff3380098a0 tx=0x7ff338009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e171299816c4f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:55.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.011+0000 7ff3437fe700 1 -- 172.21.15.133:0/4104420355 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff338004160 con 0x7ff34407be00 2023-11-27T15:13:55.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.012+0000 7ff3437fe700 1 -- 172.21.15.133:0/4104420355 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3380042c0 con 0x7ff34407be00 2023-11-27T15:13:55.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.012+0000 7ff3437fe700 1 -- 172.21.15.133:0/4104420355 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3380119b0 con 0x7ff34407be00 2023-11-27T15:13:55.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.012+0000 7ff34abc7700 1 -- 172.21.15.133:0/4104420355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34407be00 msgr2=0x7ff34407a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:55.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.012+0000 7ff34abc7700 1 --2- 172.21.15.133:0/4104420355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34407be00 0x7ff34407a300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff3380098a0 tx=0x7ff338009bb0 comp rx=0 tx=0).stop 2023-11-27T15:13:55.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.013+0000 7ff34abc7700 1 -- 172.21.15.133:0/4104420355 shutdown_connections 2023-11-27T15:13:55.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.013+0000 7ff34abc7700 1 --2- 172.21.15.133:0/4104420355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34407be00 0x7ff34407a300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:55.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.013+0000 7ff34abc7700 1 -- 172.21.15.133:0/4104420355 >> 172.21.15.133:0/4104420355 conn(0x7ff344100fa0 msgr2=0x7ff3441033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:55.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.014+0000 7ff34abc7700 1 -- 172.21.15.133:0/4104420355 shutdown_connections 2023-11-27T15:13:55.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.014+0000 7ff34abc7700 1 -- 172.21.15.133:0/4104420355 wait complete. 2023-11-27T15:13:55.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.015+0000 7ff34abc7700 1 Processor -- start 2023-11-27T15:13:55.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.016+0000 7ff34abc7700 1 -- start start 2023-11-27T15:13:55.218 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.016+0000 7ff34abc7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34407be00 0x7ff34419b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:55.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.016+0000 7ff34abc7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34419bd10 con 0x7ff34407be00 2023-11-27T15:13:55.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.016+0000 7ff348963700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34407be00 0x7ff34419b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:13:55.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.016+0000 7ff348963700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34407be00 0x7ff34419b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35854/0 (socket says 172.21.15.133:35854) 2023-11-27T15:13:55.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.016+0000 7ff348963700 1 -- 172.21.15.133:0/3691062855 learned_addr learned my addr 172.21.15.133:0/3691062855 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:13:55.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.017+0000 7ff348963700 1 -- 172.21.15.133:0/3691062855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff338009550 con 0x7ff34407be00 2023-11-27T15:13:55.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.018+0000 7ff348963700 1 --2- 172.21.15.133:0/3691062855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34407be00 0x7ff34419b7d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff338003e60 tx=0x7ff338011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:13:55.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.018+0000 7ff341ffb700 1 -- 172.21.15.133:0/3691062855 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff338003b20 con 0x7ff34407be00 2023-11-27T15:13:55.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.018+0000 7ff34abc7700 1 -- 172.21.15.133:0/3691062855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34410be30 con 0x7ff34407be00 2023-11-27T15:13:55.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.018+0000 7ff341ffb700 1 -- 172.21.15.133:0/3691062855 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff338003c80 con 0x7ff34407be00 2023-11-27T15:13:55.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.018+0000 7ff341ffb700 1 -- 172.21.15.133:0/3691062855 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff338024760 con 0x7ff34407be00 2023-11-27T15:13:55.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.018+0000 7ff34abc7700 1 -- 172.21.15.133:0/3691062855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34410bca0 con 0x7ff34407be00 2023-11-27T15:13:55.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.020+0000 7ff34abc7700 1 -- 172.21.15.133:0/3691062855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff34410bfc0 con 0x7ff34407be00 2023-11-27T15:13:55.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.020+0000 7ff341ffb700 1 -- 172.21.15.133:0/3691062855 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7ff33801a430 con 0x7ff34407be00 2023-11-27T15:13:55.219 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.020+0000 7ff341ffb700 1 --2- 172.21.15.133:0/3691062855 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7ff334038870 0x7ff33403ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:13:55.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.021+0000 7ff341ffb700 1 -- 172.21.15.133:0/3691062855 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff33804caa0 con 0x7ff34407be00 2023-11-27T15:13:55.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.021+0000 7ff343fff700 1 -- 172.21.15.133:0/3691062855 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7ff334038870 msgr2=0x7ff33403ad20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2374615220 2023-11-27T15:13:55.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.021+0000 7ff343fff700 1 --2- 172.21.15.133:0/3691062855 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7ff334038870 0x7ff33403ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:13:55.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.029+0000 7ff341ffb700 1 -- 172.21.15.133:0/3691062855 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff338018b40 con 0x7ff34407be00 2023-11-27T15:13:55.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.222+0000 7ff343fff700 1 -- 172.21.15.133:0/3691062855 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7ff334038870 msgr2=0x7ff33403ad20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2374615220 2023-11-27T15:13:55.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.222+0000 7ff343fff700 1 --2- 172.21.15.133:0/3691062855 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7ff334038870 0x7ff33403ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-27T15:13:55.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.296+0000 7ff34abc7700 1 -- 172.21.15.133:0/3691062855 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7ff34419f400 con 0x7ff34407be00 2023-11-27T15:13:55.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.297+0000 7ff341ffb700 1 -- 172.21.15.133:0/3691062855 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7ff338018b40 con 0x7ff34407be00 2023-11-27T15:13:55.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.299+0000 7ff34abc7700 1 -- 172.21.15.133:0/3691062855 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7ff334038870 msgr2=0x7ff33403ad20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:13:55.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.299+0000 7ff34abc7700 1 --2- 172.21.15.133:0/3691062855 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7ff334038870 0x7ff33403ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:55.220 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.299+0000 7ff34abc7700 1 -- 172.21.15.133:0/3691062855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34407be00 msgr2=0x7ff34419b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:13:55.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.299+0000 7ff34abc7700 1 --2- 172.21.15.133:0/3691062855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34407be00 0x7ff34419b7d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff338003e60 tx=0x7ff338011a30 comp rx=0 tx=0).stop 2023-11-27T15:13:55.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.299+0000 7ff34abc7700 1 -- 172.21.15.133:0/3691062855 shutdown_connections 2023-11-27T15:13:55.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.299+0000 7ff34abc7700 1 --2- 172.21.15.133:0/3691062855 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7ff334038870 0x7ff33403ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:55.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.299+0000 7ff34abc7700 1 --2- 172.21.15.133:0/3691062855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff34407be00 0x7ff34419b7d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:13:55.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.299+0000 7ff34abc7700 1 -- 172.21.15.133:0/3691062855 >> 172.21.15.133:0/3691062855 conn(0x7ff344100fa0 msgr2=0x7ff344101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:13:55.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.299+0000 7ff34abc7700 1 -- 172.21.15.133:0/3691062855 shutdown_connections 2023-11-27T15:13:55.221 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:54.299+0000 7ff34abc7700 1 -- 172.21.15.133:0/3691062855 wait complete. 2023-11-27T15:13:55.221 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for the mgr to restart... 2023-11-27T15:13:55.221 INFO:teuthology.orchestra.run.smithi133.stdout:Waiting for mgr epoch 9... 2023-11-27T15:13:55.253 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:13:55 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3691062855' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-11-27T15:13:55.513 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:55 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:55.371+0000 7f98cbc66200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-27T15:13:56.527 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:56 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:56.136+0000 7f98cbc66200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-27T15:13:56.528 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:56 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:56.346+0000 7f98cbc66200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-27T15:13:57.628 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:57 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:57.286+0000 7f98cbc66200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:13:57.628 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:57 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:57.404+0000 7f98cbc66200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-27T15:13:57.628 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:57 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:57.622+0000 7f98cbc66200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:13:58.263 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:57 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:57.979+0000 7f98cbc66200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-27T15:13:58.527 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:58 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:58.258+0000 7f98cbc66200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-27T15:13:58.528 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:58 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:58.449+0000 7f98cbc66200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-27T15:13:59.277 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:58 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:58.792+0000 7f98cbc66200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-27T15:14:00.027 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:59 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:59.721+0000 7f98cbc66200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-27T15:14:00.028 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:13:59 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:13:59.891+0000 7f98cbc66200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-27T15:14:00.527 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:14:00 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:14:00.137+0000 7f98cbc66200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-27T15:14:00.527 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:14:00 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:14:00.286+0000 7f98cbc66200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-27T15:14:01.027 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:14:00 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:14:00.564+0000 7f98cbc66200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-27T15:14:02.028 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:14:01 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:14:01.624+0000 7f98cbc66200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-27T15:14:02.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:01 smithi133 ceph-mon[97860]: Active manager daemon x restarted 2023-11-27T15:14:02.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:01 smithi133 ceph-mon[97860]: Activating manager daemon x 2023-11-27T15:14:02.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:01 smithi133 ceph-mon[97860]: osdmap e3: 0 total, 0 up, 0 in 2023-11-27T15:14:02.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:01 smithi133 ceph-mon[97860]: mgrmap e10: x(active, starting, since 0.00402949s) 2023-11-27T15:14:02.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:14:02.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-11-27T15:14:02.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-11-27T15:14:02.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-11-27T15:14:02.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-11-27T15:14:02.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:01 smithi133 ceph-mon[97860]: Manager daemon x is now available 2023-11-27T15:14:02.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:02.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:03.028 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout { 2023-11-27T15:14:03.028 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-11-27T15:14:03.028 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout "initialized": true 2023-11-27T15:14:03.028 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout } 2023-11-27T15:14:03.028 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.741+0000 7f1b6a65c700 1 Processor -- start 2023-11-27T15:14:03.028 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.742+0000 7f1b6a65c700 1 -- start start 2023-11-27T15:14:03.028 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.742+0000 7f1b6a65c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6407be20 0x7f1b6407a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:03.028 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.742+0000 7f1b6a65c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6407a860 con 0x7f1b6407be20 2023-11-27T15:14:03.028 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.742+0000 7f1b63fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6407be20 0x7f1b6407a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:03.028 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.742+0000 7f1b63fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6407be20 0x7f1b6407a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35868/0 (socket says 172.21.15.133:35868) 2023-11-27T15:14:03.029 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.742+0000 7f1b63fff700 1 -- 172.21.15.133:0/1233021270 learned_addr learned my addr 172.21.15.133:0/1233021270 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:14:03.029 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.743+0000 7f1b63fff700 1 -- 172.21.15.133:0/1233021270 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6407a9a0 con 0x7f1b6407be20 2023-11-27T15:14:03.029 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.743+0000 7f1b63fff700 1 --2- 172.21.15.133:0/1233021270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6407be20 0x7f1b6407a320 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1b540098a0 tx=0x7f1b54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346e1ecd01988756 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:03.029 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.743+0000 7f1b62ffd700 1 -- 172.21.15.133:0/1233021270 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b54004160 con 0x7f1b6407be20 2023-11-27T15:14:03.029 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.743+0000 7f1b62ffd700 1 -- 172.21.15.133:0/1233021270 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b540042c0 con 0x7f1b6407be20 2023-11-27T15:14:03.029 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.743+0000 7f1b62ffd700 1 -- 172.21.15.133:0/1233021270 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b540119b0 con 0x7f1b6407be20 2023-11-27T15:14:03.029 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.744+0000 7f1b6a65c700 1 -- 172.21.15.133:0/1233021270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6407be20 msgr2=0x7f1b6407a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:03.029 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.744+0000 7f1b6a65c700 1 --2- 172.21.15.133:0/1233021270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6407be20 0x7f1b6407a320 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1b540098a0 tx=0x7f1b54009bb0 comp rx=0 tx=0).stop 2023-11-27T15:14:03.029 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.744+0000 7f1b6a65c700 1 -- 172.21.15.133:0/1233021270 shutdown_connections 2023-11-27T15:14:03.029 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.744+0000 7f1b6a65c700 1 --2- 172.21.15.133:0/1233021270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6407be20 0x7f1b6407a320 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:03.029 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.744+0000 7f1b6a65c700 1 -- 172.21.15.133:0/1233021270 >> 172.21.15.133:0/1233021270 conn(0x7f1b64100fa0 msgr2=0x7f1b641033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:03.029 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.744+0000 7f1b6a65c700 1 -- 172.21.15.133:0/1233021270 shutdown_connections 2023-11-27T15:14:03.029 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.744+0000 7f1b6a65c700 1 -- 172.21.15.133:0/1233021270 wait complete. 2023-11-27T15:14:03.029 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.745+0000 7f1b6a65c700 1 Processor -- start 2023-11-27T15:14:03.030 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.745+0000 7f1b6a65c700 1 -- start start 2023-11-27T15:14:03.030 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.746+0000 7f1b6a65c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6407be20 0x7f1b6419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:03.030 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.746+0000 7f1b6a65c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6419bcf0 con 0x7f1b6407be20 2023-11-27T15:14:03.030 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.746+0000 7f1b63fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6407be20 0x7f1b6419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:03.030 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.746+0000 7f1b63fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6407be20 0x7f1b6419b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35878/0 (socket says 172.21.15.133:35878) 2023-11-27T15:14:03.030 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.746+0000 7f1b63fff700 1 -- 172.21.15.133:0/935499372 learned_addr learned my addr 172.21.15.133:0/935499372 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:14:03.030 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.746+0000 7f1b63fff700 1 -- 172.21.15.133:0/935499372 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b54009550 con 0x7f1b6407be20 2023-11-27T15:14:03.030 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.747+0000 7f1b63fff700 1 --2- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6407be20 0x7f1b6419b7b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1b54000c00 tx=0x7f1b54011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:03.030 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.747+0000 7f1b617fa700 1 -- 172.21.15.133:0/935499372 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b54003ac0 con 0x7f1b6407be20 2023-11-27T15:14:03.030 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.747+0000 7f1b617fa700 1 -- 172.21.15.133:0/935499372 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b54003c20 con 0x7f1b6407be20 2023-11-27T15:14:03.030 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.747+0000 7f1b6a65c700 1 -- 172.21.15.133:0/935499372 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6410be10 con 0x7f1b6407be20 2023-11-27T15:14:03.030 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.747+0000 7f1b617fa700 1 -- 172.21.15.133:0/935499372 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b5401a720 con 0x7f1b6407be20 2023-11-27T15:14:03.030 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.747+0000 7f1b6a65c700 1 -- 172.21.15.133:0/935499372 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6410bc80 con 0x7f1b6407be20 2023-11-27T15:14:03.031 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.748+0000 7f1b617fa700 1 -- 172.21.15.133:0/935499372 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f1b5401a990 con 0x7f1b6407be20 2023-11-27T15:14:03.031 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.749+0000 7f1b617fa700 1 --2- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f1b4c038870 0x7f1b4c03ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:03.031 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.749+0000 7f1b617fa700 1 -- 172.21.15.133:0/935499372 --> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b4c03b430 con 0x7f1b4c038870 2023-11-27T15:14:03.031 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.749+0000 7f1b617fa700 1 -- 172.21.15.133:0/935499372 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1b5404cb50 con 0x7f1b6407be20 2023-11-27T15:14:03.031 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.749+0000 7f1b637fe700 1 -- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f1b4c038870 msgr2=0x7f1b4c03ad20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2374615220 2023-11-27T15:14:03.031 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.749+0000 7f1b637fe700 1 --2- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f1b4c038870 0x7f1b4c03ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:14:03.031 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.950+0000 7f1b637fe700 1 -- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f1b4c038870 msgr2=0x7f1b4c03ad20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2374615220 2023-11-27T15:14:03.031 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:55.950+0000 7f1b637fe700 1 --2- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f1b4c038870 0x7f1b4c03ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-27T15:14:03.031 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:56.351+0000 7f1b637fe700 1 -- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f1b4c038870 msgr2=0x7f1b4c03ad20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2374615220 2023-11-27T15:14:03.031 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:56.351+0000 7f1b637fe700 1 --2- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f1b4c038870 0x7f1b4c03ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-27T15:14:03.031 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:57.152+0000 7f1b637fe700 1 -- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f1b4c038870 msgr2=0x7f1b4c03ad20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2374615220 2023-11-27T15:14:03.031 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:57.152+0000 7f1b637fe700 1 --2- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f1b4c038870 0x7f1b4c03ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-27T15:14:03.032 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:58.753+0000 7f1b637fe700 1 -- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f1b4c038870 msgr2=0x7f1b4c03ad20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.133:6800/2374615220 2023-11-27T15:14:03.032 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:13:58.753+0000 7f1b637fe700 1 --2- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f1b4c038870 0x7f1b4c03ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-11-27T15:14:03.032 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:01.629+0000 7f1b617fa700 1 -- 172.21.15.133:0/935499372 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7f1b5402d3f0 con 0x7f1b6407be20 2023-11-27T15:14:03.032 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:01.629+0000 7f1b617fa700 1 -- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f1b4c038870 msgr2=0x7f1b4c03ad20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:14:03.032 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:01.629+0000 7f1b617fa700 1 --2- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f1b4c038870 0x7f1b4c03ad20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:03.032 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:02.633+0000 7f1b617fa700 1 -- 172.21.15.133:0/935499372 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7f1b5404e370 con 0x7f1b6407be20 2023-11-27T15:14:03.032 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:02.633+0000 7f1b617fa700 1 --2- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1b4c03c1a0 0x7f1b4c03e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:03.032 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:02.633+0000 7f1b617fa700 1 -- 172.21.15.133:0/935499372 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1b4c03b430 con 0x7f1b4c03c1a0 2023-11-27T15:14:03.032 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:02.634+0000 7f1b637fe700 1 --2- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1b4c03c1a0 0x7f1b4c03e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:03.032 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:02.635+0000 7f1b637fe700 1 --2- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1b4c03c1a0 0x7f1b4c03e580 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1b58003a10 tx=0x7f1b58009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:03.032 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:02.636+0000 7f1b617fa700 1 -- 172.21.15.133:0/935499372 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f1b4c03b430 con 0x7f1b4c03c1a0 2023-11-27T15:14:03.032 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:02.647+0000 7f1b6a65c700 1 -- 172.21.15.133:0/935499372 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f1b6419caf0 con 0x7f1b4c03c1a0 2023-11-27T15:14:03.032 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:02.647+0000 7f1b617fa700 1 -- 172.21.15.133:0/935499372 <== mgr.14150 v2:172.21.15.133:6800/4209056668 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f1b6419caf0 con 0x7f1b4c03c1a0 2023-11-27T15:14:03.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:02.648+0000 7f1b6a65c700 1 -- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1b4c03c1a0 msgr2=0x7f1b4c03e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:03.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:02.648+0000 7f1b6a65c700 1 --2- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1b4c03c1a0 0x7f1b4c03e580 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1b58003a10 tx=0x7f1b58009270 comp rx=0 tx=0).stop 2023-11-27T15:14:03.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:02.648+0000 7f1b6a65c700 1 -- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6407be20 msgr2=0x7f1b6419b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:03.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:02.648+0000 7f1b6a65c700 1 --2- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6407be20 0x7f1b6419b7b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1b54000c00 tx=0x7f1b54011a80 comp rx=0 tx=0).stop 2023-11-27T15:14:03.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:02.648+0000 7f1b6a65c700 1 -- 172.21.15.133:0/935499372 shutdown_connections 2023-11-27T15:14:03.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:02.648+0000 7f1b6a65c700 1 --2- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1b4c03c1a0 0x7f1b4c03e580 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:03.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:02.648+0000 7f1b6a65c700 1 --2- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b6407be20 0x7f1b6419b7b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:03.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:02.648+0000 7f1b6a65c700 1 --2- 172.21.15.133:0/935499372 >> [v2:172.21.15.133:6800/2374615220,v1:172.21.15.133:6801/2374615220] conn(0x7f1b4c038870 0x7f1b4c03ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:03.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:02.648+0000 7f1b6a65c700 1 -- 172.21.15.133:0/935499372 >> 172.21.15.133:0/935499372 conn(0x7f1b64100fa0 msgr2=0x7f1b64101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:03.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:02.648+0000 7f1b6a65c700 1 -- 172.21.15.133:0/935499372 shutdown_connections 2023-11-27T15:14:03.033 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:02.648+0000 7f1b6a65c700 1 -- 172.21.15.133:0/935499372 wait complete. 2023-11-27T15:14:03.033 INFO:teuthology.orchestra.run.smithi133.stdout:mgr epoch 9 is available 2023-11-27T15:14:03.033 INFO:teuthology.orchestra.run.smithi133.stdout:Generating a dashboard self-signed certificate... 2023-11-27T15:14:03.034 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-11-27T15:14:03.034 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-11-27T15:14:03.034 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:02 smithi133 ceph-mon[97860]: mgrmap e11: x(active, since 1.00992s) 2023-11-27T15:14:03.956 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:03 smithi133 ceph-mon[97860]: [27/Nov/2023:15:14:02] ENGINE Bus STARTING 2023-11-27T15:14:03.956 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:03 smithi133 ceph-mon[97860]: [27/Nov/2023:15:14:02] ENGINE Serving on http://172.21.15.133:8765 2023-11-27T15:14:03.956 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:03 smithi133 ceph-mon[97860]: [27/Nov/2023:15:14:03] ENGINE Serving on https://172.21.15.133:7150 2023-11-27T15:14:03.956 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:03 smithi133 ceph-mon[97860]: [27/Nov/2023:15:14:03] ENGINE Bus STARTED 2023-11-27T15:14:04.410 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-11-27T15:14:04.410 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.571+0000 7eff1e0c9700 1 Processor -- start 2023-11-27T15:14:04.410 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.572+0000 7eff1e0c9700 1 -- start start 2023-11-27T15:14:04.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.572+0000 7eff1e0c9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff18079d00 0x7eff1810a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:04.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.572+0000 7eff1e0c9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff18106280 con 0x7eff18079d00 2023-11-27T15:14:04.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.573+0000 7eff177fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff18079d00 0x7eff1810a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:04.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.573+0000 7eff177fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff18079d00 0x7eff1810a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34160/0 (socket says 172.21.15.133:34160) 2023-11-27T15:14:04.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.573+0000 7eff177fe700 1 -- 172.21.15.133:0/2569593540 learned_addr learned my addr 172.21.15.133:0/2569593540 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:14:04.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.573+0000 7eff177fe700 1 -- 172.21.15.133:0/2569593540 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1810add0 con 0x7eff18079d00 2023-11-27T15:14:04.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.573+0000 7eff177fe700 1 --2- 172.21.15.133:0/2569593540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff18079d00 0x7eff1810a890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7eff080098a0 tx=0x7eff08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ada02a3925f2d6d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:04.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.573+0000 7eff167fc700 1 -- 172.21.15.133:0/2569593540 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff08004160 con 0x7eff18079d00 2023-11-27T15:14:04.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.574+0000 7eff167fc700 1 -- 172.21.15.133:0/2569593540 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff080042c0 con 0x7eff18079d00 2023-11-27T15:14:04.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.574+0000 7eff167fc700 1 -- 172.21.15.133:0/2569593540 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff080119b0 con 0x7eff18079d00 2023-11-27T15:14:04.411 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.574+0000 7eff1e0c9700 1 -- 172.21.15.133:0/2569593540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff18079d00 msgr2=0x7eff1810a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:04.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.574+0000 7eff1e0c9700 1 --2- 172.21.15.133:0/2569593540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff18079d00 0x7eff1810a890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7eff080098a0 tx=0x7eff08009bb0 comp rx=0 tx=0).stop 2023-11-27T15:14:04.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.574+0000 7eff1e0c9700 1 -- 172.21.15.133:0/2569593540 shutdown_connections 2023-11-27T15:14:04.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.574+0000 7eff1e0c9700 1 --2- 172.21.15.133:0/2569593540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff18079d00 0x7eff1810a890 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:04.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.574+0000 7eff1e0c9700 1 -- 172.21.15.133:0/2569593540 >> 172.21.15.133:0/2569593540 conn(0x7eff180ffe40 msgr2=0x7eff18102270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:04.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.574+0000 7eff1e0c9700 1 -- 172.21.15.133:0/2569593540 shutdown_connections 2023-11-27T15:14:04.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.575+0000 7eff1e0c9700 1 -- 172.21.15.133:0/2569593540 wait complete. 2023-11-27T15:14:04.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.576+0000 7eff1e0c9700 1 Processor -- start 2023-11-27T15:14:04.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.576+0000 7eff1e0c9700 1 -- start start 2023-11-27T15:14:04.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.576+0000 7eff1e0c9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff1819b710 0x7eff1819bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:04.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.576+0000 7eff1e0c9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1819c060 con 0x7eff1819b710 2023-11-27T15:14:04.412 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.577+0000 7eff177fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff1819b710 0x7eff1819bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:04.413 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.577+0000 7eff177fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff1819b710 0x7eff1819bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34170/0 (socket says 172.21.15.133:34170) 2023-11-27T15:14:04.413 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.577+0000 7eff177fe700 1 -- 172.21.15.133:0/2557149141 learned_addr learned my addr 172.21.15.133:0/2557149141 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:14:04.413 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.577+0000 7eff177fe700 1 -- 172.21.15.133:0/2557149141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff08009550 con 0x7eff1819b710 2023-11-27T15:14:04.413 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.577+0000 7eff177fe700 1 --2- 172.21.15.133:0/2557149141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff1819b710 0x7eff1819bb20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7eff08000c00 tx=0x7eff08011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:04.413 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.577+0000 7eff14ff9700 1 -- 172.21.15.133:0/2557149141 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff080039f0 con 0x7eff1819b710 2023-11-27T15:14:04.413 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.578+0000 7eff14ff9700 1 -- 172.21.15.133:0/2557149141 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff08003b50 con 0x7eff1819b710 2023-11-27T15:14:04.413 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.578+0000 7eff1e0c9700 1 -- 172.21.15.133:0/2557149141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff1810bda0 con 0x7eff1819b710 2023-11-27T15:14:04.413 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.578+0000 7eff14ff9700 1 -- 172.21.15.133:0/2557149141 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff0801a780 con 0x7eff1819b710 2023-11-27T15:14:04.413 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.578+0000 7eff1e0c9700 1 -- 172.21.15.133:0/2557149141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff1810bc10 con 0x7eff1819b710 2023-11-27T15:14:04.413 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.579+0000 7eff14ff9700 1 -- 172.21.15.133:0/2557149141 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7eff08024440 con 0x7eff1819b710 2023-11-27T15:14:04.413 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.579+0000 7efefe7fc700 1 -- 172.21.15.133:0/2557149141 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef80052f0 con 0x7eff1819b710 2023-11-27T15:14:04.414 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.580+0000 7eff14ff9700 1 --2- 172.21.15.133:0/2557149141 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7eff00038740 0x7eff0003abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:04.414 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.580+0000 7eff14ff9700 1 -- 172.21.15.133:0/2557149141 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7eff0804c700 con 0x7eff1819b710 2023-11-27T15:14:04.414 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.580+0000 7eff16ffd700 1 --2- 172.21.15.133:0/2557149141 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7eff00038740 0x7eff0003abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:04.414 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.581+0000 7eff16ffd700 1 --2- 172.21.15.133:0/2557149141 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7eff00038740 0x7eff0003abf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7eff0c006fd0 tx=0x7eff0c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:04.414 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.585+0000 7eff14ff9700 1 -- 172.21.15.133:0/2557149141 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7eff080189a0 con 0x7eff1819b710 2023-11-27T15:14:04.414 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.687+0000 7eff14ff9700 1 -- 172.21.15.133:0/2557149141 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7eff0802d910 con 0x7eff1819b710 2023-11-27T15:14:04.414 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.803+0000 7efefe7fc700 1 -- 172.21.15.133:0/2557149141 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7efef8000bc0 con 0x7eff00038740 2023-11-27T15:14:04.414 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.926+0000 7eff14ff9700 1 -- 172.21.15.133:0/2557149141 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7efef8000bc0 con 0x7eff00038740 2023-11-27T15:14:04.414 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.928+0000 7efefe7fc700 1 -- 172.21.15.133:0/2557149141 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7eff00038740 msgr2=0x7eff0003abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:04.414 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.928+0000 7efefe7fc700 1 --2- 172.21.15.133:0/2557149141 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7eff00038740 0x7eff0003abf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7eff0c006fd0 tx=0x7eff0c006e40 comp rx=0 tx=0).stop 2023-11-27T15:14:04.415 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.928+0000 7efefe7fc700 1 -- 172.21.15.133:0/2557149141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff1819b710 msgr2=0x7eff1819bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:04.415 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.928+0000 7efefe7fc700 1 --2- 172.21.15.133:0/2557149141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff1819b710 0x7eff1819bb20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7eff08000c00 tx=0x7eff08011a30 comp rx=0 tx=0).stop 2023-11-27T15:14:04.415 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.928+0000 7efefe7fc700 1 -- 172.21.15.133:0/2557149141 shutdown_connections 2023-11-27T15:14:04.415 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.928+0000 7efefe7fc700 1 --2- 172.21.15.133:0/2557149141 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7eff00038740 0x7eff0003abf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:04.415 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.928+0000 7efefe7fc700 1 --2- 172.21.15.133:0/2557149141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff1819b710 0x7eff1819bb20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:04.415 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.928+0000 7efefe7fc700 1 -- 172.21.15.133:0/2557149141 >> 172.21.15.133:0/2557149141 conn(0x7eff180ffe40 msgr2=0x7eff18100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:04.415 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.928+0000 7efefe7fc700 1 -- 172.21.15.133:0/2557149141 shutdown_connections 2023-11-27T15:14:04.415 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:03.928+0000 7efefe7fc700 1 -- 172.21.15.133:0/2557149141 wait complete. 2023-11-27T15:14:04.415 INFO:teuthology.orchestra.run.smithi133.stdout:Creating initial admin user... 2023-11-27T15:14:05.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:04 smithi133 ceph-mon[97860]: mgrmap e12: x(active, since 2s) 2023-11-27T15:14:05.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:04 smithi133 ceph-mon[97860]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:14:05.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:04 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:05.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:04 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:05.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:04 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:05.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$QMKcycosoBheL.6KTS/Wge1TT3/gz5zbKAL4qdCYiy/3oO92rAbIi", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1701098045, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-11-27T15:14:05.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.953+0000 7f8a1f812700 1 Processor -- start 2023-11-27T15:14:05.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.953+0000 7f8a1f812700 1 -- start start 2023-11-27T15:14:05.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.954+0000 7f8a1f812700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1807bde0 0x7f8a1807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:05.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.954+0000 7f8a1f812700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1807a820 con 0x7f8a1807bde0 2023-11-27T15:14:05.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.954+0000 7f8a1d5ae700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1807bde0 0x7f8a1807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:05.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.954+0000 7f8a1d5ae700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1807bde0 0x7f8a1807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34174/0 (socket says 172.21.15.133:34174) 2023-11-27T15:14:05.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.954+0000 7f8a1d5ae700 1 -- 172.21.15.133:0/2138264621 learned_addr learned my addr 172.21.15.133:0/2138264621 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:14:05.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.954+0000 7f8a1d5ae700 1 -- 172.21.15.133:0/2138264621 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1807a960 con 0x7f8a1807bde0 2023-11-27T15:14:05.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.955+0000 7f8a1d5ae700 1 --2- 172.21.15.133:0/2138264621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1807bde0 0x7f8a1807a2e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8a140098a0 tx=0x7f8a14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39770b6e874397af server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:05.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.955+0000 7f8a0ffff700 1 -- 172.21.15.133:0/2138264621 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a14004160 con 0x7f8a1807bde0 2023-11-27T15:14:05.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.955+0000 7f8a0ffff700 1 -- 172.21.15.133:0/2138264621 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a140042c0 con 0x7f8a1807bde0 2023-11-27T15:14:05.893 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.955+0000 7f8a0ffff700 1 -- 172.21.15.133:0/2138264621 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a140119b0 con 0x7f8a1807bde0 2023-11-27T15:14:05.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.956+0000 7f8a1f812700 1 -- 172.21.15.133:0/2138264621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1807bde0 msgr2=0x7f8a1807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:05.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.956+0000 7f8a1f812700 1 --2- 172.21.15.133:0/2138264621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1807bde0 0x7f8a1807a2e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8a140098a0 tx=0x7f8a14009bb0 comp rx=0 tx=0).stop 2023-11-27T15:14:05.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.956+0000 7f8a1f812700 1 -- 172.21.15.133:0/2138264621 shutdown_connections 2023-11-27T15:14:05.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.956+0000 7f8a1f812700 1 --2- 172.21.15.133:0/2138264621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1807bde0 0x7f8a1807a2e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:05.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.956+0000 7f8a1f812700 1 -- 172.21.15.133:0/2138264621 >> 172.21.15.133:0/2138264621 conn(0x7f8a18100fa0 msgr2=0x7f8a181033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:05.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.956+0000 7f8a1f812700 1 -- 172.21.15.133:0/2138264621 shutdown_connections 2023-11-27T15:14:05.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.956+0000 7f8a1f812700 1 -- 172.21.15.133:0/2138264621 wait complete. 2023-11-27T15:14:05.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.957+0000 7f8a1f812700 1 Processor -- start 2023-11-27T15:14:05.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.957+0000 7f8a1f812700 1 -- start start 2023-11-27T15:14:05.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.957+0000 7f8a1f812700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1807bde0 0x7f8a1819b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:05.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.957+0000 7f8a1f812700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1819bce0 con 0x7f8a1807bde0 2023-11-27T15:14:05.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.958+0000 7f8a1d5ae700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1807bde0 0x7f8a1819b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:05.894 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.958+0000 7f8a1d5ae700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1807bde0 0x7f8a1819b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34186/0 (socket says 172.21.15.133:34186) 2023-11-27T15:14:05.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.958+0000 7f8a1d5ae700 1 -- 172.21.15.133:0/666731625 learned_addr learned my addr 172.21.15.133:0/666731625 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:14:05.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.958+0000 7f8a1d5ae700 1 -- 172.21.15.133:0/666731625 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a14009550 con 0x7f8a1807bde0 2023-11-27T15:14:05.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.958+0000 7f8a1d5ae700 1 --2- 172.21.15.133:0/666731625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1807bde0 0x7f8a1819b7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a14000c00 tx=0x7f8a14011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:05.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.958+0000 7f8a0e7fc700 1 -- 172.21.15.133:0/666731625 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a140039f0 con 0x7f8a1807bde0 2023-11-27T15:14:05.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.958+0000 7f8a0e7fc700 1 -- 172.21.15.133:0/666731625 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a14003b50 con 0x7f8a1807bde0 2023-11-27T15:14:05.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.958+0000 7f8a0e7fc700 1 -- 172.21.15.133:0/666731625 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a1401a780 con 0x7f8a1807bde0 2023-11-27T15:14:05.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.958+0000 7f8a1f812700 1 -- 172.21.15.133:0/666731625 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a1810be50 con 0x7f8a1807bde0 2023-11-27T15:14:05.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.959+0000 7f8a1f812700 1 -- 172.21.15.133:0/666731625 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1810bcc0 con 0x7f8a1807bde0 2023-11-27T15:14:05.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.960+0000 7f8a1f812700 1 -- 172.21.15.133:0/666731625 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a1810bfe0 con 0x7f8a1807bde0 2023-11-27T15:14:05.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.960+0000 7f8a0e7fc700 1 -- 172.21.15.133:0/666731625 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f8a14024440 con 0x7f8a1807bde0 2023-11-27T15:14:05.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.960+0000 7f8a0e7fc700 1 --2- 172.21.15.133:0/666731625 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8a04038870 0x7f8a0403ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:05.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.960+0000 7f8a0e7fc700 1 -- 172.21.15.133:0/666731625 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8a1404c940 con 0x7f8a1807bde0 2023-11-27T15:14:05.895 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.961+0000 7f8a1cdad700 1 --2- 172.21.15.133:0/666731625 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8a04038870 0x7f8a0403ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:05.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.961+0000 7f8a1cdad700 1 --2- 172.21.15.133:0/666731625 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8a04038870 0x7f8a0403ad20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8a08006fd0 tx=0x7f8a08006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:05.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:04.967+0000 7f8a0e7fc700 1 -- 172.21.15.133:0/666731625 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8a14018db0 con 0x7f8a1807bde0 2023-11-27T15:14:05.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:05.193+0000 7f8a1f812700 1 -- 172.21.15.133:0/666731625 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f8a1819f160 con 0x7f8a04038870 2023-11-27T15:14:05.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:05.422+0000 7f8a0e7fc700 1 -- 172.21.15.133:0/666731625 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f8a1819f160 con 0x7f8a04038870 2023-11-27T15:14:05.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:05.424+0000 7f8a1f812700 1 -- 172.21.15.133:0/666731625 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8a04038870 msgr2=0x7f8a0403ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:05.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:05.424+0000 7f8a1f812700 1 --2- 172.21.15.133:0/666731625 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8a04038870 0x7f8a0403ad20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8a08006fd0 tx=0x7f8a08006e40 comp rx=0 tx=0).stop 2023-11-27T15:14:05.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:05.425+0000 7f8a1f812700 1 -- 172.21.15.133:0/666731625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1807bde0 msgr2=0x7f8a1819b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:05.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:05.425+0000 7f8a1f812700 1 --2- 172.21.15.133:0/666731625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1807bde0 0x7f8a1819b7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a14000c00 tx=0x7f8a14011a30 comp rx=0 tx=0).stop 2023-11-27T15:14:05.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:05.425+0000 7f8a1f812700 1 -- 172.21.15.133:0/666731625 shutdown_connections 2023-11-27T15:14:05.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:05.425+0000 7f8a1f812700 1 --2- 172.21.15.133:0/666731625 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8a04038870 0x7f8a0403ad20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:05.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:05.425+0000 7f8a1f812700 1 --2- 172.21.15.133:0/666731625 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a1807bde0 0x7f8a1819b7a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:05.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:05.425+0000 7f8a1f812700 1 -- 172.21.15.133:0/666731625 >> 172.21.15.133:0/666731625 conn(0x7f8a18100fa0 msgr2=0x7f8a18101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:05.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:05.425+0000 7f8a1f812700 1 -- 172.21.15.133:0/666731625 shutdown_connections 2023-11-27T15:14:05.896 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:05.425+0000 7f8a1f812700 1 -- 172.21.15.133:0/666731625 wait complete. 2023-11-27T15:14:05.897 INFO:teuthology.orchestra.run.smithi133.stdout:Fetching dashboard port number... 2023-11-27T15:14:06.713 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:06 smithi133 ceph-mon[97860]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:14:06.713 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:06 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:06.713 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:06 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:07.178 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stdout 8443 2023-11-27T15:14:07.178 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.435+0000 7f4a5954e700 1 Processor -- start 2023-11-27T15:14:07.178 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.435+0000 7f4a5954e700 1 -- start start 2023-11-27T15:14:07.178 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.435+0000 7f4a5954e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a54105910 0x7f4a54105d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:07.178 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.435+0000 7f4a5954e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a54106260 con 0x7f4a54105910 2023-11-27T15:14:07.178 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.436+0000 7f4a52ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a54105910 0x7f4a54105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:07.178 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.436+0000 7f4a52ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a54105910 0x7f4a54105d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34198/0 (socket says 172.21.15.133:34198) 2023-11-27T15:14:07.179 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.436+0000 7f4a52ffd700 1 -- 172.21.15.133:0/3431281760 learned_addr learned my addr 172.21.15.133:0/3431281760 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:14:07.179 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.436+0000 7f4a52ffd700 1 -- 172.21.15.133:0/3431281760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a541063a0 con 0x7f4a54105910 2023-11-27T15:14:07.179 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.436+0000 7f4a52ffd700 1 --2- 172.21.15.133:0/3431281760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a54105910 0x7f4a54105d20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4a440098a0 tx=0x7f4a44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba14e2567a128b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:07.179 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.437+0000 7f4a51ffb700 1 -- 172.21.15.133:0/3431281760 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a44004160 con 0x7f4a54105910 2023-11-27T15:14:07.179 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.437+0000 7f4a51ffb700 1 -- 172.21.15.133:0/3431281760 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a440042c0 con 0x7f4a54105910 2023-11-27T15:14:07.179 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.437+0000 7f4a51ffb700 1 -- 172.21.15.133:0/3431281760 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a440119b0 con 0x7f4a54105910 2023-11-27T15:14:07.179 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.438+0000 7f4a5954e700 1 -- 172.21.15.133:0/3431281760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a54105910 msgr2=0x7f4a54105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:07.179 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.438+0000 7f4a5954e700 1 --2- 172.21.15.133:0/3431281760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a54105910 0x7f4a54105d20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4a440098a0 tx=0x7f4a44009bb0 comp rx=0 tx=0).stop 2023-11-27T15:14:07.179 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.438+0000 7f4a5954e700 1 -- 172.21.15.133:0/3431281760 shutdown_connections 2023-11-27T15:14:07.179 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.438+0000 7f4a5954e700 1 --2- 172.21.15.133:0/3431281760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a54105910 0x7f4a54105d20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:07.179 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.438+0000 7f4a5954e700 1 -- 172.21.15.133:0/3431281760 >> 172.21.15.133:0/3431281760 conn(0x7f4a54100fa0 msgr2=0x7f4a541033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:07.179 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.438+0000 7f4a5954e700 1 -- 172.21.15.133:0/3431281760 shutdown_connections 2023-11-27T15:14:07.179 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.438+0000 7f4a5954e700 1 -- 172.21.15.133:0/3431281760 wait complete. 2023-11-27T15:14:07.180 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.439+0000 7f4a5954e700 1 Processor -- start 2023-11-27T15:14:07.180 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.440+0000 7f4a5954e700 1 -- start start 2023-11-27T15:14:07.180 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.440+0000 7f4a5954e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a54105910 0x7f4a5419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:07.180 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.440+0000 7f4a5954e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5419bc90 con 0x7f4a54105910 2023-11-27T15:14:07.180 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.440+0000 7f4a52ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a54105910 0x7f4a5419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:07.180 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.440+0000 7f4a52ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a54105910 0x7f4a5419b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34214/0 (socket says 172.21.15.133:34214) 2023-11-27T15:14:07.180 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.440+0000 7f4a52ffd700 1 -- 172.21.15.133:0/3831796710 learned_addr learned my addr 172.21.15.133:0/3831796710 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:14:07.180 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.441+0000 7f4a52ffd700 1 -- 172.21.15.133:0/3831796710 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a44009550 con 0x7f4a54105910 2023-11-27T15:14:07.180 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.441+0000 7f4a52ffd700 1 --2- 172.21.15.133:0/3831796710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a54105910 0x7f4a5419b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4a44011980 tx=0x7f4a44011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:07.180 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.441+0000 7f4a3bfff700 1 -- 172.21.15.133:0/3831796710 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a44003a60 con 0x7f4a54105910 2023-11-27T15:14:07.180 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.441+0000 7f4a5954e700 1 -- 172.21.15.133:0/3831796710 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a541079a0 con 0x7f4a54105910 2023-11-27T15:14:07.180 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.441+0000 7f4a3bfff700 1 -- 172.21.15.133:0/3831796710 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a44003bc0 con 0x7f4a54105910 2023-11-27T15:14:07.181 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.441+0000 7f4a3bfff700 1 -- 172.21.15.133:0/3831796710 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4a4401a670 con 0x7f4a54105910 2023-11-27T15:14:07.181 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.441+0000 7f4a5954e700 1 -- 172.21.15.133:0/3831796710 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a54107810 con 0x7f4a54105910 2023-11-27T15:14:07.181 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.442+0000 7f4a3bfff700 1 -- 172.21.15.133:0/3831796710 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f4a4401a930 con 0x7f4a54105910 2023-11-27T15:14:07.181 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.443+0000 7f4a5954e700 1 -- 172.21.15.133:0/3831796710 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a54109de0 con 0x7f4a54105910 2023-11-27T15:14:07.181 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.443+0000 7f4a3bfff700 1 --2- 172.21.15.133:0/3831796710 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4a3c038820 0x7f4a3c03acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:07.181 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.443+0000 7f4a3bfff700 1 -- 172.21.15.133:0/3831796710 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4a4404cc30 con 0x7f4a54105910 2023-11-27T15:14:07.181 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.443+0000 7f4a527fc700 1 --2- 172.21.15.133:0/3831796710 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4a3c038820 0x7f4a3c03acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:07.181 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.444+0000 7f4a527fc700 1 --2- 172.21.15.133:0/3831796710 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4a3c038820 0x7f4a3c03acd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4a48006fd0 tx=0x7f4a48006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:07.181 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.448+0000 7f4a3bfff700 1 -- 172.21.15.133:0/3831796710 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4a44018b80 con 0x7f4a54105910 2023-11-27T15:14:07.181 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.683+0000 7f4a5954e700 1 -- 172.21.15.133:0/3831796710 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f4a5419f360 con 0x7f4a54105910 2023-11-27T15:14:07.181 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.683+0000 7f4a3bfff700 1 -- 172.21.15.133:0/3831796710 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f4a44018350 con 0x7f4a54105910 2023-11-27T15:14:07.181 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.685+0000 7f4a5954e700 1 -- 172.21.15.133:0/3831796710 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4a3c038820 msgr2=0x7f4a3c03acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:07.181 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.685+0000 7f4a5954e700 1 --2- 172.21.15.133:0/3831796710 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4a3c038820 0x7f4a3c03acd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4a48006fd0 tx=0x7f4a48006e40 comp rx=0 tx=0).stop 2023-11-27T15:14:07.181 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.685+0000 7f4a5954e700 1 -- 172.21.15.133:0/3831796710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a54105910 msgr2=0x7f4a5419b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:07.182 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.685+0000 7f4a5954e700 1 --2- 172.21.15.133:0/3831796710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a54105910 0x7f4a5419b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4a44011980 tx=0x7f4a44011a60 comp rx=0 tx=0).stop 2023-11-27T15:14:07.182 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.686+0000 7f4a5954e700 1 -- 172.21.15.133:0/3831796710 shutdown_connections 2023-11-27T15:14:07.182 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.686+0000 7f4a5954e700 1 --2- 172.21.15.133:0/3831796710 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4a3c038820 0x7f4a3c03acd0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:07.182 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.686+0000 7f4a5954e700 1 --2- 172.21.15.133:0/3831796710 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a54105910 0x7f4a5419b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:07.182 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.686+0000 7f4a5954e700 1 -- 172.21.15.133:0/3831796710 >> 172.21.15.133:0/3831796710 conn(0x7f4a54100fa0 msgr2=0x7f4a54101c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:07.182 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.686+0000 7f4a5954e700 1 -- 172.21.15.133:0/3831796710 shutdown_connections 2023-11-27T15:14:07.182 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:06.686+0000 7f4a5954e700 1 -- 172.21.15.133:0/3831796710 wait complete. 2023-11-27T15:14:07.200 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-11-27T15:14:07.200 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout disabled 2023-11-27T15:14:07.223 INFO:teuthology.orchestra.run.smithi133.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-11-27T15:14:07.223 INFO:teuthology.orchestra.run.smithi133.stdout:systemctl: stdout inactive 2023-11-27T15:14:07.223 INFO:teuthology.orchestra.run.smithi133.stdout:firewalld.service is not enabled 2023-11-27T15:14:07.223 INFO:teuthology.orchestra.run.smithi133.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-11-27T15:14:07.225 INFO:teuthology.orchestra.run.smithi133.stdout:Ceph Dashboard is now available at: 2023-11-27T15:14:07.225 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:14:07.225 INFO:teuthology.orchestra.run.smithi133.stdout: URL: https://smithi133.front.sepia.ceph.com:8443/ 2023-11-27T15:14:07.225 INFO:teuthology.orchestra.run.smithi133.stdout: User: admin 2023-11-27T15:14:07.225 INFO:teuthology.orchestra.run.smithi133.stdout: Password: wwzvcfyxm3 2023-11-27T15:14:07.225 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:14:07.226 INFO:teuthology.orchestra.run.smithi133.stdout:Saving cluster configuration to /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config directory 2023-11-27T15:14:07.226 INFO:teuthology.orchestra.run.smithi133.stdout:Enabling autotune for osd_memory_target 2023-11-27T15:14:07.466 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:07 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3831796710' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-11-27T15:14:08.945 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.338+0000 7f35f0294700 1 Processor -- start 2023-11-27T15:14:08.945 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.339+0000 7f35f0294700 1 -- start start 2023-11-27T15:14:08.945 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.339+0000 7f35f0294700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e807bde0 0x7f35e807a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:08.945 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.339+0000 7f35f0294700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e807a820 con 0x7f35e807bde0 2023-11-27T15:14:08.945 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.339+0000 7f35ee030700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e807bde0 0x7f35e807a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:08.945 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.340+0000 7f35ee030700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e807bde0 0x7f35e807a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34220/0 (socket says 172.21.15.133:34220) 2023-11-27T15:14:08.945 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.340+0000 7f35ee030700 1 -- 172.21.15.133:0/1291929532 learned_addr learned my addr 172.21.15.133:0/1291929532 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:14:08.946 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.340+0000 7f35ee030700 1 -- 172.21.15.133:0/1291929532 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e807a960 con 0x7f35e807bde0 2023-11-27T15:14:08.946 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.340+0000 7f35ee030700 1 --2- 172.21.15.133:0/1291929532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e807bde0 0x7f35e807a2e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f35e40098a0 tx=0x7f35e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2cdea970e93d93f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:08.946 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.340+0000 7f35ed02e700 1 -- 172.21.15.133:0/1291929532 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35e4004160 con 0x7f35e807bde0 2023-11-27T15:14:08.946 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.340+0000 7f35ed02e700 1 -- 172.21.15.133:0/1291929532 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35e40042c0 con 0x7f35e807bde0 2023-11-27T15:14:08.946 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.340+0000 7f35ed02e700 1 -- 172.21.15.133:0/1291929532 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35e40119b0 con 0x7f35e807bde0 2023-11-27T15:14:08.946 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.341+0000 7f35f0294700 1 -- 172.21.15.133:0/1291929532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e807bde0 msgr2=0x7f35e807a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:08.946 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.341+0000 7f35f0294700 1 --2- 172.21.15.133:0/1291929532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e807bde0 0x7f35e807a2e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f35e40098a0 tx=0x7f35e4009bb0 comp rx=0 tx=0).stop 2023-11-27T15:14:08.946 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.341+0000 7f35f0294700 1 -- 172.21.15.133:0/1291929532 shutdown_connections 2023-11-27T15:14:08.946 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.341+0000 7f35f0294700 1 --2- 172.21.15.133:0/1291929532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e807bde0 0x7f35e807a2e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:08.946 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.341+0000 7f35f0294700 1 -- 172.21.15.133:0/1291929532 >> 172.21.15.133:0/1291929532 conn(0x7f35e8100fa0 msgr2=0x7f35e81033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:08.946 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.341+0000 7f35f0294700 1 -- 172.21.15.133:0/1291929532 shutdown_connections 2023-11-27T15:14:08.947 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.341+0000 7f35f0294700 1 -- 172.21.15.133:0/1291929532 wait complete. 2023-11-27T15:14:08.947 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.342+0000 7f35f0294700 1 Processor -- start 2023-11-27T15:14:08.947 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.342+0000 7f35f0294700 1 -- start start 2023-11-27T15:14:08.947 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.343+0000 7f35f0294700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e807bde0 0x7f35e819b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:08.947 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.343+0000 7f35f0294700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e819bcf0 con 0x7f35e807bde0 2023-11-27T15:14:08.947 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.343+0000 7f35ee030700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e807bde0 0x7f35e819b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:08.947 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.343+0000 7f35ee030700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e807bde0 0x7f35e819b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34236/0 (socket says 172.21.15.133:34236) 2023-11-27T15:14:08.947 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.343+0000 7f35ee030700 1 -- 172.21.15.133:0/3654295584 learned_addr learned my addr 172.21.15.133:0/3654295584 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:14:08.947 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.343+0000 7f35ee030700 1 -- 172.21.15.133:0/3654295584 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e4009550 con 0x7f35e807bde0 2023-11-27T15:14:08.947 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.343+0000 7f35ee030700 1 --2- 172.21.15.133:0/3654295584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e807bde0 0x7f35e819b7b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f35e4000c00 tx=0x7f35e4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:08.947 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.344+0000 7f35df7fe700 1 -- 172.21.15.133:0/3654295584 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35e40039f0 con 0x7f35e807bde0 2023-11-27T15:14:08.947 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.344+0000 7f35df7fe700 1 -- 172.21.15.133:0/3654295584 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35e4003b50 con 0x7f35e807bde0 2023-11-27T15:14:08.948 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.344+0000 7f35f0294700 1 -- 172.21.15.133:0/3654295584 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e810be50 con 0x7f35e807bde0 2023-11-27T15:14:08.948 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.344+0000 7f35df7fe700 1 -- 172.21.15.133:0/3654295584 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35e401a780 con 0x7f35e807bde0 2023-11-27T15:14:08.948 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.344+0000 7f35f0294700 1 -- 172.21.15.133:0/3654295584 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e810bcc0 con 0x7f35e807bde0 2023-11-27T15:14:08.948 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.345+0000 7f35df7fe700 1 -- 172.21.15.133:0/3654295584 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f35e4024440 con 0x7f35e807bde0 2023-11-27T15:14:08.948 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.345+0000 7f35f0294700 1 -- 172.21.15.133:0/3654295584 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e810bfe0 con 0x7f35e807bde0 2023-11-27T15:14:08.948 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.345+0000 7f35df7fe700 1 --2- 172.21.15.133:0/3654295584 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f35d4038820 0x7f35d403acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:08.948 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.345+0000 7f35df7fe700 1 -- 172.21.15.133:0/3654295584 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f35e404c9b0 con 0x7f35e807bde0 2023-11-27T15:14:08.948 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.345+0000 7f35ed82f700 1 --2- 172.21.15.133:0/3654295584 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f35d4038820 0x7f35d403acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:08.948 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.346+0000 7f35ed82f700 1 --2- 172.21.15.133:0/3654295584 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f35d4038820 0x7f35d403acd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f35d8006fd0 tx=0x7f35d8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:08.948 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.350+0000 7f35df7fe700 1 -- 172.21.15.133:0/3654295584 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f35e4018db0 con 0x7f35e807bde0 2023-11-27T15:14:08.948 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.430+0000 7f35df7fe700 1 -- 172.21.15.133:0/3654295584 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f35e404ed50 con 0x7f35e807bde0 2023-11-27T15:14:08.948 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.550+0000 7f35f0294700 1 -- 172.21.15.133:0/3654295584 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f35e819f430 con 0x7f35e807bde0 2023-11-27T15:14:08.949 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.551+0000 7f35df7fe700 1 -- 172.21.15.133:0/3654295584 <== mon.0 v2:172.21.15.133:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f35e40249d0 con 0x7f35e807bde0 2023-11-27T15:14:08.949 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.553+0000 7f35f0294700 1 -- 172.21.15.133:0/3654295584 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f35d4038820 msgr2=0x7f35d403acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:08.949 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.553+0000 7f35f0294700 1 --2- 172.21.15.133:0/3654295584 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f35d4038820 0x7f35d403acd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f35d8006fd0 tx=0x7f35d8006e40 comp rx=0 tx=0).stop 2023-11-27T15:14:08.949 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.553+0000 7f35f0294700 1 -- 172.21.15.133:0/3654295584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e807bde0 msgr2=0x7f35e819b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:08.949 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.553+0000 7f35f0294700 1 --2- 172.21.15.133:0/3654295584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e807bde0 0x7f35e819b7b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f35e4000c00 tx=0x7f35e4011a30 comp rx=0 tx=0).stop 2023-11-27T15:14:08.949 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.553+0000 7f35f0294700 1 -- 172.21.15.133:0/3654295584 shutdown_connections 2023-11-27T15:14:08.949 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.553+0000 7f35f0294700 1 --2- 172.21.15.133:0/3654295584 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f35d4038820 0x7f35d403acd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:08.949 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.553+0000 7f35f0294700 1 --2- 172.21.15.133:0/3654295584 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e807bde0 0x7f35e819b7b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:08.949 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.553+0000 7f35f0294700 1 -- 172.21.15.133:0/3654295584 >> 172.21.15.133:0/3654295584 conn(0x7f35e8100fa0 msgr2=0x7f35e8101c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:08.949 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.553+0000 7f35f0294700 1 -- 172.21.15.133:0/3654295584 shutdown_connections 2023-11-27T15:14:08.949 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:08.553+0000 7f35f0294700 1 -- 172.21.15.133:0/3654295584 wait complete. 2023-11-27T15:14:09.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:09 smithi133 ceph-mon[97860]: mgrmap e13: x(active, since 6s) 2023-11-27T15:14:10.229 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.455+0000 7f607bcc5700 1 Processor -- start 2023-11-27T15:14:10.229 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.456+0000 7f607bcc5700 1 -- start start 2023-11-27T15:14:10.229 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.456+0000 7f607bcc5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f607407be00 0x7f607407a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:10.229 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.456+0000 7f607bcc5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607407a840 con 0x7f607407be00 2023-11-27T15:14:10.229 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.456+0000 7f6079a61700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f607407be00 0x7f607407a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:10.229 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.456+0000 7f6079a61700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f607407be00 0x7f607407a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34240/0 (socket says 172.21.15.133:34240) 2023-11-27T15:14:10.229 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.456+0000 7f6079a61700 1 -- 172.21.15.133:0/2260447965 learned_addr learned my addr 172.21.15.133:0/2260447965 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:14:10.229 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.457+0000 7f6079a61700 1 -- 172.21.15.133:0/2260447965 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607407a980 con 0x7f607407be00 2023-11-27T15:14:10.229 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.457+0000 7f6079a61700 1 --2- 172.21.15.133:0/2260447965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f607407be00 0x7f607407a300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f60700098a0 tx=0x7f6070009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f951af0cde474e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:10.229 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.457+0000 7f6078a5f700 1 -- 172.21.15.133:0/2260447965 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6070004160 con 0x7f607407be00 2023-11-27T15:14:10.229 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.457+0000 7f6078a5f700 1 -- 172.21.15.133:0/2260447965 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60700042c0 con 0x7f607407be00 2023-11-27T15:14:10.229 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.457+0000 7f6078a5f700 1 -- 172.21.15.133:0/2260447965 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60700119b0 con 0x7f607407be00 2023-11-27T15:14:10.230 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.458+0000 7f607bcc5700 1 -- 172.21.15.133:0/2260447965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f607407be00 msgr2=0x7f607407a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:10.230 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.458+0000 7f607bcc5700 1 --2- 172.21.15.133:0/2260447965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f607407be00 0x7f607407a300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f60700098a0 tx=0x7f6070009bb0 comp rx=0 tx=0).stop 2023-11-27T15:14:10.230 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.458+0000 7f607bcc5700 1 -- 172.21.15.133:0/2260447965 shutdown_connections 2023-11-27T15:14:10.230 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.458+0000 7f607bcc5700 1 --2- 172.21.15.133:0/2260447965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f607407be00 0x7f607407a300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:10.230 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.458+0000 7f607bcc5700 1 -- 172.21.15.133:0/2260447965 >> 172.21.15.133:0/2260447965 conn(0x7f6074100fa0 msgr2=0x7f60741033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:10.230 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.458+0000 7f607bcc5700 1 -- 172.21.15.133:0/2260447965 shutdown_connections 2023-11-27T15:14:10.230 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.458+0000 7f607bcc5700 1 -- 172.21.15.133:0/2260447965 wait complete. 2023-11-27T15:14:10.230 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.459+0000 7f607bcc5700 1 Processor -- start 2023-11-27T15:14:10.230 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.459+0000 7f607bcc5700 1 -- start start 2023-11-27T15:14:10.230 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.459+0000 7f607bcc5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f607407be00 0x7f607419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:10.230 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.459+0000 7f607bcc5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607419bcf0 con 0x7f607407be00 2023-11-27T15:14:10.230 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.459+0000 7f6079a61700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f607407be00 0x7f607419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:10.230 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.460+0000 7f6079a61700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f607407be00 0x7f607419b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34242/0 (socket says 172.21.15.133:34242) 2023-11-27T15:14:10.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.460+0000 7f6079a61700 1 -- 172.21.15.133:0/1357459270 learned_addr learned my addr 172.21.15.133:0/1357459270 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:14:10.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.460+0000 7f6079a61700 1 -- 172.21.15.133:0/1357459270 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6070009550 con 0x7f607407be00 2023-11-27T15:14:10.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.460+0000 7f6079a61700 1 --2- 172.21.15.133:0/1357459270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f607407be00 0x7f607419b7b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6070003e60 tx=0x7f6070011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:10.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.460+0000 7f606affd700 1 -- 172.21.15.133:0/1357459270 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6070003b20 con 0x7f607407be00 2023-11-27T15:14:10.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.460+0000 7f606affd700 1 -- 172.21.15.133:0/1357459270 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6070003c80 con 0x7f607407be00 2023-11-27T15:14:10.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.460+0000 7f607bcc5700 1 -- 172.21.15.133:0/1357459270 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607410be30 con 0x7f607407be00 2023-11-27T15:14:10.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.460+0000 7f606affd700 1 -- 172.21.15.133:0/1357459270 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6070024760 con 0x7f607407be00 2023-11-27T15:14:10.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.460+0000 7f607bcc5700 1 -- 172.21.15.133:0/1357459270 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607410bca0 con 0x7f607407be00 2023-11-27T15:14:10.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.461+0000 7f606affd700 1 -- 172.21.15.133:0/1357459270 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f607001ab10 con 0x7f607407be00 2023-11-27T15:14:10.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.461+0000 7f607bcc5700 1 -- 172.21.15.133:0/1357459270 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607410bfc0 con 0x7f607407be00 2023-11-27T15:14:10.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.462+0000 7f606affd700 1 --2- 172.21.15.133:0/1357459270 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6060038910 0x7f606003adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:10.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.462+0000 7f606affd700 1 -- 172.21.15.133:0/1357459270 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f607004cbe0 con 0x7f607407be00 2023-11-27T15:14:10.231 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.462+0000 7f6079260700 1 --2- 172.21.15.133:0/1357459270 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6060038910 0x7f606003adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:10.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.463+0000 7f6079260700 1 --2- 172.21.15.133:0/1357459270 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6060038910 0x7f606003adc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6064006fd0 tx=0x7f6064006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:10.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.467+0000 7f606affd700 1 -- 172.21.15.133:0/1357459270 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6070018920 con 0x7f607407be00 2023-11-27T15:14:10.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.770+0000 7f607bcc5700 1 -- 172.21.15.133:0/1357459270 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f607419f3c0 con 0x7f607407be00 2023-11-27T15:14:10.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.773+0000 7f606affd700 1 -- 172.21.15.133:0/1357459270 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f6070018350 con 0x7f607407be00 2023-11-27T15:14:10.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.775+0000 7f607bcc5700 1 -- 172.21.15.133:0/1357459270 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6060038910 msgr2=0x7f606003adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:10.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.775+0000 7f607bcc5700 1 --2- 172.21.15.133:0/1357459270 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6060038910 0x7f606003adc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6064006fd0 tx=0x7f6064006e40 comp rx=0 tx=0).stop 2023-11-27T15:14:10.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.775+0000 7f607bcc5700 1 -- 172.21.15.133:0/1357459270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f607407be00 msgr2=0x7f607419b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:10.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.775+0000 7f607bcc5700 1 --2- 172.21.15.133:0/1357459270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f607407be00 0x7f607419b7b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6070003e60 tx=0x7f6070011a30 comp rx=0 tx=0).stop 2023-11-27T15:14:10.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.775+0000 7f607bcc5700 1 -- 172.21.15.133:0/1357459270 shutdown_connections 2023-11-27T15:14:10.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.775+0000 7f607bcc5700 1 --2- 172.21.15.133:0/1357459270 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6060038910 0x7f606003adc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:10.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.775+0000 7f607bcc5700 1 --2- 172.21.15.133:0/1357459270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f607407be00 0x7f607419b7b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:10.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.775+0000 7f607bcc5700 1 -- 172.21.15.133:0/1357459270 >> 172.21.15.133:0/1357459270 conn(0x7f6074100fa0 msgr2=0x7f6074101c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:10.232 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.776+0000 7f607bcc5700 1 -- 172.21.15.133:0/1357459270 shutdown_connections 2023-11-27T15:14:10.233 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr 2023-11-27T15:14:09.776+0000 7f607bcc5700 1 -- 172.21.15.133:0/1357459270 wait complete. 2023-11-27T15:14:10.233 INFO:teuthology.orchestra.run.smithi133.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-11-27T15:14:10.233 INFO:teuthology.orchestra.run.smithi133.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-11-27T15:14:10.233 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:14:10.233 INFO:teuthology.orchestra.run.smithi133.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:14:10.233 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:14:10.233 INFO:teuthology.orchestra.run.smithi133.stdout:Or, if you are only running a single cluster on this host: 2023-11-27T15:14:10.233 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:14:10.233 INFO:teuthology.orchestra.run.smithi133.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-11-27T15:14:10.233 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:14:10.233 INFO:teuthology.orchestra.run.smithi133.stdout:Please consider enabling telemetry to help improve Ceph: 2023-11-27T15:14:10.233 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:14:10.234 INFO:teuthology.orchestra.run.smithi133.stdout: ceph telemetry on 2023-11-27T15:14:10.234 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:14:10.234 INFO:teuthology.orchestra.run.smithi133.stdout:For more information see: 2023-11-27T15:14:10.234 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:14:10.234 INFO:teuthology.orchestra.run.smithi133.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-11-27T15:14:10.234 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:14:10.234 INFO:teuthology.orchestra.run.smithi133.stdout:Bootstrap complete. 2023-11-27T15:14:10.285 INFO:tasks.cephadm:Fetching config... 2023-11-27T15:14:10.285 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-11-27T15:14:10.285 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-11-27T15:14:10.328 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-11-27T15:14:10.328 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-11-27T15:14:10.328 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-11-27T15:14:10.413 INFO:tasks.cephadm:Fetching mon keyring... 2023-11-27T15:14:10.413 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-11-27T15:14:10.414 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/keyring of=/dev/stdout 2023-11-27T15:14:10.523 INFO:tasks.cephadm:Fetching pub ssh key... 2023-11-27T15:14:10.523 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-11-27T15:14:10.524 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-11-27T15:14:10.607 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-11-27T15:14:10.607 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvzDKejWOae18ZOHtCrzB9Hgme2PKW1Fi/b14QgvG3+r81TTqgD/eJ5T2CtyGQa/yktGAUNPWKGfEUd7LVRP9lbCuIdUhZR9cC7vykLXD37bR0Lg6MIFI6lhkHzR61vSPZ/YbG2rqnyUr6JexwgcyJWmBnxZoxez9sTKLQpyEzfePhNXo0YfI24kJObBA8DZR+sKqCmlDkwwNlQLFQQpfEcXCQN5NG+hwQIxrWM3luCikkNWbLhsgFNLbaredn4x0+MH6+AM5Hecp6cbZi+Xtz/VYn33S90jh5hUD2acROkpMds7IsaXHtj6vAXZafi1BokEUwE7CUOBH2ZTL4ML1DO8Tl+/zcbqf94SoMhgBAD7Fw5EM9Prh0vQdekuxJTi9RiiVnU/gmKJXKBVAeS7apurnA3u83jQgy+sN7QxQ1S1stkzTSQaC4iGnFsvpEAr94fc/BBtjcxpCUAZoyp9maoiKSV2baHlnFvqDrSLd9AMInXkm+KmPagv5TTUbK9oM= ceph-5227da4c-8d37-11ee-95a2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-11-27T15:14:10.758 INFO:teuthology.orchestra.run.smithi133.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvzDKejWOae18ZOHtCrzB9Hgme2PKW1Fi/b14QgvG3+r81TTqgD/eJ5T2CtyGQa/yktGAUNPWKGfEUd7LVRP9lbCuIdUhZR9cC7vykLXD37bR0Lg6MIFI6lhkHzR61vSPZ/YbG2rqnyUr6JexwgcyJWmBnxZoxez9sTKLQpyEzfePhNXo0YfI24kJObBA8DZR+sKqCmlDkwwNlQLFQQpfEcXCQN5NG+hwQIxrWM3luCikkNWbLhsgFNLbaredn4x0+MH6+AM5Hecp6cbZi+Xtz/VYn33S90jh5hUD2acROkpMds7IsaXHtj6vAXZafi1BokEUwE7CUOBH2ZTL4ML1DO8Tl+/zcbqf94SoMhgBAD7Fw5EM9Prh0vQdekuxJTi9RiiVnU/gmKJXKBVAeS7apurnA3u83jQgy+sN7QxQ1S1stkzTSQaC4iGnFsvpEAr94fc/BBtjcxpCUAZoyp9maoiKSV2baHlnFvqDrSLd9AMInXkm+KmPagv5TTUbK9oM= ceph-5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:14:10.789 DEBUG:teuthology.orchestra.run.smithi174:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvzDKejWOae18ZOHtCrzB9Hgme2PKW1Fi/b14QgvG3+r81TTqgD/eJ5T2CtyGQa/yktGAUNPWKGfEUd7LVRP9lbCuIdUhZR9cC7vykLXD37bR0Lg6MIFI6lhkHzR61vSPZ/YbG2rqnyUr6JexwgcyJWmBnxZoxez9sTKLQpyEzfePhNXo0YfI24kJObBA8DZR+sKqCmlDkwwNlQLFQQpfEcXCQN5NG+hwQIxrWM3luCikkNWbLhsgFNLbaredn4x0+MH6+AM5Hecp6cbZi+Xtz/VYn33S90jh5hUD2acROkpMds7IsaXHtj6vAXZafi1BokEUwE7CUOBH2ZTL4ML1DO8Tl+/zcbqf94SoMhgBAD7Fw5EM9Prh0vQdekuxJTi9RiiVnU/gmKJXKBVAeS7apurnA3u83jQgy+sN7QxQ1S1stkzTSQaC4iGnFsvpEAr94fc/BBtjcxpCUAZoyp9maoiKSV2baHlnFvqDrSLd9AMInXkm+KmPagv5TTUbK9oM= ceph-5227da4c-8d37-11ee-95a2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-11-27T15:14:10.833 INFO:teuthology.orchestra.run.smithi174.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvzDKejWOae18ZOHtCrzB9Hgme2PKW1Fi/b14QgvG3+r81TTqgD/eJ5T2CtyGQa/yktGAUNPWKGfEUd7LVRP9lbCuIdUhZR9cC7vykLXD37bR0Lg6MIFI6lhkHzR61vSPZ/YbG2rqnyUr6JexwgcyJWmBnxZoxez9sTKLQpyEzfePhNXo0YfI24kJObBA8DZR+sKqCmlDkwwNlQLFQQpfEcXCQN5NG+hwQIxrWM3luCikkNWbLhsgFNLbaredn4x0+MH6+AM5Hecp6cbZi+Xtz/VYn33S90jh5hUD2acROkpMds7IsaXHtj6vAXZafi1BokEUwE7CUOBH2ZTL4ML1DO8Tl+/zcbqf94SoMhgBAD7Fw5EM9Prh0vQdekuxJTi9RiiVnU/gmKJXKBVAeS7apurnA3u83jQgy+sN7QxQ1S1stkzTSQaC4iGnFsvpEAr94fc/BBtjcxpCUAZoyp9maoiKSV2baHlnFvqDrSLd9AMInXkm+KmPagv5TTUbK9oM= ceph-5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:14:10.845 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvzDKejWOae18ZOHtCrzB9Hgme2PKW1Fi/b14QgvG3+r81TTqgD/eJ5T2CtyGQa/yktGAUNPWKGfEUd7LVRP9lbCuIdUhZR9cC7vykLXD37bR0Lg6MIFI6lhkHzR61vSPZ/YbG2rqnyUr6JexwgcyJWmBnxZoxez9sTKLQpyEzfePhNXo0YfI24kJObBA8DZR+sKqCmlDkwwNlQLFQQpfEcXCQN5NG+hwQIxrWM3luCikkNWbLhsgFNLbaredn4x0+MH6+AM5Hecp6cbZi+Xtz/VYn33S90jh5hUD2acROkpMds7IsaXHtj6vAXZafi1BokEUwE7CUOBH2ZTL4ML1DO8Tl+/zcbqf94SoMhgBAD7Fw5EM9Prh0vQdekuxJTi9RiiVnU/gmKJXKBVAeS7apurnA3u83jQgy+sN7QxQ1S1stkzTSQaC4iGnFsvpEAr94fc/BBtjcxpCUAZoyp9maoiKSV2baHlnFvqDrSLd9AMInXkm+KmPagv5TTUbK9oM= ceph-5227da4c-8d37-11ee-95a2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-11-27T15:14:10.890 INFO:teuthology.orchestra.run.smithi177.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCvzDKejWOae18ZOHtCrzB9Hgme2PKW1Fi/b14QgvG3+r81TTqgD/eJ5T2CtyGQa/yktGAUNPWKGfEUd7LVRP9lbCuIdUhZR9cC7vykLXD37bR0Lg6MIFI6lhkHzR61vSPZ/YbG2rqnyUr6JexwgcyJWmBnxZoxez9sTKLQpyEzfePhNXo0YfI24kJObBA8DZR+sKqCmlDkwwNlQLFQQpfEcXCQN5NG+hwQIxrWM3luCikkNWbLhsgFNLbaredn4x0+MH6+AM5Hecp6cbZi+Xtz/VYn33S90jh5hUD2acROkpMds7IsaXHtj6vAXZafi1BokEUwE7CUOBH2ZTL4ML1DO8Tl+/zcbqf94SoMhgBAD7Fw5EM9Prh0vQdekuxJTi9RiiVnU/gmKJXKBVAeS7apurnA3u83jQgy+sN7QxQ1S1stkzTSQaC4iGnFsvpEAr94fc/BBtjcxpCUAZoyp9maoiKSV2baHlnFvqDrSLd9AMInXkm+KmPagv5TTUbK9oM= ceph-5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:14:10.902 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-11-27T15:14:11.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:10 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1357459270' entity='client.admin' 2023-11-27T15:14:11.934 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:14:13.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.598+0000 7f3c416b7700 1 -- 172.21.15.133:0/2104701312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c3c100660 msgr2=0x7f3c3c102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:13.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.598+0000 7f3c416b7700 1 --2- 172.21.15.133:0/2104701312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c3c100660 0x7f3c3c102a40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009910 tx=0x7f3c2c009c20 comp rx=0 tx=0).stop 2023-11-27T15:14:13.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.599+0000 7f3c416b7700 1 -- 172.21.15.133:0/2104701312 shutdown_connections 2023-11-27T15:14:13.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.599+0000 7f3c416b7700 1 --2- 172.21.15.133:0/2104701312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c3c100660 0x7f3c3c102a40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:13.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.599+0000 7f3c416b7700 1 -- 172.21.15.133:0/2104701312 >> 172.21.15.133:0/2104701312 conn(0x7f3c3c0fa080 msgr2=0x7f3c3c0fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:13.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.599+0000 7f3c416b7700 1 -- 172.21.15.133:0/2104701312 shutdown_connections 2023-11-27T15:14:13.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.599+0000 7f3c416b7700 1 -- 172.21.15.133:0/2104701312 wait complete. 2023-11-27T15:14:13.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.600+0000 7f3c416b7700 1 Processor -- start 2023-11-27T15:14:13.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.600+0000 7f3c416b7700 1 -- start start 2023-11-27T15:14:13.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.600+0000 7f3c416b7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c3c100660 0x7f3c3c196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:13.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.600+0000 7f3c416b7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c197390 con 0x7f3c3c100660 2023-11-27T15:14:13.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.601+0000 7f3c3bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c3c100660 0x7f3c3c196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:13.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.601+0000 7f3c3bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c3c100660 0x7f3c3c196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47648/0 (socket says 172.21.15.133:47648) 2023-11-27T15:14:13.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.601+0000 7f3c3bfff700 1 -- 172.21.15.133:0/3530440192 learned_addr learned my addr 172.21.15.133:0/3530440192 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:14:13.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.601+0000 7f3c3bfff700 1 -- 172.21.15.133:0/3530440192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2c0095f0 con 0x7f3c3c100660 2023-11-27T15:14:13.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.601+0000 7f3c3bfff700 1 --2- 172.21.15.133:0/3530440192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c3c100660 0x7f3c3c196e50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c006010 tx=0x7f3c2c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:13.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.601+0000 7f3c397fa700 1 -- 172.21.15.133:0/3530440192 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c2c01d070 con 0x7f3c3c100660 2023-11-27T15:14:13.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.601+0000 7f3c397fa700 1 -- 172.21.15.133:0/3530440192 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c2c0044e0 con 0x7f3c3c100660 2023-11-27T15:14:13.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.601+0000 7f3c397fa700 1 -- 172.21.15.133:0/3530440192 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c2c0039b0 con 0x7f3c3c100660 2023-11-27T15:14:13.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.602+0000 7f3c416b7700 1 -- 172.21.15.133:0/3530440192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c3c105970 con 0x7f3c3c100660 2023-11-27T15:14:13.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.602+0000 7f3c416b7700 1 -- 172.21.15.133:0/3530440192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3c1057e0 con 0x7f3c3c100660 2023-11-27T15:14:13.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.603+0000 7f3c397fa700 1 -- 172.21.15.133:0/3530440192 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f3c2c00baf0 con 0x7f3c3c100660 2023-11-27T15:14:13.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.603+0000 7f3c416b7700 1 -- 172.21.15.133:0/3530440192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c3c105b00 con 0x7f3c3c100660 2023-11-27T15:14:13.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.603+0000 7f3c397fa700 1 --2- 172.21.15.133:0/3530440192 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3c24038870 0x7f3c2403ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:13.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.603+0000 7f3c397fa700 1 -- 172.21.15.133:0/3530440192 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3c2c04c890 con 0x7f3c3c100660 2023-11-27T15:14:13.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.603+0000 7f3c3b7fe700 1 --2- 172.21.15.133:0/3530440192 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3c24038870 0x7f3c2403ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:13.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.604+0000 7f3c3b7fe700 1 --2- 172.21.15.133:0/3530440192 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3c24038870 0x7f3c2403ad20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3c30006fd0 tx=0x7f3c30006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:13.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.608+0000 7f3c397fa700 1 -- 172.21.15.133:0/3530440192 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3c2c02a930 con 0x7f3c3c100660 2023-11-27T15:14:13.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.810+0000 7f3c416b7700 1 -- 172.21.15.133:0/3530440192 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f3c3c197980 con 0x7f3c3c100660 2023-11-27T15:14:13.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.813+0000 7f3c397fa700 1 -- 172.21.15.133:0/3530440192 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f3c2c02a360 con 0x7f3c3c100660 2023-11-27T15:14:13.820 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.816+0000 7f3c416b7700 1 -- 172.21.15.133:0/3530440192 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3c24038870 msgr2=0x7f3c2403ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:13.820 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.816+0000 7f3c416b7700 1 --2- 172.21.15.133:0/3530440192 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3c24038870 0x7f3c2403ad20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3c30006fd0 tx=0x7f3c30006e40 comp rx=0 tx=0).stop 2023-11-27T15:14:13.820 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.816+0000 7f3c416b7700 1 -- 172.21.15.133:0/3530440192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c3c100660 msgr2=0x7f3c3c196e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:13.820 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.816+0000 7f3c416b7700 1 --2- 172.21.15.133:0/3530440192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c3c100660 0x7f3c3c196e50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c006010 tx=0x7f3c2c0057c0 comp rx=0 tx=0).stop 2023-11-27T15:14:13.820 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.816+0000 7f3c416b7700 1 -- 172.21.15.133:0/3530440192 shutdown_connections 2023-11-27T15:14:13.820 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.816+0000 7f3c416b7700 1 --2- 172.21.15.133:0/3530440192 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3c24038870 0x7f3c2403ad20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:13.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.816+0000 7f3c416b7700 1 --2- 172.21.15.133:0/3530440192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c3c100660 0x7f3c3c196e50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:13.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.816+0000 7f3c416b7700 1 -- 172.21.15.133:0/3530440192 >> 172.21.15.133:0/3530440192 conn(0x7f3c3c0fa080 msgr2=0x7f3c3c0fad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:13.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.816+0000 7f3c416b7700 1 -- 172.21.15.133:0/3530440192 shutdown_connections 2023-11-27T15:14:13.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:13.816+0000 7f3c416b7700 1 -- 172.21.15.133:0/3530440192 wait complete. 2023-11-27T15:14:14.229 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-11-27T15:14:14.229 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-11-27T15:14:15.067 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:14:15.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:14 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3530440192' entity='client.admin' 2023-11-27T15:14:15.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:15.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:15.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi133", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:15.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:15.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:15.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:16.447 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.442+0000 7f435a791700 1 -- 172.21.15.133:0/1715281886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43540ffd20 msgr2=0x7f4354100130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:16.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.442+0000 7f435a791700 1 --2- 172.21.15.133:0/1715281886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43540ffd20 0x7f4354100130 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4344009910 tx=0x7f4344009c20 comp rx=0 tx=0).stop 2023-11-27T15:14:16.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.443+0000 7f435a791700 1 -- 172.21.15.133:0/1715281886 shutdown_connections 2023-11-27T15:14:16.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.443+0000 7f435a791700 1 --2- 172.21.15.133:0/1715281886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43540ffd20 0x7f4354100130 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:16.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.443+0000 7f435a791700 1 -- 172.21.15.133:0/1715281886 >> 172.21.15.133:0/1715281886 conn(0x7f43540fb2b0 msgr2=0x7f43540fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:16.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.443+0000 7f435a791700 1 -- 172.21.15.133:0/1715281886 shutdown_connections 2023-11-27T15:14:16.448 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.443+0000 7f435a791700 1 -- 172.21.15.133:0/1715281886 wait complete. 2023-11-27T15:14:16.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.444+0000 7f435a791700 1 Processor -- start 2023-11-27T15:14:16.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.444+0000 7f435a791700 1 -- start start 2023-11-27T15:14:16.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.444+0000 7f435a791700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43540ffd20 0x7f4354074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:16.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.444+0000 7f435a791700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4354074dd0 con 0x7f43540ffd20 2023-11-27T15:14:16.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.444+0000 7f4353fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43540ffd20 0x7f4354074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:16.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.444+0000 7f4353fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43540ffd20 0x7f4354074890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47672/0 (socket says 172.21.15.133:47672) 2023-11-27T15:14:16.449 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.444+0000 7f4353fff700 1 -- 172.21.15.133:0/1749502778 learned_addr learned my addr 172.21.15.133:0/1749502778 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:14:16.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.445+0000 7f4353fff700 1 -- 172.21.15.133:0/1749502778 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43440095f0 con 0x7f43540ffd20 2023-11-27T15:14:16.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.445+0000 7f4353fff700 1 --2- 172.21.15.133:0/1749502778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43540ffd20 0x7f4354074890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4344004f40 tx=0x7f43440057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:16.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.445+0000 7f43517fa700 1 -- 172.21.15.133:0/1749502778 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f434401d070 con 0x7f43540ffd20 2023-11-27T15:14:16.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.445+0000 7f43517fa700 1 -- 172.21.15.133:0/1749502778 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f434400bc10 con 0x7f43540ffd20 2023-11-27T15:14:16.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.445+0000 7f43517fa700 1 -- 172.21.15.133:0/1749502778 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4344003a70 con 0x7f43540ffd20 2023-11-27T15:14:16.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.445+0000 7f435a791700 1 -- 172.21.15.133:0/1749502778 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4354105bf0 con 0x7f43540ffd20 2023-11-27T15:14:16.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.445+0000 7f435a791700 1 -- 172.21.15.133:0/1749502778 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4354105a60 con 0x7f43540ffd20 2023-11-27T15:14:16.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.446+0000 7f435a791700 1 -- 172.21.15.133:0/1749502778 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4354062120 con 0x7f43540ffd20 2023-11-27T15:14:16.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.446+0000 7f43517fa700 1 -- 172.21.15.133:0/1749502778 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f4344003ce0 con 0x7f43540ffd20 2023-11-27T15:14:16.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.447+0000 7f43517fa700 1 --2- 172.21.15.133:0/1749502778 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f433c0388c0 0x7f433c03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:16.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.447+0000 7f43517fa700 1 -- 172.21.15.133:0/1749502778 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f434404c510 con 0x7f43540ffd20 2023-11-27T15:14:16.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.447+0000 7f43537fe700 1 --2- 172.21.15.133:0/1749502778 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f433c0388c0 0x7f433c03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:16.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.448+0000 7f43537fe700 1 --2- 172.21.15.133:0/1749502778 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f433c0388c0 0x7f433c03ad70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4348006fd0 tx=0x7f4348006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:16.456 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.452+0000 7f43517fa700 1 -- 172.21.15.133:0/1749502778 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f434402abf0 con 0x7f43540ffd20 2023-11-27T15:14:16.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.652+0000 7f435a791700 1 -- 172.21.15.133:0/1749502778 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f43540732a0 con 0x7f433c0388c0 2023-11-27T15:14:16.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.656+0000 7f43517fa700 1 -- 172.21.15.133:0/1749502778 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f43540732a0 con 0x7f433c0388c0 2023-11-27T15:14:16.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.658+0000 7f435a791700 1 -- 172.21.15.133:0/1749502778 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f433c0388c0 msgr2=0x7f433c03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:16.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.658+0000 7f435a791700 1 --2- 172.21.15.133:0/1749502778 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f433c0388c0 0x7f433c03ad70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4348006fd0 tx=0x7f4348006e40 comp rx=0 tx=0).stop 2023-11-27T15:14:16.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.658+0000 7f435a791700 1 -- 172.21.15.133:0/1749502778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43540ffd20 msgr2=0x7f4354074890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:16.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.658+0000 7f435a791700 1 --2- 172.21.15.133:0/1749502778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43540ffd20 0x7f4354074890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4344004f40 tx=0x7f43440057c0 comp rx=0 tx=0).stop 2023-11-27T15:14:16.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.658+0000 7f435a791700 1 -- 172.21.15.133:0/1749502778 shutdown_connections 2023-11-27T15:14:16.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.658+0000 7f435a791700 1 --2- 172.21.15.133:0/1749502778 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f433c0388c0 0x7f433c03ad70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:16.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.658+0000 7f435a791700 1 --2- 172.21.15.133:0/1749502778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43540ffd20 0x7f4354074890 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:16.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.658+0000 7f435a791700 1 -- 172.21.15.133:0/1749502778 >> 172.21.15.133:0/1749502778 conn(0x7f43540fb2b0 msgr2=0x7f43540fbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:16.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.659+0000 7f435a791700 1 -- 172.21.15.133:0/1749502778 shutdown_connections 2023-11-27T15:14:16.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:16.659+0000 7f435a791700 1 -- 172.21.15.133:0/1749502778 wait complete. 2023-11-27T15:14:17.147 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi174 2023-11-27T15:14:17.147 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-27T15:14:17.147 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.conf 2023-11-27T15:14:17.168 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-27T15:14:17.168 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:14:17.228 INFO:tasks.cephadm:Adding host smithi174 to orchestrator... 2023-11-27T15:14:17.228 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph orch host add smithi174 2023-11-27T15:14:18.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:17 smithi133 ceph-mon[97860]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:14:18.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:17 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:18.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:17 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:18.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:17 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:18.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:17 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:18.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:17 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:14:18.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:17 smithi133 ceph-mon[97860]: Updating smithi133:/etc/ceph/ceph.conf 2023-11-27T15:14:18.103 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:14:18.976 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:18 smithi133 ceph-mon[97860]: Updating smithi133:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:14:18.976 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:18 smithi133 ceph-mon[97860]: Updating smithi133:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:14:19.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.598+0000 7fcf3b008700 1 -- 172.21.15.133:0/3786903641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf340fff90 msgr2=0x7fcf34100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:19.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.598+0000 7fcf3b008700 1 --2- 172.21.15.133:0/3786903641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf340fff90 0x7fcf34100360 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009910 tx=0x7fcf28009c20 comp rx=0 tx=0).stop 2023-11-27T15:14:19.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.598+0000 7fcf3b008700 1 -- 172.21.15.133:0/3786903641 shutdown_connections 2023-11-27T15:14:19.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.598+0000 7fcf3b008700 1 --2- 172.21.15.133:0/3786903641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf340fff90 0x7fcf34100360 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.598+0000 7fcf3b008700 1 -- 172.21.15.133:0/3786903641 >> 172.21.15.133:0/3786903641 conn(0x7fcf340fb870 msgr2=0x7fcf340fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.598+0000 7fcf3b008700 1 -- 172.21.15.133:0/3786903641 shutdown_connections 2023-11-27T15:14:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.599+0000 7fcf3b008700 1 -- 172.21.15.133:0/3786903641 wait complete. 2023-11-27T15:14:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.599+0000 7fcf3b008700 1 Processor -- start 2023-11-27T15:14:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.600+0000 7fcf3b008700 1 -- start start 2023-11-27T15:14:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.600+0000 7fcf3b008700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf340fff90 0x7fcf34197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.600+0000 7fcf3b008700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf341978a0 con 0x7fcf340fff90 2023-11-27T15:14:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.600+0000 7fcf38da4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf340fff90 0x7fcf34197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.600+0000 7fcf38da4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf340fff90 0x7fcf34197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47702/0 (socket says 172.21.15.133:47702) 2023-11-27T15:14:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.600+0000 7fcf38da4700 1 -- 172.21.15.133:0/3676675783 learned_addr learned my addr 172.21.15.133:0/3676675783 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:14:19.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.601+0000 7fcf38da4700 1 -- 172.21.15.133:0/3676675783 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf280095f0 con 0x7fcf340fff90 2023-11-27T15:14:19.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.601+0000 7fcf38da4700 1 --2- 172.21.15.133:0/3676675783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf340fff90 0x7fcf34197360 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcf28006010 tx=0x7fcf28004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:19.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.601+0000 7fcf31ffb700 1 -- 172.21.15.133:0/3676675783 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf2801d070 con 0x7fcf340fff90 2023-11-27T15:14:19.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.601+0000 7fcf3b008700 1 -- 172.21.15.133:0/3676675783 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf3410a170 con 0x7fcf340fff90 2023-11-27T15:14:19.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.601+0000 7fcf31ffb700 1 -- 172.21.15.133:0/3676675783 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf2800bc10 con 0x7fcf340fff90 2023-11-27T15:14:19.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.601+0000 7fcf31ffb700 1 -- 172.21.15.133:0/3676675783 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf28003910 con 0x7fcf340fff90 2023-11-27T15:14:19.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.601+0000 7fcf3b008700 1 -- 172.21.15.133:0/3676675783 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf34109fe0 con 0x7fcf340fff90 2023-11-27T15:14:19.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.602+0000 7fcf31ffb700 1 -- 172.21.15.133:0/3676675783 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7fcf2800bd80 con 0x7fcf340fff90 2023-11-27T15:14:19.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.602+0000 7fcf3b008700 1 -- 172.21.15.133:0/3676675783 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf3410a300 con 0x7fcf340fff90 2023-11-27T15:14:19.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.603+0000 7fcf31ffb700 1 --2- 172.21.15.133:0/3676675783 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcf24038850 0x7fcf2403ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:19.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.603+0000 7fcf31ffb700 1 -- 172.21.15.133:0/3676675783 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcf2804c600 con 0x7fcf340fff90 2023-11-27T15:14:19.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.603+0000 7fcf33fff700 1 --2- 172.21.15.133:0/3676675783 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcf24038850 0x7fcf2403ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:19.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.603+0000 7fcf33fff700 1 --2- 172.21.15.133:0/3676675783 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcf24038850 0x7fcf2403ad00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcf20006fd0 tx=0x7fcf20006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:19.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.608+0000 7fcf31ffb700 1 -- 172.21.15.133:0/3676675783 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcf28029360 con 0x7fcf340fff90 2023-11-27T15:14:19.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:19.809+0000 7fcf3b008700 1 -- 172.21.15.133:0/3676675783 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf34197ea0 con 0x7fcf24038850 2023-11-27T15:14:20.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:19 smithi133 ceph-mon[97860]: Updating smithi133:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:14:20.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:20.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:20.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:21.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:20 smithi133 ceph-mon[97860]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:14:21.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:21.916+0000 7fcf31ffb700 1 -- 172.21.15.133:0/3676675783 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fcf34197ea0 con 0x7fcf24038850 2023-11-27T15:14:21.921 INFO:teuthology.orchestra.run.smithi133.stdout:Added host 'smithi174' with addr '172.21.15.174' 2023-11-27T15:14:21.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:21.918+0000 7fcf3b008700 1 -- 172.21.15.133:0/3676675783 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcf24038850 msgr2=0x7fcf2403ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:21.918+0000 7fcf3b008700 1 --2- 172.21.15.133:0/3676675783 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcf24038850 0x7fcf2403ad00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcf20006fd0 tx=0x7fcf20006e40 comp rx=0 tx=0).stop 2023-11-27T15:14:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:21.918+0000 7fcf3b008700 1 -- 172.21.15.133:0/3676675783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf340fff90 msgr2=0x7fcf34197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:21.918+0000 7fcf3b008700 1 --2- 172.21.15.133:0/3676675783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf340fff90 0x7fcf34197360 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcf28006010 tx=0x7fcf28004ff0 comp rx=0 tx=0).stop 2023-11-27T15:14:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:21.918+0000 7fcf3b008700 1 -- 172.21.15.133:0/3676675783 shutdown_connections 2023-11-27T15:14:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:21.918+0000 7fcf3b008700 1 --2- 172.21.15.133:0/3676675783 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcf24038850 0x7fcf2403ad00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:21.918+0000 7fcf3b008700 1 --2- 172.21.15.133:0/3676675783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf340fff90 0x7fcf34197360 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:21.918+0000 7fcf3b008700 1 -- 172.21.15.133:0/3676675783 >> 172.21.15.133:0/3676675783 conn(0x7fcf340fb870 msgr2=0x7fcf341038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:21.918+0000 7fcf3b008700 1 -- 172.21.15.133:0/3676675783 shutdown_connections 2023-11-27T15:14:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:21.919+0000 7fcf3b008700 1 -- 172.21.15.133:0/3676675783 wait complete. 2023-11-27T15:14:22.098 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:21 smithi133 ceph-mon[97860]: Deploying cephadm binary to smithi174 2023-11-27T15:14:22.395 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph orch host ls --format=json 2023-11-27T15:14:23.236 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:14:23.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:22 smithi133 ceph-mon[97860]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:23.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:22 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:23.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:22 smithi133 ceph-mon[97860]: Added host smithi174 2023-11-27T15:14:23.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:22 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:23.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:22 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:24.377 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:24 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:24.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.641+0000 7f3e426b6700 1 -- 172.21.15.133:0/2192864039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e3c0fff90 msgr2=0x7f3e3c100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:24.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.641+0000 7f3e426b6700 1 --2- 172.21.15.133:0/2192864039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e3c0fff90 0x7f3e3c100360 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c009910 tx=0x7f3e2c009c20 comp rx=0 tx=0).stop 2023-11-27T15:14:24.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.641+0000 7f3e426b6700 1 -- 172.21.15.133:0/2192864039 shutdown_connections 2023-11-27T15:14:24.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.641+0000 7f3e426b6700 1 --2- 172.21.15.133:0/2192864039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e3c0fff90 0x7f3e3c100360 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:24.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.641+0000 7f3e426b6700 1 -- 172.21.15.133:0/2192864039 >> 172.21.15.133:0/2192864039 conn(0x7f3e3c0fb870 msgr2=0x7f3e3c0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:24.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.641+0000 7f3e426b6700 1 -- 172.21.15.133:0/2192864039 shutdown_connections 2023-11-27T15:14:24.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.641+0000 7f3e426b6700 1 -- 172.21.15.133:0/2192864039 wait complete. 2023-11-27T15:14:24.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.642+0000 7f3e426b6700 1 Processor -- start 2023-11-27T15:14:24.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.643+0000 7f3e426b6700 1 -- start start 2023-11-27T15:14:24.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.643+0000 7f3e426b6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e3c0fff90 0x7f3e3c1951a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:24.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.643+0000 7f3e426b6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e3c1956e0 con 0x7f3e3c0fff90 2023-11-27T15:14:24.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.644+0000 7f3e3bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e3c0fff90 0x7f3e3c1951a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:24.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.644+0000 7f3e3bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e3c0fff90 0x7f3e3c1951a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36660/0 (socket says 172.21.15.133:36660) 2023-11-27T15:14:24.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.644+0000 7f3e3bfff700 1 -- 172.21.15.133:0/2448179467 learned_addr learned my addr 172.21.15.133:0/2448179467 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:14:24.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.644+0000 7f3e3bfff700 1 -- 172.21.15.133:0/2448179467 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2c0095f0 con 0x7f3e3c0fff90 2023-11-27T15:14:24.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.644+0000 7f3e3bfff700 1 --2- 172.21.15.133:0/2448179467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e3c0fff90 0x7f3e3c1951a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c004f40 tx=0x7f3e2c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:24.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.644+0000 7f3e397fa700 1 -- 172.21.15.133:0/2448179467 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e2c01d070 con 0x7f3e3c0fff90 2023-11-27T15:14:24.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.644+0000 7f3e397fa700 1 -- 172.21.15.133:0/2448179467 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e2c004370 con 0x7f3e3c0fff90 2023-11-27T15:14:24.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.644+0000 7f3e426b6700 1 -- 172.21.15.133:0/2448179467 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3c10a170 con 0x7f3e3c0fff90 2023-11-27T15:14:24.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.644+0000 7f3e397fa700 1 -- 172.21.15.133:0/2448179467 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e2c0039b0 con 0x7f3e3c0fff90 2023-11-27T15:14:24.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.645+0000 7f3e426b6700 1 -- 172.21.15.133:0/2448179467 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e3c109fe0 con 0x7f3e3c0fff90 2023-11-27T15:14:24.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.646+0000 7f3e397fa700 1 -- 172.21.15.133:0/2448179467 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f3e2c003c20 con 0x7f3e3c0fff90 2023-11-27T15:14:24.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.646+0000 7f3e426b6700 1 -- 172.21.15.133:0/2448179467 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e3c10a300 con 0x7f3e3c0fff90 2023-11-27T15:14:24.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.646+0000 7f3e397fa700 1 --2- 172.21.15.133:0/2448179467 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3e240388c0 0x7f3e2403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:24.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.646+0000 7f3e397fa700 1 -- 172.21.15.133:0/2448179467 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3e2c04c8b0 con 0x7f3e3c0fff90 2023-11-27T15:14:24.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.646+0000 7f3e3b7fe700 1 --2- 172.21.15.133:0/2448179467 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3e240388c0 0x7f3e2403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:24.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.647+0000 7f3e3b7fe700 1 --2- 172.21.15.133:0/2448179467 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3e240388c0 0x7f3e2403ad70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3e30006fd0 tx=0x7f3e30006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:24.655 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.651+0000 7f3e397fa700 1 -- 172.21.15.133:0/2448179467 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3e2c029360 con 0x7f3e3c0fff90 2023-11-27T15:14:24.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.850+0000 7f3e426b6700 1 -- 172.21.15.133:0/2448179467 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e3c195ce0 con 0x7f3e240388c0 2023-11-27T15:14:24.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.852+0000 7f3e397fa700 1 -- 172.21.15.133:0/2448179467 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f3e3c195ce0 con 0x7f3e240388c0 2023-11-27T15:14:24.856 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:14:24.857 INFO:teuthology.orchestra.run.smithi133.stdout:[{"addr": "172.21.15.133", "hostname": "smithi133", "labels": [], "status": ""}, {"addr": "172.21.15.174", "hostname": "smithi174", "labels": [], "status": ""}] 2023-11-27T15:14:24.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.854+0000 7f3e426b6700 1 -- 172.21.15.133:0/2448179467 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3e240388c0 msgr2=0x7f3e2403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:24.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.854+0000 7f3e426b6700 1 --2- 172.21.15.133:0/2448179467 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3e240388c0 0x7f3e2403ad70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3e30006fd0 tx=0x7f3e30006e40 comp rx=0 tx=0).stop 2023-11-27T15:14:24.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.854+0000 7f3e426b6700 1 -- 172.21.15.133:0/2448179467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e3c0fff90 msgr2=0x7f3e3c1951a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:24.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.854+0000 7f3e426b6700 1 --2- 172.21.15.133:0/2448179467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e3c0fff90 0x7f3e3c1951a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c004f40 tx=0x7f3e2c005020 comp rx=0 tx=0).stop 2023-11-27T15:14:24.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.854+0000 7f3e426b6700 1 -- 172.21.15.133:0/2448179467 shutdown_connections 2023-11-27T15:14:24.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.854+0000 7f3e426b6700 1 --2- 172.21.15.133:0/2448179467 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3e240388c0 0x7f3e2403ad70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:24.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.854+0000 7f3e426b6700 1 --2- 172.21.15.133:0/2448179467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e3c0fff90 0x7f3e3c1951a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:24.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.854+0000 7f3e426b6700 1 -- 172.21.15.133:0/2448179467 >> 172.21.15.133:0/2448179467 conn(0x7f3e3c0fb870 msgr2=0x7f3e3c104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:24.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.854+0000 7f3e426b6700 1 -- 172.21.15.133:0/2448179467 shutdown_connections 2023-11-27T15:14:24.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:24.855+0000 7f3e426b6700 1 -- 172.21.15.133:0/2448179467 wait complete. 2023-11-27T15:14:25.320 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi177 2023-11-27T15:14:25.320 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-27T15:14:25.321 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/etc/ceph/ceph.conf 2023-11-27T15:14:25.341 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-27T15:14:25.342 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:14:25.401 INFO:tasks.cephadm:Adding host smithi177 to orchestrator... 2023-11-27T15:14:25.401 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph orch host add smithi177 2023-11-27T15:14:25.463 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:25 smithi133 ceph-mon[97860]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:25.463 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:25 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:26.242 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:14:26.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:26 smithi133 ceph-mon[97860]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:14:27.485 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:27 smithi133 ceph-mon[97860]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:27.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.749+0000 7fd4fbef8700 1 -- 172.21.15.133:0/1143043627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4f40fff90 msgr2=0x7fd4f4100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:27.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.749+0000 7fd4fbef8700 1 --2- 172.21.15.133:0/1143043627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4f40fff90 0x7fd4f4100360 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009910 tx=0x7fd4f0009c20 comp rx=0 tx=0).stop 2023-11-27T15:14:27.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.749+0000 7fd4fbef8700 1 -- 172.21.15.133:0/1143043627 shutdown_connections 2023-11-27T15:14:27.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.749+0000 7fd4fbef8700 1 --2- 172.21.15.133:0/1143043627 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4f40fff90 0x7fd4f4100360 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:27.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.749+0000 7fd4fbef8700 1 -- 172.21.15.133:0/1143043627 >> 172.21.15.133:0/1143043627 conn(0x7fd4f40fb870 msgr2=0x7fd4f40fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:27.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.749+0000 7fd4fbef8700 1 -- 172.21.15.133:0/1143043627 shutdown_connections 2023-11-27T15:14:27.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.749+0000 7fd4fbef8700 1 -- 172.21.15.133:0/1143043627 wait complete. 2023-11-27T15:14:27.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.751+0000 7fd4fbef8700 1 Processor -- start 2023-11-27T15:14:27.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.751+0000 7fd4fbef8700 1 -- start start 2023-11-27T15:14:27.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.752+0000 7fd4fbef8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4f40fff90 0x7fd4f41951a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:27.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.752+0000 7fd4fbef8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f41956e0 con 0x7fd4f40fff90 2023-11-27T15:14:27.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.752+0000 7fd4f9c94700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4f40fff90 0x7fd4f41951a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:27.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.752+0000 7fd4f9c94700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4f40fff90 0x7fd4f41951a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36688/0 (socket says 172.21.15.133:36688) 2023-11-27T15:14:27.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.752+0000 7fd4f9c94700 1 -- 172.21.15.133:0/1183486277 learned_addr learned my addr 172.21.15.133:0/1183486277 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:14:27.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.752+0000 7fd4f9c94700 1 -- 172.21.15.133:0/1183486277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f00095f0 con 0x7fd4f40fff90 2023-11-27T15:14:27.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.752+0000 7fd4f9c94700 1 --2- 172.21.15.133:0/1183486277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4f40fff90 0x7fd4f41951a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0006010 tx=0x7fd4f0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:27.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.753+0000 7fd4eaffd700 1 -- 172.21.15.133:0/1183486277 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4f001d070 con 0x7fd4f40fff90 2023-11-27T15:14:27.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.753+0000 7fd4eaffd700 1 -- 172.21.15.133:0/1183486277 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4f000bc10 con 0x7fd4f40fff90 2023-11-27T15:14:27.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.753+0000 7fd4fbef8700 1 -- 172.21.15.133:0/1183486277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f410a170 con 0x7fd4f40fff90 2023-11-27T15:14:27.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.753+0000 7fd4eaffd700 1 -- 172.21.15.133:0/1183486277 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4f0003910 con 0x7fd4f40fff90 2023-11-27T15:14:27.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.753+0000 7fd4fbef8700 1 -- 172.21.15.133:0/1183486277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f4109fe0 con 0x7fd4f40fff90 2023-11-27T15:14:27.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.754+0000 7fd4eaffd700 1 -- 172.21.15.133:0/1183486277 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7fd4f0003bd0 con 0x7fd4f40fff90 2023-11-27T15:14:27.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.754+0000 7fd4eaffd700 1 --2- 172.21.15.133:0/1183486277 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd4e0038850 0x7fd4e003ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:27.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.754+0000 7fd4fbef8700 1 -- 172.21.15.133:0/1183486277 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f410a300 con 0x7fd4f40fff90 2023-11-27T15:14:27.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.754+0000 7fd4eaffd700 1 -- 172.21.15.133:0/1183486277 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd4f004c590 con 0x7fd4f40fff90 2023-11-27T15:14:27.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.755+0000 7fd4f9493700 1 --2- 172.21.15.133:0/1183486277 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd4e0038850 0x7fd4e003ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:27.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.755+0000 7fd4f9493700 1 --2- 172.21.15.133:0/1183486277 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd4e0038850 0x7fd4e003ad00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4006fd0 tx=0x7fd4e4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:27.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.760+0000 7fd4eaffd700 1 -- 172.21.15.133:0/1183486277 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd4f0026070 con 0x7fd4f40fff90 2023-11-27T15:14:27.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:27.967+0000 7fd4fbef8700 1 -- 172.21.15.133:0/1183486277 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi177", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4f4195ce0 con 0x7fd4e0038850 2023-11-27T15:14:28.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:28.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:28.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:28.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:28.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:28.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:28.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:28.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:28 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:14:28.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:28 smithi133 ceph-mon[97860]: Updating smithi174:/etc/ceph/ceph.conf 2023-11-27T15:14:28.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:28.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:28.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:29.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:29 smithi133 ceph-mon[97860]: Updating smithi174:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:14:29.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:29 smithi133 ceph-mon[97860]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:14:29.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:29 smithi133 ceph-mon[97860]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:29.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:29 smithi133 ceph-mon[97860]: Updating smithi174:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:14:29.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:29 smithi133 ceph-mon[97860]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi177", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:14:30.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:30.072+0000 7fd4eaffd700 1 -- 172.21.15.133:0/1183486277 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fd4f4195ce0 con 0x7fd4e0038850 2023-11-27T15:14:30.076 INFO:teuthology.orchestra.run.smithi133.stdout:Added host 'smithi177' with addr '172.21.15.177' 2023-11-27T15:14:30.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:30.074+0000 7fd4fbef8700 1 -- 172.21.15.133:0/1183486277 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd4e0038850 msgr2=0x7fd4e003ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:30.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:30.074+0000 7fd4fbef8700 1 --2- 172.21.15.133:0/1183486277 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd4e0038850 0x7fd4e003ad00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4006fd0 tx=0x7fd4e4006e40 comp rx=0 tx=0).stop 2023-11-27T15:14:30.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:30.074+0000 7fd4fbef8700 1 -- 172.21.15.133:0/1183486277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4f40fff90 msgr2=0x7fd4f41951a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:30.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:30.074+0000 7fd4fbef8700 1 --2- 172.21.15.133:0/1183486277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4f40fff90 0x7fd4f41951a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0006010 tx=0x7fd4f0004ff0 comp rx=0 tx=0).stop 2023-11-27T15:14:30.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:30.074+0000 7fd4fbef8700 1 -- 172.21.15.133:0/1183486277 shutdown_connections 2023-11-27T15:14:30.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:30.074+0000 7fd4fbef8700 1 --2- 172.21.15.133:0/1183486277 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd4e0038850 0x7fd4e003ad00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:30.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:30.074+0000 7fd4fbef8700 1 --2- 172.21.15.133:0/1183486277 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4f40fff90 0x7fd4f41951a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:30.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:30.074+0000 7fd4fbef8700 1 -- 172.21.15.133:0/1183486277 >> 172.21.15.133:0/1183486277 conn(0x7fd4f40fb870 msgr2=0x7fd4f4104170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:30.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:30.075+0000 7fd4fbef8700 1 -- 172.21.15.133:0/1183486277 shutdown_connections 2023-11-27T15:14:30.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:30.075+0000 7fd4fbef8700 1 -- 172.21.15.133:0/1183486277 wait complete. 2023-11-27T15:14:30.513 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:30 smithi133 ceph-mon[97860]: Deploying cephadm binary to smithi177 2023-11-27T15:14:30.513 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:30 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:30.514 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:30 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:30.528 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph orch host ls --format=json 2023-11-27T15:14:31.372 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:14:31.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:31 smithi133 ceph-mon[97860]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:31.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:31 smithi133 ceph-mon[97860]: Added host smithi177 2023-11-27T15:14:31.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:31 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:32.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:32 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:32.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:32 smithi133 ceph-mon[97860]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:32.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.853+0000 7fcfd2bd9700 1 -- 172.21.15.133:0/2050383989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfcc072e90 msgr2=0x7fcfcc073260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:32.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.853+0000 7fcfd2bd9700 1 --2- 172.21.15.133:0/2050383989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfcc072e90 0x7fcfcc073260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009910 tx=0x7fcfc0009c20 comp rx=0 tx=0).stop 2023-11-27T15:14:32.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.853+0000 7fcfd2bd9700 1 -- 172.21.15.133:0/2050383989 shutdown_connections 2023-11-27T15:14:32.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.853+0000 7fcfd2bd9700 1 --2- 172.21.15.133:0/2050383989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfcc072e90 0x7fcfcc073260 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:32.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.853+0000 7fcfd2bd9700 1 -- 172.21.15.133:0/2050383989 >> 172.21.15.133:0/2050383989 conn(0x7fcfcc0fb870 msgr2=0x7fcfcc0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:32.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.853+0000 7fcfd2bd9700 1 -- 172.21.15.133:0/2050383989 shutdown_connections 2023-11-27T15:14:32.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.853+0000 7fcfd2bd9700 1 -- 172.21.15.133:0/2050383989 wait complete. 2023-11-27T15:14:32.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.854+0000 7fcfd2bd9700 1 Processor -- start 2023-11-27T15:14:32.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.855+0000 7fcfd2bd9700 1 -- start start 2023-11-27T15:14:32.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.855+0000 7fcfd2bd9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfcc072e90 0x7fcfcc1a1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:32.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.855+0000 7fcfd2bd9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc1a1aa0 con 0x7fcfcc072e90 2023-11-27T15:14:32.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.855+0000 7fcfd0975700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfcc072e90 0x7fcfcc1a1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:32.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.855+0000 7fcfd0975700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfcc072e90 0x7fcfcc1a1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51442/0 (socket says 172.21.15.133:51442) 2023-11-27T15:14:32.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.855+0000 7fcfd0975700 1 -- 172.21.15.133:0/2931457992 learned_addr learned my addr 172.21.15.133:0/2931457992 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:14:32.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.856+0000 7fcfd0975700 1 -- 172.21.15.133:0/2931457992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc00095f0 con 0x7fcfcc072e90 2023-11-27T15:14:32.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.856+0000 7fcfd0975700 1 --2- 172.21.15.133:0/2931457992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfcc072e90 0x7fcfcc1a1560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0006010 tx=0x7fcfc0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:32.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.856+0000 7fcfc9ffb700 1 -- 172.21.15.133:0/2931457992 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfc001d070 con 0x7fcfcc072e90 2023-11-27T15:14:32.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.856+0000 7fcfd2bd9700 1 -- 172.21.15.133:0/2931457992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfcc110780 con 0x7fcfcc072e90 2023-11-27T15:14:32.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.856+0000 7fcfc9ffb700 1 -- 172.21.15.133:0/2931457992 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfc00044a0 con 0x7fcfcc072e90 2023-11-27T15:14:32.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.856+0000 7fcfc9ffb700 1 -- 172.21.15.133:0/2931457992 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfc0003a70 con 0x7fcfcc072e90 2023-11-27T15:14:32.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.856+0000 7fcfd2bd9700 1 -- 172.21.15.133:0/2931457992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfcc1105f0 con 0x7fcfcc072e90 2023-11-27T15:14:32.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.857+0000 7fcfc9ffb700 1 -- 172.21.15.133:0/2931457992 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7fcfc0003cb0 con 0x7fcfcc072e90 2023-11-27T15:14:32.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.857+0000 7fcfd2bd9700 1 -- 172.21.15.133:0/2931457992 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfcc110910 con 0x7fcfcc072e90 2023-11-27T15:14:32.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.858+0000 7fcfc9ffb700 1 --2- 172.21.15.133:0/2931457992 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcfbc038870 0x7fcfbc03ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:32.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.858+0000 7fcfc9ffb700 1 -- 172.21.15.133:0/2931457992 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcfc004c690 con 0x7fcfcc072e90 2023-11-27T15:14:32.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.858+0000 7fcfcbfff700 1 --2- 172.21.15.133:0/2931457992 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcfbc038870 0x7fcfbc03ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:32.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.858+0000 7fcfcbfff700 1 --2- 172.21.15.133:0/2931457992 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcfbc038870 0x7fcfbc03ad20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8006fd0 tx=0x7fcfb8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:32.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:32.863+0000 7fcfc9ffb700 1 -- 172.21.15.133:0/2931457992 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcfc0029540 con 0x7fcfcc072e90 2023-11-27T15:14:33.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:33.064+0000 7fcfd2bd9700 1 -- 172.21.15.133:0/2931457992 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcfcc0008d0 con 0x7fcfbc038870 2023-11-27T15:14:33.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:33.065+0000 7fcfc9ffb700 1 -- 172.21.15.133:0/2931457992 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+241 (secure 0 0 0) 0x7fcfcc0008d0 con 0x7fcfbc038870 2023-11-27T15:14:33.069 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:14:33.069 INFO:teuthology.orchestra.run.smithi133.stdout:[{"addr": "172.21.15.133", "hostname": "smithi133", "labels": [], "status": ""}, {"addr": "172.21.15.174", "hostname": "smithi174", "labels": [], "status": ""}, {"addr": "172.21.15.177", "hostname": "smithi177", "labels": [], "status": ""}] 2023-11-27T15:14:33.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:33.067+0000 7fcfd2bd9700 1 -- 172.21.15.133:0/2931457992 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcfbc038870 msgr2=0x7fcfbc03ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:33.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:33.067+0000 7fcfd2bd9700 1 --2- 172.21.15.133:0/2931457992 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcfbc038870 0x7fcfbc03ad20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8006fd0 tx=0x7fcfb8006e40 comp rx=0 tx=0).stop 2023-11-27T15:14:33.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:33.067+0000 7fcfd2bd9700 1 -- 172.21.15.133:0/2931457992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfcc072e90 msgr2=0x7fcfcc1a1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:33.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:33.067+0000 7fcfd2bd9700 1 --2- 172.21.15.133:0/2931457992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfcc072e90 0x7fcfcc1a1560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0006010 tx=0x7fcfc0004ff0 comp rx=0 tx=0).stop 2023-11-27T15:14:33.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:33.068+0000 7fcfd2bd9700 1 -- 172.21.15.133:0/2931457992 shutdown_connections 2023-11-27T15:14:33.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:33.068+0000 7fcfd2bd9700 1 --2- 172.21.15.133:0/2931457992 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcfbc038870 0x7fcfbc03ad20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:33.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:33.068+0000 7fcfd2bd9700 1 --2- 172.21.15.133:0/2931457992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfcc072e90 0x7fcfcc1a1560 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:33.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:33.068+0000 7fcfd2bd9700 1 -- 172.21.15.133:0/2931457992 >> 172.21.15.133:0/2931457992 conn(0x7fcfcc0fb870 msgr2=0x7fcfcc0fc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:33.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:33.068+0000 7fcfd2bd9700 1 -- 172.21.15.133:0/2931457992 shutdown_connections 2023-11-27T15:14:33.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:33.068+0000 7fcfd2bd9700 1 -- 172.21.15.133:0/2931457992 wait complete. 2023-11-27T15:14:33.449 INFO:tasks.cephadm:Setting crush tunables to default 2023-11-27T15:14:33.449 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph osd crush tunables default 2023-11-27T15:14:33.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:33 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:33.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:33 smithi133 ceph-mon[97860]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:14:34.296 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:14:34.837 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:34 smithi133 ceph-mon[97860]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.682+0000 7ff18a60a700 1 -- 172.21.15.133:0/1075885193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff184100e00 msgr2=0x7ff1841011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.682+0000 7ff18a60a700 1 --2- 172.21.15.133:0/1075885193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff184100e00 0x7ff1841011d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff174009910 tx=0x7ff174009c20 comp rx=0 tx=0).stop 2023-11-27T15:14:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.682+0000 7ff18a60a700 1 -- 172.21.15.133:0/1075885193 shutdown_connections 2023-11-27T15:14:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.682+0000 7ff18a60a700 1 --2- 172.21.15.133:0/1075885193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff184100e00 0x7ff1841011d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.682+0000 7ff18a60a700 1 -- 172.21.15.133:0/1075885193 >> 172.21.15.133:0/1075885193 conn(0x7ff184074940 msgr2=0x7ff184074d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.682+0000 7ff18a60a700 1 -- 172.21.15.133:0/1075885193 shutdown_connections 2023-11-27T15:14:35.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.682+0000 7ff18a60a700 1 -- 172.21.15.133:0/1075885193 wait complete. 2023-11-27T15:14:35.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.684+0000 7ff18a60a700 1 Processor -- start 2023-11-27T15:14:35.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.684+0000 7ff18a60a700 1 -- start start 2023-11-27T15:14:35.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.684+0000 7ff18a60a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff184100e00 0x7ff1840fec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:35.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.685+0000 7ff18a60a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff184100b60 con 0x7ff184100e00 2023-11-27T15:14:35.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.685+0000 7ff183fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff184100e00 0x7ff1840fec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:35.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.685+0000 7ff183fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff184100e00 0x7ff1840fec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51466/0 (socket says 172.21.15.133:51466) 2023-11-27T15:14:35.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.685+0000 7ff183fff700 1 -- 172.21.15.133:0/693972224 learned_addr learned my addr 172.21.15.133:0/693972224 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:14:35.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.685+0000 7ff183fff700 1 -- 172.21.15.133:0/693972224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1740095f0 con 0x7ff184100e00 2023-11-27T15:14:35.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.685+0000 7ff183fff700 1 --2- 172.21.15.133:0/693972224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff184100e00 0x7ff1840fec30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff174006010 tx=0x7ff174004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:35.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.686+0000 7ff1817fa700 1 -- 172.21.15.133:0/693972224 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff17401d070 con 0x7ff184100e00 2023-11-27T15:14:35.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.686+0000 7ff1817fa700 1 -- 172.21.15.133:0/693972224 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff1740044a0 con 0x7ff184100e00 2023-11-27T15:14:35.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.686+0000 7ff18a60a700 1 -- 172.21.15.133:0/693972224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18410a1a0 con 0x7ff184100e00 2023-11-27T15:14:35.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.686+0000 7ff1817fa700 1 -- 172.21.15.133:0/693972224 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff174003a70 con 0x7ff184100e00 2023-11-27T15:14:35.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.686+0000 7ff18a60a700 1 -- 172.21.15.133:0/693972224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18410a010 con 0x7ff184100e00 2023-11-27T15:14:35.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.687+0000 7ff1817fa700 1 -- 172.21.15.133:0/693972224 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7ff174003cb0 con 0x7ff184100e00 2023-11-27T15:14:35.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.687+0000 7ff18a60a700 1 -- 172.21.15.133:0/693972224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff18410a330 con 0x7ff184100e00 2023-11-27T15:14:35.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.687+0000 7ff1817fa700 1 --2- 172.21.15.133:0/693972224 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff16c038870 0x7ff16c03ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:35.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.687+0000 7ff1817fa700 1 -- 172.21.15.133:0/693972224 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff17404c5c0 con 0x7ff184100e00 2023-11-27T15:14:35.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.688+0000 7ff1837fe700 1 --2- 172.21.15.133:0/693972224 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff16c038870 0x7ff16c03ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:35.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.688+0000 7ff1837fe700 1 --2- 172.21.15.133:0/693972224 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff16c038870 0x7ff16c03ad20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff178006fd0 tx=0x7ff178006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:35.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.692+0000 7ff1817fa700 1 -- 172.21.15.133:0/693972224 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff174029540 con 0x7ff184100e00 2023-11-27T15:14:35.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:35.899+0000 7ff18a60a700 1 -- 172.21.15.133:0/693972224 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ff1841074e0 con 0x7ff184100e00 2023-11-27T15:14:36.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:36.337+0000 7ff1817fa700 1 -- 172.21.15.133:0/693972224 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff174026020 con 0x7ff184100e00 2023-11-27T15:14:36.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:36.340+0000 7ff18a60a700 1 -- 172.21.15.133:0/693972224 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff16c038870 msgr2=0x7ff16c03ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:36.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:36.340+0000 7ff18a60a700 1 --2- 172.21.15.133:0/693972224 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff16c038870 0x7ff16c03ad20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff178006fd0 tx=0x7ff178006e40 comp rx=0 tx=0).stop 2023-11-27T15:14:36.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:36.340+0000 7ff18a60a700 1 -- 172.21.15.133:0/693972224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff184100e00 msgr2=0x7ff1840fec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:36.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:36.340+0000 7ff18a60a700 1 --2- 172.21.15.133:0/693972224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff184100e00 0x7ff1840fec30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff174006010 tx=0x7ff174004ff0 comp rx=0 tx=0).stop 2023-11-27T15:14:36.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:36.340+0000 7ff18a60a700 1 -- 172.21.15.133:0/693972224 shutdown_connections 2023-11-27T15:14:36.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:36.340+0000 7ff18a60a700 1 --2- 172.21.15.133:0/693972224 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff16c038870 0x7ff16c03ad20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:36.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:36.340+0000 7ff18a60a700 1 --2- 172.21.15.133:0/693972224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff184100e00 0x7ff1840fec30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:36.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:36.340+0000 7ff18a60a700 1 -- 172.21.15.133:0/693972224 >> 172.21.15.133:0/693972224 conn(0x7ff184074940 msgr2=0x7ff184104620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:36.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:36.340+0000 7ff18a60a700 1 -- 172.21.15.133:0/693972224 shutdown_connections 2023-11-27T15:14:36.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:36.340+0000 7ff18a60a700 1 -- 172.21.15.133:0/693972224 wait complete. 2023-11-27T15:14:36.345 INFO:teuthology.orchestra.run.smithi133.stderr:adjusted tunables profile to default 2023-11-27T15:14:36.776 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:36 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:36.776 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:36 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:36.776 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:36 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:36.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:36 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:36.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:36 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi177", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:36.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:36 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:36.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:36 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:36.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:36 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:14:36.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:36 smithi133 ceph-mon[97860]: Updating smithi177:/etc/ceph/ceph.conf 2023-11-27T15:14:36.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:36 smithi133 ceph-mon[97860]: Updating smithi177:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:14:36.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:36 smithi133 ceph-mon[97860]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:36.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:36 smithi133 ceph-mon[97860]: Updating smithi177:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:14:36.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:36 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/693972224' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-11-27T15:14:36.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:36 smithi133 ceph-mon[97860]: Updating smithi177:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:14:36.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:36 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:36.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:36 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:36.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:36 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:36.779 INFO:tasks.cephadm:Adding mon.a on smithi133 2023-11-27T15:14:36.779 INFO:tasks.cephadm:Adding mon.b on smithi174 2023-11-27T15:14:36.779 INFO:tasks.cephadm:Adding mon.c on smithi177 2023-11-27T15:14:36.779 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph orch apply mon '3;smithi133:172.21.15.133=a;smithi174:172.21.15.174=b;smithi177:172.21.15.177=c' 2023-11-27T15:14:37.439 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:14:37.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:37 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/693972224' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-11-27T15:14:37.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:37 smithi133 ceph-mon[97860]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:14:38.650 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.644+0000 7ffa04711700 1 -- 172.21.15.177:0/1083536886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9fc0feea0 msgr2=0x7ff9fc0ff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:38.651 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.644+0000 7ffa04711700 1 --2- 172.21.15.177:0/1083536886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9fc0feea0 0x7ff9fc0ff270 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009910 tx=0x7ff9ec009c20 comp rx=0 tx=0).stop 2023-11-27T15:14:38.651 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.644+0000 7ffa04711700 1 -- 172.21.15.177:0/1083536886 shutdown_connections 2023-11-27T15:14:38.651 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.644+0000 7ffa04711700 1 --2- 172.21.15.177:0/1083536886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9fc0feea0 0x7ff9fc0ff270 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:38.651 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.644+0000 7ffa04711700 1 -- 172.21.15.177:0/1083536886 >> 172.21.15.177:0/1083536886 conn(0x7ff9fc0fa760 msgr2=0x7ff9fc0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:38.651 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.644+0000 7ffa04711700 1 -- 172.21.15.177:0/1083536886 shutdown_connections 2023-11-27T15:14:38.651 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.644+0000 7ffa04711700 1 -- 172.21.15.177:0/1083536886 wait complete. 2023-11-27T15:14:38.651 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.645+0000 7ffa04711700 1 Processor -- start 2023-11-27T15:14:38.651 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.645+0000 7ffa04711700 1 -- start start 2023-11-27T15:14:38.651 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.645+0000 7ffa04711700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9fc0feea0 0x7ff9fc194090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:38.651 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.645+0000 7ffa04711700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9fc1945d0 con 0x7ff9fc0feea0 2023-11-27T15:14:38.651 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.645+0000 7ffa024ad700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9fc0feea0 0x7ff9fc194090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:38.652 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.645+0000 7ffa024ad700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9fc0feea0 0x7ff9fc194090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.177:44528/0 (socket says 172.21.15.177:44528) 2023-11-27T15:14:38.652 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.645+0000 7ffa024ad700 1 -- 172.21.15.177:0/2219131874 learned_addr learned my addr 172.21.15.177:0/2219131874 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-27T15:14:38.652 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.646+0000 7ffa024ad700 1 -- 172.21.15.177:0/2219131874 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec0095f0 con 0x7ff9fc0feea0 2023-11-27T15:14:38.653 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.646+0000 7ffa024ad700 1 --2- 172.21.15.177:0/2219131874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9fc0feea0 0x7ff9fc194090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec004f40 tx=0x7ff9ec005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:38.653 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.646+0000 7ff9f37fe700 1 -- 172.21.15.177:0/2219131874 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9ec01d070 con 0x7ff9fc0feea0 2023-11-27T15:14:38.653 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.646+0000 7ffa04711700 1 -- 172.21.15.177:0/2219131874 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9fc109080 con 0x7ff9fc0feea0 2023-11-27T15:14:38.653 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.646+0000 7ff9f37fe700 1 -- 172.21.15.177:0/2219131874 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9ec004370 con 0x7ff9fc0feea0 2023-11-27T15:14:38.654 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.646+0000 7ffa04711700 1 -- 172.21.15.177:0/2219131874 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9fc108ef0 con 0x7ff9fc0feea0 2023-11-27T15:14:38.654 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.646+0000 7ff9f37fe700 1 -- 172.21.15.177:0/2219131874 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9ec0039a0 con 0x7ff9fc0feea0 2023-11-27T15:14:38.654 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.647+0000 7ff9f37fe700 1 -- 172.21.15.177:0/2219131874 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7ff9ec003b00 con 0x7ff9fc0feea0 2023-11-27T15:14:38.654 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.647+0000 7ffa04711700 1 -- 172.21.15.177:0/2219131874 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9fc109210 con 0x7ff9fc0feea0 2023-11-27T15:14:38.654 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.647+0000 7ff9f37fe700 1 --2- 172.21.15.177:0/2219131874 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff9e8038870 0x7ff9e803ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:38.654 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.647+0000 7ff9f37fe700 1 -- 172.21.15.177:0/2219131874 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff9ec04c4e0 con 0x7ff9fc0feea0 2023-11-27T15:14:38.654 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.648+0000 7ffa01cac700 1 --2- 172.21.15.177:0/2219131874 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff9e8038870 0x7ff9e803ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:38.654 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.648+0000 7ffa01cac700 1 --2- 172.21.15.177:0/2219131874 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff9e8038870 0x7ff9e803ad20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8006fd0 tx=0x7ff9f8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:38.658 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.653+0000 7ff9f37fe700 1 -- 172.21.15.177:0/2219131874 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff9ec00f460 con 0x7ff9fc0feea0 2023-11-27T15:14:38.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:38 smithi133 ceph-mon[97860]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:38.852 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.848+0000 7ffa04711700 1 -- 172.21.15.177:0/2219131874 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi133:172.21.15.133=a;smithi174:172.21.15.174=b;smithi177:172.21.15.177=c", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9fc1070a0 con 0x7ff9e8038870 2023-11-27T15:14:38.862 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.858+0000 7ff9f37fe700 1 -- 172.21.15.177:0/2219131874 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff9fc1070a0 con 0x7ff9e8038870 2023-11-27T15:14:38.863 INFO:teuthology.orchestra.run.smithi177.stdout:Scheduled mon update... 2023-11-27T15:14:38.865 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.860+0000 7ffa04711700 1 -- 172.21.15.177:0/2219131874 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff9e8038870 msgr2=0x7ff9e803ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:38.865 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.860+0000 7ffa04711700 1 --2- 172.21.15.177:0/2219131874 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff9e8038870 0x7ff9e803ad20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8006fd0 tx=0x7ff9f8006e40 comp rx=0 tx=0).stop 2023-11-27T15:14:38.865 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.860+0000 7ffa04711700 1 -- 172.21.15.177:0/2219131874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9fc0feea0 msgr2=0x7ff9fc194090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:38.865 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.860+0000 7ffa04711700 1 --2- 172.21.15.177:0/2219131874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9fc0feea0 0x7ff9fc194090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec004f40 tx=0x7ff9ec005020 comp rx=0 tx=0).stop 2023-11-27T15:14:38.865 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.860+0000 7ffa04711700 1 -- 172.21.15.177:0/2219131874 shutdown_connections 2023-11-27T15:14:38.865 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.860+0000 7ffa04711700 1 --2- 172.21.15.177:0/2219131874 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff9e8038870 0x7ff9e803ad20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:38.865 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.860+0000 7ffa04711700 1 --2- 172.21.15.177:0/2219131874 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9fc0feea0 0x7ff9fc194090 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:38.865 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.860+0000 7ffa04711700 1 -- 172.21.15.177:0/2219131874 >> 172.21.15.177:0/2219131874 conn(0x7ff9fc0fa760 msgr2=0x7ff9fc103080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:38.865 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.860+0000 7ffa04711700 1 -- 172.21.15.177:0/2219131874 shutdown_connections 2023-11-27T15:14:38.865 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:38.860+0000 7ffa04711700 1 -- 172.21.15.177:0/2219131874 wait complete. 2023-11-27T15:14:39.203 DEBUG:teuthology.orchestra.run.smithi174:mon.b> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mon.b.service 2023-11-27T15:14:39.206 DEBUG:teuthology.orchestra.run.smithi177:mon.c> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mon.c.service 2023-11-27T15:14:39.208 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-11-27T15:14:39.208 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-11-27T15:14:39.237 INFO:journalctl@ceph.mon.c.smithi177.stdout:-- Logs begin at Mon 2023-11-27 14:50:38 UTC. -- 2023-11-27T15:14:39.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:-- Logs begin at Mon 2023-11-27 14:50:40 UTC. -- 2023-11-27T15:14:39.922 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:14:40.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:39 smithi133 ceph-mon[97860]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi133:172.21.15.133=a;smithi174:172.21.15.174=b;smithi177:172.21.15.177=c", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:14:40.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:39 smithi133 ceph-mon[97860]: Saving service mon spec with placement smithi133:172.21.15.133=a;smithi174:172.21.15.174=b;smithi177:172.21.15.177=c;count:3 2023-11-27T15:14:40.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:39 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:40.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:39 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:40.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:39 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:40.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:39 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:40.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:39 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:14:40.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:39 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:40.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:39 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:14:40.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:39 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:14:40.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:39 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:40.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:39 smithi133 ceph-mon[97860]: Deploying daemon mon.c on smithi177 2023-11-27T15:14:41.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:40 smithi133 ceph-mon[97860]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:41.671 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.665+0000 7f3462e1d700 1 -- 172.21.15.177:0/923841589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f345c0fed70 msgr2=0x7f345c0ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:41.671 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.665+0000 7f3462e1d700 1 --2- 172.21.15.177:0/923841589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f345c0fed70 0x7f345c0ff140 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3450009910 tx=0x7f3450009c20 comp rx=0 tx=0).stop 2023-11-27T15:14:41.671 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.665+0000 7f3462e1d700 1 -- 172.21.15.177:0/923841589 shutdown_connections 2023-11-27T15:14:41.672 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.665+0000 7f3462e1d700 1 --2- 172.21.15.177:0/923841589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f345c0fed70 0x7f345c0ff140 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:41.672 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.665+0000 7f3462e1d700 1 -- 172.21.15.177:0/923841589 >> 172.21.15.177:0/923841589 conn(0x7f345c074350 msgr2=0x7f345c074750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:41.672 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.665+0000 7f3462e1d700 1 -- 172.21.15.177:0/923841589 shutdown_connections 2023-11-27T15:14:41.672 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.665+0000 7f3462e1d700 1 -- 172.21.15.177:0/923841589 wait complete. 2023-11-27T15:14:41.675 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.666+0000 7f3462e1d700 1 Processor -- start 2023-11-27T15:14:41.675 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.667+0000 7f3462e1d700 1 -- start start 2023-11-27T15:14:41.675 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.667+0000 7f3462e1d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f345c0fed70 0x7f345c0fff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:41.676 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.667+0000 7f3462e1d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c101ed0 con 0x7f345c0fed70 2023-11-27T15:14:41.676 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.667+0000 7f3460bb9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f345c0fed70 0x7f345c0fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:41.676 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.667+0000 7f3460bb9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f345c0fed70 0x7f345c0fff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.177:44534/0 (socket says 172.21.15.177:44534) 2023-11-27T15:14:41.676 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.667+0000 7f3460bb9700 1 -- 172.21.15.177:0/3726936084 learned_addr learned my addr 172.21.15.177:0/3726936084 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-27T15:14:41.676 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.668+0000 7f3460bb9700 1 -- 172.21.15.177:0/3726936084 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34500095f0 con 0x7f345c0fed70 2023-11-27T15:14:41.676 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.668+0000 7f3460bb9700 1 --2- 172.21.15.177:0/3726936084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f345c0fed70 0x7f345c0fff50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3450004f40 tx=0x7f3450005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:41.676 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.668+0000 7f3459ffb700 1 -- 172.21.15.177:0/3726936084 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f345001d070 con 0x7f345c0fed70 2023-11-27T15:14:41.676 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.668+0000 7f3462e1d700 1 -- 172.21.15.177:0/3726936084 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345c10a1b0 con 0x7f345c0fed70 2023-11-27T15:14:41.676 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.668+0000 7f3459ffb700 1 -- 172.21.15.177:0/3726936084 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f345000be20 con 0x7f345c0fed70 2023-11-27T15:14:41.677 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.668+0000 7f3459ffb700 1 -- 172.21.15.177:0/3726936084 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3450022da0 con 0x7f345c0fed70 2023-11-27T15:14:41.677 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.668+0000 7f3462e1d700 1 -- 172.21.15.177:0/3726936084 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f345c10a020 con 0x7f345c0fed70 2023-11-27T15:14:41.677 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.669+0000 7f3459ffb700 1 -- 172.21.15.177:0/3726936084 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f345000f4b0 con 0x7f345c0fed70 2023-11-27T15:14:41.677 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.669+0000 7f3462e1d700 1 -- 172.21.15.177:0/3726936084 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f345c10a340 con 0x7f345c0fed70 2023-11-27T15:14:41.677 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.669+0000 7f3459ffb700 1 --2- 172.21.15.177:0/3726936084 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f344c038850 0x7f344c03ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:41.677 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.670+0000 7f3459ffb700 1 -- 172.21.15.177:0/3726936084 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f345004c8a0 con 0x7f345c0fed70 2023-11-27T15:14:41.677 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.670+0000 7f345bfff700 1 --2- 172.21.15.177:0/3726936084 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f344c038850 0x7f344c03ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:41.677 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.670+0000 7f345bfff700 1 --2- 172.21.15.177:0/3726936084 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f344c038850 0x7f344c03ad00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3448006fd0 tx=0x7f3448006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:41.679 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.675+0000 7f3459ffb700 1 -- 172.21.15.177:0/3726936084 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3450017440 con 0x7f345c0fed70 2023-11-27T15:14:41.924 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:41 smithi177 systemd[1]: Starting Ceph mon.c for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:14:41.959 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.955+0000 7f3462e1d700 1 -- 172.21.15.177:0/3726936084 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f345c1077f0 con 0x7f345c0fed70 2023-11-27T15:14:41.960 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.955+0000 7f3459ffb700 1 -- 172.21.15.177:0/3726936084 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+740 (secure 0 0 0) 0x7f3450027030 con 0x7f345c0fed70 2023-11-27T15:14:41.960 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-27T15:14:41.961 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":1,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","modified":"2023-11-27T15:12:58.471889Z","created":"2023-11-27T15:12:58.471889Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-11-27T15:14:41.963 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.957+0000 7f3462e1d700 1 -- 172.21.15.177:0/3726936084 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f344c038850 msgr2=0x7f344c03ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:41.963 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.957+0000 7f3462e1d700 1 --2- 172.21.15.177:0/3726936084 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f344c038850 0x7f344c03ad00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3448006fd0 tx=0x7f3448006e40 comp rx=0 tx=0).stop 2023-11-27T15:14:41.963 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.957+0000 7f3462e1d700 1 -- 172.21.15.177:0/3726936084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f345c0fed70 msgr2=0x7f345c0fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:41.963 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.957+0000 7f3462e1d700 1 --2- 172.21.15.177:0/3726936084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f345c0fed70 0x7f345c0fff50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3450004f40 tx=0x7f3450005020 comp rx=0 tx=0).stop 2023-11-27T15:14:41.963 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.958+0000 7f3462e1d700 1 -- 172.21.15.177:0/3726936084 shutdown_connections 2023-11-27T15:14:41.963 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.958+0000 7f3462e1d700 1 --2- 172.21.15.177:0/3726936084 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f344c038850 0x7f344c03ad00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:41.964 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.958+0000 7f3462e1d700 1 --2- 172.21.15.177:0/3726936084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f345c0fed70 0x7f345c0fff50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:41.964 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.958+0000 7f3462e1d700 1 -- 172.21.15.177:0/3726936084 >> 172.21.15.177:0/3726936084 conn(0x7f345c074350 msgr2=0x7f345c104930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:41.964 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.958+0000 7f3462e1d700 1 -- 172.21.15.177:0/3726936084 shutdown_connections 2023-11-27T15:14:41.964 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:41.958+0000 7f3462e1d700 1 -- 172.21.15.177:0/3726936084 wait complete. 2023-11-27T15:14:41.964 INFO:teuthology.orchestra.run.smithi177.stderr:dumped monmap epoch 1 2023-11-27T15:14:42.269 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:41 smithi177 podman[131442]: 2023-11-27T15:14:42.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 bash[131442]: c9f577aeeaa0f0b7b66ae83d54d8bb41559d8a0d51b384b530774e8d5a77190e 2023-11-27T15:14:42.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: set uid:gid to 167:167 (ceph:ceph) 2023-11-27T15:14:42.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev), process ceph-mon, pid 7 2023-11-27T15:14:42.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: pidfile_write: ignore empty --pid-file 2023-11-27T15:14:42.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: load: jerasure load: lrc 2023-11-27T15:14:42.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: RocksDB version: 7.9.2 2023-11-27T15:14:42.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Git sha 0 2023-11-27T15:14:42.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Compile date 2023-11-16 19:40:53 2023-11-27T15:14:42.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: DB SUMMARY 2023-11-27T15:14:42.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: DB Session ID: A1TFOVP6J9RXX4S5CR1S 2023-11-27T15:14:42.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: CURRENT file: CURRENT 2023-11-27T15:14:42.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: IDENTITY file: IDENTITY 2023-11-27T15:14:42.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-11-27T15:14:42.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-11-27T15:14:42.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-11-27T15:14:42.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.error_if_exists: 0 2023-11-27T15:14:42.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.create_if_missing: 0 2023-11-27T15:14:42.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.paranoid_checks: 1 2023-11-27T15:14:42.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.flush_verify_memtable_count: 1 2023-11-27T15:14:42.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-11-27T15:14:42.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-11-27T15:14:42.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.env: 0x561445cb4720 2023-11-27T15:14:42.591 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.fs: PosixFileSystem 2023-11-27T15:14:42.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.info_log: 0x561448c393c0 2023-11-27T15:14:42.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_file_opening_threads: 16 2023-11-27T15:14:42.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.statistics: (nil) 2023-11-27T15:14:42.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.use_fsync: 0 2023-11-27T15:14:42.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_log_file_size: 0 2023-11-27T15:14:42.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-11-27T15:14:42.592 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.log_file_time_to_roll: 0 2023-11-27T15:14:42.593 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.keep_log_file_num: 1000 2023-11-27T15:14:42.593 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.recycle_log_file_num: 0 2023-11-27T15:14:42.593 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.allow_fallocate: 1 2023-11-27T15:14:42.593 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.allow_mmap_reads: 0 2023-11-27T15:14:42.593 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.allow_mmap_writes: 0 2023-11-27T15:14:42.593 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.use_direct_reads: 0 2023-11-27T15:14:42.594 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-11-27T15:14:42.594 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.create_missing_column_families: 0 2023-11-27T15:14:42.594 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.db_log_dir: 2023-11-27T15:14:42.594 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.wal_dir: 2023-11-27T15:14:42.596 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.table_cache_numshardbits: 6 2023-11-27T15:14:42.596 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.WAL_ttl_seconds: 0 2023-11-27T15:14:42.596 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.WAL_size_limit_MB: 0 2023-11-27T15:14:42.596 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-11-27T15:14:42.596 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-11-27T15:14:42.597 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.is_fd_close_on_exec: 1 2023-11-27T15:14:42.597 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.advise_random_on_open: 1 2023-11-27T15:14:42.597 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.db_write_buffer_size: 0 2023-11-27T15:14:42.597 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.write_buffer_manager: 0x561447eaa3c0 2023-11-27T15:14:42.597 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-11-27T15:14:42.597 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-11-27T15:14:42.597 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.use_adaptive_mutex: 0 2023-11-27T15:14:42.597 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.rate_limiter: (nil) 2023-11-27T15:14:42.597 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-11-27T15:14:42.597 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.wal_recovery_mode: 2 2023-11-27T15:14:42.598 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.enable_thread_tracking: 0 2023-11-27T15:14:42.598 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.enable_pipelined_write: 0 2023-11-27T15:14:42.598 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.unordered_write: 0 2023-11-27T15:14:42.598 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-11-27T15:14:42.598 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-11-27T15:14:42.598 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-11-27T15:14:42.598 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-11-27T15:14:42.598 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.row_cache: None 2023-11-27T15:14:42.598 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.wal_filter: None 2023-11-27T15:14:42.598 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-11-27T15:14:42.598 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.allow_ingest_behind: 0 2023-11-27T15:14:42.599 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.two_write_queues: 0 2023-11-27T15:14:42.599 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.manual_wal_flush: 0 2023-11-27T15:14:42.599 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.wal_compression: 0 2023-11-27T15:14:42.599 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.atomic_flush: 0 2023-11-27T15:14:42.599 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-11-27T15:14:42.599 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.persist_stats_to_disk: 0 2023-11-27T15:14:42.599 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.write_dbid_to_manifest: 0 2023-11-27T15:14:42.599 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.log_readahead_size: 0 2023-11-27T15:14:42.599 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-11-27T15:14:42.599 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.best_efforts_recovery: 0 2023-11-27T15:14:42.600 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-11-27T15:14:42.600 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-11-27T15:14:42.600 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.allow_data_in_errors: 0 2023-11-27T15:14:42.600 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.db_host_id: __hostname__ 2023-11-27T15:14:42.600 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.enforce_single_del_contracts: true 2023-11-27T15:14:42.600 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_background_jobs: 2 2023-11-27T15:14:42.600 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_background_compactions: -1 2023-11-27T15:14:42.600 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_subcompactions: 1 2023-11-27T15:14:42.600 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-11-27T15:14:42.600 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-11-27T15:14:42.601 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.delayed_write_rate : 16777216 2023-11-27T15:14:42.601 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_total_wal_size: 0 2023-11-27T15:14:42.601 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-11-27T15:14:42.601 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.stats_dump_period_sec: 600 2023-11-27T15:14:42.601 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.stats_persist_period_sec: 600 2023-11-27T15:14:42.601 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-11-27T15:14:42.601 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_open_files: -1 2023-11-27T15:14:42.601 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.bytes_per_sync: 0 2023-11-27T15:14:42.601 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.wal_bytes_per_sync: 0 2023-11-27T15:14:42.601 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.strict_bytes_per_sync: 0 2023-11-27T15:14:42.602 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compaction_readahead_size: 0 2023-11-27T15:14:42.602 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_background_flushes: -1 2023-11-27T15:14:42.602 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Compression algorithms supported: 2023-11-27T15:14:42.602 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-11-27T15:14:42.602 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: kZSTD supported: 0 2023-11-27T15:14:42.602 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: kXpressCompression supported: 0 2023-11-27T15:14:42.602 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: kLZ4HCCompression supported: 1 2023-11-27T15:14:42.602 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: kZlibCompression supported: 1 2023-11-27T15:14:42.602 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: kSnappyCompression supported: 1 2023-11-27T15:14:42.602 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: kLZ4Compression supported: 1 2023-11-27T15:14:42.603 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: kBZip2Compression supported: 0 2023-11-27T15:14:42.603 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Fast CRC32 supported: Supported on x86 2023-11-27T15:14:42.603 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: DMutex implementation: pthread_mutex_t 2023-11-27T15:14:42.603 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-11-27T15:14:42.603 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-11-27T15:14:42.603 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-11-27T15:14:42.603 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.merge_operator: 2023-11-27T15:14:42.603 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compaction_filter: None 2023-11-27T15:14:42.603 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compaction_filter_factory: None 2023-11-27T15:14:42.603 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.sst_partitioner_factory: None 2023-11-27T15:14:42.603 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.memtable_factory: SkipListFactory 2023-11-27T15:14:42.604 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.table_factory: BlockBasedTable 2023-11-27T15:14:42.604 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561448c394e0) 2023-11-27T15:14:42.604 INFO:journalctl@ceph.mon.c.smithi177.stdout: cache_index_and_filter_blocks: 1 2023-11-27T15:14:42.604 INFO:journalctl@ceph.mon.c.smithi177.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-11-27T15:14:42.604 INFO:journalctl@ceph.mon.c.smithi177.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-11-27T15:14:42.604 INFO:journalctl@ceph.mon.c.smithi177.stdout: pin_top_level_index_and_filter: 1 2023-11-27T15:14:42.604 INFO:journalctl@ceph.mon.c.smithi177.stdout: index_type: 0 2023-11-27T15:14:42.604 INFO:journalctl@ceph.mon.c.smithi177.stdout: data_block_index_type: 0 2023-11-27T15:14:42.604 INFO:journalctl@ceph.mon.c.smithi177.stdout: index_shortening: 1 2023-11-27T15:14:42.604 INFO:journalctl@ceph.mon.c.smithi177.stdout: data_block_hash_table_util_ratio: 0.750000 2023-11-27T15:14:42.605 INFO:journalctl@ceph.mon.c.smithi177.stdout: checksum: 4 2023-11-27T15:14:42.605 INFO:journalctl@ceph.mon.c.smithi177.stdout: no_block_cache: 0 2023-11-27T15:14:42.605 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_cache: 0x561447f4b1f0 2023-11-27T15:14:42.605 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_cache_name: BinnedLRUCache 2023-11-27T15:14:42.605 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_cache_options: 2023-11-27T15:14:42.605 INFO:journalctl@ceph.mon.c.smithi177.stdout: capacity : 536870912 2023-11-27T15:14:42.605 INFO:journalctl@ceph.mon.c.smithi177.stdout: num_shard_bits : 4 2023-11-27T15:14:42.605 INFO:journalctl@ceph.mon.c.smithi177.stdout: strict_capacity_limit : 0 2023-11-27T15:14:42.605 INFO:journalctl@ceph.mon.c.smithi177.stdout: high_pri_pool_ratio: 0.000 2023-11-27T15:14:42.605 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_cache_compressed: (nil) 2023-11-27T15:14:42.606 INFO:journalctl@ceph.mon.c.smithi177.stdout: persistent_cache: (nil) 2023-11-27T15:14:42.606 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_size: 4096 2023-11-27T15:14:42.606 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_size_deviation: 10 2023-11-27T15:14:42.606 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_restart_interval: 16 2023-11-27T15:14:42.606 INFO:journalctl@ceph.mon.c.smithi177.stdout: index_block_restart_interval: 1 2023-11-27T15:14:42.606 INFO:journalctl@ceph.mon.c.smithi177.stdout: metadata_block_size: 4096 2023-11-27T15:14:42.606 INFO:journalctl@ceph.mon.c.smithi177.stdout: partition_filters: 0 2023-11-27T15:14:42.606 INFO:journalctl@ceph.mon.c.smithi177.stdout: use_delta_encoding: 1 2023-11-27T15:14:42.606 INFO:journalctl@ceph.mon.c.smithi177.stdout: filter_policy: bloomfilter 2023-11-27T15:14:42.606 INFO:journalctl@ceph.mon.c.smithi177.stdout: whole_key_filtering: 1 2023-11-27T15:14:42.607 INFO:journalctl@ceph.mon.c.smithi177.stdout: verify_compression: 0 2023-11-27T15:14:42.607 INFO:journalctl@ceph.mon.c.smithi177.stdout: read_amp_bytes_per_bit: 0 2023-11-27T15:14:42.607 INFO:journalctl@ceph.mon.c.smithi177.stdout: format_version: 5 2023-11-27T15:14:42.607 INFO:journalctl@ceph.mon.c.smithi177.stdout: enable_index_compression: 1 2023-11-27T15:14:42.607 INFO:journalctl@ceph.mon.c.smithi177.stdout: block_align: 0 2023-11-27T15:14:42.607 INFO:journalctl@ceph.mon.c.smithi177.stdout: max_auto_readahead_size: 262144 2023-11-27T15:14:42.607 INFO:journalctl@ceph.mon.c.smithi177.stdout: prepopulate_block_cache: 0 2023-11-27T15:14:42.607 INFO:journalctl@ceph.mon.c.smithi177.stdout: initial_auto_readahead_size: 8192 2023-11-27T15:14:42.607 INFO:journalctl@ceph.mon.c.smithi177.stdout: num_file_reads_for_auto_readahead: 2 2023-11-27T15:14:42.607 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.write_buffer_size: 33554432 2023-11-27T15:14:42.607 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_write_buffer_number: 2 2023-11-27T15:14:42.608 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compression: NoCompression 2023-11-27T15:14:42.608 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.bottommost_compression: Disabled 2023-11-27T15:14:42.608 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.prefix_extractor: nullptr 2023-11-27T15:14:42.608 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-11-27T15:14:42.608 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.num_levels: 7 2023-11-27T15:14:42.608 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-11-27T15:14:42.608 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-11-27T15:14:42.608 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-11-27T15:14:42.608 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-11-27T15:14:42.608 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-11-27T15:14:42.608 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-11-27T15:14:42.609 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-11-27T15:14:42.609 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-11-27T15:14:42.609 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-11-27T15:14:42.609 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-11-27T15:14:42.609 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-11-27T15:14:42.609 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-11-27T15:14:42.609 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compression_opts.window_bits: -14 2023-11-27T15:14:42.610 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compression_opts.level: 32767 2023-11-27T15:14:42.610 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compression_opts.strategy: 0 2023-11-27T15:14:42.610 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-11-27T15:14:42.610 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-11-27T15:14:42.610 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-11-27T15:14:42.610 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-11-27T15:14:42.610 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compression_opts.enabled: false 2023-11-27T15:14:42.611 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-11-27T15:14:42.611 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-11-27T15:14:42.611 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-11-27T15:14:42.611 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-11-27T15:14:42.611 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.target_file_size_base: 67108864 2023-11-27T15:14:42.611 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.target_file_size_multiplier: 1 2023-11-27T15:14:42.611 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-11-27T15:14:42.611 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-11-27T15:14:42.612 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-11-27T15:14:42.612 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-11-27T15:14:42.612 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-11-27T15:14:42.612 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-11-27T15:14:42.612 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-11-27T15:14:42.612 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-11-27T15:14:42.612 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-11-27T15:14:42.613 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-11-27T15:14:42.613 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-11-27T15:14:42.613 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-11-27T15:14:42.613 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-11-27T15:14:42.613 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.arena_block_size: 1048576 2023-11-27T15:14:42.613 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-11-27T15:14:42.613 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-11-27T15:14:42.613 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.disable_auto_compactions: 0 2023-11-27T15:14:42.614 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-11-27T15:14:42.614 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-11-27T15:14:42.614 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-11-27T15:14:42.614 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-11-27T15:14:42.614 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-11-27T15:14:42.614 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-11-27T15:14:42.614 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-11-27T15:14:42.614 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-11-27T15:14:42.614 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-11-27T15:14:42.614 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-11-27T15:14:42.615 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.table_properties_collectors: 2023-11-27T15:14:42.615 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.inplace_update_support: 0 2023-11-27T15:14:42.615 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.inplace_update_num_locks: 10000 2023-11-27T15:14:42.615 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-11-27T15:14:42.615 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-11-27T15:14:42.615 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.memtable_huge_page_size: 0 2023-11-27T15:14:42.615 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.bloom_locality: 0 2023-11-27T15:14:42.615 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.max_successive_merges: 0 2023-11-27T15:14:42.615 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.optimize_filters_for_hits: 0 2023-11-27T15:14:42.615 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.paranoid_file_checks: 0 2023-11-27T15:14:42.615 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.force_consistency_checks: 1 2023-11-27T15:14:42.616 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.report_bg_io_stats: 0 2023-11-27T15:14:42.616 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.ttl: 2592000 2023-11-27T15:14:42.616 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.periodic_compaction_seconds: 0 2023-11-27T15:14:42.616 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-11-27T15:14:42.616 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-11-27T15:14:42.616 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.enable_blob_files: false 2023-11-27T15:14:42.616 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.min_blob_size: 0 2023-11-27T15:14:42.616 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.blob_file_size: 268435456 2023-11-27T15:14:42.616 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.blob_compression_type: NoCompression 2023-11-27T15:14:42.616 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.enable_blob_garbage_collection: false 2023-11-27T15:14:42.617 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-11-27T15:14:42.617 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-11-27T15:14:42.617 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-11-27T15:14:42.617 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.blob_file_starting_level: 0 2023-11-27T15:14:42.617 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-11-27T15:14:42.617 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-11-27T15:14:42.617 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-11-27T15:14:42.617 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3c2a361a-3d13-4deb-9d15-2c96d9529da2 2023-11-27T15:14:42.617 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701098082277513, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-11-27T15:14:42.617 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-11-27T15:14:42.617 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701098082277917, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1701098082, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3c2a361a-3d13-4deb-9d15-2c96d9529da2", "db_session_id": "A1TFOVP6J9RXX4S5CR1S", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-11-27T15:14:42.618 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701098082277987, "job": 1, "event": "recovery_finished"} 2023-11-27T15:14:42.618 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-11-27T15:14:42.618 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-11-27T15:14:42.618 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561447fe8000 2023-11-27T15:14:42.618 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: DB pointer 0x561447fd4000 2023-11-27T15:14:42.618 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-11-27T15:14:42.618 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-11-27T15:14:42.618 INFO:journalctl@ceph.mon.c.smithi177.stdout: ** DB Stats ** 2023-11-27T15:14:42.618 INFO:journalctl@ceph.mon.c.smithi177.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-27T15:14:42.618 INFO:journalctl@ceph.mon.c.smithi177.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-11-27T15:14:42.619 INFO:journalctl@ceph.mon.c.smithi177.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-27T15:14:42.619 INFO:journalctl@ceph.mon.c.smithi177.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-27T15:14:42.619 INFO:journalctl@ceph.mon.c.smithi177.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-11-27T15:14:42.619 INFO:journalctl@ceph.mon.c.smithi177.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-27T15:14:42.619 INFO:journalctl@ceph.mon.c.smithi177.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-27T15:14:42.619 INFO:journalctl@ceph.mon.c.smithi177.stdout: 2023-11-27T15:14:42.619 INFO:journalctl@ceph.mon.c.smithi177.stdout: ** Compaction Stats [default] ** 2023-11-27T15:14:42.619 INFO:journalctl@ceph.mon.c.smithi177.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-27T15:14:42.619 INFO:journalctl@ceph.mon.c.smithi177.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-11-27T15:14:42.619 INFO:journalctl@ceph.mon.c.smithi177.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-27T15:14:42.619 INFO:journalctl@ceph.mon.c.smithi177.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-27T15:14:42.620 INFO:journalctl@ceph.mon.c.smithi177.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-27T15:14:42.620 INFO:journalctl@ceph.mon.c.smithi177.stdout: 2023-11-27T15:14:42.620 INFO:journalctl@ceph.mon.c.smithi177.stdout: ** Compaction Stats [default] ** 2023-11-27T15:14:42.620 INFO:journalctl@ceph.mon.c.smithi177.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-27T15:14:42.620 INFO:journalctl@ceph.mon.c.smithi177.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-11-27T15:14:42.620 INFO:journalctl@ceph.mon.c.smithi177.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-27T15:14:42.620 INFO:journalctl@ceph.mon.c.smithi177.stdout: 2023-11-27T15:14:42.620 INFO:journalctl@ceph.mon.c.smithi177.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-11-27T15:14:42.620 INFO:journalctl@ceph.mon.c.smithi177.stdout: 2023-11-27T15:14:42.620 INFO:journalctl@ceph.mon.c.smithi177.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-27T15:14:42.621 INFO:journalctl@ceph.mon.c.smithi177.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-11-27T15:14:42.621 INFO:journalctl@ceph.mon.c.smithi177.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-11-27T15:14:42.621 INFO:journalctl@ceph.mon.c.smithi177.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-11-27T15:14:42.621 INFO:journalctl@ceph.mon.c.smithi177.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-11-27T15:14:42.621 INFO:journalctl@ceph.mon.c.smithi177.stdout: AddFile(Keys): cumulative 0, interval 0 2023-11-27T15:14:42.621 INFO:journalctl@ceph.mon.c.smithi177.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-27T15:14:42.621 INFO:journalctl@ceph.mon.c.smithi177.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-27T15:14:42.621 INFO:journalctl@ceph.mon.c.smithi177.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-11-27T15:14:42.621 INFO:journalctl@ceph.mon.c.smithi177.stdout: Block cache BinnedLRUCache@0x561447f4b1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-11-27T15:14:42.621 INFO:journalctl@ceph.mon.c.smithi177.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-11-27T15:14:42.621 INFO:journalctl@ceph.mon.c.smithi177.stdout: 2023-11-27T15:14:42.622 INFO:journalctl@ceph.mon.c.smithi177.stdout: ** File Read Latency Histogram By Level [default] ** 2023-11-27T15:14:42.622 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-11-27T15:14:42.622 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: using public_addr v2:172.21.15.177:0/0 -> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] 2023-11-27T15:14:42.622 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: starting mon.c rank -1 at public addrs [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] at bind addrs [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:14:42.622 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: mon.c@-1(???) e0 preinit fsid 5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:14:42.622 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: mon.c@-1(synchronizing).mds e1 new map 2023-11-27T15:14:42.622 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: mon.c@-1(synchronizing).mds e1 print_map 2023-11-27T15:14:42.622 INFO:journalctl@ceph.mon.c.smithi177.stdout: e1 2023-11-27T15:14:42.622 INFO:journalctl@ceph.mon.c.smithi177.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-11-27T15:14:42.622 INFO:journalctl@ceph.mon.c.smithi177.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-27T15:14:42.622 INFO:journalctl@ceph.mon.c.smithi177.stdout: legacy client fscid: -1 2023-11-27T15:14:42.623 INFO:journalctl@ceph.mon.c.smithi177.stdout: 2023-11-27T15:14:42.623 INFO:journalctl@ceph.mon.c.smithi177.stdout: No filesystems configured 2023-11-27T15:14:42.623 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-11-27T15:14:42.623 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-11-27T15:14:42.623 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-11-27T15:14:42.623 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-11-27T15:14:42.623 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-11-27T15:14:42.623 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-11-27T15:14:42.623 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-11-27T15:14:42.623 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:14:42.624 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:14:42.624 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:14:42.624 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:14:42.624 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:42.624 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:42.624 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:42.624 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:42.624 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:42.624 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:42.624 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:42.624 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:42.625 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:14:42.625 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: Updating smithi174:/etc/ceph/ceph.conf 2023-11-27T15:14:42.625 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:42.625 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:42.625 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:42.625 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: Updating smithi174:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:14:42.625 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:14:42.625 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:42.625 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: Updating smithi174:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:14:42.625 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi177", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:14:42.626 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: Deploying cephadm binary to smithi177 2023-11-27T15:14:42.626 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:42.626 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:42.626 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:42.626 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: Added host smithi177 2023-11-27T15:14:42.626 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:42.626 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:42.626 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:42.626 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:42.626 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:14:42.627 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:42.627 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:42.627 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:42.627 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:42.627 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:42.627 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi177", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:42.627 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:42.627 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:42.627 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:14:42.627 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: Updating smithi177:/etc/ceph/ceph.conf 2023-11-27T15:14:42.627 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: Updating smithi177:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:14:42.628 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:42.628 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: Updating smithi177:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:14:42.628 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/693972224' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-11-27T15:14:42.628 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: Updating smithi177:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:14:42.628 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:42.628 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:42.628 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:42.629 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/693972224' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-11-27T15:14:42.629 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:14:42.629 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:42.629 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi133:172.21.15.133=a;smithi174:172.21.15.174=b;smithi177:172.21.15.177=c", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:14:42.629 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: Saving service mon spec with placement smithi133:172.21.15.133=a;smithi174:172.21.15.174=b;smithi177:172.21.15.177=c;count:3 2023-11-27T15:14:42.629 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:42.629 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:42.630 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:42.630 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:42.630 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:14:42.630 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:42.630 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:14:42.630 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:14:42.630 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:42.630 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: Deploying daemon mon.c on smithi177 2023-11-27T15:14:42.631 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:42.631 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-11-27T15:14:42.631 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-11-27T15:14:42.631 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: expand_channel_meta expand map: {default=false} 2023-11-27T15:14:42.631 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: expand_channel_meta from 'false' to 'false' 2023-11-27T15:14:42.631 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: expand_channel_meta expanded map: {default=false} 2023-11-27T15:14:42.631 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: expand_channel_meta expand map: {default=info} 2023-11-27T15:14:42.632 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: expand_channel_meta from 'info' to 'info' 2023-11-27T15:14:42.632 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: expand_channel_meta expanded map: {default=info} 2023-11-27T15:14:42.632 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: expand_channel_meta expand map: {default=daemon} 2023-11-27T15:14:42.632 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: expand_channel_meta from 'daemon' to 'daemon' 2023-11-27T15:14:42.632 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: expand_channel_meta expanded map: {default=daemon} 2023-11-27T15:14:42.632 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: expand_channel_meta expand map: {default=debug} 2023-11-27T15:14:42.632 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: expand_channel_meta from 'debug' to 'debug' 2023-11-27T15:14:42.633 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 ceph-mon[131504]: expand_channel_meta expanded map: {default=debug} 2023-11-27T15:14:42.633 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:42 smithi177 systemd[1]: Started Ceph mon.c for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:14:43.369 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-11-27T15:14:43.370 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-11-27T15:14:44.023 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:14:47.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:14:47.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:47.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: mon.a calling monitor election 2023-11-27T15:14:47.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:47.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:47.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:47.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: mon.c calling monitor election 2023-11-27T15:14:47.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:47.780 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:47.780 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:47.780 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:47.780 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-27T15:14:47.780 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: monmap e2: 2 mons at {a=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0],c=[v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0]} removed_ranks: {} 2023-11-27T15:14:47.780 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: fsmap 2023-11-27T15:14:47.780 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:14:47.780 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: mgrmap e13: x(active, since 45s) 2023-11-27T15:14:47.780 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: overall HEALTH_OK 2023-11-27T15:14:47.781 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:47.781 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:47.781 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:47.781 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:14:47.781 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:47.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:14:47.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:47.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: mon.a calling monitor election 2023-11-27T15:14:47.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:47.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:47.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:47.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: mon.c calling monitor election 2023-11-27T15:14:47.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:47.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:47.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:47.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:47.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-27T15:14:47.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: monmap e2: 2 mons at {a=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0],c=[v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0]} removed_ranks: {} 2023-11-27T15:14:47.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: fsmap 2023-11-27T15:14:47.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:14:47.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: mgrmap e13: x(active, since 45s) 2023-11-27T15:14:47.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: overall HEALTH_OK 2023-11-27T15:14:47.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:47.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:47.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:47.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:14:47.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:48.378 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.370+0000 7f51cf30b700 1 -- 172.21.15.177:0/1549491200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51b40035c0 msgr2=0x7f51b4005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:48.378 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.370+0000 7f51cf30b700 1 --2- 172.21.15.177:0/1549491200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51b40035c0 0x7f51b4005a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f51c80fe4c0 tx=0x7f51bc010ec0 comp rx=0 tx=0).stop 2023-11-27T15:14:48.378 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.370+0000 7f51cf30b700 1 -- 172.21.15.177:0/1549491200 shutdown_connections 2023-11-27T15:14:48.379 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.370+0000 7f51cf30b700 1 --2- 172.21.15.177:0/1549491200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51b40035c0 0x7f51b4005a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:48.379 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.370+0000 7f51cf30b700 1 -- 172.21.15.177:0/1549491200 >> 172.21.15.177:0/1549491200 conn(0x7f51c80742a0 msgr2=0x7f51c80746a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:48.379 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.370+0000 7f51cf30b700 1 -- 172.21.15.177:0/1549491200 shutdown_connections 2023-11-27T15:14:48.379 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.370+0000 7f51cf30b700 1 -- 172.21.15.177:0/1549491200 wait complete. 2023-11-27T15:14:48.379 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.371+0000 7f51cf30b700 1 Processor -- start 2023-11-27T15:14:48.379 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.371+0000 7f51cf30b700 1 -- start start 2023-11-27T15:14:48.379 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.371+0000 7f51cf30b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51b40035c0 0x7f51c8100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:48.380 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.371+0000 7f51cf30b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c8100f60 0x7f51c81013d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:48.380 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.371+0000 7f51cf30b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c81029a0 con 0x7f51c8100f60 2023-11-27T15:14:48.380 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.372+0000 7f51cf30b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c81a13c0 con 0x7f51b40035c0 2023-11-27T15:14:48.380 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.372+0000 7f51cd0a7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51b40035c0 0x7f51c8100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:48.380 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.372+0000 7f51cd0a7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51b40035c0 0x7f51c8100a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:57756/0 (socket says 172.21.15.177:57756) 2023-11-27T15:14:48.380 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.372+0000 7f51cd0a7700 1 -- 172.21.15.177:0/2416201319 learned_addr learned my addr 172.21.15.177:0/2416201319 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-27T15:14:48.380 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.372+0000 7f51cd0a7700 1 -- 172.21.15.177:0/2416201319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51b40035c0 msgr2=0x7f51c8100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-11-27T15:14:48.380 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.372+0000 7f51cd0a7700 1 -- 172.21.15.177:0/2416201319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51b40035c0 msgr2=0x7f51c8100a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-27T15:14:48.381 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.372+0000 7f51cd0a7700 1 --2- 172.21.15.177:0/2416201319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51b40035c0 0x7f51c8100a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-27T15:14:48.381 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.372+0000 7f51cd0a7700 1 --2- 172.21.15.177:0/2416201319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51b40035c0 0x7f51c8100a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:14:48.381 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.372+0000 7f51cc8a6700 1 --2- 172.21.15.177:0/2416201319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c8100f60 0x7f51c81013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:48.381 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.372+0000 7f51cc8a6700 1 -- 172.21.15.177:0/2416201319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51b40035c0 msgr2=0x7f51c8100a20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:14:48.381 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.372+0000 7f51cc8a6700 1 --2- 172.21.15.177:0/2416201319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51b40035c0 0x7f51c8100a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:48.381 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.372+0000 7f51cc8a6700 1 -- 172.21.15.177:0/2416201319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc010910 con 0x7f51c8100f60 2023-11-27T15:14:48.381 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.373+0000 7f51cc8a6700 1 --2- 172.21.15.177:0/2416201319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c8100f60 0x7f51c81013d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f51c400b4d0 tx=0x7f51c400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:48.381 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.373+0000 7f51ba7fc700 1 -- 172.21.15.177:0/2416201319 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f51c4011630 con 0x7f51c8100f60 2023-11-27T15:14:48.382 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.373+0000 7f51cf30b700 1 -- 172.21.15.177:0/2416201319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c8105ba0 con 0x7f51c8100f60 2023-11-27T15:14:48.382 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.373+0000 7f51ba7fc700 1 -- 172.21.15.177:0/2416201319 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51c4011c50 con 0x7f51c8100f60 2023-11-27T15:14:48.382 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.373+0000 7f51ba7fc700 1 -- 172.21.15.177:0/2416201319 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f51c4010d90 con 0x7f51c8100f60 2023-11-27T15:14:48.382 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.373+0000 7f51cf30b700 1 -- 172.21.15.177:0/2416201319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c8105a10 con 0x7f51c8100f60 2023-11-27T15:14:48.383 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.374+0000 7f51ba7fc700 1 -- 172.21.15.177:0/2416201319 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f51c401a400 con 0x7f51c8100f60 2023-11-27T15:14:48.383 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.374+0000 7f51cf30b700 1 -- 172.21.15.177:0/2416201319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c8105d30 con 0x7f51c8100f60 2023-11-27T15:14:48.383 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.374+0000 7f51ba7fc700 1 --2- 172.21.15.177:0/2416201319 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f51ac0386b0 0x7f51ac03ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:48.384 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.374+0000 7f51ba7fc700 1 -- 172.21.15.177:0/2416201319 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f51c404ad60 con 0x7f51c8100f60 2023-11-27T15:14:48.384 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.374+0000 7f51cd0a7700 1 --2- 172.21.15.177:0/2416201319 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f51ac0386b0 0x7f51ac03ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:48.384 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.375+0000 7f51cd0a7700 1 --2- 172.21.15.177:0/2416201319 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f51ac0386b0 0x7f51ac03ab60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f51bc010480 tx=0x7f51bc010410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:48.385 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.380+0000 7f51ba7fc700 1 -- 172.21.15.177:0/2416201319 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f51c4017680 con 0x7f51c8100f60 2023-11-27T15:14:48.648 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.643+0000 7f51cf30b700 1 -- 172.21.15.177:0/2416201319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f51c8101f50 con 0x7f51c8100f60 2023-11-27T15:14:48.649 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.644+0000 7f51ba7fc700 1 -- 172.21.15.177:0/2416201319 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1010 (secure 0 0 0) 0x7f51c40172c0 con 0x7f51c8100f60 2023-11-27T15:14:48.649 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-27T15:14:48.649 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":2,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","modified":"2023-11-27T15:14:42.308249Z","created":"2023-11-27T15:12:58.471889Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:3300","nonce":0},{"type":"v1","addr":"172.21.15.177:6789","nonce":0}]},"addr":"172.21.15.177:6789/0","public_addr":"172.21.15.177:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-11-27T15:14:48.651 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.646+0000 7f51cf30b700 1 -- 172.21.15.177:0/2416201319 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f51ac0386b0 msgr2=0x7f51ac03ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:48.651 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.646+0000 7f51cf30b700 1 --2- 172.21.15.177:0/2416201319 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f51ac0386b0 0x7f51ac03ab60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f51bc010480 tx=0x7f51bc010410 comp rx=0 tx=0).stop 2023-11-27T15:14:48.651 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.646+0000 7f51cf30b700 1 -- 172.21.15.177:0/2416201319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c8100f60 msgr2=0x7f51c81013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:48.651 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.646+0000 7f51cf30b700 1 --2- 172.21.15.177:0/2416201319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c8100f60 0x7f51c81013d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f51c400b4d0 tx=0x7f51c400b890 comp rx=0 tx=0).stop 2023-11-27T15:14:48.651 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.646+0000 7f51cf30b700 1 -- 172.21.15.177:0/2416201319 shutdown_connections 2023-11-27T15:14:48.651 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.646+0000 7f51cf30b700 1 --2- 172.21.15.177:0/2416201319 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f51ac0386b0 0x7f51ac03ab60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:48.652 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.646+0000 7f51cf30b700 1 --2- 172.21.15.177:0/2416201319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51b40035c0 0x7f51c8100a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:48.652 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.646+0000 7f51cf30b700 1 --2- 172.21.15.177:0/2416201319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51c8100f60 0x7f51c81013d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:48.652 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.646+0000 7f51cf30b700 1 -- 172.21.15.177:0/2416201319 >> 172.21.15.177:0/2416201319 conn(0x7f51c80742a0 msgr2=0x7f51c8102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:48.652 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.646+0000 7f51cf30b700 1 -- 172.21.15.177:0/2416201319 shutdown_connections 2023-11-27T15:14:48.652 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:48.646+0000 7f51cf30b700 1 -- 172.21.15.177:0/2416201319 wait complete. 2023-11-27T15:14:48.652 INFO:teuthology.orchestra.run.smithi177.stderr:dumped monmap epoch 2 2023-11-27T15:14:48.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:48 smithi177 ceph-mon[131504]: Deploying daemon mon.b on smithi174 2023-11-27T15:14:48.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:48 smithi177 ceph-mon[131504]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:48.683 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:48 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:48.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:48 smithi133 ceph-mon[97860]: Deploying daemon mon.b on smithi174 2023-11-27T15:14:48.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:48 smithi133 ceph-mon[97860]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:48.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:48 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:49.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:49 smithi133 ceph-mon[97860]: from='client.? 172.21.15.177:0/2416201319' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-27T15:14:49.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:49 smithi177 ceph-mon[131504]: from='client.? 172.21.15.177:0/2416201319' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-27T15:14:50.076 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-11-27T15:14:50.076 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph mon dump -f json 2023-11-27T15:14:50.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:49 smithi174 systemd[1]: Starting Ceph mon.b for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:14:50.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 podman[131070]: 2023-11-27T15:14:50.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 bash[131070]: 19b907d479d93f5fc141b42e05220c968dba2d801f867363cd60a6f8f1ad5e1d 2023-11-27T15:14:50.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: set uid:gid to 167:167 (ceph:ceph) 2023-11-27T15:14:50.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev), process ceph-mon, pid 7 2023-11-27T15:14:50.491 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: pidfile_write: ignore empty --pid-file 2023-11-27T15:14:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: load: jerasure load: lrc 2023-11-27T15:14:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: RocksDB version: 7.9.2 2023-11-27T15:14:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Git sha 0 2023-11-27T15:14:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Compile date 2023-11-16 19:40:53 2023-11-27T15:14:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: DB SUMMARY 2023-11-27T15:14:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: DB Session ID: 0H6P10YVT2F0ITD4UIUR 2023-11-27T15:14:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: CURRENT file: CURRENT 2023-11-27T15:14:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: IDENTITY file: IDENTITY 2023-11-27T15:14:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-11-27T15:14:50.492 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-11-27T15:14:50.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-11-27T15:14:50.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.error_if_exists: 0 2023-11-27T15:14:50.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.create_if_missing: 0 2023-11-27T15:14:50.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.paranoid_checks: 1 2023-11-27T15:14:50.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.flush_verify_memtable_count: 1 2023-11-27T15:14:50.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-11-27T15:14:50.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-11-27T15:14:50.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.env: 0x557007ce5720 2023-11-27T15:14:50.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.fs: PosixFileSystem 2023-11-27T15:14:50.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.info_log: 0x55700a2db3c0 2023-11-27T15:14:50.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_file_opening_threads: 16 2023-11-27T15:14:50.493 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.statistics: (nil) 2023-11-27T15:14:50.494 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.use_fsync: 0 2023-11-27T15:14:50.494 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_log_file_size: 0 2023-11-27T15:14:50.494 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-11-27T15:14:50.494 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.log_file_time_to_roll: 0 2023-11-27T15:14:50.494 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.keep_log_file_num: 1000 2023-11-27T15:14:50.494 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.recycle_log_file_num: 0 2023-11-27T15:14:50.494 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.allow_fallocate: 1 2023-11-27T15:14:50.494 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.allow_mmap_reads: 0 2023-11-27T15:14:50.494 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.allow_mmap_writes: 0 2023-11-27T15:14:50.494 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.use_direct_reads: 0 2023-11-27T15:14:50.494 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-11-27T15:14:50.494 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.create_missing_column_families: 0 2023-11-27T15:14:50.495 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.db_log_dir: 2023-11-27T15:14:50.495 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.wal_dir: 2023-11-27T15:14:50.495 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.table_cache_numshardbits: 6 2023-11-27T15:14:50.495 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.WAL_ttl_seconds: 0 2023-11-27T15:14:50.495 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.WAL_size_limit_MB: 0 2023-11-27T15:14:50.495 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-11-27T15:14:50.495 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-11-27T15:14:50.495 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.is_fd_close_on_exec: 1 2023-11-27T15:14:50.495 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.advise_random_on_open: 1 2023-11-27T15:14:50.495 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.db_write_buffer_size: 0 2023-11-27T15:14:50.495 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.write_buffer_manager: 0x55700954c3c0 2023-11-27T15:14:50.495 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-11-27T15:14:50.496 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-11-27T15:14:50.496 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.use_adaptive_mutex: 0 2023-11-27T15:14:50.496 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.rate_limiter: (nil) 2023-11-27T15:14:50.496 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-11-27T15:14:50.496 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.wal_recovery_mode: 2 2023-11-27T15:14:50.496 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.enable_thread_tracking: 0 2023-11-27T15:14:50.496 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.enable_pipelined_write: 0 2023-11-27T15:14:50.496 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.unordered_write: 0 2023-11-27T15:14:50.496 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-11-27T15:14:50.496 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-11-27T15:14:50.496 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-11-27T15:14:50.496 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-11-27T15:14:50.497 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.row_cache: None 2023-11-27T15:14:50.497 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.wal_filter: None 2023-11-27T15:14:50.497 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-11-27T15:14:50.497 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.allow_ingest_behind: 0 2023-11-27T15:14:50.497 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.two_write_queues: 0 2023-11-27T15:14:50.497 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.manual_wal_flush: 0 2023-11-27T15:14:50.497 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.wal_compression: 0 2023-11-27T15:14:50.497 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.atomic_flush: 0 2023-11-27T15:14:50.497 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-11-27T15:14:50.497 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.persist_stats_to_disk: 0 2023-11-27T15:14:50.497 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.write_dbid_to_manifest: 0 2023-11-27T15:14:50.497 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.log_readahead_size: 0 2023-11-27T15:14:50.497 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-11-27T15:14:50.497 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.best_efforts_recovery: 0 2023-11-27T15:14:50.498 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-11-27T15:14:50.498 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-11-27T15:14:50.498 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.allow_data_in_errors: 0 2023-11-27T15:14:50.498 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.db_host_id: __hostname__ 2023-11-27T15:14:50.498 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.enforce_single_del_contracts: true 2023-11-27T15:14:50.498 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_background_jobs: 2 2023-11-27T15:14:50.498 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_background_compactions: -1 2023-11-27T15:14:50.498 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_subcompactions: 1 2023-11-27T15:14:50.498 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-11-27T15:14:50.498 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-11-27T15:14:50.498 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.delayed_write_rate : 16777216 2023-11-27T15:14:50.499 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_total_wal_size: 0 2023-11-27T15:14:50.499 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-11-27T15:14:50.499 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.stats_dump_period_sec: 600 2023-11-27T15:14:50.499 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.stats_persist_period_sec: 600 2023-11-27T15:14:50.499 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-11-27T15:14:50.499 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_open_files: -1 2023-11-27T15:14:50.499 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.bytes_per_sync: 0 2023-11-27T15:14:50.499 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.wal_bytes_per_sync: 0 2023-11-27T15:14:50.499 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.strict_bytes_per_sync: 0 2023-11-27T15:14:50.499 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compaction_readahead_size: 0 2023-11-27T15:14:50.499 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_background_flushes: -1 2023-11-27T15:14:50.499 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Compression algorithms supported: 2023-11-27T15:14:50.500 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-11-27T15:14:50.500 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: kZSTD supported: 0 2023-11-27T15:14:50.500 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: kXpressCompression supported: 0 2023-11-27T15:14:50.500 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: kLZ4HCCompression supported: 1 2023-11-27T15:14:50.500 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: kZlibCompression supported: 1 2023-11-27T15:14:50.500 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: kSnappyCompression supported: 1 2023-11-27T15:14:50.500 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: kLZ4Compression supported: 1 2023-11-27T15:14:50.500 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: kBZip2Compression supported: 0 2023-11-27T15:14:50.500 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Fast CRC32 supported: Supported on x86 2023-11-27T15:14:50.500 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: DMutex implementation: pthread_mutex_t 2023-11-27T15:14:50.500 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-11-27T15:14:50.500 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-11-27T15:14:50.501 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-11-27T15:14:50.501 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.merge_operator: 2023-11-27T15:14:50.501 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compaction_filter: None 2023-11-27T15:14:50.501 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compaction_filter_factory: None 2023-11-27T15:14:50.501 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.sst_partitioner_factory: None 2023-11-27T15:14:50.501 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.memtable_factory: SkipListFactory 2023-11-27T15:14:50.501 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.table_factory: BlockBasedTable 2023-11-27T15:14:50.501 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55700a2db4e0) 2023-11-27T15:14:50.501 INFO:journalctl@ceph.mon.b.smithi174.stdout: cache_index_and_filter_blocks: 1 2023-11-27T15:14:50.501 INFO:journalctl@ceph.mon.b.smithi174.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-11-27T15:14:50.501 INFO:journalctl@ceph.mon.b.smithi174.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-11-27T15:14:50.501 INFO:journalctl@ceph.mon.b.smithi174.stdout: pin_top_level_index_and_filter: 1 2023-11-27T15:14:50.502 INFO:journalctl@ceph.mon.b.smithi174.stdout: index_type: 0 2023-11-27T15:14:50.502 INFO:journalctl@ceph.mon.b.smithi174.stdout: data_block_index_type: 0 2023-11-27T15:14:50.502 INFO:journalctl@ceph.mon.b.smithi174.stdout: index_shortening: 1 2023-11-27T15:14:50.502 INFO:journalctl@ceph.mon.b.smithi174.stdout: data_block_hash_table_util_ratio: 0.750000 2023-11-27T15:14:50.502 INFO:journalctl@ceph.mon.b.smithi174.stdout: checksum: 4 2023-11-27T15:14:50.502 INFO:journalctl@ceph.mon.b.smithi174.stdout: no_block_cache: 0 2023-11-27T15:14:50.502 INFO:journalctl@ceph.mon.b.smithi174.stdout: block_cache: 0x5570095ed1f0 2023-11-27T15:14:50.502 INFO:journalctl@ceph.mon.b.smithi174.stdout: block_cache_name: BinnedLRUCache 2023-11-27T15:14:50.502 INFO:journalctl@ceph.mon.b.smithi174.stdout: block_cache_options: 2023-11-27T15:14:50.502 INFO:journalctl@ceph.mon.b.smithi174.stdout: capacity : 536870912 2023-11-27T15:14:50.502 INFO:journalctl@ceph.mon.b.smithi174.stdout: num_shard_bits : 4 2023-11-27T15:14:50.502 INFO:journalctl@ceph.mon.b.smithi174.stdout: strict_capacity_limit : 0 2023-11-27T15:14:50.503 INFO:journalctl@ceph.mon.b.smithi174.stdout: high_pri_pool_ratio: 0.000 2023-11-27T15:14:50.503 INFO:journalctl@ceph.mon.b.smithi174.stdout: block_cache_compressed: (nil) 2023-11-27T15:14:50.503 INFO:journalctl@ceph.mon.b.smithi174.stdout: persistent_cache: (nil) 2023-11-27T15:14:50.503 INFO:journalctl@ceph.mon.b.smithi174.stdout: block_size: 4096 2023-11-27T15:14:50.503 INFO:journalctl@ceph.mon.b.smithi174.stdout: block_size_deviation: 10 2023-11-27T15:14:50.503 INFO:journalctl@ceph.mon.b.smithi174.stdout: block_restart_interval: 16 2023-11-27T15:14:50.503 INFO:journalctl@ceph.mon.b.smithi174.stdout: index_block_restart_interval: 1 2023-11-27T15:14:50.503 INFO:journalctl@ceph.mon.b.smithi174.stdout: metadata_block_size: 4096 2023-11-27T15:14:50.503 INFO:journalctl@ceph.mon.b.smithi174.stdout: partition_filters: 0 2023-11-27T15:14:50.503 INFO:journalctl@ceph.mon.b.smithi174.stdout: use_delta_encoding: 1 2023-11-27T15:14:50.503 INFO:journalctl@ceph.mon.b.smithi174.stdout: filter_policy: bloomfilter 2023-11-27T15:14:50.503 INFO:journalctl@ceph.mon.b.smithi174.stdout: whole_key_filtering: 1 2023-11-27T15:14:50.503 INFO:journalctl@ceph.mon.b.smithi174.stdout: verify_compression: 0 2023-11-27T15:14:50.504 INFO:journalctl@ceph.mon.b.smithi174.stdout: read_amp_bytes_per_bit: 0 2023-11-27T15:14:50.504 INFO:journalctl@ceph.mon.b.smithi174.stdout: format_version: 5 2023-11-27T15:14:50.504 INFO:journalctl@ceph.mon.b.smithi174.stdout: enable_index_compression: 1 2023-11-27T15:14:50.504 INFO:journalctl@ceph.mon.b.smithi174.stdout: block_align: 0 2023-11-27T15:14:50.504 INFO:journalctl@ceph.mon.b.smithi174.stdout: max_auto_readahead_size: 262144 2023-11-27T15:14:50.504 INFO:journalctl@ceph.mon.b.smithi174.stdout: prepopulate_block_cache: 0 2023-11-27T15:14:50.504 INFO:journalctl@ceph.mon.b.smithi174.stdout: initial_auto_readahead_size: 8192 2023-11-27T15:14:50.504 INFO:journalctl@ceph.mon.b.smithi174.stdout: num_file_reads_for_auto_readahead: 2 2023-11-27T15:14:50.504 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.write_buffer_size: 33554432 2023-11-27T15:14:50.504 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_write_buffer_number: 2 2023-11-27T15:14:50.504 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compression: NoCompression 2023-11-27T15:14:50.504 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.bottommost_compression: Disabled 2023-11-27T15:14:50.505 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.prefix_extractor: nullptr 2023-11-27T15:14:50.505 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-11-27T15:14:50.505 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.num_levels: 7 2023-11-27T15:14:50.505 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-11-27T15:14:50.505 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-11-27T15:14:50.505 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-11-27T15:14:50.505 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-11-27T15:14:50.505 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-11-27T15:14:50.505 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-11-27T15:14:50.505 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-11-27T15:14:50.505 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-11-27T15:14:50.505 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-11-27T15:14:50.506 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-11-27T15:14:50.506 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-11-27T15:14:50.506 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-11-27T15:14:50.506 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compression_opts.window_bits: -14 2023-11-27T15:14:50.506 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compression_opts.level: 32767 2023-11-27T15:14:50.506 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compression_opts.strategy: 0 2023-11-27T15:14:50.506 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-11-27T15:14:50.506 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-11-27T15:14:50.506 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-11-27T15:14:50.506 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-11-27T15:14:50.506 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compression_opts.enabled: false 2023-11-27T15:14:50.506 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-11-27T15:14:50.507 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-11-27T15:14:50.507 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-11-27T15:14:50.507 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-11-27T15:14:50.507 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.target_file_size_base: 67108864 2023-11-27T15:14:50.507 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.target_file_size_multiplier: 1 2023-11-27T15:14:50.507 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-11-27T15:14:50.507 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-11-27T15:14:50.507 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-11-27T15:14:50.507 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-11-27T15:14:50.507 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-11-27T15:14:50.507 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-11-27T15:14:50.507 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-11-27T15:14:50.508 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-11-27T15:14:50.508 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-11-27T15:14:50.508 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-11-27T15:14:50.508 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-11-27T15:14:50.508 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-11-27T15:14:50.508 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-11-27T15:14:50.508 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.arena_block_size: 1048576 2023-11-27T15:14:50.508 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-11-27T15:14:50.508 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-11-27T15:14:50.508 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.disable_auto_compactions: 0 2023-11-27T15:14:50.509 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-11-27T15:14:50.509 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-11-27T15:14:50.510 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-11-27T15:14:50.510 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-11-27T15:14:50.510 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-11-27T15:14:50.510 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-11-27T15:14:50.510 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-11-27T15:14:50.510 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-11-27T15:14:50.510 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-11-27T15:14:50.510 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-11-27T15:14:50.510 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.table_properties_collectors: 2023-11-27T15:14:50.510 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.inplace_update_support: 0 2023-11-27T15:14:50.510 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.inplace_update_num_locks: 10000 2023-11-27T15:14:50.511 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-11-27T15:14:50.511 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-11-27T15:14:50.511 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.memtable_huge_page_size: 0 2023-11-27T15:14:50.511 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.bloom_locality: 0 2023-11-27T15:14:50.511 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.max_successive_merges: 0 2023-11-27T15:14:50.511 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.optimize_filters_for_hits: 0 2023-11-27T15:14:50.511 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.paranoid_file_checks: 0 2023-11-27T15:14:50.511 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.force_consistency_checks: 1 2023-11-27T15:14:50.511 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.report_bg_io_stats: 0 2023-11-27T15:14:50.511 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.ttl: 2592000 2023-11-27T15:14:50.511 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.periodic_compaction_seconds: 0 2023-11-27T15:14:50.511 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-11-27T15:14:50.512 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-11-27T15:14:50.512 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.enable_blob_files: false 2023-11-27T15:14:50.512 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.min_blob_size: 0 2023-11-27T15:14:50.512 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.blob_file_size: 268435456 2023-11-27T15:14:50.512 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.blob_compression_type: NoCompression 2023-11-27T15:14:50.512 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.enable_blob_garbage_collection: false 2023-11-27T15:14:50.512 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-11-27T15:14:50.512 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-11-27T15:14:50.512 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-11-27T15:14:50.512 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.blob_file_starting_level: 0 2023-11-27T15:14:50.512 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-11-27T15:14:50.512 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-11-27T15:14:50.512 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-11-27T15:14:50.513 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 77288422-617f-4156-accb-33cf56bb17ec 2023-11-27T15:14:50.513 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701098090416869, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-11-27T15:14:50.513 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-11-27T15:14:50.513 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701098090417278, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1701098090, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "77288422-617f-4156-accb-33cf56bb17ec", "db_session_id": "0H6P10YVT2F0ITD4UIUR", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-11-27T15:14:50.513 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: EVENT_LOG_v1 {"time_micros": 1701098090417354, "job": 1, "event": "recovery_finished"} 2023-11-27T15:14:50.513 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-11-27T15:14:50.513 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-11-27T15:14:50.513 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55700968a000 2023-11-27T15:14:50.513 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: DB pointer 0x557009676000 2023-11-27T15:14:50.513 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-11-27T15:14:50.513 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-11-27T15:14:50.514 INFO:journalctl@ceph.mon.b.smithi174.stdout: ** DB Stats ** 2023-11-27T15:14:50.514 INFO:journalctl@ceph.mon.b.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-27T15:14:50.514 INFO:journalctl@ceph.mon.b.smithi174.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-11-27T15:14:50.514 INFO:journalctl@ceph.mon.b.smithi174.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-27T15:14:50.514 INFO:journalctl@ceph.mon.b.smithi174.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-27T15:14:50.514 INFO:journalctl@ceph.mon.b.smithi174.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-11-27T15:14:50.514 INFO:journalctl@ceph.mon.b.smithi174.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-11-27T15:14:50.514 INFO:journalctl@ceph.mon.b.smithi174.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-11-27T15:14:50.514 INFO:journalctl@ceph.mon.b.smithi174.stdout: 2023-11-27T15:14:50.514 INFO:journalctl@ceph.mon.b.smithi174.stdout: ** Compaction Stats [default] ** 2023-11-27T15:14:50.514 INFO:journalctl@ceph.mon.b.smithi174.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-27T15:14:50.514 INFO:journalctl@ceph.mon.b.smithi174.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-11-27T15:14:50.515 INFO:journalctl@ceph.mon.b.smithi174.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-27T15:14:50.515 INFO:journalctl@ceph.mon.b.smithi174.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-27T15:14:50.515 INFO:journalctl@ceph.mon.b.smithi174.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-27T15:14:50.515 INFO:journalctl@ceph.mon.b.smithi174.stdout: 2023-11-27T15:14:50.515 INFO:journalctl@ceph.mon.b.smithi174.stdout: ** Compaction Stats [default] ** 2023-11-27T15:14:50.515 INFO:journalctl@ceph.mon.b.smithi174.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-11-27T15:14:50.515 INFO:journalctl@ceph.mon.b.smithi174.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-11-27T15:14:50.515 INFO:journalctl@ceph.mon.b.smithi174.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-11-27T15:14:50.515 INFO:journalctl@ceph.mon.b.smithi174.stdout: 2023-11-27T15:14:50.515 INFO:journalctl@ceph.mon.b.smithi174.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-11-27T15:14:50.515 INFO:journalctl@ceph.mon.b.smithi174.stdout: 2023-11-27T15:14:50.515 INFO:journalctl@ceph.mon.b.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-11-27T15:14:50.516 INFO:journalctl@ceph.mon.b.smithi174.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-11-27T15:14:50.516 INFO:journalctl@ceph.mon.b.smithi174.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-11-27T15:14:50.516 INFO:journalctl@ceph.mon.b.smithi174.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-11-27T15:14:50.516 INFO:journalctl@ceph.mon.b.smithi174.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-11-27T15:14:50.516 INFO:journalctl@ceph.mon.b.smithi174.stdout: AddFile(Keys): cumulative 0, interval 0 2023-11-27T15:14:50.516 INFO:journalctl@ceph.mon.b.smithi174.stdout: Cumulative compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-27T15:14:50.516 INFO:journalctl@ceph.mon.b.smithi174.stdout: Interval compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-11-27T15:14:50.516 INFO:journalctl@ceph.mon.b.smithi174.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-11-27T15:14:50.516 INFO:journalctl@ceph.mon.b.smithi174.stdout: Block cache BinnedLRUCache@0x5570095ed1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-11-27T15:14:50.516 INFO:journalctl@ceph.mon.b.smithi174.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-11-27T15:14:50.516 INFO:journalctl@ceph.mon.b.smithi174.stdout: 2023-11-27T15:14:50.516 INFO:journalctl@ceph.mon.b.smithi174.stdout: ** File Read Latency Histogram By Level [default] ** 2023-11-27T15:14:50.517 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-11-27T15:14:50.517 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: using public_addr v2:172.21.15.174:0/0 -> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2023-11-27T15:14:50.517 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: starting mon.b rank -1 at public addrs [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] at bind addrs [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:14:50.517 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: mon.b@-1(???) e0 preinit fsid 5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:14:50.517 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: mon.b@-1(synchronizing).mds e1 new map 2023-11-27T15:14:50.517 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: mon.b@-1(synchronizing).mds e1 print_map 2023-11-27T15:14:50.517 INFO:journalctl@ceph.mon.b.smithi174.stdout: e1 2023-11-27T15:14:50.517 INFO:journalctl@ceph.mon.b.smithi174.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-11-27T15:14:50.517 INFO:journalctl@ceph.mon.b.smithi174.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-27T15:14:50.517 INFO:journalctl@ceph.mon.b.smithi174.stdout: legacy client fscid: -1 2023-11-27T15:14:50.517 INFO:journalctl@ceph.mon.b.smithi174.stdout: 2023-11-27T15:14:50.517 INFO:journalctl@ceph.mon.b.smithi174.stdout: No filesystems configured 2023-11-27T15:14:50.518 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-11-27T15:14:50.518 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-11-27T15:14:50.518 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-11-27T15:14:50.518 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-11-27T15:14:50.518 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-11-27T15:14:50.518 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-11-27T15:14:50.518 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-11-27T15:14:50.518 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:14:50.518 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:14:50.518 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-11-27T15:14:50.518 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:14:50.518 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:50.519 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.519 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.519 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.519 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.519 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:50.519 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:50.519 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:50.519 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:14:50.519 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: Updating smithi174:/etc/ceph/ceph.conf 2023-11-27T15:14:50.519 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.519 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.519 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.520 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: Updating smithi174:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:14:50.520 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:14:50.520 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:50.520 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: Updating smithi174:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:14:50.520 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi177", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:14:50.520 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: Deploying cephadm binary to smithi177 2023-11-27T15:14:50.520 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.520 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:50.520 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:50.520 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: Added host smithi177 2023-11-27T15:14:50.520 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.520 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.521 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:50.521 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.521 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:14:50.521 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:50.521 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.521 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.521 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.521 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.521 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi177", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:14:50.521 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:50.521 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:50.521 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:14:50.521 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: Updating smithi177:/etc/ceph/ceph.conf 2023-11-27T15:14:50.522 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: Updating smithi177:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:14:50.522 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:50.522 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: Updating smithi177:/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:14:50.522 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/693972224' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-11-27T15:14:50.522 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: Updating smithi177:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.client.admin.keyring 2023-11-27T15:14:50.522 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.522 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.522 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.522 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/693972224' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-11-27T15:14:50.522 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:14:50.522 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:50.522 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi133:172.21.15.133=a;smithi174:172.21.15.174=b;smithi177:172.21.15.177=c", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:14:50.523 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: Saving service mon spec with placement smithi133:172.21.15.133=a;smithi174:172.21.15.174=b;smithi177:172.21.15.177=c;count:3 2023-11-27T15:14:50.523 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.523 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:50.523 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:50.523 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:50.523 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:14:50.523 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.523 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:14:50.523 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:14:50.523 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:50.523 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: Deploying daemon mon.c on smithi177 2023-11-27T15:14:50.523 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:50.524 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:14:50.524 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:50.524 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: mon.a calling monitor election 2023-11-27T15:14:50.524 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:50.524 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:50.524 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:50.524 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: mon.c calling monitor election 2023-11-27T15:14:50.524 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:50.524 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:50.524 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:50.524 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:50.524 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-11-27T15:14:50.525 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: monmap e2: 2 mons at {a=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0],c=[v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0]} removed_ranks: {} 2023-11-27T15:14:50.525 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: fsmap 2023-11-27T15:14:50.525 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:14:50.525 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: mgrmap e13: x(active, since 45s) 2023-11-27T15:14:50.525 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: overall HEALTH_OK 2023-11-27T15:14:50.525 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.525 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.525 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:50.525 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:14:50.525 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:50.525 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: Deploying daemon mon.b on smithi174 2023-11-27T15:14:50.525 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:50.525 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:50.526 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: from='client.? 172.21.15.177:0/2416201319' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-27T15:14:50.526 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:50.526 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-11-27T15:14:50.526 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-11-27T15:14:50.526 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: expand_channel_meta expand map: {default=false} 2023-11-27T15:14:50.526 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: expand_channel_meta from 'false' to 'false' 2023-11-27T15:14:50.526 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: expand_channel_meta expanded map: {default=false} 2023-11-27T15:14:50.526 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: expand_channel_meta expand map: {default=info} 2023-11-27T15:14:50.526 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: expand_channel_meta from 'info' to 'info' 2023-11-27T15:14:50.526 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: expand_channel_meta expanded map: {default=info} 2023-11-27T15:14:50.526 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: expand_channel_meta expand map: {default=daemon} 2023-11-27T15:14:50.526 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: expand_channel_meta from 'daemon' to 'daemon' 2023-11-27T15:14:50.527 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: expand_channel_meta expanded map: {default=daemon} 2023-11-27T15:14:50.527 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: expand_channel_meta expand map: {default=debug} 2023-11-27T15:14:50.527 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: expand_channel_meta from 'debug' to 'debug' 2023-11-27T15:14:50.527 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 ceph-mon[131110]: expand_channel_meta expanded map: {default=debug} 2023-11-27T15:14:50.527 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:50 smithi174 systemd[1]: Started Ceph mon.b for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:14:50.727 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:14:50.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:50 smithi133 ceph-mon[97860]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:50.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:50 smithi177 ceph-mon[131504]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:55.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:14:55.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:14:55.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: mon.a calling monitor election 2023-11-27T15:14:55.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:55.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: mon.c calling monitor election 2023-11-27T15:14:55.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:14:55.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:55.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:14:55.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: mon.b calling monitor election 2023-11-27T15:14:55.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:14:55.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:55.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:14:55.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:14:55.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-11-27T15:14:55.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: monmap e3: 3 mons at {a=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0],b=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0],c=[v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0]} removed_ranks: {} 2023-11-27T15:14:55.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: fsmap 2023-11-27T15:14:55.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:14:55.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: mgrmap e13: x(active, since 53s) 2023-11-27T15:14:55.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: overall HEALTH_OK 2023-11-27T15:14:55.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:55.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:55.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:55 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:55.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:14:55.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:14:55.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: mon.a calling monitor election 2023-11-27T15:14:55.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:55.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: mon.c calling monitor election 2023-11-27T15:14:55.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:14:55.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:55.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:14:55.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: mon.b calling monitor election 2023-11-27T15:14:55.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:14:55.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:55.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:14:55.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:14:55.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-11-27T15:14:55.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: monmap e3: 3 mons at {a=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0],b=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0],c=[v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0]} removed_ranks: {} 2023-11-27T15:14:55.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: fsmap 2023-11-27T15:14:55.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:14:55.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: mgrmap e13: x(active, since 53s) 2023-11-27T15:14:55.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: overall HEALTH_OK 2023-11-27T15:14:55.905 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:55.905 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:55.905 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:55 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:56.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:14:56.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:14:56.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: mon.a calling monitor election 2023-11-27T15:14:56.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:14:56.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: mon.c calling monitor election 2023-11-27T15:14:56.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:14:56.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:56.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:14:56.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: mon.b calling monitor election 2023-11-27T15:14:56.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:14:56.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:56.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:14:56.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:14:56.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-11-27T15:14:56.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: monmap e3: 3 mons at {a=[v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0],b=[v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0],c=[v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0]} removed_ranks: {} 2023-11-27T15:14:56.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: fsmap 2023-11-27T15:14:56.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: osdmap e4: 0 total, 0 up, 0 in 2023-11-27T15:14:56.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: mgrmap e13: x(active, since 53s) 2023-11-27T15:14:56.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: overall HEALTH_OK 2023-11-27T15:14:56.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:56.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:56.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:55 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:56.226 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.222+0000 7fe48afbc700 1 -- 172.21.15.177:0/3354014463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4740035c0 msgr2=0x7fe474005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:56.226 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.222+0000 7fe48afbc700 1 --2- 172.21.15.177:0/3354014463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4740035c0 0x7fe474005a40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe4840fec70 tx=0x7fe47800fe60 comp rx=0 tx=0).stop 2023-11-27T15:14:56.226 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.222+0000 7fe48afbc700 1 -- 172.21.15.177:0/3354014463 shutdown_connections 2023-11-27T15:14:56.226 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.222+0000 7fe48afbc700 1 --2- 172.21.15.177:0/3354014463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4740035c0 0x7fe474005a40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:56.226 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.222+0000 7fe48afbc700 1 -- 172.21.15.177:0/3354014463 >> 172.21.15.177:0/3354014463 conn(0x7fe4840fa860 msgr2=0x7fe4840fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:56.226 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.222+0000 7fe48afbc700 1 -- 172.21.15.177:0/3354014463 shutdown_connections 2023-11-27T15:14:56.226 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.222+0000 7fe48afbc700 1 -- 172.21.15.177:0/3354014463 wait complete. 2023-11-27T15:14:56.229 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.223+0000 7fe48afbc700 1 Processor -- start 2023-11-27T15:14:56.229 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.223+0000 7fe48afbc700 1 -- start start 2023-11-27T15:14:56.229 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.223+0000 7fe48afbc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4740035c0 0x7fe484194280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:56.229 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.223+0000 7fe48afbc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4841947c0 0x7fe484198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:56.229 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.223+0000 7fe48afbc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe484199170 0x7fe484199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:56.229 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.223+0000 7fe48afbc700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe484194e90 con 0x7fe4740035c0 2023-11-27T15:14:56.229 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.223+0000 7fe48afbc700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4841a0ee0 con 0x7fe484199170 2023-11-27T15:14:56.229 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.223+0000 7fe48afbc700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4841a1020 con 0x7fe4841947c0 2023-11-27T15:14:56.230 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.224+0000 7fe483fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4841947c0 0x7fe484198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:56.230 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.224+0000 7fe483fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4841947c0 0x7fe484198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:57782/0 (socket says 172.21.15.177:57782) 2023-11-27T15:14:56.230 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.224+0000 7fe483fff700 1 -- 172.21.15.177:0/1444567358 learned_addr learned my addr 172.21.15.177:0/1444567358 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-27T15:14:56.230 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.224+0000 7fe489559700 1 --2- 172.21.15.177:0/1444567358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe484199170 0x7fe484199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:56.230 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.224+0000 7fe483fff700 1 -- 172.21.15.177:0/1444567358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4841947c0 msgr2=0x7fe484198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-11-27T15:14:56.230 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.224+0000 7fe483fff700 1 -- 172.21.15.177:0/1444567358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4841947c0 msgr2=0x7fe484198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-27T15:14:56.230 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.224+0000 7fe483fff700 1 --2- 172.21.15.177:0/1444567358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4841947c0 0x7fe484198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-27T15:14:56.230 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.224+0000 7fe483fff700 1 --2- 172.21.15.177:0/1444567358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4841947c0 0x7fe484198c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:14:56.230 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.224+0000 7fe489559700 1 -- 172.21.15.177:0/1444567358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe484199170 msgr2=0x7fe484199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-11-27T15:14:56.230 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.224+0000 7fe489559700 1 -- 172.21.15.177:0/1444567358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe484199170 msgr2=0x7fe484199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-11-27T15:14:56.230 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.224+0000 7fe489559700 1 --2- 172.21.15.177:0/1444567358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe484199170 0x7fe484199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-27T15:14:56.231 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.224+0000 7fe489559700 1 --2- 172.21.15.177:0/1444567358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe484199170 0x7fe484199610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-27T15:14:56.231 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.224+0000 7fe488d58700 1 --2- 172.21.15.177:0/1444567358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4740035c0 0x7fe484194280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:56.231 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.224+0000 7fe488d58700 1 -- 172.21.15.177:0/1444567358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4841947c0 msgr2=0x7fe484198c30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:14:56.231 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.224+0000 7fe488d58700 1 --2- 172.21.15.177:0/1444567358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4841947c0 0x7fe484198c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:56.231 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.224+0000 7fe488d58700 1 -- 172.21.15.177:0/1444567358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe484199170 msgr2=0x7fe484199610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:14:56.231 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.224+0000 7fe488d58700 1 --2- 172.21.15.177:0/1444567358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe484199170 0x7fe484199610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:56.231 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.224+0000 7fe488d58700 1 -- 172.21.15.177:0/1444567358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47800f8b0 con 0x7fe4740035c0 2023-11-27T15:14:56.231 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.224+0000 7fe488d58700 1 --2- 172.21.15.177:0/1444567358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4740035c0 0x7fe484194280 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe478015040 tx=0x7fe478012e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:56.231 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.225+0000 7fe481ffb700 1 -- 172.21.15.177:0/1444567358 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4780040b0 con 0x7fe4740035c0 2023-11-27T15:14:56.231 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.225+0000 7fe48afbc700 1 -- 172.21.15.177:0/1444567358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe484106b40 con 0x7fe4740035c0 2023-11-27T15:14:56.231 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.225+0000 7fe481ffb700 1 -- 172.21.15.177:0/1444567358 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe478004210 con 0x7fe4740035c0 2023-11-27T15:14:56.231 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.225+0000 7fe48afbc700 1 -- 172.21.15.177:0/1444567358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4841069b0 con 0x7fe4740035c0 2023-11-27T15:14:56.232 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.225+0000 7fe481ffb700 1 -- 172.21.15.177:0/1444567358 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe478027830 con 0x7fe4740035c0 2023-11-27T15:14:56.232 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.226+0000 7fe481ffb700 1 -- 172.21.15.177:0/1444567358 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7fe478024050 con 0x7fe4740035c0 2023-11-27T15:14:56.233 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.226+0000 7fe48afbc700 1 -- 172.21.15.177:0/1444567358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe484106cd0 con 0x7fe4740035c0 2023-11-27T15:14:56.233 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.226+0000 7fe481ffb700 1 --2- 172.21.15.177:0/1444567358 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe468038d40 0x7fe46803b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:56.233 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.226+0000 7fe481ffb700 1 -- 172.21.15.177:0/1444567358 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe47804e570 con 0x7fe4740035c0 2023-11-27T15:14:56.233 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.226+0000 7fe483fff700 1 --2- 172.21.15.177:0/1444567358 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe468038d40 0x7fe46803b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:56.233 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.227+0000 7fe483fff700 1 --2- 172.21.15.177:0/1444567358 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe468038d40 0x7fe46803b1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe470009180 tx=0x7fe470008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:56.234 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.231+0000 7fe481ffb700 1 -- 172.21.15.177:0/1444567358 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe478027e00 con 0x7fe4740035c0 2023-11-27T15:14:56.505 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.501+0000 7fe48afbc700 1 -- 172.21.15.177:0/1444567358 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe484062120 con 0x7fe4740035c0 2023-11-27T15:14:56.506 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.502+0000 7fe481ffb700 1 -- 172.21.15.177:0/1444567358 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1280 (secure 0 0 0) 0x7fe478035020 con 0x7fe4740035c0 2023-11-27T15:14:56.506 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-11-27T15:14:56.506 INFO:teuthology.orchestra.run.smithi177.stdout:{"epoch":3,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","modified":"2023-11-27T15:14:50.439147Z","created":"2023-11-27T15:12:58.471889Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:3300","nonce":0},{"type":"v1","addr":"172.21.15.133:6789","nonce":0}]},"addr":"172.21.15.133:6789/0","public_addr":"172.21.15.133:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:3300","nonce":0},{"type":"v1","addr":"172.21.15.177:6789","nonce":0}]},"addr":"172.21.15.177:6789/0","public_addr":"172.21.15.177:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-11-27T15:14:56.508 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.504+0000 7fe48afbc700 1 -- 172.21.15.177:0/1444567358 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe468038d40 msgr2=0x7fe46803b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:56.508 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.504+0000 7fe48afbc700 1 --2- 172.21.15.177:0/1444567358 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe468038d40 0x7fe46803b1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe470009180 tx=0x7fe470008040 comp rx=0 tx=0).stop 2023-11-27T15:14:56.509 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.504+0000 7fe48afbc700 1 -- 172.21.15.177:0/1444567358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4740035c0 msgr2=0x7fe484194280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:56.509 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.504+0000 7fe48afbc700 1 --2- 172.21.15.177:0/1444567358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4740035c0 0x7fe484194280 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe478015040 tx=0x7fe478012e10 comp rx=0 tx=0).stop 2023-11-27T15:14:56.509 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.504+0000 7fe48afbc700 1 -- 172.21.15.177:0/1444567358 shutdown_connections 2023-11-27T15:14:56.509 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.504+0000 7fe48afbc700 1 --2- 172.21.15.177:0/1444567358 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe468038d40 0x7fe46803b1f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:56.509 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.504+0000 7fe48afbc700 1 --2- 172.21.15.177:0/1444567358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe484199170 0x7fe484199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:56.509 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.504+0000 7fe48afbc700 1 --2- 172.21.15.177:0/1444567358 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4740035c0 0x7fe484194280 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:56.509 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.504+0000 7fe48afbc700 1 --2- 172.21.15.177:0/1444567358 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4841947c0 0x7fe484198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:56.509 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.504+0000 7fe48afbc700 1 -- 172.21.15.177:0/1444567358 >> 172.21.15.177:0/1444567358 conn(0x7fe4840fa860 msgr2=0x7fe48418d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:56.509 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.504+0000 7fe48afbc700 1 -- 172.21.15.177:0/1444567358 shutdown_connections 2023-11-27T15:14:56.509 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:14:56.504+0000 7fe48afbc700 1 -- 172.21.15.177:0/1444567358 wait complete. 2023-11-27T15:14:56.510 INFO:teuthology.orchestra.run.smithi177.stderr:dumped monmap epoch 3 2023-11-27T15:14:56.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:56 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:56.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:56 smithi177 ceph-mon[131504]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:56.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:56 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:56.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:56 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:14:56.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:56 smithi177 ceph-mon[131504]: from='client.? 172.21.15.177:0/1444567358' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-27T15:14:56.891 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-11-27T15:14:56.891 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph config generate-minimal-conf 2023-11-27T15:14:56.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:56 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:56.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:56 smithi174 ceph-mon[131110]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:56.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:56 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:56.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:56 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:14:56.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:56 smithi174 ceph-mon[131110]: from='client.? 172.21.15.177:0/1444567358' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-27T15:14:56.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:56 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:56.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:56 smithi133 ceph-mon[97860]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:56.960 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:56 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:14:56.961 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:56 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:14:56.961 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:56 smithi133 ceph-mon[97860]: from='client.? 172.21.15.177:0/1444567358' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-11-27T15:14:57.777 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:14:58.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:58.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:58.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:58.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:58.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:58.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:58.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:58.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:58.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:58.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:58.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:58.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:58.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:58.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:58.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:58.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:58.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:58.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:58.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:58.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:58.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:14:59.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:58 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:14:59.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:58 smithi133 ceph-mon[97860]: Updating smithi133:/etc/ceph/ceph.conf 2023-11-27T15:14:59.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:58 smithi133 ceph-mon[97860]: Updating smithi174:/etc/ceph/ceph.conf 2023-11-27T15:14:59.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:58 smithi133 ceph-mon[97860]: Updating smithi177:/etc/ceph/ceph.conf 2023-11-27T15:14:59.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:58 smithi133 ceph-mon[97860]: Updating smithi174:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:14:59.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:58 smithi133 ceph-mon[97860]: Updating smithi177:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:14:59.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:58 smithi133 ceph-mon[97860]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:59.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:58 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:59.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:58 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:59.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:58 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:59.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:58 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:59.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:58 smithi133 ceph-mon[97860]: Updating smithi133:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:14:59.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:58 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:59.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:58 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:59.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:58 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:59.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:58 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:14:59.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:58 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:14:59.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:58 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:59.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:58 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:14:59.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:58 smithi177 ceph-mon[131504]: Updating smithi133:/etc/ceph/ceph.conf 2023-11-27T15:14:59.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:58 smithi177 ceph-mon[131504]: Updating smithi174:/etc/ceph/ceph.conf 2023-11-27T15:14:59.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:58 smithi177 ceph-mon[131504]: Updating smithi177:/etc/ceph/ceph.conf 2023-11-27T15:14:59.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:58 smithi177 ceph-mon[131504]: Updating smithi174:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:14:59.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:58 smithi177 ceph-mon[131504]: Updating smithi177:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:14:59.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:58 smithi177 ceph-mon[131504]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:59.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:58 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:59.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:58 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:59.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:58 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:59.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:58 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:59.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:58 smithi177 ceph-mon[131504]: Updating smithi133:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:14:59.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:58 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:59.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:58 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:59.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:58 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:59.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:58 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:14:59.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:58 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:14:59.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:58 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:59.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:58 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:14:59.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:58 smithi174 ceph-mon[131110]: Updating smithi133:/etc/ceph/ceph.conf 2023-11-27T15:14:59.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:58 smithi174 ceph-mon[131110]: Updating smithi174:/etc/ceph/ceph.conf 2023-11-27T15:14:59.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:58 smithi174 ceph-mon[131110]: Updating smithi177:/etc/ceph/ceph.conf 2023-11-27T15:14:59.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:58 smithi174 ceph-mon[131110]: Updating smithi174:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:14:59.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:58 smithi174 ceph-mon[131110]: Updating smithi177:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:14:59.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:58 smithi174 ceph-mon[131110]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:14:59.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:58 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:59.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:58 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:59.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:58 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:59.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:58 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:59.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:58 smithi174 ceph-mon[131110]: Updating smithi133:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/config/ceph.conf 2023-11-27T15:14:59.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:58 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:59.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:58 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:59.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:58 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:14:59.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:58 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:14:59.155 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:58 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:14:59.155 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:58 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:14:59.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.355+0000 7f1c5ae8f700 1 -- 172.21.15.133:0/1142712415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c540fff90 msgr2=0x7f1c54100360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:59.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.355+0000 7f1c5ae8f700 1 --2- 172.21.15.133:0/1142712415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c540fff90 0x7f1c54100360 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1c50009910 tx=0x7f1c50009c20 comp rx=0 tx=0).stop 2023-11-27T15:14:59.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.355+0000 7f1c5ae8f700 1 -- 172.21.15.133:0/1142712415 shutdown_connections 2023-11-27T15:14:59.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.355+0000 7f1c5ae8f700 1 --2- 172.21.15.133:0/1142712415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c540fff90 0x7f1c54100360 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:59.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.355+0000 7f1c5ae8f700 1 -- 172.21.15.133:0/1142712415 >> 172.21.15.133:0/1142712415 conn(0x7f1c540fb870 msgr2=0x7f1c540fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:59.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.355+0000 7f1c5ae8f700 1 -- 172.21.15.133:0/1142712415 shutdown_connections 2023-11-27T15:14:59.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.355+0000 7f1c5ae8f700 1 -- 172.21.15.133:0/1142712415 wait complete. 2023-11-27T15:14:59.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.356+0000 7f1c5ae8f700 1 Processor -- start 2023-11-27T15:14:59.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.356+0000 7f1c5ae8f700 1 -- start start 2023-11-27T15:14:59.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.357+0000 7f1c5ae8f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c540fff90 0x7f1c54197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:59.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.357+0000 7f1c5ae8f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c54197a40 0x7f1c5419beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:59.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.357+0000 7f1c5ae8f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c5419c3f0 0x7f1c5419c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:59.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.357+0000 7f1c5ae8f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c54198110 con 0x7f1c540fff90 2023-11-27T15:14:59.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.357+0000 7f1c5ae8f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c541a4160 con 0x7f1c5419c3f0 2023-11-27T15:14:59.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.357+0000 7f1c5ae8f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c541a42a0 con 0x7f1c54197a40 2023-11-27T15:14:59.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.357+0000 7f1c58c2b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c540fff90 0x7f1c54197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:59.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.357+0000 7f1c58c2b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c540fff90 0x7f1c54197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48410/0 (socket says 172.21.15.133:48410) 2023-11-27T15:14:59.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.357+0000 7f1c58c2b700 1 -- 172.21.15.133:0/2149907891 learned_addr learned my addr 172.21.15.133:0/2149907891 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:14:59.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.357+0000 7f1c4bfff700 1 --2- 172.21.15.133:0/2149907891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c54197a40 0x7f1c5419beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:59.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.357+0000 7f1c5942c700 1 --2- 172.21.15.133:0/2149907891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c5419c3f0 0x7f1c5419c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:59.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.357+0000 7f1c58c2b700 1 -- 172.21.15.133:0/2149907891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c54197a40 msgr2=0x7f1c5419beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:59.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.357+0000 7f1c58c2b700 1 --2- 172.21.15.133:0/2149907891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c54197a40 0x7f1c5419beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:59.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.357+0000 7f1c58c2b700 1 -- 172.21.15.133:0/2149907891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c5419c3f0 msgr2=0x7f1c5419c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:59.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.357+0000 7f1c58c2b700 1 --2- 172.21.15.133:0/2149907891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c5419c3f0 0x7f1c5419c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:59.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.357+0000 7f1c58c2b700 1 -- 172.21.15.133:0/2149907891 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c500095f0 con 0x7f1c540fff90 2023-11-27T15:14:59.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.357+0000 7f1c5942c700 1 --2- 172.21.15.133:0/2149907891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c5419c3f0 0x7f1c5419c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:14:59.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.358+0000 7f1c58c2b700 1 --2- 172.21.15.133:0/2149907891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c540fff90 0x7f1c54197500 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1c50004ca0 tx=0x7f1c50004d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:59.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.358+0000 7f1c49ffb700 1 -- 172.21.15.133:0/2149907891 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5001d070 con 0x7f1c540fff90 2023-11-27T15:14:59.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.358+0000 7f1c49ffb700 1 -- 172.21.15.133:0/2149907891 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c5000be20 con 0x7f1c540fff90 2023-11-27T15:14:59.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.358+0000 7f1c5ae8f700 1 -- 172.21.15.133:0/2149907891 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5410a170 con 0x7f1c540fff90 2023-11-27T15:14:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.358+0000 7f1c49ffb700 1 -- 172.21.15.133:0/2149907891 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c50022dc0 con 0x7f1c540fff90 2023-11-27T15:14:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.358+0000 7f1c5ae8f700 1 -- 172.21.15.133:0/2149907891 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c54109fe0 con 0x7f1c540fff90 2023-11-27T15:14:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.359+0000 7f1c49ffb700 1 -- 172.21.15.133:0/2149907891 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f1c50022470 con 0x7f1c540fff90 2023-11-27T15:14:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.359+0000 7f1c5ae8f700 1 -- 172.21.15.133:0/2149907891 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c5410a300 con 0x7f1c540fff90 2023-11-27T15:14:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.359+0000 7f1c49ffb700 1 --2- 172.21.15.133:0/2149907891 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1c3c038d40 0x7f1c3c03b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:14:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.359+0000 7f1c49ffb700 1 -- 172.21.15.133:0/2149907891 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1c5004c9d0 con 0x7f1c540fff90 2023-11-27T15:14:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.360+0000 7f1c4bfff700 1 --2- 172.21.15.133:0/2149907891 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1c3c038d40 0x7f1c3c03b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:14:59.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.360+0000 7f1c4bfff700 1 --2- 172.21.15.133:0/2149907891 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1c3c038d40 0x7f1c3c03b1f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009850 tx=0x7f1c44008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:14:59.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.365+0000 7f1c49ffb700 1 -- 172.21.15.133:0/2149907891 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1c50020370 con 0x7f1c540fff90 2023-11-27T15:14:59.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.580+0000 7f1c5ae8f700 1 -- 172.21.15.133:0/2149907891 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f1c54062120 con 0x7f1c540fff90 2023-11-27T15:14:59.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.580+0000 7f1c49ffb700 1 -- 172.21.15.133:0/2149907891 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7f1c50027020 con 0x7f1c540fff90 2023-11-27T15:14:59.586 INFO:teuthology.orchestra.run.smithi133.stdout:# minimal ceph.conf for 5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:14:59.586 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2023-11-27T15:14:59.586 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = 5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:14:59.586 INFO:teuthology.orchestra.run.smithi133.stdout: mon_host = [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] 2023-11-27T15:14:59.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.583+0000 7f1c5ae8f700 1 -- 172.21.15.133:0/2149907891 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1c3c038d40 msgr2=0x7f1c3c03b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:59.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.583+0000 7f1c5ae8f700 1 --2- 172.21.15.133:0/2149907891 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1c3c038d40 0x7f1c3c03b1f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009850 tx=0x7f1c44008040 comp rx=0 tx=0).stop 2023-11-27T15:14:59.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.583+0000 7f1c5ae8f700 1 -- 172.21.15.133:0/2149907891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c540fff90 msgr2=0x7f1c54197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:14:59.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.583+0000 7f1c5ae8f700 1 --2- 172.21.15.133:0/2149907891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c540fff90 0x7f1c54197500 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1c50004ca0 tx=0x7f1c50004d80 comp rx=0 tx=0).stop 2023-11-27T15:14:59.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.583+0000 7f1c5ae8f700 1 -- 172.21.15.133:0/2149907891 shutdown_connections 2023-11-27T15:14:59.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.583+0000 7f1c5ae8f700 1 --2- 172.21.15.133:0/2149907891 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1c3c038d40 0x7f1c3c03b1f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:59.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.583+0000 7f1c5ae8f700 1 --2- 172.21.15.133:0/2149907891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c5419c3f0 0x7f1c5419c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:59.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.583+0000 7f1c5ae8f700 1 --2- 172.21.15.133:0/2149907891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c540fff90 0x7f1c54197500 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:59.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.583+0000 7f1c5ae8f700 1 --2- 172.21.15.133:0/2149907891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c54197a40 0x7f1c5419beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:14:59.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.583+0000 7f1c5ae8f700 1 -- 172.21.15.133:0/2149907891 >> 172.21.15.133:0/2149907891 conn(0x7f1c540fb870 msgr2=0x7f1c541038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:14:59.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.583+0000 7f1c5ae8f700 1 -- 172.21.15.133:0/2149907891 shutdown_connections 2023-11-27T15:14:59.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:14:59.583+0000 7f1c5ae8f700 1 -- 172.21.15.133:0/2149907891 wait complete. 2023-11-27T15:15:00.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:59 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:15:00.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:59 smithi174 ceph-mon[131110]: Reconfiguring mon.a (unknown last config time)... 2023-11-27T15:15:00.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:59 smithi174 ceph-mon[131110]: Reconfiguring daemon mon.a on smithi133 2023-11-27T15:15:00.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:14:59 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2149907891' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:00.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:59 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:15:00.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:59 smithi133 ceph-mon[97860]: Reconfiguring mon.a (unknown last config time)... 2023-11-27T15:15:00.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:59 smithi133 ceph-mon[97860]: Reconfiguring daemon mon.a on smithi133 2023-11-27T15:15:00.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:14:59 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2149907891' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:00.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:59 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:15:00.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:59 smithi177 ceph-mon[131504]: Reconfiguring mon.a (unknown last config time)... 2023-11-27T15:15:00.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:59 smithi177 ceph-mon[131504]: Reconfiguring daemon mon.a on smithi133 2023-11-27T15:15:00.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:14:59 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2149907891' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:00.382 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-11-27T15:15:00.382 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-11-27T15:15:00.382 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.conf 2023-11-27T15:15:00.461 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-11-27T15:15:00.461 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:15:00.569 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-27T15:15:00.569 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.conf 2023-11-27T15:15:00.600 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-27T15:15:00.601 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:15:00.668 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-27T15:15:00.668 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/ceph/ceph.conf 2023-11-27T15:15:00.703 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-27T15:15:00.703 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-11-27T15:15:00.773 INFO:tasks.cephadm:Adding mgr.x on smithi133 2023-11-27T15:15:00.773 INFO:tasks.cephadm:Adding mgr.y on smithi174 2023-11-27T15:15:00.773 INFO:tasks.cephadm:Adding mgr.z on smithi177 2023-11-27T15:15:00.773 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph orch apply mgr '3;smithi133=x;smithi174=y;smithi177=z' 2023-11-27T15:15:00.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:00 smithi174 ceph-mon[131110]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:00.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:00 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:00.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:00 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:00.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:00 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:15:00.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:00 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:15:00.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:00 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:01.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:00 smithi177 ceph-mon[131504]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:01.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:00 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:01.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:00 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:01.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:00 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:15:01.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:00 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:15:01.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:00 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:01.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:00 smithi133 ceph-mon[97860]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:01.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:00 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:01.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:00 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:01.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:00 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:15:01.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:00 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:15:01.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:00 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:01.449 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:15:01.966 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:01 smithi177 ceph-mon[131504]: Reconfiguring mon.b (monmap changed)... 2023-11-27T15:15:01.966 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:01 smithi177 ceph-mon[131504]: Reconfiguring daemon mon.b on smithi174 2023-11-27T15:15:01.966 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:01.967 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:01.967 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:15:01.967 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:15:01.967 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:02.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:01 smithi174 ceph-mon[131110]: Reconfiguring mon.b (monmap changed)... 2023-11-27T15:15:02.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:01 smithi174 ceph-mon[131110]: Reconfiguring daemon mon.b on smithi174 2023-11-27T15:15:02.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:02.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:02.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:15:02.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:15:02.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:02.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:01 smithi133 ceph-mon[97860]: Reconfiguring mon.b (monmap changed)... 2023-11-27T15:15:02.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:01 smithi133 ceph-mon[97860]: Reconfiguring daemon mon.b on smithi174 2023-11-27T15:15:02.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:02.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:02.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-11-27T15:15:02.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-11-27T15:15:02.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:02.760 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.754+0000 7f014a744700 1 -- 172.21.15.177:0/1585709923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0144068f80 msgr2=0x7f01441029c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:02.760 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.754+0000 7f014a744700 1 --2- 172.21.15.177:0/1585709923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0144068f80 0x7f01441029c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f012c009910 tx=0x7f012c009c20 comp rx=0 tx=0).stop 2023-11-27T15:15:02.761 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.754+0000 7f014a744700 1 -- 172.21.15.177:0/1585709923 shutdown_connections 2023-11-27T15:15:02.761 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.754+0000 7f014a744700 1 --2- 172.21.15.177:0/1585709923 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0144068f80 0x7f01441029c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:02.761 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.754+0000 7f014a744700 1 -- 172.21.15.177:0/1585709923 >> 172.21.15.177:0/1585709923 conn(0x7f01440fa390 msgr2=0x7f01440fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:02.761 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.754+0000 7f014a744700 1 -- 172.21.15.177:0/1585709923 shutdown_connections 2023-11-27T15:15:02.761 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.754+0000 7f014a744700 1 -- 172.21.15.177:0/1585709923 wait complete. 2023-11-27T15:15:02.761 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.755+0000 7f014a744700 1 Processor -- start 2023-11-27T15:15:02.761 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.755+0000 7f014a744700 1 -- start start 2023-11-27T15:15:02.761 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.755+0000 7f014a744700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0144194ff0 0x7f0144195400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:02.761 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.755+0000 7f014a744700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0144195940 0x7f014419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:02.761 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.755+0000 7f014a744700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f014419ab20 0x7f014419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:02.761 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.755+0000 7f014a744700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014419d6e0 con 0x7f0144194ff0 2023-11-27T15:15:02.762 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.755+0000 7f014a744700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014419d850 con 0x7f014419ab20 2023-11-27T15:15:02.762 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.755+0000 7f014a744700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014419da10 con 0x7f0144195940 2023-11-27T15:15:02.762 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.756+0000 7f01437fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0144195940 0x7f014419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:02.762 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.756+0000 7f0148ce1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f014419ab20 0x7f014419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:02.762 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.756+0000 7f01437fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0144195940 0x7f014419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:57792/0 (socket says 172.21.15.177:57792) 2023-11-27T15:15:02.762 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.756+0000 7f0148ce1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f014419ab20 0x7f014419cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.177:42494/0 (socket says 172.21.15.177:42494) 2023-11-27T15:15:02.762 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.756+0000 7f01437fe700 1 -- 172.21.15.177:0/20315985 learned_addr learned my addr 172.21.15.177:0/20315985 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-27T15:15:02.762 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.756+0000 7f0143fff700 1 --2- 172.21.15.177:0/20315985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0144194ff0 0x7f0144195400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:02.762 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.756+0000 7f01437fe700 1 -- 172.21.15.177:0/20315985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f014419ab20 msgr2=0x7f014419cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:02.762 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.756+0000 7f01437fe700 1 --2- 172.21.15.177:0/20315985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f014419ab20 0x7f014419cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:02.762 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.756+0000 7f01437fe700 1 -- 172.21.15.177:0/20315985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0144194ff0 msgr2=0x7f0144195400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:02.763 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.756+0000 7f01437fe700 1 --2- 172.21.15.177:0/20315985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0144194ff0 0x7f0144195400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:02.763 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.756+0000 7f01437fe700 1 -- 172.21.15.177:0/20315985 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012c0095f0 con 0x7f0144195940 2023-11-27T15:15:02.763 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.756+0000 7f01437fe700 1 --2- 172.21.15.177:0/20315985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0144195940 0x7f014419a5e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f013400b780 tx=0x7f013400ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:02.763 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.756+0000 7f01417fa700 1 -- 172.21.15.177:0/20315985 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134004230 con 0x7f0144195940 2023-11-27T15:15:02.763 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.756+0000 7f014a744700 1 -- 172.21.15.177:0/20315985 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0144105c20 con 0x7f0144195940 2023-11-27T15:15:02.763 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.756+0000 7f01417fa700 1 -- 172.21.15.177:0/20315985 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0134004390 con 0x7f0144195940 2023-11-27T15:15:02.763 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.756+0000 7f01417fa700 1 -- 172.21.15.177:0/20315985 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134011b90 con 0x7f0144195940 2023-11-27T15:15:02.763 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.756+0000 7f014a744700 1 -- 172.21.15.177:0/20315985 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0144105a90 con 0x7f0144195940 2023-11-27T15:15:02.764 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.757+0000 7f01417fa700 1 -- 172.21.15.177:0/20315985 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7f0134011cf0 con 0x7f0144195940 2023-11-27T15:15:02.764 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.757+0000 7f01417fa700 1 --2- 172.21.15.177:0/20315985 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0130038cf0 0x7f013003b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:02.764 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.757+0000 7f014a744700 1 -- 172.21.15.177:0/20315985 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f014418e950 con 0x7f0144195940 2023-11-27T15:15:02.764 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.757+0000 7f01417fa700 1 -- 172.21.15.177:0/20315985 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0134049f60 con 0x7f0144195940 2023-11-27T15:15:02.764 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.758+0000 7f0143fff700 1 --2- 172.21.15.177:0/20315985 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0130038cf0 0x7f013003b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:02.764 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.758+0000 7f0143fff700 1 --2- 172.21.15.177:0/20315985 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0130038cf0 0x7f013003b1a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f012c005130 tx=0x7f012c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:02.766 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.762+0000 7f01417fa700 1 -- 172.21.15.177:0/20315985 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0134016070 con 0x7f0144195940 2023-11-27T15:15:02.981 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.977+0000 7f014a744700 1 -- 172.21.15.177:0/20315985 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi133=x;smithi174=y;smithi177=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f0144060f30 con 0x7f0130038cf0 2023-11-27T15:15:02.988 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.984+0000 7f01417fa700 1 -- 172.21.15.177:0/20315985 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0144060f30 con 0x7f0130038cf0 2023-11-27T15:15:02.989 INFO:teuthology.orchestra.run.smithi177.stdout:Scheduled mgr update... 2023-11-27T15:15:02.991 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.987+0000 7f014a744700 1 -- 172.21.15.177:0/20315985 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0130038cf0 msgr2=0x7f013003b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:02.991 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.987+0000 7f014a744700 1 --2- 172.21.15.177:0/20315985 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0130038cf0 0x7f013003b1a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f012c005130 tx=0x7f012c005fd0 comp rx=0 tx=0).stop 2023-11-27T15:15:02.991 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.987+0000 7f014a744700 1 -- 172.21.15.177:0/20315985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0144195940 msgr2=0x7f014419a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:02.991 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.987+0000 7f014a744700 1 --2- 172.21.15.177:0/20315985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0144195940 0x7f014419a5e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f013400b780 tx=0x7f013400ba90 comp rx=0 tx=0).stop 2023-11-27T15:15:02.991 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.987+0000 7f014a744700 1 -- 172.21.15.177:0/20315985 shutdown_connections 2023-11-27T15:15:02.991 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.987+0000 7f014a744700 1 --2- 172.21.15.177:0/20315985 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0130038cf0 0x7f013003b1a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:02.991 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.987+0000 7f014a744700 1 --2- 172.21.15.177:0/20315985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f014419ab20 0x7f014419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:02.991 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.987+0000 7f014a744700 1 --2- 172.21.15.177:0/20315985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0144194ff0 0x7f0144195400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:02.991 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.987+0000 7f014a744700 1 --2- 172.21.15.177:0/20315985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0144195940 0x7f014419a5e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:02.992 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.987+0000 7f014a744700 1 -- 172.21.15.177:0/20315985 >> 172.21.15.177:0/20315985 conn(0x7f01440fa390 msgr2=0x7f01440faff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:02.992 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.987+0000 7f014a744700 1 -- 172.21.15.177:0/20315985 shutdown_connections 2023-11-27T15:15:02.992 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:02.987+0000 7f014a744700 1 -- 172.21.15.177:0/20315985 wait complete. 2023-11-27T15:15:03.241 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:03 smithi177 ceph-mon[131504]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:03.241 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:03 smithi177 ceph-mon[131504]: Reconfiguring mon.c (monmap changed)... 2023-11-27T15:15:03.241 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:03 smithi177 ceph-mon[131504]: Reconfiguring daemon mon.c on smithi177 2023-11-27T15:15:03.241 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:03 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:03.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:03 smithi133 ceph-mon[97860]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:03.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:03 smithi133 ceph-mon[97860]: Reconfiguring mon.c (monmap changed)... 2023-11-27T15:15:03.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:03 smithi133 ceph-mon[97860]: Reconfiguring daemon mon.c on smithi177 2023-11-27T15:15:03.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:03 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:03.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:03 smithi174 ceph-mon[131110]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:03.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:03 smithi174 ceph-mon[131110]: Reconfiguring mon.c (monmap changed)... 2023-11-27T15:15:03.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:03 smithi174 ceph-mon[131110]: Reconfiguring daemon mon.c on smithi177 2023-11-27T15:15:03.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:03 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:03.509 DEBUG:teuthology.orchestra.run.smithi174:mgr.y> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mgr.y.service 2023-11-27T15:15:03.512 DEBUG:teuthology.orchestra.run.smithi177:mgr.z> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mgr.z.service 2023-11-27T15:15:03.514 INFO:tasks.cephadm:Deploying OSDs... 2023-11-27T15:15:03.514 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-11-27T15:15:03.514 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/scratch_devs of=/dev/stdout 2023-11-27T15:15:03.542 INFO:journalctl@ceph.mgr.z.smithi177.stdout:-- Logs begin at Mon 2023-11-27 14:50:38 UTC. -- 2023-11-27T15:15:03.546 INFO:journalctl@ceph.mgr.y.smithi174.stdout:-- Logs begin at Mon 2023-11-27 14:50:40 UTC. -- 2023-11-27T15:15:03.561 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-27T15:15:03.561 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_1 2023-11-27T15:15:03.646 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-27T15:15:03.646 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:15:03.646 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-11-27T15:15:03.646 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:15:03.647 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:15:03.647 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2023-11-27 15:14:13.275618820 +0000 2023-11-27T15:15:03.647 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2023-11-27 15:08:08.825453650 +0000 2023-11-27T15:15:03.647 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2023-11-27 15:08:08.825453650 +0000 2023-11-27T15:15:03.647 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2023-11-27 15:08:08.797454164 +0000 2023-11-27T15:15:03.647 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-27T15:15:03.754 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2023-11-27T15:15:03.754 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2023-11-27T15:15:03.755 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000351583 s, 1.5 MB/s 2023-11-27T15:15:03.757 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-27T15:15:03.802 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_2 2023-11-27T15:15:03.886 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-27T15:15:03.886 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:15:03.886 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-11-27T15:15:03.886 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:15:03.886 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:15:03.886 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2023-11-27 15:14:13.275618820 +0000 2023-11-27T15:15:03.886 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2023-11-27 15:08:09.638438734 +0000 2023-11-27T15:15:03.887 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2023-11-27 15:08:09.638438734 +0000 2023-11-27T15:15:03.887 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2023-11-27 15:08:09.613439193 +0000 2023-11-27T15:15:03.887 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-27T15:15:03.991 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2023-11-27T15:15:03.991 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2023-11-27T15:15:03.991 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000319973 s, 1.6 MB/s 2023-11-27T15:15:03.993 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-27T15:15:04.039 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_3 2023-11-27T15:15:04.123 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-27T15:15:04.123 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:15:04.123 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-11-27T15:15:04.123 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:15:04.123 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:15:04.123 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2023-11-27 15:14:13.275618820 +0000 2023-11-27T15:15:04.123 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2023-11-27 15:08:10.452423800 +0000 2023-11-27T15:15:04.124 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2023-11-27 15:08:10.452423800 +0000 2023-11-27T15:15:04.124 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2023-11-27 15:08:10.427424259 +0000 2023-11-27T15:15:04.124 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-27T15:15:04.228 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2023-11-27T15:15:04.228 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2023-11-27T15:15:04.228 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000303941 s, 1.7 MB/s 2023-11-27T15:15:04.231 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-27T15:15:04.276 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_4 2023-11-27T15:15:04.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:04 smithi133 ceph-mon[97860]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi133=x;smithi174=y;smithi177=z", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:15:04.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:04 smithi133 ceph-mon[97860]: Saving service mgr spec with placement smithi133=x;smithi174=y;smithi177=z;count:3 2023-11-27T15:15:04.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:04 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:04.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:04 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:04.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:04 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:04.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:04 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:04.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:04 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:04.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:04 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:04.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:04 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:15:04.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:04 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-27T15:15:04.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:04 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:15:04.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:04 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:04.322 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-27T15:15:04.322 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:15:04.322 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-11-27T15:15:04.322 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:15:04.322 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:15:04.322 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2023-11-27 15:14:13.275618820 +0000 2023-11-27T15:15:04.322 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2023-11-27 15:08:11.267408848 +0000 2023-11-27T15:15:04.322 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2023-11-27 15:08:11.267408848 +0000 2023-11-27T15:15:04.323 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2023-11-27 15:08:11.240409343 +0000 2023-11-27T15:15:04.323 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-27T15:15:04.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:04 smithi177 ceph-mon[131504]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi133=x;smithi174=y;smithi177=z", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:15:04.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:04 smithi177 ceph-mon[131504]: Saving service mgr spec with placement smithi133=x;smithi174=y;smithi177=z;count:3 2023-11-27T15:15:04.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:04 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:04.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:04 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:04.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:04 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:04.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:04 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:04.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:04 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:04.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:04 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:04.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:04 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:15:04.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:04 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-27T15:15:04.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:04 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:15:04.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:04 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:04.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:04 smithi174 ceph-mon[131110]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi133=x;smithi174=y;smithi177=z", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:15:04.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:04 smithi174 ceph-mon[131110]: Saving service mgr spec with placement smithi133=x;smithi174=y;smithi177=z;count:3 2023-11-27T15:15:04.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:04 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:04.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:04 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:04.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:04 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:04.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:04 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:04.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:04 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:04.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:04 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:04.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:04 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:15:04.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:04 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-27T15:15:04.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:04 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:15:04.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:04 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:04.427 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2023-11-27T15:15:04.427 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2023-11-27T15:15:04.427 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000304542 s, 1.7 MB/s 2023-11-27T15:15:04.430 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-27T15:15:04.475 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2023-11-27T15:15:04.475 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/scratch_devs of=/dev/stdout 2023-11-27T15:15:04.496 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-27T15:15:04.497 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_1 2023-11-27T15:15:04.556 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-27T15:15:04.556 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:15:04.557 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 6h/6d Inode: 380708 Links: 1 2023-11-27T15:15:04.557 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:15:04.557 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:15:04.557 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2023-11-27 15:14:49.855580606 +0000 2023-11-27T15:15:04.557 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2023-11-27 15:04:54.101409678 +0000 2023-11-27T15:15:04.557 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2023-11-27 15:04:54.101409678 +0000 2023-11-27T15:15:04.557 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: - 2023-11-27T15:15:04.557 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-27T15:15:04.625 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2023-11-27T15:15:04.626 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2023-11-27T15:15:04.626 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000242921 s, 2.1 MB/s 2023-11-27T15:15:04.627 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-27T15:15:04.686 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_2 2023-11-27T15:15:04.745 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-27T15:15:04.746 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:15:04.746 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 6h/6d Inode: 391636 Links: 1 2023-11-27T15:15:04.746 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:15:04.746 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:15:04.746 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2023-11-27 15:14:49.855580606 +0000 2023-11-27T15:15:04.746 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2023-11-27 15:04:54.599402916 +0000 2023-11-27T15:15:04.746 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2023-11-27 15:04:54.599402916 +0000 2023-11-27T15:15:04.747 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: - 2023-11-27T15:15:04.747 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-27T15:15:04.813 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2023-11-27T15:15:04.813 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2023-11-27T15:15:04.813 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000229412 s, 2.2 MB/s 2023-11-27T15:15:04.814 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-27T15:15:04.873 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_3 2023-11-27T15:15:04.932 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-27T15:15:04.932 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:15:04.932 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 6h/6d Inode: 389948 Links: 1 2023-11-27T15:15:04.932 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:15:04.932 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:15:04.932 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2023-11-27 15:14:49.855580606 +0000 2023-11-27T15:15:04.932 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2023-11-27 15:04:55.099396128 +0000 2023-11-27T15:15:04.932 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2023-11-27 15:04:55.099396128 +0000 2023-11-27T15:15:04.932 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: - 2023-11-27T15:15:04.933 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-27T15:15:05.000 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2023-11-27T15:15:05.000 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2023-11-27T15:15:05.000 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000228868 s, 2.2 MB/s 2023-11-27T15:15:05.001 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-27T15:15:05.001 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:04 smithi177 systemd[1]: Starting Ceph mgr.z for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:15:05.060 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_4 2023-11-27T15:15:05.119 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-27T15:15:05.119 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:15:05.119 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 6h/6d Inode: 392508 Links: 1 2023-11-27T15:15:05.119 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:15:05.119 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:15:05.120 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2023-11-27 15:14:49.855580606 +0000 2023-11-27T15:15:05.120 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2023-11-27 15:04:55.612389162 +0000 2023-11-27T15:15:05.120 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2023-11-27 15:04:55.612389162 +0000 2023-11-27T15:15:05.120 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: - 2023-11-27T15:15:05.120 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-27T15:15:05.186 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2023-11-27T15:15:05.186 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2023-11-27T15:15:05.186 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000232584 s, 2.2 MB/s 2023-11-27T15:15:05.187 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-27T15:15:05.246 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-11-27T15:15:05.246 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/scratch_devs of=/dev/stdout 2023-11-27T15:15:05.256 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:04 smithi177 podman[133349]: 2023-11-27T15:15:05.256 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:05 smithi177 ceph-mgr[133389]: -- 172.21.15.177:0/541506504 <== mon.1 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559be9e8f520 con 0x559be9e64c00 2023-11-27T15:15:05.256 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:05 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:15:05.256 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:05 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:15:05.256 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:05 smithi177 ceph-mon[131504]: Deploying daemon mgr.z on smithi177 2023-11-27T15:15:05.256 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:05 smithi177 ceph-mon[131504]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:05.265 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-27T15:15:05.265 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_1 2023-11-27T15:15:05.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:05 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:15:05.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:05 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:15:05.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:05 smithi133 ceph-mon[97860]: Deploying daemon mgr.z on smithi177 2023-11-27T15:15:05.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:05 smithi133 ceph-mon[97860]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:05.324 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-27T15:15:05.324 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:15:05.324 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 384320 Links: 1 2023-11-27T15:15:05.324 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:15:05.325 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:15:05.325 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2023-11-27 15:15:04.676984529 +0000 2023-11-27T15:15:05.325 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2023-11-27 15:05:09.997596548 +0000 2023-11-27T15:15:05.325 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2023-11-27 15:05:09.997596548 +0000 2023-11-27T15:15:05.325 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2023-11-27T15:15:05.325 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-27T15:15:05.395 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2023-11-27T15:15:05.395 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2023-11-27T15:15:05.395 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000227606 s, 2.2 MB/s 2023-11-27T15:15:05.396 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-27T15:15:05.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:05 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:15:05.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:05 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:15:05.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:05 smithi174 ceph-mon[131110]: Deploying daemon mgr.z on smithi177 2023-11-27T15:15:05.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:05 smithi174 ceph-mon[131110]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:05.454 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_2 2023-11-27T15:15:05.511 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-27T15:15:05.512 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:15:05.512 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 391972 Links: 1 2023-11-27T15:15:05.512 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:15:05.512 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:15:05.512 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2023-11-27 15:15:04.676984529 +0000 2023-11-27T15:15:05.512 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2023-11-27 15:05:10.509581733 +0000 2023-11-27T15:15:05.512 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2023-11-27 15:05:10.509581733 +0000 2023-11-27T15:15:05.512 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2023-11-27T15:15:05.512 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-27T15:15:05.531 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:05 smithi177 bash[133349]: e663062808458de9db1b5d148cbbc5f4eb99dfbc3d82d01713b6208baa2c06cd 2023-11-27T15:15:05.531 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:05 smithi177 systemd[1]: Started Ceph mgr.z for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:15:05.531 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:05 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:05.345+0000 7f679f226200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-27T15:15:05.531 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:05 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:05.432+0000 7f679f226200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-27T15:15:05.531 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:05 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:05.526+0000 7f679f226200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-27T15:15:05.556 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2023-11-27T15:15:05.556 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2023-11-27T15:15:05.556 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000217445 s, 2.4 MB/s 2023-11-27T15:15:05.558 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-27T15:15:05.616 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_3 2023-11-27T15:15:05.673 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-27T15:15:05.674 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:15:05.674 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 384374 Links: 1 2023-11-27T15:15:05.674 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:15:05.674 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:15:05.674 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2023-11-27 15:15:04.676984529 +0000 2023-11-27T15:15:05.674 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2023-11-27 15:05:11.025566801 +0000 2023-11-27T15:15:05.674 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2023-11-27 15:05:11.025566801 +0000 2023-11-27T15:15:05.674 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2023-11-27T15:15:05.674 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-27T15:15:05.739 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2023-11-27T15:15:05.739 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2023-11-27T15:15:05.739 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000245392 s, 2.1 MB/s 2023-11-27T15:15:05.740 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-27T15:15:05.798 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_4 2023-11-27T15:15:05.816 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:05 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:05.628+0000 7f679f226200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-27T15:15:05.832 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-27T15:15:05.832 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-27T15:15:05.832 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 394664 Links: 1 2023-11-27T15:15:05.832 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:15:05.832 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2023-11-27T15:15:05.832 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2023-11-27 15:15:04.676984529 +0000 2023-11-27T15:15:05.832 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2023-11-27 15:05:11.527552275 +0000 2023-11-27T15:15:05.833 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2023-11-27 15:05:11.527552275 +0000 2023-11-27T15:15:05.833 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2023-11-27T15:15:05.833 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-27T15:15:05.898 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2023-11-27T15:15:05.899 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2023-11-27T15:15:05.899 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000196813 s, 2.6 MB/s 2023-11-27T15:15:05.900 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-27T15:15:05.958 INFO:tasks.cephadm:Deploying osd.0 on smithi133 with /dev/vg_nvme/lv_4... 2023-11-27T15:15:05.958 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-11-27T15:15:06.088 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:05 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:05.811+0000 7f679f226200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-27T15:15:06.446 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:06 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:06.133+0000 7f679f226200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-27T15:15:06.447 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:06 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:06.219+0000 7f679f226200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-27T15:15:06.447 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:06 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:06.346+0000 7f679f226200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-27T15:15:06.447 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:06 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:06.447 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:06 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:06.447 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:06 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:06.447 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:06 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:15:06.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:06 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-27T15:15:06.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:06 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:15:06.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:06 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:06.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:06 smithi177 ceph-mon[131504]: Deploying daemon mgr.y on smithi174 2023-11-27T15:15:06.448 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:06 smithi177 ceph-mon[131504]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:06.629 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:06 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:06.629 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:06 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:06.629 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:06 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:06.629 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:06 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:15:06.629 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:06 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-27T15:15:06.629 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:06 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:15:06.629 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:06 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:06.629 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:06 smithi174 ceph-mon[131110]: Deploying daemon mgr.y on smithi174 2023-11-27T15:15:06.630 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:06 smithi174 ceph-mon[131110]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:06.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:06 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:06.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:06 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:06.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:06 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:06.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:06 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:15:06.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:06 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-11-27T15:15:06.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:06 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:15:06.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:06 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:06.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:06 smithi133 ceph-mon[97860]: Deploying daemon mgr.y on smithi174 2023-11-27T15:15:06.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:06 smithi133 ceph-mon[97860]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:06.807 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:15:06.838 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:06 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:06.441+0000 7f679f226200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-27T15:15:06.903 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:06 smithi174 systemd[1]: Starting Ceph mgr.y for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:15:07.338 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:06 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:06.859+0000 7f679f226200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-27T15:15:07.448 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:07 smithi174 podman[132553]: 2023-11-27T15:15:07.448 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:07 smithi174 bash[132553]: cf69fe65b2dc587fa5ddab09572a61caaa5bbcdee26eca6cedf3ccc562afe487 2023-11-27T15:15:07.798 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:07 smithi174 systemd[1]: Started Ceph mgr.y for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:15:07.798 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:07 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:07.527+0000 7fc092bc0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-27T15:15:07.798 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:07 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:07.605+0000 7fc092bc0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-27T15:15:07.798 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:07 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:07.693+0000 7fc092bc0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-27T15:15:07.838 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:07 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:07.433+0000 7f679f226200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-27T15:15:07.838 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:07 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:07.605+0000 7f679f226200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-27T15:15:08.153 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:07 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:07.793+0000 7fc092bc0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-27T15:15:08.153 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:07 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:07.970+0000 7fc092bc0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-27T15:15:08.626 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:08 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:08.290+0000 7fc092bc0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-27T15:15:08.627 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:08 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:08.384+0000 7fc092bc0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-27T15:15:08.627 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:08 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:08.521+0000 7fc092bc0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-27T15:15:08.627 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:08.627 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:08.627 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:08.627 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:08.627 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:08 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:15:08.627 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:08 smithi174 ceph-mon[131110]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:08.627 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:08.692 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:08 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:08.415+0000 7f679f226200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:15:08.692 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:08 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:08.512+0000 7f679f226200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-27T15:15:08.692 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:08 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:08.687+0000 7f679f226200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:15:08.693 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:08.693 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:08.693 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:08.693 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:08.693 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:08 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:15:08.693 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:08 smithi177 ceph-mon[131504]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:08.693 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:08.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:08.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:08.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:08.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:08.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:08 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:15:08.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:08 smithi133 ceph-mon[97860]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:08.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:08.822 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-11-27T15:15:08.823 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-11-27T15:15:08.823 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2023-11-27T15:15:08.823 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2023-11-27T15:15:08.823 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0306233 s, 342 MB/s 2023-11-27T15:15:08.823 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2023-11-27T15:15:08.903 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:08 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:08.621+0000 7fc092bc0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-27T15:15:09.296 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi133:vg_nvme/lv_4 2023-11-27T15:15:09.338 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:08 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:08.977+0000 7f679f226200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-27T15:15:09.338 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:09 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:09.216+0000 7f679f226200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-27T15:15:09.403 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:09 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:09.073+0000 7fc092bc0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-27T15:15:09.636 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:09 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:09.367+0000 7f679f226200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-27T15:15:10.088 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:09 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:09.631+0000 7f679f226200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-27T15:15:10.153 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:09 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:09.660+0000 7fc092bc0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-27T15:15:10.153 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:09 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:09.828+0000 7fc092bc0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-27T15:15:10.161 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:15:10.524 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:10 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:10.525 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:10 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:10.525 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:10 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:10.525 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:10 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:10.525 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:10 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:10.525 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:10 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:10.525 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:10 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:15:10.525 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:10 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:10.525 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:10 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:15:10.525 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:10 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:15:10.525 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:10 smithi133 ceph-mon[97860]: Reconfiguring mgr.x (unknown last config time)... 2023-11-27T15:15:10.525 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:10 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:15:10.526 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:10 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:15:10.526 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:10 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:10.526 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:10 smithi133 ceph-mon[97860]: Reconfiguring daemon mgr.x on smithi133 2023-11-27T15:15:10.526 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:10 smithi133 ceph-mon[97860]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:10.586 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:10 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:10.586 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:10 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:10.586 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:10 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:10.587 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:10 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:10.587 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:10 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:10.587 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:10 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:10.587 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:10 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:15:10.587 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:10 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:10.587 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:10 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:15:10.587 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:10 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:15:10.587 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:10 smithi174 ceph-mon[131110]: Reconfiguring mgr.x (unknown last config time)... 2023-11-27T15:15:10.587 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:10 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:15:10.587 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:10 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:15:10.587 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:10 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:10.587 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:10 smithi174 ceph-mon[131110]: Reconfiguring daemon mgr.x on smithi133 2023-11-27T15:15:10.587 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:10 smithi174 ceph-mon[131110]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:10.588 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:10 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:10.327+0000 7f679f226200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-27T15:15:10.588 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:10 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:10.457+0000 7f679f226200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-27T15:15:10.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:10 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:10.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:10 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:10.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:10 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:10.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:10 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:10.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:10 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:10.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:10 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:10.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:10 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:15:10.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:10 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:10.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:10 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:15:10.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:10 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:15:10.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:10 smithi177 ceph-mon[131504]: Reconfiguring mgr.x (unknown last config time)... 2023-11-27T15:15:10.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:10 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-11-27T15:15:10.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:10 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-11-27T15:15:10.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:10 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:10.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:10 smithi177 ceph-mon[131504]: Reconfiguring daemon mgr.x on smithi133 2023-11-27T15:15:10.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:10 smithi177 ceph-mon[131504]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:10.842 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:10 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:10.582+0000 7fc092bc0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:15:10.842 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:10 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:10.671+0000 7fc092bc0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-27T15:15:10.974 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:10 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:10.645+0000 7f679f226200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-27T15:15:10.974 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:10 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:10.758+0000 7f679f226200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-27T15:15:11.111 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:10 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:10.838+0000 7fc092bc0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-27T15:15:11.338 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:10 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:10.969+0000 7f679f226200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-27T15:15:11.403 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:11 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:11.107+0000 7fc092bc0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-27T15:15:11.403 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:11 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:11.331+0000 7fc092bc0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-27T15:15:11.733 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:11 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:11.475+0000 7fc092bc0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-27T15:15:11.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.851+0000 7fcfbfacd700 1 -- 172.21.15.133:0/1787378341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfb80754a0 msgr2=0x7fcfb80758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:11.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.851+0000 7fcfbfacd700 1 --2- 172.21.15.133:0/1787378341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfb80754a0 0x7fcfb80758b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4009990 tx=0x7fcfb4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:15:11.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.851+0000 7fcfbfacd700 1 -- 172.21.15.133:0/1787378341 shutdown_connections 2023-11-27T15:15:11.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.851+0000 7fcfbfacd700 1 --2- 172.21.15.133:0/1787378341 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb80770a0 0x7fcfb8077930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:11.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.851+0000 7fcfbfacd700 1 --2- 172.21.15.133:0/1787378341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfb80754a0 0x7fcfb80758b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:11.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.851+0000 7fcfbfacd700 1 --2- 172.21.15.133:0/1787378341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfb80766f0 0x7fcfb8076b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:11.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.851+0000 7fcfbfacd700 1 -- 172.21.15.133:0/1787378341 >> 172.21.15.133:0/1787378341 conn(0x7fcfb80fdbd0 msgr2=0x7fcfb8100020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:11.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.851+0000 7fcfbfacd700 1 -- 172.21.15.133:0/1787378341 shutdown_connections 2023-11-27T15:15:11.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.851+0000 7fcfbfacd700 1 -- 172.21.15.133:0/1787378341 wait complete. 2023-11-27T15:15:11.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.852+0000 7fcfbfacd700 1 Processor -- start 2023-11-27T15:15:11.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.852+0000 7fcfbfacd700 1 -- start start 2023-11-27T15:15:11.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.852+0000 7fcfbfacd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb80754a0 0x7fcfb819c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:11.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.852+0000 7fcfbfacd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfb80766f0 0x7fcfb819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:11.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.853+0000 7fcfbfacd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfb80770a0 0x7fcfb81a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:11.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.853+0000 7fcfbfacd700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb81a41c0 con 0x7fcfb80770a0 2023-11-27T15:15:11.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.853+0000 7fcfbfacd700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb81a4300 con 0x7fcfb80766f0 2023-11-27T15:15:11.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.853+0000 7fcfbfacd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb81a4490 con 0x7fcfb80754a0 2023-11-27T15:15:11.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.853+0000 7fcfbd869700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb80754a0 0x7fcfb819c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:11.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.853+0000 7fcfbd869700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb80754a0 0x7fcfb819c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:55324/0 (socket says 172.21.15.133:55324) 2023-11-27T15:15:11.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.853+0000 7fcfbd869700 1 -- 172.21.15.133:0/2891832509 learned_addr learned my addr 172.21.15.133:0/2891832509 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:15:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.853+0000 7fcfbd068700 1 --2- 172.21.15.133:0/2891832509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfb80766f0 0x7fcfb819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.853+0000 7fcfbe06a700 1 --2- 172.21.15.133:0/2891832509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfb80770a0 0x7fcfb81a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.853+0000 7fcfbd869700 1 -- 172.21.15.133:0/2891832509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfb80766f0 msgr2=0x7fcfb819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.853+0000 7fcfbd869700 1 --2- 172.21.15.133:0/2891832509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfb80766f0 0x7fcfb819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.853+0000 7fcfbd869700 1 -- 172.21.15.133:0/2891832509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfb80770a0 msgr2=0x7fcfb81a3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.853+0000 7fcfbd869700 1 --2- 172.21.15.133:0/2891832509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfb80770a0 0x7fcfb81a3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.853+0000 7fcfbd869700 1 -- 172.21.15.133:0/2891832509 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb40095f0 con 0x7fcfb80754a0 2023-11-27T15:15:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.853+0000 7fcfbe06a700 1 --2- 172.21.15.133:0/2891832509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfb80770a0 0x7fcfb81a3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:15:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.853+0000 7fcfbd869700 1 --2- 172.21.15.133:0/2891832509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb80754a0 0x7fcfb819c680 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4004770 tx=0x7fcfb4009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.854+0000 7fcfaeffd700 1 -- 172.21.15.133:0/2891832509 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb401d070 con 0x7fcfb80754a0 2023-11-27T15:15:11.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.854+0000 7fcfaeffd700 1 -- 172.21.15.133:0/2891832509 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfb40054f0 con 0x7fcfb80754a0 2023-11-27T15:15:11.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.854+0000 7fcfaeffd700 1 -- 172.21.15.133:0/2891832509 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4004c70 con 0x7fcfb80754a0 2023-11-27T15:15:11.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.854+0000 7fcfbfacd700 1 -- 172.21.15.133:0/2891832509 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb8112390 con 0x7fcfb80754a0 2023-11-27T15:15:11.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.854+0000 7fcfbfacd700 1 -- 172.21.15.133:0/2891832509 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb8112200 con 0x7fcfb80754a0 2023-11-27T15:15:11.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.855+0000 7fcfaeffd700 1 -- 172.21.15.133:0/2891832509 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 13) v1 ==== 45540+0+0 (secure 0 0 0) 0x7fcfb4005030 con 0x7fcfb80754a0 2023-11-27T15:15:11.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.855+0000 7fcfbfacd700 1 -- 172.21.15.133:0/2891832509 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb810efb0 con 0x7fcfb80754a0 2023-11-27T15:15:11.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.855+0000 7fcfaeffd700 1 --2- 172.21.15.133:0/2891832509 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcfa40389e0 0x7fcfa403ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:11.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.855+0000 7fcfaeffd700 1 -- 172.21.15.133:0/2891832509 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcfb404c610 con 0x7fcfb80754a0 2023-11-27T15:15:11.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.856+0000 7fcfbd068700 1 --2- 172.21.15.133:0/2891832509 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcfa40389e0 0x7fcfa403ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:11.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.856+0000 7fcfbd068700 1 --2- 172.21.15.133:0/2891832509 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcfa40389e0 0x7fcfa403ae90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009c60 tx=0x7fcfa8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:11.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:11.860+0000 7fcfaeffd700 1 -- 172.21.15.133:0/2891832509 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcfb4004dd0 con 0x7fcfb80754a0 2023-11-27T15:15:12.088 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:15:11 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z[133365]: 2023-11-27T15:15:11.819+0000 7f679f226200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-27T15:15:12.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:12.085+0000 7fcfbfacd700 1 -- 172.21.15.133:0/2891832509 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfb8061cb0 con 0x7fcfa40389e0 2023-11-27T15:15:12.153 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:11 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:11.729+0000 7fc092bc0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-27T15:15:12.692 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:12.427+0000 7fc092bc0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-27T15:15:12.692 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:12.553+0000 7fc092bc0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-27T15:15:12.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:12.703+0000 7fcfaeffd700 1 -- 172.21.15.133:0/2891832509 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mgrmap(e 14) v1 ==== 90600+0+0 (secure 0 0 0) 0x7fcfb400fc10 con 0x7fcfb80754a0 2023-11-27T15:15:13.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:12 smithi133 ceph-mon[97860]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:13.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:13.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:12 smithi133 ceph-mon[97860]: Standby manager daemon z started 2023-11-27T15:15:13.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:12 smithi133 ceph-mon[97860]: from='mgr.? 172.21.15.177:0/292892710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-27T15:15:13.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:12 smithi133 ceph-mon[97860]: from='mgr.? 172.21.15.177:0/292892710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:15:13.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:12 smithi133 ceph-mon[97860]: from='mgr.? 172.21.15.177:0/292892710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-27T15:15:13.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:12 smithi133 ceph-mon[97860]: from='mgr.? 172.21.15.177:0/292892710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:15:13.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:13.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:13.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:13.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:13.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:13.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:12 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:15:13.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:13.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:12 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:15:13.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:12 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:15:13.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:12 smithi133 ceph-mon[97860]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:15:13.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:15:13.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:15:13.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:13.068 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:12.740+0000 7fc092bc0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-27T15:15:13.068 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:12.852+0000 7fc092bc0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-27T15:15:13.068 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-mon[131110]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:13.068 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:13.068 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-mon[131110]: Standby manager daemon z started 2023-11-27T15:15:13.069 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-mon[131110]: from='mgr.? 172.21.15.177:0/292892710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-27T15:15:13.069 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-mon[131110]: from='mgr.? 172.21.15.177:0/292892710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:15:13.069 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-mon[131110]: from='mgr.? 172.21.15.177:0/292892710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-27T15:15:13.069 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-mon[131110]: from='mgr.? 172.21.15.177:0/292892710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:15:13.069 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:13.069 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:13.069 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:13.069 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:13.069 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:13.069 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:15:13.070 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:13.070 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:15:13.070 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:15:13.070 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-mon[131110]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:15:13.070 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:15:13.070 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:15:13.070 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:13.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:12 smithi177 ceph-mon[131504]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:13.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:13.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:12 smithi177 ceph-mon[131504]: Standby manager daemon z started 2023-11-27T15:15:13.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:12 smithi177 ceph-mon[131504]: from='mgr.? 172.21.15.177:0/292892710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-11-27T15:15:13.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:12 smithi177 ceph-mon[131504]: from='mgr.? 172.21.15.177:0/292892710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:15:13.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:12 smithi177 ceph-mon[131504]: from='mgr.? 172.21.15.177:0/292892710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-11-27T15:15:13.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:12 smithi177 ceph-mon[131504]: from='mgr.? 172.21.15.177:0/292892710' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:15:13.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:13.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:13.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:13.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:13.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:13.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:12 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:15:13.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:13.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:12 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:15:13.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:12 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:15:13.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:12 smithi177 ceph-mon[131504]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:15:13.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:15:13.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:15:13.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:13.403 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:13 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:13.064+0000 7fc092bc0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-27T15:15:13.889 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:13 smithi133 ceph-mon[97860]: mgrmap e14: x(active, since 71s), standbys: z 2023-11-27T15:15:13.889 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:13 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-27T15:15:14.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:13 smithi177 ceph-mon[131504]: mgrmap e14: x(active, since 71s), standbys: z 2023-11-27T15:15:14.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:13 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-27T15:15:14.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:13 smithi174 ceph-mon[131110]: mgrmap e14: x(active, since 71s), standbys: z 2023-11-27T15:15:14.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:13 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-11-27T15:15:14.153 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:15:13 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y[132569]: 2023-11-27T15:15:13.913+0000 7fc092bc0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-27T15:15:14.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:14.722+0000 7fcfaeffd700 1 -- 172.21.15.133:0/2891832509 <== mon.1 v2:172.21.15.177:3300/0 8 ==== mgrmap(e 15) v1 ==== 135660+0+0 (secure 0 0 0) 0x7fcfb407e440 con 0x7fcfb80754a0 2023-11-27T15:15:15.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:14 smithi133 ceph-mon[97860]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:15.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:14 smithi133 ceph-mon[97860]: Standby manager daemon y started 2023-11-27T15:15:15.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:14 smithi133 ceph-mon[97860]: from='mgr.? 172.21.15.174:0/1358534639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-27T15:15:15.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:14 smithi133 ceph-mon[97860]: from='mgr.? 172.21.15.174:0/1358534639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:15:15.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:14 smithi133 ceph-mon[97860]: from='mgr.? 172.21.15.174:0/1358534639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-27T15:15:15.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:14 smithi133 ceph-mon[97860]: from='mgr.? 172.21.15.174:0/1358534639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:15:15.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:14 smithi177 ceph-mon[131504]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:15.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:14 smithi177 ceph-mon[131504]: Standby manager daemon y started 2023-11-27T15:15:15.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:14 smithi177 ceph-mon[131504]: from='mgr.? 172.21.15.174:0/1358534639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-27T15:15:15.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:14 smithi177 ceph-mon[131504]: from='mgr.? 172.21.15.174:0/1358534639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:15:15.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:14 smithi177 ceph-mon[131504]: from='mgr.? 172.21.15.174:0/1358534639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-27T15:15:15.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:14 smithi177 ceph-mon[131504]: from='mgr.? 172.21.15.174:0/1358534639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:15:15.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:14 smithi174 ceph-mon[131110]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:15.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:14 smithi174 ceph-mon[131110]: Standby manager daemon y started 2023-11-27T15:15:15.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:14 smithi174 ceph-mon[131110]: from='mgr.? 172.21.15.174:0/1358534639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-11-27T15:15:15.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:14 smithi174 ceph-mon[131110]: from='mgr.? 172.21.15.174:0/1358534639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-11-27T15:15:15.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:14 smithi174 ceph-mon[131110]: from='mgr.? 172.21.15.174:0/1358534639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-11-27T15:15:15.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:14 smithi174 ceph-mon[131110]: from='mgr.? 172.21.15.174:0/1358534639' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-11-27T15:15:16.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:15 smithi133 ceph-mon[97860]: mgrmap e15: x(active, since 73s), standbys: z, y 2023-11-27T15:15:16.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:15 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-27T15:15:16.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:15 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1746889615' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ebdea23-8131-43b6-b3b1-06d1337bf063"}]: dispatch 2023-11-27T15:15:16.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:15 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1746889615' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ebdea23-8131-43b6-b3b1-06d1337bf063"}]': finished 2023-11-27T15:15:16.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:15 smithi133 ceph-mon[97860]: osdmap e5: 1 total, 0 up, 1 in 2023-11-27T15:15:16.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:15 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:15:16.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:15 smithi177 ceph-mon[131504]: mgrmap e15: x(active, since 73s), standbys: z, y 2023-11-27T15:15:16.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:15 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-27T15:15:16.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:15 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1746889615' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ebdea23-8131-43b6-b3b1-06d1337bf063"}]: dispatch 2023-11-27T15:15:16.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:15 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1746889615' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ebdea23-8131-43b6-b3b1-06d1337bf063"}]': finished 2023-11-27T15:15:16.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:15 smithi177 ceph-mon[131504]: osdmap e5: 1 total, 0 up, 1 in 2023-11-27T15:15:16.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:15 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:15:16.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:15 smithi174 ceph-mon[131110]: mgrmap e15: x(active, since 73s), standbys: z, y 2023-11-27T15:15:16.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:15 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-11-27T15:15:16.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:15 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1746889615' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ebdea23-8131-43b6-b3b1-06d1337bf063"}]: dispatch 2023-11-27T15:15:16.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:15 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1746889615' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ebdea23-8131-43b6-b3b1-06d1337bf063"}]': finished 2023-11-27T15:15:16.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:15 smithi174 ceph-mon[131110]: osdmap e5: 1 total, 0 up, 1 in 2023-11-27T15:15:16.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:15 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:15:17.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:16 smithi133 ceph-mon[97860]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:17.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:16 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2106326589' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:15:17.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:16 smithi177 ceph-mon[131504]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:17.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:16 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2106326589' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:15:17.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:16 smithi174 ceph-mon[131110]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:17.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:16 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2106326589' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:15:19.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:19 smithi133 ceph-mon[97860]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:19.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:19 smithi177 ceph-mon[131504]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:19.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:19 smithi174 ceph-mon[131110]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:21.196 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:21 smithi133 ceph-mon[97860]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:21.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:21 smithi177 ceph-mon[131504]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:21.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:21 smithi174 ceph-mon[131110]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:23.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:23 smithi133 ceph-mon[97860]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:23.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:23 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-11-27T15:15:23.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:23 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:23.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:23 smithi177 ceph-mon[131504]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:23.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:23 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-11-27T15:15:23.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:23 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:23.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:23 smithi174 ceph-mon[131110]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:23.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:23 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-11-27T15:15:23.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:23 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:24.170 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:24 smithi133 ceph-mon[97860]: Deploying daemon osd.0 on smithi133 2023-11-27T15:15:24.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:24 smithi177 ceph-mon[131504]: Deploying daemon osd.0 on smithi133 2023-11-27T15:15:24.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:24 smithi174 ceph-mon[131110]: Deploying daemon osd.0 on smithi133 2023-11-27T15:15:25.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:25 smithi133 ceph-mon[97860]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:25.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:25 smithi177 ceph-mon[131504]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:25.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:25 smithi174 ceph-mon[131110]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:27.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:27 smithi177 ceph-mon[131504]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:27.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:27 smithi174 ceph-mon[131110]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:27.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:27 smithi133 ceph-mon[97860]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:29.108 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:29 smithi133 ceph-mon[97860]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:29.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:29 smithi177 ceph-mon[131504]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:29.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:29 smithi174 ceph-mon[131110]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:30.466 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:30 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:30.467 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:30 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:30.467 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:30 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:30.467 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:30 smithi133 ceph-mon[97860]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:30.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:30 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:30.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:30 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:30.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:30 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:30.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:30 smithi177 ceph-mon[131504]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:30.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:30 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:30.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:30 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:30.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:30 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:30.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:30 smithi174 ceph-mon[131110]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:32.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:32.800+0000 7fcfaeffd700 1 -- 172.21.15.133:0/2891832509 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcfb8061cb0 con 0x7fcfa40389e0 2023-11-27T15:15:32.804 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 0 on host 'smithi133' 2023-11-27T15:15:32.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:32.802+0000 7fcfbfacd700 1 -- 172.21.15.133:0/2891832509 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcfa40389e0 msgr2=0x7fcfa403ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:32.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:32.802+0000 7fcfbfacd700 1 --2- 172.21.15.133:0/2891832509 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcfa40389e0 0x7fcfa403ae90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009c60 tx=0x7fcfa8009260 comp rx=0 tx=0).stop 2023-11-27T15:15:32.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:32.802+0000 7fcfbfacd700 1 -- 172.21.15.133:0/2891832509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb80754a0 msgr2=0x7fcfb819c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:32.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:32.802+0000 7fcfbfacd700 1 --2- 172.21.15.133:0/2891832509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb80754a0 0x7fcfb819c680 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4004770 tx=0x7fcfb4009580 comp rx=0 tx=0).stop 2023-11-27T15:15:32.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:32.802+0000 7fcfbfacd700 1 -- 172.21.15.133:0/2891832509 shutdown_connections 2023-11-27T15:15:32.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:32.802+0000 7fcfbfacd700 1 --2- 172.21.15.133:0/2891832509 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcfa40389e0 0x7fcfa403ae90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:32.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:32.802+0000 7fcfbfacd700 1 --2- 172.21.15.133:0/2891832509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfb80770a0 0x7fcfb81a3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:32.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:32.802+0000 7fcfbfacd700 1 --2- 172.21.15.133:0/2891832509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfb80754a0 0x7fcfb819c680 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:32.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:32.802+0000 7fcfbfacd700 1 --2- 172.21.15.133:0/2891832509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfb80766f0 0x7fcfb819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:32.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:32.802+0000 7fcfbfacd700 1 -- 172.21.15.133:0/2891832509 >> 172.21.15.133:0/2891832509 conn(0x7fcfb80fdbd0 msgr2=0x7fcfb80ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:32.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:32.802+0000 7fcfbfacd700 1 -- 172.21.15.133:0/2891832509 shutdown_connections 2023-11-27T15:15:32.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:15:32.802+0000 7fcfbfacd700 1 -- 172.21.15.133:0/2891832509 wait complete. 2023-11-27T15:15:33.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:32 smithi133 ceph-mon[97860]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:33.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:32 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:33.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:32 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:33.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:32 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:33.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:32 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:33.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:32 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:15:33.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:32 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:33.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:32 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:15:33.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:32 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:15:33.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:32 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:33.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:32 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:33.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:32 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:33.293 DEBUG:teuthology.orchestra.run.smithi133:osd.0> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.0.service 2023-11-27T15:15:33.299 INFO:tasks.cephadm:Deploying osd.1 on smithi174 with /dev/vg_nvme/lv_4... 2023-11-27T15:15:33.299 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-11-27T15:15:33.331 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:32 smithi174 ceph-mon[131110]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:33.331 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:32 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:33.331 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:32 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:33.331 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:32 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:33.331 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:32 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:33.331 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:32 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:15:33.331 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:32 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:33.331 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:32 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:15:33.332 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:32 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:15:33.332 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:32 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:33.332 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:32 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:33.332 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:32 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:33.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:32 smithi177 ceph-mon[131504]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:33.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:32 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:33.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:32 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:33.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:32 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:33.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:32 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:33.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:32 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:15:33.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:32 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:33.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:32 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:15:33.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:32 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:15:33.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:32 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:33.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:32 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:33.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:32 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:33.370 INFO:journalctl@ceph.osd.0.smithi133.stdout:-- Logs begin at Mon 2023-11-27 14:54:33 UTC. -- 2023-11-27T15:15:33.944 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:15:34.200 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:33 smithi174 ceph-mon[131110]: from='osd.0 [v2:172.21.15.133:6802/3850463495,v1:172.21.15.133:6803/3850463495]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-27T15:15:34.200 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:33 smithi174 ceph-mon[131110]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-27T15:15:34.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:33 smithi133 ceph-mon[97860]: from='osd.0 [v2:172.21.15.133:6802/3850463495,v1:172.21.15.133:6803/3850463495]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-27T15:15:34.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:33 smithi133 ceph-mon[97860]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-27T15:15:34.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:33 smithi177 ceph-mon[131504]: from='osd.0 [v2:172.21.15.133:6802/3850463495,v1:172.21.15.133:6803/3850463495]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-27T15:15:34.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:33 smithi177 ceph-mon[131504]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-11-27T15:15:35.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:34 smithi133 ceph-mon[97860]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:35.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:34 smithi133 ceph-mon[97860]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-11-27T15:15:35.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:34 smithi133 ceph-mon[97860]: osdmap e6: 1 total, 0 up, 1 in 2023-11-27T15:15:35.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:34 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:15:35.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:34 smithi133 ceph-mon[97860]: from='osd.0 [v2:172.21.15.133:6802/3850463495,v1:172.21.15.133:6803/3850463495]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-11-27T15:15:35.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:34 smithi133 ceph-mon[97860]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-11-27T15:15:35.279 INFO:journalctl@ceph.osd.0.smithi133.stdout:Nov 27 15:15:34 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-0[110842]: 2023-11-27T15:15:34.950+0000 7f02a350a700 -1 osd.0 0 waiting for initial osdmap 2023-11-27T15:15:35.279 INFO:journalctl@ceph.osd.0.smithi133.stdout:Nov 27 15:15:34 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-0[110842]: 2023-11-27T15:15:34.954+0000 7f029d2fb700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:15:35.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:34 smithi177 ceph-mon[131504]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:35.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:34 smithi177 ceph-mon[131504]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-11-27T15:15:35.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:34 smithi177 ceph-mon[131504]: osdmap e6: 1 total, 0 up, 1 in 2023-11-27T15:15:35.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:34 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:15:35.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:34 smithi177 ceph-mon[131504]: from='osd.0 [v2:172.21.15.133:6802/3850463495,v1:172.21.15.133:6803/3850463495]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-11-27T15:15:35.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:34 smithi177 ceph-mon[131504]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-11-27T15:15:35.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:34 smithi174 ceph-mon[131110]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:35.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:34 smithi174 ceph-mon[131110]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-11-27T15:15:35.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:34 smithi174 ceph-mon[131110]: osdmap e6: 1 total, 0 up, 1 in 2023-11-27T15:15:35.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:34 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:15:35.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:34 smithi174 ceph-mon[131110]: from='osd.0 [v2:172.21.15.133:6802/3850463495,v1:172.21.15.133:6803/3850463495]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-11-27T15:15:35.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:34 smithi174 ceph-mon[131110]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-11-27T15:15:35.486 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-11-27T15:15:35.486 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-11-27T15:15:35.486 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2023-11-27T15:15:35.486 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2023-11-27T15:15:35.486 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126386 s, 830 MB/s 2023-11-27T15:15:35.486 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2023-11-27T15:15:35.887 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi174:vg_nvme/lv_4 2023-11-27T15:15:36.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:35 smithi133 ceph-mon[97860]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-11-27T15:15:36.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:35 smithi133 ceph-mon[97860]: osdmap e7: 1 total, 0 up, 1 in 2023-11-27T15:15:36.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:35 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:15:36.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:35 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:15:36.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:35 smithi174 ceph-mon[131110]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-11-27T15:15:36.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:35 smithi174 ceph-mon[131110]: osdmap e7: 1 total, 0 up, 1 in 2023-11-27T15:15:36.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:35 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:15:36.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:35 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:15:36.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:35 smithi177 ceph-mon[131504]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-11-27T15:15:36.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:35 smithi177 ceph-mon[131504]: osdmap e7: 1 total, 0 up, 1 in 2023-11-27T15:15:36.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:35 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:15:36.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:35 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:15:36.566 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:15:37.091 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:36 smithi174 ceph-mon[131110]: purged_snaps scrub starts 2023-11-27T15:15:37.091 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:36 smithi174 ceph-mon[131110]: purged_snaps scrub ok 2023-11-27T15:15:37.092 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:36 smithi174 ceph-mon[131110]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:37.092 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:36 smithi174 ceph-mon[131110]: osd.0 [v2:172.21.15.133:6802/3850463495,v1:172.21.15.133:6803/3850463495] boot 2023-11-27T15:15:37.092 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:36 smithi174 ceph-mon[131110]: osdmap e8: 1 total, 1 up, 1 in 2023-11-27T15:15:37.092 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:36 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:15:37.092 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:36 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:37.092 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:36 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:37.092 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:36 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:37.092 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:36 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:37.092 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:36 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:37.092 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:36 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:37.092 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:36 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:37.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:36 smithi133 ceph-mon[97860]: purged_snaps scrub starts 2023-11-27T15:15:37.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:36 smithi133 ceph-mon[97860]: purged_snaps scrub ok 2023-11-27T15:15:37.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:36 smithi133 ceph-mon[97860]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:37.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:36 smithi133 ceph-mon[97860]: osd.0 [v2:172.21.15.133:6802/3850463495,v1:172.21.15.133:6803/3850463495] boot 2023-11-27T15:15:37.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:36 smithi133 ceph-mon[97860]: osdmap e8: 1 total, 1 up, 1 in 2023-11-27T15:15:37.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:36 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:15:37.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:36 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:37.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:36 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:37.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:36 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:37.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:36 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:37.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:36 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:37.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:36 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:37.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:36 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:37.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:36 smithi177 ceph-mon[131504]: purged_snaps scrub starts 2023-11-27T15:15:37.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:36 smithi177 ceph-mon[131504]: purged_snaps scrub ok 2023-11-27T15:15:37.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:36 smithi177 ceph-mon[131504]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-11-27T15:15:37.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:36 smithi177 ceph-mon[131504]: osd.0 [v2:172.21.15.133:6802/3850463495,v1:172.21.15.133:6803/3850463495] boot 2023-11-27T15:15:37.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:36 smithi177 ceph-mon[131504]: osdmap e8: 1 total, 1 up, 1 in 2023-11-27T15:15:37.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:36 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-11-27T15:15:37.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:36 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:37.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:36 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:37.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:36 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:37.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:36 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:37.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:36 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:37.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:36 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:37.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:36 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:37.797 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.791+0000 7f90cab15700 1 -- 172.21.15.174:0/3998477824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90c4105fb0 msgr2=0x7f90c4108410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:37.798 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.791+0000 7f90cab15700 1 --2- 172.21.15.174:0/3998477824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90c4105fb0 0x7f90c4108410 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00b0a0 tx=0x7f90bc00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:15:37.798 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.791+0000 7f90cab15700 1 -- 172.21.15.174:0/3998477824 shutdown_connections 2023-11-27T15:15:37.798 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.791+0000 7f90cab15700 1 --2- 172.21.15.174:0/3998477824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90c4105fb0 0x7f90c4108410 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:37.798 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.791+0000 7f90cab15700 1 --2- 172.21.15.174:0/3998477824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90c4100d70 0x7f90c4103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:37.798 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.791+0000 7f90cab15700 1 --2- 172.21.15.174:0/3998477824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90c4103690 0x7f90c4105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:37.798 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.791+0000 7f90cab15700 1 -- 172.21.15.174:0/3998477824 >> 172.21.15.174:0/3998477824 conn(0x7f90c40fa7f0 msgr2=0x7f90c40fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:37.798 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.791+0000 7f90cab15700 1 -- 172.21.15.174:0/3998477824 shutdown_connections 2023-11-27T15:15:37.798 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.791+0000 7f90cab15700 1 -- 172.21.15.174:0/3998477824 wait complete. 2023-11-27T15:15:37.798 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.792+0000 7f90cab15700 1 Processor -- start 2023-11-27T15:15:37.798 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.792+0000 7f90cab15700 1 -- start start 2023-11-27T15:15:37.799 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.792+0000 7f90cab15700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90c4100d70 0x7f90c4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:37.799 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.792+0000 7f90cab15700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90c4103690 0x7f90c4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:37.799 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.792+0000 7f90cab15700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90c4105fb0 0x7f90c419f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:37.799 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.792+0000 7f90cab15700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c419ffa0 con 0x7f90c4103690 2023-11-27T15:15:37.799 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.792+0000 7f90cab15700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c41a00e0 con 0x7f90c4105fb0 2023-11-27T15:15:37.799 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.792+0000 7f90cab15700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c41a0270 con 0x7f90c4100d70 2023-11-27T15:15:37.799 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.792+0000 7f90c90b2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90c4105fb0 0x7f90c419f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:37.799 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.792+0000 7f90c90b2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90c4105fb0 0x7f90c419f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:48306/0 (socket says 172.21.15.174:48306) 2023-11-27T15:15:37.799 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.792+0000 7f90c90b2700 1 -- 172.21.15.174:0/914723881 learned_addr learned my addr 172.21.15.174:0/914723881 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-27T15:15:37.799 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.792+0000 7f90c88b1700 1 --2- 172.21.15.174:0/914723881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90c4100d70 0x7f90c4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:37.799 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.792+0000 7f90c90b2700 1 -- 172.21.15.174:0/914723881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90c4100d70 msgr2=0x7f90c4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:37.799 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.792+0000 7f90c90b2700 1 --2- 172.21.15.174:0/914723881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90c4100d70 0x7f90c4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:37.799 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.792+0000 7f90c90b2700 1 -- 172.21.15.174:0/914723881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90c4103690 msgr2=0x7f90c4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:37.800 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.792+0000 7f90c90b2700 1 --2- 172.21.15.174:0/914723881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90c4103690 0x7f90c4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:37.800 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.792+0000 7f90c90b2700 1 -- 172.21.15.174:0/914723881 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90bc009d00 con 0x7f90c4105fb0 2023-11-27T15:15:37.800 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.793+0000 7f90c90b2700 1 --2- 172.21.15.174:0/914723881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90c4105fb0 0x7f90c419f9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0062a0 tx=0x7f90bc0094e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:37.800 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.793+0000 7f90c1ffb700 1 -- 172.21.15.174:0/914723881 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc00e040 con 0x7f90c4105fb0 2023-11-27T15:15:37.800 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.793+0000 7f90c1ffb700 1 -- 172.21.15.174:0/914723881 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90bc004de0 con 0x7f90c4105fb0 2023-11-27T15:15:37.800 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.793+0000 7f90cab15700 1 -- 172.21.15.174:0/914723881 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c410bcb0 con 0x7f90c4105fb0 2023-11-27T15:15:37.800 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.793+0000 7f90c1ffb700 1 -- 172.21.15.174:0/914723881 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90bc01d480 con 0x7f90c4105fb0 2023-11-27T15:15:37.800 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.793+0000 7f90cab15700 1 -- 172.21.15.174:0/914723881 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c410bb20 con 0x7f90c4105fb0 2023-11-27T15:15:37.801 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.794+0000 7f90cab15700 1 -- 172.21.15.174:0/914723881 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c4107b20 con 0x7f90c4105fb0 2023-11-27T15:15:37.801 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.795+0000 7f90c1ffb700 1 -- 172.21.15.174:0/914723881 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 15) v1 ==== 135660+0+0 (secure 0 0 0) 0x7f90bc004020 con 0x7f90c4105fb0 2023-11-27T15:15:37.801 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.796+0000 7f90c1ffb700 1 --2- 172.21.15.174:0/914723881 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f90b40a1140 0x7f90b40a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:37.801 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.796+0000 7f90c1ffb700 1 -- 172.21.15.174:0/914723881 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f90bc0cc910 con 0x7f90c4105fb0 2023-11-27T15:15:37.801 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.796+0000 7f90c88b1700 1 --2- 172.21.15.174:0/914723881 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f90b40a1140 0x7f90b40a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:37.801 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.797+0000 7f90c88b1700 1 --2- 172.21.15.174:0/914723881 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f90b40a1140 0x7f90b40a35f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009a10 tx=0x7f90b8009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:37.805 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.800+0000 7f90c1ffb700 1 -- 172.21.15.174:0/914723881 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f90bc097ac0 con 0x7f90c4105fb0 2023-11-27T15:15:37.995 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:37.991+0000 7f90cab15700 1 -- 172.21.15.174:0/914723881 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f90c4061ae0 con 0x7f90b40a1140 2023-11-27T15:15:38.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:38 smithi133 ceph-mon[97860]: Detected new or changed devices on smithi133 2023-11-27T15:15:38.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:38 smithi133 ceph-mon[97860]: Adjusting osd_memory_target on smithi133 to 17271M 2023-11-27T15:15:38.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:38 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:15:38.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:38 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:15:38.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:38 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:15:38.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:38 smithi133 ceph-mon[97860]: osdmap e9: 1 total, 1 up, 1 in 2023-11-27T15:15:38.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:38 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:15:38.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:38 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:15:38.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:38 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:38.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:38 smithi177 ceph-mon[131504]: Detected new or changed devices on smithi133 2023-11-27T15:15:38.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:38 smithi177 ceph-mon[131504]: Adjusting osd_memory_target on smithi133 to 17271M 2023-11-27T15:15:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:38 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:15:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:38 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:15:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:38 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:15:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:38 smithi177 ceph-mon[131504]: osdmap e9: 1 total, 1 up, 1 in 2023-11-27T15:15:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:38 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:15:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:38 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:15:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:38 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:38 smithi174 ceph-mon[131110]: Detected new or changed devices on smithi133 2023-11-27T15:15:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:38 smithi174 ceph-mon[131110]: Adjusting osd_memory_target on smithi133 to 17271M 2023-11-27T15:15:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:38 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:15:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:38 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:15:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:38 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:15:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:38 smithi174 ceph-mon[131110]: osdmap e9: 1 total, 1 up, 1 in 2023-11-27T15:15:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:38 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:15:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:38 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:15:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:38 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:39.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:39 smithi174 ceph-mon[131110]: pgmap v47: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:39.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:39 smithi174 ceph-mon[131110]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:15:39.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:39 smithi177 ceph-mon[131504]: pgmap v47: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:39.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:39 smithi177 ceph-mon[131504]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:15:39.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:39 smithi133 ceph-mon[97860]: pgmap v47: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:39.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:39 smithi133 ceph-mon[97860]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:15:41.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:41 smithi177 ceph-mon[131504]: pgmap v48: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:41.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:41 smithi177 ceph-mon[131504]: from='client.? 172.21.15.174:0/381672191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc933082-b3e3-43d5-b110-c4d118df4418"}]: dispatch 2023-11-27T15:15:41.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:41 smithi177 ceph-mon[131504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc933082-b3e3-43d5-b110-c4d118df4418"}]: dispatch 2023-11-27T15:15:41.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:41 smithi177 ceph-mon[131504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc933082-b3e3-43d5-b110-c4d118df4418"}]': finished 2023-11-27T15:15:41.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:41 smithi177 ceph-mon[131504]: osdmap e10: 2 total, 1 up, 2 in 2023-11-27T15:15:41.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:41 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:15:41.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:41 smithi174 ceph-mon[131110]: pgmap v48: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:41.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:41 smithi174 ceph-mon[131110]: from='client.? 172.21.15.174:0/381672191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc933082-b3e3-43d5-b110-c4d118df4418"}]: dispatch 2023-11-27T15:15:41.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:41 smithi174 ceph-mon[131110]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc933082-b3e3-43d5-b110-c4d118df4418"}]: dispatch 2023-11-27T15:15:41.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:41 smithi174 ceph-mon[131110]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc933082-b3e3-43d5-b110-c4d118df4418"}]': finished 2023-11-27T15:15:41.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:41 smithi174 ceph-mon[131110]: osdmap e10: 2 total, 1 up, 2 in 2023-11-27T15:15:41.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:41 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:15:41.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:41 smithi133 ceph-mon[97860]: pgmap v48: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:41.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:41 smithi133 ceph-mon[97860]: from='client.? 172.21.15.174:0/381672191' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc933082-b3e3-43d5-b110-c4d118df4418"}]: dispatch 2023-11-27T15:15:41.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:41 smithi133 ceph-mon[97860]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc933082-b3e3-43d5-b110-c4d118df4418"}]: dispatch 2023-11-27T15:15:41.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:41 smithi133 ceph-mon[97860]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc933082-b3e3-43d5-b110-c4d118df4418"}]': finished 2023-11-27T15:15:41.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:41 smithi133 ceph-mon[97860]: osdmap e10: 2 total, 1 up, 2 in 2023-11-27T15:15:41.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:41 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:15:42.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:42 smithi177 ceph-mon[131504]: from='client.? 172.21.15.174:0/2847132314' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:15:42.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:42 smithi174 ceph-mon[131110]: from='client.? 172.21.15.174:0/2847132314' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:15:42.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:42 smithi133 ceph-mon[97860]: from='client.? 172.21.15.174:0/2847132314' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:15:43.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:43 smithi177 ceph-mon[131504]: pgmap v50: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:43.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:43 smithi174 ceph-mon[131110]: pgmap v50: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:43.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:43 smithi133 ceph-mon[97860]: pgmap v50: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:45.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:45 smithi177 ceph-mon[131504]: pgmap v51: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:45.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:45 smithi174 ceph-mon[131110]: pgmap v51: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:45.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:45 smithi133 ceph-mon[97860]: pgmap v51: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:47.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:47 smithi177 ceph-mon[131504]: pgmap v52: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:47.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:47 smithi174 ceph-mon[131110]: pgmap v52: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:47.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:47 smithi133 ceph-mon[97860]: pgmap v52: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:48.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:48 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-11-27T15:15:48.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:48 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:48.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:48 smithi177 ceph-mon[131504]: Deploying daemon osd.1 on smithi174 2023-11-27T15:15:48.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:48 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-11-27T15:15:48.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:48 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:48.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:48 smithi174 ceph-mon[131110]: Deploying daemon osd.1 on smithi174 2023-11-27T15:15:48.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:48 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-11-27T15:15:48.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:48 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:48.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:48 smithi133 ceph-mon[97860]: Deploying daemon osd.1 on smithi174 2023-11-27T15:15:49.234 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:49 smithi174 ceph-mon[131110]: pgmap v53: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:49.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:49 smithi177 ceph-mon[131504]: pgmap v53: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:49.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:49 smithi133 ceph-mon[97860]: pgmap v53: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:51.331 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:51 smithi174 ceph-mon[131110]: pgmap v54: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:51.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:51 smithi177 ceph-mon[131504]: pgmap v54: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:51.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:51 smithi133 ceph-mon[97860]: pgmap v54: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:52.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:52 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:52.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:52 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:52.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:52 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:52.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:52 smithi174 ceph-mon[131110]: pgmap v55: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:52.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:52 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:52.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:52 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:52.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:52 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:52.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:52 smithi133 ceph-mon[97860]: pgmap v55: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:52.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:52 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:52.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:52 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:52.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:52 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:52.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:52 smithi177 ceph-mon[131504]: pgmap v55: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:54.460 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:54.455+0000 7f90c1ffb700 1 -- 172.21.15.174:0/914723881 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f90c4061ae0 con 0x7f90b40a1140 2023-11-27T15:15:54.460 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 1 on host 'smithi174' 2023-11-27T15:15:54.462 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:54.458+0000 7f90cab15700 1 -- 172.21.15.174:0/914723881 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f90b40a1140 msgr2=0x7f90b40a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:54.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:54.458+0000 7f90cab15700 1 --2- 172.21.15.174:0/914723881 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f90b40a1140 0x7f90b40a35f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009a10 tx=0x7f90b8009190 comp rx=0 tx=0).stop 2023-11-27T15:15:54.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:54.458+0000 7f90cab15700 1 -- 172.21.15.174:0/914723881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90c4105fb0 msgr2=0x7f90c419f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:54.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:54.458+0000 7f90cab15700 1 --2- 172.21.15.174:0/914723881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90c4105fb0 0x7f90c419f9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0062a0 tx=0x7f90bc0094e0 comp rx=0 tx=0).stop 2023-11-27T15:15:54.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:54.458+0000 7f90cab15700 1 -- 172.21.15.174:0/914723881 shutdown_connections 2023-11-27T15:15:54.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:54.458+0000 7f90cab15700 1 --2- 172.21.15.174:0/914723881 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f90b40a1140 0x7f90b40a35f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:54.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:54.458+0000 7f90cab15700 1 --2- 172.21.15.174:0/914723881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90c4105fb0 0x7f90c419f9d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:54.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:54.458+0000 7f90cab15700 1 --2- 172.21.15.174:0/914723881 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90c4100d70 0x7f90c4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:54.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:54.458+0000 7f90cab15700 1 --2- 172.21.15.174:0/914723881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90c4103690 0x7f90c4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:54.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:54.458+0000 7f90cab15700 1 -- 172.21.15.174:0/914723881 >> 172.21.15.174:0/914723881 conn(0x7f90c40fa7f0 msgr2=0x7f90c41019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:54.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:54.458+0000 7f90cab15700 1 -- 172.21.15.174:0/914723881 shutdown_connections 2023-11-27T15:15:54.463 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:15:54.458+0000 7f90cab15700 1 -- 172.21.15.174:0/914723881 wait complete. 2023-11-27T15:15:54.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:54 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:54.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:54 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:54.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:54 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:54.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:54 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:54.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:54 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:15:54.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:54 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:54.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:54 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:15:54.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:54 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:15:54.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:54 smithi133 ceph-mon[97860]: pgmap v56: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:54.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:54 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:54.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:54 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:54.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:54 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:54.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:54 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:54.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:54 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:15:54.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:54 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:54.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:54 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:15:54.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:54 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:15:54.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:54 smithi177 ceph-mon[131504]: pgmap v56: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:54.622 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:54 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:54.622 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:54 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:54.622 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:54 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:54.622 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:54 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:54.622 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:54 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:15:54.622 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:54 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:54.622 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:54 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:15:54.622 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:54 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:15:54.623 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:54 smithi174 ceph-mon[131110]: pgmap v56: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:54.856 DEBUG:teuthology.orchestra.run.smithi174:osd.1> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.1.service 2023-11-27T15:15:54.858 INFO:tasks.cephadm:Deploying osd.2 on smithi177 with /dev/vg_nvme/lv_4... 2023-11-27T15:15:54.858 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-11-27T15:15:54.889 INFO:journalctl@ceph.osd.1.smithi174.stdout:-- Logs begin at Mon 2023-11-27 14:50:40 UTC. -- 2023-11-27T15:15:55.506 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:15:55.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:55 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:55.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:55 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:55.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:55 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:55.653 INFO:journalctl@ceph.osd.1.smithi174.stdout:Nov 27 15:15:55 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-1[136171]: 2023-11-27T15:15:55.492+0000 7f502a8ab640 -1 osd.1 0 log_to_monitors true 2023-11-27T15:15:55.729 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:55 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:55.729 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:55 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:55.729 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:55 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:55.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:55 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:55.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:55 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:55.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:55 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:15:56.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:56 smithi174 ceph-mon[131110]: from='osd.1 [v2:172.21.15.174:6800/3890663971,v1:172.21.15.174:6801/3890663971]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-27T15:15:56.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:56 smithi174 ceph-mon[131110]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-27T15:15:56.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:56 smithi174 ceph-mon[131110]: pgmap v57: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:56.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:56 smithi133 ceph-mon[97860]: from='osd.1 [v2:172.21.15.174:6800/3890663971,v1:172.21.15.174:6801/3890663971]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-27T15:15:56.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:56 smithi133 ceph-mon[97860]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-27T15:15:56.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:56 smithi133 ceph-mon[97860]: pgmap v57: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:56.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:56 smithi177 ceph-mon[131504]: from='osd.1 [v2:172.21.15.174:6800/3890663971,v1:172.21.15.174:6801/3890663971]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-27T15:15:56.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:56 smithi177 ceph-mon[131504]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-11-27T15:15:56.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:56 smithi177 ceph-mon[131504]: pgmap v57: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:57.011 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-11-27T15:15:57.011 INFO:teuthology.orchestra.run.smithi177.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-11-27T15:15:57.012 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10+0 records in 2023-11-27T15:15:57.012 INFO:teuthology.orchestra.run.smithi177.stderr:10+0 records out 2023-11-27T15:15:57.012 INFO:teuthology.orchestra.run.smithi177.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0109485 s, 958 MB/s 2023-11-27T15:15:57.012 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping successful for: 2023-11-27T15:15:57.447 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi177:vg_nvme/lv_4 2023-11-27T15:15:57.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:57 smithi133 ceph-mon[97860]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-11-27T15:15:57.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:57 smithi133 ceph-mon[97860]: osdmap e11: 2 total, 1 up, 2 in 2023-11-27T15:15:57.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:15:57.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:57 smithi133 ceph-mon[97860]: from='osd.1 [v2:172.21.15.174:6800/3890663971,v1:172.21.15.174:6801/3890663971]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:15:57.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:57 smithi133 ceph-mon[97860]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:15:57.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:57 smithi133 ceph-mon[97860]: Detected new or changed devices on smithi174 2023-11-27T15:15:57.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:57.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:57.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:57.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:57 smithi133 ceph-mon[97860]: Adjusting osd_memory_target on smithi174 to 17168M 2023-11-27T15:15:57.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:57.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:57.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:57.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:57 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:15:57.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:57.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:57 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:15:57.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:57 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:15:57.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:57 smithi177 ceph-mon[131504]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-11-27T15:15:57.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:57 smithi177 ceph-mon[131504]: osdmap e11: 2 total, 1 up, 2 in 2023-11-27T15:15:57.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:15:57.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:57 smithi177 ceph-mon[131504]: from='osd.1 [v2:172.21.15.174:6800/3890663971,v1:172.21.15.174:6801/3890663971]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:15:57.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:57 smithi177 ceph-mon[131504]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:15:57.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:57 smithi177 ceph-mon[131504]: Detected new or changed devices on smithi174 2023-11-27T15:15:57.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:57.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:57.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:57.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:57 smithi177 ceph-mon[131504]: Adjusting osd_memory_target on smithi174 to 17168M 2023-11-27T15:15:57.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:57.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:57.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:57.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:57 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:15:57.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:57.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:57 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:15:57.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:57 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:15:57.903 INFO:journalctl@ceph.osd.1.smithi174.stdout:Nov 27 15:15:57 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-1[136171]: 2023-11-27T15:15:57.426+0000 7f501ff22700 -1 osd.1 0 waiting for initial osdmap 2023-11-27T15:15:57.903 INFO:journalctl@ceph.osd.1.smithi174.stdout:Nov 27 15:15:57 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-1[136171]: 2023-11-27T15:15:57.430+0000 7f501bd17700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:15:57.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:57 smithi174 ceph-mon[131110]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-11-27T15:15:57.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:57 smithi174 ceph-mon[131110]: osdmap e11: 2 total, 1 up, 2 in 2023-11-27T15:15:57.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:15:57.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:57 smithi174 ceph-mon[131110]: from='osd.1 [v2:172.21.15.174:6800/3890663971,v1:172.21.15.174:6801/3890663971]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:15:57.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:57 smithi174 ceph-mon[131110]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:15:57.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:57 smithi174 ceph-mon[131110]: Detected new or changed devices on smithi174 2023-11-27T15:15:57.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:57.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:57.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:15:57.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:57 smithi174 ceph-mon[131110]: Adjusting osd_memory_target on smithi174 to 17168M 2023-11-27T15:15:57.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:57.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:15:57.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:15:57.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:57 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:15:57.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:15:57.905 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:57 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:15:57.905 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:57 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:15:58.085 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:15:58.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:58 smithi177 ceph-mon[131504]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2023-11-27T15:15:58.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:58 smithi177 ceph-mon[131504]: osdmap e12: 2 total, 1 up, 2 in 2023-11-27T15:15:58.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:58 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:15:58.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:58 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:15:58.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:58 smithi177 ceph-mon[131504]: pgmap v60: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:58.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:58 smithi133 ceph-mon[97860]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2023-11-27T15:15:58.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:58 smithi133 ceph-mon[97860]: osdmap e12: 2 total, 1 up, 2 in 2023-11-27T15:15:58.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:58 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:15:58.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:58 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:15:58.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:58 smithi133 ceph-mon[97860]: pgmap v60: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:58.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:58 smithi174 ceph-mon[131110]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2023-11-27T15:15:58.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:58 smithi174 ceph-mon[131110]: osdmap e12: 2 total, 1 up, 2 in 2023-11-27T15:15:58.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:58 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:15:58.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:58 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:15:58.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:58 smithi174 ceph-mon[131110]: pgmap v60: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-11-27T15:15:59.261 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.255+0000 7ff625af4700 1 -- 172.21.15.177:0/2203438221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6200ffde0 msgr2=0x7ff620107690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:59.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.255+0000 7ff625af4700 1 --2- 172.21.15.177:0/2203438221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6200ffde0 0x7ff620107690 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff61800b0a0 tx=0x7ff61800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:15:59.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.255+0000 7ff625af4700 1 -- 172.21.15.177:0/2203438221 shutdown_connections 2023-11-27T15:15:59.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.255+0000 7ff625af4700 1 --2- 172.21.15.177:0/2203438221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6200ffde0 0x7ff620107690 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:59.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.255+0000 7ff625af4700 1 --2- 172.21.15.177:0/2203438221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff620100860 0x7ff620100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:59.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.255+0000 7ff625af4700 1 --2- 172.21.15.177:0/2203438221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6200ff430 0x7ff6200ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:59.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.255+0000 7ff625af4700 1 -- 172.21.15.177:0/2203438221 >> 172.21.15.177:0/2203438221 conn(0x7ff6200f9f70 msgr2=0x7ff6200fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:15:59.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.255+0000 7ff625af4700 1 -- 172.21.15.177:0/2203438221 shutdown_connections 2023-11-27T15:15:59.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.255+0000 7ff625af4700 1 -- 172.21.15.177:0/2203438221 wait complete. 2023-11-27T15:15:59.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.255+0000 7ff625af4700 1 Processor -- start 2023-11-27T15:15:59.262 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.256+0000 7ff625af4700 1 -- start start 2023-11-27T15:15:59.263 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.256+0000 7ff625af4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6200ff430 0x7ff6201960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:59.263 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.256+0000 7ff625af4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6200ffde0 0x7ff620196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:59.263 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.256+0000 7ff625af4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff620100860 0x7ff62019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:59.263 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.256+0000 7ff625af4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62019dc80 con 0x7ff6200ff430 2023-11-27T15:15:59.263 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.256+0000 7ff625af4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62019ddc0 con 0x7ff6200ffde0 2023-11-27T15:15:59.263 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.256+0000 7ff625af4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62019df50 con 0x7ff620100860 2023-11-27T15:15:59.263 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.256+0000 7ff61ffff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff620100860 0x7ff62019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:59.263 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.256+0000 7ff61ffff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff620100860 0x7ff62019d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:57824/0 (socket says 172.21.15.177:57824) 2023-11-27T15:15:59.263 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.256+0000 7ff61ffff700 1 -- 172.21.15.177:0/1393158256 learned_addr learned my addr 172.21.15.177:0/1393158256 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-27T15:15:59.263 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.256+0000 7ff61f7fe700 1 --2- 172.21.15.177:0/1393158256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6200ff430 0x7ff6201960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:59.263 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.256+0000 7ff61effd700 1 --2- 172.21.15.177:0/1393158256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6200ffde0 0x7ff620196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:59.264 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.256+0000 7ff61ffff700 1 -- 172.21.15.177:0/1393158256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6200ffde0 msgr2=0x7ff620196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:59.264 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.256+0000 7ff61ffff700 1 --2- 172.21.15.177:0/1393158256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6200ffde0 0x7ff620196630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:59.264 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.256+0000 7ff61ffff700 1 -- 172.21.15.177:0/1393158256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6200ff430 msgr2=0x7ff6201960f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:15:59.264 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.256+0000 7ff61ffff700 1 --2- 172.21.15.177:0/1393158256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6200ff430 0x7ff6201960f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:15:59.264 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.256+0000 7ff61ffff700 1 -- 172.21.15.177:0/1393158256 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff614009520 con 0x7ff620100860 2023-11-27T15:15:59.264 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.256+0000 7ff61f7fe700 1 --2- 172.21.15.177:0/1393158256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6200ff430 0x7ff6201960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:15:59.264 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.256+0000 7ff61ffff700 1 --2- 172.21.15.177:0/1393158256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff620100860 0x7ff62019d6b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff618007a60 tx=0x7ff61800bb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:59.264 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.257+0000 7ff61cff9700 1 -- 172.21.15.177:0/1393158256 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61800e040 con 0x7ff620100860 2023-11-27T15:15:59.264 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.257+0000 7ff61cff9700 1 -- 172.21.15.177:0/1393158256 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff618004c60 con 0x7ff620100860 2023-11-27T15:15:59.264 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.257+0000 7ff61cff9700 1 -- 172.21.15.177:0/1393158256 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff618005370 con 0x7ff620100860 2023-11-27T15:15:59.264 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.257+0000 7ff625af4700 1 -- 172.21.15.177:0/1393158256 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff610009520 con 0x7ff620100860 2023-11-27T15:15:59.264 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.257+0000 7ff625af4700 1 -- 172.21.15.177:0/1393158256 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff618009d00 con 0x7ff620100860 2023-11-27T15:15:59.265 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.258+0000 7ff625af4700 1 -- 172.21.15.177:0/1393158256 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff62010aa00 con 0x7ff620100860 2023-11-27T15:15:59.265 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.258+0000 7ff61cff9700 1 -- 172.21.15.177:0/1393158256 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 15) v1 ==== 135660+0+0 (secure 0 0 0) 0x7ff61801d470 con 0x7ff620100860 2023-11-27T15:15:59.266 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.259+0000 7ff61cff9700 1 --2- 172.21.15.177:0/1393158256 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff6080a10c0 0x7ff6080a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:15:59.266 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.259+0000 7ff61cff9700 1 -- 172.21.15.177:0/1393158256 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7ff6180cc710 con 0x7ff620100860 2023-11-27T15:15:59.266 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.259+0000 7ff61f7fe700 1 --2- 172.21.15.177:0/1393158256 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff6080a10c0 0x7ff6080a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:15:59.266 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.260+0000 7ff61f7fe700 1 --2- 172.21.15.177:0/1393158256 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff6080a10c0 0x7ff6080a3570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff610009e40 tx=0x7ff610009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:15:59.268 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.263+0000 7ff61cff9700 1 -- 172.21.15.177:0/1393158256 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff618097740 con 0x7ff620100860 2023-11-27T15:15:59.463 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:15:59.459+0000 7ff625af4700 1 -- 172.21.15.177:0/1393158256 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6200faeb0 con 0x7ff6080a10c0 2023-11-27T15:15:59.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:59 smithi133 ceph-mon[97860]: purged_snaps scrub starts 2023-11-27T15:15:59.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:59 smithi133 ceph-mon[97860]: purged_snaps scrub ok 2023-11-27T15:15:59.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:59 smithi133 ceph-mon[97860]: osd.1 [v2:172.21.15.174:6800/3890663971,v1:172.21.15.174:6801/3890663971] boot 2023-11-27T15:15:59.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:59 smithi133 ceph-mon[97860]: osdmap e13: 2 total, 2 up, 2 in 2023-11-27T15:15:59.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:15:59 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:15:59.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:59 smithi177 ceph-mon[131504]: purged_snaps scrub starts 2023-11-27T15:15:59.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:59 smithi177 ceph-mon[131504]: purged_snaps scrub ok 2023-11-27T15:15:59.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:59 smithi177 ceph-mon[131504]: osd.1 [v2:172.21.15.174:6800/3890663971,v1:172.21.15.174:6801/3890663971] boot 2023-11-27T15:15:59.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:59 smithi177 ceph-mon[131504]: osdmap e13: 2 total, 2 up, 2 in 2023-11-27T15:15:59.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:15:59 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:15:59.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:59 smithi174 ceph-mon[131110]: purged_snaps scrub starts 2023-11-27T15:15:59.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:59 smithi174 ceph-mon[131110]: purged_snaps scrub ok 2023-11-27T15:15:59.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:59 smithi174 ceph-mon[131110]: osd.1 [v2:172.21.15.174:6800/3890663971,v1:172.21.15.174:6801/3890663971] boot 2023-11-27T15:15:59.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:59 smithi174 ceph-mon[131110]: osdmap e13: 2 total, 2 up, 2 in 2023-11-27T15:15:59.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:15:59 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-11-27T15:16:00.705 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:00 smithi177 ceph-mon[131504]: osdmap e14: 2 total, 2 up, 2 in 2023-11-27T15:16:00.705 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:00 smithi177 ceph-mon[131504]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:00.705 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:00 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:16:00.706 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:00 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:16:00.706 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:00 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:00.706 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:00 smithi177 ceph-mon[131504]: pgmap v63: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:00.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:00 smithi133 ceph-mon[97860]: osdmap e14: 2 total, 2 up, 2 in 2023-11-27T15:16:00.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:00 smithi133 ceph-mon[97860]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:00.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:00 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:16:00.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:00 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:16:00.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:00 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:00.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:00 smithi133 ceph-mon[97860]: pgmap v63: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:00.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:00 smithi174 ceph-mon[131110]: osdmap e14: 2 total, 2 up, 2 in 2023-11-27T15:16:00.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:00 smithi174 ceph-mon[131110]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:00.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:00 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:16:00.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:00 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:16:00.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:00 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:00.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:00 smithi174 ceph-mon[131110]: pgmap v63: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:03.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:02 smithi133 ceph-mon[97860]: pgmap v64: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:03.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:02 smithi133 ceph-mon[97860]: from='client.? 172.21.15.177:0/1372168694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a8ba7ac-d927-4a9f-9079-105004d658a0"}]: dispatch 2023-11-27T15:16:03.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:02 smithi133 ceph-mon[97860]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a8ba7ac-d927-4a9f-9079-105004d658a0"}]: dispatch 2023-11-27T15:16:03.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:02 smithi133 ceph-mon[97860]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a8ba7ac-d927-4a9f-9079-105004d658a0"}]': finished 2023-11-27T15:16:03.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:02 smithi133 ceph-mon[97860]: osdmap e15: 3 total, 2 up, 3 in 2023-11-27T15:16:03.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:16:03.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:02 smithi133 ceph-mon[97860]: from='client.? 172.21.15.177:0/150043743' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:16:03.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:02 smithi177 ceph-mon[131504]: pgmap v64: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:03.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:02 smithi177 ceph-mon[131504]: from='client.? 172.21.15.177:0/1372168694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a8ba7ac-d927-4a9f-9079-105004d658a0"}]: dispatch 2023-11-27T15:16:03.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:02 smithi177 ceph-mon[131504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a8ba7ac-d927-4a9f-9079-105004d658a0"}]: dispatch 2023-11-27T15:16:03.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:02 smithi177 ceph-mon[131504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a8ba7ac-d927-4a9f-9079-105004d658a0"}]': finished 2023-11-27T15:16:03.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:02 smithi177 ceph-mon[131504]: osdmap e15: 3 total, 2 up, 3 in 2023-11-27T15:16:03.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:16:03.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:02 smithi177 ceph-mon[131504]: from='client.? 172.21.15.177:0/150043743' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:16:03.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:02 smithi174 ceph-mon[131110]: pgmap v64: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:03.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:02 smithi174 ceph-mon[131110]: from='client.? 172.21.15.177:0/1372168694' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a8ba7ac-d927-4a9f-9079-105004d658a0"}]: dispatch 2023-11-27T15:16:03.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:02 smithi174 ceph-mon[131110]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a8ba7ac-d927-4a9f-9079-105004d658a0"}]: dispatch 2023-11-27T15:16:03.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:02 smithi174 ceph-mon[131110]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a8ba7ac-d927-4a9f-9079-105004d658a0"}]': finished 2023-11-27T15:16:03.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:02 smithi174 ceph-mon[131110]: osdmap e15: 3 total, 2 up, 3 in 2023-11-27T15:16:03.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:16:03.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:02 smithi174 ceph-mon[131110]: from='client.? 172.21.15.177:0/150043743' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:16:05.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:04 smithi133 ceph-mon[97860]: pgmap v66: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:05.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:04 smithi177 ceph-mon[131504]: pgmap v66: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:05.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:04 smithi174 ceph-mon[131110]: pgmap v66: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:07.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:06 smithi133 ceph-mon[97860]: pgmap v67: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:07.043 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:06 smithi177 ceph-mon[131504]: pgmap v67: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:07.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:06 smithi174 ceph-mon[131110]: pgmap v67: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:09.279 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:09 smithi177 ceph-mon[131504]: pgmap v68: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:09.279 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:09 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-11-27T15:16:09.279 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:09 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:09.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:09 smithi174 ceph-mon[131110]: pgmap v68: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:09.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:09 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-11-27T15:16:09.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:09 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:09.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:09 smithi133 ceph-mon[97860]: pgmap v68: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:09.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:09 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-11-27T15:16:09.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:09 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:10.215 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:10 smithi177 ceph-mon[131504]: Deploying daemon osd.2 on smithi177 2023-11-27T15:16:10.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:10 smithi174 ceph-mon[131110]: Deploying daemon osd.2 on smithi177 2023-11-27T15:16:10.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:10 smithi133 ceph-mon[97860]: Deploying daemon osd.2 on smithi177 2023-11-27T15:16:11.297 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:11 smithi177 ceph-mon[131504]: pgmap v69: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:11.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:11 smithi174 ceph-mon[131110]: pgmap v69: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:11.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:11 smithi133 ceph-mon[97860]: pgmap v69: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:13.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:13 smithi177 ceph-mon[131504]: pgmap v70: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:13.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:13 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:13.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:13 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:13.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:13 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:13.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:13 smithi174 ceph-mon[131110]: pgmap v70: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:13.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:13 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:13.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:13 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:13.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:13 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:13.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:13 smithi133 ceph-mon[97860]: pgmap v70: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:13.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:13 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:13.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:13 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:13.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:13 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:15.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:15 smithi177 ceph-mon[131504]: pgmap v71: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:15.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:15 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:15.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:15 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:15.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:15 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:15.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:15 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:16:15.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:15 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:15.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:15 smithi174 ceph-mon[131110]: pgmap v71: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:15.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:15 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:15.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:15 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:15.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:15 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:15.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:15 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:16:15.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:15 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:15.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:15 smithi133 ceph-mon[97860]: pgmap v71: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:15.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:15 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:15.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:15 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:15.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:15 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:15.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:15 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:16:15.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:15 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:15.728 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:16:15.724+0000 7ff61cff9700 1 -- 172.21.15.177:0/1393158256 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff6200faeb0 con 0x7ff6080a10c0 2023-11-27T15:16:15.728 INFO:teuthology.orchestra.run.smithi177.stdout:Created osd(s) 2 on host 'smithi177' 2023-11-27T15:16:15.730 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:16:15.726+0000 7ff625af4700 1 -- 172.21.15.177:0/1393158256 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff6080a10c0 msgr2=0x7ff6080a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:15.730 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:16:15.726+0000 7ff625af4700 1 --2- 172.21.15.177:0/1393158256 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff6080a10c0 0x7ff6080a3570 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff610009e40 tx=0x7ff610009260 comp rx=0 tx=0).stop 2023-11-27T15:16:15.730 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:16:15.726+0000 7ff625af4700 1 -- 172.21.15.177:0/1393158256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff620100860 msgr2=0x7ff62019d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:15.730 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:16:15.726+0000 7ff625af4700 1 --2- 172.21.15.177:0/1393158256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff620100860 0x7ff62019d6b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff618007a60 tx=0x7ff61800bb60 comp rx=0 tx=0).stop 2023-11-27T15:16:15.730 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:16:15.726+0000 7ff625af4700 1 -- 172.21.15.177:0/1393158256 shutdown_connections 2023-11-27T15:16:15.730 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:16:15.726+0000 7ff625af4700 1 --2- 172.21.15.177:0/1393158256 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff6080a10c0 0x7ff6080a3570 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.731 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:16:15.726+0000 7ff625af4700 1 --2- 172.21.15.177:0/1393158256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff620100860 0x7ff62019d6b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.731 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:16:15.726+0000 7ff625af4700 1 --2- 172.21.15.177:0/1393158256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6200ff430 0x7ff6201960f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.731 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:16:15.726+0000 7ff625af4700 1 --2- 172.21.15.177:0/1393158256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6200ffde0 0x7ff620196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:15.731 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:16:15.726+0000 7ff625af4700 1 -- 172.21.15.177:0/1393158256 >> 172.21.15.177:0/1393158256 conn(0x7ff6200f9f70 msgr2=0x7ff6200fa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:15.731 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:16:15.726+0000 7ff625af4700 1 -- 172.21.15.177:0/1393158256 shutdown_connections 2023-11-27T15:16:15.731 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:16:15.727+0000 7ff625af4700 1 -- 172.21.15.177:0/1393158256 wait complete. 2023-11-27T15:16:16.173 DEBUG:teuthology.orchestra.run.smithi177:osd.2> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.2.service 2023-11-27T15:16:16.175 INFO:tasks.cephadm:Deploying osd.3 on smithi133 with /dev/vg_nvme/lv_3... 2023-11-27T15:16:16.175 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-11-27T15:16:16.207 INFO:journalctl@ceph.osd.2.smithi177.stdout:-- Logs begin at Mon 2023-11-27 14:50:38 UTC. -- 2023-11-27T15:16:16.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:16 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:16:16.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:16 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:16:16.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:16 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:16:16.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:16.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:16.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:16.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:16 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:16:16.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:16 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:16:16.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:16 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:16:16.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:16.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:16.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:16.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:16 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:16:16.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:16 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:16:16.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:16 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:16:16.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:16.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:16.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:16.944 INFO:journalctl@ceph.osd.2.smithi177.stdout:Nov 27 15:16:16 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-2[137294]: 2023-11-27T15:16:16.661+0000 7fd435695640 -1 osd.2 0 log_to_monitors true 2023-11-27T15:16:17.025 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:17.338 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:17 smithi133 ceph-mon[97860]: pgmap v72: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:17.338 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:17 smithi133 ceph-mon[97860]: from='osd.2 [v2:172.21.15.177:6800/2043769283,v1:172.21.15.177:6801/2043769283]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-27T15:16:17.338 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:17 smithi133 ceph-mon[97860]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-27T15:16:17.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:17 smithi177 ceph-mon[131504]: pgmap v72: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:17.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:17 smithi177 ceph-mon[131504]: from='osd.2 [v2:172.21.15.177:6800/2043769283,v1:172.21.15.177:6801/2043769283]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-27T15:16:17.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:17 smithi177 ceph-mon[131504]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-27T15:16:17.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:17 smithi174 ceph-mon[131110]: pgmap v72: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:17.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:17 smithi174 ceph-mon[131110]: from='osd.2 [v2:172.21.15.177:6800/2043769283,v1:172.21.15.177:6801/2043769283]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-27T15:16:17.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:17 smithi174 ceph-mon[131110]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-11-27T15:16:18.275 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:18 smithi177 ceph-mon[131504]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-11-27T15:16:18.275 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:18 smithi177 ceph-mon[131504]: osdmap e16: 3 total, 2 up, 3 in 2023-11-27T15:16:18.275 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:18 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:16:18.275 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:18 smithi177 ceph-mon[131504]: from='osd.2 [v2:172.21.15.177:6800/2043769283,v1:172.21.15.177:6801/2043769283]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:16:18.275 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:18 smithi177 ceph-mon[131504]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:16:18.275 INFO:journalctl@ceph.osd.2.smithi177.stdout:Nov 27 15:16:18 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-2[137294]: 2023-11-27T15:16:18.072+0000 7fd42ad0c700 -1 osd.2 0 waiting for initial osdmap 2023-11-27T15:16:18.275 INFO:journalctl@ceph.osd.2.smithi177.stdout:Nov 27 15:16:18 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-2[137294]: 2023-11-27T15:16:18.076+0000 7fd426b01700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:16:18.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:18 smithi133 ceph-mon[97860]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-11-27T15:16:18.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:18 smithi133 ceph-mon[97860]: osdmap e16: 3 total, 2 up, 3 in 2023-11-27T15:16:18.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:18 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:16:18.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:18 smithi133 ceph-mon[97860]: from='osd.2 [v2:172.21.15.177:6800/2043769283,v1:172.21.15.177:6801/2043769283]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:16:18.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:18 smithi133 ceph-mon[97860]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:16:18.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:18 smithi174 ceph-mon[131110]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-11-27T15:16:18.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:18 smithi174 ceph-mon[131110]: osdmap e16: 3 total, 2 up, 3 in 2023-11-27T15:16:18.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:18 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:16:18.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:18 smithi174 ceph-mon[131110]: from='osd.2 [v2:172.21.15.177:6800/2043769283,v1:172.21.15.177:6801/2043769283]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:16:18.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:18 smithi174 ceph-mon[131110]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:16:18.998 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-11-27T15:16:18.998 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-11-27T15:16:18.998 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2023-11-27T15:16:18.998 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2023-11-27T15:16:18.999 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0380172 s, 276 MB/s 2023-11-27T15:16:18.999 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2023-11-27T15:16:19.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:19 smithi177 ceph-mon[131504]: pgmap v74: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:19.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:19 smithi177 ceph-mon[131504]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2023-11-27T15:16:19.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:19 smithi177 ceph-mon[131504]: osdmap e17: 3 total, 2 up, 3 in 2023-11-27T15:16:19.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:16:19.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:16:19.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:19.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:19.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:16:19.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:19.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:19.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:16:19.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:19.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:19 smithi174 ceph-mon[131110]: pgmap v74: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:19.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:19 smithi174 ceph-mon[131110]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2023-11-27T15:16:19.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:19 smithi174 ceph-mon[131110]: osdmap e17: 3 total, 2 up, 3 in 2023-11-27T15:16:19.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:16:19.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:16:19.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:19.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:19.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:16:19.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:19.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:19.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:16:19.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:19.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:19 smithi133 ceph-mon[97860]: pgmap v74: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-11-27T15:16:19.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:19 smithi133 ceph-mon[97860]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2023-11-27T15:16:19.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:19 smithi133 ceph-mon[97860]: osdmap e17: 3 total, 2 up, 3 in 2023-11-27T15:16:19.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:16:19.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:16:19.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:19.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:19.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:16:19.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:19.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:19.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:16:19.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:19.602 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi133:vg_nvme/lv_3 2023-11-27T15:16:20.455 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:16:20.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:20 smithi133 ceph-mon[97860]: purged_snaps scrub starts 2023-11-27T15:16:20.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:20 smithi133 ceph-mon[97860]: purged_snaps scrub ok 2023-11-27T15:16:20.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:20 smithi133 ceph-mon[97860]: Detected new or changed devices on smithi177 2023-11-27T15:16:20.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:20 smithi133 ceph-mon[97860]: Adjusting osd_memory_target on smithi177 to 17168M 2023-11-27T15:16:20.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:20 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:16:20.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:20 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:16:20.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:20 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:16:20.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:20 smithi133 ceph-mon[97860]: osd.2 [v2:172.21.15.177:6800/2043769283,v1:172.21.15.177:6801/2043769283] boot 2023-11-27T15:16:20.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:20 smithi133 ceph-mon[97860]: osdmap e18: 3 total, 3 up, 3 in 2023-11-27T15:16:20.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:20 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:16:20.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:20 smithi177 ceph-mon[131504]: purged_snaps scrub starts 2023-11-27T15:16:20.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:20 smithi177 ceph-mon[131504]: purged_snaps scrub ok 2023-11-27T15:16:20.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:20 smithi177 ceph-mon[131504]: Detected new or changed devices on smithi177 2023-11-27T15:16:20.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:20 smithi177 ceph-mon[131504]: Adjusting osd_memory_target on smithi177 to 17168M 2023-11-27T15:16:20.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:20 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:16:20.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:20 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:16:20.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:20 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:16:20.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:20 smithi177 ceph-mon[131504]: osd.2 [v2:172.21.15.177:6800/2043769283,v1:172.21.15.177:6801/2043769283] boot 2023-11-27T15:16:20.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:20 smithi177 ceph-mon[131504]: osdmap e18: 3 total, 3 up, 3 in 2023-11-27T15:16:20.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:20 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:16:20.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:20 smithi174 ceph-mon[131110]: purged_snaps scrub starts 2023-11-27T15:16:20.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:20 smithi174 ceph-mon[131110]: purged_snaps scrub ok 2023-11-27T15:16:20.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:20 smithi174 ceph-mon[131110]: Detected new or changed devices on smithi177 2023-11-27T15:16:20.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:20 smithi174 ceph-mon[131110]: Adjusting osd_memory_target on smithi177 to 17168M 2023-11-27T15:16:20.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:20 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:16:20.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:20 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:16:20.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:20 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:16:20.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:20 smithi174 ceph-mon[131110]: osd.2 [v2:172.21.15.177:6800/2043769283,v1:172.21.15.177:6801/2043769283] boot 2023-11-27T15:16:20.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:20 smithi174 ceph-mon[131110]: osdmap e18: 3 total, 3 up, 3 in 2023-11-27T15:16:20.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:20 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-11-27T15:16:21.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:21 smithi133 ceph-mon[97860]: pgmap v77: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:21.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:21 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:16:21.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:21 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-11-27T15:16:21.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:21 smithi133 ceph-mon[97860]: osdmap e19: 3 total, 3 up, 3 in 2023-11-27T15:16:21.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:21 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:16:21.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:21 smithi177 ceph-mon[131504]: pgmap v77: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:21.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:21 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:16:21.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:21 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-11-27T15:16:21.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:21 smithi177 ceph-mon[131504]: osdmap e19: 3 total, 3 up, 3 in 2023-11-27T15:16:21.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:21 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:16:21.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:21 smithi174 ceph-mon[131110]: pgmap v77: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:21.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:21 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:16:21.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:21 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-11-27T15:16:21.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:21 smithi174 ceph-mon[131110]: osdmap e19: 3 total, 3 up, 3 in 2023-11-27T15:16:21.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:21 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:16:21.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.952+0000 7f2df3e8f700 1 -- 172.21.15.133:0/1218786606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dec101e80 msgr2=0x7f2dec108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:21.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.952+0000 7f2df3e8f700 1 --2- 172.21.15.133:0/1218786606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dec101e80 0x7f2dec108740 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2de400b0a0 tx=0x7f2de400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:16:21.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.952+0000 7f2df3e8f700 1 -- 172.21.15.133:0/1218786606 shutdown_connections 2023-11-27T15:16:21.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.952+0000 7f2df3e8f700 1 --2- 172.21.15.133:0/1218786606 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dec101e80 0x7f2dec108740 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:21.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.952+0000 7f2df3e8f700 1 --2- 172.21.15.133:0/1218786606 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dec1002f0 0x7f2dec100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:21.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.952+0000 7f2df3e8f700 1 --2- 172.21.15.133:0/1218786606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dec1014f0 0x7f2dec101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:21.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.952+0000 7f2df3e8f700 1 -- 172.21.15.133:0/1218786606 >> 172.21.15.133:0/1218786606 conn(0x7f2dec0fb900 msgr2=0x7f2dec0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:21.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.952+0000 7f2df3e8f700 1 -- 172.21.15.133:0/1218786606 shutdown_connections 2023-11-27T15:16:21.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.952+0000 7f2df3e8f700 1 -- 172.21.15.133:0/1218786606 wait complete. 2023-11-27T15:16:21.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.953+0000 7f2df3e8f700 1 Processor -- start 2023-11-27T15:16:21.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.953+0000 7f2df3e8f700 1 -- start start 2023-11-27T15:16:21.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.953+0000 7f2df3e8f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dec1002f0 0x7f2dec074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:21.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.953+0000 7f2df3e8f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dec1014f0 0x7f2dec072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:21.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.953+0000 7f2df3e8f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dec101e80 0x7f2dec073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:21.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.953+0000 7f2df3e8f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec073ba0 con 0x7f2dec101e80 2023-11-27T15:16:21.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.953+0000 7f2df3e8f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec073d10 con 0x7f2dec1002f0 2023-11-27T15:16:21.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.953+0000 7f2df3e8f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec1a01e0 con 0x7f2dec1014f0 2023-11-27T15:16:21.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.954+0000 7f2df242c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dec101e80 0x7f2dec073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:21.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.954+0000 7f2df1c2b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dec1002f0 0x7f2dec074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:21.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.954+0000 7f2df1c2b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dec1002f0 0x7f2dec074890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:43986/0 (socket says 172.21.15.133:43986) 2023-11-27T15:16:21.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.954+0000 7f2df242c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dec101e80 0x7f2dec073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:50750/0 (socket says 172.21.15.133:50750) 2023-11-27T15:16:21.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.954+0000 7f2df1c2b700 1 -- 172.21.15.133:0/2820017807 learned_addr learned my addr 172.21.15.133:0/2820017807 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:16:21.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.954+0000 7f2df142a700 1 --2- 172.21.15.133:0/2820017807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dec1014f0 0x7f2dec072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:21.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.954+0000 7f2df242c700 1 -- 172.21.15.133:0/2820017807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dec1014f0 msgr2=0x7f2dec072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:21.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.954+0000 7f2df242c700 1 --2- 172.21.15.133:0/2820017807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dec1014f0 0x7f2dec072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:21.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.954+0000 7f2df242c700 1 -- 172.21.15.133:0/2820017807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dec1002f0 msgr2=0x7f2dec074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:21.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.954+0000 7f2df242c700 1 --2- 172.21.15.133:0/2820017807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dec1002f0 0x7f2dec074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:21.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.954+0000 7f2df242c700 1 -- 172.21.15.133:0/2820017807 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc009520 con 0x7f2dec101e80 2023-11-27T15:16:21.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.954+0000 7f2df242c700 1 --2- 172.21.15.133:0/2820017807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dec101e80 0x7f2dec073420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2de4009fd0 tx=0x7f2de4007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:21.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.955+0000 7f2de2ffd700 1 -- 172.21.15.133:0/2820017807 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de400e040 con 0x7f2dec101e80 2023-11-27T15:16:21.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.955+0000 7f2de2ffd700 1 -- 172.21.15.133:0/2820017807 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2de4004c50 con 0x7f2dec101e80 2023-11-27T15:16:21.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.955+0000 7f2df3e8f700 1 -- 172.21.15.133:0/2820017807 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de4009d00 con 0x7f2dec101e80 2023-11-27T15:16:21.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.955+0000 7f2de2ffd700 1 -- 172.21.15.133:0/2820017807 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de401b6c0 con 0x7f2dec101e80 2023-11-27T15:16:21.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.955+0000 7f2df3e8f700 1 -- 172.21.15.133:0/2820017807 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dec10bcb0 con 0x7f2dec101e80 2023-11-27T15:16:21.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.956+0000 7f2df3e8f700 1 -- 172.21.15.133:0/2820017807 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dec107830 con 0x7f2dec101e80 2023-11-27T15:16:21.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.958+0000 7f2de2ffd700 1 -- 172.21.15.133:0/2820017807 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 15) v1 ==== 135660+0+0 (secure 0 0 0) 0x7f2de401d470 con 0x7f2dec101e80 2023-11-27T15:16:21.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.959+0000 7f2de2ffd700 1 --2- 172.21.15.133:0/2820017807 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2dd80a1140 0x7f2dd80a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:21.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.959+0000 7f2de2ffd700 1 -- 172.21.15.133:0/2820017807 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f2de40cca10 con 0x7f2dec101e80 2023-11-27T15:16:21.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.959+0000 7f2df1c2b700 1 --2- 172.21.15.133:0/2820017807 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2dd80a1140 0x7f2dd80a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:21.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.960+0000 7f2df1c2b700 1 --2- 172.21.15.133:0/2820017807 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2dd80a1140 0x7f2dd80a35f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2de8005950 tx=0x7f2de8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:21.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:21.962+0000 7f2de2ffd700 1 -- 172.21.15.133:0/2820017807 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2de4097600 con 0x7f2dec101e80 2023-11-27T15:16:22.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:22.187+0000 7f2df3e8f700 1 -- 172.21.15.133:0/2820017807 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f2dec061900 con 0x7f2dd80a1140 2023-11-27T15:16:22.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:22 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-11-27T15:16:22.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:22 smithi133 ceph-mon[97860]: osdmap e20: 3 total, 3 up, 3 in 2023-11-27T15:16:22.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:22 smithi133 ceph-mon[97860]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:22.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:22 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:16:22.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:22 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:16:22.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:22 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:22.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:22 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-11-27T15:16:22.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:22 smithi177 ceph-mon[131504]: osdmap e20: 3 total, 3 up, 3 in 2023-11-27T15:16:22.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:22 smithi177 ceph-mon[131504]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:22.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:22 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:16:22.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:22 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:16:22.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:22 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:22.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:22 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-11-27T15:16:22.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:22 smithi174 ceph-mon[131110]: osdmap e20: 3 total, 3 up, 3 in 2023-11-27T15:16:22.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:22 smithi174 ceph-mon[131110]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:22.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:22 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:16:22.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:22 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:16:22.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:22 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:23.027 INFO:journalctl@ceph.osd.0.smithi133.stdout:Nov 27 15:16:22 smithi133 sudo[113849]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-27T15:16:23.027 INFO:journalctl@ceph.osd.0.smithi133.stdout:Nov 27 15:16:22 smithi133 sudo[113849]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:16:23.028 INFO:journalctl@ceph.osd.0.smithi133.stdout:Nov 27 15:16:22 smithi133 sudo[113849]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:16:23.028 INFO:journalctl@ceph.osd.0.smithi133.stdout:Nov 27 15:16:22 smithi133 sudo[113849]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:16:23.527 INFO:journalctl@ceph.osd.0.smithi133.stdout:Nov 27 15:16:23 smithi133 sudo[113896]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-27T15:16:23.528 INFO:journalctl@ceph.osd.0.smithi133.stdout:Nov 27 15:16:23 smithi133 sudo[113896]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:16:23.528 INFO:journalctl@ceph.osd.0.smithi133.stdout:Nov 27 15:16:23 smithi133 sudo[113896]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:16:23.528 INFO:journalctl@ceph.osd.0.smithi133.stdout:Nov 27 15:16:23 smithi133 sudo[113896]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:16:23.792 INFO:journalctl@ceph.osd.1.smithi174.stdout:Nov 27 15:16:23 smithi174 sudo[138455]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-27T15:16:23.792 INFO:journalctl@ceph.osd.1.smithi174.stdout:Nov 27 15:16:23 smithi174 sudo[138455]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:16:23.792 INFO:journalctl@ceph.osd.1.smithi174.stdout:Nov 27 15:16:23 smithi174 sudo[138455]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:16:23.792 INFO:journalctl@ceph.osd.1.smithi174.stdout:Nov 27 15:16:23 smithi174 sudo[138455]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:16:24.042 INFO:journalctl@ceph.osd.1.smithi174.stdout:Nov 27 15:16:23 smithi174 sudo[138458]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-27T15:16:24.042 INFO:journalctl@ceph.osd.1.smithi174.stdout:Nov 27 15:16:23 smithi174 sudo[138458]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:16:24.042 INFO:journalctl@ceph.osd.1.smithi174.stdout:Nov 27 15:16:23 smithi174 sudo[138458]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:16:24.042 INFO:journalctl@ceph.osd.1.smithi174.stdout:Nov 27 15:16:23 smithi174 sudo[138458]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:16:24.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:24 smithi133 ceph-mon[97860]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:24.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:24 smithi133 ceph-mon[97860]: osdmap e21: 3 total, 3 up, 3 in 2023-11-27T15:16:24.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:24 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:24.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:24 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:24.311 INFO:journalctl@ceph.osd.2.smithi177.stdout:Nov 27 15:16:24 smithi177 sudo[139427]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-27T15:16:24.311 INFO:journalctl@ceph.osd.2.smithi177.stdout:Nov 27 15:16:24 smithi177 sudo[139427]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:16:24.311 INFO:journalctl@ceph.osd.2.smithi177.stdout:Nov 27 15:16:24 smithi177 sudo[139427]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:16:24.311 INFO:journalctl@ceph.osd.2.smithi177.stdout:Nov 27 15:16:24 smithi177 sudo[139427]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:16:24.311 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:24 smithi177 ceph-mon[131504]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:24.311 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:24 smithi177 ceph-mon[131504]: osdmap e21: 3 total, 3 up, 3 in 2023-11-27T15:16:24.311 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:24 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:24.311 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:24 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:24.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:24.310+0000 7f2de2ffd700 1 -- 172.21.15.133:0/2820017807 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2de40979d0 con 0x7f2dec101e80 2023-11-27T15:16:24.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:24 smithi174 ceph-mon[131110]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:24.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:24 smithi174 ceph-mon[131110]: osdmap e21: 3 total, 3 up, 3 in 2023-11-27T15:16:24.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:24 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:24.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:24 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:24.588 INFO:journalctl@ceph.osd.2.smithi177.stdout:Nov 27 15:16:24 smithi177 sudo[139430]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-27T15:16:24.588 INFO:journalctl@ceph.osd.2.smithi177.stdout:Nov 27 15:16:24 smithi177 sudo[139430]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:16:24.588 INFO:journalctl@ceph.osd.2.smithi177.stdout:Nov 27 15:16:24 smithi177 sudo[139430]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:16:24.588 INFO:journalctl@ceph.osd.2.smithi177.stdout:Nov 27 15:16:24 smithi177 sudo[139430]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:16:25.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:24 smithi133 sudo[114009]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-27T15:16:25.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:24 smithi133 sudo[114009]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:16:25.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:24 smithi133 sudo[114009]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:16:25.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:24 smithi133 sudo[114009]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:16:25.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:25 smithi177 ceph-mon[131504]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:25.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:25 smithi177 ceph-mon[131504]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-11-27T15:16:25.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:25 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:25.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:25 smithi177 sudo[139433]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-27T15:16:25.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:25 smithi177 sudo[139433]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:16:25.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:25 smithi177 sudo[139433]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:16:25.674 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:25 smithi177 sudo[139433]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:16:25.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:25 smithi133 sudo[114037]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-27T15:16:25.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:25 smithi133 sudo[114037]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:16:25.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:25 smithi133 sudo[114037]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:16:25.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:25 smithi133 sudo[114037]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:16:25.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:25 smithi133 ceph-mon[97860]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:25.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:25 smithi133 ceph-mon[97860]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-11-27T15:16:25.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:25 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:25.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:25 smithi174 ceph-mon[131110]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:25.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:25 smithi174 ceph-mon[131110]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-11-27T15:16:25.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:25 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:26.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:25 smithi177 sudo[139436]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-27T15:16:26.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:25 smithi177 sudo[139436]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:16:26.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:25 smithi177 sudo[139436]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:16:26.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:25 smithi177 sudo[139436]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:16:26.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 sudo[138461]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-11-27T15:16:26.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 sudo[138461]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:16:26.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 sudo[138461]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:16:26.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 sudo[138461]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:16:26.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 sudo[138464]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-11-27T15:16:26.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 sudo[138464]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-11-27T15:16:26.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 sudo[138464]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-11-27T15:16:26.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 sudo[138464]: pam_unix(sudo:session): session closed for user root 2023-11-27T15:16:26.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 ceph-mon[131110]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:16:26.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2718329937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "452fbca8-3b28-4258-a86b-1042888fe294"}]: dispatch 2023-11-27T15:16:26.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 ceph-mon[131110]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:16:26.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:16:26.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 ceph-mon[131110]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "452fbca8-3b28-4258-a86b-1042888fe294"}]: dispatch 2023-11-27T15:16:26.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:16:26.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:16:26.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 ceph-mon[131110]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:16:26.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:16:26.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:16:26.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:16:26.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 ceph-mon[131110]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "452fbca8-3b28-4258-a86b-1042888fe294"}]': finished 2023-11-27T15:16:26.655 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 ceph-mon[131110]: osdmap e22: 4 total, 3 up, 4 in 2023-11-27T15:16:26.655 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:16:26.655 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 ceph-mon[131110]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:26.655 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 ceph-mon[131110]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:16:26.655 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 ceph-mon[131110]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:16:26.655 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:16:26.655 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:16:26.655 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:16:26.655 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:26 smithi174 ceph-mon[131110]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:16:26.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:26 smithi133 ceph-mon[97860]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:16:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:26 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2718329937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "452fbca8-3b28-4258-a86b-1042888fe294"}]: dispatch 2023-11-27T15:16:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:26 smithi133 ceph-mon[97860]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:16:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:16:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:26 smithi133 ceph-mon[97860]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "452fbca8-3b28-4258-a86b-1042888fe294"}]: dispatch 2023-11-27T15:16:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:16:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:16:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:26 smithi133 ceph-mon[97860]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:16:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:16:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:16:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:16:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:26 smithi133 ceph-mon[97860]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "452fbca8-3b28-4258-a86b-1042888fe294"}]': finished 2023-11-27T15:16:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:26 smithi133 ceph-mon[97860]: osdmap e22: 4 total, 3 up, 4 in 2023-11-27T15:16:26.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:16:26.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:26 smithi133 ceph-mon[97860]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:26.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:26 smithi133 ceph-mon[97860]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:16:26.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:26 smithi133 ceph-mon[97860]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:16:26.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:16:26.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:16:26.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:16:26.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:26 smithi133 ceph-mon[97860]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:16:26.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:26 smithi177 ceph-mon[131504]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:16:26.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:26 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2718329937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "452fbca8-3b28-4258-a86b-1042888fe294"}]: dispatch 2023-11-27T15:16:26.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:26 smithi177 ceph-mon[131504]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:16:26.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:16:26.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:26 smithi177 ceph-mon[131504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "452fbca8-3b28-4258-a86b-1042888fe294"}]: dispatch 2023-11-27T15:16:26.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:16:26.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:16:26.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:26 smithi177 ceph-mon[131504]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:16:26.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:16:26.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:16:26.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:16:26.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:26 smithi177 ceph-mon[131504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "452fbca8-3b28-4258-a86b-1042888fe294"}]': finished 2023-11-27T15:16:26.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:26 smithi177 ceph-mon[131504]: osdmap e22: 4 total, 3 up, 4 in 2023-11-27T15:16:26.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:16:26.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:26 smithi177 ceph-mon[131504]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 18 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:26.841 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:26 smithi177 ceph-mon[131504]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:16:26.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:26 smithi177 ceph-mon[131504]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-11-27T15:16:26.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-11-27T15:16:26.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-11-27T15:16:26.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-11-27T15:16:26.842 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:26 smithi177 ceph-mon[131504]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-11-27T15:16:27.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:27 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2395771427' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:16:27.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:27 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2395771427' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:16:27.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:27 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2395771427' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:16:28.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:28 smithi133 ceph-mon[97860]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:28.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:28 smithi177 ceph-mon[131504]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:28.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:28 smithi174 ceph-mon[131110]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:31.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:30 smithi133 ceph-mon[97860]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 2.1 KiB/s rd, 62 KiB/s wr, 5 op/s 2023-11-27T15:16:31.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:30 smithi177 ceph-mon[131504]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 2.1 KiB/s rd, 62 KiB/s wr, 5 op/s 2023-11-27T15:16:31.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:30 smithi174 ceph-mon[131110]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 2.1 KiB/s rd, 62 KiB/s wr, 5 op/s 2023-11-27T15:16:33.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:33 smithi177 ceph-mon[131504]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 42 KiB/s wr, 3 op/s 2023-11-27T15:16:33.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:33 smithi174 ceph-mon[131110]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 42 KiB/s wr, 3 op/s 2023-11-27T15:16:33.404 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:33 smithi133 ceph-mon[97860]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 42 KiB/s wr, 3 op/s 2023-11-27T15:16:34.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:34 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-11-27T15:16:34.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:34 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:34.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:34 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-11-27T15:16:34.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:34 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:34.442 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:34 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-11-27T15:16:34.442 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:34 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:35.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:35 smithi177 ceph-mon[131504]: Deploying daemon osd.3 on smithi133 2023-11-27T15:16:35.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:35 smithi177 ceph-mon[131504]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 32 KiB/s wr, 2 op/s 2023-11-27T15:16:35.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:35 smithi174 ceph-mon[131110]: Deploying daemon osd.3 on smithi133 2023-11-27T15:16:35.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:35 smithi174 ceph-mon[131110]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 32 KiB/s wr, 2 op/s 2023-11-27T15:16:35.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:35 smithi133 ceph-mon[97860]: Deploying daemon osd.3 on smithi133 2023-11-27T15:16:35.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:35 smithi133 ceph-mon[97860]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 32 KiB/s wr, 2 op/s 2023-11-27T15:16:37.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:37 smithi133 ceph-mon[97860]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 900 B/s rd, 26 KiB/s wr, 2 op/s 2023-11-27T15:16:37.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:37 smithi177 ceph-mon[131504]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 900 B/s rd, 26 KiB/s wr, 2 op/s 2023-11-27T15:16:37.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:37 smithi174 ceph-mon[131110]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 900 B/s rd, 26 KiB/s wr, 2 op/s 2023-11-27T15:16:39.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:39 smithi177 ceph-mon[131504]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-11-27T15:16:39.362 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:39 smithi133 ceph-mon[97860]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-11-27T15:16:39.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:39 smithi174 ceph-mon[131110]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-11-27T15:16:40.997 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:40 smithi133 ceph-mon[97860]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-11-27T15:16:40.998 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:40 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:40.998 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:40 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:40.998 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:40 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:41.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:40 smithi177 ceph-mon[131504]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-11-27T15:16:41.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:40 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:41.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:40 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:41.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:40 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:41.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:40 smithi174 ceph-mon[131110]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-11-27T15:16:41.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:40 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:41.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:40 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:41.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:40 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:43.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:43 smithi133 ceph-mon[97860]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:43.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:43 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:43.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:43 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:43.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:43 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:43.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:43 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:16:43.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:43 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:43.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:43 smithi177 ceph-mon[131504]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:43.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:43 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:43.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:43 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:43.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:43 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:43.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:43 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:16:43.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:43 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:43.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:43 smithi174 ceph-mon[131110]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:43.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:43 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:43.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:43 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:43.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:43 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:43.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:43 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:16:43.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:43 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:43.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:43.403+0000 7f2de2ffd700 1 -- 172.21.15.133:0/2820017807 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2dec061900 con 0x7f2dd80a1140 2023-11-27T15:16:43.408 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 3 on host 'smithi133' 2023-11-27T15:16:43.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:43.405+0000 7f2df3e8f700 1 -- 172.21.15.133:0/2820017807 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2dd80a1140 msgr2=0x7f2dd80a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:43.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:43.405+0000 7f2df3e8f700 1 --2- 172.21.15.133:0/2820017807 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2dd80a1140 0x7f2dd80a35f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2de8005950 tx=0x7f2de8009260 comp rx=0 tx=0).stop 2023-11-27T15:16:43.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:43.405+0000 7f2df3e8f700 1 -- 172.21.15.133:0/2820017807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dec101e80 msgr2=0x7f2dec073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:43.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:43.405+0000 7f2df3e8f700 1 --2- 172.21.15.133:0/2820017807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dec101e80 0x7f2dec073420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2de4009fd0 tx=0x7f2de4007930 comp rx=0 tx=0).stop 2023-11-27T15:16:43.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:43.405+0000 7f2df3e8f700 1 -- 172.21.15.133:0/2820017807 shutdown_connections 2023-11-27T15:16:43.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:43.405+0000 7f2df3e8f700 1 --2- 172.21.15.133:0/2820017807 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2dd80a1140 0x7f2dd80a35f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:43.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:43.406+0000 7f2df3e8f700 1 --2- 172.21.15.133:0/2820017807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dec101e80 0x7f2dec073420 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:43.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:43.406+0000 7f2df3e8f700 1 --2- 172.21.15.133:0/2820017807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dec1002f0 0x7f2dec074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:43.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:43.406+0000 7f2df3e8f700 1 --2- 172.21.15.133:0/2820017807 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dec1014f0 0x7f2dec072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:43.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:43.406+0000 7f2df3e8f700 1 -- 172.21.15.133:0/2820017807 >> 172.21.15.133:0/2820017807 conn(0x7f2dec0fb900 msgr2=0x7f2dec104970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:43.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:43.406+0000 7f2df3e8f700 1 -- 172.21.15.133:0/2820017807 shutdown_connections 2023-11-27T15:16:43.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:16:43.406+0000 7f2df3e8f700 1 -- 172.21.15.133:0/2820017807 wait complete. 2023-11-27T15:16:43.890 DEBUG:teuthology.orchestra.run.smithi133:osd.3> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.3.service 2023-11-27T15:16:43.894 INFO:tasks.cephadm:Deploying osd.4 on smithi174 with /dev/vg_nvme/lv_3... 2023-11-27T15:16:43.895 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-11-27T15:16:43.967 INFO:journalctl@ceph.osd.3.smithi133.stdout:-- Logs begin at Mon 2023-11-27 14:54:33 UTC. -- 2023-11-27T15:16:44.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:44 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:16:44.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:44 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:16:44.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:44 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:16:44.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:44 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:44.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:44 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:44.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:44 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:44.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:44 smithi133 ceph-mon[97860]: from='osd.3 [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-11-27T15:16:44.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:44 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:16:44.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:44 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:16:44.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:44 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:16:44.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:44 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:44.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:44 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:44.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:44 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:44.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:44 smithi177 ceph-mon[131504]: from='osd.3 [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-11-27T15:16:44.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:44 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:16:44.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:44 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:16:44.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:44 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:16:44.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:44 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:44.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:44 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:44.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:44 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:16:44.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:44 smithi174 ceph-mon[131110]: from='osd.3 [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-11-27T15:16:44.545 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:16:45.236 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:45 smithi133 ceph-mon[97860]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:45.236 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:45 smithi133 ceph-mon[97860]: from='osd.3 [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-11-27T15:16:45.236 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:45 smithi133 ceph-mon[97860]: osdmap e23: 4 total, 3 up, 4 in 2023-11-27T15:16:45.237 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:45 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:16:45.237 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:45 smithi133 ceph-mon[97860]: from='osd.3 [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-11-27T15:16:45.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:45 smithi177 ceph-mon[131504]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:45.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:45 smithi177 ceph-mon[131504]: from='osd.3 [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-11-27T15:16:45.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:45 smithi177 ceph-mon[131504]: osdmap e23: 4 total, 3 up, 4 in 2023-11-27T15:16:45.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:45 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:16:45.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:45 smithi177 ceph-mon[131504]: from='osd.3 [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-11-27T15:16:45.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:45 smithi174 ceph-mon[131110]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:45.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:45 smithi174 ceph-mon[131110]: from='osd.3 [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-11-27T15:16:45.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:45 smithi174 ceph-mon[131110]: osdmap e23: 4 total, 3 up, 4 in 2023-11-27T15:16:45.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:45 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:16:45.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:45 smithi174 ceph-mon[131110]: from='osd.3 [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-11-27T15:16:45.527 INFO:journalctl@ceph.osd.3.smithi133.stdout:Nov 27 15:16:45 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-3[116279]: 2023-11-27T15:16:45.414+0000 7fa57ca40700 -1 osd.3 0 waiting for initial osdmap 2023-11-27T15:16:45.528 INFO:journalctl@ceph.osd.3.smithi133.stdout:Nov 27 15:16:45 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-3[116279]: 2023-11-27T15:16:45.421+0000 7fa578835700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:16:46.025 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-11-27T15:16:46.026 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-11-27T15:16:46.026 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2023-11-27T15:16:46.026 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2023-11-27T15:16:46.026 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111085 s, 944 MB/s 2023-11-27T15:16:46.026 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2023-11-27T15:16:46.399 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi174:vg_nvme/lv_3 2023-11-27T15:16:46.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:46 smithi133 ceph-mon[97860]: from='osd.3 [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-11-27T15:16:46.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:46 smithi133 ceph-mon[97860]: osdmap e24: 4 total, 3 up, 4 in 2023-11-27T15:16:46.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:46 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:16:46.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:46 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:16:46.652 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:46 smithi133 ceph-mon[97860]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:46.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:46 smithi174 ceph-mon[131110]: from='osd.3 [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-11-27T15:16:46.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:46 smithi174 ceph-mon[131110]: osdmap e24: 4 total, 3 up, 4 in 2023-11-27T15:16:46.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:46 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:16:46.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:46 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:16:46.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:46 smithi174 ceph-mon[131110]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:46.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:46 smithi177 ceph-mon[131504]: from='osd.3 [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-11-27T15:16:46.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:46 smithi177 ceph-mon[131504]: osdmap e24: 4 total, 3 up, 4 in 2023-11-27T15:16:46.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:46 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:16:46.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:46 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:16:46.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:46 smithi177 ceph-mon[131504]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 19 MiB used, 268 GiB / 268 GiB avail 2023-11-27T15:16:47.042 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:16:47.581 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:47 smithi174 ceph-mon[131110]: purged_snaps scrub starts 2023-11-27T15:16:47.581 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:47 smithi174 ceph-mon[131110]: purged_snaps scrub ok 2023-11-27T15:16:47.581 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:47 smithi174 ceph-mon[131110]: osd.3 [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009] boot 2023-11-27T15:16:47.581 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:47 smithi174 ceph-mon[131110]: osdmap e25: 4 total, 4 up, 4 in 2023-11-27T15:16:47.581 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:47 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:16:47.582 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:47 smithi174 ceph-mon[131110]: Detected new or changed devices on smithi133 2023-11-27T15:16:47.582 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:47 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:47.582 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:47 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:47.582 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:47 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:16:47.582 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:47 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:16:47.582 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:47 smithi174 ceph-mon[131110]: Adjusting osd_memory_target on smithi133 to 8635M 2023-11-27T15:16:47.582 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:47 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:47.582 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:47 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:47.582 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:47 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:16:47.582 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:47 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:16:47.582 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:47 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:47.582 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:47 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:16:47.583 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:47 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:16:47.583 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:47 smithi174 ceph-mon[131110]: osdmap e26: 4 total, 4 up, 4 in 2023-11-27T15:16:47.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:47 smithi133 ceph-mon[97860]: purged_snaps scrub starts 2023-11-27T15:16:47.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:47 smithi133 ceph-mon[97860]: purged_snaps scrub ok 2023-11-27T15:16:47.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:47 smithi133 ceph-mon[97860]: osd.3 [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009] boot 2023-11-27T15:16:47.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:47 smithi133 ceph-mon[97860]: osdmap e25: 4 total, 4 up, 4 in 2023-11-27T15:16:47.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:16:47.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:47 smithi133 ceph-mon[97860]: Detected new or changed devices on smithi133 2023-11-27T15:16:47.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:47.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:47.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:16:47.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:16:47.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:47 smithi133 ceph-mon[97860]: Adjusting osd_memory_target on smithi133 to 8635M 2023-11-27T15:16:47.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:47.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:47.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:16:47.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:47 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:16:47.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:47.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:47 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:16:47.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:47 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:16:47.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:47 smithi133 ceph-mon[97860]: osdmap e26: 4 total, 4 up, 4 in 2023-11-27T15:16:47.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:47 smithi177 ceph-mon[131504]: purged_snaps scrub starts 2023-11-27T15:16:47.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:47 smithi177 ceph-mon[131504]: purged_snaps scrub ok 2023-11-27T15:16:47.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:47 smithi177 ceph-mon[131504]: osd.3 [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009] boot 2023-11-27T15:16:47.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:47 smithi177 ceph-mon[131504]: osdmap e25: 4 total, 4 up, 4 in 2023-11-27T15:16:47.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-11-27T15:16:47.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:47 smithi177 ceph-mon[131504]: Detected new or changed devices on smithi133 2023-11-27T15:16:47.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:47.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:47.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:16:47.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:16:47.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:47 smithi177 ceph-mon[131504]: Adjusting osd_memory_target on smithi133 to 8635M 2023-11-27T15:16:47.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:47.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:47.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:16:47.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:47 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:16:47.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:16:47.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:47 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:16:47.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:47 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:16:47.840 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:47 smithi177 ceph-mon[131504]: osdmap e26: 4 total, 4 up, 4 in 2023-11-27T15:16:48.203 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.197+0000 7fb857abd700 1 -- 172.21.15.174:0/2625605595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8500fe960 msgr2=0x7fb8500fed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:48.204 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.197+0000 7fb857abd700 1 --2- 172.21.15.174:0/2625605595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8500fe960 0x7fb8500fed70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb84c009990 tx=0x7fb84c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:16:48.204 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.197+0000 7fb857abd700 1 -- 172.21.15.174:0/2625605595 shutdown_connections 2023-11-27T15:16:48.204 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.197+0000 7fb857abd700 1 --2- 172.21.15.174:0/2625605595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8501004f0 0x7fb850106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:48.204 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.197+0000 7fb857abd700 1 --2- 172.21.15.174:0/2625605595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8500fe960 0x7fb8500fed70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:48.204 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.197+0000 7fb857abd700 1 --2- 172.21.15.174:0/2625605595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8500ffb60 0x7fb8500fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:48.204 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.197+0000 7fb857abd700 1 -- 172.21.15.174:0/2625605595 >> 172.21.15.174:0/2625605595 conn(0x7fb8500f9f70 msgr2=0x7fb8500fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:16:48.204 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.197+0000 7fb857abd700 1 -- 172.21.15.174:0/2625605595 shutdown_connections 2023-11-27T15:16:48.204 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.197+0000 7fb857abd700 1 -- 172.21.15.174:0/2625605595 wait complete. 2023-11-27T15:16:48.204 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.197+0000 7fb857abd700 1 Processor -- start 2023-11-27T15:16:48.205 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.197+0000 7fb857abd700 1 -- start start 2023-11-27T15:16:48.205 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb857abd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8500fe960 0x7fb850194f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:48.205 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb857abd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8500ffb60 0x7fb850195450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:48.205 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb857abd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8501004f0 0x7fb85019c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:48.205 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb857abd700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85019caa0 con 0x7fb8500ffb60 2023-11-27T15:16:48.205 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb857abd700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85019cbe0 con 0x7fb8500fe960 2023-11-27T15:16:48.205 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb857abd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85019cd70 con 0x7fb8501004f0 2023-11-27T15:16:48.205 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb855859700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8500fe960 0x7fb850194f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:48.205 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb855859700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8500fe960 0x7fb850194f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:48384/0 (socket says 172.21.15.174:48384) 2023-11-27T15:16:48.205 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb855859700 1 -- 172.21.15.174:0/2695947491 learned_addr learned my addr 172.21.15.174:0/2695947491 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-27T15:16:48.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb85605a700 1 --2- 172.21.15.174:0/2695947491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8501004f0 0x7fb85019c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:48.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb855058700 1 --2- 172.21.15.174:0/2695947491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8500ffb60 0x7fb850195450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:48.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb855859700 1 -- 172.21.15.174:0/2695947491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8501004f0 msgr2=0x7fb85019c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:48.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb855859700 1 --2- 172.21.15.174:0/2695947491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8501004f0 0x7fb85019c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:48.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb855859700 1 -- 172.21.15.174:0/2695947491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8500ffb60 msgr2=0x7fb850195450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:16:48.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb855859700 1 --2- 172.21.15.174:0/2695947491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8500ffb60 0x7fb850195450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:16:48.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb855859700 1 -- 172.21.15.174:0/2695947491 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84c0095f0 con 0x7fb8500fe960 2023-11-27T15:16:48.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb85605a700 1 --2- 172.21.15.174:0/2695947491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8501004f0 0x7fb85019c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:16:48.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb855058700 1 --2- 172.21.15.174:0/2695947491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8500ffb60 0x7fb850195450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:16:48.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb855859700 1 --2- 172.21.15.174:0/2695947491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8500fe960 0x7fb850194f10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb84c000c00 tx=0x7fb84c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:48.206 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb846ffd700 1 -- 172.21.15.174:0/2695947491 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c01d070 con 0x7fb8500fe960 2023-11-27T15:16:48.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb857abd700 1 -- 172.21.15.174:0/2695947491 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb85010a330 con 0x7fb8500fe960 2023-11-27T15:16:48.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb846ffd700 1 -- 172.21.15.174:0/2695947491 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb84c00fc00 con 0x7fb8500fe960 2023-11-27T15:16:48.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb846ffd700 1 -- 172.21.15.174:0/2695947491 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c0176c0 con 0x7fb8500fe960 2023-11-27T15:16:48.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.198+0000 7fb857abd700 1 -- 172.21.15.174:0/2695947491 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85010a1a0 con 0x7fb8500fe960 2023-11-27T15:16:48.207 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.199+0000 7fb857abd700 1 -- 172.21.15.174:0/2695947491 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb850066ea0 con 0x7fb8500fe960 2023-11-27T15:16:48.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.200+0000 7fb846ffd700 1 -- 172.21.15.174:0/2695947491 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb84c022470 con 0x7fb8500fe960 2023-11-27T15:16:48.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.201+0000 7fb846ffd700 1 --2- 172.21.15.174:0/2695947491 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb83c0a1150 0x7fb83c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:16:48.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.201+0000 7fb846ffd700 1 -- 172.21.15.174:0/2695947491 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7fb84c0cca60 con 0x7fb8500fe960 2023-11-27T15:16:48.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.201+0000 7fb855058700 1 --2- 172.21.15.174:0/2695947491 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb83c0a1150 0x7fb83c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:16:48.208 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.202+0000 7fb855058700 1 --2- 172.21.15.174:0/2695947491 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb83c0a1150 0x7fb83c0a3600 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb840005fd0 tx=0x7fb840005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:16:48.209 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.205+0000 7fb846ffd700 1 -- 172.21.15.174:0/2695947491 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb84c05c730 con 0x7fb8500fe960 2023-11-27T15:16:48.402 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:16:48.398+0000 7fb857abd700 1 -- 172.21.15.174:0/2695947491 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8500fc750 con 0x7fb83c0a1150 2023-11-27T15:16:49.331 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:49 smithi174 ceph-mon[131110]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:16:49.331 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:49 smithi174 ceph-mon[131110]: osdmap e27: 4 total, 4 up, 4 in 2023-11-27T15:16:49.331 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:49 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:16:49.331 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:49 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:16:49.331 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:49 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:49.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:49 smithi177 ceph-mon[131504]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:16:49.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:49 smithi177 ceph-mon[131504]: osdmap e27: 4 total, 4 up, 4 in 2023-11-27T15:16:49.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:49 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:16:49.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:49 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:16:49.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:49 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:49.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:49 smithi133 ceph-mon[97860]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:16:49.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:49 smithi133 ceph-mon[97860]: osdmap e27: 4 total, 4 up, 4 in 2023-11-27T15:16:49.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:49 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:16:49.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:49 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:16:49.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:49 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:50.104 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:50 smithi174 ceph-mon[131110]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:50.104 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:50 smithi174 ceph-mon[131110]: osdmap e28: 4 total, 4 up, 4 in 2023-11-27T15:16:50.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:50 smithi177 ceph-mon[131504]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:50.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:50 smithi177 ceph-mon[131504]: osdmap e28: 4 total, 4 up, 4 in 2023-11-27T15:16:50.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:50 smithi133 ceph-mon[97860]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:16:50.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:50 smithi133 ceph-mon[97860]: osdmap e28: 4 total, 4 up, 4 in 2023-11-27T15:16:51.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:51 smithi177 ceph-mon[131504]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-27T15:16:51.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:51 smithi177 ceph-mon[131504]: osdmap e29: 4 total, 4 up, 4 in 2023-11-27T15:16:51.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:51 smithi177 ceph-mon[131504]: from='client.? 172.21.15.174:0/3857501099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd04b180-bbcc-4611-9c42-936c87827fe7"}]: dispatch 2023-11-27T15:16:51.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:51 smithi177 ceph-mon[131504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd04b180-bbcc-4611-9c42-936c87827fe7"}]: dispatch 2023-11-27T15:16:51.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:51 smithi177 ceph-mon[131504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dd04b180-bbcc-4611-9c42-936c87827fe7"}]': finished 2023-11-27T15:16:51.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:51 smithi177 ceph-mon[131504]: osdmap e30: 5 total, 4 up, 5 in 2023-11-27T15:16:51.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:51 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:16:51.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:51 smithi174 ceph-mon[131110]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-27T15:16:51.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:51 smithi174 ceph-mon[131110]: osdmap e29: 4 total, 4 up, 4 in 2023-11-27T15:16:51.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:51 smithi174 ceph-mon[131110]: from='client.? 172.21.15.174:0/3857501099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd04b180-bbcc-4611-9c42-936c87827fe7"}]: dispatch 2023-11-27T15:16:51.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:51 smithi174 ceph-mon[131110]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd04b180-bbcc-4611-9c42-936c87827fe7"}]: dispatch 2023-11-27T15:16:51.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:51 smithi174 ceph-mon[131110]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dd04b180-bbcc-4611-9c42-936c87827fe7"}]': finished 2023-11-27T15:16:51.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:51 smithi174 ceph-mon[131110]: osdmap e30: 5 total, 4 up, 5 in 2023-11-27T15:16:51.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:51 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:16:51.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:51 smithi133 ceph-mon[97860]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-27T15:16:51.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:51 smithi133 ceph-mon[97860]: osdmap e29: 4 total, 4 up, 4 in 2023-11-27T15:16:51.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:51 smithi133 ceph-mon[97860]: from='client.? 172.21.15.174:0/3857501099' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd04b180-bbcc-4611-9c42-936c87827fe7"}]: dispatch 2023-11-27T15:16:51.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:51 smithi133 ceph-mon[97860]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd04b180-bbcc-4611-9c42-936c87827fe7"}]: dispatch 2023-11-27T15:16:51.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:51 smithi133 ceph-mon[97860]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dd04b180-bbcc-4611-9c42-936c87827fe7"}]': finished 2023-11-27T15:16:51.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:51 smithi133 ceph-mon[97860]: osdmap e30: 5 total, 4 up, 5 in 2023-11-27T15:16:51.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:51 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:16:52.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:52 smithi177 ceph-mon[131504]: from='client.? 172.21.15.174:0/667967251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:16:52.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:52 smithi174 ceph-mon[131110]: from='client.? 172.21.15.174:0/667967251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:16:52.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:52 smithi133 ceph-mon[97860]: from='client.? 172.21.15.174:0/667967251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:16:53.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:53 smithi177 ceph-mon[131504]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 25 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-27T15:16:53.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:53 smithi174 ceph-mon[131110]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 25 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-27T15:16:53.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:53 smithi133 ceph-mon[97860]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 25 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-11-27T15:16:55.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:55 smithi177 ceph-mon[131504]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-11-27T15:16:55.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:55 smithi174 ceph-mon[131110]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-11-27T15:16:55.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:55 smithi133 ceph-mon[97860]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-11-27T15:16:57.303 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:57 smithi174 ceph-mon[131110]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2023-11-27T15:16:57.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:57 smithi177 ceph-mon[131504]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2023-11-27T15:16:57.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:57 smithi133 ceph-mon[97860]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2023-11-27T15:16:58.387 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:58 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-11-27T15:16:58.387 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:58 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:58.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:58 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-11-27T15:16:58.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:58 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:58.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:58 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-11-27T15:16:58.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:58 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:16:59.268 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:59 smithi174 ceph-mon[131110]: Deploying daemon osd.4 on smithi174 2023-11-27T15:16:59.268 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:16:59 smithi174 ceph-mon[131110]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:16:59.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:59 smithi133 ceph-mon[97860]: Deploying daemon osd.4 on smithi174 2023-11-27T15:16:59.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:16:59 smithi133 ceph-mon[97860]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:16:59.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:59 smithi177 ceph-mon[131504]: Deploying daemon osd.4 on smithi174 2023-11-27T15:16:59.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:16:59 smithi177 ceph-mon[131504]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:17:01.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:01 smithi174 ceph-mon[131110]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:17:01.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:01 smithi133 ceph-mon[97860]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:17:01.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:01 smithi177 ceph-mon[131504]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:17:02.995 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:02 smithi174 ceph-mon[131110]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:17:02.995 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:02.995 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:02.995 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:03.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:02 smithi177 ceph-mon[131504]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:17:03.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:03.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:03.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:03.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:02 smithi133 ceph-mon[97860]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:17:03.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:03.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:03.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:04.564 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:04.559+0000 7fb846ffd700 1 -- 172.21.15.174:0/2695947491 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb8500fc750 con 0x7fb83c0a1150 2023-11-27T15:17:04.565 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 4 on host 'smithi174' 2023-11-27T15:17:04.566 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:04.562+0000 7fb857abd700 1 -- 172.21.15.174:0/2695947491 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb83c0a1150 msgr2=0x7fb83c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:04.566 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:04.562+0000 7fb857abd700 1 --2- 172.21.15.174:0/2695947491 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb83c0a1150 0x7fb83c0a3600 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb840005fd0 tx=0x7fb840005e30 comp rx=0 tx=0).stop 2023-11-27T15:17:04.567 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:04.562+0000 7fb857abd700 1 -- 172.21.15.174:0/2695947491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8500fe960 msgr2=0x7fb850194f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:04.567 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:04.562+0000 7fb857abd700 1 --2- 172.21.15.174:0/2695947491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8500fe960 0x7fb850194f10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb84c000c00 tx=0x7fb84c00f790 comp rx=0 tx=0).stop 2023-11-27T15:17:04.567 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:04.562+0000 7fb857abd700 1 -- 172.21.15.174:0/2695947491 shutdown_connections 2023-11-27T15:17:04.567 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:04.562+0000 7fb857abd700 1 --2- 172.21.15.174:0/2695947491 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb83c0a1150 0x7fb83c0a3600 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:04.567 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:04.562+0000 7fb857abd700 1 --2- 172.21.15.174:0/2695947491 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8501004f0 0x7fb85019c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:04.567 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:04.562+0000 7fb857abd700 1 --2- 172.21.15.174:0/2695947491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8500fe960 0x7fb850194f10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:04.567 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:04.562+0000 7fb857abd700 1 --2- 172.21.15.174:0/2695947491 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8500ffb60 0x7fb850195450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:04.567 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:04.562+0000 7fb857abd700 1 -- 172.21.15.174:0/2695947491 >> 172.21.15.174:0/2695947491 conn(0x7fb8500f9f70 msgr2=0x7fb850103620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:04.567 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:04.562+0000 7fb857abd700 1 -- 172.21.15.174:0/2695947491 shutdown_connections 2023-11-27T15:17:04.567 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:04.562+0000 7fb857abd700 1 -- 172.21.15.174:0/2695947491 wait complete. 2023-11-27T15:17:04.921 DEBUG:teuthology.orchestra.run.smithi174:osd.4> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.4.service 2023-11-27T15:17:04.923 INFO:tasks.cephadm:Deploying osd.5 on smithi177 with /dev/vg_nvme/lv_3... 2023-11-27T15:17:04.923 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-11-27T15:17:04.965 INFO:journalctl@ceph.osd.4.smithi174.stdout:-- Logs begin at Mon 2023-11-27 14:50:40 UTC. -- 2023-11-27T15:17:05.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:05 smithi177 ceph-mon[131504]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:17:05.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:05.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:05.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:05.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:05.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:05 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:17:05.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:05.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:05 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:17:05.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:05 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:17:05.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:05.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:05.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:05.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:05 smithi174 ceph-mon[131110]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:17:05.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:05.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:05.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:05.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:05.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:05 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:17:05.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:05.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:05 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:17:05.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:05 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:17:05.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:05.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:05.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:05.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:05 smithi133 ceph-mon[97860]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:17:05.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:05.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:05.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:05.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:05.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:05 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:17:05.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:05.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:05 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:17:05.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:05 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:17:05.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:05.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:05.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:05.587 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:17:06.087 INFO:journalctl@ceph.osd.4.smithi174.stdout:Nov 27 15:17:05 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-4[141652]: 2023-11-27T15:17:05.805+0000 7fe4100d6640 -1 osd.4 0 log_to_monitors true 2023-11-27T15:17:06.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:06 smithi174 ceph-mon[131110]: from='osd.4 [v2:172.21.15.174:6808/959536963,v1:172.21.15.174:6809/959536963]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-27T15:17:06.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:06 smithi174 ceph-mon[131110]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-27T15:17:06.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:06 smithi133 ceph-mon[97860]: from='osd.4 [v2:172.21.15.174:6808/959536963,v1:172.21.15.174:6809/959536963]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-27T15:17:06.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:06 smithi133 ceph-mon[97860]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-27T15:17:06.542 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:06 smithi177 ceph-mon[131504]: from='osd.4 [v2:172.21.15.174:6808/959536963,v1:172.21.15.174:6809/959536963]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-27T15:17:06.542 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:06 smithi177 ceph-mon[131504]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-11-27T15:17:07.017 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-11-27T15:17:07.017 INFO:teuthology.orchestra.run.smithi177.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-11-27T15:17:07.017 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10+0 records in 2023-11-27T15:17:07.017 INFO:teuthology.orchestra.run.smithi177.stderr:10+0 records out 2023-11-27T15:17:07.017 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110189 s, 952 MB/s 2023-11-27T15:17:07.017 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping successful for: 2023-11-27T15:17:07.328 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:07 smithi177 ceph-mon[131504]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:17:07.328 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:07 smithi177 ceph-mon[131504]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-11-27T15:17:07.328 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:07 smithi177 ceph-mon[131504]: osdmap e31: 5 total, 4 up, 5 in 2023-11-27T15:17:07.328 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:07 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:17:07.328 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:07 smithi177 ceph-mon[131504]: from='osd.4 [v2:172.21.15.174:6808/959536963,v1:172.21.15.174:6809/959536963]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:17:07.328 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:07 smithi177 ceph-mon[131504]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:17:07.341 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi177:vg_nvme/lv_3 2023-11-27T15:17:07.358 INFO:journalctl@ceph.osd.4.smithi174.stdout:Nov 27 15:17:07 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-4[141652]: 2023-11-27T15:17:07.143+0000 7fe40574d700 -1 osd.4 0 waiting for initial osdmap 2023-11-27T15:17:07.358 INFO:journalctl@ceph.osd.4.smithi174.stdout:Nov 27 15:17:07 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-4[141652]: 2023-11-27T15:17:07.152+0000 7fe3fed3d700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:17:07.358 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:07 smithi174 ceph-mon[131110]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:17:07.358 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:07 smithi174 ceph-mon[131110]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-11-27T15:17:07.359 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:07 smithi174 ceph-mon[131110]: osdmap e31: 5 total, 4 up, 5 in 2023-11-27T15:17:07.359 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:07 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:17:07.359 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:07 smithi174 ceph-mon[131110]: from='osd.4 [v2:172.21.15.174:6808/959536963,v1:172.21.15.174:6809/959536963]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:17:07.359 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:07 smithi174 ceph-mon[131110]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:17:07.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:07 smithi133 ceph-mon[97860]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:17:07.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:07 smithi133 ceph-mon[97860]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-11-27T15:17:07.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:07 smithi133 ceph-mon[97860]: osdmap e31: 5 total, 4 up, 5 in 2023-11-27T15:17:07.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:07 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:17:07.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:07 smithi133 ceph-mon[97860]: from='osd.4 [v2:172.21.15.174:6808/959536963,v1:172.21.15.174:6809/959536963]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:17:07.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:07 smithi133 ceph-mon[97860]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:17:07.979 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:17:08.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:08 smithi174 ceph-mon[131110]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2023-11-27T15:17:08.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:08 smithi174 ceph-mon[131110]: osdmap e32: 5 total, 4 up, 5 in 2023-11-27T15:17:08.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:17:08.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:17:08.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:08.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:08.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:08.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:08.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:08.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:08.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:08.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:08.417 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:08 smithi177 ceph-mon[131504]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2023-11-27T15:17:08.417 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:08 smithi177 ceph-mon[131504]: osdmap e32: 5 total, 4 up, 5 in 2023-11-27T15:17:08.417 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:17:08.417 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:17:08.417 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:08.417 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:08.417 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:08.417 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:08.417 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:08.417 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:08.418 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:08.418 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:08.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:08 smithi133 ceph-mon[97860]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2023-11-27T15:17:08.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:08 smithi133 ceph-mon[97860]: osdmap e32: 5 total, 4 up, 5 in 2023-11-27T15:17:08.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:17:08.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:17:08.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:08.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:08.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:08.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:08.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:08.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:08.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:08.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:09.116 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.109+0000 7fe23f487700 1 -- 172.21.15.177:0/1890796840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe238100d70 msgr2=0x7fe238107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:09.116 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.109+0000 7fe23f487700 1 --2- 172.21.15.177:0/1890796840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe238100d70 0x7fe238107630 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe23000b0a0 tx=0x7fe23000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.116 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.109+0000 7fe23f487700 1 -- 172.21.15.177:0/1890796840 shutdown_connections 2023-11-27T15:17:09.116 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.109+0000 7fe23f487700 1 --2- 172.21.15.177:0/1890796840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe238100d70 0x7fe238107630 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.116 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.110+0000 7fe23f487700 1 --2- 172.21.15.177:0/1890796840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2380ff1e0 0x7fe2380ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.116 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.110+0000 7fe23f487700 1 --2- 172.21.15.177:0/1890796840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2381003e0 0x7fe238100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.117 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.110+0000 7fe23f487700 1 -- 172.21.15.177:0/1890796840 >> 172.21.15.177:0/1890796840 conn(0x7fe2380fa7f0 msgr2=0x7fe2380fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:09.117 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.110+0000 7fe23f487700 1 -- 172.21.15.177:0/1890796840 shutdown_connections 2023-11-27T15:17:09.117 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.110+0000 7fe23f487700 1 -- 172.21.15.177:0/1890796840 wait complete. 2023-11-27T15:17:09.117 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.110+0000 7fe23f487700 1 Processor -- start 2023-11-27T15:17:09.117 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.110+0000 7fe23f487700 1 -- start start 2023-11-27T15:17:09.117 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.110+0000 7fe23f487700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2380ff1e0 0x7fe238103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:09.117 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.110+0000 7fe23f487700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2381003e0 0x7fe2381017c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:09.117 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.110+0000 7fe23f487700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe238100d70 0x7fe238101d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:09.117 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.110+0000 7fe23f487700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe238102480 con 0x7fe2380ff1e0 2023-11-27T15:17:09.117 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.110+0000 7fe23f487700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2381025f0 con 0x7fe2381003e0 2023-11-27T15:17:09.117 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.110+0000 7fe23f487700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2381a01a0 con 0x7fe238100d70 2023-11-27T15:17:09.117 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.110+0000 7fe23da24700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe238100d70 0x7fe238101d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:09.118 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.111+0000 7fe23da24700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe238100d70 0x7fe238101d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:57906/0 (socket says 172.21.15.177:57906) 2023-11-27T15:17:09.118 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.111+0000 7fe23da24700 1 -- 172.21.15.177:0/1965111145 learned_addr learned my addr 172.21.15.177:0/1965111145 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-27T15:17:09.118 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.111+0000 7fe23ca22700 1 --2- 172.21.15.177:0/1965111145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2381003e0 0x7fe2381017c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:09.118 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.111+0000 7fe23d223700 1 --2- 172.21.15.177:0/1965111145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2380ff1e0 0x7fe238103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:09.118 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.111+0000 7fe23da24700 1 -- 172.21.15.177:0/1965111145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2381003e0 msgr2=0x7fe2381017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:09.118 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.111+0000 7fe23da24700 1 --2- 172.21.15.177:0/1965111145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2381003e0 0x7fe2381017c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.118 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.111+0000 7fe23da24700 1 -- 172.21.15.177:0/1965111145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2380ff1e0 msgr2=0x7fe238103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:09.118 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.111+0000 7fe23da24700 1 --2- 172.21.15.177:0/1965111145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2380ff1e0 0x7fe238103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:09.118 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.111+0000 7fe23da24700 1 -- 172.21.15.177:0/1965111145 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe230009d00 con 0x7fe238100d70 2023-11-27T15:17:09.118 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.111+0000 7fe23d223700 1 --2- 172.21.15.177:0/1965111145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2380ff1e0 0x7fe238103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:17:09.118 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.111+0000 7fe23da24700 1 --2- 172.21.15.177:0/1965111145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe238100d70 0x7fe238101d00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe2300062a0 tx=0x7fe230004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:09.118 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.111+0000 7fe22e7fc700 1 -- 172.21.15.177:0/1965111145 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23000e040 con 0x7fe238100d70 2023-11-27T15:17:09.119 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.111+0000 7fe22e7fc700 1 -- 172.21.15.177:0/1965111145 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe23000cec0 con 0x7fe238100d70 2023-11-27T15:17:09.119 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.111+0000 7fe23f487700 1 -- 172.21.15.177:0/1965111145 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23810abb0 con 0x7fe238100d70 2023-11-27T15:17:09.119 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.111+0000 7fe22e7fc700 1 -- 172.21.15.177:0/1965111145 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe230004020 con 0x7fe238100d70 2023-11-27T15:17:09.119 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.111+0000 7fe23f487700 1 -- 172.21.15.177:0/1965111145 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe23810aa20 con 0x7fe238100d70 2023-11-27T15:17:09.120 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.112+0000 7fe23f487700 1 -- 172.21.15.177:0/1965111145 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe238066ea0 con 0x7fe238100d70 2023-11-27T15:17:09.120 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.113+0000 7fe22e7fc700 1 -- 172.21.15.177:0/1965111145 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe23001d690 con 0x7fe238100d70 2023-11-27T15:17:09.120 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.114+0000 7fe22e7fc700 1 --2- 172.21.15.177:0/1965111145 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe2240a11d0 0x7fe2240a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:09.120 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.114+0000 7fe22e7fc700 1 -- 172.21.15.177:0/1965111145 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fe2300cdbd0 con 0x7fe238100d70 2023-11-27T15:17:09.120 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.114+0000 7fe23d223700 1 --2- 172.21.15.177:0/1965111145 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe2240a11d0 0x7fe2240a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:09.120 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.115+0000 7fe23d223700 1 --2- 172.21.15.177:0/1965111145 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe2240a11d0 0x7fe2240a3680 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe234005950 tx=0x7fe2340058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:09.122 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.117+0000 7fe22e7fc700 1 -- 172.21.15.177:0/1965111145 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe230098420 con 0x7fe238100d70 2023-11-27T15:17:09.316 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:09.312+0000 7fe23f487700 1 -- 172.21.15.177:0/1965111145 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2381a0c30 con 0x7fe2240a11d0 2023-11-27T15:17:09.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:09 smithi133 ceph-mon[97860]: purged_snaps scrub starts 2023-11-27T15:17:09.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:09 smithi133 ceph-mon[97860]: purged_snaps scrub ok 2023-11-27T15:17:09.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:09 smithi133 ceph-mon[97860]: Detected new or changed devices on smithi174 2023-11-27T15:17:09.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:09 smithi133 ceph-mon[97860]: Adjusting osd_memory_target on smithi174 to 8584M 2023-11-27T15:17:09.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:09 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:17:09.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:09 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:17:09.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:09 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:17:09.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:09 smithi133 ceph-mon[97860]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:17:09.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:09 smithi133 ceph-mon[97860]: osd.4 [v2:172.21.15.174:6808/959536963,v1:172.21.15.174:6809/959536963] boot 2023-11-27T15:17:09.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:09 smithi133 ceph-mon[97860]: osdmap e33: 5 total, 5 up, 5 in 2023-11-27T15:17:09.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:09 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:17:09.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:09 smithi177 ceph-mon[131504]: purged_snaps scrub starts 2023-11-27T15:17:09.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:09 smithi177 ceph-mon[131504]: purged_snaps scrub ok 2023-11-27T15:17:09.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:09 smithi177 ceph-mon[131504]: Detected new or changed devices on smithi174 2023-11-27T15:17:09.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:09 smithi177 ceph-mon[131504]: Adjusting osd_memory_target on smithi174 to 8584M 2023-11-27T15:17:09.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:09 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:17:09.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:09 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:17:09.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:09 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:17:09.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:09 smithi177 ceph-mon[131504]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:17:09.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:09 smithi177 ceph-mon[131504]: osd.4 [v2:172.21.15.174:6808/959536963,v1:172.21.15.174:6809/959536963] boot 2023-11-27T15:17:09.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:09 smithi177 ceph-mon[131504]: osdmap e33: 5 total, 5 up, 5 in 2023-11-27T15:17:09.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:09 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:17:09.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:09 smithi174 ceph-mon[131110]: purged_snaps scrub starts 2023-11-27T15:17:09.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:09 smithi174 ceph-mon[131110]: purged_snaps scrub ok 2023-11-27T15:17:09.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:09 smithi174 ceph-mon[131110]: Detected new or changed devices on smithi174 2023-11-27T15:17:09.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:09 smithi174 ceph-mon[131110]: Adjusting osd_memory_target on smithi174 to 8584M 2023-11-27T15:17:09.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:09 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:17:09.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:09 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:17:09.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:09 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:17:09.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:09 smithi174 ceph-mon[131110]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 24 MiB used, 358 GiB / 358 GiB avail 2023-11-27T15:17:09.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:09 smithi174 ceph-mon[131110]: osd.4 [v2:172.21.15.174:6808/959536963,v1:172.21.15.174:6809/959536963] boot 2023-11-27T15:17:09.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:09 smithi174 ceph-mon[131110]: osdmap e33: 5 total, 5 up, 5 in 2023-11-27T15:17:09.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:09 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-11-27T15:17:10.418 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:10 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:17:10.418 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:10 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:17:10.418 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:10 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:10.418 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:10 smithi177 ceph-mon[131504]: osdmap e34: 5 total, 5 up, 5 in 2023-11-27T15:17:10.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:10 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:17:10.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:10 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:17:10.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:10 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:10.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:10 smithi133 ceph-mon[97860]: osdmap e34: 5 total, 5 up, 5 in 2023-11-27T15:17:10.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:10 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:17:10.661 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:10 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:17:10.662 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:10 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:10.662 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:10 smithi174 ceph-mon[131110]: osdmap e34: 5 total, 5 up, 5 in 2023-11-27T15:17:11.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:11 smithi133 ceph-mon[97860]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:11.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:11 smithi133 ceph-mon[97860]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:11.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:11 smithi177 ceph-mon[131504]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:11.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:11 smithi177 ceph-mon[131504]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:11.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:11 smithi174 ceph-mon[131110]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:11.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:11 smithi174 ceph-mon[131110]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:12.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:12 smithi133 ceph-mon[97860]: from='client.? 172.21.15.177:0/99880716' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec77b94b-3eb1-41a2-a319-fd446ce04816"}]: dispatch 2023-11-27T15:17:12.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:12 smithi133 ceph-mon[97860]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec77b94b-3eb1-41a2-a319-fd446ce04816"}]: dispatch 2023-11-27T15:17:12.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:12 smithi133 ceph-mon[97860]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec77b94b-3eb1-41a2-a319-fd446ce04816"}]': finished 2023-11-27T15:17:12.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:12 smithi133 ceph-mon[97860]: osdmap e35: 6 total, 5 up, 6 in 2023-11-27T15:17:12.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:17:12.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:12 smithi177 ceph-mon[131504]: from='client.? 172.21.15.177:0/99880716' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec77b94b-3eb1-41a2-a319-fd446ce04816"}]: dispatch 2023-11-27T15:17:12.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:12 smithi177 ceph-mon[131504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec77b94b-3eb1-41a2-a319-fd446ce04816"}]: dispatch 2023-11-27T15:17:12.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:12 smithi177 ceph-mon[131504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec77b94b-3eb1-41a2-a319-fd446ce04816"}]': finished 2023-11-27T15:17:12.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:12 smithi177 ceph-mon[131504]: osdmap e35: 6 total, 5 up, 6 in 2023-11-27T15:17:12.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:17:12.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:12 smithi174 ceph-mon[131110]: from='client.? 172.21.15.177:0/99880716' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec77b94b-3eb1-41a2-a319-fd446ce04816"}]: dispatch 2023-11-27T15:17:12.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:12 smithi174 ceph-mon[131110]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec77b94b-3eb1-41a2-a319-fd446ce04816"}]: dispatch 2023-11-27T15:17:12.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:12 smithi174 ceph-mon[131110]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec77b94b-3eb1-41a2-a319-fd446ce04816"}]': finished 2023-11-27T15:17:12.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:12 smithi174 ceph-mon[131110]: osdmap e35: 6 total, 5 up, 6 in 2023-11-27T15:17:12.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:17:13.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:13 smithi133 ceph-mon[97860]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:13.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:13 smithi133 ceph-mon[97860]: from='client.? 172.21.15.177:0/2689965560' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:17:13.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:13 smithi177 ceph-mon[131504]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:13.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:13 smithi177 ceph-mon[131504]: from='client.? 172.21.15.177:0/2689965560' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:17:13.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:13 smithi174 ceph-mon[131110]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 29 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:13.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:13 smithi174 ceph-mon[131110]: from='client.? 172.21.15.177:0/2689965560' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:17:15.442 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:15 smithi177 ceph-mon[131504]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:15.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:15 smithi133 ceph-mon[97860]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:15.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:15 smithi174 ceph-mon[131110]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:17.422 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:17 smithi177 ceph-mon[131504]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:17.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:17 smithi133 ceph-mon[97860]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:17.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:17 smithi174 ceph-mon[131110]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:19.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:19 smithi133 ceph-mon[97860]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:19.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-11-27T15:17:19.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:19.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:19 smithi177 ceph-mon[131504]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:19.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-11-27T15:17:19.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:19.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:19 smithi174 ceph-mon[131110]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:19.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-11-27T15:17:19.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:20.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:20 smithi177 ceph-mon[131504]: Deploying daemon osd.5 on smithi177 2023-11-27T15:17:20.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:20 smithi133 ceph-mon[97860]: Deploying daemon osd.5 on smithi177 2023-11-27T15:17:20.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:20 smithi174 ceph-mon[131110]: Deploying daemon osd.5 on smithi177 2023-11-27T15:17:21.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:21 smithi177 ceph-mon[131504]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:21.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:21 smithi133 ceph-mon[97860]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:21.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:21 smithi174 ceph-mon[131110]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:23.435 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:23 smithi177 ceph-mon[131504]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:23.436 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:23 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:23.436 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:23 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:23.436 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:23 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:23.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:23 smithi133 ceph-mon[97860]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:23.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:23 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:23.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:23 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:23.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:23 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:23.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:23 smithi174 ceph-mon[131110]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:23.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:23 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:23.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:23 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:23.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:23 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:25.329 INFO:teuthology.orchestra.run.smithi177.stdout:Created osd(s) 5 on host 'smithi177' 2023-11-27T15:17:25.329 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:25.325+0000 7fe22e7fc700 1 -- 172.21.15.177:0/1965111145 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe2381a0c30 con 0x7fe2240a11d0 2023-11-27T15:17:25.331 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:25.327+0000 7fe23f487700 1 -- 172.21.15.177:0/1965111145 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe2240a11d0 msgr2=0x7fe2240a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:25.331 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:25.327+0000 7fe23f487700 1 --2- 172.21.15.177:0/1965111145 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe2240a11d0 0x7fe2240a3680 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe234005950 tx=0x7fe2340058e0 comp rx=0 tx=0).stop 2023-11-27T15:17:25.331 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:25.327+0000 7fe23f487700 1 -- 172.21.15.177:0/1965111145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe238100d70 msgr2=0x7fe238101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:25.331 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:25.327+0000 7fe23f487700 1 --2- 172.21.15.177:0/1965111145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe238100d70 0x7fe238101d00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe2300062a0 tx=0x7fe230004d00 comp rx=0 tx=0).stop 2023-11-27T15:17:25.331 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:25.327+0000 7fe23f487700 1 -- 172.21.15.177:0/1965111145 shutdown_connections 2023-11-27T15:17:25.332 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:25.327+0000 7fe23f487700 1 --2- 172.21.15.177:0/1965111145 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe2240a11d0 0x7fe2240a3680 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:25.332 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:25.327+0000 7fe23f487700 1 --2- 172.21.15.177:0/1965111145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe238100d70 0x7fe238101d00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:25.332 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:25.328+0000 7fe23f487700 1 --2- 172.21.15.177:0/1965111145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2380ff1e0 0x7fe238103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:25.332 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:25.328+0000 7fe23f487700 1 --2- 172.21.15.177:0/1965111145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2381003e0 0x7fe2381017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:25.332 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:25.328+0000 7fe23f487700 1 -- 172.21.15.177:0/1965111145 >> 172.21.15.177:0/1965111145 conn(0x7fe2380fa7f0 msgr2=0x7fe238068ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:25.332 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:25.328+0000 7fe23f487700 1 -- 172.21.15.177:0/1965111145 shutdown_connections 2023-11-27T15:17:25.332 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:17:25.328+0000 7fe23f487700 1 -- 172.21.15.177:0/1965111145 wait complete. 2023-11-27T15:17:25.498 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:25 smithi177 ceph-mon[131504]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:25.499 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:25 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:25.499 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:25 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:25.499 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:25 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:25.499 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:25 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:25.499 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:25 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:25.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:25 smithi133 ceph-mon[97860]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:25.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:25 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:25.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:25 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:25.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:25 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:25.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:25 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:25.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:25 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:25.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:25 smithi174 ceph-mon[131110]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:25.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:25 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:25.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:25 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:25.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:25 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:25.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:25 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:25.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:25 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:25.770 DEBUG:teuthology.orchestra.run.smithi177:osd.5> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.5.service 2023-11-27T15:17:25.772 INFO:tasks.cephadm:Deploying osd.6 on smithi133 with /dev/vg_nvme/lv_2... 2023-11-27T15:17:25.772 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-11-27T15:17:25.804 INFO:journalctl@ceph.osd.5.smithi177.stdout:-- Logs begin at Mon 2023-11-27 14:50:38 UTC. -- 2023-11-27T15:17:26.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:26 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:17:26.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:26 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:17:26.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:26 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:17:26.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:26.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:26.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:26.557 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:17:26 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:17:26.414+0000 7f854ee4c640 -1 osd.5 0 log_to_monitors true 2023-11-27T15:17:26.557 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:26 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:17:26.557 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:26 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:17:26.557 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:26 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:17:26.557 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:26.557 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:26.557 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:26.639 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:17:26.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:26 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:17:26.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:26 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:17:26.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:26 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:17:26.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:26.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:26.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:27.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:27 smithi177 ceph-mon[131504]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:27.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:27 smithi177 ceph-mon[131504]: from='osd.5 [v2:172.21.15.177:6808/3863044388,v1:172.21.15.177:6809/3863044388]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-27T15:17:27.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:27 smithi177 ceph-mon[131504]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-27T15:17:27.458 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:27 smithi133 ceph-mon[97860]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:27.458 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:27 smithi133 ceph-mon[97860]: from='osd.5 [v2:172.21.15.177:6808/3863044388,v1:172.21.15.177:6809/3863044388]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-27T15:17:27.458 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:27 smithi133 ceph-mon[97860]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-27T15:17:27.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:27 smithi174 ceph-mon[131110]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:27.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:27 smithi174 ceph-mon[131110]: from='osd.5 [v2:172.21.15.177:6808/3863044388,v1:172.21.15.177:6809/3863044388]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-27T15:17:27.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:27 smithi174 ceph-mon[131110]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-11-27T15:17:28.588 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:17:28 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:17:28.309+0000 7f85444c3700 -1 osd.5 0 waiting for initial osdmap 2023-11-27T15:17:28.588 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:17:28 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:17:28.316+0000 7f853dab3700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:17:28.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:28 smithi177 ceph-mon[131504]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-11-27T15:17:28.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:28 smithi177 ceph-mon[131504]: osdmap e36: 6 total, 5 up, 6 in 2023-11-27T15:17:28.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:28 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:17:28.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:28 smithi177 ceph-mon[131504]: from='osd.5 [v2:172.21.15.177:6808/3863044388,v1:172.21.15.177:6809/3863044388]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:17:28.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:28 smithi177 ceph-mon[131504]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:17:28.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:28 smithi177 ceph-mon[131504]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:28.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:28 smithi177 ceph-mon[131504]: Detected new or changed devices on smithi177 2023-11-27T15:17:28.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:28 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:28.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:28 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:28.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:28 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:28.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:28 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:28.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:28 smithi177 ceph-mon[131504]: Adjusting osd_memory_target on smithi177 to 8584M 2023-11-27T15:17:28.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:28 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:28.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:28 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:28.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:28 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:28.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:28 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:17:28.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:28 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:28.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:28 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:17:28.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:28 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:17:28.627 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-11-27T15:17:28.627 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-11-27T15:17:28.627 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2023-11-27T15:17:28.627 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2023-11-27T15:17:28.627 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0391072 s, 268 MB/s 2023-11-27T15:17:28.628 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2023-11-27T15:17:28.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:28 smithi174 ceph-mon[131110]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-11-27T15:17:28.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:28 smithi174 ceph-mon[131110]: osdmap e36: 6 total, 5 up, 6 in 2023-11-27T15:17:28.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:28 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:17:28.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:28 smithi174 ceph-mon[131110]: from='osd.5 [v2:172.21.15.177:6808/3863044388,v1:172.21.15.177:6809/3863044388]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:17:28.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:28 smithi174 ceph-mon[131110]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:17:28.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:28 smithi174 ceph-mon[131110]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:28.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:28 smithi174 ceph-mon[131110]: Detected new or changed devices on smithi177 2023-11-27T15:17:28.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:28 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:28.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:28 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:28.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:28 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:28.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:28 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:28.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:28 smithi174 ceph-mon[131110]: Adjusting osd_memory_target on smithi177 to 8584M 2023-11-27T15:17:28.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:28 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:28.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:28 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:28.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:28 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:28.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:28 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:17:28.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:28 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:28.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:28 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:17:28.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:28 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:17:28.664 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:28 smithi133 ceph-mon[97860]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-11-27T15:17:28.664 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:28 smithi133 ceph-mon[97860]: osdmap e36: 6 total, 5 up, 6 in 2023-11-27T15:17:28.664 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:17:28.664 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:28 smithi133 ceph-mon[97860]: from='osd.5 [v2:172.21.15.177:6808/3863044388,v1:172.21.15.177:6809/3863044388]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:17:28.664 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:28 smithi133 ceph-mon[97860]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:17:28.664 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:28 smithi133 ceph-mon[97860]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:28 smithi133 ceph-mon[97860]: Detected new or changed devices on smithi177 2023-11-27T15:17:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:28 smithi133 ceph-mon[97860]: Adjusting osd_memory_target on smithi177 to 8584M 2023-11-27T15:17:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:28.665 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:28 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:17:28.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:28.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:28 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:17:28.666 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:28 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:17:29.118 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi133:vg_nvme/lv_2 2023-11-27T15:17:29.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:29 smithi177 ceph-mon[131504]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2023-11-27T15:17:29.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:29 smithi177 ceph-mon[131504]: osdmap e37: 6 total, 5 up, 6 in 2023-11-27T15:17:29.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:29 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:17:29.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:29 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:17:29.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:29 smithi174 ceph-mon[131110]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2023-11-27T15:17:29.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:29 smithi174 ceph-mon[131110]: osdmap e37: 6 total, 5 up, 6 in 2023-11-27T15:17:29.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:29 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:17:29.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:29 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:17:29.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:29 smithi133 ceph-mon[97860]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2023-11-27T15:17:29.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:29 smithi133 ceph-mon[97860]: osdmap e37: 6 total, 5 up, 6 in 2023-11-27T15:17:29.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:29 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:17:29.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:29 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:17:29.970 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:17:30.523 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:30 smithi133 ceph-mon[97860]: purged_snaps scrub starts 2023-11-27T15:17:30.523 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:30 smithi133 ceph-mon[97860]: purged_snaps scrub ok 2023-11-27T15:17:30.524 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:30 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:17:30.524 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:30 smithi133 ceph-mon[97860]: osd.5 [v2:172.21.15.177:6808/3863044388,v1:172.21.15.177:6809/3863044388] boot 2023-11-27T15:17:30.524 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:30 smithi133 ceph-mon[97860]: osdmap e38: 6 total, 6 up, 6 in 2023-11-27T15:17:30.524 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:30 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:17:30.524 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:30 smithi133 ceph-mon[97860]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:30.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:30 smithi177 ceph-mon[131504]: purged_snaps scrub starts 2023-11-27T15:17:30.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:30 smithi177 ceph-mon[131504]: purged_snaps scrub ok 2023-11-27T15:17:30.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:30 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:17:30.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:30 smithi177 ceph-mon[131504]: osd.5 [v2:172.21.15.177:6808/3863044388,v1:172.21.15.177:6809/3863044388] boot 2023-11-27T15:17:30.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:30 smithi177 ceph-mon[131504]: osdmap e38: 6 total, 6 up, 6 in 2023-11-27T15:17:30.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:30 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:17:30.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:30 smithi177 ceph-mon[131504]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:30.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:30 smithi174 ceph-mon[131110]: purged_snaps scrub starts 2023-11-27T15:17:30.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:30 smithi174 ceph-mon[131110]: purged_snaps scrub ok 2023-11-27T15:17:30.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:30 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:17:30.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:30 smithi174 ceph-mon[131110]: osd.5 [v2:172.21.15.177:6808/3863044388,v1:172.21.15.177:6809/3863044388] boot 2023-11-27T15:17:30.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:30 smithi174 ceph-mon[131110]: osdmap e38: 6 total, 6 up, 6 in 2023-11-27T15:17:30.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:30 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-11-27T15:17:30.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:30 smithi174 ceph-mon[131110]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 30 MiB used, 447 GiB / 447 GiB avail 2023-11-27T15:17:31.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.358+0000 7ff24a125700 1 -- 172.21.15.133:0/2995281977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff244074b20 msgr2=0x7ff244072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:31.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.358+0000 7ff24a125700 1 --2- 172.21.15.133:0/2995281977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff244074b20 0x7ff244072f80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff2340099c0 tx=0x7ff234009cd0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.358+0000 7ff24a125700 1 -- 172.21.15.133:0/2995281977 shutdown_connections 2023-11-27T15:17:31.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.358+0000 7ff24a125700 1 --2- 172.21.15.133:0/2995281977 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff24410c8d0 0x7ff24410ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.358+0000 7ff24a125700 1 --2- 172.21.15.133:0/2995281977 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff244074b20 0x7ff244072f80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.358+0000 7ff24a125700 1 --2- 172.21.15.133:0/2995281977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff244073550 0x7ff2440739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.358+0000 7ff24a125700 1 -- 172.21.15.133:0/2995281977 >> 172.21.15.133:0/2995281977 conn(0x7ff2440fb900 msgr2=0x7ff2440fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:31.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.358+0000 7ff24a125700 1 -- 172.21.15.133:0/2995281977 shutdown_connections 2023-11-27T15:17:31.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.359+0000 7ff24a125700 1 -- 172.21.15.133:0/2995281977 wait complete. 2023-11-27T15:17:31.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.359+0000 7ff24a125700 1 Processor -- start 2023-11-27T15:17:31.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.360+0000 7ff24a125700 1 -- start start 2023-11-27T15:17:31.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.360+0000 7ff24a125700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff244073550 0x7ff24419c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:31.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.360+0000 7ff24a125700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff244074b20 0x7ff24419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:31.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.360+0000 7ff24a125700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff24410c8d0 0x7ff2441a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:31.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.360+0000 7ff24a125700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2441a41e0 con 0x7ff24410c8d0 2023-11-27T15:17:31.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.360+0000 7ff24a125700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2441a4320 con 0x7ff244074b20 2023-11-27T15:17:31.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.360+0000 7ff24a125700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2441a44b0 con 0x7ff244073550 2023-11-27T15:17:31.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.360+0000 7ff2437fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff244073550 0x7ff24419c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:31.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.360+0000 7ff2437fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff244073550 0x7ff24419c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:43174/0 (socket says 172.21.15.133:43174) 2023-11-27T15:17:31.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.360+0000 7ff2437fe700 1 -- 172.21.15.133:0/539487687 learned_addr learned my addr 172.21.15.133:0/539487687 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:17:31.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.360+0000 7ff243fff700 1 --2- 172.21.15.133:0/539487687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff24410c8d0 0x7ff2441a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:31.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.360+0000 7ff242ffd700 1 --2- 172.21.15.133:0/539487687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff244074b20 0x7ff24419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:31.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.361+0000 7ff2437fe700 1 -- 172.21.15.133:0/539487687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff244074b20 msgr2=0x7ff24419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:31.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.361+0000 7ff2437fe700 1 --2- 172.21.15.133:0/539487687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff244074b20 0x7ff24419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.361+0000 7ff2437fe700 1 -- 172.21.15.133:0/539487687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff24410c8d0 msgr2=0x7ff2441a3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:31.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.361+0000 7ff2437fe700 1 --2- 172.21.15.133:0/539487687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff24410c8d0 0x7ff2441a3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:31.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.361+0000 7ff2437fe700 1 -- 172.21.15.133:0/539487687 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2340095f0 con 0x7ff244073550 2023-11-27T15:17:31.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.361+0000 7ff2437fe700 1 --2- 172.21.15.133:0/539487687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff244073550 0x7ff24419c650 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff2340098e0 tx=0x7ff23400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:31.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.361+0000 7ff240ff9700 1 -- 172.21.15.133:0/539487687 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23401d070 con 0x7ff244073550 2023-11-27T15:17:31.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.361+0000 7ff24a125700 1 -- 172.21.15.133:0/539487687 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2441122a0 con 0x7ff244073550 2023-11-27T15:17:31.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.361+0000 7ff240ff9700 1 -- 172.21.15.133:0/539487687 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff234005030 con 0x7ff244073550 2023-11-27T15:17:31.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.361+0000 7ff240ff9700 1 -- 172.21.15.133:0/539487687 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff234004a70 con 0x7ff244073550 2023-11-27T15:17:31.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.361+0000 7ff24a125700 1 -- 172.21.15.133:0/539487687 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff244112110 con 0x7ff244073550 2023-11-27T15:17:31.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.362+0000 7ff24a125700 1 -- 172.21.15.133:0/539487687 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff244102420 con 0x7ff244073550 2023-11-27T15:17:31.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.364+0000 7ff240ff9700 1 -- 172.21.15.133:0/539487687 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff2340051a0 con 0x7ff244073550 2023-11-27T15:17:31.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.366+0000 7ff240ff9700 1 --2- 172.21.15.133:0/539487687 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff22c0a1150 0x7ff22c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:31.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.366+0000 7ff240ff9700 1 -- 172.21.15.133:0/539487687 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7ff2340cc890 con 0x7ff244073550 2023-11-27T15:17:31.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.366+0000 7ff242ffd700 1 --2- 172.21.15.133:0/539487687 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff22c0a1150 0x7ff22c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:31.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.366+0000 7ff242ffd700 1 --2- 172.21.15.133:0/539487687 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff22c0a1150 0x7ff22c0a3600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff238009c60 tx=0x7ff238009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:31.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.369+0000 7ff240ff9700 1 -- 172.21.15.133:0/539487687 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff234096eb0 con 0x7ff244073550 2023-11-27T15:17:31.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:31 smithi133 ceph-mon[97860]: osdmap e39: 6 total, 6 up, 6 in 2023-11-27T15:17:31.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:31.578+0000 7ff24a125700 1 -- 172.21.15.133:0/539487687 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2440619f0 con 0x7ff22c0a1150 2023-11-27T15:17:31.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:31 smithi177 ceph-mon[131504]: osdmap e39: 6 total, 6 up, 6 in 2023-11-27T15:17:31.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:31 smithi174 ceph-mon[131110]: osdmap e39: 6 total, 6 up, 6 in 2023-11-27T15:17:32.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:32 smithi177 ceph-mon[131504]: osdmap e40: 6 total, 6 up, 6 in 2023-11-27T15:17:32.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:32 smithi177 ceph-mon[131504]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:32.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:32 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:17:32.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:32 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:17:32.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:32 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:32.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:32 smithi177 ceph-mon[131504]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:32.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:32 smithi177 ceph-mon[131504]: osdmap e41: 6 total, 6 up, 6 in 2023-11-27T15:17:32.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:32 smithi174 ceph-mon[131110]: osdmap e40: 6 total, 6 up, 6 in 2023-11-27T15:17:32.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:32 smithi174 ceph-mon[131110]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:32.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:32 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:17:32.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:32 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:17:32.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:32 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:32.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:32 smithi174 ceph-mon[131110]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:32.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:32 smithi174 ceph-mon[131110]: osdmap e41: 6 total, 6 up, 6 in 2023-11-27T15:17:32.767 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:32 smithi133 ceph-mon[97860]: osdmap e40: 6 total, 6 up, 6 in 2023-11-27T15:17:32.768 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:32 smithi133 ceph-mon[97860]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:32.768 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:32 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:17:32.768 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:32 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:17:32.768 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:32 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:32.768 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:32 smithi133 ceph-mon[97860]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:32.768 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:32 smithi133 ceph-mon[97860]: osdmap e41: 6 total, 6 up, 6 in 2023-11-27T15:17:34.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:34 smithi177 ceph-mon[131504]: osdmap e42: 6 total, 6 up, 6 in 2023-11-27T15:17:34.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:34 smithi174 ceph-mon[131110]: osdmap e42: 6 total, 6 up, 6 in 2023-11-27T15:17:34.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:34 smithi133 ceph-mon[97860]: osdmap e42: 6 total, 6 up, 6 in 2023-11-27T15:17:35.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:35 smithi177 ceph-mon[131504]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:35.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:35 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3018163632' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2abbda56-2257-4a87-9f75-8491329c2df8"}]: dispatch 2023-11-27T15:17:35.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:35 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3018163632' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2abbda56-2257-4a87-9f75-8491329c2df8"}]': finished 2023-11-27T15:17:35.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:35 smithi177 ceph-mon[131504]: osdmap e43: 7 total, 6 up, 7 in 2023-11-27T15:17:35.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:35 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:17:35.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:35 smithi174 ceph-mon[131110]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:35.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:35 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3018163632' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2abbda56-2257-4a87-9f75-8491329c2df8"}]: dispatch 2023-11-27T15:17:35.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:35 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3018163632' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2abbda56-2257-4a87-9f75-8491329c2df8"}]': finished 2023-11-27T15:17:35.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:35 smithi174 ceph-mon[131110]: osdmap e43: 7 total, 6 up, 7 in 2023-11-27T15:17:35.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:35 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:17:35.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:35 smithi133 ceph-mon[97860]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 35 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:35.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:35 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3018163632' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2abbda56-2257-4a87-9f75-8491329c2df8"}]: dispatch 2023-11-27T15:17:35.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:35 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3018163632' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2abbda56-2257-4a87-9f75-8491329c2df8"}]': finished 2023-11-27T15:17:35.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:35 smithi133 ceph-mon[97860]: osdmap e43: 7 total, 6 up, 7 in 2023-11-27T15:17:35.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:35 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:17:36.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:36 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2611305283' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:17:36.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:36 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2611305283' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:17:36.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:36 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2611305283' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:17:37.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:37 smithi177 ceph-mon[131504]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 133 KiB/s, 0 objects/s recovering 2023-11-27T15:17:37.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:37 smithi174 ceph-mon[131110]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 133 KiB/s, 0 objects/s recovering 2023-11-27T15:17:37.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:37 smithi133 ceph-mon[97860]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 133 KiB/s, 0 objects/s recovering 2023-11-27T15:17:39.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:39 smithi177 ceph-mon[131504]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-27T15:17:39.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:39 smithi174 ceph-mon[131110]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-27T15:17:39.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:39 smithi133 ceph-mon[97860]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-27T15:17:41.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:41 smithi177 ceph-mon[131504]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-11-27T15:17:41.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:41 smithi174 ceph-mon[131110]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-11-27T15:17:41.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:41 smithi133 ceph-mon[97860]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-11-27T15:17:43.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:43 smithi177 ceph-mon[131504]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2023-11-27T15:17:43.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:43 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-11-27T15:17:43.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:43 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:43.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:43 smithi177 ceph-mon[131504]: Deploying daemon osd.6 on smithi133 2023-11-27T15:17:43.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:43 smithi133 ceph-mon[97860]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2023-11-27T15:17:43.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:43 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-11-27T15:17:43.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:43 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:43.397 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:43 smithi133 ceph-mon[97860]: Deploying daemon osd.6 on smithi133 2023-11-27T15:17:43.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:43 smithi174 ceph-mon[131110]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2023-11-27T15:17:43.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:43 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-11-27T15:17:43.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:43 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:43.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:43 smithi174 ceph-mon[131110]: Deploying daemon osd.6 on smithi133 2023-11-27T15:17:45.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:45 smithi174 ceph-mon[131110]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-11-27T15:17:45.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:45 smithi133 ceph-mon[97860]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-11-27T15:17:45.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:45 smithi177 ceph-mon[131504]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-11-27T15:17:47.352 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:47 smithi133 ceph-mon[97860]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:47.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:47 smithi174 ceph-mon[131110]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:47.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:47 smithi177 ceph-mon[131504]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:49.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:49 smithi174 ceph-mon[131110]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:49.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:49 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:49.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:49 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:49.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:49 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:49.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:49 smithi133 ceph-mon[97860]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:49.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:49 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:49.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:49 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:49.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:49 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:49.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:49 smithi177 ceph-mon[131504]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:49.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:49 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:49.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:49 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:49.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:49 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:51.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:51 smithi174 ceph-mon[131110]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:51.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:51 smithi133 ceph-mon[97860]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:51.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:51 smithi177 ceph-mon[131504]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:52.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:52.037+0000 7ff240ff9700 1 -- 172.21.15.133:0/539487687 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff2440619f0 con 0x7ff22c0a1150 2023-11-27T15:17:52.042 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 6 on host 'smithi133' 2023-11-27T15:17:52.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:52.039+0000 7ff24a125700 1 -- 172.21.15.133:0/539487687 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff22c0a1150 msgr2=0x7ff22c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:52.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:52.039+0000 7ff24a125700 1 --2- 172.21.15.133:0/539487687 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff22c0a1150 0x7ff22c0a3600 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff238009c60 tx=0x7ff238009260 comp rx=0 tx=0).stop 2023-11-27T15:17:52.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:52.040+0000 7ff24a125700 1 -- 172.21.15.133:0/539487687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff244073550 msgr2=0x7ff24419c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:52.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:52.040+0000 7ff24a125700 1 --2- 172.21.15.133:0/539487687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff244073550 0x7ff24419c650 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff2340098e0 tx=0x7ff23400f690 comp rx=0 tx=0).stop 2023-11-27T15:17:52.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:52.040+0000 7ff24a125700 1 -- 172.21.15.133:0/539487687 shutdown_connections 2023-11-27T15:17:52.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:52.040+0000 7ff24a125700 1 --2- 172.21.15.133:0/539487687 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff22c0a1150 0x7ff22c0a3600 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:52.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:52.040+0000 7ff24a125700 1 --2- 172.21.15.133:0/539487687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff24410c8d0 0x7ff2441a3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:52.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:52.040+0000 7ff24a125700 1 --2- 172.21.15.133:0/539487687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff244073550 0x7ff24419c650 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:52.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:52.040+0000 7ff24a125700 1 --2- 172.21.15.133:0/539487687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff244074b20 0x7ff24419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:52.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:52.040+0000 7ff24a125700 1 -- 172.21.15.133:0/539487687 >> 172.21.15.133:0/539487687 conn(0x7ff2440fb900 msgr2=0x7ff2440fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:52.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:52.040+0000 7ff24a125700 1 -- 172.21.15.133:0/539487687 shutdown_connections 2023-11-27T15:17:52.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:17:52.040+0000 7ff24a125700 1 -- 172.21.15.133:0/539487687 wait complete. 2023-11-27T15:17:52.507 DEBUG:teuthology.orchestra.run.smithi133:osd.6> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.6.service 2023-11-27T15:17:52.511 INFO:tasks.cephadm:Deploying osd.7 on smithi174 with /dev/vg_nvme/lv_2... 2023-11-27T15:17:52.511 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-11-27T15:17:52.589 INFO:journalctl@ceph.osd.6.smithi133.stdout:-- Logs begin at Mon 2023-11-27 14:54:33 UTC. -- 2023-11-27T15:17:52.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:52 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:52.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:52 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:52.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:52 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:52.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:52 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:52.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:52 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:17:52.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:52 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:52.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:52 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:17:52.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:52 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:17:52.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:52 smithi133 ceph-mon[97860]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:52.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:52 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:52.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:52 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:52.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:52 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:52.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:52 smithi133 ceph-mon[97860]: from='osd.6 [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-27T15:17:52.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:52 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:52.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:52 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:52.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:52 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:52.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:52 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:52.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:52 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:17:52.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:52 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:52.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:52 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:17:52.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:52 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:17:52.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:52 smithi177 ceph-mon[131504]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:52.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:52 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:52.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:52 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:52.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:52 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:52.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:52 smithi177 ceph-mon[131504]: from='osd.6 [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-27T15:17:52.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:52 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:52.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:52 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:52.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:52 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:52.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:52 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:52.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:52 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:17:52.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:52 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:52.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:52 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:17:52.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:52 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:17:52.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:52 smithi174 ceph-mon[131110]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:52.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:52 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:52.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:52 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:52.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:52 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:17:52.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:52 smithi174 ceph-mon[131110]: from='osd.6 [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-11-27T15:17:53.163 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:17:54.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:54 smithi177 ceph-mon[131504]: from='osd.6 [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-11-27T15:17:54.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:54 smithi177 ceph-mon[131504]: osdmap e44: 7 total, 6 up, 7 in 2023-11-27T15:17:54.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:54 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:17:54.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:54 smithi177 ceph-mon[131504]: from='osd.6 [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-11-27T15:17:54.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:54 smithi174 ceph-mon[131110]: from='osd.6 [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-11-27T15:17:54.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:54 smithi174 ceph-mon[131110]: osdmap e44: 7 total, 6 up, 7 in 2023-11-27T15:17:54.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:54 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:17:54.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:54 smithi174 ceph-mon[131110]: from='osd.6 [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-11-27T15:17:54.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:54 smithi133 ceph-mon[97860]: from='osd.6 [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-11-27T15:17:54.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:54 smithi133 ceph-mon[97860]: osdmap e44: 7 total, 6 up, 7 in 2023-11-27T15:17:54.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:54 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:17:54.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:54 smithi133 ceph-mon[97860]: from='osd.6 [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-11-27T15:17:54.529 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:17:54 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:17:54.072+0000 7f3ae58f9700 -1 osd.6 0 waiting for initial osdmap 2023-11-27T15:17:54.529 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:17:54 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:17:54.088+0000 7f3adf6ea700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:17:54.735 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-11-27T15:17:54.735 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-11-27T15:17:54.735 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2023-11-27T15:17:54.736 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2023-11-27T15:17:54.736 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0148756 s, 705 MB/s 2023-11-27T15:17:54.736 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2023-11-27T15:17:55.117 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi174:vg_nvme/lv_2 2023-11-27T15:17:55.226 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:55 smithi133 ceph-mon[97860]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:55.226 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:55 smithi133 ceph-mon[97860]: from='osd.6 [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-11-27T15:17:55.226 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:55 smithi133 ceph-mon[97860]: osdmap e45: 7 total, 6 up, 7 in 2023-11-27T15:17:55.226 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:55 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:17:55.226 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:55 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:17:55.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:55 smithi177 ceph-mon[131504]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:55.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:55 smithi177 ceph-mon[131504]: from='osd.6 [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-11-27T15:17:55.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:55 smithi177 ceph-mon[131504]: osdmap e45: 7 total, 6 up, 7 in 2023-11-27T15:17:55.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:55 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:17:55.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:55 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:17:55.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:55 smithi174 ceph-mon[131110]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:55.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:55 smithi174 ceph-mon[131110]: from='osd.6 [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-11-27T15:17:55.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:55 smithi174 ceph-mon[131110]: osdmap e45: 7 total, 6 up, 7 in 2023-11-27T15:17:55.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:55 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:17:55.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:55 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:17:55.761 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:17:56.262 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:56 smithi174 ceph-mon[131110]: purged_snaps scrub starts 2023-11-27T15:17:56.263 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:56 smithi174 ceph-mon[131110]: purged_snaps scrub ok 2023-11-27T15:17:56.263 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:56 smithi174 ceph-mon[131110]: osd.6 [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592] boot 2023-11-27T15:17:56.263 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:56 smithi174 ceph-mon[131110]: osdmap e46: 7 total, 7 up, 7 in 2023-11-27T15:17:56.263 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:56 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:17:56.263 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:56 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:56.263 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:56 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:56.263 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:56 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:56.263 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:56 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:56.264 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:56 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:56.264 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:56 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:56.264 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:56 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:56.264 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:56 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:56.264 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:56 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:56.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:56 smithi177 ceph-mon[131504]: purged_snaps scrub starts 2023-11-27T15:17:56.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:56 smithi177 ceph-mon[131504]: purged_snaps scrub ok 2023-11-27T15:17:56.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:56 smithi177 ceph-mon[131504]: osd.6 [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592] boot 2023-11-27T15:17:56.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:56 smithi177 ceph-mon[131504]: osdmap e46: 7 total, 7 up, 7 in 2023-11-27T15:17:56.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:56 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:17:56.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:56 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:56.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:56 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:56.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:56 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:56.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:56 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:56.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:56 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:56.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:56 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:56.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:56 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:56.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:56 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:56.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:56 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:56.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:56 smithi133 ceph-mon[97860]: purged_snaps scrub starts 2023-11-27T15:17:56.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:56 smithi133 ceph-mon[97860]: purged_snaps scrub ok 2023-11-27T15:17:56.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:56 smithi133 ceph-mon[97860]: osd.6 [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592] boot 2023-11-27T15:17:56.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:56 smithi133 ceph-mon[97860]: osdmap e46: 7 total, 7 up, 7 in 2023-11-27T15:17:56.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:56 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-11-27T15:17:56.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:56 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:56.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:56 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:56.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:56 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:56.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:56 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:56.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:56 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:17:56.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:56 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:56.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:56 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:56.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:56 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:17:56.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:56 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:17:56.910 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.906+0000 7f477d187700 1 -- 172.21.15.174:0/1369192408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47780ff1e0 msgr2=0x7f47780ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:56.911 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.906+0000 7f477d187700 1 --2- 172.21.15.174:0/1369192408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47780ff1e0 0x7f47780ff5f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f476c009990 tx=0x7f476c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.911 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.906+0000 7f477d187700 1 -- 172.21.15.174:0/1369192408 shutdown_connections 2023-11-27T15:17:56.911 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.906+0000 7f477d187700 1 --2- 172.21.15.174:0/1369192408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4778100d70 0x7f4778107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.911 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.906+0000 7f477d187700 1 --2- 172.21.15.174:0/1369192408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47780ff1e0 0x7f47780ff5f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.911 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.906+0000 7f477d187700 1 --2- 172.21.15.174:0/1369192408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47781003e0 0x7f4778100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.911 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.906+0000 7f477d187700 1 -- 172.21.15.174:0/1369192408 >> 172.21.15.174:0/1369192408 conn(0x7f47780fa7f0 msgr2=0x7f47780fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:17:56.911 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.906+0000 7f477d187700 1 -- 172.21.15.174:0/1369192408 shutdown_connections 2023-11-27T15:17:56.911 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.906+0000 7f477d187700 1 -- 172.21.15.174:0/1369192408 wait complete. 2023-11-27T15:17:56.911 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.907+0000 7f477d187700 1 Processor -- start 2023-11-27T15:17:56.913 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.907+0000 7f477d187700 1 -- start start 2023-11-27T15:17:56.913 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.907+0000 7f477d187700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47780ff1e0 0x7f4778195090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:56.913 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.907+0000 7f477d187700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47781003e0 0x7f47781955d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:56.914 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.907+0000 7f477d187700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4778100d70 0x7f477819c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:56.914 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.907+0000 7f477d187700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477819cc20 con 0x7f47780ff1e0 2023-11-27T15:17:56.914 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.907+0000 7f477d187700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477819cd60 con 0x7f4778100d70 2023-11-27T15:17:56.914 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.907+0000 7f477d187700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477819cef0 con 0x7f47781003e0 2023-11-27T15:17:56.914 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.907+0000 7f477759e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4778100d70 0x7f477819c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:56.914 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.907+0000 7f477759e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4778100d70 0x7f477819c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:48496/0 (socket says 172.21.15.174:48496) 2023-11-27T15:17:56.914 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.907+0000 7f477759e700 1 -- 172.21.15.174:0/2656725766 learned_addr learned my addr 172.21.15.174:0/2656725766 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-27T15:17:56.914 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.907+0000 7f477659c700 1 --2- 172.21.15.174:0/2656725766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47781003e0 0x7f47781955d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:56.914 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.907+0000 7f477759e700 1 -- 172.21.15.174:0/2656725766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47781003e0 msgr2=0x7f47781955d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:56.915 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.907+0000 7f477759e700 1 --2- 172.21.15.174:0/2656725766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47781003e0 0x7f47781955d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.915 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.907+0000 7f477759e700 1 -- 172.21.15.174:0/2656725766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47780ff1e0 msgr2=0x7f4778195090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:17:56.915 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.907+0000 7f477759e700 1 --2- 172.21.15.174:0/2656725766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47780ff1e0 0x7f4778195090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:17:56.915 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.907+0000 7f477759e700 1 -- 172.21.15.174:0/2656725766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c0095f0 con 0x7f4778100d70 2023-11-27T15:17:56.915 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.907+0000 7f477759e700 1 --2- 172.21.15.174:0/2656725766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4778100d70 0x7f477819c650 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f477000db90 tx=0x7f477000df50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:56.915 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.908+0000 7f475ffff700 1 -- 172.21.15.174:0/2656725766 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477000e800 con 0x7f4778100d70 2023-11-27T15:17:56.915 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.908+0000 7f477d187700 1 -- 172.21.15.174:0/2656725766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477810ac30 con 0x7f4778100d70 2023-11-27T15:17:56.915 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.908+0000 7f475ffff700 1 -- 172.21.15.174:0/2656725766 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f477000bac0 con 0x7f4778100d70 2023-11-27T15:17:56.915 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.908+0000 7f475ffff700 1 -- 172.21.15.174:0/2656725766 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47700163f0 con 0x7f4778100d70 2023-11-27T15:17:56.915 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.908+0000 7f477d187700 1 -- 172.21.15.174:0/2656725766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477810aaa0 con 0x7f4778100d70 2023-11-27T15:17:56.915 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.909+0000 7f477d187700 1 -- 172.21.15.174:0/2656725766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4778066ea0 con 0x7f4778100d70 2023-11-27T15:17:56.916 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.910+0000 7f475ffff700 1 -- 172.21.15.174:0/2656725766 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4770009580 con 0x7f4778100d70 2023-11-27T15:17:56.916 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.910+0000 7f475ffff700 1 --2- 172.21.15.174:0/2656725766 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f47600a1150 0x7f47600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:17:56.916 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.911+0000 7f475ffff700 1 -- 172.21.15.174:0/2656725766 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f4770006e40 con 0x7f4778100d70 2023-11-27T15:17:56.916 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.911+0000 7f4776d9d700 1 --2- 172.21.15.174:0/2656725766 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f47600a1150 0x7f47600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:17:56.916 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.912+0000 7f4776d9d700 1 --2- 172.21.15.174:0/2656725766 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f47600a1150 0x7f47600a3600 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f476c0098e0 tx=0x7f476c003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:17:56.918 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:56.914+0000 7f475ffff700 1 -- 172.21.15.174:0/2656725766 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f47700977d0 con 0x7f4778100d70 2023-11-27T15:17:57.112 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:17:57.109+0000 7f477d187700 1 -- 172.21.15.174:0/2656725766 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f4778060f30 con 0x7f47600a1150 2023-11-27T15:17:57.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:57 smithi177 ceph-mon[131504]: Detected new or changed devices on smithi133 2023-11-27T15:17:57.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:57 smithi177 ceph-mon[131504]: Adjusting osd_memory_target on smithi133 to 5757M 2023-11-27T15:17:57.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:57 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:17:57.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:57 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:17:57.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:57 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:17:57.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:57 smithi177 ceph-mon[131504]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:57.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:57 smithi177 ceph-mon[131504]: osdmap e47: 7 total, 7 up, 7 in 2023-11-27T15:17:57.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:57 smithi174 ceph-mon[131110]: Detected new or changed devices on smithi133 2023-11-27T15:17:57.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:57 smithi174 ceph-mon[131110]: Adjusting osd_memory_target on smithi133 to 5757M 2023-11-27T15:17:57.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:57 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:17:57.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:57 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:17:57.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:57 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:17:57.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:57 smithi174 ceph-mon[131110]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:57.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:57 smithi174 ceph-mon[131110]: osdmap e47: 7 total, 7 up, 7 in 2023-11-27T15:17:57.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:57 smithi133 ceph-mon[97860]: Detected new or changed devices on smithi133 2023-11-27T15:17:57.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:57 smithi133 ceph-mon[97860]: Adjusting osd_memory_target on smithi133 to 5757M 2023-11-27T15:17:57.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:57 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:17:57.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:57 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:17:57.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:57 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:17:57.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:57 smithi133 ceph-mon[97860]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 36 MiB used, 536 GiB / 536 GiB avail 2023-11-27T15:17:57.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:57 smithi133 ceph-mon[97860]: osdmap e47: 7 total, 7 up, 7 in 2023-11-27T15:17:58.302 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:58 smithi174 ceph-mon[131110]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:58.302 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:58 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:17:58.303 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:58 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:17:58.303 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:58 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:58.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:58 smithi133 ceph-mon[97860]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:58.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:58 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:17:58.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:58 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:17:58.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:58 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:58.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:58 smithi177 ceph-mon[131504]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:17:58.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:58 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:17:58.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:58 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:17:58.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:58 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:17:59.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:17:59 smithi174 ceph-mon[131110]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:17:59.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:17:59 smithi133 ceph-mon[97860]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:17:59.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:17:59 smithi177 ceph-mon[131504]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:00.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:00 smithi174 ceph-mon[131110]: from='client.? 172.21.15.174:0/3962283280' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "106a6b4b-99fc-44a8-a4a4-c635b25101d3"}]: dispatch 2023-11-27T15:18:00.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:00 smithi174 ceph-mon[131110]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "106a6b4b-99fc-44a8-a4a4-c635b25101d3"}]: dispatch 2023-11-27T15:18:00.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:00 smithi174 ceph-mon[131110]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "106a6b4b-99fc-44a8-a4a4-c635b25101d3"}]': finished 2023-11-27T15:18:00.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:00 smithi174 ceph-mon[131110]: osdmap e48: 8 total, 7 up, 8 in 2023-11-27T15:18:00.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:00 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:18:00.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:00 smithi133 ceph-mon[97860]: from='client.? 172.21.15.174:0/3962283280' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "106a6b4b-99fc-44a8-a4a4-c635b25101d3"}]: dispatch 2023-11-27T15:18:00.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:00 smithi133 ceph-mon[97860]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "106a6b4b-99fc-44a8-a4a4-c635b25101d3"}]: dispatch 2023-11-27T15:18:00.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:00 smithi133 ceph-mon[97860]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "106a6b4b-99fc-44a8-a4a4-c635b25101d3"}]': finished 2023-11-27T15:18:00.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:00 smithi133 ceph-mon[97860]: osdmap e48: 8 total, 7 up, 8 in 2023-11-27T15:18:00.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:00 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:18:00.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:00 smithi177 ceph-mon[131504]: from='client.? 172.21.15.174:0/3962283280' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "106a6b4b-99fc-44a8-a4a4-c635b25101d3"}]: dispatch 2023-11-27T15:18:00.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:00 smithi177 ceph-mon[131504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "106a6b4b-99fc-44a8-a4a4-c635b25101d3"}]: dispatch 2023-11-27T15:18:00.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:00 smithi177 ceph-mon[131504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "106a6b4b-99fc-44a8-a4a4-c635b25101d3"}]': finished 2023-11-27T15:18:00.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:00 smithi177 ceph-mon[131504]: osdmap e48: 8 total, 7 up, 8 in 2023-11-27T15:18:00.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:00 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:18:01.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:01 smithi174 ceph-mon[131110]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:01.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:01 smithi174 ceph-mon[131110]: from='client.? 172.21.15.174:0/446004130' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:18:01.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:01 smithi133 ceph-mon[97860]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:01.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:01 smithi133 ceph-mon[97860]: from='client.? 172.21.15.174:0/446004130' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:18:01.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:01 smithi177 ceph-mon[131504]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:01.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:01 smithi177 ceph-mon[131504]: from='client.? 172.21.15.174:0/446004130' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:18:03.360 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:03 smithi174 ceph-mon[131110]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:03.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:03 smithi133 ceph-mon[97860]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:03.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:03 smithi177 ceph-mon[131504]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 41 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:05.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:05 smithi174 ceph-mon[131110]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:05.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:05 smithi133 ceph-mon[97860]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:05.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:05 smithi177 ceph-mon[131504]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:07.276 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:07 smithi174 ceph-mon[131110]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:07.276 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:07 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-11-27T15:18:07.276 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:07 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:07.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:07 smithi133 ceph-mon[97860]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:07.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:07 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-11-27T15:18:07.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:07 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:07.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:07 smithi177 ceph-mon[131504]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:07.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:07 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-11-27T15:18:07.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:07 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:08.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:08 smithi174 ceph-mon[131110]: Deploying daemon osd.7 on smithi174 2023-11-27T15:18:08.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:08 smithi133 ceph-mon[97860]: Deploying daemon osd.7 on smithi174 2023-11-27T15:18:08.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:08 smithi177 ceph-mon[131504]: Deploying daemon osd.7 on smithi174 2023-11-27T15:18:09.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:09 smithi174 ceph-mon[131110]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:09.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:09 smithi133 ceph-mon[97860]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:09.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:09 smithi177 ceph-mon[131504]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:11.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:11 smithi174 ceph-mon[131110]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:11.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:11 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:11.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:11 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:11.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:11 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:11.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:11 smithi133 ceph-mon[97860]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:11.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:11 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:11.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:11 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:11.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:11 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:11.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:11 smithi177 ceph-mon[131504]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:11.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:11 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:11.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:11 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:11.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:11 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:13.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:13 smithi174 ceph-mon[131110]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:13.476 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:18:13.471+0000 7f475ffff700 1 -- 172.21.15.174:0/2656725766 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4778060f30 con 0x7f47600a1150 2023-11-27T15:18:13.477 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 7 on host 'smithi174' 2023-11-27T15:18:13.478 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:18:13.473+0000 7f477d187700 1 -- 172.21.15.174:0/2656725766 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f47600a1150 msgr2=0x7f47600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:13.478 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:18:13.473+0000 7f477d187700 1 --2- 172.21.15.174:0/2656725766 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f47600a1150 0x7f47600a3600 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f476c0098e0 tx=0x7f476c003680 comp rx=0 tx=0).stop 2023-11-27T15:18:13.478 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:18:13.473+0000 7f477d187700 1 -- 172.21.15.174:0/2656725766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4778100d70 msgr2=0x7f477819c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:13.478 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:18:13.473+0000 7f477d187700 1 --2- 172.21.15.174:0/2656725766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4778100d70 0x7f477819c650 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f477000db90 tx=0x7f477000df50 comp rx=0 tx=0).stop 2023-11-27T15:18:13.478 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:18:13.473+0000 7f477d187700 1 -- 172.21.15.174:0/2656725766 shutdown_connections 2023-11-27T15:18:13.479 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:18:13.473+0000 7f477d187700 1 --2- 172.21.15.174:0/2656725766 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f47600a1150 0x7f47600a3600 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:13.479 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:18:13.473+0000 7f477d187700 1 --2- 172.21.15.174:0/2656725766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4778100d70 0x7f477819c650 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:13.479 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:18:13.473+0000 7f477d187700 1 --2- 172.21.15.174:0/2656725766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47780ff1e0 0x7f4778195090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:13.479 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:18:13.473+0000 7f477d187700 1 --2- 172.21.15.174:0/2656725766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47781003e0 0x7f47781955d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:13.479 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:18:13.473+0000 7f477d187700 1 -- 172.21.15.174:0/2656725766 >> 172.21.15.174:0/2656725766 conn(0x7f47780fa7f0 msgr2=0x7f47780fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:13.479 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:18:13.473+0000 7f477d187700 1 -- 172.21.15.174:0/2656725766 shutdown_connections 2023-11-27T15:18:13.479 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:18:13.473+0000 7f477d187700 1 -- 172.21.15.174:0/2656725766 wait complete. 2023-11-27T15:18:13.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:13 smithi133 ceph-mon[97860]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:13.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:13 smithi177 ceph-mon[131504]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:13.947 DEBUG:teuthology.orchestra.run.smithi174:osd.7> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.7.service 2023-11-27T15:18:13.949 INFO:tasks.cephadm:Deploying osd.8 on smithi177 with /dev/vg_nvme/lv_2... 2023-11-27T15:18:13.949 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-11-27T15:18:13.979 INFO:journalctl@ceph.osd.7.smithi174.stdout:-- Logs begin at Mon 2023-11-27 14:50:40 UTC. -- 2023-11-27T15:18:14.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:14 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:14.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:14 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:14.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:14 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:14.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:14 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:14.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:14 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:18:14.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:14 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:14.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:14 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:18:14.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:14 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:18:14.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:14 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:14.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:14 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:14.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:14 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:14.446 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:14 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:14.446 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:14 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:14.446 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:14 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:14.447 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:14 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:14.447 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:14 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:18:14.447 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:14 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:14.447 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:14 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:18:14.447 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:14 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:18:14.447 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:14 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:14.447 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:14 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:14.447 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:14 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:14.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:14.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:14.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:14.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:14.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:14 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:18:14.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:14.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:14 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:18:14.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:14 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:18:14.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:14.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:14.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:14.600 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:18:14.717 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:18:14 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:18:14.624+0000 7fc0da2bc640 -1 osd.7 0 log_to_monitors true 2023-11-27T15:18:15.417 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:15 smithi177 ceph-mon[131504]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:15.417 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:15 smithi177 ceph-mon[131504]: from='osd.7 [v2:172.21.15.174:6816/2275667196,v1:172.21.15.174:6817/2275667196]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-27T15:18:15.417 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:15 smithi177 ceph-mon[131504]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-27T15:18:15.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:15 smithi133 ceph-mon[97860]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:15.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:15 smithi133 ceph-mon[97860]: from='osd.7 [v2:172.21.15.174:6816/2275667196,v1:172.21.15.174:6817/2275667196]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-27T15:18:15.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:15 smithi133 ceph-mon[97860]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-27T15:18:15.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:15 smithi174 ceph-mon[131110]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:15.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:15 smithi174 ceph-mon[131110]: from='osd.7 [v2:172.21.15.174:6816/2275667196,v1:172.21.15.174:6817/2275667196]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-27T15:18:15.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:15 smithi174 ceph-mon[131110]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-11-27T15:18:16.088 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-11-27T15:18:16.088 INFO:teuthology.orchestra.run.smithi177.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-11-27T15:18:16.088 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10+0 records in 2023-11-27T15:18:16.088 INFO:teuthology.orchestra.run.smithi177.stderr:10+0 records out 2023-11-27T15:18:16.088 INFO:teuthology.orchestra.run.smithi177.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.01083 s, 968 MB/s 2023-11-27T15:18:16.088 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping successful for: 2023-11-27T15:18:16.552 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi177:vg_nvme/lv_2 2023-11-27T15:18:16.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:16 smithi133 ceph-mon[97860]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-11-27T15:18:16.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:16 smithi133 ceph-mon[97860]: osdmap e49: 8 total, 7 up, 8 in 2023-11-27T15:18:16.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:18:16.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:16 smithi133 ceph-mon[97860]: from='osd.7 [v2:172.21.15.174:6816/2275667196,v1:172.21.15.174:6817/2275667196]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:18:16.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:16 smithi133 ceph-mon[97860]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:18:16.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:16 smithi133 ceph-mon[97860]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:16.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:16 smithi133 ceph-mon[97860]: Detected new or changed devices on smithi174 2023-11-27T15:18:16.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:16.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:16.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:16.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:16.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:16.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:16.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:16.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:16.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:16.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:16 smithi177 ceph-mon[131504]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-11-27T15:18:16.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:16 smithi177 ceph-mon[131504]: osdmap e49: 8 total, 7 up, 8 in 2023-11-27T15:18:16.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:18:16.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:16 smithi177 ceph-mon[131504]: from='osd.7 [v2:172.21.15.174:6816/2275667196,v1:172.21.15.174:6817/2275667196]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:18:16.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:16 smithi177 ceph-mon[131504]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:18:16.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:16 smithi177 ceph-mon[131504]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:16.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:16 smithi177 ceph-mon[131504]: Detected new or changed devices on smithi174 2023-11-27T15:18:16.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:16.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:16.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:16.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:16.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:16.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:16.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:16.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:16.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:16.903 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:18:16 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:18:16.439+0000 7fc0cf933700 -1 osd.7 0 waiting for initial osdmap 2023-11-27T15:18:16.903 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:18:16 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:18:16.451+0000 7fc0c9f25700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:18:16.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:16 smithi174 ceph-mon[131110]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-11-27T15:18:16.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:16 smithi174 ceph-mon[131110]: osdmap e49: 8 total, 7 up, 8 in 2023-11-27T15:18:16.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:18:16.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:16 smithi174 ceph-mon[131110]: from='osd.7 [v2:172.21.15.174:6816/2275667196,v1:172.21.15.174:6817/2275667196]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:18:16.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:16 smithi174 ceph-mon[131110]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:18:16.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:16 smithi174 ceph-mon[131110]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 42 MiB used, 626 GiB / 626 GiB avail 2023-11-27T15:18:16.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:16 smithi174 ceph-mon[131110]: Detected new or changed devices on smithi174 2023-11-27T15:18:16.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:16.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:16.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:16.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:16.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:16.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:16.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:16.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:16.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:17.191 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:18:17.723 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:17 smithi177 ceph-mon[131504]: Adjusting osd_memory_target on smithi174 to 5722M 2023-11-27T15:18:17.723 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:17 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:18:17.723 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:17 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:18:17.723 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:17 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:18:17.723 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:17 smithi177 ceph-mon[131504]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2023-11-27T15:18:17.723 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:17 smithi177 ceph-mon[131504]: osdmap e50: 8 total, 7 up, 8 in 2023-11-27T15:18:17.723 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:17 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:18:17.723 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:17 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:18:17.724 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:17 smithi177 ceph-mon[131504]: osd.7 [v2:172.21.15.174:6816/2275667196,v1:172.21.15.174:6817/2275667196] boot 2023-11-27T15:18:17.724 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:17 smithi177 ceph-mon[131504]: osdmap e51: 8 total, 8 up, 8 in 2023-11-27T15:18:17.724 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:17 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:18:17.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:17 smithi133 ceph-mon[97860]: Adjusting osd_memory_target on smithi174 to 5722M 2023-11-27T15:18:17.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:17 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:18:17.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:17 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:18:17.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:17 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:18:17.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:17 smithi133 ceph-mon[97860]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2023-11-27T15:18:17.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:17 smithi133 ceph-mon[97860]: osdmap e50: 8 total, 7 up, 8 in 2023-11-27T15:18:17.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:17 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:18:17.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:17 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:18:17.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:17 smithi133 ceph-mon[97860]: osd.7 [v2:172.21.15.174:6816/2275667196,v1:172.21.15.174:6817/2275667196] boot 2023-11-27T15:18:17.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:17 smithi133 ceph-mon[97860]: osdmap e51: 8 total, 8 up, 8 in 2023-11-27T15:18:17.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:17 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:18:17.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:17 smithi174 ceph-mon[131110]: Adjusting osd_memory_target on smithi174 to 5722M 2023-11-27T15:18:17.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:17 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:18:17.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:17 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:18:17.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:17 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:18:17.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:17 smithi174 ceph-mon[131110]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2023-11-27T15:18:17.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:17 smithi174 ceph-mon[131110]: osdmap e50: 8 total, 7 up, 8 in 2023-11-27T15:18:17.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:17 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:18:17.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:17 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:18:17.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:17 smithi174 ceph-mon[131110]: osd.7 [v2:172.21.15.174:6816/2275667196,v1:172.21.15.174:6817/2275667196] boot 2023-11-27T15:18:17.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:17 smithi174 ceph-mon[131110]: osdmap e51: 8 total, 8 up, 8 in 2023-11-27T15:18:17.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:17 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-11-27T15:18:18.377 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.371+0000 7ff7ad14d700 1 -- 172.21.15.177:0/4234596784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7a81014f0 msgr2=0x7ff7a8101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:18.378 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.371+0000 7ff7ad14d700 1 --2- 172.21.15.177:0/4234596784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7a81014f0 0x7ff7a8101940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff79c009990 tx=0x7ff79c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.378 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.371+0000 7ff7ad14d700 1 -- 172.21.15.177:0/4234596784 shutdown_connections 2023-11-27T15:18:18.378 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.371+0000 7ff7ad14d700 1 --2- 172.21.15.177:0/4234596784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7a8101e80 0x7ff7a8108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.378 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.371+0000 7ff7ad14d700 1 --2- 172.21.15.177:0/4234596784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a81002f0 0x7ff7a8100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.378 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.371+0000 7ff7ad14d700 1 --2- 172.21.15.177:0/4234596784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7a81014f0 0x7ff7a8101940 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.378 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.371+0000 7ff7ad14d700 1 -- 172.21.15.177:0/4234596784 >> 172.21.15.177:0/4234596784 conn(0x7ff7a80fb900 msgr2=0x7ff7a80fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:18.378 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.371+0000 7ff7ad14d700 1 -- 172.21.15.177:0/4234596784 shutdown_connections 2023-11-27T15:18:18.378 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.371+0000 7ff7ad14d700 1 -- 172.21.15.177:0/4234596784 wait complete. 2023-11-27T15:18:18.378 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.372+0000 7ff7ad14d700 1 Processor -- start 2023-11-27T15:18:18.379 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.372+0000 7ff7ad14d700 1 -- start start 2023-11-27T15:18:18.379 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.372+0000 7ff7ad14d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7a81002f0 0x7ff7a8196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.379 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.372+0000 7ff7ad14d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a81014f0 0x7ff7a81965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.379 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.372+0000 7ff7ad14d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7a8101e80 0x7ff7a819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.379 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.372+0000 7ff7ad14d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a819dbf0 con 0x7ff7a81014f0 2023-11-27T15:18:18.379 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.372+0000 7ff7ad14d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a819dd30 con 0x7ff7a81002f0 2023-11-27T15:18:18.379 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.372+0000 7ff7ad14d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a819dec0 con 0x7ff7a8101e80 2023-11-27T15:18:18.379 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.372+0000 7ff7a759e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7a8101e80 0x7ff7a819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.379 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.372+0000 7ff7a759e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7a8101e80 0x7ff7a819d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:58024/0 (socket says 172.21.15.177:58024) 2023-11-27T15:18:18.379 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.372+0000 7ff7a759e700 1 -- 172.21.15.177:0/634643594 learned_addr learned my addr 172.21.15.177:0/634643594 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-27T15:18:18.379 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.372+0000 7ff7a6d9d700 1 --2- 172.21.15.177:0/634643594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7a81002f0 0x7ff7a8196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.379 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.372+0000 7ff7a659c700 1 --2- 172.21.15.177:0/634643594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a81014f0 0x7ff7a81965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.379 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.372+0000 7ff7a759e700 1 -- 172.21.15.177:0/634643594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7a81002f0 msgr2=0x7ff7a8196060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:18.380 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.372+0000 7ff7a759e700 1 --2- 172.21.15.177:0/634643594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7a81002f0 0x7ff7a8196060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.380 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.372+0000 7ff7a759e700 1 -- 172.21.15.177:0/634643594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a81014f0 msgr2=0x7ff7a81965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:18.380 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.372+0000 7ff7a759e700 1 --2- 172.21.15.177:0/634643594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a81014f0 0x7ff7a81965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:18.380 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.372+0000 7ff7a759e700 1 -- 172.21.15.177:0/634643594 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79c0095f0 con 0x7ff7a8101e80 2023-11-27T15:18:18.380 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.372+0000 7ff7a659c700 1 --2- 172.21.15.177:0/634643594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a81014f0 0x7ff7a81965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:18:18.380 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.373+0000 7ff7a759e700 1 --2- 172.21.15.177:0/634643594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7a8101e80 0x7ff7a819d620 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0007d40 tx=0x7ff7a000a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:18.380 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.373+0000 7ff78ffff700 1 -- 172.21.15.177:0/634643594 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a000ad60 con 0x7ff7a8101e80 2023-11-27T15:18:18.380 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.373+0000 7ff78ffff700 1 -- 172.21.15.177:0/634643594 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7a00128e0 con 0x7ff7a8101e80 2023-11-27T15:18:18.380 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.373+0000 7ff78ffff700 1 -- 172.21.15.177:0/634643594 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a00117e0 con 0x7ff7a8101e80 2023-11-27T15:18:18.380 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.373+0000 7ff7ad14d700 1 -- 172.21.15.177:0/634643594 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a810bcf0 con 0x7ff7a8101e80 2023-11-27T15:18:18.380 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.373+0000 7ff7ad14d700 1 -- 172.21.15.177:0/634643594 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a810bb60 con 0x7ff7a8101e80 2023-11-27T15:18:18.381 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.374+0000 7ff7ad14d700 1 -- 172.21.15.177:0/634643594 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a81078a0 con 0x7ff7a8101e80 2023-11-27T15:18:18.381 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.375+0000 7ff78ffff700 1 -- 172.21.15.177:0/634643594 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff7a0012420 con 0x7ff7a8101e80 2023-11-27T15:18:18.381 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.375+0000 7ff78ffff700 1 --2- 172.21.15.177:0/634643594 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff7900a11d0 0x7ff7900a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:18.381 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.375+0000 7ff78ffff700 1 -- 172.21.15.177:0/634643594 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5426+0+0 (secure 0 0 0) 0x7ff7a00cc080 con 0x7ff7a8101e80 2023-11-27T15:18:18.382 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.376+0000 7ff7a6d9d700 1 --2- 172.21.15.177:0/634643594 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff7900a11d0 0x7ff7900a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:18.382 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.376+0000 7ff7a6d9d700 1 --2- 172.21.15.177:0/634643594 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff7900a11d0 0x7ff7900a3680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff798006fd0 tx=0x7ff798008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:18.384 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.380+0000 7ff78ffff700 1 -- 172.21.15.177:0/634643594 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff7a0096310 con 0x7ff7a8101e80 2023-11-27T15:18:18.578 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:18.574+0000 7ff7ad14d700 1 -- 172.21.15.177:0/634643594 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7a819e080 con 0x7ff7900a11d0 2023-11-27T15:18:18.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:18 smithi133 ceph-mon[97860]: purged_snaps scrub starts 2023-11-27T15:18:18.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:18 smithi133 ceph-mon[97860]: purged_snaps scrub ok 2023-11-27T15:18:18.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:18 smithi133 ceph-mon[97860]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:18.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:18 smithi133 ceph-mon[97860]: osdmap e52: 8 total, 8 up, 8 in 2023-11-27T15:18:18.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:18 smithi177 ceph-mon[131504]: purged_snaps scrub starts 2023-11-27T15:18:18.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:18 smithi177 ceph-mon[131504]: purged_snaps scrub ok 2023-11-27T15:18:18.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:18 smithi177 ceph-mon[131504]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:18.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:18 smithi177 ceph-mon[131504]: osdmap e52: 8 total, 8 up, 8 in 2023-11-27T15:18:18.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:18 smithi174 ceph-mon[131110]: purged_snaps scrub starts 2023-11-27T15:18:18.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:18 smithi174 ceph-mon[131110]: purged_snaps scrub ok 2023-11-27T15:18:18.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:18 smithi174 ceph-mon[131110]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:18.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:18 smithi174 ceph-mon[131110]: osdmap e52: 8 total, 8 up, 8 in 2023-11-27T15:18:19.777 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:19 smithi177 ceph-mon[131504]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:18:19.777 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:18:19.777 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:18:19.778 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:19.778 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:19 smithi177 ceph-mon[131504]: osdmap e53: 8 total, 8 up, 8 in 2023-11-27T15:18:19.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:19 smithi133 ceph-mon[97860]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:18:19.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:18:19.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:18:19.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:19.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:19 smithi133 ceph-mon[97860]: osdmap e53: 8 total, 8 up, 8 in 2023-11-27T15:18:19.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:19 smithi174 ceph-mon[131110]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:18:19.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:18:19.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:18:19.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:19.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:19 smithi174 ceph-mon[131110]: osdmap e53: 8 total, 8 up, 8 in 2023-11-27T15:18:21.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:21 smithi177 ceph-mon[131504]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:21.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:21 smithi177 ceph-mon[131504]: osdmap e54: 8 total, 8 up, 8 in 2023-11-27T15:18:21.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:21 smithi177 ceph-mon[131504]: from='client.? 172.21.15.177:0/3639660545' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1541498-5236-447b-b194-d8da4226318e"}]: dispatch 2023-11-27T15:18:21.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:21 smithi177 ceph-mon[131504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1541498-5236-447b-b194-d8da4226318e"}]: dispatch 2023-11-27T15:18:21.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:21 smithi177 ceph-mon[131504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1541498-5236-447b-b194-d8da4226318e"}]': finished 2023-11-27T15:18:21.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:21 smithi177 ceph-mon[131504]: osdmap e55: 9 total, 8 up, 9 in 2023-11-27T15:18:21.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:21 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:18:21.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:21 smithi174 ceph-mon[131110]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:21.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:21 smithi174 ceph-mon[131110]: osdmap e54: 8 total, 8 up, 8 in 2023-11-27T15:18:21.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:21 smithi174 ceph-mon[131110]: from='client.? 172.21.15.177:0/3639660545' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1541498-5236-447b-b194-d8da4226318e"}]: dispatch 2023-11-27T15:18:21.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:21 smithi174 ceph-mon[131110]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1541498-5236-447b-b194-d8da4226318e"}]: dispatch 2023-11-27T15:18:21.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:21 smithi174 ceph-mon[131110]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1541498-5236-447b-b194-d8da4226318e"}]': finished 2023-11-27T15:18:21.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:21 smithi174 ceph-mon[131110]: osdmap e55: 9 total, 8 up, 9 in 2023-11-27T15:18:21.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:21 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:18:21.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:21 smithi133 ceph-mon[97860]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 47 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:21.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:21 smithi133 ceph-mon[97860]: osdmap e54: 8 total, 8 up, 8 in 2023-11-27T15:18:21.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:21 smithi133 ceph-mon[97860]: from='client.? 172.21.15.177:0/3639660545' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1541498-5236-447b-b194-d8da4226318e"}]: dispatch 2023-11-27T15:18:21.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:21 smithi133 ceph-mon[97860]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d1541498-5236-447b-b194-d8da4226318e"}]: dispatch 2023-11-27T15:18:21.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:21 smithi133 ceph-mon[97860]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d1541498-5236-447b-b194-d8da4226318e"}]': finished 2023-11-27T15:18:21.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:21 smithi133 ceph-mon[97860]: osdmap e55: 9 total, 8 up, 9 in 2023-11-27T15:18:21.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:21 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:18:22.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:22 smithi177 ceph-mon[131504]: from='client.? 172.21.15.177:0/1899711810' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:18:22.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:22 smithi174 ceph-mon[131110]: from='client.? 172.21.15.177:0/1899711810' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:18:22.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:22 smithi133 ceph-mon[97860]: from='client.? 172.21.15.177:0/1899711810' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:18:23.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:23 smithi177 ceph-mon[131504]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:23.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:23 smithi174 ceph-mon[131110]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:23.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:23 smithi133 ceph-mon[97860]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:25.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:25 smithi177 ceph-mon[131504]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:25.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:25 smithi174 ceph-mon[131110]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:25.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:25 smithi133 ceph-mon[97860]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:27.346 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:27 smithi177 ceph-mon[131504]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:27.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:27 smithi174 ceph-mon[131110]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:27.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:27 smithi133 ceph-mon[97860]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:28.352 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:28 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-11-27T15:18:28.354 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:28 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:28.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:28 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-11-27T15:18:28.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:28 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:28.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-11-27T15:18:28.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:29.219 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:29 smithi177 ceph-mon[131504]: Deploying daemon osd.8 on smithi177 2023-11-27T15:18:29.219 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:29 smithi177 ceph-mon[131504]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:29.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:29 smithi174 ceph-mon[131110]: Deploying daemon osd.8 on smithi177 2023-11-27T15:18:29.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:29 smithi174 ceph-mon[131110]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:29.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:29 smithi133 ceph-mon[97860]: Deploying daemon osd.8 on smithi177 2023-11-27T15:18:29.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:29 smithi133 ceph-mon[97860]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:31.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:31 smithi174 ceph-mon[131110]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:31.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:31 smithi133 ceph-mon[97860]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:31.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:31 smithi177 ceph-mon[131504]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:33.247 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:32 smithi177 ceph-mon[131504]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:33.247 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:32 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:33.247 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:32 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:33.247 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:32 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:33.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:32 smithi133 ceph-mon[97860]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:33.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:32 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:33.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:32 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:33.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:32 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:33.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:32 smithi174 ceph-mon[131110]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:33.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:32 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:33.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:32 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:33.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:32 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:34.797 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:34.793+0000 7ff78ffff700 1 -- 172.21.15.177:0/634643594 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff7a819e080 con 0x7ff7900a11d0 2023-11-27T15:18:34.797 INFO:teuthology.orchestra.run.smithi177.stdout:Created osd(s) 8 on host 'smithi177' 2023-11-27T15:18:34.799 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:34.795+0000 7ff7ad14d700 1 -- 172.21.15.177:0/634643594 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff7900a11d0 msgr2=0x7ff7900a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:34.799 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:34.795+0000 7ff7ad14d700 1 --2- 172.21.15.177:0/634643594 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff7900a11d0 0x7ff7900a3680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff798006fd0 tx=0x7ff798008040 comp rx=0 tx=0).stop 2023-11-27T15:18:34.799 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:34.795+0000 7ff7ad14d700 1 -- 172.21.15.177:0/634643594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7a8101e80 msgr2=0x7ff7a819d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:34.799 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:34.795+0000 7ff7ad14d700 1 --2- 172.21.15.177:0/634643594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7a8101e80 0x7ff7a819d620 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0007d40 tx=0x7ff7a000a340 comp rx=0 tx=0).stop 2023-11-27T15:18:34.799 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:34.795+0000 7ff7ad14d700 1 -- 172.21.15.177:0/634643594 shutdown_connections 2023-11-27T15:18:34.799 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:34.795+0000 7ff7ad14d700 1 --2- 172.21.15.177:0/634643594 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff7900a11d0 0x7ff7900a3680 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:34.799 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:34.795+0000 7ff7ad14d700 1 --2- 172.21.15.177:0/634643594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7a8101e80 0x7ff7a819d620 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:34.799 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:34.795+0000 7ff7ad14d700 1 --2- 172.21.15.177:0/634643594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7a81002f0 0x7ff7a8196060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:34.799 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:34.795+0000 7ff7ad14d700 1 --2- 172.21.15.177:0/634643594 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7a81014f0 0x7ff7a81965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:34.799 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:34.795+0000 7ff7ad14d700 1 -- 172.21.15.177:0/634643594 >> 172.21.15.177:0/634643594 conn(0x7ff7a80fb900 msgr2=0x7ff7a81049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:34.799 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:34.795+0000 7ff7ad14d700 1 -- 172.21.15.177:0/634643594 shutdown_connections 2023-11-27T15:18:34.800 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:18:34.795+0000 7ff7ad14d700 1 -- 172.21.15.177:0/634643594 wait complete. 2023-11-27T15:18:35.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:35 smithi177 ceph-mon[131504]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:35.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:35 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:35.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:35 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:35.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:35 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:35.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:35 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:35.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:35 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:35.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:35 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:35.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:35 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:35.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:35 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:35.175 DEBUG:teuthology.orchestra.run.smithi177:osd.8> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.8.service 2023-11-27T15:18:35.177 INFO:tasks.cephadm:Deploying osd.9 on smithi133 with /dev/vg_nvme/lv_1... 2023-11-27T15:18:35.178 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-11-27T15:18:35.209 INFO:journalctl@ceph.osd.8.smithi177.stdout:-- Logs begin at Mon 2023-11-27 14:50:38 UTC. -- 2023-11-27T15:18:35.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:35 smithi133 ceph-mon[97860]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:35.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:35 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:35.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:35 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:35.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:35 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:35.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:35 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:35.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:35 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:35.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:35 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:35.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:35 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:35.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:35 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:35.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:35 smithi174 ceph-mon[131110]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:35.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:35 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:35.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:35 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:35.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:35 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:35.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:35 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:35.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:35 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:35.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:35 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:35.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:35 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:35.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:35 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:36.009 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:18:35 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:18:35.964+0000 7f2cf2182640 -1 osd.8 0 log_to_monitors true 2023-11-27T15:18:36.028 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:18:36.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:36 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:18:36.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:36 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:18:36.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:36 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:18:36.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:36 smithi133 ceph-mon[97860]: from='osd.8 [v2:172.21.15.177:6816/3783894564,v1:172.21.15.177:6817/3783894564]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-27T15:18:36.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:36 smithi133 ceph-mon[97860]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-27T15:18:36.287 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:36 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:18:36.287 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:36 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:18:36.287 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:36 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:18:36.287 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:36 smithi177 ceph-mon[131504]: from='osd.8 [v2:172.21.15.177:6816/3783894564,v1:172.21.15.177:6817/3783894564]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-27T15:18:36.287 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:36 smithi177 ceph-mon[131504]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-27T15:18:36.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:36 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:18:36.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:36 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:18:36.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:36 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:18:36.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:36 smithi174 ceph-mon[131110]: from='osd.8 [v2:172.21.15.177:6816/3783894564,v1:172.21.15.177:6817/3783894564]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-27T15:18:36.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:36 smithi174 ceph-mon[131110]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-11-27T15:18:37.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:37 smithi133 ceph-mon[97860]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:37.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:37 smithi133 ceph-mon[97860]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-11-27T15:18:37.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:37 smithi133 ceph-mon[97860]: osdmap e56: 9 total, 8 up, 9 in 2023-11-27T15:18:37.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:37 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:18:37.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:37 smithi133 ceph-mon[97860]: from='osd.8 [v2:172.21.15.177:6816/3783894564,v1:172.21.15.177:6817/3783894564]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:18:37.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:37 smithi133 ceph-mon[97860]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:18:37.338 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:18:37 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:18:37.015+0000 7f2ce77f9700 -1 osd.8 0 waiting for initial osdmap 2023-11-27T15:18:37.338 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:18:37 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:18:37.029+0000 7f2ce35ee700 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:18:37.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:37 smithi177 ceph-mon[131504]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:37.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:37 smithi177 ceph-mon[131504]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-11-27T15:18:37.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:37 smithi177 ceph-mon[131504]: osdmap e56: 9 total, 8 up, 9 in 2023-11-27T15:18:37.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:37 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:18:37.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:37 smithi177 ceph-mon[131504]: from='osd.8 [v2:172.21.15.177:6816/3783894564,v1:172.21.15.177:6817/3783894564]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:18:37.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:37 smithi177 ceph-mon[131504]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:18:37.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:37 smithi174 ceph-mon[131110]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:37.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:37 smithi174 ceph-mon[131110]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-11-27T15:18:37.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:37 smithi174 ceph-mon[131110]: osdmap e56: 9 total, 8 up, 9 in 2023-11-27T15:18:37.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:37 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:18:37.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:37 smithi174 ceph-mon[131110]: from='osd.8 [v2:172.21.15.177:6816/3783894564,v1:172.21.15.177:6817/3783894564]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:18:37.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:37 smithi174 ceph-mon[131110]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:18:37.899 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-11-27T15:18:37.899 INFO:teuthology.orchestra.run.smithi133.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-11-27T15:18:37.899 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10+0 records in 2023-11-27T15:18:37.900 INFO:teuthology.orchestra.run.smithi133.stderr:10+0 records out 2023-11-27T15:18:37.900 INFO:teuthology.orchestra.run.smithi133.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0303867 s, 345 MB/s 2023-11-27T15:18:37.900 INFO:teuthology.orchestra.run.smithi133.stderr:--> Zapping successful for: 2023-11-27T15:18:38.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:38 smithi133 ceph-mon[97860]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2023-11-27T15:18:38.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:38 smithi133 ceph-mon[97860]: osdmap e57: 9 total, 8 up, 9 in 2023-11-27T15:18:38.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:38 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:18:38.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:38 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:18:38.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:38 smithi133 ceph-mon[97860]: osd.8 [v2:172.21.15.177:6816/3783894564,v1:172.21.15.177:6817/3783894564] boot 2023-11-27T15:18:38.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:38 smithi133 ceph-mon[97860]: osdmap e58: 9 total, 9 up, 9 in 2023-11-27T15:18:38.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:38 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:18:38.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:38 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:38.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:38 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:38.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:38 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:38.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:38 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:38.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:38 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:38.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:38 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:38.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:38 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:38.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:38 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:38.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:38 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:38.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:38 smithi177 ceph-mon[131504]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2023-11-27T15:18:38.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:38 smithi177 ceph-mon[131504]: osdmap e57: 9 total, 8 up, 9 in 2023-11-27T15:18:38.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:38 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:18:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:38 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:18:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:38 smithi177 ceph-mon[131504]: osd.8 [v2:172.21.15.177:6816/3783894564,v1:172.21.15.177:6817/3783894564] boot 2023-11-27T15:18:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:38 smithi177 ceph-mon[131504]: osdmap e58: 9 total, 9 up, 9 in 2023-11-27T15:18:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:38 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:18:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:38 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:38 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:38 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:38 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:38 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:38 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:38 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:38.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:38 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:38.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:38 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:38 smithi174 ceph-mon[131110]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2023-11-27T15:18:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:38 smithi174 ceph-mon[131110]: osdmap e57: 9 total, 8 up, 9 in 2023-11-27T15:18:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:38 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:18:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:38 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:18:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:38 smithi174 ceph-mon[131110]: osd.8 [v2:172.21.15.177:6816/3783894564,v1:172.21.15.177:6817/3783894564] boot 2023-11-27T15:18:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:38 smithi174 ceph-mon[131110]: osdmap e58: 9 total, 9 up, 9 in 2023-11-27T15:18:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:38 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-11-27T15:18:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:38 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:38 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:38 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:38.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:38 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:38.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:38 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:18:38.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:38 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:38.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:38 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:38.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:38 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:18:38.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:38 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:38.409 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi133:vg_nvme/lv_1 2023-11-27T15:18:39.262 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:18:39.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:39 smithi174 ceph-mon[131110]: purged_snaps scrub starts 2023-11-27T15:18:39.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:39 smithi174 ceph-mon[131110]: purged_snaps scrub ok 2023-11-27T15:18:39.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:39 smithi174 ceph-mon[131110]: Detected new or changed devices on smithi177 2023-11-27T15:18:39.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:39 smithi174 ceph-mon[131110]: Adjusting osd_memory_target on smithi177 to 5722M 2023-11-27T15:18:39.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:39 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:18:39.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:39 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:18:39.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:39 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:18:39.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:39 smithi174 ceph-mon[131110]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:39.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:39 smithi174 ceph-mon[131110]: osdmap e59: 9 total, 9 up, 9 in 2023-11-27T15:18:39.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:39 smithi133 ceph-mon[97860]: purged_snaps scrub starts 2023-11-27T15:18:39.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:39 smithi133 ceph-mon[97860]: purged_snaps scrub ok 2023-11-27T15:18:39.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:39 smithi133 ceph-mon[97860]: Detected new or changed devices on smithi177 2023-11-27T15:18:39.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:39 smithi133 ceph-mon[97860]: Adjusting osd_memory_target on smithi177 to 5722M 2023-11-27T15:18:39.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:39 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:18:39.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:39 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:18:39.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:39 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:18:39.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:39 smithi133 ceph-mon[97860]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:39.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:39 smithi133 ceph-mon[97860]: osdmap e59: 9 total, 9 up, 9 in 2023-11-27T15:18:39.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:39 smithi177 ceph-mon[131504]: purged_snaps scrub starts 2023-11-27T15:18:39.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:39 smithi177 ceph-mon[131504]: purged_snaps scrub ok 2023-11-27T15:18:39.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:39 smithi177 ceph-mon[131504]: Detected new or changed devices on smithi177 2023-11-27T15:18:39.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:39 smithi177 ceph-mon[131504]: Adjusting osd_memory_target on smithi177 to 5722M 2023-11-27T15:18:39.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:39 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:18:39.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:39 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:18:39.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:39 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:18:39.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:39 smithi177 ceph-mon[131504]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 48 MiB used, 715 GiB / 715 GiB avail 2023-11-27T15:18:39.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:39 smithi177 ceph-mon[131504]: osdmap e59: 9 total, 9 up, 9 in 2023-11-27T15:18:40.387 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:40 smithi133 ceph-mon[97860]: osdmap e60: 9 total, 9 up, 9 in 2023-11-27T15:18:40.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:40 smithi174 ceph-mon[131110]: osdmap e60: 9 total, 9 up, 9 in 2023-11-27T15:18:40.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:40 smithi177 ceph-mon[131504]: osdmap e60: 9 total, 9 up, 9 in 2023-11-27T15:18:40.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.629+0000 7fbf895df700 1 -- 172.21.15.133:0/4294941127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf84101e80 msgr2=0x7fbf84108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:40.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.629+0000 7fbf895df700 1 --2- 172.21.15.133:0/4294941127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf84101e80 0x7fbf84108740 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c00b0a0 tx=0x7fbf7c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:18:40.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.629+0000 7fbf895df700 1 -- 172.21.15.133:0/4294941127 shutdown_connections 2023-11-27T15:18:40.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.629+0000 7fbf895df700 1 --2- 172.21.15.133:0/4294941127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf84101e80 0x7fbf84108740 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:40.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.629+0000 7fbf895df700 1 --2- 172.21.15.133:0/4294941127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf841002f0 0x7fbf84100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:40.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.629+0000 7fbf895df700 1 --2- 172.21.15.133:0/4294941127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf841014f0 0x7fbf84101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:40.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.629+0000 7fbf895df700 1 -- 172.21.15.133:0/4294941127 >> 172.21.15.133:0/4294941127 conn(0x7fbf840fb900 msgr2=0x7fbf840fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:18:40.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.629+0000 7fbf895df700 1 -- 172.21.15.133:0/4294941127 shutdown_connections 2023-11-27T15:18:40.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.629+0000 7fbf895df700 1 -- 172.21.15.133:0/4294941127 wait complete. 2023-11-27T15:18:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.630+0000 7fbf895df700 1 Processor -- start 2023-11-27T15:18:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.630+0000 7fbf895df700 1 -- start start 2023-11-27T15:18:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.630+0000 7fbf895df700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf841002f0 0x7fbf84198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.630+0000 7fbf895df700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf841014f0 0x7fbf84198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.630+0000 7fbf895df700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf84101e80 0x7fbf8419f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.631+0000 7fbf895df700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8419fed0 con 0x7fbf841002f0 2023-11-27T15:18:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.631+0000 7fbf895df700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf841a0010 con 0x7fbf841014f0 2023-11-27T15:18:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.631+0000 7fbf895df700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf841a01a0 con 0x7fbf84101e80 2023-11-27T15:18:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.631+0000 7fbf82ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf841002f0 0x7fbf84198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:40.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.631+0000 7fbf82ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf841002f0 0x7fbf84198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39216/0 (socket says 172.21.15.133:39216) 2023-11-27T15:18:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.631+0000 7fbf82ffd700 1 -- 172.21.15.133:0/3836043314 learned_addr learned my addr 172.21.15.133:0/3836043314 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:18:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.631+0000 7fbf827fc700 1 --2- 172.21.15.133:0/3836043314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf841014f0 0x7fbf84198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.631+0000 7fbf837fe700 1 --2- 172.21.15.133:0/3836043314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf84101e80 0x7fbf8419f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.631+0000 7fbf82ffd700 1 -- 172.21.15.133:0/3836043314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf84101e80 msgr2=0x7fbf8419f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.631+0000 7fbf82ffd700 1 --2- 172.21.15.133:0/3836043314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf84101e80 0x7fbf8419f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.631+0000 7fbf82ffd700 1 -- 172.21.15.133:0/3836043314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf841014f0 msgr2=0x7fbf84198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:18:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.631+0000 7fbf82ffd700 1 --2- 172.21.15.133:0/3836043314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf841014f0 0x7fbf84198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:18:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.631+0000 7fbf82ffd700 1 -- 172.21.15.133:0/3836043314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c009d00 con 0x7fbf841002f0 2023-11-27T15:18:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.631+0000 7fbf837fe700 1 --2- 172.21.15.133:0/3836043314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf84101e80 0x7fbf8419f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:18:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.631+0000 7fbf82ffd700 1 --2- 172.21.15.133:0/3836043314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf841002f0 0x7fbf84198340 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400e9a0 tx=0x7fbf7400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.632+0000 7fbf6bfff700 1 -- 172.21.15.133:0/3836043314 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf74009790 con 0x7fbf841002f0 2023-11-27T15:18:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.632+0000 7fbf895df700 1 -- 172.21.15.133:0/3836043314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf8410bc90 con 0x7fbf841002f0 2023-11-27T15:18:40.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.632+0000 7fbf6bfff700 1 -- 172.21.15.133:0/3836043314 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf740044e0 con 0x7fbf841002f0 2023-11-27T15:18:40.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.632+0000 7fbf6bfff700 1 -- 172.21.15.133:0/3836043314 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf74003cc0 con 0x7fbf841002f0 2023-11-27T15:18:40.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.632+0000 7fbf895df700 1 -- 172.21.15.133:0/3836043314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8410bb00 con 0x7fbf841002f0 2023-11-27T15:18:40.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.633+0000 7fbf895df700 1 -- 172.21.15.133:0/3836043314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf841a0930 con 0x7fbf841002f0 2023-11-27T15:18:40.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.635+0000 7fbf6bfff700 1 -- 172.21.15.133:0/3836043314 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fbf74003e80 con 0x7fbf841002f0 2023-11-27T15:18:40.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.637+0000 7fbf6bfff700 1 --2- 172.21.15.133:0/3836043314 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbf6c0a1150 0x7fbf6c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:18:40.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.637+0000 7fbf6bfff700 1 -- 172.21.15.133:0/3836043314 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5829+0+0 (secure 0 0 0) 0x7fbf74014070 con 0x7fbf841002f0 2023-11-27T15:18:40.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.637+0000 7fbf827fc700 1 --2- 172.21.15.133:0/3836043314 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbf6c0a1150 0x7fbf6c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:18:40.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.638+0000 7fbf827fc700 1 --2- 172.21.15.133:0/3836043314 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbf6c0a1150 0x7fbf6c0a3600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009c60 tx=0x7fbf78009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:18:40.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.642+0000 7fbf6bfff700 1 -- 172.21.15.133:0/3836043314 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbf740962b0 con 0x7fbf841002f0 2023-11-27T15:18:40.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:18:40.847+0000 7fbf895df700 1 -- 172.21.15.133:0/3836043314 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf841a0ee0 con 0x7fbf6c0a1150 2023-11-27T15:18:41.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:41 smithi174 ceph-mon[131110]: pgmap v189: 1 pgs: 1 remapped+peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:18:41.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:41 smithi174 ceph-mon[131110]: osdmap e61: 9 total, 9 up, 9 in 2023-11-27T15:18:41.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:41 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:18:41.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:41 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:18:41.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:41 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:41.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:41 smithi133 ceph-mon[97860]: pgmap v189: 1 pgs: 1 remapped+peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:18:41.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:41 smithi133 ceph-mon[97860]: osdmap e61: 9 total, 9 up, 9 in 2023-11-27T15:18:41.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:41 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:18:41.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:41 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:18:41.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:41 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:41.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:41 smithi177 ceph-mon[131504]: pgmap v189: 1 pgs: 1 remapped+peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:18:41.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:41 smithi177 ceph-mon[131504]: osdmap e61: 9 total, 9 up, 9 in 2023-11-27T15:18:41.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:41 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:18:41.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:41 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:18:41.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:41 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:42.328 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:42 smithi133 ceph-mon[97860]: from='client.14325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:18:42.328 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:42 smithi133 ceph-mon[97860]: osdmap e62: 9 total, 9 up, 9 in 2023-11-27T15:18:42.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:42 smithi174 ceph-mon[131110]: from='client.14325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:18:42.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:42 smithi174 ceph-mon[131110]: osdmap e62: 9 total, 9 up, 9 in 2023-11-27T15:18:42.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:42 smithi177 ceph-mon[131504]: from='client.14325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi133:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:18:42.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:42 smithi177 ceph-mon[131504]: osdmap e62: 9 total, 9 up, 9 in 2023-11-27T15:18:43.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:43 smithi174 ceph-mon[131110]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:18:43.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:43 smithi133 ceph-mon[97860]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:18:43.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:43 smithi177 ceph-mon[131504]: pgmap v192: 1 pgs: 1 remapped+peering; 577 KiB data, 54 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:18:44.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:44 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3052483031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e115ec1-93e3-41ea-a785-18b6c70b7b9c"}]: dispatch 2023-11-27T15:18:44.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:44 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3052483031' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e115ec1-93e3-41ea-a785-18b6c70b7b9c"}]': finished 2023-11-27T15:18:44.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:44 smithi174 ceph-mon[131110]: osdmap e63: 10 total, 9 up, 10 in 2023-11-27T15:18:44.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:44 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:18:44.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:44 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3052483031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e115ec1-93e3-41ea-a785-18b6c70b7b9c"}]: dispatch 2023-11-27T15:18:44.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:44 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3052483031' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e115ec1-93e3-41ea-a785-18b6c70b7b9c"}]': finished 2023-11-27T15:18:44.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:44 smithi133 ceph-mon[97860]: osdmap e63: 10 total, 9 up, 10 in 2023-11-27T15:18:44.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:44 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:18:44.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:44 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3052483031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5e115ec1-93e3-41ea-a785-18b6c70b7b9c"}]: dispatch 2023-11-27T15:18:44.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:44 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3052483031' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5e115ec1-93e3-41ea-a785-18b6c70b7b9c"}]': finished 2023-11-27T15:18:44.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:44 smithi177 ceph-mon[131504]: osdmap e63: 10 total, 9 up, 10 in 2023-11-27T15:18:44.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:44 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:18:45.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:45 smithi174 ceph-mon[131110]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-11-27T15:18:45.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:45 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3680564371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:18:45.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:45 smithi133 ceph-mon[97860]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-11-27T15:18:45.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:45 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3680564371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:18:45.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:45 smithi177 ceph-mon[131504]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-11-27T15:18:45.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:45 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3680564371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:18:47.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:47 smithi174 ceph-mon[131110]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-27T15:18:47.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:47 smithi133 ceph-mon[97860]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-27T15:18:47.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:47 smithi177 ceph-mon[131504]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-11-27T15:18:49.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:49 smithi174 ceph-mon[131110]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-11-27T15:18:49.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:49 smithi133 ceph-mon[97860]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-11-27T15:18:49.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:49 smithi177 ceph-mon[131504]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 76 KiB/s, 0 objects/s recovering 2023-11-27T15:18:51.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:51 smithi174 ceph-mon[131110]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2023-11-27T15:18:51.508 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:51 smithi133 ceph-mon[97860]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2023-11-27T15:18:51.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:51 smithi177 ceph-mon[131504]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2023-11-27T15:18:52.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:52 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-11-27T15:18:52.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:52 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:52.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:52 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-11-27T15:18:52.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:52 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:52.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:52 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-11-27T15:18:52.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:52 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:18:53.188 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:53 smithi133 ceph-mon[97860]: Deploying daemon osd.9 on smithi133 2023-11-27T15:18:53.188 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:53 smithi133 ceph-mon[97860]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-11-27T15:18:53.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:53 smithi174 ceph-mon[131110]: Deploying daemon osd.9 on smithi133 2023-11-27T15:18:53.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:53 smithi174 ceph-mon[131110]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-11-27T15:18:53.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:53 smithi177 ceph-mon[131504]: Deploying daemon osd.9 on smithi133 2023-11-27T15:18:53.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:53 smithi177 ceph-mon[131504]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-11-27T15:18:55.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:55 smithi174 ceph-mon[131110]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:18:55.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:55 smithi133 ceph-mon[97860]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:18:55.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:55 smithi177 ceph-mon[131504]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:18:57.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:57 smithi174 ceph-mon[131110]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:18:57.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:57 smithi133 ceph-mon[97860]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:18:57.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:57 smithi177 ceph-mon[131504]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:18:59.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:59 smithi133 ceph-mon[97860]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:18:59.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:59 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:59.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:59 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:59.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:18:59 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:59.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:59 smithi177 ceph-mon[131504]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:18:59.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:59 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:59.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:59 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:59.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:18:59 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:18:59.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:59 smithi174 ceph-mon[131110]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:18:59.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:59 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:59.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:59 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:18:59.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:18:59 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:01.266 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:01 smithi133 ceph-mon[97860]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:19:01.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:01 smithi177 ceph-mon[131504]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:19:01.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:01 smithi174 ceph-mon[131110]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:19:01.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:01.664+0000 7fbf6bfff700 1 -- 172.21.15.133:0/3836043314 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbf841a0ee0 con 0x7fbf6c0a1150 2023-11-27T15:19:01.668 INFO:teuthology.orchestra.run.smithi133.stdout:Created osd(s) 9 on host 'smithi133' 2023-11-27T15:19:01.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:01.667+0000 7fbf895df700 1 -- 172.21.15.133:0/3836043314 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbf6c0a1150 msgr2=0x7fbf6c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:01.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:01.667+0000 7fbf895df700 1 --2- 172.21.15.133:0/3836043314 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbf6c0a1150 0x7fbf6c0a3600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009c60 tx=0x7fbf78009260 comp rx=0 tx=0).stop 2023-11-27T15:19:01.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:01.667+0000 7fbf895df700 1 -- 172.21.15.133:0/3836043314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf841002f0 msgr2=0x7fbf84198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:01.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:01.667+0000 7fbf895df700 1 --2- 172.21.15.133:0/3836043314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf841002f0 0x7fbf84198340 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400e9a0 tx=0x7fbf7400ecb0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:01.667+0000 7fbf895df700 1 -- 172.21.15.133:0/3836043314 shutdown_connections 2023-11-27T15:19:01.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:01.667+0000 7fbf895df700 1 --2- 172.21.15.133:0/3836043314 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbf6c0a1150 0x7fbf6c0a3600 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:01.667+0000 7fbf895df700 1 --2- 172.21.15.133:0/3836043314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbf84101e80 0x7fbf8419f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:01.667+0000 7fbf895df700 1 --2- 172.21.15.133:0/3836043314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf841002f0 0x7fbf84198340 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:01.667+0000 7fbf895df700 1 --2- 172.21.15.133:0/3836043314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf841014f0 0x7fbf84198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:01.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:01.667+0000 7fbf895df700 1 -- 172.21.15.133:0/3836043314 >> 172.21.15.133:0/3836043314 conn(0x7fbf840fb900 msgr2=0x7fbf840fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:01.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:01.667+0000 7fbf895df700 1 -- 172.21.15.133:0/3836043314 shutdown_connections 2023-11-27T15:19:01.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:01.667+0000 7fbf895df700 1 -- 172.21.15.133:0/3836043314 wait complete. 2023-11-27T15:19:02.186 DEBUG:teuthology.orchestra.run.smithi133:osd.9> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.9.service 2023-11-27T15:19:02.191 INFO:tasks.cephadm:Deploying osd.10 on smithi174 with /dev/vg_nvme/lv_1... 2023-11-27T15:19:02.191 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-11-27T15:19:02.260 INFO:journalctl@ceph.osd.9.smithi133.stdout:-- Logs begin at Mon 2023-11-27 14:54:33 UTC. -- 2023-11-27T15:19:02.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:02.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:02.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:02.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:02.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:02 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:19:02.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:02.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:02 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:19:02.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:02 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:19:02.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:02.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:02.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:02.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:02 smithi133 ceph-mon[97860]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:19:02.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:02 smithi133 ceph-mon[97860]: from='osd.9 [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-27T15:19:02.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:02.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:02.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:02.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:02.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:02 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:19:02.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:02.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:02 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:19:02.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:02 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:19:02.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:02.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:02.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:02.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:02 smithi177 ceph-mon[131504]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:19:02.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:02 smithi177 ceph-mon[131504]: from='osd.9 [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-27T15:19:02.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:02.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:02.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:02.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:02.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:02 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:19:02.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:02.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:02 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:19:02.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:02 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:19:02.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:02.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:02.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:02.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:02 smithi174 ceph-mon[131110]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:19:02.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:02 smithi174 ceph-mon[131110]: from='osd.9 [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-11-27T15:19:02.844 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:19:03.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:03 smithi133 ceph-mon[97860]: from='osd.9 [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-11-27T15:19:03.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:03 smithi133 ceph-mon[97860]: osdmap e64: 10 total, 9 up, 10 in 2023-11-27T15:19:03.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:03 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:19:03.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:03 smithi133 ceph-mon[97860]: from='osd.9 [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-11-27T15:19:03.778 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:19:03 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:19:03.641+0000 7f88392eb700 -1 osd.9 0 waiting for initial osdmap 2023-11-27T15:19:03.778 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:19:03 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:19:03.658+0000 7f88330dc700 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:19:03.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:03 smithi174 ceph-mon[131110]: from='osd.9 [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-11-27T15:19:03.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:03 smithi174 ceph-mon[131110]: osdmap e64: 10 total, 9 up, 10 in 2023-11-27T15:19:03.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:03 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:19:03.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:03 smithi174 ceph-mon[131110]: from='osd.9 [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-11-27T15:19:04.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:03 smithi177 ceph-mon[131504]: from='osd.9 [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-11-27T15:19:04.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:03 smithi177 ceph-mon[131504]: osdmap e64: 10 total, 9 up, 10 in 2023-11-27T15:19:04.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:03 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:19:04.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:03 smithi177 ceph-mon[131504]: from='osd.9 [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]: dispatch 2023-11-27T15:19:04.332 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-11-27T15:19:04.333 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-11-27T15:19:04.333 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2023-11-27T15:19:04.333 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2023-11-27T15:19:04.333 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112184 s, 935 MB/s 2023-11-27T15:19:04.333 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2023-11-27T15:19:04.690 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi174:vg_nvme/lv_1 2023-11-27T15:19:04.902 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:04 smithi174 ceph-mon[131110]: from='osd.9 [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-11-27T15:19:04.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:04 smithi174 ceph-mon[131110]: osdmap e65: 10 total, 9 up, 10 in 2023-11-27T15:19:04.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:04 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:19:04.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:04 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:19:04.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:04 smithi174 ceph-mon[131110]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:19:04.921 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:04 smithi133 ceph-mon[97860]: from='osd.9 [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-11-27T15:19:04.921 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:04 smithi133 ceph-mon[97860]: osdmap e65: 10 total, 9 up, 10 in 2023-11-27T15:19:04.921 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:04 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:19:04.921 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:04 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:19:04.921 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:04 smithi133 ceph-mon[97860]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:19:05.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:04 smithi177 ceph-mon[131504]: from='osd.9 [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi133", "root=default"]}]': finished 2023-11-27T15:19:05.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:04 smithi177 ceph-mon[131504]: osdmap e65: 10 total, 9 up, 10 in 2023-11-27T15:19:05.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:04 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:19:05.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:04 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:19:05.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:04 smithi177 ceph-mon[131504]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 55 MiB used, 805 GiB / 805 GiB avail 2023-11-27T15:19:05.333 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:19:05.867 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:05 smithi174 ceph-mon[131110]: purged_snaps scrub starts 2023-11-27T15:19:05.867 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:05 smithi174 ceph-mon[131110]: purged_snaps scrub ok 2023-11-27T15:19:05.867 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:05 smithi174 ceph-mon[131110]: osd.9 [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790] boot 2023-11-27T15:19:05.867 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:05 smithi174 ceph-mon[131110]: osdmap e66: 10 total, 10 up, 10 in 2023-11-27T15:19:05.868 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:19:05.868 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:05.868 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:05.868 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:05.868 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:05.868 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:05.868 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:05.868 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:05.868 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:05.868 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:05.868 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:06.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:05 smithi133 ceph-mon[97860]: purged_snaps scrub starts 2023-11-27T15:19:06.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:05 smithi133 ceph-mon[97860]: purged_snaps scrub ok 2023-11-27T15:19:06.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:05 smithi133 ceph-mon[97860]: osd.9 [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790] boot 2023-11-27T15:19:06.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:05 smithi133 ceph-mon[97860]: osdmap e66: 10 total, 10 up, 10 in 2023-11-27T15:19:06.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:19:06.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:06.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:06.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:06.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:06.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:06.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:06.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:06.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:06.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:06.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:06.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:05 smithi177 ceph-mon[131504]: purged_snaps scrub starts 2023-11-27T15:19:06.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:05 smithi177 ceph-mon[131504]: purged_snaps scrub ok 2023-11-27T15:19:06.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:05 smithi177 ceph-mon[131504]: osd.9 [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790] boot 2023-11-27T15:19:06.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:05 smithi177 ceph-mon[131504]: osdmap e66: 10 total, 10 up, 10 in 2023-11-27T15:19:06.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-11-27T15:19:06.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:06.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:06.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:06.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:06.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:06.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:06.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:06.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:06.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:06.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:06.515 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.509+0000 7f3e2dac2700 1 -- 172.21.15.174:0/3463469516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e28100d70 msgr2=0x7f3e28107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:06.515 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.509+0000 7f3e2dac2700 1 --2- 172.21.15.174:0/3463469516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e28100d70 0x7f3e28107630 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3e2000b0a0 tx=0x7f3e2000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:19:06.515 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.509+0000 7f3e2dac2700 1 -- 172.21.15.174:0/3463469516 shutdown_connections 2023-11-27T15:19:06.515 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.509+0000 7f3e2dac2700 1 --2- 172.21.15.174:0/3463469516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e28100d70 0x7f3e28107630 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:06.515 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.509+0000 7f3e2dac2700 1 --2- 172.21.15.174:0/3463469516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e280ff1e0 0x7f3e280ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:06.515 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.509+0000 7f3e2dac2700 1 --2- 172.21.15.174:0/3463469516 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e281003e0 0x7f3e28100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:06.515 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.509+0000 7f3e2dac2700 1 -- 172.21.15.174:0/3463469516 >> 172.21.15.174:0/3463469516 conn(0x7f3e280fa7f0 msgr2=0x7f3e280fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:06.515 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.509+0000 7f3e2dac2700 1 -- 172.21.15.174:0/3463469516 shutdown_connections 2023-11-27T15:19:06.515 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.509+0000 7f3e2dac2700 1 -- 172.21.15.174:0/3463469516 wait complete. 2023-11-27T15:19:06.515 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.510+0000 7f3e2dac2700 1 Processor -- start 2023-11-27T15:19:06.516 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.510+0000 7f3e2dac2700 1 -- start start 2023-11-27T15:19:06.516 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.510+0000 7f3e2dac2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e280ff1e0 0x7f3e28069130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:06.516 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.510+0000 7f3e2dac2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e281003e0 0x7f3e2819b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:06.516 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.510+0000 7f3e2dac2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e28100d70 0x7f3e2819d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:06.516 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.510+0000 7f3e2dac2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2819de30 con 0x7f3e281003e0 2023-11-27T15:19:06.516 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.510+0000 7f3e2dac2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2819dfa0 con 0x7f3e28100d70 2023-11-27T15:19:06.516 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.510+0000 7f3e2dac2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2819e160 con 0x7f3e280ff1e0 2023-11-27T15:19:06.516 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.510+0000 7f3e27fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e28100d70 0x7f3e2819d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:06.516 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.510+0000 7f3e27fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e28100d70 0x7f3e2819d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:48636/0 (socket says 172.21.15.174:48636) 2023-11-27T15:19:06.516 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.510+0000 7f3e27fff700 1 -- 172.21.15.174:0/3183841921 learned_addr learned my addr 172.21.15.174:0/3183841921 (peer_addr_for_me v2:172.21.15.174:0/0) 2023-11-27T15:19:06.517 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.510+0000 7f3e277fe700 1 --2- 172.21.15.174:0/3183841921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e280ff1e0 0x7f3e28069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:06.517 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.510+0000 7f3e27fff700 1 -- 172.21.15.174:0/3183841921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e280ff1e0 msgr2=0x7f3e28069130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:06.517 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.510+0000 7f3e26ffd700 1 --2- 172.21.15.174:0/3183841921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e281003e0 0x7f3e2819b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:06.517 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.510+0000 7f3e27fff700 1 --2- 172.21.15.174:0/3183841921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e280ff1e0 0x7f3e28069130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:06.517 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.510+0000 7f3e27fff700 1 -- 172.21.15.174:0/3183841921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e281003e0 msgr2=0x7f3e2819b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:06.517 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.511+0000 7f3e27fff700 1 --2- 172.21.15.174:0/3183841921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e281003e0 0x7f3e2819b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:06.517 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.511+0000 7f3e27fff700 1 -- 172.21.15.174:0/3183841921 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e20009d00 con 0x7f3e28100d70 2023-11-27T15:19:06.517 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.511+0000 7f3e277fe700 1 --2- 172.21.15.174:0/3183841921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e280ff1e0 0x7f3e28069130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:19:06.517 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.511+0000 7f3e26ffd700 1 --2- 172.21.15.174:0/3183841921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e281003e0 0x7f3e2819b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:19:06.517 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.511+0000 7f3e27fff700 1 --2- 172.21.15.174:0/3183841921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e28100d70 0x7f3e2819d690 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e200062a0 tx=0x7f3e20004db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:06.517 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.511+0000 7f3e24ff9700 1 -- 172.21.15.174:0/3183841921 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2000e040 con 0x7f3e28100d70 2023-11-27T15:19:06.517 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.511+0000 7f3e2dac2700 1 -- 172.21.15.174:0/3183841921 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e2810ac50 con 0x7f3e28100d70 2023-11-27T15:19:06.518 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.511+0000 7f3e24ff9700 1 -- 172.21.15.174:0/3183841921 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e2001b660 con 0x7f3e28100d70 2023-11-27T15:19:06.518 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.511+0000 7f3e24ff9700 1 -- 172.21.15.174:0/3183841921 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e20004280 con 0x7f3e28100d70 2023-11-27T15:19:06.518 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.511+0000 7f3e2dac2700 1 -- 172.21.15.174:0/3183841921 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e2810aac0 con 0x7f3e28100d70 2023-11-27T15:19:06.519 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.512+0000 7f3e2dac2700 1 -- 172.21.15.174:0/3183841921 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e28066ea0 con 0x7f3e28100d70 2023-11-27T15:19:06.519 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.513+0000 7f3e24ff9700 1 -- 172.21.15.174:0/3183841921 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3e2001ba40 con 0x7f3e28100d70 2023-11-27T15:19:06.519 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.514+0000 7f3e24ff9700 1 --2- 172.21.15.174:0/3183841921 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3e100a1150 0x7f3e100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:06.519 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.514+0000 7f3e24ff9700 1 -- 172.21.15.174:0/3183841921 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f3e200cdf00 con 0x7f3e28100d70 2023-11-27T15:19:06.519 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.514+0000 7f3e277fe700 1 --2- 172.21.15.174:0/3183841921 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3e100a1150 0x7f3e100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:06.519 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.515+0000 7f3e277fe700 1 --2- 172.21.15.174:0/3183841921 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3e100a1150 0x7f3e100a3600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3e28100240 tx=0x7f3e1c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:06.520 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.517+0000 7f3e24ff9700 1 -- 172.21.15.174:0/3183841921 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3e20097ee0 con 0x7f3e28100d70 2023-11-27T15:19:06.719 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:06.715+0000 7f3e2dac2700 1 -- 172.21.15.174:0/3183841921 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e2819ebf0 con 0x7f3e100a1150 2023-11-27T15:19:07.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:06 smithi133 ceph-mon[97860]: Detected new or changed devices on smithi133 2023-11-27T15:19:07.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:06 smithi133 ceph-mon[97860]: Adjusting osd_memory_target on smithi133 to 4317M 2023-11-27T15:19:07.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:06 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:19:07.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:06 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:19:07.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:06 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:19:07.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:06 smithi133 ceph-mon[97860]: osdmap e67: 10 total, 10 up, 10 in 2023-11-27T15:19:07.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:06 smithi133 ceph-mon[97860]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:07.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:06 smithi177 ceph-mon[131504]: Detected new or changed devices on smithi133 2023-11-27T15:19:07.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:06 smithi177 ceph-mon[131504]: Adjusting osd_memory_target on smithi133 to 4317M 2023-11-27T15:19:07.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:06 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:19:07.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:06 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:19:07.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:06 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:19:07.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:06 smithi177 ceph-mon[131504]: osdmap e67: 10 total, 10 up, 10 in 2023-11-27T15:19:07.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:06 smithi177 ceph-mon[131504]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:07.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:06 smithi174 ceph-mon[131110]: Detected new or changed devices on smithi133 2023-11-27T15:19:07.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:06 smithi174 ceph-mon[131110]: Adjusting osd_memory_target on smithi133 to 4317M 2023-11-27T15:19:07.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:06 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:19:07.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:06 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:19:07.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:06 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:19:07.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:06 smithi174 ceph-mon[131110]: osdmap e67: 10 total, 10 up, 10 in 2023-11-27T15:19:07.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:06 smithi174 ceph-mon[131110]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 60 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:07.884 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:07 smithi174 ceph-mon[131110]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:19:07.884 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:07 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:19:07.884 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:07 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:19:07.884 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:07 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:08.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:07 smithi133 ceph-mon[97860]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:19:08.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:07 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:19:08.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:07 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:19:08.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:07 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:08.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:07 smithi177 ceph-mon[131504]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:19:08.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:07 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:19:08.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:07 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:19:08.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:07 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:09.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:08 smithi133 ceph-mon[97860]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:09.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:08 smithi177 ceph-mon[131504]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:09.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:08 smithi174 ceph-mon[131110]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:10.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:09 smithi133 ceph-mon[97860]: from='client.? 172.21.15.174:0/3495578785' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "342cb818-eaf2-4961-81aa-fd849bfd7be5"}]: dispatch 2023-11-27T15:19:10.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:09 smithi133 ceph-mon[97860]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "342cb818-eaf2-4961-81aa-fd849bfd7be5"}]: dispatch 2023-11-27T15:19:10.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:09 smithi133 ceph-mon[97860]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "342cb818-eaf2-4961-81aa-fd849bfd7be5"}]': finished 2023-11-27T15:19:10.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:09 smithi133 ceph-mon[97860]: osdmap e68: 11 total, 10 up, 11 in 2023-11-27T15:19:10.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:09 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:19:10.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:09 smithi177 ceph-mon[131504]: from='client.? 172.21.15.174:0/3495578785' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "342cb818-eaf2-4961-81aa-fd849bfd7be5"}]: dispatch 2023-11-27T15:19:10.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:09 smithi177 ceph-mon[131504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "342cb818-eaf2-4961-81aa-fd849bfd7be5"}]: dispatch 2023-11-27T15:19:10.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:09 smithi177 ceph-mon[131504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "342cb818-eaf2-4961-81aa-fd849bfd7be5"}]': finished 2023-11-27T15:19:10.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:09 smithi177 ceph-mon[131504]: osdmap e68: 11 total, 10 up, 11 in 2023-11-27T15:19:10.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:09 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:19:10.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:09 smithi174 ceph-mon[131110]: from='client.? 172.21.15.174:0/3495578785' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "342cb818-eaf2-4961-81aa-fd849bfd7be5"}]: dispatch 2023-11-27T15:19:10.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:09 smithi174 ceph-mon[131110]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "342cb818-eaf2-4961-81aa-fd849bfd7be5"}]: dispatch 2023-11-27T15:19:10.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:09 smithi174 ceph-mon[131110]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "342cb818-eaf2-4961-81aa-fd849bfd7be5"}]': finished 2023-11-27T15:19:10.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:09 smithi174 ceph-mon[131110]: osdmap e68: 11 total, 10 up, 11 in 2023-11-27T15:19:10.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:09 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:19:11.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:10 smithi133 ceph-mon[97860]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:11.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:10 smithi133 ceph-mon[97860]: from='client.? 172.21.15.174:0/1868296621' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:19:11.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:10 smithi177 ceph-mon[131504]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:11.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:10 smithi177 ceph-mon[131504]: from='client.? 172.21.15.174:0/1868296621' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:19:11.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:10 smithi174 ceph-mon[131110]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:11.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:10 smithi174 ceph-mon[131110]: from='client.? 172.21.15.174:0/1868296621' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:19:13.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:13 smithi174 ceph-mon[131110]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:13.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:13 smithi133 ceph-mon[97860]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:13.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:13 smithi177 ceph-mon[131504]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:15.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:15 smithi174 ceph-mon[131110]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:15.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:15 smithi133 ceph-mon[97860]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:15.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:15 smithi177 ceph-mon[131504]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:16.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-11-27T15:19:16.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:16.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-11-27T15:19:16.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:16.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-11-27T15:19:16.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:17.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:17 smithi174 ceph-mon[131110]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:17.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:17 smithi174 ceph-mon[131110]: Deploying daemon osd.10 on smithi174 2023-11-27T15:19:17.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:17 smithi133 ceph-mon[97860]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:17.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:17 smithi133 ceph-mon[97860]: Deploying daemon osd.10 on smithi174 2023-11-27T15:19:17.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:17 smithi177 ceph-mon[131504]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:17.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:17 smithi177 ceph-mon[131504]: Deploying daemon osd.10 on smithi174 2023-11-27T15:19:19.379 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:19 smithi174 ceph-mon[131110]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:19.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:19 smithi133 ceph-mon[97860]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:19.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:19 smithi177 ceph-mon[131504]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:21.363 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:21 smithi174 ceph-mon[131110]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:21.363 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:21 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:21.363 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:21 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:21.363 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:21 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:21.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:21 smithi133 ceph-mon[97860]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:21.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:21 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:21.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:21 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:21.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:21 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:21.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:21 smithi177 ceph-mon[131504]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:21.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:21 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:21.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:21 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:21.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:21 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:23.299 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:23 smithi174 ceph-mon[131110]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:23.332 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:23.328+0000 7f3e24ff9700 1 -- 172.21.15.174:0/3183841921 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f3e2819ebf0 con 0x7f3e100a1150 2023-11-27T15:19:23.332 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 10 on host 'smithi174' 2023-11-27T15:19:23.334 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:23.330+0000 7f3e2dac2700 1 -- 172.21.15.174:0/3183841921 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3e100a1150 msgr2=0x7f3e100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:23.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:23.330+0000 7f3e2dac2700 1 --2- 172.21.15.174:0/3183841921 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3e100a1150 0x7f3e100a3600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3e28100240 tx=0x7f3e1c008040 comp rx=0 tx=0).stop 2023-11-27T15:19:23.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:23.330+0000 7f3e2dac2700 1 -- 172.21.15.174:0/3183841921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e28100d70 msgr2=0x7f3e2819d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:23.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:23.330+0000 7f3e2dac2700 1 --2- 172.21.15.174:0/3183841921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e28100d70 0x7f3e2819d690 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e200062a0 tx=0x7f3e20004db0 comp rx=0 tx=0).stop 2023-11-27T15:19:23.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:23.330+0000 7f3e2dac2700 1 -- 172.21.15.174:0/3183841921 shutdown_connections 2023-11-27T15:19:23.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:23.330+0000 7f3e2dac2700 1 --2- 172.21.15.174:0/3183841921 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3e100a1150 0x7f3e100a3600 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:23.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:23.330+0000 7f3e2dac2700 1 --2- 172.21.15.174:0/3183841921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e28100d70 0x7f3e2819d690 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:23.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:23.330+0000 7f3e2dac2700 1 --2- 172.21.15.174:0/3183841921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3e280ff1e0 0x7f3e28069130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:23.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:23.330+0000 7f3e2dac2700 1 --2- 172.21.15.174:0/3183841921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3e281003e0 0x7f3e2819b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:23.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:23.330+0000 7f3e2dac2700 1 -- 172.21.15.174:0/3183841921 >> 172.21.15.174:0/3183841921 conn(0x7f3e280fa7f0 msgr2=0x7f3e280fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:23.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:23.330+0000 7f3e2dac2700 1 -- 172.21.15.174:0/3183841921 shutdown_connections 2023-11-27T15:19:23.335 INFO:teuthology.orchestra.run.smithi174.stderr:2023-11-27T15:19:23.330+0000 7f3e2dac2700 1 -- 172.21.15.174:0/3183841921 wait complete. 2023-11-27T15:19:23.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:23 smithi133 ceph-mon[97860]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:23.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:23 smithi177 ceph-mon[131504]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:23.723 DEBUG:teuthology.orchestra.run.smithi174:osd.10> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.10.service 2023-11-27T15:19:23.725 INFO:tasks.cephadm:Deploying osd.11 on smithi177 with /dev/vg_nvme/lv_1... 2023-11-27T15:19:23.725 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-11-27T15:19:23.756 INFO:journalctl@ceph.osd.10.smithi174.stdout:-- Logs begin at Mon 2023-11-27 14:50:40 UTC. -- 2023-11-27T15:19:24.373 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:19:24.506 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:19:24 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:19:24.226+0000 7f2648d84640 -1 osd.10 0 log_to_monitors true 2023-11-27T15:19:24.506 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:24 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:24.507 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:24 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:24.507 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:24 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:24.507 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:24 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:24.507 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:24 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:19:24.507 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:24 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:24.507 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:24 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:19:24.507 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:24 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:19:24.507 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:24 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:24.507 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:24 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:24.508 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:24 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:24.508 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:24 smithi174 ceph-mon[131110]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:24.508 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:24 smithi174 ceph-mon[131110]: from='osd.10 [v2:172.21.15.174:6824/880237758,v1:172.21.15.174:6825/880237758]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-27T15:19:24.508 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:24 smithi174 ceph-mon[131110]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-27T15:19:24.575 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:24 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:24.575 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:24 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:24.575 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:24 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:24.575 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:24 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:24.575 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:24 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:19:24.575 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:24 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:24.575 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:24 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:19:24.575 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:24 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:19:24.575 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:24 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:24.576 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:24 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:24.576 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:24 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:24.576 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:24 smithi177 ceph-mon[131504]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:24.576 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:24 smithi177 ceph-mon[131504]: from='osd.10 [v2:172.21.15.174:6824/880237758,v1:172.21.15.174:6825/880237758]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-27T15:19:24.576 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:24 smithi177 ceph-mon[131504]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-27T15:19:24.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:24 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:24.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:24 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:24.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:24 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:24.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:24 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:24.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:24 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:19:24.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:24 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:24.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:24 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:19:24.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:24 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:19:24.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:24 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:24.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:24 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:24.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:24 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:24.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:24 smithi133 ceph-mon[97860]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:24.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:24 smithi133 ceph-mon[97860]: from='osd.10 [v2:172.21.15.174:6824/880237758,v1:172.21.15.174:6825/880237758]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-27T15:19:24.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:24 smithi133 ceph-mon[97860]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-11-27T15:19:25.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:25 smithi174 ceph-mon[131110]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-11-27T15:19:25.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:25 smithi174 ceph-mon[131110]: osdmap e69: 11 total, 10 up, 11 in 2023-11-27T15:19:25.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:25 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:19:25.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:25 smithi174 ceph-mon[131110]: from='osd.10 [v2:172.21.15.174:6824/880237758,v1:172.21.15.174:6825/880237758]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:19:25.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:25 smithi174 ceph-mon[131110]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:19:25.654 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:19:25 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:19:25.346+0000 7f263e3fb700 -1 osd.10 0 waiting for initial osdmap 2023-11-27T15:19:25.654 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:19:25 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:19:25.362+0000 7f26381ec700 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:19:25.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:25 smithi133 ceph-mon[97860]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-11-27T15:19:25.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:25 smithi133 ceph-mon[97860]: osdmap e69: 11 total, 10 up, 11 in 2023-11-27T15:19:25.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:25 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:19:25.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:25 smithi133 ceph-mon[97860]: from='osd.10 [v2:172.21.15.174:6824/880237758,v1:172.21.15.174:6825/880237758]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:19:25.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:25 smithi133 ceph-mon[97860]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:19:25.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:25 smithi177 ceph-mon[131504]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-11-27T15:19:25.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:25 smithi177 ceph-mon[131504]: osdmap e69: 11 total, 10 up, 11 in 2023-11-27T15:19:25.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:25 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:19:25.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:25 smithi177 ceph-mon[131504]: from='osd.10 [v2:172.21.15.174:6824/880237758,v1:172.21.15.174:6825/880237758]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:19:25.839 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:25 smithi177 ceph-mon[131504]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2023-11-27T15:19:25.870 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-11-27T15:19:25.871 INFO:teuthology.orchestra.run.smithi177.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-11-27T15:19:25.871 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10+0 records in 2023-11-27T15:19:25.871 INFO:teuthology.orchestra.run.smithi177.stderr:10+0 records out 2023-11-27T15:19:25.871 INFO:teuthology.orchestra.run.smithi177.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0173535 s, 604 MB/s 2023-11-27T15:19:25.871 INFO:teuthology.orchestra.run.smithi177.stderr:--> Zapping successful for: 2023-11-27T15:19:26.224 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph orch daemon add osd smithi177:vg_nvme/lv_1 2023-11-27T15:19:26.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:26 smithi177 ceph-mon[131504]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2023-11-27T15:19:26.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:26 smithi177 ceph-mon[131504]: osdmap e70: 11 total, 10 up, 11 in 2023-11-27T15:19:26.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:19:26.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:19:26.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:26 smithi177 ceph-mon[131504]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:26.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:26.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:26.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:26.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:26.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:26.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:26.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:26.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:26.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:26.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:26.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:26 smithi177 ceph-mon[131504]: osd.10 [v2:172.21.15.174:6824/880237758,v1:172.21.15.174:6825/880237758] boot 2023-11-27T15:19:26.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:26 smithi177 ceph-mon[131504]: osdmap e71: 11 total, 11 up, 11 in 2023-11-27T15:19:26.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:19:26.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:26 smithi174 ceph-mon[131110]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2023-11-27T15:19:26.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:26 smithi174 ceph-mon[131110]: osdmap e70: 11 total, 10 up, 11 in 2023-11-27T15:19:26.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:19:26.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:19:26.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:26 smithi174 ceph-mon[131110]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:26.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:26.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:26.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:26.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:26.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:26.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:26.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:26.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:26.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:26.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:26.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:26 smithi174 ceph-mon[131110]: osd.10 [v2:172.21.15.174:6824/880237758,v1:172.21.15.174:6825/880237758] boot 2023-11-27T15:19:26.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:26 smithi174 ceph-mon[131110]: osdmap e71: 11 total, 11 up, 11 in 2023-11-27T15:19:26.654 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:19:26.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:26 smithi133 ceph-mon[97860]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2023-11-27T15:19:26.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:26 smithi133 ceph-mon[97860]: osdmap e70: 11 total, 10 up, 11 in 2023-11-27T15:19:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:19:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:19:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:26 smithi133 ceph-mon[97860]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 61 MiB used, 894 GiB / 894 GiB avail 2023-11-27T15:19:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:26.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:26.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:26.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:26.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:26 smithi133 ceph-mon[97860]: osd.10 [v2:172.21.15.174:6824/880237758,v1:172.21.15.174:6825/880237758] boot 2023-11-27T15:19:26.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:26 smithi133 ceph-mon[97860]: osdmap e71: 11 total, 11 up, 11 in 2023-11-27T15:19:26.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-11-27T15:19:26.873 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:19:28.102 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.097+0000 7fb278ae8700 1 -- 172.21.15.177:0/1756283240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb274100d70 msgr2=0x7fb274107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:28.102 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.097+0000 7fb278ae8700 1 --2- 172.21.15.177:0/1756283240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb274100d70 0x7fb274107630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00b0a0 tx=0x7fb26c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.103 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.097+0000 7fb278ae8700 1 -- 172.21.15.177:0/1756283240 shutdown_connections 2023-11-27T15:19:28.103 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.097+0000 7fb278ae8700 1 --2- 172.21.15.177:0/1756283240 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb274100d70 0x7fb274107630 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.103 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.097+0000 7fb278ae8700 1 --2- 172.21.15.177:0/1756283240 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2740ff1e0 0x7fb2740ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.103 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.097+0000 7fb278ae8700 1 --2- 172.21.15.177:0/1756283240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2741003e0 0x7fb274100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.103 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.097+0000 7fb278ae8700 1 -- 172.21.15.177:0/1756283240 >> 172.21.15.177:0/1756283240 conn(0x7fb2740fa7f0 msgr2=0x7fb2740fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:28.103 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.097+0000 7fb278ae8700 1 -- 172.21.15.177:0/1756283240 shutdown_connections 2023-11-27T15:19:28.103 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.097+0000 7fb278ae8700 1 -- 172.21.15.177:0/1756283240 wait complete. 2023-11-27T15:19:28.105 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.098+0000 7fb278ae8700 1 Processor -- start 2023-11-27T15:19:28.105 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.098+0000 7fb278ae8700 1 -- start start 2023-11-27T15:19:28.105 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.098+0000 7fb278ae8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2740ff1e0 0x7fb274069110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.105 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.099+0000 7fb278ae8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2741003e0 0x7fb27419d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.105 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.099+0000 7fb278ae8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb274100d70 0x7fb27419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.106 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.099+0000 7fb278ae8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27419fdf0 con 0x7fb274100d70 2023-11-27T15:19:28.106 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.099+0000 7fb278ae8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27419ff60 con 0x7fb2740ff1e0 2023-11-27T15:19:28.106 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.099+0000 7fb278ae8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2741a0120 con 0x7fb2741003e0 2023-11-27T15:19:28.106 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.099+0000 7fb26bfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2741003e0 0x7fb27419d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.106 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.099+0000 7fb26bfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2741003e0 0x7fb27419d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:58174/0 (socket says 172.21.15.177:58174) 2023-11-27T15:19:28.106 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.099+0000 7fb26bfff700 1 -- 172.21.15.177:0/341709548 learned_addr learned my addr 172.21.15.177:0/341709548 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-27T15:19:28.106 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.099+0000 7fb27259c700 1 --2- 172.21.15.177:0/341709548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2740ff1e0 0x7fb274069110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.106 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.099+0000 7fb272d9d700 1 --2- 172.21.15.177:0/341709548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb274100d70 0x7fb27419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.106 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.099+0000 7fb26bfff700 1 -- 172.21.15.177:0/341709548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2740ff1e0 msgr2=0x7fb274069110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:28.106 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.099+0000 7fb26bfff700 1 --2- 172.21.15.177:0/341709548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2740ff1e0 0x7fb274069110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.107 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.099+0000 7fb26bfff700 1 -- 172.21.15.177:0/341709548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb274100d70 msgr2=0x7fb27419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:28.107 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.099+0000 7fb26bfff700 1 --2- 172.21.15.177:0/341709548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb274100d70 0x7fb27419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:28.107 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.099+0000 7fb26bfff700 1 -- 172.21.15.177:0/341709548 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c009d00 con 0x7fb2741003e0 2023-11-27T15:19:28.107 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.099+0000 7fb27259c700 1 --2- 172.21.15.177:0/341709548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2740ff1e0 0x7fb274069110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:19:28.107 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.099+0000 7fb26bfff700 1 --2- 172.21.15.177:0/341709548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2741003e0 0x7fb27419d1f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00d980 tx=0x7fb25c00dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:28.107 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.100+0000 7fb26b7fe700 1 -- 172.21.15.177:0/341709548 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c00bb60 con 0x7fb2741003e0 2023-11-27T15:19:28.107 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.100+0000 7fb26b7fe700 1 -- 172.21.15.177:0/341709548 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb25c007640 con 0x7fb2741003e0 2023-11-27T15:19:28.107 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.100+0000 7fb26b7fe700 1 -- 172.21.15.177:0/341709548 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c004e90 con 0x7fb2741003e0 2023-11-27T15:19:28.107 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.100+0000 7fb278ae8700 1 -- 172.21.15.177:0/341709548 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb27410ab30 con 0x7fb2741003e0 2023-11-27T15:19:28.108 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.100+0000 7fb278ae8700 1 -- 172.21.15.177:0/341709548 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb27410a9a0 con 0x7fb2741003e0 2023-11-27T15:19:28.108 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.101+0000 7fb278ae8700 1 -- 172.21.15.177:0/341709548 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2741a0800 con 0x7fb2741003e0 2023-11-27T15:19:28.108 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.102+0000 7fb26b7fe700 1 -- 172.21.15.177:0/341709548 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb25c005020 con 0x7fb2741003e0 2023-11-27T15:19:28.109 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.102+0000 7fb26b7fe700 1 --2- 172.21.15.177:0/341709548 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb2600a11d0 0x7fb2600a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:28.109 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.102+0000 7fb26b7fe700 1 -- 172.21.15.177:0/341709548 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6723+0+0 (secure 0 0 0) 0x7fb25c0cc110 con 0x7fb2741003e0 2023-11-27T15:19:28.109 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.103+0000 7fb27259c700 1 --2- 172.21.15.177:0/341709548 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb2600a11d0 0x7fb2600a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:28.109 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.103+0000 7fb27259c700 1 --2- 172.21.15.177:0/341709548 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb2600a11d0 0x7fb2600a3680 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb274100240 tx=0x7fb26400a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:28.111 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.107+0000 7fb26b7fe700 1 -- 172.21.15.177:0/341709548 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb25c099920 con 0x7fb2741003e0 2023-11-27T15:19:28.305 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:28.300+0000 7fb278ae8700 1 -- 172.21.15.177:0/341709548 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2741a0e70 con 0x7fb2600a11d0 2023-11-27T15:19:28.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:28 smithi177 ceph-mon[131504]: purged_snaps scrub starts 2023-11-27T15:19:28.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:28 smithi177 ceph-mon[131504]: purged_snaps scrub ok 2023-11-27T15:19:28.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:28 smithi177 ceph-mon[131504]: Detected new or changed devices on smithi174 2023-11-27T15:19:28.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:28 smithi177 ceph-mon[131504]: Adjusting osd_memory_target on smithi174 to 4292M 2023-11-27T15:19:28.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:28 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:19:28.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:28 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:19:28.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:28 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:19:28.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:28 smithi177 ceph-mon[131504]: osdmap e72: 11 total, 11 up, 11 in 2023-11-27T15:19:28.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:28 smithi174 ceph-mon[131110]: purged_snaps scrub starts 2023-11-27T15:19:28.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:28 smithi174 ceph-mon[131110]: purged_snaps scrub ok 2023-11-27T15:19:28.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:28 smithi174 ceph-mon[131110]: Detected new or changed devices on smithi174 2023-11-27T15:19:28.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:28 smithi174 ceph-mon[131110]: Adjusting osd_memory_target on smithi174 to 4292M 2023-11-27T15:19:28.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:28 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:19:28.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:28 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:19:28.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:28 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:19:28.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:28 smithi174 ceph-mon[131110]: osdmap e72: 11 total, 11 up, 11 in 2023-11-27T15:19:28.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:28 smithi133 ceph-mon[97860]: purged_snaps scrub starts 2023-11-27T15:19:28.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:28 smithi133 ceph-mon[97860]: purged_snaps scrub ok 2023-11-27T15:19:28.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:28 smithi133 ceph-mon[97860]: Detected new or changed devices on smithi174 2023-11-27T15:19:28.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:28 smithi133 ceph-mon[97860]: Adjusting osd_memory_target on smithi174 to 4292M 2023-11-27T15:19:28.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:28 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:19:28.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:28 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:19:28.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:28 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:19:28.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:28 smithi133 ceph-mon[97860]: osdmap e72: 11 total, 11 up, 11 in 2023-11-27T15:19:29.370 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:29 smithi177 ceph-mon[131504]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:19:29.370 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:29 smithi177 ceph-mon[131504]: osdmap e73: 11 total, 11 up, 11 in 2023-11-27T15:19:29.370 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:29 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:19:29.371 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:29 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:19:29.371 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:29 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:29.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:29 smithi174 ceph-mon[131110]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:19:29.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:29 smithi174 ceph-mon[131110]: osdmap e73: 11 total, 11 up, 11 in 2023-11-27T15:19:29.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:29 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:19:29.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:29 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:19:29.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:29 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:29.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:29 smithi133 ceph-mon[97860]: pgmap v225: 1 pgs: 1 remapped+peering; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:19:29.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:29 smithi133 ceph-mon[97860]: osdmap e73: 11 total, 11 up, 11 in 2023-11-27T15:19:29.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:29 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-11-27T15:19:29.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:29 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-11-27T15:19:29.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:29 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:30.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:30 smithi177 ceph-mon[131504]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:19:30.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:30 smithi177 ceph-mon[131504]: osdmap e74: 11 total, 11 up, 11 in 2023-11-27T15:19:30.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:30 smithi174 ceph-mon[131110]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:19:30.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:30 smithi174 ceph-mon[131110]: osdmap e74: 11 total, 11 up, 11 in 2023-11-27T15:19:30.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:30 smithi133 ceph-mon[97860]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi177:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:19:30.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:30 smithi133 ceph-mon[97860]: osdmap e74: 11 total, 11 up, 11 in 2023-11-27T15:19:31.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:31 smithi174 ceph-mon[131110]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:19:31.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:31 smithi174 ceph-mon[131110]: osdmap e75: 11 total, 11 up, 11 in 2023-11-27T15:19:31.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:31 smithi174 ceph-mon[131110]: from='client.? 172.21.15.177:0/1766401530' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9a72525-14d3-4ff6-8a06-abcd6af6396d"}]: dispatch 2023-11-27T15:19:31.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:31 smithi174 ceph-mon[131110]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9a72525-14d3-4ff6-8a06-abcd6af6396d"}]: dispatch 2023-11-27T15:19:31.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:31 smithi174 ceph-mon[131110]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9a72525-14d3-4ff6-8a06-abcd6af6396d"}]': finished 2023-11-27T15:19:31.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:31 smithi174 ceph-mon[131110]: osdmap e76: 12 total, 11 up, 12 in 2023-11-27T15:19:31.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:31 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:19:31.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:31 smithi133 ceph-mon[97860]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:19:31.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:31 smithi133 ceph-mon[97860]: osdmap e75: 11 total, 11 up, 11 in 2023-11-27T15:19:31.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:31 smithi133 ceph-mon[97860]: from='client.? 172.21.15.177:0/1766401530' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9a72525-14d3-4ff6-8a06-abcd6af6396d"}]: dispatch 2023-11-27T15:19:31.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:31 smithi133 ceph-mon[97860]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9a72525-14d3-4ff6-8a06-abcd6af6396d"}]: dispatch 2023-11-27T15:19:31.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:31 smithi133 ceph-mon[97860]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9a72525-14d3-4ff6-8a06-abcd6af6396d"}]': finished 2023-11-27T15:19:31.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:31 smithi133 ceph-mon[97860]: osdmap e76: 12 total, 11 up, 12 in 2023-11-27T15:19:31.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:31 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:19:31.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:31 smithi177 ceph-mon[131504]: pgmap v228: 1 pgs: 1 remapped+peering; 577 KiB data, 67 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:19:31.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:31 smithi177 ceph-mon[131504]: osdmap e75: 11 total, 11 up, 11 in 2023-11-27T15:19:31.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:31 smithi177 ceph-mon[131504]: from='client.? 172.21.15.177:0/1766401530' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9a72525-14d3-4ff6-8a06-abcd6af6396d"}]: dispatch 2023-11-27T15:19:31.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:31 smithi177 ceph-mon[131504]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9a72525-14d3-4ff6-8a06-abcd6af6396d"}]: dispatch 2023-11-27T15:19:31.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:31 smithi177 ceph-mon[131504]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9a72525-14d3-4ff6-8a06-abcd6af6396d"}]': finished 2023-11-27T15:19:31.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:31 smithi177 ceph-mon[131504]: osdmap e76: 12 total, 11 up, 12 in 2023-11-27T15:19:31.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:31 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:19:32.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:32 smithi174 ceph-mon[131110]: from='client.? 172.21.15.177:0/3974350386' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:19:32.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:32 smithi133 ceph-mon[97860]: from='client.? 172.21.15.177:0/3974350386' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:19:32.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:32 smithi177 ceph-mon[131504]: from='client.? 172.21.15.177:0/3974350386' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-11-27T15:19:33.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:33 smithi174 ceph-mon[131110]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:19:33.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:33 smithi133 ceph-mon[97860]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:19:33.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:33 smithi177 ceph-mon[131504]: pgmap v231: 1 pgs: 1 remapped+peering; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:19:35.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:35 smithi174 ceph-mon[131110]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-11-27T15:19:35.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:35 smithi133 ceph-mon[97860]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-11-27T15:19:35.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:35 smithi177 ceph-mon[131504]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-11-27T15:19:37.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:37 smithi177 ceph-mon[131504]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2023-11-27T15:19:37.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:37 smithi174 ceph-mon[131110]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2023-11-27T15:19:37.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:37 smithi133 ceph-mon[97860]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 68 MiB used, 983 GiB / 983 GiB avail; 87 KiB/s, 0 objects/s recovering 2023-11-27T15:19:38.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:38 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-11-27T15:19:38.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:38 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:38 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-11-27T15:19:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:38 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:38.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:38 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-11-27T15:19:38.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:38 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:39.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:39 smithi177 ceph-mon[131504]: Deploying daemon osd.11 on smithi177 2023-11-27T15:19:39.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:39 smithi177 ceph-mon[131504]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-27T15:19:39.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:39 smithi174 ceph-mon[131110]: Deploying daemon osd.11 on smithi177 2023-11-27T15:19:39.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:39 smithi174 ceph-mon[131110]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-27T15:19:39.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:39 smithi133 ceph-mon[97860]: Deploying daemon osd.11 on smithi177 2023-11-27T15:19:39.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:39 smithi133 ceph-mon[97860]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-11-27T15:19:41.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:41 smithi177 ceph-mon[131504]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-11-27T15:19:41.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:41 smithi174 ceph-mon[131110]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-11-27T15:19:41.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:41 smithi133 ceph-mon[97860]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-11-27T15:19:43.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:42 smithi133 ceph-mon[97860]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-11-27T15:19:43.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:42 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:43.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:42 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:43.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:42 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:43.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:42 smithi177 ceph-mon[131504]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-11-27T15:19:43.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:43.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:43.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:42 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:43.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:42 smithi174 ceph-mon[131110]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-11-27T15:19:43.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:42 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:43.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:42 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:43.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:42 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:44.970 INFO:teuthology.orchestra.run.smithi177.stdout:Created osd(s) 11 on host 'smithi177' 2023-11-27T15:19:44.971 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:44.967+0000 7fb26b7fe700 1 -- 172.21.15.177:0/341709548 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fb2741a0e70 con 0x7fb2600a11d0 2023-11-27T15:19:44.973 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:44.969+0000 7fb278ae8700 1 -- 172.21.15.177:0/341709548 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb2600a11d0 msgr2=0x7fb2600a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:44.973 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:44.969+0000 7fb278ae8700 1 --2- 172.21.15.177:0/341709548 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb2600a11d0 0x7fb2600a3680 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb274100240 tx=0x7fb26400a680 comp rx=0 tx=0).stop 2023-11-27T15:19:44.973 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:44.969+0000 7fb278ae8700 1 -- 172.21.15.177:0/341709548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2741003e0 msgr2=0x7fb27419d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:44.973 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:44.969+0000 7fb278ae8700 1 --2- 172.21.15.177:0/341709548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2741003e0 0x7fb27419d1f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00d980 tx=0x7fb25c00dd40 comp rx=0 tx=0).stop 2023-11-27T15:19:44.973 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:44.969+0000 7fb278ae8700 1 -- 172.21.15.177:0/341709548 shutdown_connections 2023-11-27T15:19:44.973 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:44.969+0000 7fb278ae8700 1 --2- 172.21.15.177:0/341709548 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb2600a11d0 0x7fb2600a3680 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:44.973 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:44.969+0000 7fb278ae8700 1 --2- 172.21.15.177:0/341709548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb274100d70 0x7fb27419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:44.973 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:44.969+0000 7fb278ae8700 1 --2- 172.21.15.177:0/341709548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2740ff1e0 0x7fb274069110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:44.973 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:44.969+0000 7fb278ae8700 1 --2- 172.21.15.177:0/341709548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2741003e0 0x7fb27419d1f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:44.973 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:44.969+0000 7fb278ae8700 1 -- 172.21.15.177:0/341709548 >> 172.21.15.177:0/341709548 conn(0x7fb2740fa7f0 msgr2=0x7fb2740fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:44.973 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:44.969+0000 7fb278ae8700 1 -- 172.21.15.177:0/341709548 shutdown_connections 2023-11-27T15:19:44.973 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:44.970+0000 7fb278ae8700 1 -- 172.21.15.177:0/341709548 wait complete. 2023-11-27T15:19:45.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:44 smithi133 ceph-mon[97860]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-27T15:19:45.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:44 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:45.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:44 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:45.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:44 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:45.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:44 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:45.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:44 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:45.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:44 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:45.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:44 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:45.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:44 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:45.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:44 smithi177 ceph-mon[131504]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-27T15:19:45.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:44 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:45.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:44 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:45.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:44 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:45.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:44 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:45.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:44 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:45.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:44 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:45.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:44 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:45.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:44 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:45.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:44 smithi174 ceph-mon[131110]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-11-27T15:19:45.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:44 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:45.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:44 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:45.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:44 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:45.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:44 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:45.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:44 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:45.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:44 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:45.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:44 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:45.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:44 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:45.468 DEBUG:teuthology.orchestra.run.smithi177:osd.11> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.11.service 2023-11-27T15:19:45.470 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-11-27T15:19:45.470 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph osd stat -f json 2023-11-27T15:19:45.499 INFO:journalctl@ceph.osd.11.smithi177.stdout:-- Logs begin at Mon 2023-11-27 14:50:38 UTC. -- 2023-11-27T15:19:46.165 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:19:45 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:19:45.886+0000 7f362cfdd640 -1 osd.11 0 log_to_monitors true 2023-11-27T15:19:46.165 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:45 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:19:46.165 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:45 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:19:46.165 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:45 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:19:46.165 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:45 smithi177 ceph-mon[131504]: from='osd.11 [v2:172.21.15.177:6824/1870203795,v1:172.21.15.177:6825/1870203795]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-27T15:19:46.165 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:45 smithi177 ceph-mon[131504]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-27T15:19:46.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:45 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:19:46.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:45 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:19:46.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:45 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:19:46.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:45 smithi133 ceph-mon[97860]: from='osd.11 [v2:172.21.15.177:6824/1870203795,v1:172.21.15.177:6825/1870203795]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-27T15:19:46.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:45 smithi133 ceph-mon[97860]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-27T15:19:46.326 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:19:46.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:45 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:19:46.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:45 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:19:46.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:45 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:19:46.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:45 smithi174 ceph-mon[131110]: from='osd.11 [v2:172.21.15.177:6824/1870203795,v1:172.21.15.177:6825/1870203795]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-27T15:19:46.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:45 smithi174 ceph-mon[131110]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-11-27T15:19:47.168 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:46 smithi133 ceph-mon[97860]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:19:47.168 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:46 smithi133 ceph-mon[97860]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-11-27T15:19:47.168 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:46 smithi133 ceph-mon[97860]: osdmap e77: 12 total, 11 up, 12 in 2023-11-27T15:19:47.168 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:46 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:19:47.168 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:46 smithi133 ceph-mon[97860]: from='osd.11 [v2:172.21.15.177:6824/1870203795,v1:172.21.15.177:6825/1870203795]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:19:47.168 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:46 smithi133 ceph-mon[97860]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:19:47.276 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:19:46 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:19:46.986+0000 7f3622e55700 -1 osd.11 0 waiting for initial osdmap 2023-11-27T15:19:47.277 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:19:47 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:19:47.003+0000 7f361a441700 -1 osd.11 78 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-27T15:19:47.277 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:46 smithi177 ceph-mon[131504]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:19:47.277 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:46 smithi177 ceph-mon[131504]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-11-27T15:19:47.277 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:46 smithi177 ceph-mon[131504]: osdmap e77: 12 total, 11 up, 12 in 2023-11-27T15:19:47.277 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:46 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:19:47.277 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:46 smithi177 ceph-mon[131504]: from='osd.11 [v2:172.21.15.177:6824/1870203795,v1:172.21.15.177:6825/1870203795]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:19:47.277 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:46 smithi177 ceph-mon[131504]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:19:47.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:46 smithi174 ceph-mon[131110]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:19:47.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:46 smithi174 ceph-mon[131110]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-11-27T15:19:47.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:46 smithi174 ceph-mon[131110]: osdmap e77: 12 total, 11 up, 12 in 2023-11-27T15:19:47.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:46 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:19:47.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:46 smithi174 ceph-mon[131110]: from='osd.11 [v2:172.21.15.177:6824/1870203795,v1:172.21.15.177:6825/1870203795]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:19:47.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:46 smithi174 ceph-mon[131110]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]: dispatch 2023-11-27T15:19:47.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.821+0000 7f2392537700 1 -- 172.21.15.133:0/1140846303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c074b20 msgr2=0x7f238c072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:47.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.821+0000 7f2392537700 1 --2- 172.21.15.133:0/1140846303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c074b20 0x7f238c072f80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f23880099c0 tx=0x7f2388009cd0 comp rx=0 tx=0).stop 2023-11-27T15:19:47.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.821+0000 7f2392537700 1 -- 172.21.15.133:0/1140846303 shutdown_connections 2023-11-27T15:19:47.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.821+0000 7f2392537700 1 --2- 172.21.15.133:0/1140846303 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f238c10c6f0 0x7f238c10ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:47.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.821+0000 7f2392537700 1 --2- 172.21.15.133:0/1140846303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c074b20 0x7f238c072f80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:47.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.821+0000 7f2392537700 1 --2- 172.21.15.133:0/1140846303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f238c073550 0x7f238c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:47.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.821+0000 7f2392537700 1 -- 172.21.15.133:0/1140846303 >> 172.21.15.133:0/1140846303 conn(0x7f238c0fb720 msgr2=0x7f238c0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:47.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.821+0000 7f2392537700 1 -- 172.21.15.133:0/1140846303 shutdown_connections 2023-11-27T15:19:47.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.821+0000 7f2392537700 1 -- 172.21.15.133:0/1140846303 wait complete. 2023-11-27T15:19:47.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.822+0000 7f2392537700 1 Processor -- start 2023-11-27T15:19:47.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.822+0000 7f2392537700 1 -- start start 2023-11-27T15:19:47.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.822+0000 7f2392537700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c073550 0x7f238c19c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:47.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.823+0000 7f2392537700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f238c074b20 0x7f238c19c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:47.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.823+0000 7f2392537700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f238c10c6f0 0x7f238c1a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:47.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.823+0000 7f2392537700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238c1a3ff0 con 0x7f238c073550 2023-11-27T15:19:47.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.823+0000 7f2392537700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238c1a4130 con 0x7f238c10c6f0 2023-11-27T15:19:47.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.823+0000 7f2392537700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238c1a42c0 con 0x7f238c074b20 2023-11-27T15:19:47.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.823+0000 7f2391535700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c073550 0x7f238c19c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:47.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.823+0000 7f2390d34700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f238c074b20 0x7f238c19c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:47.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.823+0000 7f2391535700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c073550 0x7f238c19c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54720/0 (socket says 172.21.15.133:54720) 2023-11-27T15:19:47.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.823+0000 7f2391535700 1 -- 172.21.15.133:0/3655994752 learned_addr learned my addr 172.21.15.133:0/3655994752 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:19:47.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.823+0000 7f2391d36700 1 --2- 172.21.15.133:0/3655994752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f238c10c6f0 0x7f238c1a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:47.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.823+0000 7f2391535700 1 -- 172.21.15.133:0/3655994752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f238c074b20 msgr2=0x7f238c19c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:47.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.823+0000 7f2391535700 1 --2- 172.21.15.133:0/3655994752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f238c074b20 0x7f238c19c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:47.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.823+0000 7f2391535700 1 -- 172.21.15.133:0/3655994752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f238c10c6f0 msgr2=0x7f238c1a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:47.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.823+0000 7f2391535700 1 --2- 172.21.15.133:0/3655994752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f238c10c6f0 0x7f238c1a3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:47.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.823+0000 7f2391535700 1 -- 172.21.15.133:0/3655994752 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23880095f0 con 0x7f238c073550 2023-11-27T15:19:47.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.823+0000 7f2390d34700 1 --2- 172.21.15.133:0/3655994752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f238c074b20 0x7f238c19c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:47.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.823+0000 7f2391d36700 1 --2- 172.21.15.133:0/3655994752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f238c10c6f0 0x7f238c1a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:19:47.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.824+0000 7f2391535700 1 --2- 172.21.15.133:0/3655994752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c073550 0x7f238c19c460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f238800b580 tx=0x7f238800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:47.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.824+0000 7f23827fc700 1 -- 172.21.15.133:0/3655994752 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238801d070 con 0x7f238c073550 2023-11-27T15:19:47.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.824+0000 7f2392537700 1 -- 172.21.15.133:0/3655994752 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f238c112060 con 0x7f238c073550 2023-11-27T15:19:47.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.824+0000 7f2392537700 1 -- 172.21.15.133:0/3655994752 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238c111ed0 con 0x7f238c073550 2023-11-27T15:19:47.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.824+0000 7f23827fc700 1 -- 172.21.15.133:0/3655994752 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2388005510 con 0x7f238c073550 2023-11-27T15:19:47.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.824+0000 7f23827fc700 1 -- 172.21.15.133:0/3655994752 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388004b10 con 0x7f238c073550 2023-11-27T15:19:47.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.825+0000 7f2392537700 1 -- 172.21.15.133:0/3655994752 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f238c102240 con 0x7f238c073550 2023-11-27T15:19:47.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.827+0000 7f23827fc700 1 -- 172.21.15.133:0/3655994752 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f238800fb70 con 0x7f238c073550 2023-11-27T15:19:47.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.828+0000 7f23827fc700 1 --2- 172.21.15.133:0/3655994752 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f23780a1150 0x7f23780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:47.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.829+0000 7f23827fc700 1 -- 172.21.15.133:0/3655994752 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f23880cd1e0 con 0x7f238c073550 2023-11-27T15:19:47.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.829+0000 7f2390d34700 1 --2- 172.21.15.133:0/3655994752 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f23780a1150 0x7f23780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:47.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.829+0000 7f2390d34700 1 --2- 172.21.15.133:0/3655994752 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f23780a1150 0x7f23780a3600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f237c011490 tx=0x7f237c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:47.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:47.833+0000 7f23827fc700 1 -- 172.21.15.133:0/3655994752 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f238805b730 con 0x7f238c073550 2023-11-27T15:19:48.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:48.041+0000 7f2392537700 1 -- 172.21.15.133:0/3655994752 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f238c061700 con 0x7f238c073550 2023-11-27T15:19:48.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:48.041+0000 7f23827fc700 1 -- 172.21.15.133:0/3655994752 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7f2388096ee0 con 0x7f238c073550 2023-11-27T15:19:48.045 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:19:48.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:48.043+0000 7f2392537700 1 -- 172.21.15.133:0/3655994752 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f23780a1150 msgr2=0x7f23780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:48.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:48.043+0000 7f2392537700 1 --2- 172.21.15.133:0/3655994752 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f23780a1150 0x7f23780a3600 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f237c011490 tx=0x7f237c009260 comp rx=0 tx=0).stop 2023-11-27T15:19:48.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:48.043+0000 7f2392537700 1 -- 172.21.15.133:0/3655994752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c073550 msgr2=0x7f238c19c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:48.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:48.043+0000 7f2392537700 1 --2- 172.21.15.133:0/3655994752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c073550 0x7f238c19c460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f238800b580 tx=0x7f238800f690 comp rx=0 tx=0).stop 2023-11-27T15:19:48.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:48.044+0000 7f2392537700 1 -- 172.21.15.133:0/3655994752 shutdown_connections 2023-11-27T15:19:48.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:48.044+0000 7f2392537700 1 --2- 172.21.15.133:0/3655994752 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f23780a1150 0x7f23780a3600 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:48.044+0000 7f2392537700 1 --2- 172.21.15.133:0/3655994752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f238c10c6f0 0x7f238c1a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:48.044+0000 7f2392537700 1 --2- 172.21.15.133:0/3655994752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f238c073550 0x7f238c19c460 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:48.044+0000 7f2392537700 1 --2- 172.21.15.133:0/3655994752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f238c074b20 0x7f238c19c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:48.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:48.044+0000 7f2392537700 1 -- 172.21.15.133:0/3655994752 >> 172.21.15.133:0/3655994752 conn(0x7f238c0fb720 msgr2=0x7f238c0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:48.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:48.044+0000 7f2392537700 1 -- 172.21.15.133:0/3655994752 shutdown_connections 2023-11-27T15:19:48.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:48.044+0000 7f2392537700 1 -- 172.21.15.133:0/3655994752 wait complete. 2023-11-27T15:19:48.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:47 smithi133 ceph-mon[97860]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2023-11-27T15:19:48.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:47 smithi133 ceph-mon[97860]: osdmap e78: 12 total, 11 up, 12 in 2023-11-27T15:19:48.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:19:48.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:19:48.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:48.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:48.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:48.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:48.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:48.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:48.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:48.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:48.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:48.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:47 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:48.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:47 smithi177 ceph-mon[131504]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2023-11-27T15:19:48.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:47 smithi177 ceph-mon[131504]: osdmap e78: 12 total, 11 up, 12 in 2023-11-27T15:19:48.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:19:48.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:19:48.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:48.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:48.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:48.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:48.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:48.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:48.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:48.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:48.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:48.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:47 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:48.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:47 smithi174 ceph-mon[131110]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi177", "root=default"]}]': finished 2023-11-27T15:19:48.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:47 smithi174 ceph-mon[131110]: osdmap e78: 12 total, 11 up, 12 in 2023-11-27T15:19:48.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:47 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:19:48.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:47 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:19:48.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:47 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:48.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:47 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:48.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:47 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:48.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:47 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:48.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:47 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:48.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:47 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:19:48.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:47 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:48.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:47 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:48.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:47 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:48.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:47 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:48.504 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1701098387,"num_in_osds":12,"osd_in_since":1701098370,"num_remapped_pgs":0} 2023-11-27T15:19:48.505 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph osd dump --format=json 2023-11-27T15:19:49.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:48 smithi133 ceph-mon[97860]: purged_snaps scrub starts 2023-11-27T15:19:49.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:48 smithi133 ceph-mon[97860]: purged_snaps scrub ok 2023-11-27T15:19:49.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:48 smithi133 ceph-mon[97860]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:19:49.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:48 smithi133 ceph-mon[97860]: Detected new or changed devices on smithi177 2023-11-27T15:19:49.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:48 smithi133 ceph-mon[97860]: Adjusting osd_memory_target on smithi177 to 4292M 2023-11-27T15:19:49.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:48 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:19:49.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:48 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:19:49.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:48 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:19:49.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:48 smithi133 ceph-mon[97860]: osd.11 [v2:172.21.15.177:6824/1870203795,v1:172.21.15.177:6825/1870203795] boot 2023-11-27T15:19:49.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:48 smithi133 ceph-mon[97860]: osdmap e79: 12 total, 12 up, 12 in 2023-11-27T15:19:49.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:48 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:19:49.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:48 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3655994752' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-27T15:19:49.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:48 smithi177 ceph-mon[131504]: purged_snaps scrub starts 2023-11-27T15:19:49.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:48 smithi177 ceph-mon[131504]: purged_snaps scrub ok 2023-11-27T15:19:49.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:48 smithi177 ceph-mon[131504]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:19:49.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:48 smithi177 ceph-mon[131504]: Detected new or changed devices on smithi177 2023-11-27T15:19:49.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:48 smithi177 ceph-mon[131504]: Adjusting osd_memory_target on smithi177 to 4292M 2023-11-27T15:19:49.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:48 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:19:49.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:48 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:19:49.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:48 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:19:49.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:48 smithi177 ceph-mon[131504]: osd.11 [v2:172.21.15.177:6824/1870203795,v1:172.21.15.177:6825/1870203795] boot 2023-11-27T15:19:49.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:48 smithi177 ceph-mon[131504]: osdmap e79: 12 total, 12 up, 12 in 2023-11-27T15:19:49.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:48 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:19:49.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:48 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3655994752' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-27T15:19:49.352 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:19:49.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:48 smithi174 ceph-mon[131110]: purged_snaps scrub starts 2023-11-27T15:19:49.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:48 smithi174 ceph-mon[131110]: purged_snaps scrub ok 2023-11-27T15:19:49.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:48 smithi174 ceph-mon[131110]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 69 MiB used, 983 GiB / 983 GiB avail 2023-11-27T15:19:49.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:48 smithi174 ceph-mon[131110]: Detected new or changed devices on smithi177 2023-11-27T15:19:49.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:48 smithi174 ceph-mon[131110]: Adjusting osd_memory_target on smithi177 to 4292M 2023-11-27T15:19:49.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:48 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:19:49.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:48 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:19:49.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:48 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:19:49.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:48 smithi174 ceph-mon[131110]: osd.11 [v2:172.21.15.177:6824/1870203795,v1:172.21.15.177:6825/1870203795] boot 2023-11-27T15:19:49.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:48 smithi174 ceph-mon[131110]: osdmap e79: 12 total, 12 up, 12 in 2023-11-27T15:19:49.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:48 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-11-27T15:19:49.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:48 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3655994752' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-11-27T15:19:50.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:50 smithi133 ceph-mon[97860]: osdmap e80: 12 total, 12 up, 12 in 2023-11-27T15:19:50.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:50 smithi174 ceph-mon[131110]: osdmap e80: 12 total, 12 up, 12 in 2023-11-27T15:19:50.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:50 smithi177 ceph-mon[131504]: osdmap e80: 12 total, 12 up, 12 in 2023-11-27T15:19:50.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.828+0000 7fd18d363700 1 -- 172.21.15.133:0/3523060856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd188100ee0 msgr2=0x7fd188104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:50.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.828+0000 7fd18d363700 1 --2- 172.21.15.133:0/3523060856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd188100ee0 0x7fd188104eb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0099c0 tx=0x7fd17c009cd0 comp rx=0 tx=0).stop 2023-11-27T15:19:50.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.829+0000 7fd18d363700 1 -- 172.21.15.133:0/3523060856 shutdown_connections 2023-11-27T15:19:50.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.829+0000 7fd18d363700 1 --2- 172.21.15.133:0/3523060856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1881055e0 0x7fd18810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:50.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.829+0000 7fd18d363700 1 --2- 172.21.15.133:0/3523060856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd188100540 0x7fd188100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:50.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.829+0000 7fd18d363700 1 --2- 172.21.15.133:0/3523060856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd188100ee0 0x7fd188104eb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:50.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.829+0000 7fd18d363700 1 -- 172.21.15.133:0/3523060856 >> 172.21.15.133:0/3523060856 conn(0x7fd1880fbdc0 msgr2=0x7fd1880fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:50.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.829+0000 7fd18d363700 1 -- 172.21.15.133:0/3523060856 shutdown_connections 2023-11-27T15:19:50.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.829+0000 7fd18d363700 1 -- 172.21.15.133:0/3523060856 wait complete. 2023-11-27T15:19:50.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.830+0000 7fd18d363700 1 Processor -- start 2023-11-27T15:19:50.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.830+0000 7fd18d363700 1 -- start start 2023-11-27T15:19:50.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.830+0000 7fd18d363700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd188100540 0x7fd1881963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:50.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.830+0000 7fd18d363700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd188100ee0 0x7fd188196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:50.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.830+0000 7fd18d363700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1881055e0 0x7fd18819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:50.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.830+0000 7fd18d363700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18819b3b0 con 0x7fd1881055e0 2023-11-27T15:19:50.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.830+0000 7fd18d363700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18819b520 con 0x7fd188100540 2023-11-27T15:19:50.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.830+0000 7fd18d363700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18819b6e0 con 0x7fd188100ee0 2023-11-27T15:19:50.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.830+0000 7fd1867fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd188100ee0 0x7fd188196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:50.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.830+0000 7fd1867fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd188100ee0 0x7fd188196930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:43398/0 (socket says 172.21.15.133:43398) 2023-11-27T15:19:50.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.830+0000 7fd1867fc700 1 -- 172.21.15.133:0/4077923319 learned_addr learned my addr 172.21.15.133:0/4077923319 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:19:50.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.830+0000 7fd186ffd700 1 --2- 172.21.15.133:0/4077923319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd188100540 0x7fd1881963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:50.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.830+0000 7fd1877fe700 1 --2- 172.21.15.133:0/4077923319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1881055e0 0x7fd18819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:50.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.831+0000 7fd1877fe700 1 -- 172.21.15.133:0/4077923319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd188100ee0 msgr2=0x7fd188196930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:50.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.831+0000 7fd1877fe700 1 --2- 172.21.15.133:0/4077923319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd188100ee0 0x7fd188196930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:50.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.831+0000 7fd1877fe700 1 -- 172.21.15.133:0/4077923319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd188100540 msgr2=0x7fd1881963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:50.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.831+0000 7fd1877fe700 1 --2- 172.21.15.133:0/4077923319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd188100540 0x7fd1881963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:50.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.831+0000 7fd1877fe700 1 -- 172.21.15.133:0/4077923319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c0095f0 con 0x7fd1881055e0 2023-11-27T15:19:50.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.831+0000 7fd1877fe700 1 --2- 172.21.15.133:0/4077923319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1881055e0 0x7fd18819ada0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd180007f10 tx=0x7fd18000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:50.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.831+0000 7fd16ffff700 1 -- 172.21.15.133:0/4077923319 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180009ea0 con 0x7fd1881055e0 2023-11-27T15:19:50.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.831+0000 7fd16ffff700 1 -- 172.21.15.133:0/4077923319 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd18000f040 con 0x7fd1881055e0 2023-11-27T15:19:50.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.831+0000 7fd16ffff700 1 -- 172.21.15.133:0/4077923319 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180004350 con 0x7fd1881055e0 2023-11-27T15:19:50.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.831+0000 7fd18d363700 1 -- 172.21.15.133:0/4077923319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd188110360 con 0x7fd1881055e0 2023-11-27T15:19:50.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.831+0000 7fd18d363700 1 -- 172.21.15.133:0/4077923319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1881101d0 con 0x7fd1881055e0 2023-11-27T15:19:50.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.833+0000 7fd18d363700 1 -- 172.21.15.133:0/4077923319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1881104f0 con 0x7fd1881055e0 2023-11-27T15:19:50.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.835+0000 7fd16ffff700 1 -- 172.21.15.133:0/4077923319 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd18000db20 con 0x7fd1881055e0 2023-11-27T15:19:50.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.836+0000 7fd16ffff700 1 --2- 172.21.15.133:0/4077923319 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd1700a11d0 0x7fd1700a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:50.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.836+0000 7fd16ffff700 1 -- 172.21.15.133:0/4077923319 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd1800cbcd0 con 0x7fd1881055e0 2023-11-27T15:19:50.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.836+0000 7fd186ffd700 1 --2- 172.21.15.133:0/4077923319 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd1700a11d0 0x7fd1700a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:50.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.837+0000 7fd186ffd700 1 --2- 172.21.15.133:0/4077923319 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd1700a11d0 0x7fd1700a3680 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd178009c60 tx=0x7fd178009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:50.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:50.840+0000 7fd16ffff700 1 -- 172.21.15.133:0/4077923319 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd180095820 con 0x7fd1881055e0 2023-11-27T15:19:51.040 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:51.036+0000 7fd18d363700 1 -- 172.21.15.133:0/4077923319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd188066ea0 con 0x7fd1881055e0 2023-11-27T15:19:51.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:51.037+0000 7fd16ffff700 1 -- 172.21.15.133:0/4077923319 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18458 (secure 0 0 0) 0x7fd180098e40 con 0x7fd1881055e0 2023-11-27T15:19:51.042 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:19:51.042 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":80,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:19:49.094979+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:19:51.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:51.040+0000 7fd18d363700 1 -- 172.21.15.133:0/4077923319 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd1700a11d0 msgr2=0x7fd1700a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:51.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:51.040+0000 7fd18d363700 1 --2- 172.21.15.133:0/4077923319 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd1700a11d0 0x7fd1700a3680 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd178009c60 tx=0x7fd178009260 comp rx=0 tx=0).stop 2023-11-27T15:19:51.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:51.040+0000 7fd18d363700 1 -- 172.21.15.133:0/4077923319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1881055e0 msgr2=0x7fd18819ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:51.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:51.040+0000 7fd18d363700 1 --2- 172.21.15.133:0/4077923319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1881055e0 0x7fd18819ada0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd180007f10 tx=0x7fd18000d3b0 comp rx=0 tx=0).stop 2023-11-27T15:19:51.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:51.040+0000 7fd18d363700 1 -- 172.21.15.133:0/4077923319 shutdown_connections 2023-11-27T15:19:51.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:51.040+0000 7fd18d363700 1 --2- 172.21.15.133:0/4077923319 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd1700a11d0 0x7fd1700a3680 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:51.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:51.040+0000 7fd18d363700 1 --2- 172.21.15.133:0/4077923319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd1881055e0 0x7fd18819ada0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:51.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:51.040+0000 7fd18d363700 1 --2- 172.21.15.133:0/4077923319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd188100540 0x7fd1881963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:51.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:51.040+0000 7fd18d363700 1 --2- 172.21.15.133:0/4077923319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd188100ee0 0x7fd188196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:51.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:51.040+0000 7fd18d363700 1 -- 172.21.15.133:0/4077923319 >> 172.21.15.133:0/4077923319 conn(0x7fd1880fbdc0 msgr2=0x7fd1880fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:51.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:51.040+0000 7fd18d363700 1 -- 172.21.15.133:0/4077923319 shutdown_connections 2023-11-27T15:19:51.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:51.040+0000 7fd18d363700 1 -- 172.21.15.133:0/4077923319 wait complete. 2023-11-27T15:19:51.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:51 smithi174 ceph-mon[131110]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:19:51.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:51 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/4077923319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:19:51.414 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:51 smithi133 ceph-mon[97860]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:19:51.414 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:51 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/4077923319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:19:51.434 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:16:20.216150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-27T15:19:51.435 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph osd pool get .mgr pg_num 2023-11-27T15:19:51.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:51 smithi177 ceph-mon[131504]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:19:51.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:51 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/4077923319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:19:52.280 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:19:53.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:53 smithi133 ceph-mon[97860]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:19:53.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:53 smithi174 ceph-mon[131110]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:19:53.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:53 smithi177 ceph-mon[131504]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:19:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.757+0000 7fa653093700 1 -- 172.21.15.133:0/3925423050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa64c100540 msgr2=0x7fa64c100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.757+0000 7fa653093700 1 --2- 172.21.15.133:0/3925423050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa64c100540 0x7fa64c100910 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa6400098f0 tx=0x7fa640009c00 comp rx=0 tx=0).stop 2023-11-27T15:19:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.757+0000 7fa653093700 1 -- 172.21.15.133:0/3925423050 shutdown_connections 2023-11-27T15:19:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.757+0000 7fa653093700 1 --2- 172.21.15.133:0/3925423050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa64c1055e0 0x7fa64c10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.757+0000 7fa653093700 1 --2- 172.21.15.133:0/3925423050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa64c100540 0x7fa64c100910 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.757+0000 7fa653093700 1 --2- 172.21.15.133:0/3925423050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa64c100ee0 0x7fa64c104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.757+0000 7fa653093700 1 -- 172.21.15.133:0/3925423050 >> 172.21.15.133:0/3925423050 conn(0x7fa64c0fbdc0 msgr2=0x7fa64c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.757+0000 7fa653093700 1 -- 172.21.15.133:0/3925423050 shutdown_connections 2023-11-27T15:19:53.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.757+0000 7fa653093700 1 -- 172.21.15.133:0/3925423050 wait complete. 2023-11-27T15:19:53.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.758+0000 7fa653093700 1 Processor -- start 2023-11-27T15:19:53.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.758+0000 7fa653093700 1 -- start start 2023-11-27T15:19:53.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.759+0000 7fa653093700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa64c100540 0x7fa64c196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.759+0000 7fa653093700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa64c100ee0 0x7fa64c196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.759+0000 7fa653093700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa64c1055e0 0x7fa64c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.759+0000 7fa653093700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c19b3d0 con 0x7fa64c1055e0 2023-11-27T15:19:53.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.759+0000 7fa653093700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c19b540 con 0x7fa64c100540 2023-11-27T15:19:53.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.759+0000 7fa653093700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c19b700 con 0x7fa64c100ee0 2023-11-27T15:19:53.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.759+0000 7fa650e2f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa64c100540 0x7fa64c196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.759+0000 7fa651630700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa64c1055e0 0x7fa64c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.759+0000 7fa650e2f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa64c100540 0x7fa64c196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:42990/0 (socket says 172.21.15.133:42990) 2023-11-27T15:19:53.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.759+0000 7fa650e2f700 1 -- 172.21.15.133:0/863755394 learned_addr learned my addr 172.21.15.133:0/863755394 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:19:53.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.759+0000 7fa64bfff700 1 --2- 172.21.15.133:0/863755394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa64c100ee0 0x7fa64c196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.759+0000 7fa650e2f700 1 -- 172.21.15.133:0/863755394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa64c100ee0 msgr2=0x7fa64c196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:53.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.759+0000 7fa650e2f700 1 --2- 172.21.15.133:0/863755394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa64c100ee0 0x7fa64c196950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.759+0000 7fa650e2f700 1 -- 172.21.15.133:0/863755394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa64c1055e0 msgr2=0x7fa64c19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:53.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.759+0000 7fa650e2f700 1 --2- 172.21.15.133:0/863755394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa64c1055e0 0x7fa64c19adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.759+0000 7fa650e2f700 1 -- 172.21.15.133:0/863755394 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa644009d00 con 0x7fa64c100540 2023-11-27T15:19:53.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.760+0000 7fa650e2f700 1 --2- 172.21.15.133:0/863755394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa64c100540 0x7fa64c196410 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa640009230 tx=0x7fa64000fa80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:53.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.760+0000 7fa649ffb700 1 -- 172.21.15.133:0/863755394 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64001c070 con 0x7fa64c100540 2023-11-27T15:19:53.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.760+0000 7fa653093700 1 -- 172.21.15.133:0/863755394 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa640009520 con 0x7fa64c100540 2023-11-27T15:19:53.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.760+0000 7fa649ffb700 1 -- 172.21.15.133:0/863755394 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6400058e0 con 0x7fa64c100540 2023-11-27T15:19:53.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.760+0000 7fa649ffb700 1 -- 172.21.15.133:0/863755394 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6400038e0 con 0x7fa64c100540 2023-11-27T15:19:53.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.760+0000 7fa653093700 1 -- 172.21.15.133:0/863755394 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64c110320 con 0x7fa64c100540 2023-11-27T15:19:53.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.761+0000 7fa653093700 1 -- 172.21.15.133:0/863755394 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa64c110190 con 0x7fa64c100540 2023-11-27T15:19:53.767 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.763+0000 7fa649ffb700 1 -- 172.21.15.133:0/863755394 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa640020030 con 0x7fa64c100540 2023-11-27T15:19:53.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.764+0000 7fa649ffb700 1 --2- 172.21.15.133:0/863755394 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa63c0a1150 0x7fa63c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:53.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.764+0000 7fa649ffb700 1 -- 172.21.15.133:0/863755394 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa6400cd720 con 0x7fa64c100540 2023-11-27T15:19:53.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.765+0000 7fa64bfff700 1 --2- 172.21.15.133:0/863755394 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa63c0a1150 0x7fa63c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:53.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.765+0000 7fa64bfff700 1 --2- 172.21.15.133:0/863755394 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa63c0a1150 0x7fa63c0a3600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa64c197a30 tx=0x7fa638009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:53.770 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.767+0000 7fa649ffb700 1 -- 172.21.15.133:0/863755394 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa640097300 con 0x7fa64c100540 2023-11-27T15:19:53.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.963+0000 7fa653093700 1 -- 172.21.15.133:0/863755394 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa64c1104b0 con 0x7fa64c100540 2023-11-27T15:19:53.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.964+0000 7fa649ffb700 1 -- 172.21.15.133:0/863755394 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa64009a920 con 0x7fa64c100540 2023-11-27T15:19:53.967 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-11-27T15:19:53.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.966+0000 7fa653093700 1 -- 172.21.15.133:0/863755394 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa63c0a1150 msgr2=0x7fa63c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:53.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.966+0000 7fa653093700 1 --2- 172.21.15.133:0/863755394 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa63c0a1150 0x7fa63c0a3600 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa64c197a30 tx=0x7fa638009260 comp rx=0 tx=0).stop 2023-11-27T15:19:53.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.966+0000 7fa653093700 1 -- 172.21.15.133:0/863755394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa64c100540 msgr2=0x7fa64c196410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:53.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.966+0000 7fa653093700 1 --2- 172.21.15.133:0/863755394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa64c100540 0x7fa64c196410 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa640009230 tx=0x7fa64000fa80 comp rx=0 tx=0).stop 2023-11-27T15:19:53.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.966+0000 7fa653093700 1 -- 172.21.15.133:0/863755394 shutdown_connections 2023-11-27T15:19:53.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.966+0000 7fa653093700 1 --2- 172.21.15.133:0/863755394 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa63c0a1150 0x7fa63c0a3600 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.966+0000 7fa653093700 1 --2- 172.21.15.133:0/863755394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa64c1055e0 0x7fa64c19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.966+0000 7fa653093700 1 --2- 172.21.15.133:0/863755394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa64c100540 0x7fa64c196410 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.966+0000 7fa653093700 1 --2- 172.21.15.133:0/863755394 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa64c100ee0 0x7fa64c196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:53.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.966+0000 7fa653093700 1 -- 172.21.15.133:0/863755394 >> 172.21.15.133:0/863755394 conn(0x7fa64c0fbdc0 msgr2=0x7fa64c0fc880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:53.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.966+0000 7fa653093700 1 -- 172.21.15.133:0/863755394 shutdown_connections 2023-11-27T15:19:53.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:53.966+0000 7fa653093700 1 -- 172.21.15.133:0/863755394 wait complete. 2023-11-27T15:19:54.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:54 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/863755394' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:19:54.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:54 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/863755394' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:19:54.458 INFO:tasks.cephadm:Adding mds.a on smithi133 2023-11-27T15:19:54.458 INFO:tasks.cephadm:Adding mds.d on smithi133 2023-11-27T15:19:54.458 INFO:tasks.cephadm:Adding mds.g on smithi133 2023-11-27T15:19:54.459 INFO:tasks.cephadm:Adding mds.j on smithi133 2023-11-27T15:19:54.459 INFO:tasks.cephadm:Adding mds.b on smithi174 2023-11-27T15:19:54.459 INFO:tasks.cephadm:Adding mds.e on smithi174 2023-11-27T15:19:54.459 INFO:tasks.cephadm:Adding mds.h on smithi174 2023-11-27T15:19:54.459 INFO:tasks.cephadm:Adding mds.k on smithi174 2023-11-27T15:19:54.459 INFO:tasks.cephadm:Adding mds.c on smithi177 2023-11-27T15:19:54.459 INFO:tasks.cephadm:Adding mds.f on smithi177 2023-11-27T15:19:54.459 INFO:tasks.cephadm:Adding mds.i on smithi177 2023-11-27T15:19:54.459 INFO:tasks.cephadm:Adding mds.l on smithi177 2023-11-27T15:19:54.459 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph orch apply mds all '12;smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi174=b;smithi174=e;smithi174=h;smithi174=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l' 2023-11-27T15:19:54.492 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:54 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/863755394' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:19:55.114 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:19:55.376 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:55 smithi177 ceph-mon[131504]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:19:55.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:55 smithi174 ceph-mon[131110]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:19:55.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:55 smithi133 ceph-mon[97860]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:19:56.312 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.305+0000 7f5cc424f700 1 -- 172.21.15.177:0/14931665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cbc100d70 msgr2=0x7f5cbc107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:56.312 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.305+0000 7f5cc424f700 1 --2- 172.21.15.177:0/14931665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cbc100d70 0x7f5cbc107630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400b0a0 tx=0x7f5cb400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.312 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.305+0000 7f5cc424f700 1 -- 172.21.15.177:0/14931665 shutdown_connections 2023-11-27T15:19:56.312 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.305+0000 7f5cc424f700 1 --2- 172.21.15.177:0/14931665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cbc100d70 0x7f5cbc107630 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.312 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.305+0000 7f5cc424f700 1 --2- 172.21.15.177:0/14931665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cbc0ff1e0 0x7f5cbc0ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.312 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.305+0000 7f5cc424f700 1 --2- 172.21.15.177:0/14931665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cbc1003e0 0x7f5cbc100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.312 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.305+0000 7f5cc424f700 1 -- 172.21.15.177:0/14931665 >> 172.21.15.177:0/14931665 conn(0x7f5cbc0fa7f0 msgr2=0x7f5cbc0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:56.312 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.305+0000 7f5cc424f700 1 -- 172.21.15.177:0/14931665 shutdown_connections 2023-11-27T15:19:56.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.305+0000 7f5cc424f700 1 -- 172.21.15.177:0/14931665 wait complete. 2023-11-27T15:19:56.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.306+0000 7f5cc424f700 1 Processor -- start 2023-11-27T15:19:56.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.306+0000 7f5cc424f700 1 -- start start 2023-11-27T15:19:56.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.306+0000 7f5cc424f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cbc0ff1e0 0x7f5cbc109a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.306+0000 7f5cc424f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cbc1003e0 0x7f5cbc1080e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.306+0000 7f5cc424f700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cbc100d70 0x7f5cbc108620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.306+0000 7f5cc424f700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc108da0 con 0x7f5cbc100d70 2023-11-27T15:19:56.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.306+0000 7f5cc424f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc108f10 con 0x7f5cbc1003e0 2023-11-27T15:19:56.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.306+0000 7f5cc424f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc1a0290 con 0x7f5cbc0ff1e0 2023-11-27T15:19:56.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.306+0000 7f5cc1feb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cbc0ff1e0 0x7f5cbc109a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.306+0000 7f5cc1feb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cbc0ff1e0 0x7f5cbc109a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:58292/0 (socket says 172.21.15.177:58292) 2023-11-27T15:19:56.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.306+0000 7f5cc1feb700 1 -- 172.21.15.177:0/2071803735 learned_addr learned my addr 172.21.15.177:0/2071803735 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-11-27T15:19:56.313 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.306+0000 7f5cc17ea700 1 --2- 172.21.15.177:0/2071803735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cbc1003e0 0x7f5cbc1080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.314 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.307+0000 7f5cc27ec700 1 --2- 172.21.15.177:0/2071803735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cbc100d70 0x7f5cbc108620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.314 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.307+0000 7f5cc1feb700 1 -- 172.21.15.177:0/2071803735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cbc1003e0 msgr2=0x7f5cbc1080e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:56.314 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.307+0000 7f5cc1feb700 1 --2- 172.21.15.177:0/2071803735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cbc1003e0 0x7f5cbc1080e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.314 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.307+0000 7f5cc1feb700 1 -- 172.21.15.177:0/2071803735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cbc100d70 msgr2=0x7f5cbc108620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:56.314 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.307+0000 7f5cc1feb700 1 --2- 172.21.15.177:0/2071803735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cbc100d70 0x7f5cbc108620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.314 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.307+0000 7f5cc1feb700 1 -- 172.21.15.177:0/2071803735 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb4009d00 con 0x7f5cbc0ff1e0 2023-11-27T15:19:56.314 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.307+0000 7f5cc1feb700 1 --2- 172.21.15.177:0/2071803735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cbc0ff1e0 0x7f5cbc109a90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00c970 tx=0x7f5cac00cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:56.314 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.307+0000 7f5cb2ffd700 1 -- 172.21.15.177:0/2071803735 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac004220 con 0x7f5cbc0ff1e0 2023-11-27T15:19:56.314 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.307+0000 7f5cb2ffd700 1 -- 172.21.15.177:0/2071803735 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cac007d60 con 0x7f5cbc0ff1e0 2023-11-27T15:19:56.314 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.307+0000 7f5cc424f700 1 -- 172.21.15.177:0/2071803735 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cbc10bd40 con 0x7f5cbc0ff1e0 2023-11-27T15:19:56.314 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.307+0000 7f5cb2ffd700 1 -- 172.21.15.177:0/2071803735 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac003c90 con 0x7f5cbc0ff1e0 2023-11-27T15:19:56.314 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.307+0000 7f5cc424f700 1 -- 172.21.15.177:0/2071803735 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cbc10bbb0 con 0x7f5cbc0ff1e0 2023-11-27T15:19:56.315 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.308+0000 7f5cc424f700 1 -- 172.21.15.177:0/2071803735 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cbc1067e0 con 0x7f5cbc0ff1e0 2023-11-27T15:19:56.315 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.309+0000 7f5cb2ffd700 1 -- 172.21.15.177:0/2071803735 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5cac004380 con 0x7f5cbc0ff1e0 2023-11-27T15:19:56.315 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.310+0000 7f5cb2ffd700 1 --2- 172.21.15.177:0/2071803735 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5ca80a11d0 0x7f5ca80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:56.315 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.310+0000 7f5cb2ffd700 1 -- 172.21.15.177:0/2071803735 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5cac0cc3e0 con 0x7f5cbc0ff1e0 2023-11-27T15:19:56.315 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.310+0000 7f5cc17ea700 1 --2- 172.21.15.177:0/2071803735 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5ca80a11d0 0x7f5ca80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:56.316 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.310+0000 7f5cc17ea700 1 --2- 172.21.15.177:0/2071803735 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5ca80a11d0 0x7f5ca80a3680 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009a10 tx=0x7f5cb8009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:56.317 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.313+0000 7f5cb2ffd700 1 -- 172.21.15.177:0/2071803735 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5cac095fb0 con 0x7f5cbc0ff1e0 2023-11-27T15:19:56.511 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.507+0000 7f5cc424f700 1 -- 172.21.15.177:0/2071803735 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi174=b;smithi174=e;smithi174=h;smithi174=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cbc061af0 con 0x7f5ca80a11d0 2023-11-27T15:19:56.520 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.515+0000 7f5cb2ffd700 1 -- 172.21.15.177:0/2071803735 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f5cbc061af0 con 0x7f5ca80a11d0 2023-11-27T15:19:56.520 INFO:teuthology.orchestra.run.smithi177.stdout:Scheduled mds.all update... 2023-11-27T15:19:56.522 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.518+0000 7f5cc424f700 1 -- 172.21.15.177:0/2071803735 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5ca80a11d0 msgr2=0x7f5ca80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:56.522 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.518+0000 7f5cc424f700 1 --2- 172.21.15.177:0/2071803735 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5ca80a11d0 0x7f5ca80a3680 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8009a10 tx=0x7f5cb8009190 comp rx=0 tx=0).stop 2023-11-27T15:19:56.522 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.518+0000 7f5cc424f700 1 -- 172.21.15.177:0/2071803735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cbc0ff1e0 msgr2=0x7f5cbc109a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:56.522 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.518+0000 7f5cc424f700 1 --2- 172.21.15.177:0/2071803735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cbc0ff1e0 0x7f5cbc109a90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00c970 tx=0x7f5cac00cc80 comp rx=0 tx=0).stop 2023-11-27T15:19:56.522 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.518+0000 7f5cc424f700 1 -- 172.21.15.177:0/2071803735 shutdown_connections 2023-11-27T15:19:56.522 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.518+0000 7f5cc424f700 1 --2- 172.21.15.177:0/2071803735 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5ca80a11d0 0x7f5ca80a3680 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.523 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.518+0000 7f5cc424f700 1 --2- 172.21.15.177:0/2071803735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5cbc100d70 0x7f5cbc108620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.523 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.518+0000 7f5cc424f700 1 --2- 172.21.15.177:0/2071803735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cbc0ff1e0 0x7f5cbc109a90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.523 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.518+0000 7f5cc424f700 1 --2- 172.21.15.177:0/2071803735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cbc1003e0 0x7f5cbc1080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:56.523 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.518+0000 7f5cc424f700 1 -- 172.21.15.177:0/2071803735 >> 172.21.15.177:0/2071803735 conn(0x7f5cbc0fa7f0 msgr2=0x7f5cbc103920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:56.523 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.518+0000 7f5cc424f700 1 -- 172.21.15.177:0/2071803735 shutdown_connections 2023-11-27T15:19:56.523 INFO:teuthology.orchestra.run.smithi177.stderr:2023-11-27T15:19:56.518+0000 7f5cc424f700 1 -- 172.21.15.177:0/2071803735 wait complete. 2023-11-27T15:19:56.967 DEBUG:teuthology.orchestra.run.smithi133:mds.a> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.a.service 2023-11-27T15:19:56.973 DEBUG:teuthology.orchestra.run.smithi133:mds.d> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.d.service 2023-11-27T15:19:57.019 DEBUG:teuthology.orchestra.run.smithi133:mds.g> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.g.service 2023-11-27T15:19:57.052 INFO:journalctl@ceph.mds.a.smithi133.stdout:-- Logs begin at Mon 2023-11-27 14:54:33 UTC. -- 2023-11-27T15:19:57.056 DEBUG:teuthology.orchestra.run.smithi133:mds.j> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.j.service 2023-11-27T15:19:57.091 INFO:journalctl@ceph.mds.d.smithi133.stdout:-- Logs begin at Mon 2023-11-27 14:54:33 UTC. -- 2023-11-27T15:19:57.095 DEBUG:teuthology.orchestra.run.smithi174:mds.b> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.b.service 2023-11-27T15:19:57.098 DEBUG:teuthology.orchestra.run.smithi174:mds.e> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.e.service 2023-11-27T15:19:57.129 INFO:journalctl@ceph.mds.g.smithi133.stdout:-- Logs begin at Mon 2023-11-27 14:54:33 UTC. -- 2023-11-27T15:19:57.138 INFO:journalctl@ceph.mds.b.smithi174.stdout:-- Logs begin at Mon 2023-11-27 14:50:40 UTC. -- 2023-11-27T15:19:57.140 DEBUG:teuthology.orchestra.run.smithi174:mds.h> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.h.service 2023-11-27T15:19:57.164 INFO:journalctl@ceph.mds.j.smithi133.stdout:-- Logs begin at Mon 2023-11-27 14:54:33 UTC. -- 2023-11-27T15:19:57.168 INFO:journalctl@ceph.mds.e.smithi174.stdout:-- Logs begin at Mon 2023-11-27 14:50:40 UTC. -- 2023-11-27T15:19:57.169 DEBUG:teuthology.orchestra.run.smithi174:mds.k> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.k.service 2023-11-27T15:19:57.196 INFO:journalctl@ceph.mds.h.smithi174.stdout:-- Logs begin at Mon 2023-11-27 14:50:40 UTC. -- 2023-11-27T15:19:57.197 DEBUG:teuthology.orchestra.run.smithi177:mds.c> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.c.service 2023-11-27T15:19:57.200 DEBUG:teuthology.orchestra.run.smithi177:mds.f> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.f.service 2023-11-27T15:19:57.224 INFO:journalctl@ceph.mds.k.smithi174.stdout:-- Logs begin at Mon 2023-11-27 14:50:40 UTC. -- 2023-11-27T15:19:57.224 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:57 smithi177 ceph-mon[131504]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:19:57.224 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:57.224 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:57.224 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:57.225 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:57.225 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:57.225 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:19:57.225 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:19:57.225 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:57.226 DEBUG:teuthology.orchestra.run.smithi177:mds.i> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.i.service 2023-11-27T15:19:57.230 INFO:journalctl@ceph.mds.c.smithi177.stdout:-- Logs begin at Mon 2023-11-27 14:50:38 UTC. -- 2023-11-27T15:19:57.231 DEBUG:teuthology.orchestra.run.smithi177:mds.l> sudo journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.l.service 2023-11-27T15:19:57.251 INFO:journalctl@ceph.mds.f.smithi177.stdout:-- Logs begin at Mon 2023-11-27 14:50:38 UTC. -- 2023-11-27T15:19:57.252 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-11-27T15:19:57.253 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:19:57.256 INFO:journalctl@ceph.mds.i.smithi177.stdout:-- Logs begin at Mon 2023-11-27 14:50:38 UTC. -- 2023-11-27T15:19:57.279 INFO:journalctl@ceph.mds.l.smithi177.stdout:-- Logs begin at Mon 2023-11-27 14:50:38 UTC. -- 2023-11-27T15:19:57.312 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:57 smithi133 ceph-mon[97860]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:19:57.312 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:57.313 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:57.313 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:57.313 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:57.313 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:57.313 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:19:57.313 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:19:57.313 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:57.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:57 smithi174 ceph-mon[131110]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:19:57.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:57.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:19:57.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:57.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:19:57.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:57.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:19:57.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:19:57.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:57.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.462+0000 7f0f8f1ff700 1 -- 172.21.15.133:0/3149728159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f88101e10 msgr2=0x7f0f881086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:57.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.462+0000 7f0f8f1ff700 1 --2- 172.21.15.133:0/3149728159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f88101e10 0x7f0f881086d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000b0a0 tx=0x7f0f8000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:19:57.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.462+0000 7f0f8f1ff700 1 -- 172.21.15.133:0/3149728159 shutdown_connections 2023-11-27T15:19:57.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.462+0000 7f0f8f1ff700 1 --2- 172.21.15.133:0/3149728159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f88101e10 0x7f0f881086d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:57.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.462+0000 7f0f8f1ff700 1 --2- 172.21.15.133:0/3149728159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f88100290 0x7f0f881006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:57.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.462+0000 7f0f8f1ff700 1 --2- 172.21.15.133:0/3149728159 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f88101480 0x7f0f881018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:57.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.462+0000 7f0f8f1ff700 1 -- 172.21.15.133:0/3149728159 >> 172.21.15.133:0/3149728159 conn(0x7f0f880fb870 msgr2=0x7f0f880fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:57.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.462+0000 7f0f8f1ff700 1 -- 172.21.15.133:0/3149728159 shutdown_connections 2023-11-27T15:19:57.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.462+0000 7f0f8f1ff700 1 -- 172.21.15.133:0/3149728159 wait complete. 2023-11-27T15:19:57.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.463+0000 7f0f8f1ff700 1 Processor -- start 2023-11-27T15:19:57.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.463+0000 7f0f8f1ff700 1 -- start start 2023-11-27T15:19:57.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.463+0000 7f0f8f1ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f88100290 0x7f0f88198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:57.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.463+0000 7f0f8f1ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f88101480 0x7f0f881987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:57.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.463+0000 7f0f8f1ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f88101e10 0x7f0f8819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:57.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.463+0000 7f0f8f1ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8819fe10 con 0x7f0f88100290 2023-11-27T15:19:57.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.463+0000 7f0f8f1ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8819ff50 con 0x7f0f88101e10 2023-11-27T15:19:57.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.463+0000 7f0f8f1ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f881a00e0 con 0x7f0f88101480 2023-11-27T15:19:57.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.464+0000 7f0f8cf9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f88100290 0x7f0f88198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:57.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.464+0000 7f0f87fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f88101480 0x7f0f881987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:57.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.464+0000 7f0f8d79c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f88101e10 0x7f0f8819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:57.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.464+0000 7f0f8cf9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f88100290 0x7f0f88198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53324/0 (socket says 172.21.15.133:53324) 2023-11-27T15:19:57.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.464+0000 7f0f87fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f88101480 0x7f0f881987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:45574/0 (socket says 172.21.15.133:45574) 2023-11-27T15:19:57.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.464+0000 7f0f8cf9b700 1 -- 172.21.15.133:0/254963413 learned_addr learned my addr 172.21.15.133:0/254963413 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:19:57.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.464+0000 7f0f8d79c700 1 -- 172.21.15.133:0/254963413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f88101480 msgr2=0x7f0f881987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:57.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.464+0000 7f0f8d79c700 1 --2- 172.21.15.133:0/254963413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f88101480 0x7f0f881987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:57.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.464+0000 7f0f8d79c700 1 -- 172.21.15.133:0/254963413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f88100290 msgr2=0x7f0f88198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:57.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.464+0000 7f0f8d79c700 1 --2- 172.21.15.133:0/254963413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f88100290 0x7f0f88198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:57.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.464+0000 7f0f8d79c700 1 -- 172.21.15.133:0/254963413 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f80009d00 con 0x7f0f88101e10 2023-11-27T15:19:57.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.464+0000 7f0f8cf9b700 1 --2- 172.21.15.133:0/254963413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f88100290 0x7f0f88198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:19:57.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.464+0000 7f0f8d79c700 1 --2- 172.21.15.133:0/254963413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f88101e10 0x7f0f8819f840 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000b070 tx=0x7f0f80004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:57.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.464+0000 7f0f85ffb700 1 -- 172.21.15.133:0/254963413 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8000e040 con 0x7f0f88101e10 2023-11-27T15:19:57.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.464+0000 7f0f8f1ff700 1 -- 172.21.15.133:0/254963413 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8810bc70 con 0x7f0f88101e10 2023-11-27T15:19:57.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.465+0000 7f0f85ffb700 1 -- 172.21.15.133:0/254963413 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f80004e70 con 0x7f0f88101e10 2023-11-27T15:19:57.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.465+0000 7f0f85ffb700 1 -- 172.21.15.133:0/254963413 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f80004020 con 0x7f0f88101e10 2023-11-27T15:19:57.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.465+0000 7f0f8f1ff700 1 -- 172.21.15.133:0/254963413 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8810bae0 con 0x7f0f88101e10 2023-11-27T15:19:57.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.465+0000 7f0f8f1ff700 1 -- 172.21.15.133:0/254963413 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8810be00 con 0x7f0f88101e10 2023-11-27T15:19:57.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.467+0000 7f0f85ffb700 1 -- 172.21.15.133:0/254963413 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0f8001d690 con 0x7f0f88101e10 2023-11-27T15:19:57.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.469+0000 7f0f85ffb700 1 --2- 172.21.15.133:0/254963413 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0f780a1150 0x7f0f780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:57.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.469+0000 7f0f85ffb700 1 -- 172.21.15.133:0/254963413 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0f800cdeb0 con 0x7f0f88101e10 2023-11-27T15:19:57.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.469+0000 7f0f8cf9b700 1 --2- 172.21.15.133:0/254963413 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0f780a1150 0x7f0f780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:57.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.469+0000 7f0f8cf9b700 1 --2- 172.21.15.133:0/254963413 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0f780a1150 0x7f0f780a3600 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c007fd0 tx=0x7f0f7c007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:57.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.473+0000 7f0f85ffb700 1 -- 172.21.15.133:0/254963413 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0f80097a90 con 0x7f0f88101e10 2023-11-27T15:19:57.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.674+0000 7f0f8f1ff700 1 -- 172.21.15.133:0/254963413 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f881a0be0 con 0x7f0f88101e10 2023-11-27T15:19:57.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.675+0000 7f0f85ffb700 1 -- 172.21.15.133:0/254963413 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18458 (secure 0 0 0) 0x7f0f8009b0b0 con 0x7f0f88101e10 2023-11-27T15:19:57.679 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:19:57.679 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":80,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:19:49.094979+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:19:57.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.677+0000 7f0f8f1ff700 1 -- 172.21.15.133:0/254963413 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0f780a1150 msgr2=0x7f0f780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:57.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.677+0000 7f0f8f1ff700 1 --2- 172.21.15.133:0/254963413 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0f780a1150 0x7f0f780a3600 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c007fd0 tx=0x7f0f7c007480 comp rx=0 tx=0).stop 2023-11-27T15:19:57.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.677+0000 7f0f8f1ff700 1 -- 172.21.15.133:0/254963413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f88101e10 msgr2=0x7f0f8819f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:57.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.677+0000 7f0f8f1ff700 1 --2- 172.21.15.133:0/254963413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f88101e10 0x7f0f8819f840 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000b070 tx=0x7f0f80004c80 comp rx=0 tx=0).stop 2023-11-27T15:19:57.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.677+0000 7f0f8f1ff700 1 -- 172.21.15.133:0/254963413 shutdown_connections 2023-11-27T15:19:57.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.677+0000 7f0f8f1ff700 1 --2- 172.21.15.133:0/254963413 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0f780a1150 0x7f0f780a3600 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:57.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.677+0000 7f0f8f1ff700 1 --2- 172.21.15.133:0/254963413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f88101e10 0x7f0f8819f840 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:57.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.677+0000 7f0f8f1ff700 1 --2- 172.21.15.133:0/254963413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f88100290 0x7f0f88198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:57.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.677+0000 7f0f8f1ff700 1 --2- 172.21.15.133:0/254963413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f88101480 0x7f0f881987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:57.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.677+0000 7f0f8f1ff700 1 -- 172.21.15.133:0/254963413 >> 172.21.15.133:0/254963413 conn(0x7f0f880fb870 msgr2=0x7f0f880fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:57.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.678+0000 7f0f8f1ff700 1 -- 172.21.15.133:0/254963413 shutdown_connections 2023-11-27T15:19:57.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.678+0000 7f0f8f1ff700 1 -- 172.21.15.133:0/254963413 wait complete. 2023-11-27T15:19:57.703 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:16:20.216150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-27T15:19:57.703 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:19:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.899+0000 7f54eabff700 1 -- 172.21.15.133:0/516821929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54e41002c0 msgr2=0x7f54e41006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.899+0000 7f54eabff700 1 --2- 172.21.15.133:0/516821929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54e41002c0 0x7f54e41006d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f54d80098f0 tx=0x7f54d8009c00 comp rx=0 tx=0).stop 2023-11-27T15:19:57.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.899+0000 7f54eabff700 1 -- 172.21.15.133:0/516821929 shutdown_connections 2023-11-27T15:19:57.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.899+0000 7f54eabff700 1 --2- 172.21.15.133:0/516821929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54e4101e40 0x7f54e4108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:57.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.899+0000 7f54eabff700 1 --2- 172.21.15.133:0/516821929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54e41002c0 0x7f54e41006d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:57.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.899+0000 7f54eabff700 1 --2- 172.21.15.133:0/516821929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54e41014b0 0x7f54e4101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:57.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.899+0000 7f54eabff700 1 -- 172.21.15.133:0/516821929 >> 172.21.15.133:0/516821929 conn(0x7f54e40fb8a0 msgr2=0x7f54e40fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:57.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.899+0000 7f54eabff700 1 -- 172.21.15.133:0/516821929 shutdown_connections 2023-11-27T15:19:57.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.899+0000 7f54eabff700 1 -- 172.21.15.133:0/516821929 wait complete. 2023-11-27T15:19:57.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.900+0000 7f54eabff700 1 Processor -- start 2023-11-27T15:19:57.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.900+0000 7f54eabff700 1 -- start start 2023-11-27T15:19:57.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.901+0000 7f54eabff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54e41002c0 0x7f54e41982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:57.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.901+0000 7f54eabff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54e41014b0 0x7f54e4198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:57.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.901+0000 7f54eabff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54e4101e40 0x7f54e419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:57.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.901+0000 7f54eabff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e419fe40 con 0x7f54e41014b0 2023-11-27T15:19:57.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.901+0000 7f54eabff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e419ff80 con 0x7f54e4101e40 2023-11-27T15:19:57.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.901+0000 7f54eabff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e41a0110 con 0x7f54e41002c0 2023-11-27T15:19:57.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.901+0000 7f54e3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54e41014b0 0x7f54e4198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:57.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.901+0000 7f54e3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54e41014b0 0x7f54e4198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53344/0 (socket says 172.21.15.133:53344) 2023-11-27T15:19:57.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.901+0000 7f54e3fff700 1 -- 172.21.15.133:0/3582631827 learned_addr learned my addr 172.21.15.133:0/3582631827 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:19:57.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.901+0000 7f54e899b700 1 --2- 172.21.15.133:0/3582631827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54e41002c0 0x7f54e41982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:57.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.901+0000 7f54e919c700 1 --2- 172.21.15.133:0/3582631827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54e4101e40 0x7f54e419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:57.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.901+0000 7f54e3fff700 1 -- 172.21.15.133:0/3582631827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54e41002c0 msgr2=0x7f54e41982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:57.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.901+0000 7f54e3fff700 1 --2- 172.21.15.133:0/3582631827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54e41002c0 0x7f54e41982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:57.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.901+0000 7f54e3fff700 1 -- 172.21.15.133:0/3582631827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54e4101e40 msgr2=0x7f54e419f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:57.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.901+0000 7f54e3fff700 1 --2- 172.21.15.133:0/3582631827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54e4101e40 0x7f54e419f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:57.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.901+0000 7f54e3fff700 1 -- 172.21.15.133:0/3582631827 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d8009520 con 0x7f54e41014b0 2023-11-27T15:19:57.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.901+0000 7f54e3fff700 1 --2- 172.21.15.133:0/3582631827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54e41014b0 0x7f54e4198800 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f54d000e9c0 tx=0x7f54d000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:57.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.902+0000 7f54e1ffb700 1 -- 172.21.15.133:0/3582631827 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d000cbd0 con 0x7f54e41014b0 2023-11-27T15:19:57.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.902+0000 7f54e1ffb700 1 -- 172.21.15.133:0/3582631827 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54d000eeb0 con 0x7f54e41014b0 2023-11-27T15:19:57.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.902+0000 7f54eabff700 1 -- 172.21.15.133:0/3582631827 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e410bc90 con 0x7f54e41014b0 2023-11-27T15:19:57.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.902+0000 7f54e1ffb700 1 -- 172.21.15.133:0/3582631827 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d0005580 con 0x7f54e41014b0 2023-11-27T15:19:57.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.902+0000 7f54eabff700 1 -- 172.21.15.133:0/3582631827 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e410bb00 con 0x7f54e41014b0 2023-11-27T15:19:57.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.903+0000 7f54eabff700 1 -- 172.21.15.133:0/3582631827 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e41929f0 con 0x7f54e41014b0 2023-11-27T15:19:57.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.905+0000 7f54e1ffb700 1 -- 172.21.15.133:0/3582631827 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f54d0004750 con 0x7f54e41014b0 2023-11-27T15:19:57.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.906+0000 7f54e1ffb700 1 --2- 172.21.15.133:0/3582631827 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f54d40a1150 0x7f54d40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:57.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.906+0000 7f54e1ffb700 1 -- 172.21.15.133:0/3582631827 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f54d0014070 con 0x7f54e41014b0 2023-11-27T15:19:57.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.906+0000 7f54e899b700 1 --2- 172.21.15.133:0/3582631827 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f54d40a1150 0x7f54d40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:57.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.907+0000 7f54e899b700 1 --2- 172.21.15.133:0/3582631827 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f54d40a1150 0x7f54d40a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009fd0 tx=0x7f54d8019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:57.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:57.910+0000 7f54e1ffb700 1 -- 172.21.15.133:0/3582631827 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f54d0096440 con 0x7f54e41014b0 2023-11-27T15:19:58.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.112+0000 7f54eabff700 1 -- 172.21.15.133:0/3582631827 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f54e4061cc0 con 0x7f54e41014b0 2023-11-27T15:19:58.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.113+0000 7f54e1ffb700 1 -- 172.21.15.133:0/3582631827 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f54d0099a60 con 0x7f54e41014b0 2023-11-27T15:19:58.117 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-11-27T15:19:58.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.115+0000 7f54eabff700 1 -- 172.21.15.133:0/3582631827 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f54d40a1150 msgr2=0x7f54d40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:58.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.115+0000 7f54eabff700 1 --2- 172.21.15.133:0/3582631827 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f54d40a1150 0x7f54d40a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009fd0 tx=0x7f54d8019040 comp rx=0 tx=0).stop 2023-11-27T15:19:58.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.115+0000 7f54eabff700 1 -- 172.21.15.133:0/3582631827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54e41014b0 msgr2=0x7f54e4198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:58.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.115+0000 7f54eabff700 1 --2- 172.21.15.133:0/3582631827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54e41014b0 0x7f54e4198800 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f54d000e9c0 tx=0x7f54d000ecd0 comp rx=0 tx=0).stop 2023-11-27T15:19:58.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.116+0000 7f54eabff700 1 -- 172.21.15.133:0/3582631827 shutdown_connections 2023-11-27T15:19:58.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.116+0000 7f54eabff700 1 --2- 172.21.15.133:0/3582631827 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f54d40a1150 0x7f54d40a3600 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:58.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.116+0000 7f54eabff700 1 --2- 172.21.15.133:0/3582631827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54e4101e40 0x7f54e419f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:58.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.116+0000 7f54eabff700 1 --2- 172.21.15.133:0/3582631827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f54e41002c0 0x7f54e41982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:58.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.116+0000 7f54eabff700 1 --2- 172.21.15.133:0/3582631827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54e41014b0 0x7f54e4198800 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:58.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.116+0000 7f54eabff700 1 -- 172.21.15.133:0/3582631827 >> 172.21.15.133:0/3582631827 conn(0x7f54e40fb8a0 msgr2=0x7f54e4106df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:58.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.116+0000 7f54eabff700 1 -- 172.21.15.133:0/3582631827 shutdown_connections 2023-11-27T15:19:58.120 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.116+0000 7f54eabff700 1 -- 172.21.15.133:0/3582631827 wait complete. 2023-11-27T15:19:58.140 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:19:58.198 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:58 smithi177 ceph-mon[131504]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi174=b;smithi174=e;smithi174=h;smithi174=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:19:58.199 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:58 smithi177 ceph-mon[131504]: Saving service mds.all spec with placement smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi174=b;smithi174=e;smithi174=h;smithi174=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l;count:12 2023-11-27T15:19:58.199 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:58 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:19:58.199 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:58 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:19:58.199 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:58 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:19:58.199 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:58 smithi177 ceph-mon[131504]: Service mds.all has no related daemon already placed. 2023-11-27T15:19:58.199 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:58 smithi177 ceph-mon[131504]: Deploying daemon mds.l on smithi177 2023-11-27T15:19:58.199 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:58 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/254963413' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:19:58.199 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:58 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3582631827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:19:58.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.332+0000 7fc889fff700 1 -- 172.21.15.133:0/3633706053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8841014a0 msgr2=0x7fc8841018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:58.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.332+0000 7fc889fff700 1 --2- 172.21.15.133:0/3633706053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8841014a0 0x7fc8841018f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc878009990 tx=0x7fc878009ca0 comp rx=0 tx=0).stop 2023-11-27T15:19:58.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.332+0000 7fc889fff700 1 -- 172.21.15.133:0/3633706053 shutdown_connections 2023-11-27T15:19:58.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.332+0000 7fc889fff700 1 --2- 172.21.15.133:0/3633706053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc884101e30 0x7fc8841086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:58.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.332+0000 7fc889fff700 1 --2- 172.21.15.133:0/3633706053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8841002b0 0x7fc8841006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:58.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.332+0000 7fc889fff700 1 --2- 172.21.15.133:0/3633706053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8841014a0 0x7fc8841018f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:58.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.332+0000 7fc889fff700 1 -- 172.21.15.133:0/3633706053 >> 172.21.15.133:0/3633706053 conn(0x7fc8840fb890 msgr2=0x7fc8840fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:58.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.332+0000 7fc889fff700 1 -- 172.21.15.133:0/3633706053 shutdown_connections 2023-11-27T15:19:58.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.332+0000 7fc889fff700 1 -- 172.21.15.133:0/3633706053 wait complete. 2023-11-27T15:19:58.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.333+0000 7fc889fff700 1 Processor -- start 2023-11-27T15:19:58.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.334+0000 7fc889fff700 1 -- start start 2023-11-27T15:19:58.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.334+0000 7fc889fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8841002b0 0x7fc884195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:58.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.334+0000 7fc889fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8841014a0 0x7fc884196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:58.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.334+0000 7fc889fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc884101e30 0x7fc88419d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:58.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.334+0000 7fc889fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88419da90 con 0x7fc8841002b0 2023-11-27T15:19:58.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.334+0000 7fc889fff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88419dbd0 con 0x7fc8841014a0 2023-11-27T15:19:58.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.334+0000 7fc889fff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88419dd60 con 0x7fc884101e30 2023-11-27T15:19:58.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.334+0000 7fc8837fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8841002b0 0x7fc884195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:58.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.334+0000 7fc883fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc884101e30 0x7fc88419d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:58.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.334+0000 7fc8837fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8841002b0 0x7fc884195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53352/0 (socket says 172.21.15.133:53352) 2023-11-27T15:19:58.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.334+0000 7fc883fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc884101e30 0x7fc88419d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:45598/0 (socket says 172.21.15.133:45598) 2023-11-27T15:19:58.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.334+0000 7fc8837fe700 1 -- 172.21.15.133:0/2921937075 learned_addr learned my addr 172.21.15.133:0/2921937075 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:19:58.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.334+0000 7fc882ffd700 1 --2- 172.21.15.133:0/2921937075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8841014a0 0x7fc884196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:58.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.335+0000 7fc8837fe700 1 -- 172.21.15.133:0/2921937075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc884101e30 msgr2=0x7fc88419d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:58.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.335+0000 7fc8837fe700 1 --2- 172.21.15.133:0/2921937075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc884101e30 0x7fc88419d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:58.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.335+0000 7fc8837fe700 1 -- 172.21.15.133:0/2921937075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8841014a0 msgr2=0x7fc884196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:58.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.335+0000 7fc8837fe700 1 --2- 172.21.15.133:0/2921937075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8841014a0 0x7fc884196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:58.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.335+0000 7fc8837fe700 1 -- 172.21.15.133:0/2921937075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8780095f0 con 0x7fc8841002b0 2023-11-27T15:19:58.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.335+0000 7fc8837fe700 1 --2- 172.21.15.133:0/2921937075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8841002b0 0x7fc884195f10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc87400c990 tx=0x7fc87400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:58.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.335+0000 7fc880ff9700 1 -- 172.21.15.133:0/2921937075 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874007940 con 0x7fc8841002b0 2023-11-27T15:19:58.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.335+0000 7fc889fff700 1 -- 172.21.15.133:0/2921937075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88410bc60 con 0x7fc8841002b0 2023-11-27T15:19:58.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.335+0000 7fc880ff9700 1 -- 172.21.15.133:0/2921937075 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc87400f450 con 0x7fc8841002b0 2023-11-27T15:19:58.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.335+0000 7fc880ff9700 1 -- 172.21.15.133:0/2921937075 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874018660 con 0x7fc8841002b0 2023-11-27T15:19:58.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.335+0000 7fc889fff700 1 -- 172.21.15.133:0/2921937075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88410bad0 con 0x7fc8841002b0 2023-11-27T15:19:58.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.336+0000 7fc889fff700 1 -- 172.21.15.133:0/2921937075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc88419e3c0 con 0x7fc8841002b0 2023-11-27T15:19:58.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.338+0000 7fc880ff9700 1 -- 172.21.15.133:0/2921937075 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fc87400f8c0 con 0x7fc8841002b0 2023-11-27T15:19:58.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.339+0000 7fc880ff9700 1 --2- 172.21.15.133:0/2921937075 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc86c0a1150 0x7fc86c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:19:58.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.340+0000 7fc880ff9700 1 -- 172.21.15.133:0/2921937075 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc8740cc400 con 0x7fc8841002b0 2023-11-27T15:19:58.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.340+0000 7fc882ffd700 1 --2- 172.21.15.133:0/2921937075 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc86c0a1150 0x7fc86c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:19:58.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.340+0000 7fc882ffd700 1 --2- 172.21.15.133:0/2921937075 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc86c0a1150 0x7fc86c0a3600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc87800b2e0 tx=0x7fc878005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:19:58.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.343+0000 7fc880ff9700 1 -- 172.21.15.133:0/2921937075 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc874095f50 con 0x7fc8841002b0 2023-11-27T15:19:58.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:58 smithi174 ceph-mon[131110]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi174=b;smithi174=e;smithi174=h;smithi174=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:19:58.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:58 smithi174 ceph-mon[131110]: Saving service mds.all spec with placement smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi174=b;smithi174=e;smithi174=h;smithi174=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l;count:12 2023-11-27T15:19:58.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:58 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:19:58.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:58 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:19:58.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:58 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:19:58.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:58 smithi174 ceph-mon[131110]: Service mds.all has no related daemon already placed. 2023-11-27T15:19:58.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:58 smithi174 ceph-mon[131110]: Deploying daemon mds.l on smithi177 2023-11-27T15:19:58.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:58 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/254963413' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:19:58.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:58 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3582631827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:19:58.506 INFO:journalctl@ceph.mds.l.smithi177.stdout:Nov 27 15:19:58 smithi177 systemd[1]: Starting Ceph mds.l for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:19:58.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:58 smithi133 ceph-mon[97860]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi174=b;smithi174=e;smithi174=h;smithi174=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:19:58.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:58 smithi133 ceph-mon[97860]: Saving service mds.all spec with placement smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi174=b;smithi174=e;smithi174=h;smithi174=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l;count:12 2023-11-27T15:19:58.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:58 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:19:58.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:58 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:19:58.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:58 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:19:58.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:58 smithi133 ceph-mon[97860]: Service mds.all has no related daemon already placed. 2023-11-27T15:19:58.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:58 smithi133 ceph-mon[97860]: Deploying daemon mds.l on smithi177 2023-11-27T15:19:58.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:58 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/254963413' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:19:58.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:58 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3582631827' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:19:58.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.586+0000 7fc889fff700 1 -- 172.21.15.133:0/2921937075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc88419ecc0 con 0x7fc8841002b0 2023-11-27T15:19:58.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.586+0000 7fc880ff9700 1 -- 172.21.15.133:0/2921937075 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7fc874099570 con 0x7fc8841002b0 2023-11-27T15:19:58.591 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:19:58.591 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-11-27T15:19:58.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.589+0000 7fc889fff700 1 -- 172.21.15.133:0/2921937075 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc86c0a1150 msgr2=0x7fc86c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:58.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.589+0000 7fc889fff700 1 --2- 172.21.15.133:0/2921937075 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc86c0a1150 0x7fc86c0a3600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc87800b2e0 tx=0x7fc878005fb0 comp rx=0 tx=0).stop 2023-11-27T15:19:58.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.589+0000 7fc889fff700 1 -- 172.21.15.133:0/2921937075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8841002b0 msgr2=0x7fc884195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:19:58.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.589+0000 7fc889fff700 1 --2- 172.21.15.133:0/2921937075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8841002b0 0x7fc884195f10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc87400c990 tx=0x7fc87400cca0 comp rx=0 tx=0).stop 2023-11-27T15:19:58.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.589+0000 7fc889fff700 1 -- 172.21.15.133:0/2921937075 shutdown_connections 2023-11-27T15:19:58.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.589+0000 7fc889fff700 1 --2- 172.21.15.133:0/2921937075 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc86c0a1150 0x7fc86c0a3600 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:58.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.589+0000 7fc889fff700 1 --2- 172.21.15.133:0/2921937075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc884101e30 0x7fc88419d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:58.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.589+0000 7fc889fff700 1 --2- 172.21.15.133:0/2921937075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8841002b0 0x7fc884195f10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:58.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.589+0000 7fc889fff700 1 --2- 172.21.15.133:0/2921937075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8841014a0 0x7fc884196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:19:58.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.589+0000 7fc889fff700 1 -- 172.21.15.133:0/2921937075 >> 172.21.15.133:0/2921937075 conn(0x7fc8840fb890 msgr2=0x7fc8840fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:19:58.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.589+0000 7fc889fff700 1 -- 172.21.15.133:0/2921937075 shutdown_connections 2023-11-27T15:19:58.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:19:58.589+0000 7fc889fff700 1 -- 172.21.15.133:0/2921937075 wait complete. 2023-11-27T15:19:58.595 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 1 2023-11-27T15:19:58.778 INFO:journalctl@ceph.mds.l.smithi177.stdout:Nov 27 15:19:58 smithi177 podman[156538]: 2023-11-27T15:19:58.778 INFO:journalctl@ceph.mds.l.smithi177.stdout:Nov 27 15:19:58 smithi177 ceph-mds[156578]: -- 172.21.15.177:0/2419714600 <== mon.1 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56128c559520 con 0x56128c52e800 2023-11-27T15:19:58.778 INFO:journalctl@ceph.mds.l.smithi177.stdout:Nov 27 15:19:58 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-l[156554]: starting mds.l at 2023-11-27T15:19:59.088 INFO:journalctl@ceph.mds.l.smithi177.stdout:Nov 27 15:19:58 smithi177 bash[156538]: a96fbb654cb8277e2c117f86678266c128e1ebabba9f3f2ea06eae0f55de3f68 2023-11-27T15:19:59.088 INFO:journalctl@ceph.mds.l.smithi177.stdout:Nov 27 15:19:58 smithi177 systemd[1]: Started Ceph mds.l for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:19:59.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:59 smithi174 ceph-mon[131110]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:19:59.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:59 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2921937075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:19:59.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:59 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.177:6832/1329176662,v1:172.21.15.177:6833/1329176662] up:boot 2023-11-27T15:19:59.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:59 smithi174 ceph-mon[131110]: fsmap 1 up:standby 2023-11-27T15:19:59.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:59 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-27T15:19:59.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:59 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:59.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:59 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:59.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:59 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:59.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:59 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:19:59.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:59 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:19:59.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:19:59 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:59.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:59 smithi133 ceph-mon[97860]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:19:59.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:59 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2921937075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:19:59.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:59 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.177:6832/1329176662,v1:172.21.15.177:6833/1329176662] up:boot 2023-11-27T15:19:59.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:59 smithi133 ceph-mon[97860]: fsmap 1 up:standby 2023-11-27T15:19:59.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:59 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-27T15:19:59.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:59 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:59.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:59 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:59.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:59 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:59.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:59 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:19:59.530 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:59 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:19:59.530 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:19:59 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:19:59.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:59 smithi177 ceph-mon[131504]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:19:59.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:59 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2921937075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:19:59.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:59 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.177:6832/1329176662,v1:172.21.15.177:6833/1329176662] up:boot 2023-11-27T15:19:59.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:59 smithi177 ceph-mon[131504]: fsmap 1 up:standby 2023-11-27T15:19:59.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:59 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-11-27T15:19:59.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:59 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:59.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:59 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:59.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:59 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:19:59.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:59 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:19:59.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:59 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:19:59.590 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:19:59 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:00.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:00 smithi174 ceph-mon[131110]: Deploying daemon mds.k on smithi174 2023-11-27T15:20:00.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:00 smithi174 ceph-mon[131110]: overall HEALTH_OK 2023-11-27T15:20:00.404 INFO:journalctl@ceph.mds.k.smithi174.stdout:Nov 27 15:20:00 smithi174 systemd[1]: Starting Ceph mds.k for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:20:00.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:00 smithi133 ceph-mon[97860]: Deploying daemon mds.k on smithi174 2023-11-27T15:20:00.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:00 smithi133 ceph-mon[97860]: overall HEALTH_OK 2023-11-27T15:20:00.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:00 smithi177 ceph-mon[131504]: Deploying daemon mds.k on smithi174 2023-11-27T15:20:00.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:00 smithi177 ceph-mon[131504]: overall HEALTH_OK 2023-11-27T15:20:00.613 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:20:00.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.824+0000 7f2bd79ff700 1 -- 172.21.15.133:0/1643962123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd0101e30 msgr2=0x7f2bd01086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:00.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.824+0000 7f2bd79ff700 1 --2- 172.21.15.133:0/1643962123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd0101e30 0x7f2bd01086f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009f90 tx=0x7f2bc800b330 comp rx=0 tx=0).stop 2023-11-27T15:20:00.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.824+0000 7f2bd79ff700 1 -- 172.21.15.133:0/1643962123 shutdown_connections 2023-11-27T15:20:00.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.824+0000 7f2bd79ff700 1 --2- 172.21.15.133:0/1643962123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd0101e30 0x7f2bd01086f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:00.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.824+0000 7f2bd79ff700 1 --2- 172.21.15.133:0/1643962123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd01002b0 0x7f2bd01006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:00.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.824+0000 7f2bd79ff700 1 --2- 172.21.15.133:0/1643962123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bd01014a0 0x7f2bd01018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:00.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.824+0000 7f2bd79ff700 1 -- 172.21.15.133:0/1643962123 >> 172.21.15.133:0/1643962123 conn(0x7f2bd00fb890 msgr2=0x7f2bd00fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:00.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.824+0000 7f2bd79ff700 1 -- 172.21.15.133:0/1643962123 shutdown_connections 2023-11-27T15:20:00.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.824+0000 7f2bd79ff700 1 -- 172.21.15.133:0/1643962123 wait complete. 2023-11-27T15:20:00.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.825+0000 7f2bd79ff700 1 Processor -- start 2023-11-27T15:20:00.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.825+0000 7f2bd79ff700 1 -- start start 2023-11-27T15:20:00.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.825+0000 7f2bd79ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bd01002b0 0x7f2bd01982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:00.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.825+0000 7f2bd79ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd01014a0 0x7f2bd01987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:00.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.825+0000 7f2bd79ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd0101e30 0x7f2bd019f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:00.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.825+0000 7f2bd79ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd019fe30 con 0x7f2bd01002b0 2023-11-27T15:20:00.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.825+0000 7f2bd79ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd019ff70 con 0x7f2bd01014a0 2023-11-27T15:20:00.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.826+0000 7f2bd79ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd01a0100 con 0x7f2bd0101e30 2023-11-27T15:20:00.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.826+0000 7f2bd579b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bd01002b0 0x7f2bd01982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:00.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.826+0000 7f2bd4f9a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd01014a0 0x7f2bd01987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:00.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.826+0000 7f2bd4f9a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd01014a0 0x7f2bd01987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:43044/0 (socket says 172.21.15.133:43044) 2023-11-27T15:20:00.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.826+0000 7f2bd4f9a700 1 -- 172.21.15.133:0/626562751 learned_addr learned my addr 172.21.15.133:0/626562751 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:00.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.826+0000 7f2bd579b700 1 --2- 172.21.15.133:0/626562751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bd01002b0 0x7f2bd01982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53366/0 (socket says 172.21.15.133:53366) 2023-11-27T15:20:00.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.826+0000 7f2bd5f9c700 1 --2- 172.21.15.133:0/626562751 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd0101e30 0x7f2bd019f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:00.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.826+0000 7f2bd4f9a700 1 -- 172.21.15.133:0/626562751 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd0101e30 msgr2=0x7f2bd019f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:00.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.826+0000 7f2bd4f9a700 1 --2- 172.21.15.133:0/626562751 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd0101e30 0x7f2bd019f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:00.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.826+0000 7f2bd4f9a700 1 -- 172.21.15.133:0/626562751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bd01002b0 msgr2=0x7f2bd01982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:00.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.826+0000 7f2bd4f9a700 1 --2- 172.21.15.133:0/626562751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bd01002b0 0x7f2bd01982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:00.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.826+0000 7f2bd4f9a700 1 -- 172.21.15.133:0/626562751 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc00095f0 con 0x7f2bd01014a0 2023-11-27T15:20:00.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.826+0000 7f2bd579b700 1 --2- 172.21.15.133:0/626562751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bd01002b0 0x7f2bd01982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:00.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.826+0000 7f2bd4f9a700 1 --2- 172.21.15.133:0/626562751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd01014a0 0x7f2bd01987f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000ec10 tx=0x7f2bc000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:00.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.826+0000 7f2bc67fc700 1 -- 172.21.15.133:0/626562751 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc000cc90 con 0x7f2bd01014a0 2023-11-27T15:20:00.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.827+0000 7f2bc67fc700 1 -- 172.21.15.133:0/626562751 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bc00044e0 con 0x7f2bd01014a0 2023-11-27T15:20:00.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.827+0000 7f2bd79ff700 1 -- 172.21.15.133:0/626562751 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc8009c40 con 0x7f2bd01014a0 2023-11-27T15:20:00.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.827+0000 7f2bc67fc700 1 -- 172.21.15.133:0/626562751 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0010430 con 0x7f2bd01014a0 2023-11-27T15:20:00.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.827+0000 7f2bd79ff700 1 -- 172.21.15.133:0/626562751 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd010bc80 con 0x7f2bd01014a0 2023-11-27T15:20:00.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.827+0000 7f2bd79ff700 1 -- 172.21.15.133:0/626562751 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd0067a60 con 0x7f2bd01014a0 2023-11-27T15:20:00.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.829+0000 7f2bc67fc700 1 -- 172.21.15.133:0/626562751 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2bc0003fb0 con 0x7f2bd01014a0 2023-11-27T15:20:00.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.831+0000 7f2bc67fc700 1 --2- 172.21.15.133:0/626562751 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2bbc0a1150 0x7f2bbc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:00.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.831+0000 7f2bc67fc700 1 -- 172.21.15.133:0/626562751 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2bc0014070 con 0x7f2bd01014a0 2023-11-27T15:20:00.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.831+0000 7f2bd579b700 1 --2- 172.21.15.133:0/626562751 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2bbc0a1150 0x7f2bbc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:00.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.831+0000 7f2bd579b700 1 --2- 172.21.15.133:0/626562751 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2bbc0a1150 0x7f2bbc0a3600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0101300 tx=0x7f2bcc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:00.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:00.833+0000 7f2bc67fc700 1 -- 172.21.15.133:0/626562751 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2bc0096be0 con 0x7f2bd01014a0 2023-11-27T15:20:00.866 INFO:journalctl@ceph.mds.k.smithi174.stdout:Nov 27 15:20:00 smithi174 podman[155389]: 2023-11-27T15:20:00.866 INFO:journalctl@ceph.mds.k.smithi174.stdout:Nov 27 15:20:00 smithi174 bash[155389]: cc229957c0029a079db3352f1f7df12b0092e0c3665d10bbb5dd1cf1c5c6051e 2023-11-27T15:20:01.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:01.091+0000 7f2bd79ff700 1 -- 172.21.15.133:0/626562751 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bd01a0c70 con 0x7f2bd01014a0 2023-11-27T15:20:01.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:01.091+0000 7f2bc67fc700 1 -- 172.21.15.133:0/626562751 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7f2bc009a200 con 0x7f2bd01014a0 2023-11-27T15:20:01.096 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:01.096 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-11-27T15:20:01.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:01.094+0000 7f2bd79ff700 1 -- 172.21.15.133:0/626562751 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2bbc0a1150 msgr2=0x7f2bbc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:01.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:01.094+0000 7f2bd79ff700 1 --2- 172.21.15.133:0/626562751 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2bbc0a1150 0x7f2bbc0a3600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0101300 tx=0x7f2bcc009260 comp rx=0 tx=0).stop 2023-11-27T15:20:01.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:01.094+0000 7f2bd79ff700 1 -- 172.21.15.133:0/626562751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd01014a0 msgr2=0x7f2bd01987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:01.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:01.094+0000 7f2bd79ff700 1 --2- 172.21.15.133:0/626562751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd01014a0 0x7f2bd01987f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000ec10 tx=0x7f2bc000ef20 comp rx=0 tx=0).stop 2023-11-27T15:20:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:01.094+0000 7f2bd79ff700 1 -- 172.21.15.133:0/626562751 shutdown_connections 2023-11-27T15:20:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:01.094+0000 7f2bd79ff700 1 --2- 172.21.15.133:0/626562751 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2bbc0a1150 0x7f2bbc0a3600 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:01.094+0000 7f2bd79ff700 1 --2- 172.21.15.133:0/626562751 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bd0101e30 0x7f2bd019f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:01.094+0000 7f2bd79ff700 1 --2- 172.21.15.133:0/626562751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bd01002b0 0x7f2bd01982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:01.094+0000 7f2bd79ff700 1 --2- 172.21.15.133:0/626562751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd01014a0 0x7f2bd01987f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:01.094+0000 7f2bd79ff700 1 -- 172.21.15.133:0/626562751 >> 172.21.15.133:0/626562751 conn(0x7f2bd00fb890 msgr2=0x7f2bd0106d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:01.094+0000 7f2bd79ff700 1 -- 172.21.15.133:0/626562751 shutdown_connections 2023-11-27T15:20:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:01.094+0000 7f2bd79ff700 1 -- 172.21.15.133:0/626562751 wait complete. 2023-11-27T15:20:01.099 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 2 2023-11-27T15:20:01.137 INFO:journalctl@ceph.mds.k.smithi174.stdout:Nov 27 15:20:00 smithi174 ceph-mds[155429]: -- 172.21.15.174:0/3282386141 <== mon.2 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a0de329520 con 0x55a0de2ff000 2023-11-27T15:20:01.137 INFO:journalctl@ceph.mds.k.smithi174.stdout:Nov 27 15:20:00 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-k[155405]: starting mds.k at 2023-11-27T15:20:01.137 INFO:journalctl@ceph.mds.k.smithi174.stdout:Nov 27 15:20:00 smithi174 systemd[1]: Started Ceph mds.k for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:20:01.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:01 smithi174 ceph-mon[131110]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:01.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:01.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:01.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:01.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:01.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:01.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:01.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:01 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/626562751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:01.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:01 smithi133 ceph-mon[97860]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:01.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:01.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:01.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:01.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:01.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:01.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:01.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:01 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/626562751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:01.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:01 smithi177 ceph-mon[131504]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:01.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:01.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:01.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:01.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:01.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:01.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:01.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:01 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/626562751' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:02.380 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:02 smithi133 ceph-mon[97860]: Deploying daemon mds.j on smithi133 2023-11-27T15:20:02.380 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:02 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.174:6832/2946349291,v1:172.21.15.174:6833/2946349291] up:boot 2023-11-27T15:20:02.381 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:02 smithi133 ceph-mon[97860]: fsmap 2 up:standby 2023-11-27T15:20:02.381 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-27T15:20:02.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:02 smithi174 ceph-mon[131110]: Deploying daemon mds.j on smithi133 2023-11-27T15:20:02.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:02 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.174:6832/2946349291,v1:172.21.15.174:6833/2946349291] up:boot 2023-11-27T15:20:02.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:02 smithi174 ceph-mon[131110]: fsmap 2 up:standby 2023-11-27T15:20:02.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-27T15:20:02.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:02 smithi177 ceph-mon[131504]: Deploying daemon mds.j on smithi133 2023-11-27T15:20:02.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:02 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.174:6832/2946349291,v1:172.21.15.174:6833/2946349291] up:boot 2023-11-27T15:20:02.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:02 smithi177 ceph-mon[131504]: fsmap 2 up:standby 2023-11-27T15:20:02.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-11-27T15:20:03.119 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:20:03.263 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:03 smithi133 ceph-mon[97860]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:03.263 INFO:journalctl@ceph.mds.j.smithi133.stdout:Nov 27 15:20:03 smithi133 systemd[1]: Starting Ceph mds.j for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:20:03.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.386+0000 7f2bc77ff700 1 -- 172.21.15.133:0/2886825795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bc0105fd0 msgr2=0x7f2bc01083b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:03.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.386+0000 7f2bc77ff700 1 --2- 172.21.15.133:0/2886825795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bc0105fd0 0x7f2bc01083b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800b0a0 tx=0x7f2bb800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.386+0000 7f2bc77ff700 1 -- 172.21.15.133:0/2886825795 shutdown_connections 2023-11-27T15:20:03.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.386+0000 7f2bc77ff700 1 --2- 172.21.15.133:0/2886825795 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bc0105fd0 0x7f2bc01083b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.386+0000 7f2bc77ff700 1 --2- 172.21.15.133:0/2886825795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bc0100d90 0x7f2bc0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.386+0000 7f2bc77ff700 1 --2- 172.21.15.133:0/2886825795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bc01036b0 0x7f2bc0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.386+0000 7f2bc77ff700 1 -- 172.21.15.133:0/2886825795 >> 172.21.15.133:0/2886825795 conn(0x7f2bc00fa750 msgr2=0x7f2bc00fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:03.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.386+0000 7f2bc77ff700 1 -- 172.21.15.133:0/2886825795 shutdown_connections 2023-11-27T15:20:03.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.386+0000 7f2bc77ff700 1 -- 172.21.15.133:0/2886825795 wait complete. 2023-11-27T15:20:03.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.387+0000 7f2bc77ff700 1 Processor -- start 2023-11-27T15:20:03.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.387+0000 7f2bc77ff700 1 -- start start 2023-11-27T15:20:03.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.388+0000 7f2bc77ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bc0100d90 0x7f2bc01982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:03.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.388+0000 7f2bc77ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bc01036b0 0x7f2bc0198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:03.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.388+0000 7f2bc77ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bc0105fd0 0x7f2bc019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:03.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.388+0000 7f2bc77ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc019fe70 con 0x7f2bc01036b0 2023-11-27T15:20:03.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.388+0000 7f2bc77ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc019ffb0 con 0x7f2bc0100d90 2023-11-27T15:20:03.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.388+0000 7f2bc77ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc01a0140 con 0x7f2bc0105fd0 2023-11-27T15:20:03.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.388+0000 7f2bc4d9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bc01036b0 0x7f2bc0198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:03.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.388+0000 7f2bc4d9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bc01036b0 0x7f2bc0198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35564/0 (socket says 172.21.15.133:35564) 2023-11-27T15:20:03.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.388+0000 7f2bc4d9a700 1 -- 172.21.15.133:0/3474259421 learned_addr learned my addr 172.21.15.133:0/3474259421 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:03.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.388+0000 7f2bc559b700 1 --2- 172.21.15.133:0/3474259421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bc0100d90 0x7f2bc01982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:03.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.388+0000 7f2bc5d9c700 1 --2- 172.21.15.133:0/3474259421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bc0105fd0 0x7f2bc019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:03.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.388+0000 7f2bc4d9a700 1 -- 172.21.15.133:0/3474259421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bc0105fd0 msgr2=0x7f2bc019f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:03.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.388+0000 7f2bc4d9a700 1 --2- 172.21.15.133:0/3474259421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bc0105fd0 0x7f2bc019f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.388+0000 7f2bc4d9a700 1 -- 172.21.15.133:0/3474259421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bc0100d90 msgr2=0x7f2bc01982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:03.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.388+0000 7f2bc4d9a700 1 --2- 172.21.15.133:0/3474259421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bc0100d90 0x7f2bc01982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.388+0000 7f2bc4d9a700 1 -- 172.21.15.133:0/3474259421 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb8009d00 con 0x7f2bc01036b0 2023-11-27T15:20:03.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.388+0000 7f2bc559b700 1 --2- 172.21.15.133:0/3474259421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bc0100d90 0x7f2bc01982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:20:03.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.389+0000 7f2bc4d9a700 1 --2- 172.21.15.133:0/3474259421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bc01036b0 0x7f2bc0198830 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000e9a0 tx=0x7f2bb000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:03.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.389+0000 7f2bb67fc700 1 -- 172.21.15.133:0/3474259421 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0009790 con 0x7f2bc01036b0 2023-11-27T15:20:03.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.389+0000 7f2bb67fc700 1 -- 172.21.15.133:0/3474259421 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bb000cbd0 con 0x7f2bc01036b0 2023-11-27T15:20:03.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.389+0000 7f2bc77ff700 1 -- 172.21.15.133:0/3474259421 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc010bc60 con 0x7f2bc01036b0 2023-11-27T15:20:03.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.389+0000 7f2bb67fc700 1 -- 172.21.15.133:0/3474259421 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0018680 con 0x7f2bc01036b0 2023-11-27T15:20:03.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.389+0000 7f2bc77ff700 1 -- 172.21.15.133:0/3474259421 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc010bad0 con 0x7f2bc01036b0 2023-11-27T15:20:03.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.390+0000 7f2bc77ff700 1 -- 172.21.15.133:0/3474259421 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc0107b40 con 0x7f2bc01036b0 2023-11-27T15:20:03.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.392+0000 7f2bb67fc700 1 -- 172.21.15.133:0/3474259421 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2bb000cd40 con 0x7f2bc01036b0 2023-11-27T15:20:03.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.394+0000 7f2bb67fc700 1 --2- 172.21.15.133:0/3474259421 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2bac0a11d0 0x7f2bac0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:03.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.394+0000 7f2bb67fc700 1 -- 172.21.15.133:0/3474259421 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2bb0014070 con 0x7f2bc01036b0 2023-11-27T15:20:03.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.394+0000 7f2bc559b700 1 --2- 172.21.15.133:0/3474259421 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2bac0a11d0 0x7f2bac0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:03.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.395+0000 7f2bc559b700 1 --2- 172.21.15.133:0/3474259421 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2bac0a11d0 0x7f2bac0a3680 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0068890 tx=0x7f2bbc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:03.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:03 smithi174 ceph-mon[131110]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:03.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.401+0000 7f2bb67fc700 1 -- 172.21.15.133:0/3474259421 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2bb0051730 con 0x7f2bc01036b0 2023-11-27T15:20:03.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:03 smithi177 ceph-mon[131504]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:03.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.672+0000 7f2bc77ff700 1 -- 172.21.15.133:0/3474259421 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bc002cfe0 con 0x7f2bc01036b0 2023-11-27T15:20:03.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.673+0000 7f2bb67fc700 1 -- 172.21.15.133:0/3474259421 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2020 (secure 0 0 0) 0x7f2bb0096230 con 0x7f2bc01036b0 2023-11-27T15:20:03.677 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:03.677 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-27T15:20:03.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.675+0000 7f2bc77ff700 1 -- 172.21.15.133:0/3474259421 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2bac0a11d0 msgr2=0x7f2bac0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:03.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.675+0000 7f2bc77ff700 1 --2- 172.21.15.133:0/3474259421 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2bac0a11d0 0x7f2bac0a3680 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0068890 tx=0x7f2bbc009260 comp rx=0 tx=0).stop 2023-11-27T15:20:03.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.675+0000 7f2bc77ff700 1 -- 172.21.15.133:0/3474259421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bc01036b0 msgr2=0x7f2bc0198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:03.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.675+0000 7f2bc77ff700 1 --2- 172.21.15.133:0/3474259421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bc01036b0 0x7f2bc0198830 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000e9a0 tx=0x7f2bb000ecb0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.675+0000 7f2bc77ff700 1 -- 172.21.15.133:0/3474259421 shutdown_connections 2023-11-27T15:20:03.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.675+0000 7f2bc77ff700 1 --2- 172.21.15.133:0/3474259421 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2bac0a11d0 0x7f2bac0a3680 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0068890 tx=0x7f2bbc009260 comp rx=0 tx=0).stop 2023-11-27T15:20:03.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.675+0000 7f2bc77ff700 1 --2- 172.21.15.133:0/3474259421 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bc0105fd0 0x7f2bc019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.675+0000 7f2bc77ff700 1 --2- 172.21.15.133:0/3474259421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bc0100d90 0x7f2bc01982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.675+0000 7f2bc77ff700 1 --2- 172.21.15.133:0/3474259421 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bc01036b0 0x7f2bc0198830 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:03.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.675+0000 7f2bc77ff700 1 -- 172.21.15.133:0/3474259421 >> 172.21.15.133:0/3474259421 conn(0x7f2bc00fa750 msgr2=0x7f2bc00ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:03.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.675+0000 7f2bc77ff700 1 -- 172.21.15.133:0/3474259421 shutdown_connections 2023-11-27T15:20:03.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:03.675+0000 7f2bc77ff700 1 -- 172.21.15.133:0/3474259421 wait complete. 2023-11-27T15:20:03.680 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 3 2023-11-27T15:20:03.988 INFO:journalctl@ceph.mds.j.smithi133.stdout:Nov 27 15:20:03 smithi133 podman[131041]: 2023-11-27T15:20:04.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:04 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3474259421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:04.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:04 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:04.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:04 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:04.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:04 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:04.278 INFO:journalctl@ceph.mds.j.smithi133.stdout:Nov 27 15:20:03 smithi133 ceph-mds[131083]: -- 172.21.15.133:0/946085973 <== mon.0 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55905d5e7520 con 0x55905d5bec00 2023-11-27T15:20:04.278 INFO:journalctl@ceph.mds.j.smithi133.stdout:Nov 27 15:20:03 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-j[131059]: starting mds.j at 2023-11-27T15:20:04.278 INFO:journalctl@ceph.mds.j.smithi133.stdout:Nov 27 15:20:03 smithi133 bash[131041]: 0ef752b6cdaf31b8c409a69e3f348631450ddc9b08a141be308bc4bf848a7d0a 2023-11-27T15:20:04.278 INFO:journalctl@ceph.mds.j.smithi133.stdout:Nov 27 15:20:04 smithi133 systemd[1]: Started Ceph mds.j for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:20:04.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:04 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3474259421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:04.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:04 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:04.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:04 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:04.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:04 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:04.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:04 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3474259421' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:04.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:04 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:04.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:04 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:04.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:04 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:05.522 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:05 smithi133 ceph-mon[97860]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:05.522 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:05.522 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:05 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.133:6834/1877742522,v1:172.21.15.133:6835/1877742522] up:boot 2023-11-27T15:20:05.522 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:05 smithi133 ceph-mon[97860]: fsmap 3 up:standby 2023-11-27T15:20:05.522 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:05.523 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-27T15:20:05.523 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:05 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:05.523 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:05 smithi133 ceph-mon[97860]: Deploying daemon mds.d on smithi133 2023-11-27T15:20:05.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:05 smithi177 ceph-mon[131504]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:05.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:05.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:05 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.133:6834/1877742522,v1:172.21.15.133:6835/1877742522] up:boot 2023-11-27T15:20:05.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:05 smithi177 ceph-mon[131504]: fsmap 3 up:standby 2023-11-27T15:20:05.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:05.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-27T15:20:05.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:05 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:05.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:05 smithi177 ceph-mon[131504]: Deploying daemon mds.d on smithi133 2023-11-27T15:20:05.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:05 smithi174 ceph-mon[131110]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:05.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:05.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:05 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.133:6834/1877742522,v1:172.21.15.133:6835/1877742522] up:boot 2023-11-27T15:20:05.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:05 smithi174 ceph-mon[131110]: fsmap 3 up:standby 2023-11-27T15:20:05.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:05.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-11-27T15:20:05.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:05 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:05.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:05 smithi174 ceph-mon[131110]: Deploying daemon mds.d on smithi133 2023-11-27T15:20:05.701 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:20:05.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.922+0000 7f350159d700 1 -- 172.21.15.133:0/4220481818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34fc101cf0 msgr2=0x7f34fc108590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:05.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.922+0000 7f350159d700 1 --2- 172.21.15.133:0/4220481818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34fc101cf0 0x7f34fc108590 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f34f400b0a0 tx=0x7f34f400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.922+0000 7f350159d700 1 -- 172.21.15.133:0/4220481818 shutdown_connections 2023-11-27T15:20:05.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.922+0000 7f350159d700 1 --2- 172.21.15.133:0/4220481818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34fc101cf0 0x7f34fc108590 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.922+0000 7f350159d700 1 --2- 172.21.15.133:0/4220481818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34fc1000b0 0x7f34fc1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.922+0000 7f350159d700 1 --2- 172.21.15.133:0/4220481818 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34fc101340 0x7f34fc1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.922+0000 7f350159d700 1 -- 172.21.15.133:0/4220481818 >> 172.21.15.133:0/4220481818 conn(0x7f34fc0fb6d0 msgr2=0x7f34fc0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:05.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.923+0000 7f350159d700 1 -- 172.21.15.133:0/4220481818 shutdown_connections 2023-11-27T15:20:05.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.923+0000 7f350159d700 1 -- 172.21.15.133:0/4220481818 wait complete. 2023-11-27T15:20:05.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.924+0000 7f350159d700 1 Processor -- start 2023-11-27T15:20:05.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.924+0000 7f350159d700 1 -- start start 2023-11-27T15:20:05.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.924+0000 7f350159d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34fc1000b0 0x7f34fc1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.924+0000 7f350159d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34fc101340 0x7f34fc198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.924+0000 7f350159d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34fc101cf0 0x7f34fc19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.924+0000 7f350159d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc19fc50 con 0x7f34fc1000b0 2023-11-27T15:20:05.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.924+0000 7f350159d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc19fd90 con 0x7f34fc101340 2023-11-27T15:20:05.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.924+0000 7f350159d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc19ff20 con 0x7f34fc101cf0 2023-11-27T15:20:05.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.924+0000 7f34fbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34fc1000b0 0x7f34fc1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.924+0000 7f34fbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34fc1000b0 0x7f34fc1980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35608/0 (socket says 172.21.15.133:35608) 2023-11-27T15:20:05.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.924+0000 7f34fbfff700 1 -- 172.21.15.133:0/3428481364 learned_addr learned my addr 172.21.15.133:0/3428481364 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:05.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.924+0000 7f34fb7fe700 1 --2- 172.21.15.133:0/3428481364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34fc101340 0x7f34fc198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.925+0000 7f3500d9c700 1 --2- 172.21.15.133:0/3428481364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34fc101cf0 0x7f34fc19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.925+0000 7f34fb7fe700 1 -- 172.21.15.133:0/3428481364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34fc101cf0 msgr2=0x7f34fc19f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:05.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.925+0000 7f34fb7fe700 1 --2- 172.21.15.133:0/3428481364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34fc101cf0 0x7f34fc19f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.925+0000 7f34fb7fe700 1 -- 172.21.15.133:0/3428481364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34fc1000b0 msgr2=0x7f34fc1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:05.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.925+0000 7f34fb7fe700 1 --2- 172.21.15.133:0/3428481364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34fc1000b0 0x7f34fc1980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:05.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.925+0000 7f34fb7fe700 1 -- 172.21.15.133:0/3428481364 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f4009d00 con 0x7f34fc101340 2023-11-27T15:20:05.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.925+0000 7f34fbfff700 1 --2- 172.21.15.133:0/3428481364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34fc1000b0 0x7f34fc1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:05.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.925+0000 7f34fb7fe700 1 --2- 172.21.15.133:0/3428481364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34fc101340 0x7f34fc198610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f34f000e9f0 tx=0x7f34f000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:05.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.925+0000 7f34f97fa700 1 -- 172.21.15.133:0/3428481364 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f000c9c0 con 0x7f34fc101340 2023-11-27T15:20:05.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.925+0000 7f350159d700 1 -- 172.21.15.133:0/3428481364 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34fc10baa0 con 0x7f34fc101340 2023-11-27T15:20:05.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.925+0000 7f34f97fa700 1 -- 172.21.15.133:0/3428481364 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34f0010430 con 0x7f34fc101340 2023-11-27T15:20:05.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.925+0000 7f34f97fa700 1 -- 172.21.15.133:0/3428481364 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0018680 con 0x7f34fc101340 2023-11-27T15:20:05.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.925+0000 7f350159d700 1 -- 172.21.15.133:0/3428481364 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34fc10b910 con 0x7f34fc101340 2023-11-27T15:20:05.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.926+0000 7f350159d700 1 -- 172.21.15.133:0/3428481364 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34fc1a0610 con 0x7f34fc101340 2023-11-27T15:20:05.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.928+0000 7f34f97fa700 1 -- 172.21.15.133:0/3428481364 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f34f0010860 con 0x7f34fc101340 2023-11-27T15:20:05.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.930+0000 7f34f97fa700 1 --2- 172.21.15.133:0/3428481364 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f34e40a1150 0x7f34e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:05.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.930+0000 7f34f97fa700 1 -- 172.21.15.133:0/3428481364 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f34f0014070 con 0x7f34fc101340 2023-11-27T15:20:05.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.930+0000 7f34fbfff700 1 --2- 172.21.15.133:0/3428481364 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f34e40a1150 0x7f34e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:05.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.930+0000 7f34fbfff700 1 --2- 172.21.15.133:0/3428481364 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f34e40a1150 0x7f34e40a3600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f34ec011550 tx=0x7f34ec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:05.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:05.932+0000 7f34f97fa700 1 -- 172.21.15.133:0/3428481364 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f34f0096130 con 0x7f34fc101340 2023-11-27T15:20:06.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:06.201+0000 7f350159d700 1 -- 172.21.15.133:0/3428481364 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34fc1a0bc0 con 0x7f34fc101340 2023-11-27T15:20:06.206 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:06.202+0000 7f34f97fa700 1 -- 172.21.15.133:0/3428481364 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2780 (secure 0 0 0) 0x7f34f0099750 con 0x7f34fc101340 2023-11-27T15:20:06.206 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:06.207 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[]} 2023-11-27T15:20:06.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:06.206+0000 7f350159d700 1 -- 172.21.15.133:0/3428481364 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f34e40a1150 msgr2=0x7f34e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:06.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:06.206+0000 7f350159d700 1 --2- 172.21.15.133:0/3428481364 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f34e40a1150 0x7f34e40a3600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f34ec011550 tx=0x7f34ec009260 comp rx=0 tx=0).stop 2023-11-27T15:20:06.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:06.206+0000 7f350159d700 1 -- 172.21.15.133:0/3428481364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34fc101340 msgr2=0x7f34fc198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:06.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:06.206+0000 7f350159d700 1 --2- 172.21.15.133:0/3428481364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34fc101340 0x7f34fc198610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f34f000e9f0 tx=0x7f34f000ed00 comp rx=0 tx=0).stop 2023-11-27T15:20:06.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:06.208+0000 7f350159d700 1 -- 172.21.15.133:0/3428481364 shutdown_connections 2023-11-27T15:20:06.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:06.208+0000 7f350159d700 1 --2- 172.21.15.133:0/3428481364 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f34e40a1150 0x7f34e40a3600 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:06.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:06.208+0000 7f350159d700 1 --2- 172.21.15.133:0/3428481364 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34fc101cf0 0x7f34fc19f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:06.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:06.208+0000 7f350159d700 1 --2- 172.21.15.133:0/3428481364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34fc1000b0 0x7f34fc1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:06.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:06.208+0000 7f350159d700 1 --2- 172.21.15.133:0/3428481364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34fc101340 0x7f34fc198610 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:06.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:06.208+0000 7f350159d700 1 -- 172.21.15.133:0/3428481364 >> 172.21.15.133:0/3428481364 conn(0x7f34fc0fb6d0 msgr2=0x7f34fc0fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:06.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:06.208+0000 7f350159d700 1 -- 172.21.15.133:0/3428481364 shutdown_connections 2023-11-27T15:20:06.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:06.208+0000 7f350159d700 1 -- 172.21.15.133:0/3428481364 wait complete. 2023-11-27T15:20:06.214 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 4 2023-11-27T15:20:06.778 INFO:journalctl@ceph.mds.d.smithi133.stdout:Nov 27 15:20:06 smithi133 systemd[1]: Starting Ceph mds.d for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:20:07.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:07 smithi133 ceph-mon[97860]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:07.177 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:07 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3428481364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:07.177 INFO:journalctl@ceph.mds.d.smithi133.stdout:Nov 27 15:20:06 smithi133 podman[131431]: 2023-11-27T15:20:07.527 INFO:journalctl@ceph.mds.d.smithi133.stdout:Nov 27 15:20:07 smithi133 bash[131431]: 5b47e112c91b4f226361749db10ec174db5580048dc09e78639542ede8565112 2023-11-27T15:20:07.527 INFO:journalctl@ceph.mds.d.smithi133.stdout:Nov 27 15:20:07 smithi133 ceph-mds[131471]: -- 172.21.15.133:0/2108367934 <== mon.0 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5576edfd1520 con 0x5576edfa9000 2023-11-27T15:20:07.527 INFO:journalctl@ceph.mds.d.smithi133.stdout:Nov 27 15:20:07 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-d[131447]: starting mds.d at 2023-11-27T15:20:07.528 INFO:journalctl@ceph.mds.d.smithi133.stdout:Nov 27 15:20:07 smithi133 systemd[1]: Started Ceph mds.d for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:20:07.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:07 smithi177 ceph-mon[131504]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:07.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:07 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3428481364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:07.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:07 smithi174 ceph-mon[131110]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:07.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:07 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3428481364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:08.238 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:20:08.333 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:08.333 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:08.333 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:08.333 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:08.333 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:08.333 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:08.333 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:08 smithi174 ceph-mon[131110]: Deploying daemon mds.e on smithi174 2023-11-27T15:20:08.333 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:08 smithi174 ceph-mon[131110]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:08.333 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:08 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.133:6836/2249018853,v1:172.21.15.133:6837/2249018853] up:boot 2023-11-27T15:20:08.333 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:08 smithi174 ceph-mon[131110]: fsmap 4 up:standby 2023-11-27T15:20:08.333 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:08 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-27T15:20:08.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.447+0000 7f93091ff700 1 -- 172.21.15.133:0/3835420105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9304101480 msgr2=0x7f93041018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:08.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.447+0000 7f93091ff700 1 --2- 172.21.15.133:0/3835420105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9304101480 0x7f93041018d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f92ec009990 tx=0x7f92ec009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.447+0000 7f93091ff700 1 -- 172.21.15.133:0/3835420105 shutdown_connections 2023-11-27T15:20:08.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.447+0000 7f93091ff700 1 --2- 172.21.15.133:0/3835420105 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9304101e10 0x7f93041086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.447+0000 7f93091ff700 1 --2- 172.21.15.133:0/3835420105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9304100290 0x7f93041006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.447+0000 7f93091ff700 1 --2- 172.21.15.133:0/3835420105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9304101480 0x7f93041018d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.447+0000 7f93091ff700 1 -- 172.21.15.133:0/3835420105 >> 172.21.15.133:0/3835420105 conn(0x7f93040fb870 msgr2=0x7f93040fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:08.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.447+0000 7f93091ff700 1 -- 172.21.15.133:0/3835420105 shutdown_connections 2023-11-27T15:20:08.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.447+0000 7f93091ff700 1 -- 172.21.15.133:0/3835420105 wait complete. 2023-11-27T15:20:08.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.448+0000 7f93091ff700 1 Processor -- start 2023-11-27T15:20:08.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.448+0000 7f93091ff700 1 -- start start 2023-11-27T15:20:08.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.448+0000 7f93091ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9304100290 0x7f93040748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.448+0000 7f93091ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9304101480 0x7f9304072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.448+0000 7f93091ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9304101e10 0x7f9304073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.448+0000 7f93091ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9304073bc0 con 0x7f9304100290 2023-11-27T15:20:08.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.448+0000 7f93091ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9304073d30 con 0x7f9304101e10 2023-11-27T15:20:08.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.448+0000 7f93091ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93041a0030 con 0x7f9304101480 2023-11-27T15:20:08.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.449+0000 7f9302d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9304100290 0x7f93040748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.449+0000 7f9302d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9304100290 0x7f93040748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35628/0 (socket says 172.21.15.133:35628) 2023-11-27T15:20:08.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.449+0000 7f9302d9d700 1 -- 172.21.15.133:0/3609109400 learned_addr learned my addr 172.21.15.133:0/3609109400 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:08.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.449+0000 7f92fbfff700 1 --2- 172.21.15.133:0/3609109400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9304101480 0x7f9304072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.449+0000 7f930359e700 1 --2- 172.21.15.133:0/3609109400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9304101e10 0x7f9304073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.449+0000 7f9302d9d700 1 -- 172.21.15.133:0/3609109400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9304101480 msgr2=0x7f9304072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:08.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.449+0000 7f9302d9d700 1 --2- 172.21.15.133:0/3609109400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9304101480 0x7f9304072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.449+0000 7f9302d9d700 1 -- 172.21.15.133:0/3609109400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9304101e10 msgr2=0x7f9304073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:08.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.449+0000 7f9302d9d700 1 --2- 172.21.15.133:0/3609109400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9304101e10 0x7f9304073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.449+0000 7f9302d9d700 1 -- 172.21.15.133:0/3609109400 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f4009520 con 0x7f9304100290 2023-11-27T15:20:08.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.449+0000 7f9302d9d700 1 --2- 172.21.15.133:0/3609109400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9304100290 0x7f93040748a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f92f400ea50 tx=0x7f92f400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:08.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.449+0000 7f9300d99700 1 -- 172.21.15.133:0/3609109400 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f400cbd0 con 0x7f9304100290 2023-11-27T15:20:08.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.450+0000 7f9300d99700 1 -- 172.21.15.133:0/3609109400 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92f400cd30 con 0x7f9304100290 2023-11-27T15:20:08.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.450+0000 7f93091ff700 1 -- 172.21.15.133:0/3609109400 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ec0095f0 con 0x7f9304100290 2023-11-27T15:20:08.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.450+0000 7f9300d99700 1 -- 172.21.15.133:0/3609109400 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f4018680 con 0x7f9304100290 2023-11-27T15:20:08.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.450+0000 7f93091ff700 1 -- 172.21.15.133:0/3609109400 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930410bc50 con 0x7f9304100290 2023-11-27T15:20:08.456 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.451+0000 7f93091ff700 1 -- 172.21.15.133:0/3609109400 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9304107210 con 0x7f9304100290 2023-11-27T15:20:08.456 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.452+0000 7f9300d99700 1 -- 172.21.15.133:0/3609109400 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f92f4010430 con 0x7f9304100290 2023-11-27T15:20:08.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.454+0000 7f9300d99700 1 --2- 172.21.15.133:0/3609109400 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92f00a1150 0x7f92f00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:08.458 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.454+0000 7f9300d99700 1 -- 172.21.15.133:0/3609109400 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f92f4014070 con 0x7f9304100290 2023-11-27T15:20:08.458 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.454+0000 7f92fbfff700 1 --2- 172.21.15.133:0/3609109400 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92f00a1150 0x7f92f00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:08.458 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.454+0000 7f92fbfff700 1 --2- 172.21.15.133:0/3609109400 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92f00a1150 0x7f92f00a3600 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00b2e0 tx=0x7f92ec005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:08.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.458+0000 7f9300d99700 1 -- 172.21.15.133:0/3609109400 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f92f4096500 con 0x7f9304100290 2023-11-27T15:20:08.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:08.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:08.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:08.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:08.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:08.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:08.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:08 smithi133 ceph-mon[97860]: Deploying daemon mds.e on smithi174 2023-11-27T15:20:08.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:08 smithi133 ceph-mon[97860]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:08.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:08 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.133:6836/2249018853,v1:172.21.15.133:6837/2249018853] up:boot 2023-11-27T15:20:08.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:08 smithi133 ceph-mon[97860]: fsmap 4 up:standby 2023-11-27T15:20:08.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:08 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-27T15:20:08.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:08.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:08.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:08.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:08.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:08.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:08.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:08 smithi177 ceph-mon[131504]: Deploying daemon mds.e on smithi174 2023-11-27T15:20:08.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:08 smithi177 ceph-mon[131504]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:08.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:08 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.133:6836/2249018853,v1:172.21.15.133:6837/2249018853] up:boot 2023-11-27T15:20:08.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:08 smithi177 ceph-mon[131504]: fsmap 4 up:standby 2023-11-27T15:20:08.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:08 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-11-27T15:20:08.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.697+0000 7f93091ff700 1 -- 172.21.15.133:0/3609109400 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f930402cfe0 con 0x7f9304100290 2023-11-27T15:20:08.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.697+0000 7f9300d99700 1 -- 172.21.15.133:0/3609109400 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3540 (secure 0 0 0) 0x7f92f4099b20 con 0x7f9304100290 2023-11-27T15:20:08.701 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:08.701 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[]} 2023-11-27T15:20:08.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.699+0000 7f93091ff700 1 -- 172.21.15.133:0/3609109400 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92f00a1150 msgr2=0x7f92f00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:08.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.699+0000 7f93091ff700 1 --2- 172.21.15.133:0/3609109400 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92f00a1150 0x7f92f00a3600 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00b2e0 tx=0x7f92ec005fb0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.699+0000 7f93091ff700 1 -- 172.21.15.133:0/3609109400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9304100290 msgr2=0x7f93040748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:08.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.699+0000 7f93091ff700 1 --2- 172.21.15.133:0/3609109400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9304100290 0x7f93040748a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f92f400ea50 tx=0x7f92f400ed60 comp rx=0 tx=0).stop 2023-11-27T15:20:08.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.700+0000 7f93091ff700 1 -- 172.21.15.133:0/3609109400 shutdown_connections 2023-11-27T15:20:08.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.700+0000 7f93091ff700 1 --2- 172.21.15.133:0/3609109400 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92f00a1150 0x7f92f00a3600 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.700+0000 7f93091ff700 1 --2- 172.21.15.133:0/3609109400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9304101e10 0x7f9304073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.700+0000 7f93091ff700 1 --2- 172.21.15.133:0/3609109400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9304100290 0x7f93040748a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.700+0000 7f93091ff700 1 --2- 172.21.15.133:0/3609109400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9304101480 0x7f9304072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:08.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.700+0000 7f93091ff700 1 -- 172.21.15.133:0/3609109400 >> 172.21.15.133:0/3609109400 conn(0x7f93040fb870 msgr2=0x7f93040fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:08.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.700+0000 7f93091ff700 1 -- 172.21.15.133:0/3609109400 shutdown_connections 2023-11-27T15:20:08.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:08.700+0000 7f93091ff700 1 -- 172.21.15.133:0/3609109400 wait complete. 2023-11-27T15:20:08.705 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 5 2023-11-27T15:20:08.903 INFO:journalctl@ceph.mds.e.smithi174.stdout:Nov 27 15:20:08 smithi174 systemd[1]: Starting Ceph mds.e for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:20:09.347 INFO:journalctl@ceph.mds.e.smithi174.stdout:Nov 27 15:20:09 smithi174 podman[155727]: 2023-11-27T15:20:09.347 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:09 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3609109400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:09.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:09 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3609109400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:09.653 INFO:journalctl@ceph.mds.e.smithi174.stdout:Nov 27 15:20:09 smithi174 ceph-mds[155767]: -- 172.21.15.174:0/2995917412 <== mon.2 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556c71693520 con 0x556c71669000 2023-11-27T15:20:09.653 INFO:journalctl@ceph.mds.e.smithi174.stdout:Nov 27 15:20:09 smithi174 bash[155727]: 6ff342b4d31e03dd758676daeef54c211ffbac011a9129f08a8af2222388d717 2023-11-27T15:20:09.653 INFO:journalctl@ceph.mds.e.smithi174.stdout:Nov 27 15:20:09 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-e[155743]: starting mds.e at 2023-11-27T15:20:09.653 INFO:journalctl@ceph.mds.e.smithi174.stdout:Nov 27 15:20:09 smithi174 systemd[1]: Started Ceph mds.e for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:20:09.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:09 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3609109400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:10.527 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:10 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:10.527 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:10 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:10.527 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:10 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:10.527 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:10 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:10.527 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:10 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:10.527 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:10 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:10.528 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:10 smithi177 ceph-mon[131504]: Deploying daemon mds.f on smithi177 2023-11-27T15:20:10.528 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:10 smithi177 ceph-mon[131504]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:10.528 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:10 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.174:6834/637171798,v1:172.21.15.174:6835/637171798] up:boot 2023-11-27T15:20:10.528 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:10 smithi177 ceph-mon[131504]: fsmap 5 up:standby 2023-11-27T15:20:10.528 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:10 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:20:10.724 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:20:10.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:10 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:10.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:10 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:10.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:10 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:10.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:10 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:10.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:10 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:10.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:10 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:10.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:10 smithi133 ceph-mon[97860]: Deploying daemon mds.f on smithi177 2023-11-27T15:20:10.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:10 smithi133 ceph-mon[97860]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:10.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:10 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.174:6834/637171798,v1:172.21.15.174:6835/637171798] up:boot 2023-11-27T15:20:10.778 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:10 smithi133 ceph-mon[97860]: fsmap 5 up:standby 2023-11-27T15:20:10.779 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:10 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:20:10.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:10 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:10.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:10 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:10.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:10 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:10.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:10 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:10.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:10 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:10.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:10 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:10.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:10 smithi174 ceph-mon[131110]: Deploying daemon mds.f on smithi177 2023-11-27T15:20:10.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:10 smithi174 ceph-mon[131110]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:10.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:10 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.174:6834/637171798,v1:172.21.15.174:6835/637171798] up:boot 2023-11-27T15:20:10.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:10 smithi174 ceph-mon[131110]: fsmap 5 up:standby 2023-11-27T15:20:10.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:10 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:20:10.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.932+0000 7f4c345ff700 1 -- 172.21.15.133:0/730421312 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c2c101e10 msgr2=0x7f4c2c1086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:10.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.932+0000 7f4c345ff700 1 --2- 172.21.15.133:0/730421312 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c2c101e10 0x7f4c2c1086d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400b0a0 tx=0x7f4c2400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:20:10.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.932+0000 7f4c345ff700 1 -- 172.21.15.133:0/730421312 shutdown_connections 2023-11-27T15:20:10.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.932+0000 7f4c345ff700 1 --2- 172.21.15.133:0/730421312 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c2c101e10 0x7f4c2c1086d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:10.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.932+0000 7f4c345ff700 1 --2- 172.21.15.133:0/730421312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c100290 0x7f4c2c1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:10.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.932+0000 7f4c345ff700 1 --2- 172.21.15.133:0/730421312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c2c101480 0x7f4c2c1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:10.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.932+0000 7f4c345ff700 1 -- 172.21.15.133:0/730421312 >> 172.21.15.133:0/730421312 conn(0x7f4c2c0fb870 msgr2=0x7f4c2c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:10.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.932+0000 7f4c345ff700 1 -- 172.21.15.133:0/730421312 shutdown_connections 2023-11-27T15:20:10.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.932+0000 7f4c345ff700 1 -- 172.21.15.133:0/730421312 wait complete. 2023-11-27T15:20:10.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.934+0000 7f4c345ff700 1 Processor -- start 2023-11-27T15:20:10.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.934+0000 7f4c345ff700 1 -- start start 2023-11-27T15:20:10.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.934+0000 7f4c345ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c100290 0x7f4c2c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:10.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.934+0000 7f4c345ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c2c101480 0x7f4c2c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:10.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.934+0000 7f4c345ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c2c101e10 0x7f4c2c19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:10.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.934+0000 7f4c345ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c19fc00 con 0x7f4c2c101480 2023-11-27T15:20:10.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.934+0000 7f4c345ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c19fd40 con 0x7f4c2c100290 2023-11-27T15:20:10.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.934+0000 7f4c345ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c19fed0 con 0x7f4c2c101e10 2023-11-27T15:20:10.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.935+0000 7f4c31b9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c2c101480 0x7f4c2c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:10.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.935+0000 7f4c32b9c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c2c101e10 0x7f4c2c19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:10.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.935+0000 7f4c32b9c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c2c101e10 0x7f4c2c19f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:36118/0 (socket says 172.21.15.133:36118) 2023-11-27T15:20:10.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.935+0000 7f4c31b9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c2c101480 0x7f4c2c1985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35646/0 (socket says 172.21.15.133:35646) 2023-11-27T15:20:10.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.935+0000 7f4c32b9c700 1 -- 172.21.15.133:0/554581130 learned_addr learned my addr 172.21.15.133:0/554581130 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:10.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.935+0000 7f4c3239b700 1 --2- 172.21.15.133:0/554581130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c100290 0x7f4c2c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:10.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.935+0000 7f4c32b9c700 1 -- 172.21.15.133:0/554581130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c100290 msgr2=0x7f4c2c198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:10.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.935+0000 7f4c32b9c700 1 --2- 172.21.15.133:0/554581130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c100290 0x7f4c2c198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:10.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.935+0000 7f4c32b9c700 1 -- 172.21.15.133:0/554581130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c2c101480 msgr2=0x7f4c2c1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:10.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.935+0000 7f4c32b9c700 1 --2- 172.21.15.133:0/554581130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c2c101480 0x7f4c2c1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:10.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.935+0000 7f4c32b9c700 1 -- 172.21.15.133:0/554581130 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c24009d00 con 0x7f4c2c101e10 2023-11-27T15:20:10.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.935+0000 7f4c32b9c700 1 --2- 172.21.15.133:0/554581130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c2c101e10 0x7f4c2c19f630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009850 tx=0x7f4c2400bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:10.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.935+0000 7f4c237fe700 1 -- 172.21.15.133:0/554581130 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400e040 con 0x7f4c2c101e10 2023-11-27T15:20:10.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.935+0000 7f4c237fe700 1 -- 172.21.15.133:0/554581130 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4c24003e10 con 0x7f4c2c101e10 2023-11-27T15:20:10.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.935+0000 7f4c237fe700 1 -- 172.21.15.133:0/554581130 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2401b6c0 con 0x7f4c2c101e10 2023-11-27T15:20:10.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.935+0000 7f4c345ff700 1 -- 172.21.15.133:0/554581130 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c2c10bb90 con 0x7f4c2c101e10 2023-11-27T15:20:10.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.936+0000 7f4c345ff700 1 -- 172.21.15.133:0/554581130 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c2c10ba00 con 0x7f4c2c101e10 2023-11-27T15:20:10.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.937+0000 7f4c345ff700 1 -- 172.21.15.133:0/554581130 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c2c067a60 con 0x7f4c2c101e10 2023-11-27T15:20:10.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.938+0000 7f4c237fe700 1 -- 172.21.15.133:0/554581130 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4c2401d470 con 0x7f4c2c101e10 2023-11-27T15:20:10.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.940+0000 7f4c237fe700 1 --2- 172.21.15.133:0/554581130 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4c180a1150 0x7f4c180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:10.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.940+0000 7f4c237fe700 1 -- 172.21.15.133:0/554581130 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4c240cd9f0 con 0x7f4c2c101e10 2023-11-27T15:20:10.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.940+0000 7f4c3239b700 1 --2- 172.21.15.133:0/554581130 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4c180a1150 0x7f4c180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:10.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.940+0000 7f4c3239b700 1 --2- 172.21.15.133:0/554581130 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4c180a1150 0x7f4c180a3600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c1012e0 tx=0x7f4c28007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:10.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:10.944+0000 7f4c237fe700 1 -- 172.21.15.133:0/554581130 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4c240975d0 con 0x7f4c2c101e10 2023-11-27T15:20:11.089 INFO:journalctl@ceph.mds.f.smithi177.stdout:Nov 27 15:20:10 smithi177 systemd[1]: Starting Ceph mds.f for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:20:11.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:11.184+0000 7f4c345ff700 1 -- 172.21.15.133:0/554581130 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c2c1a0900 con 0x7f4c2c101e10 2023-11-27T15:20:11.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:11.185+0000 7f4c237fe700 1 -- 172.21.15.133:0/554581130 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4297 (secure 0 0 0) 0x7f4c2409abf0 con 0x7f4c2c101e10 2023-11-27T15:20:11.189 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:11.189 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-11-27T15:20:11.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:11.187+0000 7f4c345ff700 1 -- 172.21.15.133:0/554581130 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4c180a1150 msgr2=0x7f4c180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:11.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:11.187+0000 7f4c345ff700 1 --2- 172.21.15.133:0/554581130 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4c180a1150 0x7f4c180a3600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c1012e0 tx=0x7f4c28007400 comp rx=0 tx=0).stop 2023-11-27T15:20:11.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:11.187+0000 7f4c345ff700 1 -- 172.21.15.133:0/554581130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c2c101e10 msgr2=0x7f4c2c19f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:11.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:11.187+0000 7f4c345ff700 1 --2- 172.21.15.133:0/554581130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c2c101e10 0x7f4c2c19f630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009850 tx=0x7f4c2400bce0 comp rx=0 tx=0).stop 2023-11-27T15:20:11.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:11.187+0000 7f4c345ff700 1 -- 172.21.15.133:0/554581130 shutdown_connections 2023-11-27T15:20:11.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:11.187+0000 7f4c345ff700 1 --2- 172.21.15.133:0/554581130 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4c180a1150 0x7f4c180a3600 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:11.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:11.187+0000 7f4c345ff700 1 --2- 172.21.15.133:0/554581130 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c2c101e10 0x7f4c2c19f630 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:11.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:11.187+0000 7f4c345ff700 1 --2- 172.21.15.133:0/554581130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c100290 0x7f4c2c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:11.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:11.187+0000 7f4c345ff700 1 --2- 172.21.15.133:0/554581130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c2c101480 0x7f4c2c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:11.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:11.187+0000 7f4c345ff700 1 -- 172.21.15.133:0/554581130 >> 172.21.15.133:0/554581130 conn(0x7f4c2c0fb870 msgr2=0x7f4c2c0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:11.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:11.187+0000 7f4c345ff700 1 -- 172.21.15.133:0/554581130 shutdown_connections 2023-11-27T15:20:11.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:11.187+0000 7f4c345ff700 1 -- 172.21.15.133:0/554581130 wait complete. 2023-11-27T15:20:11.192 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 6 2023-11-27T15:20:11.533 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:11 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/554581130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:11.533 INFO:journalctl@ceph.mds.f.smithi177.stdout:Nov 27 15:20:11 smithi177 podman[156881]: 2023-11-27T15:20:11.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:11 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/554581130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:11.838 INFO:journalctl@ceph.mds.f.smithi177.stdout:Nov 27 15:20:11 smithi177 ceph-mds[156922]: -- 172.21.15.177:0/344476580 <== mon.1 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559d8127f520 con 0x559d81254800 2023-11-27T15:20:11.838 INFO:journalctl@ceph.mds.f.smithi177.stdout:Nov 27 15:20:11 smithi177 bash[156881]: 63f6ff1d61b88a365871e36bed2ad7ac3e8b6c7ce1bb628ce7795fa69ef9083e 2023-11-27T15:20:11.838 INFO:journalctl@ceph.mds.f.smithi177.stdout:Nov 27 15:20:11 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-f[156898]: starting mds.f at 2023-11-27T15:20:11.838 INFO:journalctl@ceph.mds.f.smithi177.stdout:Nov 27 15:20:11 smithi177 systemd[1]: Started Ceph mds.f for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:20:11.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:11 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/554581130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:12.862 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:12.862 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:12.862 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:12.862 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:12.862 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:12.862 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:12.862 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:12 smithi177 ceph-mon[131504]: Deploying daemon mds.c on smithi177 2023-11-27T15:20:12.862 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:12 smithi177 ceph-mon[131504]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:12.862 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:12 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.177:6834/3620466749,v1:172.21.15.177:6835/3620466749] up:boot 2023-11-27T15:20:12.863 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:12 smithi177 ceph-mon[131504]: fsmap 6 up:standby 2023-11-27T15:20:12.863 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-27T15:20:12.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:12.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:12.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:12.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:12.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:12.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:12.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:12 smithi174 ceph-mon[131110]: Deploying daemon mds.c on smithi177 2023-11-27T15:20:12.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:12 smithi174 ceph-mon[131110]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:12.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:12 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.177:6834/3620466749,v1:172.21.15.177:6835/3620466749] up:boot 2023-11-27T15:20:12.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:12 smithi174 ceph-mon[131110]: fsmap 6 up:standby 2023-11-27T15:20:12.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-27T15:20:13.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:13.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:13.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:13.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:13.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:13.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:13.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:12 smithi133 ceph-mon[97860]: Deploying daemon mds.c on smithi177 2023-11-27T15:20:13.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:12 smithi133 ceph-mon[97860]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:13.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:12 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.177:6834/3620466749,v1:172.21.15.177:6835/3620466749] up:boot 2023-11-27T15:20:13.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:12 smithi133 ceph-mon[97860]: fsmap 6 up:standby 2023-11-27T15:20:13.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-27T15:20:13.139 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:20:13 smithi177 systemd[1]: Starting Ceph mds.c for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:20:13.211 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:20:13.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.414+0000 7f7fed7ff700 1 -- 172.21.15.133:0/313930467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8101480 msgr2=0x7f7fe81018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:13.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.414+0000 7f7fed7ff700 1 --2- 172.21.15.133:0/313930467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8101480 0x7f7fe81018d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009990 tx=0x7f7fdc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:13.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.414+0000 7f7fed7ff700 1 -- 172.21.15.133:0/313930467 shutdown_connections 2023-11-27T15:20:13.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.414+0000 7f7fed7ff700 1 --2- 172.21.15.133:0/313930467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fe8101e10 0x7f7fe81086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:13.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.414+0000 7f7fed7ff700 1 --2- 172.21.15.133:0/313930467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fe8100290 0x7f7fe81006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:13.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.414+0000 7f7fed7ff700 1 --2- 172.21.15.133:0/313930467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8101480 0x7f7fe81018d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:13.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.414+0000 7f7fed7ff700 1 -- 172.21.15.133:0/313930467 >> 172.21.15.133:0/313930467 conn(0x7f7fe80fb870 msgr2=0x7f7fe80fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:13.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.414+0000 7f7fed7ff700 1 -- 172.21.15.133:0/313930467 shutdown_connections 2023-11-27T15:20:13.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.414+0000 7f7fed7ff700 1 -- 172.21.15.133:0/313930467 wait complete. 2023-11-27T15:20:13.422 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.416+0000 7f7fed7ff700 1 Processor -- start 2023-11-27T15:20:13.422 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.416+0000 7f7fed7ff700 1 -- start start 2023-11-27T15:20:13.422 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.416+0000 7f7fed7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8100290 0x7f7fe80748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:13.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.416+0000 7f7fed7ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fe8101480 0x7f7fe8072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:13.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.416+0000 7f7fed7ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fe8101e10 0x7f7fe8073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:13.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.416+0000 7f7fed7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe8073bc0 con 0x7f7fe8100290 2023-11-27T15:20:13.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.416+0000 7f7fed7ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe8073d30 con 0x7f7fe8101480 2023-11-27T15:20:13.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.416+0000 7f7fed7ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe81a0000 con 0x7f7fe8101e10 2023-11-27T15:20:13.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.417+0000 7f7fe6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8100290 0x7f7fe80748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:13.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.417+0000 7f7fe6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8100290 0x7f7fe80748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53246/0 (socket says 172.21.15.133:53246) 2023-11-27T15:20:13.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.417+0000 7f7fe6ffd700 1 -- 172.21.15.133:0/4050896415 learned_addr learned my addr 172.21.15.133:0/4050896415 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:13.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.417+0000 7f7fe67fc700 1 --2- 172.21.15.133:0/4050896415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fe8101480 0x7f7fe8072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:13.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.417+0000 7f7fe6ffd700 1 -- 172.21.15.133:0/4050896415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fe8101e10 msgr2=0x7f7fe8073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:13.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.417+0000 7f7fe6ffd700 1 --2- 172.21.15.133:0/4050896415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fe8101e10 0x7f7fe8073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:13.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.417+0000 7f7fe6ffd700 1 -- 172.21.15.133:0/4050896415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fe8101480 msgr2=0x7f7fe8072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:13.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.417+0000 7f7fe6ffd700 1 --2- 172.21.15.133:0/4050896415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fe8101480 0x7f7fe8072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:13.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.417+0000 7f7fe6ffd700 1 -- 172.21.15.133:0/4050896415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd8009520 con 0x7f7fe8100290 2023-11-27T15:20:13.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.417+0000 7f7fe6ffd700 1 --2- 172.21.15.133:0/4050896415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8100290 0x7f7fe80748a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800ea50 tx=0x7f7fd800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:13.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.417+0000 7f7fcffff700 1 -- 172.21.15.133:0/4050896415 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd800cb50 con 0x7f7fe8100290 2023-11-27T15:20:13.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.417+0000 7f7fcffff700 1 -- 172.21.15.133:0/4050896415 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fd800ccb0 con 0x7f7fe8100290 2023-11-27T15:20:13.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.417+0000 7f7fcffff700 1 -- 172.21.15.133:0/4050896415 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8018680 con 0x7f7fe8100290 2023-11-27T15:20:13.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.418+0000 7f7fed7ff700 1 -- 172.21.15.133:0/4050896415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fdc0095f0 con 0x7f7fe8100290 2023-11-27T15:20:13.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.418+0000 7f7fed7ff700 1 -- 172.21.15.133:0/4050896415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe810bc50 con 0x7f7fe8100290 2023-11-27T15:20:13.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.419+0000 7f7fed7ff700 1 -- 172.21.15.133:0/4050896415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe81071c0 con 0x7f7fe8100290 2023-11-27T15:20:13.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.420+0000 7f7fcffff700 1 -- 172.21.15.133:0/4050896415 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7fd8010430 con 0x7f7fe8100290 2023-11-27T15:20:13.426 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.422+0000 7f7fcffff700 1 --2- 172.21.15.133:0/4050896415 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7fd00a1150 0x7f7fd00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:13.427 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.422+0000 7f7fcffff700 1 -- 172.21.15.133:0/4050896415 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7fd8014070 con 0x7f7fe8100290 2023-11-27T15:20:13.427 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.422+0000 7f7fe67fc700 1 --2- 172.21.15.133:0/4050896415 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7fd00a1150 0x7f7fd00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:13.427 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.422+0000 7f7fe67fc700 1 --2- 172.21.15.133:0/4050896415 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7fd00a1150 0x7f7fd00a3600 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00b2e0 tx=0x7f7fdc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:13.430 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.426+0000 7f7fcffff700 1 -- 172.21.15.133:0/4050896415 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7fd80964d0 con 0x7f7fe8100290 2023-11-27T15:20:13.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.666+0000 7f7fed7ff700 1 -- 172.21.15.133:0/4050896415 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fe802cfe0 con 0x7f7fe8100290 2023-11-27T15:20:13.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.667+0000 7f7fcffff700 1 -- 172.21.15.133:0/4050896415 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5057 (secure 0 0 0) 0x7f7fd8099af0 con 0x7f7fe8100290 2023-11-27T15:20:13.672 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:13.672 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24496,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2023-11-27T15:20:13.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.670+0000 7f7fed7ff700 1 -- 172.21.15.133:0/4050896415 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7fd00a1150 msgr2=0x7f7fd00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:13.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.670+0000 7f7fed7ff700 1 --2- 172.21.15.133:0/4050896415 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7fd00a1150 0x7f7fd00a3600 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc00b2e0 tx=0x7f7fdc005fb0 comp rx=0 tx=0).stop 2023-11-27T15:20:13.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.670+0000 7f7fed7ff700 1 -- 172.21.15.133:0/4050896415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8100290 msgr2=0x7f7fe80748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:13.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.670+0000 7f7fed7ff700 1 --2- 172.21.15.133:0/4050896415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8100290 0x7f7fe80748a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800ea50 tx=0x7f7fd800ed60 comp rx=0 tx=0).stop 2023-11-27T15:20:13.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.670+0000 7f7fed7ff700 1 -- 172.21.15.133:0/4050896415 shutdown_connections 2023-11-27T15:20:13.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.670+0000 7f7fed7ff700 1 --2- 172.21.15.133:0/4050896415 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7fd00a1150 0x7f7fd00a3600 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:13.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.670+0000 7f7fed7ff700 1 --2- 172.21.15.133:0/4050896415 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fe8101e10 0x7f7fe8073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:13.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.670+0000 7f7fed7ff700 1 --2- 172.21.15.133:0/4050896415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fe8100290 0x7f7fe80748a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:13.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.670+0000 7f7fed7ff700 1 --2- 172.21.15.133:0/4050896415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fe8101480 0x7f7fe8072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:13.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.670+0000 7f7fed7ff700 1 -- 172.21.15.133:0/4050896415 >> 172.21.15.133:0/4050896415 conn(0x7f7fe80fb870 msgr2=0x7f7fe80fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:13.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.670+0000 7f7fed7ff700 1 -- 172.21.15.133:0/4050896415 shutdown_connections 2023-11-27T15:20:13.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:13.670+0000 7f7fed7ff700 1 -- 172.21.15.133:0/4050896415 wait complete. 2023-11-27T15:20:13.677 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 7 2023-11-27T15:20:13.691 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:20:13 smithi177 podman[157224]: 2023-11-27T15:20:14.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:13 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/4050896415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:14.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:13 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/4050896415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:14.088 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:20:13 smithi177 ceph-mds[157264]: -- 172.21.15.177:0/1147359498 <== mon.1 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c04f629520 con 0x55c04f5fec00 2023-11-27T15:20:14.088 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:20:13 smithi177 bash[157224]: b94086894dfe4e80f8f52352f2f0942e7b2d5d6df61183a0760ae54bb753e171 2023-11-27T15:20:14.089 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:20:13 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-c[157240]: starting mds.c at 2023-11-27T15:20:14.089 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:20:13 smithi177 systemd[1]: Started Ceph mds.c for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:20:14.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:13 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/4050896415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:14.882 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:14 smithi174 ceph-mon[131110]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:14.883 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:14 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.177:6836/3492215689,v1:172.21.15.177:6837/3492215689] up:boot 2023-11-27T15:20:14.883 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:14 smithi174 ceph-mon[131110]: fsmap 7 up:standby 2023-11-27T15:20:14.883 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:14 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:20:14.883 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:14 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:14.884 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:14 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:14.884 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:14 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:14.884 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:14 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:14.885 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:14 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:14.885 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:14 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:14.885 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:14 smithi174 ceph-mon[131110]: Deploying daemon mds.h on smithi174 2023-11-27T15:20:15.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:14 smithi133 ceph-mon[97860]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:15.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:14 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.177:6836/3492215689,v1:172.21.15.177:6837/3492215689] up:boot 2023-11-27T15:20:15.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:14 smithi133 ceph-mon[97860]: fsmap 7 up:standby 2023-11-27T15:20:15.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:20:15.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:15.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:15.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:15.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:15.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:15.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:15.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:14 smithi133 ceph-mon[97860]: Deploying daemon mds.h on smithi174 2023-11-27T15:20:15.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:14 smithi177 ceph-mon[131504]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:15.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:14 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.177:6836/3492215689,v1:172.21.15.177:6837/3492215689] up:boot 2023-11-27T15:20:15.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:14 smithi177 ceph-mon[131504]: fsmap 7 up:standby 2023-11-27T15:20:15.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:14 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:20:15.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:14 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:15.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:14 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:15.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:14 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:15.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:14 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:15.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:14 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:15.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:14 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:15.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:14 smithi177 ceph-mon[131504]: Deploying daemon mds.h on smithi174 2023-11-27T15:20:15.653 INFO:journalctl@ceph.mds.h.smithi174.stdout:Nov 27 15:20:15 smithi174 systemd[1]: Starting Ceph mds.h for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:20:15.695 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:20:15.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.897+0000 7f7aba7ff700 1 -- 172.21.15.133:0/3947307992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab4073550 msgr2=0x7f7ab40739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:15.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.897+0000 7f7aba7ff700 1 --2- 172.21.15.133:0/3947307992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab4073550 0x7f7ab40739c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009990 tx=0x7f7aa8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:15.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.897+0000 7f7aba7ff700 1 -- 172.21.15.133:0/3947307992 shutdown_connections 2023-11-27T15:20:15.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.897+0000 7f7aba7ff700 1 --2- 172.21.15.133:0/3947307992 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ab410c860 0x7f7ab410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:15.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.897+0000 7f7aba7ff700 1 --2- 172.21.15.133:0/3947307992 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ab4074b30 0x7f7ab4072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:15.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.897+0000 7f7aba7ff700 1 --2- 172.21.15.133:0/3947307992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab4073550 0x7f7ab40739c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:15.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.897+0000 7f7aba7ff700 1 -- 172.21.15.133:0/3947307992 >> 172.21.15.133:0/3947307992 conn(0x7f7ab40fb870 msgr2=0x7f7ab40fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:15.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.897+0000 7f7aba7ff700 1 -- 172.21.15.133:0/3947307992 shutdown_connections 2023-11-27T15:20:15.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.897+0000 7f7aba7ff700 1 -- 172.21.15.133:0/3947307992 wait complete. 2023-11-27T15:20:15.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.898+0000 7f7aba7ff700 1 Processor -- start 2023-11-27T15:20:15.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.898+0000 7f7aba7ff700 1 -- start start 2023-11-27T15:20:15.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.899+0000 7f7aba7ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab4073550 0x7f7ab419c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:15.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.899+0000 7f7aba7ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ab4074b30 0x7f7ab419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:15.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.899+0000 7f7aba7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ab410c860 0x7f7ab41a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:15.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.899+0000 7f7aba7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab41a4140 con 0x7f7ab410c860 2023-11-27T15:20:15.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.899+0000 7f7aba7ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab41a4280 con 0x7f7ab4073550 2023-11-27T15:20:15.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.899+0000 7f7aba7ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab41a4410 con 0x7f7ab4074b30 2023-11-27T15:20:15.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.899+0000 7f7ab3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab4073550 0x7f7ab419c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:15.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.899+0000 7f7ab3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab4073550 0x7f7ab419c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:37470/0 (socket says 172.21.15.133:37470) 2023-11-27T15:20:15.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.899+0000 7f7ab3fff700 1 -- 172.21.15.133:0/2689459896 learned_addr learned my addr 172.21.15.133:0/2689459896 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:15.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.899+0000 7f7ab8d9c700 1 --2- 172.21.15.133:0/2689459896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ab410c860 0x7f7ab41a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:15.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.899+0000 7f7ab37fe700 1 --2- 172.21.15.133:0/2689459896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ab4074b30 0x7f7ab419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:15.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.899+0000 7f7ab3fff700 1 -- 172.21.15.133:0/2689459896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ab4074b30 msgr2=0x7f7ab419cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:15.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.899+0000 7f7ab3fff700 1 --2- 172.21.15.133:0/2689459896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ab4074b30 0x7f7ab419cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:15.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.899+0000 7f7ab3fff700 1 -- 172.21.15.133:0/2689459896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ab410c860 msgr2=0x7f7ab41a3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:15.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.899+0000 7f7ab3fff700 1 --2- 172.21.15.133:0/2689459896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ab410c860 0x7f7ab41a3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:15.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.899+0000 7f7ab3fff700 1 -- 172.21.15.133:0/2689459896 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa80095f0 con 0x7f7ab4073550 2023-11-27T15:20:15.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.899+0000 7f7ab3fff700 1 --2- 172.21.15.133:0/2689459896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab4073550 0x7f7ab419c5c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400e970 tx=0x7f7aa400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:15.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.900+0000 7f7ab17fa700 1 -- 172.21.15.133:0/2689459896 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4009790 con 0x7f7ab4073550 2023-11-27T15:20:15.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.900+0000 7f7aba7ff700 1 -- 172.21.15.133:0/2689459896 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab4112270 con 0x7f7ab4073550 2023-11-27T15:20:15.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.900+0000 7f7aba7ff700 1 -- 172.21.15.133:0/2689459896 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab41120e0 con 0x7f7ab4073550 2023-11-27T15:20:15.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.900+0000 7f7ab17fa700 1 -- 172.21.15.133:0/2689459896 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7aa400cca0 con 0x7f7ab4073550 2023-11-27T15:20:15.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.900+0000 7f7ab17fa700 1 -- 172.21.15.133:0/2689459896 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4021de0 con 0x7f7ab4073550 2023-11-27T15:20:15.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.901+0000 7f7aba7ff700 1 -- 172.21.15.133:0/2689459896 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa0005310 con 0x7f7ab4073550 2023-11-27T15:20:15.911 INFO:journalctl@ceph.mds.h.smithi174.stdout:Nov 27 15:20:15 smithi174 podman[156064]: 2023-11-27T15:20:15.911 INFO:journalctl@ceph.mds.h.smithi174.stdout:Nov 27 15:20:15 smithi174 ceph-mds[156104]: -- 172.21.15.174:0/2192879507 <== mon.2 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a18478b520 con 0x55a184760c00 2023-11-27T15:20:15.911 INFO:journalctl@ceph.mds.h.smithi174.stdout:Nov 27 15:20:15 smithi174 bash[156064]: 9c672774345dc8bdc69619e84fcd00b20f5d270fe11b20c8a8f81d7747321dfa 2023-11-27T15:20:15.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.903+0000 7f7ab17fa700 1 -- 172.21.15.133:0/2689459896 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7aa40214d0 con 0x7f7ab4073550 2023-11-27T15:20:15.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.904+0000 7f7ab17fa700 1 --2- 172.21.15.133:0/2689459896 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7a9c0a11d0 0x7f7a9c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:15.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.904+0000 7f7ab17fa700 1 -- 172.21.15.133:0/2689459896 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7aa4014070 con 0x7f7ab4073550 2023-11-27T15:20:15.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.904+0000 7f7ab37fe700 1 --2- 172.21.15.133:0/2689459896 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7a9c0a11d0 0x7f7a9c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:15.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.905+0000 7f7ab37fe700 1 --2- 172.21.15.133:0/2689459896 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7a9c0a11d0 0x7f7a9c0a3680 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800b2e0 tx=0x7f7aa8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:15.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:15.906+0000 7f7ab17fa700 1 -- 172.21.15.133:0/2689459896 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7aa4096210 con 0x7f7ab4073550 2023-11-27T15:20:16.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:16.153+0000 7f7aba7ff700 1 -- 172.21.15.133:0/2689459896 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7aa0005180 con 0x7f7ab4073550 2023-11-27T15:20:16.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:16.154+0000 7f7ab17fa700 1 -- 172.21.15.133:0/2689459896 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5817 (secure 0 0 0) 0x7f7aa4099830 con 0x7f7ab4073550 2023-11-27T15:20:16.159 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:16.160 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24496,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/3492215689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3492215689},{"type":"v1","addr":"172.21.15.177:6837","nonce":3492215689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-11-27T15:20:16.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:16.157+0000 7f7aba7ff700 1 -- 172.21.15.133:0/2689459896 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7a9c0a11d0 msgr2=0x7f7a9c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:16.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:16.157+0000 7f7aba7ff700 1 --2- 172.21.15.133:0/2689459896 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7a9c0a11d0 0x7f7a9c0a3680 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800b2e0 tx=0x7f7aa8005fb0 comp rx=0 tx=0).stop 2023-11-27T15:20:16.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:16.157+0000 7f7aba7ff700 1 -- 172.21.15.133:0/2689459896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab4073550 msgr2=0x7f7ab419c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:16.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:16.157+0000 7f7aba7ff700 1 --2- 172.21.15.133:0/2689459896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab4073550 0x7f7ab419c5c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7aa400e970 tx=0x7f7aa400ec80 comp rx=0 tx=0).stop 2023-11-27T15:20:16.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:16.157+0000 7f7aba7ff700 1 -- 172.21.15.133:0/2689459896 shutdown_connections 2023-11-27T15:20:16.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:16.157+0000 7f7aba7ff700 1 --2- 172.21.15.133:0/2689459896 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7a9c0a11d0 0x7f7a9c0a3680 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:16.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:16.157+0000 7f7aba7ff700 1 --2- 172.21.15.133:0/2689459896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ab410c860 0x7f7ab41a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:16.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:16.157+0000 7f7aba7ff700 1 --2- 172.21.15.133:0/2689459896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab4073550 0x7f7ab419c5c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:16.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:16.157+0000 7f7aba7ff700 1 --2- 172.21.15.133:0/2689459896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ab4074b30 0x7f7ab419cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:16.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:16.157+0000 7f7aba7ff700 1 -- 172.21.15.133:0/2689459896 >> 172.21.15.133:0/2689459896 conn(0x7f7ab40fb870 msgr2=0x7f7ab40fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:16.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:16.157+0000 7f7aba7ff700 1 -- 172.21.15.133:0/2689459896 shutdown_connections 2023-11-27T15:20:16.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:16.157+0000 7f7aba7ff700 1 -- 172.21.15.133:0/2689459896 wait complete. 2023-11-27T15:20:16.164 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 8 2023-11-27T15:20:16.403 INFO:journalctl@ceph.mds.h.smithi174.stdout:Nov 27 15:20:15 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-h[156080]: starting mds.h at 2023-11-27T15:20:16.403 INFO:journalctl@ceph.mds.h.smithi174.stdout:Nov 27 15:20:15 smithi174 systemd[1]: Started Ceph mds.h for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:20:17.264 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:16 smithi177 ceph-mon[131504]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:17.264 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:17.264 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:17.265 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:17.265 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:17.265 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:17.265 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:17.265 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:16 smithi177 ceph-mon[131504]: Deploying daemon mds.i on smithi177 2023-11-27T15:20:17.265 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:16 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2689459896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:17.265 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:16 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] up:boot 2023-11-27T15:20:17.265 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:16 smithi177 ceph-mon[131504]: fsmap 8 up:standby 2023-11-27T15:20:17.265 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-27T15:20:17.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:16 smithi133 ceph-mon[97860]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:17.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:17.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:17.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:17.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:17.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:17.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:17.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:16 smithi133 ceph-mon[97860]: Deploying daemon mds.i on smithi177 2023-11-27T15:20:17.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:16 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2689459896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:17.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:16 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] up:boot 2023-11-27T15:20:17.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:16 smithi133 ceph-mon[97860]: fsmap 8 up:standby 2023-11-27T15:20:17.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-27T15:20:17.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:16 smithi174 ceph-mon[131110]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:17.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:17.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:17.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:17.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:17.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:17.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:17.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:16 smithi174 ceph-mon[131110]: Deploying daemon mds.i on smithi177 2023-11-27T15:20:17.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:16 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2689459896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:17.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:16 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] up:boot 2023-11-27T15:20:17.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:16 smithi174 ceph-mon[131110]: fsmap 8 up:standby 2023-11-27T15:20:17.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-27T15:20:17.588 INFO:journalctl@ceph.mds.i.smithi177.stdout:Nov 27 15:20:17 smithi177 systemd[1]: Starting Ceph mds.i for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:20:18.099 INFO:journalctl@ceph.mds.i.smithi177.stdout:Nov 27 15:20:17 smithi177 podman[157565]: 2023-11-27T15:20:18.099 INFO:journalctl@ceph.mds.i.smithi177.stdout:Nov 27 15:20:18 smithi177 ceph-mds[157605]: -- 172.21.15.177:0/1807155130 <== mon.1 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558d7628f520 con 0x558d76265000 2023-11-27T15:20:18.182 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:20:18.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.398+0000 7fc1eabff700 1 -- 172.21.15.133:0/1998310152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1e4074b30 msgr2=0x7fc1e4072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:18.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.398+0000 7fc1eabff700 1 --2- 172.21.15.133:0/1998310152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1e4074b30 0x7fc1e4072f80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009990 tx=0x7fc1d8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:18.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.398+0000 7fc1eabff700 1 -- 172.21.15.133:0/1998310152 shutdown_connections 2023-11-27T15:20:18.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.398+0000 7fc1eabff700 1 --2- 172.21.15.133:0/1998310152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1e410c860 0x7fc1e410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:18.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.398+0000 7fc1eabff700 1 --2- 172.21.15.133:0/1998310152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1e4074b30 0x7fc1e4072f80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:18.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.398+0000 7fc1eabff700 1 --2- 172.21.15.133:0/1998310152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1e4073550 0x7fc1e40739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:18.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.398+0000 7fc1eabff700 1 -- 172.21.15.133:0/1998310152 >> 172.21.15.133:0/1998310152 conn(0x7fc1e40fb870 msgr2=0x7fc1e40fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:18.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.399+0000 7fc1eabff700 1 -- 172.21.15.133:0/1998310152 shutdown_connections 2023-11-27T15:20:18.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.399+0000 7fc1eabff700 1 -- 172.21.15.133:0/1998310152 wait complete. 2023-11-27T15:20:18.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.399+0000 7fc1eabff700 1 Processor -- start 2023-11-27T15:20:18.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.400+0000 7fc1eabff700 1 -- start start 2023-11-27T15:20:18.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.400+0000 7fc1eabff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1e4073550 0x7fc1e419c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:18.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.400+0000 7fc1eabff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1e4074b30 0x7fc1e419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:18.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.400+0000 7fc1eabff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1e410c860 0x7fc1e41a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:18.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.400+0000 7fc1eabff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e41a4130 con 0x7fc1e4074b30 2023-11-27T15:20:18.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.400+0000 7fc1eabff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e41a4270 con 0x7fc1e410c860 2023-11-27T15:20:18.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.400+0000 7fc1eabff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e41a4400 con 0x7fc1e4073550 2023-11-27T15:20:18.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.400+0000 7fc1e899b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1e4073550 0x7fc1e419c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:18.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.400+0000 7fc1e3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1e4074b30 0x7fc1e419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:18.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.400+0000 7fc1e899b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1e4073550 0x7fc1e419c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:34154/0 (socket says 172.21.15.133:34154) 2023-11-27T15:20:18.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.400+0000 7fc1e899b700 1 -- 172.21.15.133:0/2009845588 learned_addr learned my addr 172.21.15.133:0/2009845588 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:18.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.400+0000 7fc1e919c700 1 --2- 172.21.15.133:0/2009845588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1e410c860 0x7fc1e41a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:18.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.401+0000 7fc1e899b700 1 -- 172.21.15.133:0/2009845588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1e410c860 msgr2=0x7fc1e41a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:18.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.401+0000 7fc1e899b700 1 --2- 172.21.15.133:0/2009845588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1e410c860 0x7fc1e41a3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:18.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.401+0000 7fc1e899b700 1 -- 172.21.15.133:0/2009845588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1e4074b30 msgr2=0x7fc1e419caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:18.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.401+0000 7fc1e899b700 1 --2- 172.21.15.133:0/2009845588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1e4074b30 0x7fc1e419caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:18.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.401+0000 7fc1e899b700 1 -- 172.21.15.133:0/2009845588 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d80095f0 con 0x7fc1e4073550 2023-11-27T15:20:18.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.401+0000 7fc1e919c700 1 --2- 172.21.15.133:0/2009845588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1e410c860 0x7fc1e41a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:20:18.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.401+0000 7fc1e899b700 1 --2- 172.21.15.133:0/2009845588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1e4073550 0x7fc1e419c5b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8004710 tx=0x7fc1d8009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:18.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.401+0000 7fc1e1ffb700 1 -- 172.21.15.133:0/2009845588 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d801d070 con 0x7fc1e4073550 2023-11-27T15:20:18.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.401+0000 7fc1e1ffb700 1 -- 172.21.15.133:0/2009845588 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1d8021410 con 0x7fc1e4073550 2023-11-27T15:20:18.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.401+0000 7fc1eabff700 1 -- 172.21.15.133:0/2009845588 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e4112250 con 0x7fc1e4073550 2023-11-27T15:20:18.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.401+0000 7fc1e1ffb700 1 -- 172.21.15.133:0/2009845588 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8017650 con 0x7fc1e4073550 2023-11-27T15:20:18.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.401+0000 7fc1eabff700 1 -- 172.21.15.133:0/2009845588 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e41120c0 con 0x7fc1e4073550 2023-11-27T15:20:18.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.402+0000 7fc1eabff700 1 -- 172.21.15.133:0/2009845588 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e41023b0 con 0x7fc1e4073550 2023-11-27T15:20:18.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.404+0000 7fc1e1ffb700 1 -- 172.21.15.133:0/2009845588 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fc1d8021580 con 0x7fc1e4073550 2023-11-27T15:20:18.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.405+0000 7fc1e1ffb700 1 --2- 172.21.15.133:0/2009845588 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc1d40a11d0 0x7fc1d40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:18.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.406+0000 7fc1e1ffb700 1 -- 172.21.15.133:0/2009845588 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc1d80cd840 con 0x7fc1e4073550 2023-11-27T15:20:18.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.406+0000 7fc1e3fff700 1 --2- 172.21.15.133:0/2009845588 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc1d40a11d0 0x7fc1d40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:18.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.406+0000 7fc1e3fff700 1 --2- 172.21.15.133:0/2009845588 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc1d40a11d0 0x7fc1d40a3680 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4074850 tx=0x7fc1d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:18.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.409+0000 7fc1e1ffb700 1 -- 172.21.15.133:0/2009845588 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc1d80974c0 con 0x7fc1e4073550 2023-11-27T15:20:18.588 INFO:journalctl@ceph.mds.i.smithi177.stdout:Nov 27 15:20:18 smithi177 bash[157565]: 630cb31efe89340cae07f940ff8f5132d654b3bd2d32b729107af44a971634cf 2023-11-27T15:20:18.588 INFO:journalctl@ceph.mds.i.smithi177.stdout:Nov 27 15:20:18 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-i[157581]: starting mds.i at 2023-11-27T15:20:18.588 INFO:journalctl@ceph.mds.i.smithi177.stdout:Nov 27 15:20:18 smithi177 systemd[1]: Started Ceph mds.i for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:20:18.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.661+0000 7fc1eabff700 1 -- 172.21.15.133:0/2009845588 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1e402cfe0 con 0x7fc1e4073550 2023-11-27T15:20:18.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.662+0000 7fc1e1ffb700 1 -- 172.21.15.133:0/2009845588 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7339 (secure 0 0 0) 0x7fc1d809aae0 con 0x7fc1e4073550 2023-11-27T15:20:18.667 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:18.667 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24496,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/3492215689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3492215689},{"type":"v1","addr":"172.21.15.177:6837","nonce":3492215689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-11-27T15:20:18.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.665+0000 7fc1eabff700 1 -- 172.21.15.133:0/2009845588 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc1d40a11d0 msgr2=0x7fc1d40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:18.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.665+0000 7fc1eabff700 1 --2- 172.21.15.133:0/2009845588 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc1d40a11d0 0x7fc1d40a3680 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4074850 tx=0x7fc1d0009260 comp rx=0 tx=0).stop 2023-11-27T15:20:18.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.665+0000 7fc1eabff700 1 -- 172.21.15.133:0/2009845588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1e4073550 msgr2=0x7fc1e419c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:18.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.665+0000 7fc1eabff700 1 --2- 172.21.15.133:0/2009845588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1e4073550 0x7fc1e419c5b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8004710 tx=0x7fc1d8009580 comp rx=0 tx=0).stop 2023-11-27T15:20:18.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.665+0000 7fc1eabff700 1 -- 172.21.15.133:0/2009845588 shutdown_connections 2023-11-27T15:20:18.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.665+0000 7fc1eabff700 1 --2- 172.21.15.133:0/2009845588 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc1d40a11d0 0x7fc1d40a3680 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:18.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.665+0000 7fc1eabff700 1 --2- 172.21.15.133:0/2009845588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1e410c860 0x7fc1e41a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:18.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.665+0000 7fc1eabff700 1 --2- 172.21.15.133:0/2009845588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1e4073550 0x7fc1e419c5b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:18.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.665+0000 7fc1eabff700 1 --2- 172.21.15.133:0/2009845588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1e4074b30 0x7fc1e419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:18.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.665+0000 7fc1eabff700 1 -- 172.21.15.133:0/2009845588 >> 172.21.15.133:0/2009845588 conn(0x7fc1e40fb870 msgr2=0x7fc1e40fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:18.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.665+0000 7fc1eabff700 1 -- 172.21.15.133:0/2009845588 shutdown_connections 2023-11-27T15:20:18.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:18.665+0000 7fc1eabff700 1 -- 172.21.15.133:0/2009845588 wait complete. 2023-11-27T15:20:18.671 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 10 2023-11-27T15:20:19.392 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:19 smithi133 ceph-mon[97860]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:19.392 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:19 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] up:boot 2023-11-27T15:20:19.392 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:19 smithi133 ceph-mon[97860]: fsmap 9 up:standby 2023-11-27T15:20:19.393 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:20:19.393 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:19.393 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:19.393 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:19.393 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:19.393 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:19.393 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:19.393 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:19 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2009845588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:19.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:19 smithi174 ceph-mon[131110]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:19.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:19 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] up:boot 2023-11-27T15:20:19.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:19 smithi174 ceph-mon[131110]: fsmap 9 up:standby 2023-11-27T15:20:19.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:20:19.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:19.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:19.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:19.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:19.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:19.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:19.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:19 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2009845588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:19.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:19 smithi177 ceph-mon[131504]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:19.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:19 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] up:boot 2023-11-27T15:20:19.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:19 smithi177 ceph-mon[131504]: fsmap 9 up:standby 2023-11-27T15:20:19.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:20:19.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:19.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:19.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:19.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:19.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:19.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:19.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:19 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2009845588' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:20.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:20 smithi174 ceph-mon[131110]: Deploying daemon mds.g on smithi133 2023-11-27T15:20:20.453 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:20 smithi133 ceph-mon[97860]: Deploying daemon mds.g on smithi133 2023-11-27T15:20:20.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:20 smithi177 ceph-mon[131504]: Deploying daemon mds.g on smithi133 2023-11-27T15:20:20.698 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:20:20.767 INFO:journalctl@ceph.mds.g.smithi133.stdout:Nov 27 15:20:20 smithi133 systemd[1]: Starting Ceph mds.g for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:20:20.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.924+0000 7f7ac73ff700 1 -- 172.21.15.133:0/2418922759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ac0108140 msgr2=0x7f7ac010a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:20.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.924+0000 7f7ac73ff700 1 --2- 172.21.15.133:0/2418922759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ac0108140 0x7f7ac010a5c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7ab800b0a0 tx=0x7f7ab800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:20:20.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.924+0000 7f7ac73ff700 1 -- 172.21.15.133:0/2418922759 shutdown_connections 2023-11-27T15:20:20.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.924+0000 7f7ac73ff700 1 --2- 172.21.15.133:0/2418922759 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ac0108140 0x7f7ac010a5c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:20.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.924+0000 7f7ac73ff700 1 --2- 172.21.15.133:0/2418922759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ac0074ad0 0x7f7ac0072f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:20.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.924+0000 7f7ac73ff700 1 --2- 172.21.15.133:0/2418922759 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ac00734f0 0x7f7ac0073960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:20.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.924+0000 7f7ac73ff700 1 -- 172.21.15.133:0/2418922759 >> 172.21.15.133:0/2418922759 conn(0x7f7ac00fb890 msgr2=0x7f7ac00fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:20.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.924+0000 7f7ac73ff700 1 -- 172.21.15.133:0/2418922759 shutdown_connections 2023-11-27T15:20:20.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.924+0000 7f7ac73ff700 1 -- 172.21.15.133:0/2418922759 wait complete. 2023-11-27T15:20:20.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.926+0000 7f7ac73ff700 1 Processor -- start 2023-11-27T15:20:20.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.926+0000 7f7ac73ff700 1 -- start start 2023-11-27T15:20:20.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.926+0000 7f7ac73ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ac00734f0 0x7f7ac0198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:20.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.926+0000 7f7ac73ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ac0074ad0 0x7f7ac0198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:20.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.926+0000 7f7ac73ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ac0108140 0x7f7ac019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:20.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.926+0000 7f7ac73ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac019fdd0 con 0x7f7ac00734f0 2023-11-27T15:20:20.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.926+0000 7f7ac73ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac019ff10 con 0x7f7ac0108140 2023-11-27T15:20:20.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.926+0000 7f7ac73ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac01a00a0 con 0x7f7ac0074ad0 2023-11-27T15:20:20.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.926+0000 7f7ac519b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ac00734f0 0x7f7ac0198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:20.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.926+0000 7f7ac499a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ac0074ad0 0x7f7ac0198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:20.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.926+0000 7f7ac519b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ac00734f0 0x7f7ac0198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53322/0 (socket says 172.21.15.133:53322) 2023-11-27T15:20:20.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.926+0000 7f7ac519b700 1 -- 172.21.15.133:0/837210376 learned_addr learned my addr 172.21.15.133:0/837210376 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:20.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.927+0000 7f7ac599c700 1 --2- 172.21.15.133:0/837210376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ac0108140 0x7f7ac019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:20.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.927+0000 7f7ac519b700 1 -- 172.21.15.133:0/837210376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ac0074ad0 msgr2=0x7f7ac0198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:20.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.927+0000 7f7ac519b700 1 --2- 172.21.15.133:0/837210376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ac0074ad0 0x7f7ac0198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:20.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.927+0000 7f7ac519b700 1 -- 172.21.15.133:0/837210376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ac0108140 msgr2=0x7f7ac019f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:20.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.927+0000 7f7ac519b700 1 --2- 172.21.15.133:0/837210376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ac0108140 0x7f7ac019f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:20.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.927+0000 7f7ac519b700 1 -- 172.21.15.133:0/837210376 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab8009d00 con 0x7f7ac00734f0 2023-11-27T15:20:20.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.927+0000 7f7ac519b700 1 --2- 172.21.15.133:0/837210376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ac00734f0 0x7f7ac0198250 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00e9a0 tx=0x7f7abc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:20.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.927+0000 7f7ab67fc700 1 -- 172.21.15.133:0/837210376 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc009790 con 0x7f7ac00734f0 2023-11-27T15:20:20.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.927+0000 7f7ab67fc700 1 -- 172.21.15.133:0/837210376 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7abc010430 con 0x7f7ac00734f0 2023-11-27T15:20:20.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.927+0000 7f7ac73ff700 1 -- 172.21.15.133:0/837210376 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac010de40 con 0x7f7ac00734f0 2023-11-27T15:20:20.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.927+0000 7f7ab67fc700 1 -- 172.21.15.133:0/837210376 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc018680 con 0x7f7ac00734f0 2023-11-27T15:20:20.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.927+0000 7f7ac73ff700 1 -- 172.21.15.133:0/837210376 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac010dcb0 con 0x7f7ac00734f0 2023-11-27T15:20:20.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.929+0000 7f7ac73ff700 1 -- 172.21.15.133:0/837210376 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac0106880 con 0x7f7ac00734f0 2023-11-27T15:20:20.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.930+0000 7f7ab67fc700 1 -- 172.21.15.133:0/837210376 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7abc0105a0 con 0x7f7ac00734f0 2023-11-27T15:20:20.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.932+0000 7f7ab67fc700 1 --2- 172.21.15.133:0/837210376 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7aac0a1150 0x7f7aac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:20.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.933+0000 7f7ab67fc700 1 -- 172.21.15.133:0/837210376 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7abc014070 con 0x7f7ac00734f0 2023-11-27T15:20:20.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.933+0000 7f7ac499a700 1 --2- 172.21.15.133:0/837210376 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7aac0a1150 0x7f7aac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:20.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.933+0000 7f7ac499a700 1 --2- 172.21.15.133:0/837210376 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7aac0a1150 0x7f7aac0a3600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0005950 tx=0x7f7ab000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:20.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:20.937+0000 7f7ab67fc700 1 -- 172.21.15.133:0/837210376 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7abc051730 con 0x7f7ac00734f0 2023-11-27T15:20:21.027 INFO:journalctl@ceph.mds.g.smithi133.stdout:Nov 27 15:20:20 smithi133 podman[132017]: 2023-11-27T15:20:21.204 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:21.199+0000 7f7ac73ff700 1 -- 172.21.15.133:0/837210376 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ac002cfe0 con 0x7f7ac00734f0 2023-11-27T15:20:21.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:21.200+0000 7f7ab67fc700 1 -- 172.21.15.133:0/837210376 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7339 (secure 0 0 0) 0x7f7abc096220 con 0x7f7ac00734f0 2023-11-27T15:20:21.205 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:21.205 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24496,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/3492215689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3492215689},{"type":"v1","addr":"172.21.15.177:6837","nonce":3492215689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-11-27T15:20:21.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:21.202+0000 7f7ac73ff700 1 -- 172.21.15.133:0/837210376 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7aac0a1150 msgr2=0x7f7aac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:21.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:21.202+0000 7f7ac73ff700 1 --2- 172.21.15.133:0/837210376 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7aac0a1150 0x7f7aac0a3600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0005950 tx=0x7f7ab000b410 comp rx=0 tx=0).stop 2023-11-27T15:20:21.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:21.202+0000 7f7ac73ff700 1 -- 172.21.15.133:0/837210376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ac00734f0 msgr2=0x7f7ac0198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:21.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:21.202+0000 7f7ac73ff700 1 --2- 172.21.15.133:0/837210376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ac00734f0 0x7f7ac0198250 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7abc00e9a0 tx=0x7f7abc00ecb0 comp rx=0 tx=0).stop 2023-11-27T15:20:21.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:21.202+0000 7f7ac73ff700 1 -- 172.21.15.133:0/837210376 shutdown_connections 2023-11-27T15:20:21.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:21.202+0000 7f7ac73ff700 1 --2- 172.21.15.133:0/837210376 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7aac0a1150 0x7f7aac0a3600 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:21.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:21.202+0000 7f7ac73ff700 1 --2- 172.21.15.133:0/837210376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ac0108140 0x7f7ac019f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:21.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:21.202+0000 7f7ac73ff700 1 --2- 172.21.15.133:0/837210376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ac00734f0 0x7f7ac0198250 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:21.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:21.202+0000 7f7ac73ff700 1 --2- 172.21.15.133:0/837210376 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ac0074ad0 0x7f7ac0198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:21.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:21.202+0000 7f7ac73ff700 1 -- 172.21.15.133:0/837210376 >> 172.21.15.133:0/837210376 conn(0x7f7ac00fb890 msgr2=0x7f7ac00fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:21.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:21.202+0000 7f7ac73ff700 1 -- 172.21.15.133:0/837210376 shutdown_connections 2023-11-27T15:20:21.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:21.202+0000 7f7ac73ff700 1 -- 172.21.15.133:0/837210376 wait complete. 2023-11-27T15:20:21.209 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 10 2023-11-27T15:20:21.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:21 smithi174 ceph-mon[131110]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:21.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:21 smithi133 ceph-mon[97860]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:21.528 INFO:journalctl@ceph.mds.g.smithi133.stdout:Nov 27 15:20:21 smithi133 bash[132017]: 19d3af6baf87afe665463b155a736f63aa17742bd0f07fcfe0c322b5cf4a3a1e 2023-11-27T15:20:21.528 INFO:journalctl@ceph.mds.g.smithi133.stdout:Nov 27 15:20:21 smithi133 ceph-mds[132081]: -- 172.21.15.133:0/804434421 <== mon.0 v2:172.21.15.133:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ea63871520 con 0x55ea63848800 2023-11-27T15:20:21.528 INFO:journalctl@ceph.mds.g.smithi133.stdout:Nov 27 15:20:21 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-g[132055]: starting mds.g at 2023-11-27T15:20:21.528 INFO:journalctl@ceph.mds.g.smithi133.stdout:Nov 27 15:20:21 smithi133 systemd[1]: Started Ceph mds.g for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:20:21.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:21 smithi177 ceph-mon[131504]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:22.394 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:22 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/837210376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:22.395 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:22 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:22.395 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:22 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:22.395 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:22 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:22.395 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:22 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:22.395 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:22 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:22.396 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:22 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:22.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:22 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/837210376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:22.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:22 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:22.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:22 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:22.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:22 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:22.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:22 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:22.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:22 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:22.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:22 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:22.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:22 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/837210376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:22.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:22 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:22.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:22 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:22.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:22 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:22.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:22 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:22.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:22 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:22.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:22 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:23.228 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:20:23.304 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:23 smithi133 ceph-mon[97860]: Deploying daemon mds.a on smithi133 2023-11-27T15:20:23.304 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:23 smithi133 ceph-mon[97860]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:23.304 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:23 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.133:6838/2381290259,v1:172.21.15.133:6839/2381290259] up:boot 2023-11-27T15:20:23.304 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:23 smithi133 ceph-mon[97860]: fsmap 10 up:standby 2023-11-27T15:20:23.304 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:23 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-27T15:20:23.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:23 smithi174 ceph-mon[131110]: Deploying daemon mds.a on smithi133 2023-11-27T15:20:23.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:23 smithi174 ceph-mon[131110]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:23.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:23 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.133:6838/2381290259,v1:172.21.15.133:6839/2381290259] up:boot 2023-11-27T15:20:23.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:23 smithi174 ceph-mon[131110]: fsmap 10 up:standby 2023-11-27T15:20:23.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:23 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-27T15:20:23.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.462+0000 7f6a905ff700 1 -- 172.21.15.133:0/2490595847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a88108160 msgr2=0x7f6a8810a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:23.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.462+0000 7f6a905ff700 1 --2- 172.21.15.133:0/2490595847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a88108160 0x7f6a8810a5c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6a8000b0a0 tx=0x7f6a8000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:20:23.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.462+0000 7f6a905ff700 1 -- 172.21.15.133:0/2490595847 shutdown_connections 2023-11-27T15:20:23.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.462+0000 7f6a905ff700 1 --2- 172.21.15.133:0/2490595847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a88108160 0x7f6a8810a5c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:23.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.462+0000 7f6a905ff700 1 --2- 172.21.15.133:0/2490595847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a88074ab0 0x7f6a88072f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:23.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.462+0000 7f6a905ff700 1 --2- 172.21.15.133:0/2490595847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a880734d0 0x7f6a88073940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:23.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.462+0000 7f6a905ff700 1 -- 172.21.15.133:0/2490595847 >> 172.21.15.133:0/2490595847 conn(0x7f6a880fb890 msgr2=0x7f6a880fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:23.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.462+0000 7f6a905ff700 1 -- 172.21.15.133:0/2490595847 shutdown_connections 2023-11-27T15:20:23.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.463+0000 7f6a905ff700 1 -- 172.21.15.133:0/2490595847 wait complete. 2023-11-27T15:20:23.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.464+0000 7f6a905ff700 1 Processor -- start 2023-11-27T15:20:23.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.464+0000 7f6a905ff700 1 -- start start 2023-11-27T15:20:23.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.464+0000 7f6a905ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a880734d0 0x7f6a88198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:23.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.464+0000 7f6a905ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a88074ab0 0x7f6a881987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:23.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.465+0000 7f6a905ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a88108160 0x7f6a8819f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:23.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.465+0000 7f6a905ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8819fe00 con 0x7f6a88074ab0 2023-11-27T15:20:23.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.465+0000 7f6a905ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8819ff40 con 0x7f6a88108160 2023-11-27T15:20:23.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.465+0000 7f6a905ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a881a00d0 con 0x7f6a880734d0 2023-11-27T15:20:23.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.465+0000 7f6a8db9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a88074ab0 0x7f6a881987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:23.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.465+0000 7f6a8e39b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a880734d0 0x7f6a88198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:23.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.465+0000 7f6a8db9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a88074ab0 0x7f6a881987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39314/0 (socket says 172.21.15.133:39314) 2023-11-27T15:20:23.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.465+0000 7f6a8db9a700 1 -- 172.21.15.133:0/3836334332 learned_addr learned my addr 172.21.15.133:0/3836334332 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:23.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.465+0000 7f6a8eb9c700 1 --2- 172.21.15.133:0/3836334332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a88108160 0x7f6a8819f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:23.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.465+0000 7f6a8e39b700 1 -- 172.21.15.133:0/3836334332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a88108160 msgr2=0x7f6a8819f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:23.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.465+0000 7f6a8e39b700 1 --2- 172.21.15.133:0/3836334332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a88108160 0x7f6a8819f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:23.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.465+0000 7f6a8e39b700 1 -- 172.21.15.133:0/3836334332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a88074ab0 msgr2=0x7f6a881987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:23.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.465+0000 7f6a8e39b700 1 --2- 172.21.15.133:0/3836334332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a88074ab0 0x7f6a881987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:23.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.465+0000 7f6a8e39b700 1 -- 172.21.15.133:0/3836334332 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a80009d00 con 0x7f6a880734d0 2023-11-27T15:20:23.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.465+0000 7f6a8db9a700 1 --2- 172.21.15.133:0/3836334332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a88074ab0 0x7f6a881987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:23.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.466+0000 7f6a8e39b700 1 --2- 172.21.15.133:0/3836334332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a880734d0 0x7f6a88198280 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400e9f0 tx=0x7f6a8400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:23.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.466+0000 7f6a7f7fe700 1 -- 172.21.15.133:0/3836334332 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8400c9c0 con 0x7f6a880734d0 2023-11-27T15:20:23.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.466+0000 7f6a905ff700 1 -- 172.21.15.133:0/3836334332 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8810de40 con 0x7f6a880734d0 2023-11-27T15:20:23.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.466+0000 7f6a905ff700 1 -- 172.21.15.133:0/3836334332 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8810dcb0 con 0x7f6a880734d0 2023-11-27T15:20:23.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.466+0000 7f6a7f7fe700 1 -- 172.21.15.133:0/3836334332 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a84010430 con 0x7f6a880734d0 2023-11-27T15:20:23.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.466+0000 7f6a7f7fe700 1 -- 172.21.15.133:0/3836334332 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a84018680 con 0x7f6a880734d0 2023-11-27T15:20:23.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.467+0000 7f6a905ff700 1 -- 172.21.15.133:0/3836334332 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a881068a0 con 0x7f6a880734d0 2023-11-27T15:20:23.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.469+0000 7f6a7f7fe700 1 -- 172.21.15.133:0/3836334332 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6a84010810 con 0x7f6a880734d0 2023-11-27T15:20:23.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.470+0000 7f6a7f7fe700 1 --2- 172.21.15.133:0/3836334332 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6a740a1150 0x7f6a740a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:23.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.470+0000 7f6a7f7fe700 1 -- 172.21.15.133:0/3836334332 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6a84014070 con 0x7f6a880734d0 2023-11-27T15:20:23.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.470+0000 7f6a8db9a700 1 --2- 172.21.15.133:0/3836334332 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6a740a1150 0x7f6a740a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:23.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.471+0000 7f6a8db9a700 1 --2- 172.21.15.133:0/3836334332 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6a740a1150 0x7f6a740a3600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6a78005950 tx=0x7f6a7800b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:23.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.473+0000 7f6a7f7fe700 1 -- 172.21.15.133:0/3836334332 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6a84051730 con 0x7f6a880734d0 2023-11-27T15:20:23.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:23 smithi177 ceph-mon[131504]: Deploying daemon mds.a on smithi133 2023-11-27T15:20:23.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:23 smithi177 ceph-mon[131504]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:23.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:23 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.133:6838/2381290259,v1:172.21.15.133:6839/2381290259] up:boot 2023-11-27T15:20:23.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:23 smithi177 ceph-mon[131504]: fsmap 10 up:standby 2023-11-27T15:20:23.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:23 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-11-27T15:20:23.742 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.737+0000 7f6a905ff700 1 -- 172.21.15.133:0/3836334332 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a88061700 con 0x7f6a880734d0 2023-11-27T15:20:23.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.738+0000 7f6a7f7fe700 1 -- 172.21.15.133:0/3836334332 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8100 (secure 0 0 0) 0x7f6a840961f0 con 0x7f6a880734d0 2023-11-27T15:20:23.743 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:23.743 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14493,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2381290259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2381290259},{"type":"v1","addr":"172.21.15.133:6839","nonce":2381290259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24496,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/3492215689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3492215689},{"type":"v1","addr":"172.21.15.177:6837","nonce":3492215689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-11-27T15:20:23.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.740+0000 7f6a905ff700 1 -- 172.21.15.133:0/3836334332 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6a740a1150 msgr2=0x7f6a740a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:23.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.740+0000 7f6a905ff700 1 --2- 172.21.15.133:0/3836334332 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6a740a1150 0x7f6a740a3600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6a78005950 tx=0x7f6a7800b500 comp rx=0 tx=0).stop 2023-11-27T15:20:23.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.741+0000 7f6a905ff700 1 -- 172.21.15.133:0/3836334332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a880734d0 msgr2=0x7f6a88198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:23.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.741+0000 7f6a905ff700 1 --2- 172.21.15.133:0/3836334332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a880734d0 0x7f6a88198280 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400e9f0 tx=0x7f6a8400ed00 comp rx=0 tx=0).stop 2023-11-27T15:20:23.746 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.741+0000 7f6a905ff700 1 -- 172.21.15.133:0/3836334332 shutdown_connections 2023-11-27T15:20:23.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.741+0000 7f6a905ff700 1 --2- 172.21.15.133:0/3836334332 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6a740a1150 0x7f6a740a3600 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:23.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.741+0000 7f6a905ff700 1 --2- 172.21.15.133:0/3836334332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a88108160 0x7f6a8819f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:23.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.741+0000 7f6a905ff700 1 --2- 172.21.15.133:0/3836334332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6a880734d0 0x7f6a88198280 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:23.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.741+0000 7f6a905ff700 1 --2- 172.21.15.133:0/3836334332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a88074ab0 0x7f6a881987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:23.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.741+0000 7f6a905ff700 1 -- 172.21.15.133:0/3836334332 >> 172.21.15.133:0/3836334332 conn(0x7f6a880fb890 msgr2=0x7f6a880fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:23.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.741+0000 7f6a905ff700 1 -- 172.21.15.133:0/3836334332 shutdown_connections 2023-11-27T15:20:23.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:23.741+0000 7f6a905ff700 1 -- 172.21.15.133:0/3836334332 wait complete. 2023-11-27T15:20:23.748 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 11 2023-11-27T15:20:24.027 INFO:journalctl@ceph.mds.a.smithi133.stdout:Nov 27 15:20:23 smithi133 systemd[1]: Starting Ceph mds.a for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:20:24.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:24 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3836334332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:24.405 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:24 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3836334332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:24.405 INFO:journalctl@ceph.mds.a.smithi133.stdout:Nov 27 15:20:24 smithi133 podman[132431]: 2023-11-27T15:20:24.405 INFO:journalctl@ceph.mds.a.smithi133.stdout:Nov 27 15:20:24 smithi133 bash[132431]: dbae139ff71a58e2b9ea93c4446371cc952dfb10337c7b43e67b7c7c2f9729f0 2023-11-27T15:20:24.406 INFO:journalctl@ceph.mds.a.smithi133.stdout:Nov 27 15:20:24 smithi133 ceph-mds[132471]: -- 172.21.15.133:0/338729245 <== mon.1 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55eb54551520 con 0x55eb54528c00 2023-11-27T15:20:24.406 INFO:journalctl@ceph.mds.a.smithi133.stdout:Nov 27 15:20:24 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-a[132447]: starting mds.a at 2023-11-27T15:20:24.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:24 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3836334332' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:24.777 INFO:journalctl@ceph.mds.a.smithi133.stdout:Nov 27 15:20:24 smithi133 systemd[1]: Started Ceph mds.a for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:20:25.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:25 smithi174 ceph-mon[131110]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:25.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:25 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:25.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:25 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:25.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:25 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:25.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:25 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:25.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:25 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:25.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:25 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:25.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:25 smithi133 ceph-mon[97860]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:25.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:25 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:25.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:25 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:25.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:25 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:25.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:25 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:25.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:25 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:25.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:25 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:25.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:25 smithi177 ceph-mon[131504]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:25.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:25 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:25.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:25 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:25.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:25 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:25.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:25 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-11-27T15:20:25.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:25 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-11-27T15:20:25.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:25 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:25.768 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:20:25.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.973+0000 7f65c5dff700 1 -- 172.21.15.133:0/2686381641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65c010c860 msgr2=0x7f65c010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:25.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.973+0000 7f65c5dff700 1 --2- 172.21.15.133:0/2686381641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65c010c860 0x7f65c010ec40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f65b800b0a0 tx=0x7f65b800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:20:25.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.974+0000 7f65c5dff700 1 -- 172.21.15.133:0/2686381641 shutdown_connections 2023-11-27T15:20:25.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.974+0000 7f65c5dff700 1 --2- 172.21.15.133:0/2686381641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65c010c860 0x7f65c010ec40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:25.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.974+0000 7f65c5dff700 1 --2- 172.21.15.133:0/2686381641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65c0074b30 0x7f65c0072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:25.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.974+0000 7f65c5dff700 1 --2- 172.21.15.133:0/2686381641 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65c0073550 0x7f65c00739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:25.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.974+0000 7f65c5dff700 1 -- 172.21.15.133:0/2686381641 >> 172.21.15.133:0/2686381641 conn(0x7f65c00fb870 msgr2=0x7f65c00fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:25.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.974+0000 7f65c5dff700 1 -- 172.21.15.133:0/2686381641 shutdown_connections 2023-11-27T15:20:25.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.974+0000 7f65c5dff700 1 -- 172.21.15.133:0/2686381641 wait complete. 2023-11-27T15:20:25.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.975+0000 7f65c5dff700 1 Processor -- start 2023-11-27T15:20:25.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.975+0000 7f65c5dff700 1 -- start start 2023-11-27T15:20:25.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.975+0000 7f65c5dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65c0073550 0x7f65c019c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:25.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.975+0000 7f65c5dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65c0074b30 0x7f65c019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:25.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.975+0000 7f65c5dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65c010c860 0x7f65c01a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:25.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.975+0000 7f65c5dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c01a41b0 con 0x7f65c0073550 2023-11-27T15:20:25.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.975+0000 7f65c5dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c01a42f0 con 0x7f65c010c860 2023-11-27T15:20:25.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.975+0000 7f65c5dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c01a4480 con 0x7f65c0074b30 2023-11-27T15:20:25.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.975+0000 7f65bf7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65c0073550 0x7f65c019c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:25.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.975+0000 7f65bf7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65c0073550 0x7f65c019c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39336/0 (socket says 172.21.15.133:39336) 2023-11-27T15:20:25.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.975+0000 7f65bffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65c010c860 0x7f65c01a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:25.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.975+0000 7f65bf7fe700 1 -- 172.21.15.133:0/2356626711 learned_addr learned my addr 172.21.15.133:0/2356626711 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:25.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.975+0000 7f65beffd700 1 --2- 172.21.15.133:0/2356626711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65c0074b30 0x7f65c019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:25.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.976+0000 7f65bffff700 1 -- 172.21.15.133:0/2356626711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65c0074b30 msgr2=0x7f65c019cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:25.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.976+0000 7f65bffff700 1 --2- 172.21.15.133:0/2356626711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65c0074b30 0x7f65c019cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:25.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.976+0000 7f65bffff700 1 -- 172.21.15.133:0/2356626711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65c0073550 msgr2=0x7f65c019c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:25.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.976+0000 7f65bffff700 1 --2- 172.21.15.133:0/2356626711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65c0073550 0x7f65c019c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:25.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.976+0000 7f65bffff700 1 -- 172.21.15.133:0/2356626711 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b8009d00 con 0x7f65c010c860 2023-11-27T15:20:25.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.976+0000 7f65bf7fe700 1 --2- 172.21.15.133:0/2356626711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65c0073550 0x7f65c019c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:25.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.976+0000 7f65bffff700 1 --2- 172.21.15.133:0/2356626711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65c010c860 0x7f65c01a3be0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009fd0 tx=0x7f65b8004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:25.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.976+0000 7f65bcff9700 1 -- 172.21.15.133:0/2356626711 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b800e040 con 0x7f65c010c860 2023-11-27T15:20:25.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.976+0000 7f65c5dff700 1 -- 172.21.15.133:0/2356626711 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c0112270 con 0x7f65c010c860 2023-11-27T15:20:25.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.976+0000 7f65bcff9700 1 -- 172.21.15.133:0/2356626711 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65b8004e70 con 0x7f65c010c860 2023-11-27T15:20:25.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.976+0000 7f65bcff9700 1 -- 172.21.15.133:0/2356626711 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b801da00 con 0x7f65c010c860 2023-11-27T15:20:25.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.976+0000 7f65c5dff700 1 -- 172.21.15.133:0/2356626711 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c01120e0 con 0x7f65c010c860 2023-11-27T15:20:25.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.977+0000 7f65c5dff700 1 -- 172.21.15.133:0/2356626711 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c01023b0 con 0x7f65c010c860 2023-11-27T15:20:25.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.979+0000 7f65bcff9700 1 -- 172.21.15.133:0/2356626711 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f65b8005880 con 0x7f65c010c860 2023-11-27T15:20:25.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.981+0000 7f65bcff9700 1 --2- 172.21.15.133:0/2356626711 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f65a80a11d0 0x7f65a80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:25.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.981+0000 7f65bcff9700 1 -- 172.21.15.133:0/2356626711 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f65b80cdd90 con 0x7f65c010c860 2023-11-27T15:20:25.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.981+0000 7f65bf7fe700 1 --2- 172.21.15.133:0/2356626711 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f65a80a11d0 0x7f65a80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:25.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.981+0000 7f65bf7fe700 1 --2- 172.21.15.133:0/2356626711 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f65a80a11d0 0x7f65a80a3680 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f65b000f480 tx=0x7f65b000a400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:25.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:25.985+0000 7f65bcff9700 1 -- 172.21.15.133:0/2356626711 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f65b8097970 con 0x7f65c010c860 2023-11-27T15:20:26.146 INFO:journalctl@ceph.mds.b.smithi174.stdout:Nov 27 15:20:25 smithi174 systemd[1]: Starting Ceph mds.b for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:20:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:26.224+0000 7f65c5dff700 1 -- 172.21.15.133:0/2356626711 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65c002cfe0 con 0x7f65c010c860 2023-11-27T15:20:26.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:26.225+0000 7f65bcff9700 1 -- 172.21.15.133:0/2356626711 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8861 (secure 0 0 0) 0x7f65b809af90 con 0x7f65c010c860 2023-11-27T15:20:26.230 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:26.230 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14493,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2381290259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2381290259},{"type":"v1","addr":"172.21.15.133:6839","nonce":2381290259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1704875895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1704875895},{"type":"v1","addr":"172.21.15.133:6841","nonce":1704875895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24496,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/3492215689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3492215689},{"type":"v1","addr":"172.21.15.177:6837","nonce":3492215689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-11-27T15:20:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:26.227+0000 7f65c5dff700 1 -- 172.21.15.133:0/2356626711 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f65a80a11d0 msgr2=0x7f65a80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:26.227+0000 7f65c5dff700 1 --2- 172.21.15.133:0/2356626711 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f65a80a11d0 0x7f65a80a3680 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f65b000f480 tx=0x7f65b000a400 comp rx=0 tx=0).stop 2023-11-27T15:20:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:26.227+0000 7f65c5dff700 1 -- 172.21.15.133:0/2356626711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65c010c860 msgr2=0x7f65c01a3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:26.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:26.227+0000 7f65c5dff700 1 --2- 172.21.15.133:0/2356626711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65c010c860 0x7f65c01a3be0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f65b8009fd0 tx=0x7f65b8004c80 comp rx=0 tx=0).stop 2023-11-27T15:20:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:26.227+0000 7f65c5dff700 1 -- 172.21.15.133:0/2356626711 shutdown_connections 2023-11-27T15:20:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:26.227+0000 7f65c5dff700 1 --2- 172.21.15.133:0/2356626711 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f65a80a11d0 0x7f65a80a3680 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:26.227+0000 7f65c5dff700 1 --2- 172.21.15.133:0/2356626711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65c010c860 0x7f65c01a3be0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:26.227+0000 7f65c5dff700 1 --2- 172.21.15.133:0/2356626711 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f65c0073550 0x7f65c019c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:26.227+0000 7f65c5dff700 1 --2- 172.21.15.133:0/2356626711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65c0074b30 0x7f65c019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:26.227+0000 7f65c5dff700 1 -- 172.21.15.133:0/2356626711 >> 172.21.15.133:0/2356626711 conn(0x7f65c00fb870 msgr2=0x7f65c00fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:26.227+0000 7f65c5dff700 1 -- 172.21.15.133:0/2356626711 shutdown_connections 2023-11-27T15:20:26.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:26.227+0000 7f65c5dff700 1 -- 172.21.15.133:0/2356626711 wait complete. 2023-11-27T15:20:26.234 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 12 2023-11-27T15:20:26.403 INFO:journalctl@ceph.mds.b.smithi174.stdout:Nov 27 15:20:26 smithi174 podman[156404]: 2023-11-27T15:20:26.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:26 smithi174 ceph-mon[131110]: Deploying daemon mds.b on smithi174 2023-11-27T15:20:26.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:26 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.133:6840/1704875895,v1:172.21.15.133:6841/1704875895] up:boot 2023-11-27T15:20:26.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:26 smithi174 ceph-mon[131110]: fsmap 11 up:standby 2023-11-27T15:20:26.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:26 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-27T15:20:26.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:26 smithi133 ceph-mon[97860]: Deploying daemon mds.b on smithi174 2023-11-27T15:20:26.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:26 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.133:6840/1704875895,v1:172.21.15.133:6841/1704875895] up:boot 2023-11-27T15:20:26.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:26 smithi133 ceph-mon[97860]: fsmap 11 up:standby 2023-11-27T15:20:26.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:26 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-27T15:20:26.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:26 smithi177 ceph-mon[131504]: Deploying daemon mds.b on smithi174 2023-11-27T15:20:26.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:26 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.133:6840/1704875895,v1:172.21.15.133:6841/1704875895] up:boot 2023-11-27T15:20:26.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:26 smithi177 ceph-mon[131504]: fsmap 11 up:standby 2023-11-27T15:20:26.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:26 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-11-27T15:20:26.903 INFO:journalctl@ceph.mds.b.smithi174.stdout:Nov 27 15:20:26 smithi174 ceph-mds[156444]: -- 172.21.15.174:0/2284597063 <== mon.2 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56477c0cf520 con 0x56477c0a4800 2023-11-27T15:20:26.903 INFO:journalctl@ceph.mds.b.smithi174.stdout:Nov 27 15:20:26 smithi174 bash[156404]: 328951b76ca096e0e636730932baa31d997fa94b01f35501e34ff8e499e3540d 2023-11-27T15:20:26.903 INFO:journalctl@ceph.mds.b.smithi174.stdout:Nov 27 15:20:26 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-b[156420]: starting mds.b at 2023-11-27T15:20:26.903 INFO:journalctl@ceph.mds.b.smithi174.stdout:Nov 27 15:20:26 smithi174 systemd[1]: Started Ceph mds.b for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:20:27.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:27 smithi174 ceph-mon[131110]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:27.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:27 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2356626711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:27.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:27 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:27.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:27 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:27.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:27 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:27.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:27 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:27.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:27 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:27.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:27 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:27.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:27 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:27.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:27 smithi133 ceph-mon[97860]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:27.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:27 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2356626711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:27.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:27 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:27.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:27 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:27.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:27 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:27.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:27 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:27.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:27 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:27.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:27 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:27.529 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:27 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:27.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:27 smithi177 ceph-mon[131504]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:27.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:27 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2356626711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:27.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:27 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:27.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:27 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:27.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:27 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:27.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:27 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:27.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:27 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:27.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:27 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:27.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:27 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:28.252 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:20:28.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.457+0000 7fa79bdff700 1 -- 172.21.15.133:0/3545673393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7940734f0 msgr2=0x7fa794073960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:28.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.457+0000 7fa79bdff700 1 --2- 172.21.15.133:0/3545673393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7940734f0 0x7fa794073960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa7840098f0 tx=0x7fa784009c00 comp rx=0 tx=0).stop 2023-11-27T15:20:28.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.457+0000 7fa79bdff700 1 -- 172.21.15.133:0/3545673393 shutdown_connections 2023-11-27T15:20:28.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.457+0000 7fa79bdff700 1 --2- 172.21.15.133:0/3545673393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa794108140 0x7fa79410a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:28.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.457+0000 7fa79bdff700 1 --2- 172.21.15.133:0/3545673393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa794074ad0 0x7fa794072f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:28.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.457+0000 7fa79bdff700 1 --2- 172.21.15.133:0/3545673393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7940734f0 0x7fa794073960 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:28.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.457+0000 7fa79bdff700 1 -- 172.21.15.133:0/3545673393 >> 172.21.15.133:0/3545673393 conn(0x7fa7940fb890 msgr2=0x7fa7940fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:28.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.457+0000 7fa79bdff700 1 -- 172.21.15.133:0/3545673393 shutdown_connections 2023-11-27T15:20:28.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.457+0000 7fa79bdff700 1 -- 172.21.15.133:0/3545673393 wait complete. 2023-11-27T15:20:28.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.458+0000 7fa79bdff700 1 Processor -- start 2023-11-27T15:20:28.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.458+0000 7fa79bdff700 1 -- start start 2023-11-27T15:20:28.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.459+0000 7fa79bdff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7940734f0 0x7fa7941981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:28.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.459+0000 7fa79bdff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa794074ad0 0x7fa7941986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:28.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.459+0000 7fa79bdff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa794108140 0x7fa79419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:28.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.459+0000 7fa79bdff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79419fd20 con 0x7fa794108140 2023-11-27T15:20:28.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.459+0000 7fa79bdff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79419fe60 con 0x7fa794074ad0 2023-11-27T15:20:28.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.459+0000 7fa79bdff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79419fff0 con 0x7fa7940734f0 2023-11-27T15:20:28.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.459+0000 7fa79939a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa794074ad0 0x7fa7941986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:28.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.459+0000 7fa79939a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa794074ad0 0x7fa7941986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:52076/0 (socket says 172.21.15.133:52076) 2023-11-27T15:20:28.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.459+0000 7fa79939a700 1 -- 172.21.15.133:0/535379929 learned_addr learned my addr 172.21.15.133:0/535379929 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:28.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.459+0000 7fa79a39c700 1 --2- 172.21.15.133:0/535379929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa794108140 0x7fa79419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:28.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.459+0000 7fa799b9b700 1 --2- 172.21.15.133:0/535379929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7940734f0 0x7fa7941981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:28.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.459+0000 7fa79939a700 1 -- 172.21.15.133:0/535379929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7940734f0 msgr2=0x7fa7941981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:28.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.459+0000 7fa79939a700 1 --2- 172.21.15.133:0/535379929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7940734f0 0x7fa7941981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:28.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.459+0000 7fa79939a700 1 -- 172.21.15.133:0/535379929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa794108140 msgr2=0x7fa79419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:28.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.459+0000 7fa79939a700 1 --2- 172.21.15.133:0/535379929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa794108140 0x7fa79419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:28.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.459+0000 7fa79939a700 1 -- 172.21.15.133:0/535379929 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7900095f0 con 0x7fa794074ad0 2023-11-27T15:20:28.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.460+0000 7fa79939a700 1 --2- 172.21.15.133:0/535379929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa794074ad0 0x7fa7941986e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa784009fd0 tx=0x7fa78400b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:28.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.460+0000 7fa78affd700 1 -- 172.21.15.133:0/535379929 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78401c070 con 0x7fa794074ad0 2023-11-27T15:20:28.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.460+0000 7fa78affd700 1 -- 172.21.15.133:0/535379929 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa78400f7d0 con 0x7fa794074ad0 2023-11-27T15:20:28.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.460+0000 7fa79bdff700 1 -- 172.21.15.133:0/535379929 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa784009520 con 0x7fa794074ad0 2023-11-27T15:20:28.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.460+0000 7fa78affd700 1 -- 172.21.15.133:0/535379929 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa784004ad0 con 0x7fa794074ad0 2023-11-27T15:20:28.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.460+0000 7fa79bdff700 1 -- 172.21.15.133:0/535379929 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa79410de40 con 0x7fa794074ad0 2023-11-27T15:20:28.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.461+0000 7fa79bdff700 1 -- 172.21.15.133:0/535379929 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa794106880 con 0x7fa794074ad0 2023-11-27T15:20:28.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.463+0000 7fa78affd700 1 -- 172.21.15.133:0/535379929 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa784020070 con 0x7fa794074ad0 2023-11-27T15:20:28.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.464+0000 7fa78affd700 1 --2- 172.21.15.133:0/535379929 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa7800a1150 0x7fa7800a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:28.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.464+0000 7fa78affd700 1 -- 172.21.15.133:0/535379929 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa7840cd130 con 0x7fa794074ad0 2023-11-27T15:20:28.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.464+0000 7fa799b9b700 1 --2- 172.21.15.133:0/535379929 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa7800a1150 0x7fa7800a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:28.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.465+0000 7fa799b9b700 1 --2- 172.21.15.133:0/535379929 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa7800a1150 0x7fa7800a3600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa790005fd0 tx=0x7fa790009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:28.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.468+0000 7fa78affd700 1 -- 172.21.15.133:0/535379929 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa78405c730 con 0x7fa794074ad0 2023-11-27T15:20:28.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:28 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] up:boot 2023-11-27T15:20:28.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:28 smithi133 ceph-mon[97860]: fsmap 12 up:standby 2023-11-27T15:20:28.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:28 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-27T15:20:28.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:28 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] up:boot 2023-11-27T15:20:28.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:28 smithi177 ceph-mon[131504]: fsmap 12 up:standby 2023-11-27T15:20:28.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:28 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-27T15:20:28.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:28 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] up:boot 2023-11-27T15:20:28.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:28 smithi174 ceph-mon[131110]: fsmap 12 up:standby 2023-11-27T15:20:28.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:28 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-27T15:20:28.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.775+0000 7fa79bdff700 1 -- 172.21.15.133:0/535379929 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa79402cfe0 con 0x7fa794074ad0 2023-11-27T15:20:28.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.776+0000 7fa78affd700 1 -- 172.21.15.133:0/535379929 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9622 (secure 0 0 0) 0x7fa784096d10 con 0x7fa794074ad0 2023-11-27T15:20:28.780 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:28.780 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14493,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2381290259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2381290259},{"type":"v1","addr":"172.21.15.133:6839","nonce":2381290259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1704875895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1704875895},{"type":"v1","addr":"172.21.15.133:6841","nonce":1704875895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24496,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/3492215689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3492215689},{"type":"v1","addr":"172.21.15.177:6837","nonce":3492215689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24560,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/2781994447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2781994447},{"type":"v1","addr":"172.21.15.174:6839","nonce":2781994447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-11-27T15:20:28.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.778+0000 7fa79bdff700 1 -- 172.21.15.133:0/535379929 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa7800a1150 msgr2=0x7fa7800a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:28.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.778+0000 7fa79bdff700 1 --2- 172.21.15.133:0/535379929 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa7800a1150 0x7fa7800a3600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa790005fd0 tx=0x7fa790009310 comp rx=0 tx=0).stop 2023-11-27T15:20:28.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.778+0000 7fa79bdff700 1 -- 172.21.15.133:0/535379929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa794074ad0 msgr2=0x7fa7941986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:28.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.778+0000 7fa79bdff700 1 --2- 172.21.15.133:0/535379929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa794074ad0 0x7fa7941986e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa784009fd0 tx=0x7fa78400b790 comp rx=0 tx=0).stop 2023-11-27T15:20:28.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.778+0000 7fa79bdff700 1 -- 172.21.15.133:0/535379929 shutdown_connections 2023-11-27T15:20:28.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.778+0000 7fa79bdff700 1 --2- 172.21.15.133:0/535379929 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa7800a1150 0x7fa7800a3600 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:28.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.778+0000 7fa79bdff700 1 --2- 172.21.15.133:0/535379929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa794108140 0x7fa79419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:28.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.779+0000 7fa79bdff700 1 --2- 172.21.15.133:0/535379929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7940734f0 0x7fa7941981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:28.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.779+0000 7fa79bdff700 1 --2- 172.21.15.133:0/535379929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa794074ad0 0x7fa7941986e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:28.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.779+0000 7fa79bdff700 1 -- 172.21.15.133:0/535379929 >> 172.21.15.133:0/535379929 conn(0x7fa7940fb890 msgr2=0x7fa7940fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:28.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.779+0000 7fa79bdff700 1 -- 172.21.15.133:0/535379929 shutdown_connections 2023-11-27T15:20:28.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:28.779+0000 7fa79bdff700 1 -- 172.21.15.133:0/535379929 wait complete. 2023-11-27T15:20:28.784 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 13 2023-11-27T15:20:28.804 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:20:29.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.009+0000 7fe0e73ff700 1 -- 172.21.15.133:0/4075954411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0e0100d20 msgr2=0x7fe0e0103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:29.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.009+0000 7fe0e73ff700 1 --2- 172.21.15.133:0/4075954411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0e0100d20 0x7fe0e0103100 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00b0a0 tx=0x7fe0dc00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.009+0000 7fe0e73ff700 1 -- 172.21.15.133:0/4075954411 shutdown_connections 2023-11-27T15:20:29.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.009+0000 7fe0e73ff700 1 --2- 172.21.15.133:0/4075954411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0e0105f60 0x7fe0e01083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.009+0000 7fe0e73ff700 1 --2- 172.21.15.133:0/4075954411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0e0100d20 0x7fe0e0103100 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.009+0000 7fe0e73ff700 1 --2- 172.21.15.133:0/4075954411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0e0103640 0x7fe0e0105a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.010+0000 7fe0e73ff700 1 -- 172.21.15.133:0/4075954411 >> 172.21.15.133:0/4075954411 conn(0x7fe0e00fa7a0 msgr2=0x7fe0e00fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:29.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.010+0000 7fe0e73ff700 1 -- 172.21.15.133:0/4075954411 shutdown_connections 2023-11-27T15:20:29.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.010+0000 7fe0e73ff700 1 -- 172.21.15.133:0/4075954411 wait complete. 2023-11-27T15:20:29.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.011+0000 7fe0e73ff700 1 Processor -- start 2023-11-27T15:20:29.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.011+0000 7fe0e73ff700 1 -- start start 2023-11-27T15:20:29.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.011+0000 7fe0e73ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0e0100d20 0x7fe0e01961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:29.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.011+0000 7fe0e73ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0e0103640 0x7fe0e01966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:29.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.011+0000 7fe0e73ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0e0105f60 0x7fe0e019d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:29.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.011+0000 7fe0e73ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e019dd20 con 0x7fe0e0100d20 2023-11-27T15:20:29.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.011+0000 7fe0e73ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e019de60 con 0x7fe0e0103640 2023-11-27T15:20:29.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.011+0000 7fe0e73ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0e019dff0 con 0x7fe0e0105f60 2023-11-27T15:20:29.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.011+0000 7fe0e519b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0e0100d20 0x7fe0e01961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:29.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.011+0000 7fe0e499a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0e0103640 0x7fe0e01966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:29.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.011+0000 7fe0e499a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0e0103640 0x7fe0e01966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:52094/0 (socket says 172.21.15.133:52094) 2023-11-27T15:20:29.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.011+0000 7fe0e499a700 1 -- 172.21.15.133:0/3573978319 learned_addr learned my addr 172.21.15.133:0/3573978319 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:29.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.011+0000 7fe0e599c700 1 --2- 172.21.15.133:0/3573978319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0e0105f60 0x7fe0e019d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:29.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.012+0000 7fe0e519b700 1 -- 172.21.15.133:0/3573978319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0e0105f60 msgr2=0x7fe0e019d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:29.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.012+0000 7fe0e519b700 1 --2- 172.21.15.133:0/3573978319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0e0105f60 0x7fe0e019d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.012+0000 7fe0e519b700 1 -- 172.21.15.133:0/3573978319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0e0103640 msgr2=0x7fe0e01966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:29.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.012+0000 7fe0e519b700 1 --2- 172.21.15.133:0/3573978319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0e0103640 0x7fe0e01966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.012+0000 7fe0e519b700 1 -- 172.21.15.133:0/3573978319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0dc009d00 con 0x7fe0e0100d20 2023-11-27T15:20:29.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.012+0000 7fe0e499a700 1 --2- 172.21.15.133:0/3573978319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0e0103640 0x7fe0e01966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:29.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.012+0000 7fe0e519b700 1 --2- 172.21.15.133:0/3573978319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0e0100d20 0x7fe0e01961a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc0062a0 tx=0x7fe0dc0094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:29.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.012+0000 7fe0d67fc700 1 -- 172.21.15.133:0/3573978319 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc00e040 con 0x7fe0e0100d20 2023-11-27T15:20:29.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.012+0000 7fe0d67fc700 1 -- 172.21.15.133:0/3573978319 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0dc004d60 con 0x7fe0e0100d20 2023-11-27T15:20:29.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.012+0000 7fe0e73ff700 1 -- 172.21.15.133:0/3573978319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e010bc50 con 0x7fe0e0100d20 2023-11-27T15:20:29.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.012+0000 7fe0d67fc700 1 -- 172.21.15.133:0/3573978319 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc01d480 con 0x7fe0e0100d20 2023-11-27T15:20:29.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.012+0000 7fe0e73ff700 1 -- 172.21.15.133:0/3573978319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0e010bac0 con 0x7fe0e0100d20 2023-11-27T15:20:29.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.013+0000 7fe0e73ff700 1 -- 172.21.15.133:0/3573978319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0e010bde0 con 0x7fe0e0100d20 2023-11-27T15:20:29.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.015+0000 7fe0d67fc700 1 -- 172.21.15.133:0/3573978319 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe0dc003fe0 con 0x7fe0e0100d20 2023-11-27T15:20:29.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.016+0000 7fe0d67fc700 1 --2- 172.21.15.133:0/3573978319 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe0cc0a1150 0x7fe0cc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:29.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.016+0000 7fe0d67fc700 1 -- 172.21.15.133:0/3573978319 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe0dc0cdb90 con 0x7fe0e0100d20 2023-11-27T15:20:29.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.016+0000 7fe0e499a700 1 --2- 172.21.15.133:0/3573978319 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe0cc0a1150 0x7fe0cc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:29.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.017+0000 7fe0e499a700 1 --2- 172.21.15.133:0/3573978319 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe0cc0a1150 0x7fe0cc0a3600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe0d00104e0 tx=0x7fe0d0007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:29.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.021+0000 7fe0d67fc700 1 -- 172.21.15.133:0/3573978319 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe0dc097770 con 0x7fe0e0100d20 2023-11-27T15:20:29.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.228+0000 7fe0e73ff700 1 -- 172.21.15.133:0/3573978319 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0e0061910 con 0x7fe0e0100d20 2023-11-27T15:20:29.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.229+0000 7fe0d67fc700 1 -- 172.21.15.133:0/3573978319 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+18458 (secure 0 0 0) 0x7fe0dc09ad90 con 0x7fe0e0100d20 2023-11-27T15:20:29.234 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:29.234 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":80,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:19:49.094979+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:20:29.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.231+0000 7fe0e73ff700 1 -- 172.21.15.133:0/3573978319 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe0cc0a1150 msgr2=0x7fe0cc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:29.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.231+0000 7fe0e73ff700 1 --2- 172.21.15.133:0/3573978319 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe0cc0a1150 0x7fe0cc0a3600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe0d00104e0 tx=0x7fe0d0007480 comp rx=0 tx=0).stop 2023-11-27T15:20:29.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.231+0000 7fe0e73ff700 1 -- 172.21.15.133:0/3573978319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0e0100d20 msgr2=0x7fe0e01961a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:29.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.231+0000 7fe0e73ff700 1 --2- 172.21.15.133:0/3573978319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0e0100d20 0x7fe0e01961a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc0062a0 tx=0x7fe0dc0094e0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.232+0000 7fe0e73ff700 1 -- 172.21.15.133:0/3573978319 shutdown_connections 2023-11-27T15:20:29.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.232+0000 7fe0e73ff700 1 --2- 172.21.15.133:0/3573978319 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe0cc0a1150 0x7fe0cc0a3600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.236 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.232+0000 7fe0e73ff700 1 --2- 172.21.15.133:0/3573978319 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe0e0105f60 0x7fe0e019d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.232+0000 7fe0e73ff700 1 --2- 172.21.15.133:0/3573978319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0e0100d20 0x7fe0e01961a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.232+0000 7fe0e73ff700 1 --2- 172.21.15.133:0/3573978319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe0e0103640 0x7fe0e01966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.232+0000 7fe0e73ff700 1 -- 172.21.15.133:0/3573978319 >> 172.21.15.133:0/3573978319 conn(0x7fe0e00fa7a0 msgr2=0x7fe0e0101980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:29.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.232+0000 7fe0e73ff700 1 -- 172.21.15.133:0/3573978319 shutdown_connections 2023-11-27T15:20:29.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.232+0000 7fe0e73ff700 1 -- 172.21.15.133:0/3573978319 wait complete. 2023-11-27T15:20:29.255 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:16:20.216150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-27T15:20:29.255 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:20:29.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.461+0000 7f346a9ff700 1 -- 172.21.15.133:0/1054449643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f346410c890 msgr2=0x7f346410ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:29.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.461+0000 7f346a9ff700 1 --2- 172.21.15.133:0/1054449643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f346410c890 0x7f346410ec70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f345c00b0a0 tx=0x7f345c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.461+0000 7f346a9ff700 1 -- 172.21.15.133:0/1054449643 shutdown_connections 2023-11-27T15:20:29.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.461+0000 7f346a9ff700 1 --2- 172.21.15.133:0/1054449643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f346410c890 0x7f346410ec70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.461+0000 7f346a9ff700 1 --2- 172.21.15.133:0/1054449643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3464074b30 0x7f3464072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.461+0000 7f346a9ff700 1 --2- 172.21.15.133:0/1054449643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3464073550 0x7f34640739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.461+0000 7f346a9ff700 1 -- 172.21.15.133:0/1054449643 >> 172.21.15.133:0/1054449643 conn(0x7f34640fb8a0 msgr2=0x7f34640fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:29.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.461+0000 7f346a9ff700 1 -- 172.21.15.133:0/1054449643 shutdown_connections 2023-11-27T15:20:29.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.461+0000 7f346a9ff700 1 -- 172.21.15.133:0/1054449643 wait complete. 2023-11-27T15:20:29.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.462+0000 7f346a9ff700 1 Processor -- start 2023-11-27T15:20:29.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.463+0000 7f346a9ff700 1 -- start start 2023-11-27T15:20:29.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.463+0000 7f346a9ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3464073550 0x7f346419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:29.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.463+0000 7f346a9ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3464074b30 0x7f346419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:29.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.463+0000 7f346a9ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f346410c890 0x7f34641a3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:29.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.463+0000 7f346a9ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34641a42e0 con 0x7f3464074b30 2023-11-27T15:20:29.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.463+0000 7f346a9ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34641a4420 con 0x7f346410c890 2023-11-27T15:20:29.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.463+0000 7f346a9ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34641a45b0 con 0x7f3464073550 2023-11-27T15:20:29.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.463+0000 7f3468f9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f346410c890 0x7f34641a3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:29.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.463+0000 7f34637fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3464074b30 0x7f346419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:29.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.463+0000 7f3463fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3464073550 0x7f346419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:29.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.463+0000 7f3468f9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f346410c890 0x7f34641a3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:52116/0 (socket says 172.21.15.133:52116) 2023-11-27T15:20:29.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.463+0000 7f3468f9c700 1 -- 172.21.15.133:0/2791405589 learned_addr learned my addr 172.21.15.133:0/2791405589 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:29.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.464+0000 7f3468f9c700 1 -- 172.21.15.133:0/2791405589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3464073550 msgr2=0x7f346419c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:29.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.464+0000 7f3468f9c700 1 --2- 172.21.15.133:0/2791405589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3464073550 0x7f346419c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.464+0000 7f3468f9c700 1 -- 172.21.15.133:0/2791405589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3464074b30 msgr2=0x7f346419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:29.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.464+0000 7f3468f9c700 1 --2- 172.21.15.133:0/2791405589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3464074b30 0x7f346419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.464+0000 7f3468f9c700 1 -- 172.21.15.133:0/2791405589 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345c009d00 con 0x7f346410c890 2023-11-27T15:20:29.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.464+0000 7f34637fe700 1 --2- 172.21.15.133:0/2791405589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3464074b30 0x7f346419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:29.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.464+0000 7f3463fff700 1 --2- 172.21.15.133:0/2791405589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3464073550 0x7f346419c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:29.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.464+0000 7f3468f9c700 1 --2- 172.21.15.133:0/2791405589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f346410c890 0x7f34641a3d10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f345c009fd0 tx=0x7f345c00bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:29.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.464+0000 7f34617fa700 1 -- 172.21.15.133:0/2791405589 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c00e040 con 0x7f346410c890 2023-11-27T15:20:29.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.464+0000 7f34617fa700 1 -- 172.21.15.133:0/2791405589 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f345c0094a0 con 0x7f346410c890 2023-11-27T15:20:29.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.464+0000 7f346a9ff700 1 -- 172.21.15.133:0/2791405589 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34641122a0 con 0x7f346410c890 2023-11-27T15:20:29.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.464+0000 7f34617fa700 1 -- 172.21.15.133:0/2791405589 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c01b6c0 con 0x7f346410c890 2023-11-27T15:20:29.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.464+0000 7f346a9ff700 1 -- 172.21.15.133:0/2791405589 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3464112110 con 0x7f346410c890 2023-11-27T15:20:29.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.465+0000 7f346a9ff700 1 -- 172.21.15.133:0/2791405589 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34641023e0 con 0x7f346410c890 2023-11-27T15:20:29.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.467+0000 7f34617fa700 1 -- 172.21.15.133:0/2791405589 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f345c01b820 con 0x7f346410c890 2023-11-27T15:20:29.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.468+0000 7f34617fa700 1 --2- 172.21.15.133:0/2791405589 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f344c0a1150 0x7f344c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:29.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.469+0000 7f34617fa700 1 -- 172.21.15.133:0/2791405589 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f345c0cd700 con 0x7f346410c890 2023-11-27T15:20:29.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.469+0000 7f3463fff700 1 --2- 172.21.15.133:0/2791405589 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f344c0a1150 0x7f344c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:29.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.469+0000 7f3463fff700 1 --2- 172.21.15.133:0/2791405589 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f344c0a1150 0x7f344c0a3600 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3464074f00 tx=0x7f3454009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:29.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.470+0000 7f34617fa700 1 -- 172.21.15.133:0/2791405589 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f345c097400 con 0x7f346410c890 2023-11-27T15:20:29.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:29 smithi133 ceph-mon[97860]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:29.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:29 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/535379929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:29.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:29 smithi177 ceph-mon[131504]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:29.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:29 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/535379929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:29.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:29 smithi174 ceph-mon[131110]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:29.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:29 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/535379929' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:29.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.699+0000 7f346a9ff700 1 -- 172.21.15.133:0/2791405589 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3464061700 con 0x7f346410c890 2023-11-27T15:20:29.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.700+0000 7f34617fa700 1 -- 172.21.15.133:0/2791405589 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f345c09aa20 con 0x7f346410c890 2023-11-27T15:20:29.704 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-11-27T15:20:29.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.702+0000 7f346a9ff700 1 -- 172.21.15.133:0/2791405589 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f344c0a1150 msgr2=0x7f344c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:29.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.702+0000 7f346a9ff700 1 --2- 172.21.15.133:0/2791405589 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f344c0a1150 0x7f344c0a3600 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3464074f00 tx=0x7f3454009260 comp rx=0 tx=0).stop 2023-11-27T15:20:29.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.702+0000 7f346a9ff700 1 -- 172.21.15.133:0/2791405589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f346410c890 msgr2=0x7f34641a3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:29.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.702+0000 7f346a9ff700 1 --2- 172.21.15.133:0/2791405589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f346410c890 0x7f34641a3d10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f345c009fd0 tx=0x7f345c00bce0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.702+0000 7f346a9ff700 1 -- 172.21.15.133:0/2791405589 shutdown_connections 2023-11-27T15:20:29.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.702+0000 7f346a9ff700 1 --2- 172.21.15.133:0/2791405589 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f344c0a1150 0x7f344c0a3600 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.702+0000 7f346a9ff700 1 --2- 172.21.15.133:0/2791405589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f346410c890 0x7f34641a3d10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.702+0000 7f346a9ff700 1 --2- 172.21.15.133:0/2791405589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3464073550 0x7f346419c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.702+0000 7f346a9ff700 1 --2- 172.21.15.133:0/2791405589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3464074b30 0x7f346419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.702+0000 7f346a9ff700 1 -- 172.21.15.133:0/2791405589 >> 172.21.15.133:0/2791405589 conn(0x7f34640fb8a0 msgr2=0x7f34640fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:29.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.702+0000 7f346a9ff700 1 -- 172.21.15.133:0/2791405589 shutdown_connections 2023-11-27T15:20:29.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.702+0000 7f346a9ff700 1 -- 172.21.15.133:0/2791405589 wait complete. 2023-11-27T15:20:29.726 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-27T15:20:29.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.929+0000 7ff90e1ff700 1 -- 172.21.15.133:0/4088893897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff908100d20 msgr2=0x7ff908103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:29.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.929+0000 7ff90e1ff700 1 --2- 172.21.15.133:0/4088893897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff908100d20 0x7ff908103100 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8009990 tx=0x7ff8f8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.929+0000 7ff90e1ff700 1 -- 172.21.15.133:0/4088893897 shutdown_connections 2023-11-27T15:20:29.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.929+0000 7ff90e1ff700 1 --2- 172.21.15.133:0/4088893897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff908105f60 0x7ff9081083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.929+0000 7ff90e1ff700 1 --2- 172.21.15.133:0/4088893897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff908100d20 0x7ff908103100 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.929+0000 7ff90e1ff700 1 --2- 172.21.15.133:0/4088893897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff908103640 0x7ff908105a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.929+0000 7ff90e1ff700 1 -- 172.21.15.133:0/4088893897 >> 172.21.15.133:0/4088893897 conn(0x7ff9080fa7a0 msgr2=0x7ff9080fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:29.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.929+0000 7ff90e1ff700 1 -- 172.21.15.133:0/4088893897 shutdown_connections 2023-11-27T15:20:29.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.929+0000 7ff90e1ff700 1 -- 172.21.15.133:0/4088893897 wait complete. 2023-11-27T15:20:29.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.930+0000 7ff90e1ff700 1 Processor -- start 2023-11-27T15:20:29.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.931+0000 7ff90e1ff700 1 -- start start 2023-11-27T15:20:29.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.931+0000 7ff90e1ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff908100d20 0x7ff908196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:29.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.931+0000 7ff90e1ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff908103640 0x7ff908196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:29.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.931+0000 7ff90e1ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff908105f60 0x7ff90819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:29.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.931+0000 7ff90e1ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90819db80 con 0x7ff908103640 2023-11-27T15:20:29.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.931+0000 7ff90e1ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90819dcc0 con 0x7ff908105f60 2023-11-27T15:20:29.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.931+0000 7ff90e1ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90819de50 con 0x7ff908100d20 2023-11-27T15:20:29.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.931+0000 7ff8fffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff908103640 0x7ff908196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:29.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.931+0000 7ff8fffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff908103640 0x7ff908196540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39414/0 (socket says 172.21.15.133:39414) 2023-11-27T15:20:29.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.931+0000 7ff8fffff700 1 -- 172.21.15.133:0/900112263 learned_addr learned my addr 172.21.15.133:0/900112263 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:29.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.931+0000 7ff9077fe700 1 --2- 172.21.15.133:0/900112263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff908100d20 0x7ff908196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.931+0000 7ff907fff700 1 --2- 172.21.15.133:0/900112263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff908105f60 0x7ff90819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.932+0000 7ff9077fe700 1 -- 172.21.15.133:0/900112263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff908105f60 msgr2=0x7ff90819d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.932+0000 7ff9077fe700 1 --2- 172.21.15.133:0/900112263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff908105f60 0x7ff90819d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.932+0000 7ff9077fe700 1 -- 172.21.15.133:0/900112263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff908103640 msgr2=0x7ff908196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.932+0000 7ff9077fe700 1 --2- 172.21.15.133:0/900112263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff908103640 0x7ff908196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.932+0000 7ff9077fe700 1 -- 172.21.15.133:0/900112263 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f80095f0 con 0x7ff908100d20 2023-11-27T15:20:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.932+0000 7ff9077fe700 1 --2- 172.21.15.133:0/900112263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff908100d20 0x7ff908196000 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff8f80098e0 tx=0x7ff8f8005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.932+0000 7ff9057fa700 1 -- 172.21.15.133:0/900112263 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f801d070 con 0x7ff908100d20 2023-11-27T15:20:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.932+0000 7ff9057fa700 1 -- 172.21.15.133:0/900112263 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8f8005030 con 0x7ff908100d20 2023-11-27T15:20:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.932+0000 7ff90e1ff700 1 -- 172.21.15.133:0/900112263 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90810bc80 con 0x7ff908100d20 2023-11-27T15:20:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.932+0000 7ff9057fa700 1 -- 172.21.15.133:0/900112263 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8004ab0 con 0x7ff908100d20 2023-11-27T15:20:29.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.932+0000 7ff90e1ff700 1 -- 172.21.15.133:0/900112263 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff90810baf0 con 0x7ff908100d20 2023-11-27T15:20:29.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.933+0000 7ff90e1ff700 1 -- 172.21.15.133:0/900112263 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9080fc310 con 0x7ff908100d20 2023-11-27T15:20:29.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.935+0000 7ff9057fa700 1 -- 172.21.15.133:0/900112263 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff8f80051a0 con 0x7ff908100d20 2023-11-27T15:20:29.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.936+0000 7ff9057fa700 1 --2- 172.21.15.133:0/900112263 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff8f40a1150 0x7ff8f40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:29.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.936+0000 7ff9057fa700 1 -- 172.21.15.133:0/900112263 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff8f80cd5f0 con 0x7ff908100d20 2023-11-27T15:20:29.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.937+0000 7ff8fffff700 1 --2- 172.21.15.133:0/900112263 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff8f40a1150 0x7ff8f40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:29.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.937+0000 7ff8fffff700 1 --2- 172.21.15.133:0/900112263 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff8f40a1150 0x7ff8f40a3600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009d20 tx=0x7ff8f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:29.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:29.940+0000 7ff9057fa700 1 -- 172.21.15.133:0/900112263 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff8f805a730 con 0x7ff908100d20 2023-11-27T15:20:30.169 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.165+0000 7ff90e1ff700 1 -- 172.21.15.133:0/900112263 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff90802cfe0 con 0x7ff908100d20 2023-11-27T15:20:30.170 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.166+0000 7ff9057fa700 1 -- 172.21.15.133:0/900112263 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff8f80971d0 con 0x7ff908100d20 2023-11-27T15:20:30.170 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:30.170 INFO:teuthology.orchestra.run.smithi133.stdout:[ 2023-11-27T15:20:30.170 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:20:30.170 INFO:teuthology.orchestra.run.smithi133.stdout: "poolnum": 1, 2023-11-27T15:20:30.170 INFO:teuthology.orchestra.run.smithi133.stdout: "poolname": ".mgr" 2023-11-27T15:20:30.171 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:20:30.171 INFO:teuthology.orchestra.run.smithi133.stdout:] 2023-11-27T15:20:30.173 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.169+0000 7ff90e1ff700 1 -- 172.21.15.133:0/900112263 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff8f40a1150 msgr2=0x7ff8f40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:30.173 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.169+0000 7ff90e1ff700 1 --2- 172.21.15.133:0/900112263 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff8f40a1150 0x7ff8f40a3600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009d20 tx=0x7ff8f0009260 comp rx=0 tx=0).stop 2023-11-27T15:20:30.173 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.169+0000 7ff90e1ff700 1 -- 172.21.15.133:0/900112263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff908100d20 msgr2=0x7ff908196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:30.173 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.169+0000 7ff90e1ff700 1 --2- 172.21.15.133:0/900112263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff908100d20 0x7ff908196000 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff8f80098e0 tx=0x7ff8f8005b00 comp rx=0 tx=0).stop 2023-11-27T15:20:30.173 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.169+0000 7ff90e1ff700 1 -- 172.21.15.133:0/900112263 shutdown_connections 2023-11-27T15:20:30.173 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.169+0000 7ff90e1ff700 1 --2- 172.21.15.133:0/900112263 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff8f40a1150 0x7ff8f40a3600 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:30.173 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.169+0000 7ff90e1ff700 1 --2- 172.21.15.133:0/900112263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff908105f60 0x7ff90819d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:30.173 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.169+0000 7ff90e1ff700 1 --2- 172.21.15.133:0/900112263 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff908100d20 0x7ff908196000 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:30.173 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.169+0000 7ff90e1ff700 1 --2- 172.21.15.133:0/900112263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff908103640 0x7ff908196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:30.173 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.169+0000 7ff90e1ff700 1 -- 172.21.15.133:0/900112263 >> 172.21.15.133:0/900112263 conn(0x7ff9080fa7a0 msgr2=0x7ff908101980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:30.174 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.169+0000 7ff90e1ff700 1 -- 172.21.15.133:0/900112263 shutdown_connections 2023-11-27T15:20:30.174 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.169+0000 7ff90e1ff700 1 -- 172.21.15.133:0/900112263 wait complete. 2023-11-27T15:20:30.192 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-27T15:20:30.193 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-27T15:20:30.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.396+0000 7fdce2fff700 1 -- 172.21.15.133:0/835848531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcdc101e40 msgr2=0x7fdcdc108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:30.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.396+0000 7fdce2fff700 1 --2- 172.21.15.133:0/835848531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcdc101e40 0x7fdcdc108700 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400b0a0 tx=0x7fdcd400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:20:30.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.396+0000 7fdce2fff700 1 -- 172.21.15.133:0/835848531 shutdown_connections 2023-11-27T15:20:30.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.396+0000 7fdce2fff700 1 --2- 172.21.15.133:0/835848531 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcdc101e40 0x7fdcdc108700 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:30.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.396+0000 7fdce2fff700 1 --2- 172.21.15.133:0/835848531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcdc1002c0 0x7fdcdc1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:30.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.396+0000 7fdce2fff700 1 --2- 172.21.15.133:0/835848531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcdc1014b0 0x7fdcdc101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:30.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.396+0000 7fdce2fff700 1 -- 172.21.15.133:0/835848531 >> 172.21.15.133:0/835848531 conn(0x7fdcdc0fb8a0 msgr2=0x7fdcdc0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:30.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.396+0000 7fdce2fff700 1 -- 172.21.15.133:0/835848531 shutdown_connections 2023-11-27T15:20:30.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.396+0000 7fdce2fff700 1 -- 172.21.15.133:0/835848531 wait complete. 2023-11-27T15:20:30.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.397+0000 7fdce2fff700 1 Processor -- start 2023-11-27T15:20:30.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.398+0000 7fdce2fff700 1 -- start start 2023-11-27T15:20:30.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.398+0000 7fdce2fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcdc1002c0 0x7fdcdc0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:30.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.398+0000 7fdce2fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcdc1014b0 0x7fdcdc072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:30.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.398+0000 7fdce2fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcdc101e40 0x7fdcdc073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:30.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.398+0000 7fdce2fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc073bc0 con 0x7fdcdc101e40 2023-11-27T15:20:30.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.398+0000 7fdce2fff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc073d30 con 0x7fdcdc1002c0 2023-11-27T15:20:30.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.398+0000 7fdce2fff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc19ff70 con 0x7fdcdc1014b0 2023-11-27T15:20:30.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.398+0000 7fdce0d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcdc1002c0 0x7fdcdc0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:30.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.398+0000 7fdce0d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcdc1002c0 0x7fdcdc0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:52146/0 (socket says 172.21.15.133:52146) 2023-11-27T15:20:30.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.398+0000 7fdce0d9b700 1 -- 172.21.15.133:0/1609820392 learned_addr learned my addr 172.21.15.133:0/1609820392 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:30.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.398+0000 7fdce159c700 1 --2- 172.21.15.133:0/1609820392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcdc101e40 0x7fdcdc073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:30.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.398+0000 7fdcdbfff700 1 --2- 172.21.15.133:0/1609820392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcdc1014b0 0x7fdcdc072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:30.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.398+0000 7fdce0d9b700 1 -- 172.21.15.133:0/1609820392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcdc1014b0 msgr2=0x7fdcdc072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:30.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.398+0000 7fdce0d9b700 1 --2- 172.21.15.133:0/1609820392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcdc1014b0 0x7fdcdc072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:30.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.398+0000 7fdce0d9b700 1 -- 172.21.15.133:0/1609820392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcdc101e40 msgr2=0x7fdcdc073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:30.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.398+0000 7fdce0d9b700 1 --2- 172.21.15.133:0/1609820392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcdc101e40 0x7fdcdc073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:30.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.398+0000 7fdce0d9b700 1 -- 172.21.15.133:0/1609820392 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd4009d00 con 0x7fdcdc1002c0 2023-11-27T15:20:30.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.399+0000 7fdce0d9b700 1 --2- 172.21.15.133:0/1609820392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcdc1002c0 0x7fdcdc0748a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000e970 tx=0x7fdcd000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:30.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.399+0000 7fdcd9ffb700 1 -- 172.21.15.133:0/1609820392 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0009790 con 0x7fdcdc1002c0 2023-11-27T15:20:30.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.399+0000 7fdcd9ffb700 1 -- 172.21.15.133:0/1609820392 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcd000c7e0 con 0x7fdcdc1002c0 2023-11-27T15:20:30.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.399+0000 7fdcd9ffb700 1 -- 172.21.15.133:0/1609820392 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0003c70 con 0x7fdcdc1002c0 2023-11-27T15:20:30.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.399+0000 7fdce2fff700 1 -- 172.21.15.133:0/1609820392 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcdc10bbc0 con 0x7fdcdc1002c0 2023-11-27T15:20:30.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.399+0000 7fdce2fff700 1 -- 172.21.15.133:0/1609820392 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcdc10ba30 con 0x7fdcdc1002c0 2023-11-27T15:20:30.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.400+0000 7fdce2fff700 1 -- 172.21.15.133:0/1609820392 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcdc1a0640 con 0x7fdcdc1002c0 2023-11-27T15:20:30.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.402+0000 7fdcd9ffb700 1 -- 172.21.15.133:0/1609820392 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fdcd000cc50 con 0x7fdcdc1002c0 2023-11-27T15:20:30.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.403+0000 7fdcd9ffb700 1 --2- 172.21.15.133:0/1609820392 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fdccc0a11d0 0x7fdccc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:30.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.404+0000 7fdcd9ffb700 1 -- 172.21.15.133:0/1609820392 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdcd0014070 con 0x7fdcdc1002c0 2023-11-27T15:20:30.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.404+0000 7fdcdbfff700 1 --2- 172.21.15.133:0/1609820392 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fdccc0a11d0 0x7fdccc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:30.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.404+0000 7fdcdbfff700 1 --2- 172.21.15.133:0/1609820392 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fdccc0a11d0 0x7fdccc0a3680 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009c60 tx=0x7fdcc8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:30.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.407+0000 7fdcd9ffb700 1 -- 172.21.15.133:0/1609820392 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdcd0096170 con 0x7fdcdc1002c0 2023-11-27T15:20:30.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:30 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3573978319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:20:30.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:30 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:30.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:30 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:30.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:30 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:30.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:30 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:30.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:30 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2791405589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:20:30.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:30 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3573978319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:20:30.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:30 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:30.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:30 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:30.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:30 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:30.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:30 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:30.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:30 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2791405589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:20:30.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:30.621+0000 7fdce2fff700 1 -- 172.21.15.133:0/1609820392 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fdcdc107110 con 0x7fdcdc1002c0 2023-11-27T15:20:30.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:30 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3573978319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:20:30.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:30 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:30.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:30 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:30.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:30 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:30.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:30 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:30.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:30 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2791405589' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:20:31.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:31 smithi133 ceph-mon[97860]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:31.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:31 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/900112263' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-11-27T15:20:31.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:31 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:31.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:31 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1609820392' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-27T15:20:31.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:31 smithi133 ceph-mon[97860]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-27T15:20:31.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:31 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:31.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:31 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:31.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:31 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:31.528 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:31 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:31.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:31 smithi177 ceph-mon[131504]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:31.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:31 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/900112263' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-11-27T15:20:31.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:31 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:31.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:31 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1609820392' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-27T15:20:31.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:31 smithi177 ceph-mon[131504]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-27T15:20:31.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:31 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:31.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:31 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:31.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:31 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:31.589 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:31 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:31.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.626+0000 7fdcd9ffb700 1 -- 172.21.15.133:0/1609820392 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v81) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdcd0003dd0 con 0x7fdcdc1002c0 2023-11-27T15:20:31.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.632+0000 7fdce2fff700 1 -- 172.21.15.133:0/1609820392 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fdccc0a11d0 msgr2=0x7fdccc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:31.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.632+0000 7fdce2fff700 1 --2- 172.21.15.133:0/1609820392 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fdccc0a11d0 0x7fdccc0a3680 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009c60 tx=0x7fdcc8009260 comp rx=0 tx=0).stop 2023-11-27T15:20:31.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.632+0000 7fdce2fff700 1 -- 172.21.15.133:0/1609820392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcdc1002c0 msgr2=0x7fdcdc0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:31.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.632+0000 7fdce2fff700 1 --2- 172.21.15.133:0/1609820392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcdc1002c0 0x7fdcdc0748a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000e970 tx=0x7fdcd000ec80 comp rx=0 tx=0).stop 2023-11-27T15:20:31.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.633+0000 7fdce2fff700 1 -- 172.21.15.133:0/1609820392 shutdown_connections 2023-11-27T15:20:31.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.633+0000 7fdce2fff700 1 --2- 172.21.15.133:0/1609820392 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fdccc0a11d0 0x7fdccc0a3680 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:31.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.633+0000 7fdce2fff700 1 --2- 172.21.15.133:0/1609820392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdcdc101e40 0x7fdcdc073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:31.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.633+0000 7fdce2fff700 1 --2- 172.21.15.133:0/1609820392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcdc1002c0 0x7fdcdc0748a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:31.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.633+0000 7fdce2fff700 1 --2- 172.21.15.133:0/1609820392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcdc1014b0 0x7fdcdc072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:31.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.633+0000 7fdce2fff700 1 -- 172.21.15.133:0/1609820392 >> 172.21.15.133:0/1609820392 conn(0x7fdcdc0fb8a0 msgr2=0x7fdcdc0fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:31.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.633+0000 7fdce2fff700 1 -- 172.21.15.133:0/1609820392 shutdown_connections 2023-11-27T15:20:31.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.633+0000 7fdce2fff700 1 -- 172.21.15.133:0/1609820392 wait complete. 2023-11-27T15:20:31.638 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' created 2023-11-27T15:20:31.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:31 smithi174 ceph-mon[131110]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:31.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:31 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/900112263' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-11-27T15:20:31.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:31 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:31.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:31 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1609820392' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-27T15:20:31.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:31 smithi174 ceph-mon[131110]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-11-27T15:20:31.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:31 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:31.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:31 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:31.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:31 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:31.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:31 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:31.656 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-11-27T15:20:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.876+0000 7fd24dfff700 1 -- 172.21.15.133:0/2975272369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd248073550 msgr2=0x7fd2480739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.876+0000 7fd24dfff700 1 --2- 172.21.15.133:0/2975272369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd248073550 0x7fd2480739c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009990 tx=0x7fd23c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.877+0000 7fd24dfff700 1 -- 172.21.15.133:0/2975272369 shutdown_connections 2023-11-27T15:20:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.877+0000 7fd24dfff700 1 --2- 172.21.15.133:0/2975272369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd248108180 0x7fd24810a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.877+0000 7fd24dfff700 1 --2- 172.21.15.133:0/2975272369 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd248074b30 0x7fd248072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.877+0000 7fd24dfff700 1 --2- 172.21.15.133:0/2975272369 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd248073550 0x7fd2480739c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.877+0000 7fd24dfff700 1 -- 172.21.15.133:0/2975272369 >> 172.21.15.133:0/2975272369 conn(0x7fd2480fb8a0 msgr2=0x7fd2480fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.877+0000 7fd24dfff700 1 -- 172.21.15.133:0/2975272369 shutdown_connections 2023-11-27T15:20:31.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.877+0000 7fd24dfff700 1 -- 172.21.15.133:0/2975272369 wait complete. 2023-11-27T15:20:31.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.878+0000 7fd24dfff700 1 Processor -- start 2023-11-27T15:20:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.878+0000 7fd24dfff700 1 -- start start 2023-11-27T15:20:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.878+0000 7fd24dfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd248073550 0x7fd2481982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.878+0000 7fd24dfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd248074b30 0x7fd2481987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.878+0000 7fd24dfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd248108180 0x7fd24819f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.878+0000 7fd24dfff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24819fe20 con 0x7fd248108180 2023-11-27T15:20:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.878+0000 7fd24dfff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24819ff60 con 0x7fd248073550 2023-11-27T15:20:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.878+0000 7fd24dfff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2481a00f0 con 0x7fd248074b30 2023-11-27T15:20:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.878+0000 7fd246ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd248074b30 0x7fd2481987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:31.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.878+0000 7fd246ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd248074b30 0x7fd2481987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:35936/0 (socket says 172.21.15.133:35936) 2023-11-27T15:20:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.878+0000 7fd246ffd700 1 -- 172.21.15.133:0/1470094825 learned_addr learned my addr 172.21.15.133:0/1470094825 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.878+0000 7fd2477fe700 1 --2- 172.21.15.133:0/1470094825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd248073550 0x7fd2481982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.878+0000 7fd247fff700 1 --2- 172.21.15.133:0/1470094825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd248108180 0x7fd24819f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.879+0000 7fd246ffd700 1 -- 172.21.15.133:0/1470094825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd248073550 msgr2=0x7fd2481982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.879+0000 7fd246ffd700 1 --2- 172.21.15.133:0/1470094825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd248073550 0x7fd2481982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.879+0000 7fd246ffd700 1 -- 172.21.15.133:0/1470094825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd248108180 msgr2=0x7fd24819f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.879+0000 7fd246ffd700 1 --2- 172.21.15.133:0/1470094825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd248108180 0x7fd24819f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.879+0000 7fd246ffd700 1 -- 172.21.15.133:0/1470094825 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23c0095f0 con 0x7fd248074b30 2023-11-27T15:20:31.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.879+0000 7fd246ffd700 1 --2- 172.21.15.133:0/1470094825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd248074b30 0x7fd2481987e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd23c004710 tx=0x7fd23c009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:31.886 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.879+0000 7fd244ff9700 1 -- 172.21.15.133:0/1470094825 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c01d070 con 0x7fd248074b30 2023-11-27T15:20:31.886 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.879+0000 7fd24dfff700 1 -- 172.21.15.133:0/1470094825 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24810dea0 con 0x7fd248074b30 2023-11-27T15:20:31.886 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.879+0000 7fd244ff9700 1 -- 172.21.15.133:0/1470094825 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd23c021410 con 0x7fd248074b30 2023-11-27T15:20:31.886 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.879+0000 7fd244ff9700 1 -- 172.21.15.133:0/1470094825 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c017650 con 0x7fd248074b30 2023-11-27T15:20:31.886 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.879+0000 7fd24dfff700 1 -- 172.21.15.133:0/1470094825 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24810dd10 con 0x7fd248074b30 2023-11-27T15:20:31.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.880+0000 7fd24dfff700 1 -- 172.21.15.133:0/1470094825 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd248106830 con 0x7fd248074b30 2023-11-27T15:20:31.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.882+0000 7fd244ff9700 1 -- 172.21.15.133:0/1470094825 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd23c021580 con 0x7fd248074b30 2023-11-27T15:20:31.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.883+0000 7fd244ff9700 1 --2- 172.21.15.133:0/1470094825 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd2300a1150 0x7fd2300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:31.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.884+0000 7fd244ff9700 1 -- 172.21.15.133:0/1470094825 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fd23c0cdbf0 con 0x7fd248074b30 2023-11-27T15:20:31.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.884+0000 7fd2477fe700 1 --2- 172.21.15.133:0/1470094825 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd2300a1150 0x7fd2300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:31.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.884+0000 7fd2477fe700 1 --2- 172.21.15.133:0/1470094825 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd2300a1150 0x7fd2300a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd248074f00 tx=0x7fd238009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:31.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:31.886+0000 7fd244ff9700 1 -- 172.21.15.133:0/1470094825 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd23c097660 con 0x7fd248074b30 2023-11-27T15:20:32.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.098+0000 7fd24dfff700 1 -- 172.21.15.133:0/1470094825 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fd24802cfe0 con 0x7fd248074b30 2023-11-27T15:20:32.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.644+0000 7fd244ff9700 1 -- 172.21.15.133:0/1470094825 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v82) v1 ==== 171+0+0 (secure 0 0 0) 0x7fd23c09ac80 con 0x7fd248074b30 2023-11-27T15:20:32.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.653+0000 7fd24dfff700 1 -- 172.21.15.133:0/1470094825 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd2300a1150 msgr2=0x7fd2300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.653+0000 7fd24dfff700 1 --2- 172.21.15.133:0/1470094825 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd2300a1150 0x7fd2300a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd248074f00 tx=0x7fd238009260 comp rx=0 tx=0).stop 2023-11-27T15:20:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.654+0000 7fd24dfff700 1 -- 172.21.15.133:0/1470094825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd248074b30 msgr2=0x7fd2481987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.654+0000 7fd24dfff700 1 --2- 172.21.15.133:0/1470094825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd248074b30 0x7fd2481987e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd23c004710 tx=0x7fd23c009580 comp rx=0 tx=0).stop 2023-11-27T15:20:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.654+0000 7fd24dfff700 1 -- 172.21.15.133:0/1470094825 shutdown_connections 2023-11-27T15:20:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.654+0000 7fd24dfff700 1 --2- 172.21.15.133:0/1470094825 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd2300a1150 0x7fd2300a3600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.654+0000 7fd24dfff700 1 --2- 172.21.15.133:0/1470094825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd248108180 0x7fd24819f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.654+0000 7fd24dfff700 1 --2- 172.21.15.133:0/1470094825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd248073550 0x7fd2481982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:32.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.654+0000 7fd24dfff700 1 --2- 172.21.15.133:0/1470094825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd248074b30 0x7fd2481987e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:32.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.654+0000 7fd24dfff700 1 -- 172.21.15.133:0/1470094825 >> 172.21.15.133:0/1470094825 conn(0x7fd2480fb8a0 msgr2=0x7fd2480fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:32.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.655+0000 7fd24dfff700 1 -- 172.21.15.133:0/1470094825 shutdown_connections 2023-11-27T15:20:32.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.655+0000 7fd24dfff700 1 -- 172.21.15.133:0/1470094825 wait complete. 2023-11-27T15:20:32.662 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' created 2023-11-27T15:20:32.684 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-27T15:20:32.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.895+0000 7f24a3dff700 1 -- 172.21.15.133:0/3315847458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f249c074ad0 msgr2=0x7f249c072f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:32.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.895+0000 7f24a3dff700 1 --2- 172.21.15.133:0/3315847458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f249c074ad0 0x7f249c072f20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2498009990 tx=0x7f2498009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:32.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.895+0000 7f24a3dff700 1 -- 172.21.15.133:0/3315847458 shutdown_connections 2023-11-27T15:20:32.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.895+0000 7f24a3dff700 1 --2- 172.21.15.133:0/3315847458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f249c108160 0x7f249c10a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:32.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.895+0000 7f24a3dff700 1 --2- 172.21.15.133:0/3315847458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f249c074ad0 0x7f249c072f20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:32.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.895+0000 7f24a3dff700 1 --2- 172.21.15.133:0/3315847458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f249c0734f0 0x7f249c073960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:32.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.895+0000 7f24a3dff700 1 -- 172.21.15.133:0/3315847458 >> 172.21.15.133:0/3315847458 conn(0x7f249c0fb8b0 msgr2=0x7f249c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:32.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.895+0000 7f24a3dff700 1 -- 172.21.15.133:0/3315847458 shutdown_connections 2023-11-27T15:20:32.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.895+0000 7f24a3dff700 1 -- 172.21.15.133:0/3315847458 wait complete. 2023-11-27T15:20:32.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.896+0000 7f24a3dff700 1 Processor -- start 2023-11-27T15:20:32.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.897+0000 7f24a3dff700 1 -- start start 2023-11-27T15:20:32.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.897+0000 7f24a3dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f249c0734f0 0x7f249c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:32.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.897+0000 7f24a3dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f249c074ad0 0x7f249c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:32.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.897+0000 7f24a3dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f249c108160 0x7f249c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:32.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.897+0000 7f24a3dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c19fde0 con 0x7f249c0734f0 2023-11-27T15:20:32.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.897+0000 7f24a3dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c19ff20 con 0x7f249c074ad0 2023-11-27T15:20:32.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.897+0000 7f24a3dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f249c1a00b0 con 0x7f249c108160 2023-11-27T15:20:32.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.897+0000 7f24a139a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f249c074ad0 0x7f249c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:32.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.897+0000 7f24a139a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f249c074ad0 0x7f249c1987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:57034/0 (socket says 172.21.15.133:57034) 2023-11-27T15:20:32.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.897+0000 7f24a139a700 1 -- 172.21.15.133:0/1732652793 learned_addr learned my addr 172.21.15.133:0/1732652793 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:32.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.897+0000 7f24a1b9b700 1 --2- 172.21.15.133:0/1732652793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f249c0734f0 0x7f249c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:32.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.897+0000 7f24a239c700 1 --2- 172.21.15.133:0/1732652793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f249c108160 0x7f249c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:32.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.898+0000 7f24a139a700 1 -- 172.21.15.133:0/1732652793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f249c108160 msgr2=0x7f249c19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:32.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.898+0000 7f24a139a700 1 --2- 172.21.15.133:0/1732652793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f249c108160 0x7f249c19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:32.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.898+0000 7f24a139a700 1 -- 172.21.15.133:0/1732652793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f249c0734f0 msgr2=0x7f249c198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:32.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.898+0000 7f24a139a700 1 --2- 172.21.15.133:0/1732652793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f249c0734f0 0x7f249c198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:32.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.898+0000 7f24a139a700 1 -- 172.21.15.133:0/1732652793 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248c009520 con 0x7f249c074ad0 2023-11-27T15:20:32.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.898+0000 7f24a239c700 1 --2- 172.21.15.133:0/1732652793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f249c108160 0x7f249c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:32.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.898+0000 7f24a1b9b700 1 --2- 172.21.15.133:0/1732652793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f249c0734f0 0x7f249c198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:32.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.898+0000 7f24a139a700 1 --2- 172.21.15.133:0/1732652793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f249c074ad0 0x7f249c1987a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f248c00eb30 tx=0x7f248c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:32.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:32 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:20:32.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:32 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:20:32.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:32 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:20:32.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:32 smithi174 ceph-mon[131110]: Service mds.all has no related daemon already placed. 2023-11-27T15:20:32.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:32 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:32.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:32 smithi174 ceph-mon[131110]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-11-27T15:20:32.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:32 smithi174 ceph-mon[131110]: osdmap e81: 12 total, 12 up, 12 in 2023-11-27T15:20:32.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:32 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1470094825' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-27T15:20:32.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.898+0000 7f2492ffd700 1 -- 172.21.15.133:0/1732652793 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c009880 con 0x7f249c074ad0 2023-11-27T15:20:32.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.898+0000 7f24a3dff700 1 -- 172.21.15.133:0/1732652793 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24980095f0 con 0x7f249c074ad0 2023-11-27T15:20:32.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.898+0000 7f2492ffd700 1 -- 172.21.15.133:0/1732652793 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f248c0044e0 con 0x7f249c074ad0 2023-11-27T15:20:32.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.898+0000 7f2492ffd700 1 -- 172.21.15.133:0/1732652793 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c003d80 con 0x7f249c074ad0 2023-11-27T15:20:32.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.898+0000 7f24a3dff700 1 -- 172.21.15.133:0/1732652793 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249c10de60 con 0x7f249c074ad0 2023-11-27T15:20:32.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.899+0000 7f24a3dff700 1 -- 172.21.15.133:0/1732652793 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f249c10dff0 con 0x7f249c074ad0 2023-11-27T15:20:32.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.902+0000 7f2492ffd700 1 -- 172.21.15.133:0/1732652793 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f248c003f20 con 0x7f249c074ad0 2023-11-27T15:20:32.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.903+0000 7f2492ffd700 1 --2- 172.21.15.133:0/1732652793 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f24880a1150 0x7f24880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:32.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.904+0000 7f2492ffd700 1 -- 172.21.15.133:0/1732652793 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f248c014070 con 0x7f249c074ad0 2023-11-27T15:20:32.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.904+0000 7f24a1b9b700 1 --2- 172.21.15.133:0/1732652793 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f24880a1150 0x7f24880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:32.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.904+0000 7f24a1b9b700 1 --2- 172.21.15.133:0/1732652793 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f24880a1150 0x7f24880a3600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f249800b2e0 tx=0x7f2498005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:32.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:32.907+0000 7f2492ffd700 1 -- 172.21.15.133:0/1732652793 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f248c096590 con 0x7f249c074ad0 2023-11-27T15:20:33.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:32 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:20:33.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:32 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:20:33.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:32 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:20:33.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:32 smithi133 ceph-mon[97860]: Service mds.all has no related daemon already placed. 2023-11-27T15:20:33.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:32 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:33.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:32 smithi133 ceph-mon[97860]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-11-27T15:20:33.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:32 smithi133 ceph-mon[97860]: osdmap e81: 12 total, 12 up, 12 in 2023-11-27T15:20:33.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:32 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1470094825' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-27T15:20:33.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:32 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:20:33.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:32 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:20:33.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:32 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:20:33.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:32 smithi177 ceph-mon[131504]: Service mds.all has no related daemon already placed. 2023-11-27T15:20:33.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:32 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:20:33.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:32 smithi177 ceph-mon[131504]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-11-27T15:20:33.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:32 smithi177 ceph-mon[131504]: osdmap e81: 12 total, 12 up, 12 in 2023-11-27T15:20:33.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:32 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1470094825' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-27T15:20:33.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.198+0000 7f24a3dff700 1 -- 172.21.15.133:0/1732652793 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f249c1068a0 con 0x7f249c074ad0 2023-11-27T15:20:33.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.208+0000 7f2492ffd700 1 -- 172.21.15.133:0/1732652793 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f248c094e60 con 0x7f249c074ad0 2023-11-27T15:20:33.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.214+0000 7f24a3dff700 1 -- 172.21.15.133:0/1732652793 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f24880a1150 msgr2=0x7f24880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:33.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.214+0000 7f24a3dff700 1 --2- 172.21.15.133:0/1732652793 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f24880a1150 0x7f24880a3600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f249800b2e0 tx=0x7f2498005fb0 comp rx=0 tx=0).stop 2023-11-27T15:20:33.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.214+0000 7f24a3dff700 1 -- 172.21.15.133:0/1732652793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f249c074ad0 msgr2=0x7f249c1987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:33.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.214+0000 7f24a3dff700 1 --2- 172.21.15.133:0/1732652793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f249c074ad0 0x7f249c1987a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f248c00eb30 tx=0x7f248c00ee40 comp rx=0 tx=0).stop 2023-11-27T15:20:33.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.215+0000 7f24a3dff700 1 -- 172.21.15.133:0/1732652793 shutdown_connections 2023-11-27T15:20:33.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.215+0000 7f24a3dff700 1 --2- 172.21.15.133:0/1732652793 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f24880a1150 0x7f24880a3600 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:33.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.215+0000 7f24a3dff700 1 --2- 172.21.15.133:0/1732652793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f249c108160 0x7f249c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:33.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.215+0000 7f24a3dff700 1 --2- 172.21.15.133:0/1732652793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f249c0734f0 0x7f249c198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:33.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.215+0000 7f24a3dff700 1 --2- 172.21.15.133:0/1732652793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f249c074ad0 0x7f249c1987a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:33.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.215+0000 7f24a3dff700 1 -- 172.21.15.133:0/1732652793 >> 172.21.15.133:0/1732652793 conn(0x7f249c0fb8b0 msgr2=0x7f249c0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:33.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.216+0000 7f24a3dff700 1 -- 172.21.15.133:0/1732652793 shutdown_connections 2023-11-27T15:20:33.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.216+0000 7f24a3dff700 1 -- 172.21.15.133:0/1732652793 wait complete. 2023-11-27T15:20:33.223 INFO:teuthology.orchestra.run.smithi133.stderr:new fs with metadata pool 2 and data pool 3 2023-11-27T15:20:33.254 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-11-27T15:20:33.255 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-11-27T15:20:33.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.462+0000 7f86929ff700 1 -- 172.21.15.133:0/2122431383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868c073550 msgr2=0x7f868c0739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:33.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.462+0000 7f86929ff700 1 --2- 172.21.15.133:0/2122431383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868c073550 0x7f868c0739c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8680009990 tx=0x7f8680009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:33.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.463+0000 7f86929ff700 1 -- 172.21.15.133:0/2122431383 shutdown_connections 2023-11-27T15:20:33.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.463+0000 7f86929ff700 1 --2- 172.21.15.133:0/2122431383 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f868c10c8a0 0x7f868c10ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:33.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.463+0000 7f86929ff700 1 --2- 172.21.15.133:0/2122431383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f868c074b30 0x7f868c072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:33.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.463+0000 7f86929ff700 1 --2- 172.21.15.133:0/2122431383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868c073550 0x7f868c0739c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:33.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.463+0000 7f86929ff700 1 -- 172.21.15.133:0/2122431383 >> 172.21.15.133:0/2122431383 conn(0x7f868c0fb8b0 msgr2=0x7f868c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:33.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.463+0000 7f86929ff700 1 -- 172.21.15.133:0/2122431383 shutdown_connections 2023-11-27T15:20:33.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.463+0000 7f86929ff700 1 -- 172.21.15.133:0/2122431383 wait complete. 2023-11-27T15:20:33.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.464+0000 7f86929ff700 1 Processor -- start 2023-11-27T15:20:33.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.464+0000 7f86929ff700 1 -- start start 2023-11-27T15:20:33.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.464+0000 7f86929ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f868c073550 0x7f868c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:33.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.464+0000 7f86929ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868c074b30 0x7f868c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:33.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.464+0000 7f86929ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f868c10c8a0 0x7f868c1a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:33.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.464+0000 7f86929ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c1a4250 con 0x7f868c074b30 2023-11-27T15:20:33.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.464+0000 7f86929ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c1a4390 con 0x7f868c073550 2023-11-27T15:20:33.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.464+0000 7f86929ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c1a4520 con 0x7f868c10c8a0 2023-11-27T15:20:33.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.464+0000 7f868bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f868c073550 0x7f868c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:33.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.464+0000 7f868bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f868c073550 0x7f868c19c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:57048/0 (socket says 172.21.15.133:57048) 2023-11-27T15:20:33.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.464+0000 7f868bfff700 1 -- 172.21.15.133:0/3649278485 learned_addr learned my addr 172.21.15.133:0/3649278485 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:33.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.464+0000 7f868b7fe700 1 --2- 172.21.15.133:0/3649278485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868c074b30 0x7f868c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:33.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.464+0000 7f8690f9c700 1 --2- 172.21.15.133:0/3649278485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f868c10c8a0 0x7f868c1a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:33.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.465+0000 7f868bfff700 1 -- 172.21.15.133:0/3649278485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f868c10c8a0 msgr2=0x7f868c1a3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:33.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.465+0000 7f868bfff700 1 --2- 172.21.15.133:0/3649278485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f868c10c8a0 0x7f868c1a3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:33.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.465+0000 7f868bfff700 1 -- 172.21.15.133:0/3649278485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868c074b30 msgr2=0x7f868c19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:33.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.465+0000 7f868bfff700 1 --2- 172.21.15.133:0/3649278485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868c074b30 0x7f868c19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:33.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.465+0000 7f868bfff700 1 -- 172.21.15.133:0/3649278485 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86800095f0 con 0x7f868c073550 2023-11-27T15:20:33.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.465+0000 7f8690f9c700 1 --2- 172.21.15.133:0/3649278485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f868c10c8a0 0x7f868c1a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:20:33.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.465+0000 7f868bfff700 1 --2- 172.21.15.133:0/3649278485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f868c073550 0x7f868c19c6d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f867c00e970 tx=0x7f867c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:33.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.465+0000 7f86897fa700 1 -- 172.21.15.133:0/3649278485 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c009790 con 0x7f868c073550 2023-11-27T15:20:33.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.465+0000 7f86929ff700 1 -- 172.21.15.133:0/3649278485 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868c1122b0 con 0x7f868c073550 2023-11-27T15:20:33.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.465+0000 7f86897fa700 1 -- 172.21.15.133:0/3649278485 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f867c00eec0 con 0x7f868c073550 2023-11-27T15:20:33.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.465+0000 7f86897fa700 1 -- 172.21.15.133:0/3649278485 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c005590 con 0x7f868c073550 2023-11-27T15:20:33.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.465+0000 7f86929ff700 1 -- 172.21.15.133:0/3649278485 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868c112120 con 0x7f868c073550 2023-11-27T15:20:33.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.466+0000 7f86929ff700 1 -- 172.21.15.133:0/3649278485 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f868c1023f0 con 0x7f868c073550 2023-11-27T15:20:33.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.469+0000 7f86897fa700 1 -- 172.21.15.133:0/3649278485 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f867c00cbd0 con 0x7f868c073550 2023-11-27T15:20:33.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.470+0000 7f86897fa700 1 --2- 172.21.15.133:0/3649278485 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f86740a1150 0x7f86740a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:33.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.471+0000 7f86897fa700 1 -- 172.21.15.133:0/3649278485 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f867c014070 con 0x7f868c073550 2023-11-27T15:20:33.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.471+0000 7f868b7fe700 1 --2- 172.21.15.133:0/3649278485 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f86740a1150 0x7f86740a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:33.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.471+0000 7f868b7fe700 1 --2- 172.21.15.133:0/3649278485 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f86740a1150 0x7f86740a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f868000b2e0 tx=0x7f8680005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:33.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.471+0000 7f86897fa700 1 -- 172.21.15.133:0/3649278485 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f867c096930 con 0x7f868c073550 2023-11-27T15:20:33.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:33.695+0000 7f86929ff700 1 -- 172.21.15.133:0/3649278485 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f868c02cfe0 con 0x7f868c073550 2023-11-27T15:20:34.632 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:33 smithi177 ceph-mon[131504]: pgmap v266: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:34.632 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:33 smithi177 ceph-mon[131504]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-27T15:20:34.632 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:33 smithi177 ceph-mon[131504]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-11-27T15:20:34.632 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:33 smithi177 ceph-mon[131504]: osdmap e82: 12 total, 12 up, 12 in 2023-11-27T15:20:34.632 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:33 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1732652793' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-27T15:20:34.632 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:33 smithi177 ceph-mon[131504]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-27T15:20:34.632 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:33 smithi177 ceph-mon[131504]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-11-27T15:20:34.632 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:33 smithi177 ceph-mon[131504]: osdmap e83: 12 total, 12 up, 12 in 2023-11-27T15:20:34.632 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:33 smithi177 ceph-mon[131504]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-11-27T15:20:34.632 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:33 smithi177 ceph-mon[131504]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-11-27T15:20:34.632 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:33 smithi177 ceph-mon[131504]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-11-27T15:20:34.633 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:33 smithi174 ceph-mon[131110]: pgmap v266: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:34.633 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:33 smithi174 ceph-mon[131110]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-27T15:20:34.633 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:33 smithi174 ceph-mon[131110]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-11-27T15:20:34.633 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:33 smithi174 ceph-mon[131110]: osdmap e82: 12 total, 12 up, 12 in 2023-11-27T15:20:34.633 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:33 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1732652793' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-27T15:20:34.633 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:33 smithi174 ceph-mon[131110]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-27T15:20:34.633 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:33 smithi174 ceph-mon[131110]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-11-27T15:20:34.633 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:33 smithi174 ceph-mon[131110]: osdmap e83: 12 total, 12 up, 12 in 2023-11-27T15:20:34.633 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:33 smithi174 ceph-mon[131110]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-11-27T15:20:34.633 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:33 smithi174 ceph-mon[131110]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-11-27T15:20:34.633 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:33 smithi174 ceph-mon[131110]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-11-27T15:20:34.633 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:33 smithi133 ceph-mon[97860]: pgmap v266: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:34.633 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:33 smithi133 ceph-mon[97860]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-11-27T15:20:34.634 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:33 smithi133 ceph-mon[97860]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-11-27T15:20:34.634 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:33 smithi133 ceph-mon[97860]: osdmap e82: 12 total, 12 up, 12 in 2023-11-27T15:20:34.634 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:33 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1732652793' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-27T15:20:34.634 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:33 smithi133 ceph-mon[97860]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-11-27T15:20:34.634 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:33 smithi133 ceph-mon[97860]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-11-27T15:20:34.634 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:33 smithi133 ceph-mon[97860]: osdmap e83: 12 total, 12 up, 12 in 2023-11-27T15:20:34.634 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:33 smithi133 ceph-mon[97860]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-11-27T15:20:34.634 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:33 smithi133 ceph-mon[97860]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-11-27T15:20:34.634 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:33 smithi133 ceph-mon[97860]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-11-27T15:20:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.213+0000 7f86897fa700 1 -- 172.21.15.133:0/3649278485 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v84) v1 ==== 157+0+0 (secure 0 0 0) 0x7f867c004ef0 con 0x7f868c073550 2023-11-27T15:20:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.216+0000 7f86929ff700 1 -- 172.21.15.133:0/3649278485 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f86740a1150 msgr2=0x7f86740a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.216+0000 7f86929ff700 1 --2- 172.21.15.133:0/3649278485 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f86740a1150 0x7f86740a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f868000b2e0 tx=0x7f8680005fb0 comp rx=0 tx=0).stop 2023-11-27T15:20:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.216+0000 7f86929ff700 1 -- 172.21.15.133:0/3649278485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f868c073550 msgr2=0x7f868c19c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.216+0000 7f86929ff700 1 --2- 172.21.15.133:0/3649278485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f868c073550 0x7f868c19c6d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f867c00e970 tx=0x7f867c00ec80 comp rx=0 tx=0).stop 2023-11-27T15:20:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.216+0000 7f86929ff700 1 -- 172.21.15.133:0/3649278485 shutdown_connections 2023-11-27T15:20:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.216+0000 7f86929ff700 1 --2- 172.21.15.133:0/3649278485 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f86740a1150 0x7f86740a3600 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.216+0000 7f86929ff700 1 --2- 172.21.15.133:0/3649278485 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f868c10c8a0 0x7f868c1a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.216+0000 7f86929ff700 1 --2- 172.21.15.133:0/3649278485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f868c073550 0x7f868c19c6d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.216+0000 7f86929ff700 1 --2- 172.21.15.133:0/3649278485 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f868c074b30 0x7f868c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.216+0000 7f86929ff700 1 -- 172.21.15.133:0/3649278485 >> 172.21.15.133:0/3649278485 conn(0x7f868c0fb8b0 msgr2=0x7f868c0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.216+0000 7f86929ff700 1 -- 172.21.15.133:0/3649278485 shutdown_connections 2023-11-27T15:20:34.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.216+0000 7f86929ff700 1 -- 172.21.15.133:0/3649278485 wait complete. 2023-11-27T15:20:34.637 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-11-27T15:20:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.846+0000 7f667efff700 1 -- 172.21.15.133:0/240639704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6678074b30 msgr2=0x7f6678072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.846+0000 7f667efff700 1 --2- 172.21.15.133:0/240639704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6678074b30 0x7f6678072f80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f666c009990 tx=0x7f666c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.847+0000 7f667efff700 1 -- 172.21.15.133:0/240639704 shutdown_connections 2023-11-27T15:20:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.847+0000 7f667efff700 1 --2- 172.21.15.133:0/240639704 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f667810c8b0 0x7f667810ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.847+0000 7f667efff700 1 --2- 172.21.15.133:0/240639704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6678074b30 0x7f6678072f80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.847+0000 7f667efff700 1 --2- 172.21.15.133:0/240639704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6678073550 0x7f66780739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.847+0000 7f667efff700 1 -- 172.21.15.133:0/240639704 >> 172.21.15.133:0/240639704 conn(0x7f66780fb8c0 msgr2=0x7f66780fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:34.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.847+0000 7f667efff700 1 -- 172.21.15.133:0/240639704 shutdown_connections 2023-11-27T15:20:34.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.847+0000 7f667efff700 1 -- 172.21.15.133:0/240639704 wait complete. 2023-11-27T15:20:34.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.848+0000 7f667efff700 1 Processor -- start 2023-11-27T15:20:34.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.848+0000 7f667efff700 1 -- start start 2023-11-27T15:20:34.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.849+0000 7f667efff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6678073550 0x7f667819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:34.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.849+0000 7f667efff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6678074b30 0x7f667819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:34.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.849+0000 7f667efff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f667810c8b0 0x7f66781a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:34.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.849+0000 7f667efff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66781a4240 con 0x7f6678074b30 2023-11-27T15:20:34.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.849+0000 7f667efff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66781a4380 con 0x7f6678073550 2023-11-27T15:20:34.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.849+0000 7f667efff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66781a4510 con 0x7f667810c8b0 2023-11-27T15:20:34.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.849+0000 7f6677fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6678074b30 0x7f667819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:34.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.849+0000 7f6677fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6678074b30 0x7f667819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44764/0 (socket says 172.21.15.133:44764) 2023-11-27T15:20:34.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.849+0000 7f6677fff700 1 -- 172.21.15.133:0/828473515 learned_addr learned my addr 172.21.15.133:0/828473515 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:34.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.849+0000 7f667cd9b700 1 --2- 172.21.15.133:0/828473515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6678073550 0x7f667819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:34.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.849+0000 7f667d59c700 1 --2- 172.21.15.133:0/828473515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f667810c8b0 0x7f66781a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:34.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.849+0000 7f667cd9b700 1 -- 172.21.15.133:0/828473515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f667810c8b0 msgr2=0x7f66781a3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:34.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.849+0000 7f667cd9b700 1 --2- 172.21.15.133:0/828473515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f667810c8b0 0x7f66781a3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:34.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.849+0000 7f667cd9b700 1 -- 172.21.15.133:0/828473515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6678074b30 msgr2=0x7f667819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:34.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.849+0000 7f667cd9b700 1 --2- 172.21.15.133:0/828473515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6678074b30 0x7f667819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:34.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.849+0000 7f667cd9b700 1 -- 172.21.15.133:0/828473515 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666c0095f0 con 0x7f6678073550 2023-11-27T15:20:34.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.849+0000 7f6677fff700 1 --2- 172.21.15.133:0/828473515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6678074b30 0x7f667819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:34.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.850+0000 7f667cd9b700 1 --2- 172.21.15.133:0/828473515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6678073550 0x7f667819c6c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f666c0098e0 tx=0x7f666c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:34.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.850+0000 7f6675ffb700 1 -- 172.21.15.133:0/828473515 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c01d070 con 0x7f6678073550 2023-11-27T15:20:34.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.850+0000 7f6675ffb700 1 -- 172.21.15.133:0/828473515 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f666c021410 con 0x7f6678073550 2023-11-27T15:20:34.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.850+0000 7f667efff700 1 -- 172.21.15.133:0/828473515 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66781122a0 con 0x7f6678073550 2023-11-27T15:20:34.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.850+0000 7f6675ffb700 1 -- 172.21.15.133:0/828473515 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c005720 con 0x7f6678073550 2023-11-27T15:20:34.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.850+0000 7f667efff700 1 -- 172.21.15.133:0/828473515 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6678112110 con 0x7f6678073550 2023-11-27T15:20:34.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.851+0000 7f667efff700 1 -- 172.21.15.133:0/828473515 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6678102400 con 0x7f6678073550 2023-11-27T15:20:34.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.853+0000 7f6675ffb700 1 -- 172.21.15.133:0/828473515 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f666c021580 con 0x7f6678073550 2023-11-27T15:20:34.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.854+0000 7f6675ffb700 1 --2- 172.21.15.133:0/828473515 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f66680a11d0 0x7f66680a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:34.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.854+0000 7f6675ffb700 1 -- 172.21.15.133:0/828473515 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f666c0cdae0 con 0x7f6678073550 2023-11-27T15:20:34.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.854+0000 7f6677fff700 1 --2- 172.21.15.133:0/828473515 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f66680a11d0 0x7f66680a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:34.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.855+0000 7f6677fff700 1 --2- 172.21.15.133:0/828473515 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f66680a11d0 0x7f66680a3680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6664005950 tx=0x7f6664009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:34.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:34.858+0000 7f6675ffb700 1 -- 172.21.15.133:0/828473515 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f666c097280 con 0x7f6678073550 2023-11-27T15:20:34.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:34 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3649278485' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-11-27T15:20:34.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:34 smithi174 ceph-mon[131110]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-11-27T15:20:34.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:34 smithi174 ceph-mon[131110]: pgmap v269: 97 pgs: 7 creating+peering, 83 unknown, 7 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:34.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:34 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:34.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:34 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:34.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:34 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:34.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:34 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] up:active 2023-11-27T15:20:34.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:34 smithi174 ceph-mon[131110]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:20:34.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:34 smithi174 ceph-mon[131110]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-11-27T15:20:34.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:34 smithi174 ceph-mon[131110]: osdmap e84: 12 total, 12 up, 12 in 2023-11-27T15:20:34.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:34 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:34.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:34 smithi174 ceph-mon[131110]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:20:35.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:34 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3649278485' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-11-27T15:20:35.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:34 smithi133 ceph-mon[97860]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-11-27T15:20:35.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:34 smithi133 ceph-mon[97860]: pgmap v269: 97 pgs: 7 creating+peering, 83 unknown, 7 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:35.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:34 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:35.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:34 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:35.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:34 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:35.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:34 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] up:active 2023-11-27T15:20:35.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:34 smithi133 ceph-mon[97860]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:20:35.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:34 smithi133 ceph-mon[97860]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-11-27T15:20:35.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:34 smithi133 ceph-mon[97860]: osdmap e84: 12 total, 12 up, 12 in 2023-11-27T15:20:35.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:34 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:35.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:34 smithi133 ceph-mon[97860]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:20:35.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:35.081+0000 7f667efff700 1 -- 172.21.15.133:0/828473515 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f66780617e0 con 0x7f6678073550 2023-11-27T15:20:35.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:34 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3649278485' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-11-27T15:20:35.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:34 smithi177 ceph-mon[131504]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-11-27T15:20:35.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:34 smithi177 ceph-mon[131504]: pgmap v269: 97 pgs: 7 creating+peering, 83 unknown, 7 active+clean; 577 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail 2023-11-27T15:20:35.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:34 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:35.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:34 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:35.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:34 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:35.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:34 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] up:active 2023-11-27T15:20:35.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:34 smithi177 ceph-mon[131504]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:20:35.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:34 smithi177 ceph-mon[131504]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-11-27T15:20:35.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:34 smithi177 ceph-mon[131504]: osdmap e84: 12 total, 12 up, 12 in 2023-11-27T15:20:35.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:34 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:35.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:34 smithi177 ceph-mon[131504]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:20:35.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:35 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/828473515' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-27T15:20:35.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:35 smithi174 ceph-mon[131110]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-27T15:20:35.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:35 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:35.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:35 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:35.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:35 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:35.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:35 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:35.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:35 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:20:35.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:35 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:35.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:35 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:20:35.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:35 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:20:35.904 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:35 smithi174 ceph-mon[131110]: Service mds.all has no related daemon already placed. 2023-11-27T15:20:36.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:35 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/828473515' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-27T15:20:36.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:35 smithi133 ceph-mon[97860]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-27T15:20:36.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:35 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:36.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:35 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:36.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:35 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:36.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:35 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:36.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:35 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:20:36.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:35 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:36.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:35 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:20:36.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:35 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:20:36.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:35 smithi133 ceph-mon[97860]: Service mds.all has no related daemon already placed. 2023-11-27T15:20:36.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:35 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/828473515' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-27T15:20:36.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:35 smithi177 ceph-mon[131504]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-11-27T15:20:36.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:35 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:36.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:35 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:36.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:35 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:20:36.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:35 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:20:36.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:35 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:20:36.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:35 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:20:36.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:35 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:20:36.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:35 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:20:36.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:35 smithi177 ceph-mon[131504]: Service mds.all has no related daemon already placed. 2023-11-27T15:20:36.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.646+0000 7f6675ffb700 1 -- 172.21.15.133:0/828473515 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v86) v1 ==== 228+0+0 (secure 0 0 0) 0x7f666c09a8a0 con 0x7f6678073550 2023-11-27T15:20:36.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.652+0000 7f667efff700 1 -- 172.21.15.133:0/828473515 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f66680a11d0 msgr2=0x7f66680a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:36.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.652+0000 7f667efff700 1 --2- 172.21.15.133:0/828473515 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f66680a11d0 0x7f66680a3680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6664005950 tx=0x7f6664009260 comp rx=0 tx=0).stop 2023-11-27T15:20:36.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.652+0000 7f667efff700 1 -- 172.21.15.133:0/828473515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6678073550 msgr2=0x7f667819c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:36.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.652+0000 7f667efff700 1 --2- 172.21.15.133:0/828473515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6678073550 0x7f667819c6c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f666c0098e0 tx=0x7f666c009580 comp rx=0 tx=0).stop 2023-11-27T15:20:36.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.653+0000 7f667efff700 1 -- 172.21.15.133:0/828473515 shutdown_connections 2023-11-27T15:20:36.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.653+0000 7f667efff700 1 --2- 172.21.15.133:0/828473515 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f66680a11d0 0x7f66680a3680 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:36.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.653+0000 7f667efff700 1 --2- 172.21.15.133:0/828473515 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f667810c8b0 0x7f66781a3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:36.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.653+0000 7f667efff700 1 --2- 172.21.15.133:0/828473515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6678073550 0x7f667819c6c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:36.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.653+0000 7f667efff700 1 --2- 172.21.15.133:0/828473515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6678074b30 0x7f667819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:36.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.653+0000 7f667efff700 1 -- 172.21.15.133:0/828473515 >> 172.21.15.133:0/828473515 conn(0x7f66780fb8c0 msgr2=0x7f66780fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:36.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.653+0000 7f667efff700 1 -- 172.21.15.133:0/828473515 shutdown_connections 2023-11-27T15:20:36.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.654+0000 7f667efff700 1 -- 172.21.15.133:0/828473515 wait complete. 2023-11-27T15:20:36.660 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data_ec' created 2023-11-27T15:20:36.679 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-11-27T15:20:36.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.880+0000 7fd94c7ff700 1 -- 172.21.15.133:0/3174023509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd944074b30 msgr2=0x7fd944072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:36.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.880+0000 7fd94c7ff700 1 --2- 172.21.15.133:0/3174023509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd944074b30 0x7fd944072f80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd940009990 tx=0x7fd940009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:36.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.880+0000 7fd94c7ff700 1 -- 172.21.15.133:0/3174023509 shutdown_connections 2023-11-27T15:20:36.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.880+0000 7fd94c7ff700 1 --2- 172.21.15.133:0/3174023509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd944108190 0x7fd94410a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:36.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.880+0000 7fd94c7ff700 1 --2- 172.21.15.133:0/3174023509 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd944074b30 0x7fd944072f80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:36.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.880+0000 7fd94c7ff700 1 --2- 172.21.15.133:0/3174023509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd944073550 0x7fd9440739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:36.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.880+0000 7fd94c7ff700 1 -- 172.21.15.133:0/3174023509 >> 172.21.15.133:0/3174023509 conn(0x7fd9440fb8b0 msgr2=0x7fd9440fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:36.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.880+0000 7fd94c7ff700 1 -- 172.21.15.133:0/3174023509 shutdown_connections 2023-11-27T15:20:36.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.880+0000 7fd94c7ff700 1 -- 172.21.15.133:0/3174023509 wait complete. 2023-11-27T15:20:36.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.881+0000 7fd94c7ff700 1 Processor -- start 2023-11-27T15:20:36.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.881+0000 7fd94c7ff700 1 -- start start 2023-11-27T15:20:36.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.882+0000 7fd94c7ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd944073550 0x7fd944198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:36.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.882+0000 7fd94c7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd944074b30 0x7fd9441987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:36.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.882+0000 7fd94c7ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd944108190 0x7fd94419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:36.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.882+0000 7fd94c7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94419fe10 con 0x7fd944074b30 2023-11-27T15:20:36.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.882+0000 7fd94c7ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94419ff50 con 0x7fd944073550 2023-11-27T15:20:36.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.882+0000 7fd94c7ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9441a00e0 con 0x7fd944108190 2023-11-27T15:20:36.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.882+0000 7fd949d9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd944074b30 0x7fd9441987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:36.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.882+0000 7fd949d9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd944074b30 0x7fd9441987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44776/0 (socket says 172.21.15.133:44776) 2023-11-27T15:20:36.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.882+0000 7fd949d9a700 1 -- 172.21.15.133:0/1848760947 learned_addr learned my addr 172.21.15.133:0/1848760947 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:36.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.882+0000 7fd94a59b700 1 --2- 172.21.15.133:0/1848760947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd944073550 0x7fd944198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:36.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.882+0000 7fd94ad9c700 1 --2- 172.21.15.133:0/1848760947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd944108190 0x7fd94419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:36.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.883+0000 7fd949d9a700 1 -- 172.21.15.133:0/1848760947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd944108190 msgr2=0x7fd94419f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:36.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.883+0000 7fd949d9a700 1 --2- 172.21.15.133:0/1848760947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd944108190 0x7fd94419f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:36.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.883+0000 7fd949d9a700 1 -- 172.21.15.133:0/1848760947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd944073550 msgr2=0x7fd944198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:36.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.883+0000 7fd949d9a700 1 --2- 172.21.15.133:0/1848760947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd944073550 0x7fd944198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:36.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.883+0000 7fd949d9a700 1 -- 172.21.15.133:0/1848760947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9400095f0 con 0x7fd944074b30 2023-11-27T15:20:36.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.883+0000 7fd94ad9c700 1 --2- 172.21.15.133:0/1848760947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd944108190 0x7fd94419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:20:36.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.883+0000 7fd949d9a700 1 --2- 172.21.15.133:0/1848760947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd944074b30 0x7fd9441987d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd93400e970 tx=0x7fd93400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:36.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.883+0000 7fd93b7fe700 1 -- 172.21.15.133:0/1848760947 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd934009790 con 0x7fd944074b30 2023-11-27T15:20:36.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.883+0000 7fd94c7ff700 1 -- 172.21.15.133:0/1848760947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd94410de40 con 0x7fd944074b30 2023-11-27T15:20:36.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.883+0000 7fd93b7fe700 1 -- 172.21.15.133:0/1848760947 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd93400eec0 con 0x7fd944074b30 2023-11-27T15:20:36.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.883+0000 7fd93b7fe700 1 -- 172.21.15.133:0/1848760947 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd934003cd0 con 0x7fd944074b30 2023-11-27T15:20:36.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.883+0000 7fd94c7ff700 1 -- 172.21.15.133:0/1848760947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94410dcb0 con 0x7fd944074b30 2023-11-27T15:20:36.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.884+0000 7fd94c7ff700 1 -- 172.21.15.133:0/1848760947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd944106840 con 0x7fd944074b30 2023-11-27T15:20:36.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.886+0000 7fd93b7fe700 1 -- 172.21.15.133:0/1848760947 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd93400ccd0 con 0x7fd944074b30 2023-11-27T15:20:36.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.887+0000 7fd93b7fe700 1 --2- 172.21.15.133:0/1848760947 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9300a1150 0x7fd9300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:36.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.888+0000 7fd93b7fe700 1 -- 172.21.15.133:0/1848760947 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fd934014070 con 0x7fd944074b30 2023-11-27T15:20:36.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.888+0000 7fd94a59b700 1 --2- 172.21.15.133:0/1848760947 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9300a1150 0x7fd9300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:36.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.888+0000 7fd94a59b700 1 --2- 172.21.15.133:0/1848760947 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9300a1150 0x7fd9300a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd94000b2e0 tx=0x7fd940005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:36.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:36.892+0000 7fd93b7fe700 1 -- 172.21.15.133:0/1848760947 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd934096280 con 0x7fd944074b30 2023-11-27T15:20:36.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:36 smithi174 ceph-mon[131110]: pgmap v272: 97 pgs: 7 creating+peering, 42 unknown, 48 active+clean; 577 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 3 op/s 2023-11-27T15:20:36.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:36 smithi174 ceph-mon[131110]: osdmap e85: 12 total, 12 up, 12 in 2023-11-27T15:20:37.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:36 smithi133 ceph-mon[97860]: pgmap v272: 97 pgs: 7 creating+peering, 42 unknown, 48 active+clean; 577 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 3 op/s 2023-11-27T15:20:37.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:36 smithi133 ceph-mon[97860]: osdmap e85: 12 total, 12 up, 12 in 2023-11-27T15:20:37.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:36 smithi177 ceph-mon[131504]: pgmap v272: 97 pgs: 7 creating+peering, 42 unknown, 48 active+clean; 577 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 3 op/s 2023-11-27T15:20:37.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:36 smithi177 ceph-mon[131504]: osdmap e85: 12 total, 12 up, 12 in 2023-11-27T15:20:37.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.091+0000 7fd94c7ff700 1 -- 172.21.15.133:0/1848760947 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fd94402cfe0 con 0x7fd944074b30 2023-11-27T15:20:37.647 INFO:journalctl@ceph.mds.b.smithi174.stdout:Nov 27 15:20:37 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-b[156420]: 2023-11-27T15:20:37.149+0000 7ff29549d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-27T15:20:37.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.653+0000 7fd93b7fe700 1 -- 172.21.15.133:0/1848760947 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd9340998a0 con 0x7fd944074b30 2023-11-27T15:20:37.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.658+0000 7fd94c7ff700 1 -- 172.21.15.133:0/1848760947 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9300a1150 msgr2=0x7fd9300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:37.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.658+0000 7fd94c7ff700 1 --2- 172.21.15.133:0/1848760947 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9300a1150 0x7fd9300a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd94000b2e0 tx=0x7fd940005fb0 comp rx=0 tx=0).stop 2023-11-27T15:20:37.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.658+0000 7fd94c7ff700 1 -- 172.21.15.133:0/1848760947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd944074b30 msgr2=0x7fd9441987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:37.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.658+0000 7fd94c7ff700 1 --2- 172.21.15.133:0/1848760947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd944074b30 0x7fd9441987d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd93400e970 tx=0x7fd93400ec80 comp rx=0 tx=0).stop 2023-11-27T15:20:37.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.659+0000 7fd94c7ff700 1 -- 172.21.15.133:0/1848760947 shutdown_connections 2023-11-27T15:20:37.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.659+0000 7fd94c7ff700 1 --2- 172.21.15.133:0/1848760947 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9300a1150 0x7fd9300a3600 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:37.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.659+0000 7fd94c7ff700 1 --2- 172.21.15.133:0/1848760947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd944108190 0x7fd94419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:37.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.659+0000 7fd94c7ff700 1 --2- 172.21.15.133:0/1848760947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd944073550 0x7fd944198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:37.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.659+0000 7fd94c7ff700 1 --2- 172.21.15.133:0/1848760947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd944074b30 0x7fd9441987d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:37.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.659+0000 7fd94c7ff700 1 -- 172.21.15.133:0/1848760947 >> 172.21.15.133:0/1848760947 conn(0x7fd9440fb8b0 msgr2=0x7fd9440fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:37.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.660+0000 7fd94c7ff700 1 -- 172.21.15.133:0/1848760947 shutdown_connections 2023-11-27T15:20:37.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.660+0000 7fd94c7ff700 1 -- 172.21.15.133:0/1848760947 wait complete. 2023-11-27T15:20:37.667 INFO:teuthology.orchestra.run.smithi133.stderr:set pool 4 allow_ec_overwrites to true 2023-11-27T15:20:37.694 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-11-27T15:20:37.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.893+0000 7fe24c1ff700 1 -- 172.21.15.133:0/858157377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2441002d0 msgr2=0x7fe2441006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:37.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.893+0000 7fe24c1ff700 1 --2- 172.21.15.133:0/858157377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2441002d0 0x7fe2441006e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe240009990 tx=0x7fe240009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:37.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.893+0000 7fe24c1ff700 1 -- 172.21.15.133:0/858157377 shutdown_connections 2023-11-27T15:20:37.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.893+0000 7fe24c1ff700 1 --2- 172.21.15.133:0/858157377 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe244101e50 0x7fe244108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:37.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.893+0000 7fe24c1ff700 1 --2- 172.21.15.133:0/858157377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2441002d0 0x7fe2441006e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:37.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.893+0000 7fe24c1ff700 1 --2- 172.21.15.133:0/858157377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2441014c0 0x7fe244101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:37.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.893+0000 7fe24c1ff700 1 -- 172.21.15.133:0/858157377 >> 172.21.15.133:0/858157377 conn(0x7fe2440fb8b0 msgr2=0x7fe2440fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:37.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.893+0000 7fe24c1ff700 1 -- 172.21.15.133:0/858157377 shutdown_connections 2023-11-27T15:20:37.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.893+0000 7fe24c1ff700 1 -- 172.21.15.133:0/858157377 wait complete. 2023-11-27T15:20:37.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.894+0000 7fe24c1ff700 1 Processor -- start 2023-11-27T15:20:37.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.894+0000 7fe24c1ff700 1 -- start start 2023-11-27T15:20:37.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.895+0000 7fe24c1ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2441002d0 0x7fe244198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:37.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.895+0000 7fe24c1ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe2441014c0 0x7fe2441987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:37.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.895+0000 7fe24c1ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe244101e50 0x7fe24419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:37.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.895+0000 7fe24c1ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24419fe10 con 0x7fe244101e50 2023-11-27T15:20:37.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.895+0000 7fe24c1ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24419ff50 con 0x7fe2441002d0 2023-11-27T15:20:37.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.895+0000 7fe24c1ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2441a00e0 con 0x7fe2441014c0 2023-11-27T15:20:37.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.895+0000 7fe24a79c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe244101e50 0x7fe24419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:37.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.895+0000 7fe24a79c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe244101e50 0x7fe24419f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44792/0 (socket says 172.21.15.133:44792) 2023-11-27T15:20:37.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.895+0000 7fe24a79c700 1 -- 172.21.15.133:0/3358885588 learned_addr learned my addr 172.21.15.133:0/3358885588 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:37.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.895+0000 7fe249f9b700 1 --2- 172.21.15.133:0/3358885588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2441002d0 0x7fe244198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:37.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.895+0000 7fe24979a700 1 --2- 172.21.15.133:0/3358885588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe2441014c0 0x7fe2441987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:37.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.895+0000 7fe24a79c700 1 -- 172.21.15.133:0/3358885588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe2441014c0 msgr2=0x7fe2441987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:37.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.895+0000 7fe24a79c700 1 --2- 172.21.15.133:0/3358885588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe2441014c0 0x7fe2441987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:37.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.895+0000 7fe24a79c700 1 -- 172.21.15.133:0/3358885588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2441002d0 msgr2=0x7fe244198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:37.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.895+0000 7fe24a79c700 1 --2- 172.21.15.133:0/3358885588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2441002d0 0x7fe244198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:37.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.895+0000 7fe24a79c700 1 -- 172.21.15.133:0/3358885588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2400095f0 con 0x7fe244101e50 2023-11-27T15:20:37.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.895+0000 7fe24979a700 1 --2- 172.21.15.133:0/3358885588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe2441014c0 0x7fe2441987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:20:37.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.896+0000 7fe24a79c700 1 --2- 172.21.15.133:0/3358885588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe244101e50 0x7fe24419f840 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00ef30 tx=0x7fe23c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:37.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.896+0000 7fe23affd700 1 -- 172.21.15.133:0/3358885588 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c018070 con 0x7fe244101e50 2023-11-27T15:20:37.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.896+0000 7fe23affd700 1 -- 172.21.15.133:0/3358885588 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe23c00f040 con 0x7fe244101e50 2023-11-27T15:20:37.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.896+0000 7fe24c1ff700 1 -- 172.21.15.133:0/3358885588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24410bca0 con 0x7fe244101e50 2023-11-27T15:20:37.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.896+0000 7fe23affd700 1 -- 172.21.15.133:0/3358885588 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c00cdb0 con 0x7fe244101e50 2023-11-27T15:20:37.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.896+0000 7fe24c1ff700 1 -- 172.21.15.133:0/3358885588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24410bb10 con 0x7fe244101e50 2023-11-27T15:20:37.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.897+0000 7fe24c1ff700 1 -- 172.21.15.133:0/3358885588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe244192a00 con 0x7fe244101e50 2023-11-27T15:20:37.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:37 smithi174 ceph-mon[131110]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-11-27T15:20:37.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:37 smithi174 ceph-mon[131110]: osdmap e86: 12 total, 12 up, 12 in 2023-11-27T15:20:37.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:37 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1848760947' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-11-27T15:20:37.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.900+0000 7fe23affd700 1 -- 172.21.15.133:0/3358885588 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe23c003a00 con 0x7fe244101e50 2023-11-27T15:20:37.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.902+0000 7fe23affd700 1 --2- 172.21.15.133:0/3358885588 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe2300a11d0 0x7fe2300a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:37.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.902+0000 7fe23affd700 1 -- 172.21.15.133:0/3358885588 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fe23c0ccf90 con 0x7fe244101e50 2023-11-27T15:20:37.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.902+0000 7fe249f9b700 1 --2- 172.21.15.133:0/3358885588 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe2300a11d0 0x7fe2300a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:37.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.902+0000 7fe249f9b700 1 --2- 172.21.15.133:0/3358885588 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe2300a11d0 0x7fe2300a3680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe24000b2e0 tx=0x7fe240005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:37.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:37.904+0000 7fe23affd700 1 -- 172.21.15.133:0/3358885588 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe23c0965a0 con 0x7fe244101e50 2023-11-27T15:20:38.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:37 smithi133 ceph-mon[97860]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-11-27T15:20:38.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:37 smithi133 ceph-mon[97860]: osdmap e86: 12 total, 12 up, 12 in 2023-11-27T15:20:38.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:37 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1848760947' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-11-27T15:20:38.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:37 smithi177 ceph-mon[131504]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-11-27T15:20:38.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:37 smithi177 ceph-mon[131504]: osdmap e86: 12 total, 12 up, 12 in 2023-11-27T15:20:38.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:37 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1848760947' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-11-27T15:20:38.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.145+0000 7fe24c1ff700 1 -- 172.21.15.133:0/3358885588 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fe24402cfe0 con 0x7fe244101e50 2023-11-27T15:20:38.156 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.152+0000 7fe23affd700 1 -- 172.21.15.133:0/3358885588 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7fe23c099bc0 con 0x7fe244101e50 2023-11-27T15:20:38.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.154+0000 7fe24c1ff700 1 -- 172.21.15.133:0/3358885588 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe2300a11d0 msgr2=0x7fe2300a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:38.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.154+0000 7fe24c1ff700 1 --2- 172.21.15.133:0/3358885588 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe2300a11d0 0x7fe2300a3680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe24000b2e0 tx=0x7fe240005fb0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.154+0000 7fe24c1ff700 1 -- 172.21.15.133:0/3358885588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe244101e50 msgr2=0x7fe24419f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:38.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.154+0000 7fe24c1ff700 1 --2- 172.21.15.133:0/3358885588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe244101e50 0x7fe24419f840 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00ef30 tx=0x7fe23c00c3a0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.154+0000 7fe24c1ff700 1 -- 172.21.15.133:0/3358885588 shutdown_connections 2023-11-27T15:20:38.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.154+0000 7fe24c1ff700 1 --2- 172.21.15.133:0/3358885588 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe2300a11d0 0x7fe2300a3680 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.154+0000 7fe24c1ff700 1 --2- 172.21.15.133:0/3358885588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe244101e50 0x7fe24419f840 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.154+0000 7fe24c1ff700 1 --2- 172.21.15.133:0/3358885588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2441002d0 0x7fe244198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.154+0000 7fe24c1ff700 1 --2- 172.21.15.133:0/3358885588 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe2441014c0 0x7fe2441987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.154+0000 7fe24c1ff700 1 -- 172.21.15.133:0/3358885588 >> 172.21.15.133:0/3358885588 conn(0x7fe2440fb8b0 msgr2=0x7fe244106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:38.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.155+0000 7fe24c1ff700 1 -- 172.21.15.133:0/3358885588 shutdown_connections 2023-11-27T15:20:38.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.155+0000 7fe24c1ff700 1 -- 172.21.15.133:0/3358885588 wait complete. 2023-11-27T15:20:38.160 INFO:teuthology.orchestra.run.smithi133.stderr:added data pool 4 to fsmap 2023-11-27T15:20:38.177 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:20:38.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.371+0000 7f92f53ff700 1 -- 172.21.15.133:0/4241156862 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92f0101480 msgr2=0x7f92f01018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:38.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.371+0000 7f92f53ff700 1 --2- 172.21.15.133:0/4241156862 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92f0101480 0x7f92f01018d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009990 tx=0x7f92d8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.371+0000 7f92f53ff700 1 -- 172.21.15.133:0/4241156862 shutdown_connections 2023-11-27T15:20:38.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.371+0000 7f92f53ff700 1 --2- 172.21.15.133:0/4241156862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92f0101e10 0x7f92f01086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.371+0000 7f92f53ff700 1 --2- 172.21.15.133:0/4241156862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f0100290 0x7f92f01006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.371+0000 7f92f53ff700 1 --2- 172.21.15.133:0/4241156862 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92f0101480 0x7f92f01018d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.371+0000 7f92f53ff700 1 -- 172.21.15.133:0/4241156862 >> 172.21.15.133:0/4241156862 conn(0x7f92f00fb870 msgr2=0x7f92f00fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:38.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.371+0000 7f92f53ff700 1 -- 172.21.15.133:0/4241156862 shutdown_connections 2023-11-27T15:20:38.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.371+0000 7f92f53ff700 1 -- 172.21.15.133:0/4241156862 wait complete. 2023-11-27T15:20:38.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.372+0000 7f92f53ff700 1 Processor -- start 2023-11-27T15:20:38.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.372+0000 7f92f53ff700 1 -- start start 2023-11-27T15:20:38.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.372+0000 7f92f53ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92f0100290 0x7f92f00748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:38.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.372+0000 7f92f53ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92f0101480 0x7f92f0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:38.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.372+0000 7f92f53ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f0101e10 0x7f92f0073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:38.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.372+0000 7f92f53ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f0073bc0 con 0x7f92f0101e10 2023-11-27T15:20:38.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.372+0000 7f92f53ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f0073d30 con 0x7f92f0100290 2023-11-27T15:20:38.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.372+0000 7f92f53ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f019ff70 con 0x7f92f0101480 2023-11-27T15:20:38.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.373+0000 7f92ef7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f0101e10 0x7f92f0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:38.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.373+0000 7f92eeffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92f0100290 0x7f92f00748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:38.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.373+0000 7f92eeffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92f0100290 0x7f92f00748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:57112/0 (socket says 172.21.15.133:57112) 2023-11-27T15:20:38.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.373+0000 7f92ef7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f0101e10 0x7f92f0073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44810/0 (socket says 172.21.15.133:44810) 2023-11-27T15:20:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.373+0000 7f92e7fff700 1 --2- 172.21.15.133:0/362419621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92f0101480 0x7f92f0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.373+0000 7f92eeffd700 1 -- 172.21.15.133:0/362419621 learned_addr learned my addr 172.21.15.133:0/362419621 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.373+0000 7f92eeffd700 1 -- 172.21.15.133:0/362419621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92f0101480 msgr2=0x7f92f0072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.373+0000 7f92eeffd700 1 --2- 172.21.15.133:0/362419621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92f0101480 0x7f92f0072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.373+0000 7f92eeffd700 1 -- 172.21.15.133:0/362419621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f0101e10 msgr2=0x7f92f0073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.373+0000 7f92eeffd700 1 --2- 172.21.15.133:0/362419621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f0101e10 0x7f92f0073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.373+0000 7f92eeffd700 1 -- 172.21.15.133:0/362419621 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e0009520 con 0x7f92f0100290 2023-11-27T15:20:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.373+0000 7f92eeffd700 1 --2- 172.21.15.133:0/362419621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92f0100290 0x7f92f00748a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f92e000ea50 tx=0x7f92e000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.374+0000 7f92ecff9700 1 -- 172.21.15.133:0/362419621 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e000cbd0 con 0x7f92f0100290 2023-11-27T15:20:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.374+0000 7f92f53ff700 1 -- 172.21.15.133:0/362419621 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d80095f0 con 0x7f92f0100290 2023-11-27T15:20:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.374+0000 7f92ecff9700 1 -- 172.21.15.133:0/362419621 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92e000cd30 con 0x7f92f0100290 2023-11-27T15:20:38.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.374+0000 7f92ecff9700 1 -- 172.21.15.133:0/362419621 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e0018680 con 0x7f92f0100290 2023-11-27T15:20:38.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.374+0000 7f92f53ff700 1 -- 172.21.15.133:0/362419621 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f010bbf0 con 0x7f92f0100290 2023-11-27T15:20:38.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.375+0000 7f92f53ff700 1 -- 172.21.15.133:0/362419621 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f0107850 con 0x7f92f0100290 2023-11-27T15:20:38.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.377+0000 7f92ecff9700 1 -- 172.21.15.133:0/362419621 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f92e0010430 con 0x7f92f0100290 2023-11-27T15:20:38.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.378+0000 7f92ecff9700 1 --2- 172.21.15.133:0/362419621 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92dc0a11d0 0x7f92dc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:38.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.378+0000 7f92ecff9700 1 -- 172.21.15.133:0/362419621 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f92e0014070 con 0x7f92f0100290 2023-11-27T15:20:38.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.378+0000 7f92e7fff700 1 --2- 172.21.15.133:0/362419621 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92dc0a11d0 0x7f92dc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:38.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.379+0000 7f92e7fff700 1 --2- 172.21.15.133:0/362419621 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92dc0a11d0 0x7f92dc0a3680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f92d800b2e0 tx=0x7f92d8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:38.386 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.382+0000 7f92ecff9700 1 -- 172.21.15.133:0/362419621 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f92e0096490 con 0x7f92f0100290 2023-11-27T15:20:38.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.630+0000 7f92f53ff700 1 -- 172.21.15.133:0/362419621 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92f0067a60 con 0x7f92f0100290 2023-11-27T15:20:38.635 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.631+0000 7f92ecff9700 1 -- 172.21.15.133:0/362419621 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10768 (secure 0 0 0) 0x7f92e0099ab0 con 0x7f92f0100290 2023-11-27T15:20:38.635 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:38.635 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14493,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2381290259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2381290259},{"type":"v1","addr":"172.21.15.133:6839","nonce":2381290259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1704875895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1704875895},{"type":"v1","addr":"172.21.15.133:6841","nonce":1704875895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24496,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/3492215689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3492215689},{"type":"v1","addr":"172.21.15.177:6837","nonce":3492215689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-27T15:20:33.201023+0000","modified":"2023-11-27T15:20:38.147285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24560},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6839/2781994447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2781994447},{"type":"v1","addr":"172.21.15.174:6839","nonce":2781994447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-27T15:20:38.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.633+0000 7f92f53ff700 1 -- 172.21.15.133:0/362419621 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92dc0a11d0 msgr2=0x7f92dc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:38.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.633+0000 7f92f53ff700 1 --2- 172.21.15.133:0/362419621 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92dc0a11d0 0x7f92dc0a3680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f92d800b2e0 tx=0x7f92d8005fb0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.633+0000 7f92f53ff700 1 -- 172.21.15.133:0/362419621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92f0100290 msgr2=0x7f92f00748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:38.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.633+0000 7f92f53ff700 1 --2- 172.21.15.133:0/362419621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92f0100290 0x7f92f00748a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f92e000ea50 tx=0x7f92e000ed60 comp rx=0 tx=0).stop 2023-11-27T15:20:38.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.633+0000 7f92f53ff700 1 -- 172.21.15.133:0/362419621 shutdown_connections 2023-11-27T15:20:38.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.633+0000 7f92f53ff700 1 --2- 172.21.15.133:0/362419621 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92dc0a11d0 0x7f92dc0a3680 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.633+0000 7f92f53ff700 1 --2- 172.21.15.133:0/362419621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92f0101e10 0x7f92f0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.633+0000 7f92f53ff700 1 --2- 172.21.15.133:0/362419621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92f0100290 0x7f92f00748a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.633+0000 7f92f53ff700 1 --2- 172.21.15.133:0/362419621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92f0101480 0x7f92f0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.633+0000 7f92f53ff700 1 -- 172.21.15.133:0/362419621 >> 172.21.15.133:0/362419621 conn(0x7f92f00fb870 msgr2=0x7f92f0104990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:38.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.634+0000 7f92f53ff700 1 -- 172.21.15.133:0/362419621 shutdown_connections 2023-11-27T15:20:38.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.634+0000 7f92f53ff700 1 -- 172.21.15.133:0/362419621 wait complete. 2023-11-27T15:20:38.639 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 17 2023-11-27T15:20:38.657 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:20:38.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.852+0000 7f251d7ff700 1 -- 172.21.15.133:0/3696250029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2518101480 msgr2=0x7f25181018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:38.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.852+0000 7f251d7ff700 1 --2- 172.21.15.133:0/3696250029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2518101480 0x7f25181018d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f250c009990 tx=0x7f250c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.852+0000 7f251d7ff700 1 -- 172.21.15.133:0/3696250029 shutdown_connections 2023-11-27T15:20:38.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.852+0000 7f251d7ff700 1 --2- 172.21.15.133:0/3696250029 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2518101e10 0x7f25181086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.852+0000 7f251d7ff700 1 --2- 172.21.15.133:0/3696250029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2518100290 0x7f25181006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.852+0000 7f251d7ff700 1 --2- 172.21.15.133:0/3696250029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2518101480 0x7f25181018d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.852+0000 7f251d7ff700 1 -- 172.21.15.133:0/3696250029 >> 172.21.15.133:0/3696250029 conn(0x7f25180fb870 msgr2=0x7f25180fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:38.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.852+0000 7f251d7ff700 1 -- 172.21.15.133:0/3696250029 shutdown_connections 2023-11-27T15:20:38.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.852+0000 7f251d7ff700 1 -- 172.21.15.133:0/3696250029 wait complete. 2023-11-27T15:20:38.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.853+0000 7f251d7ff700 1 Processor -- start 2023-11-27T15:20:38.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.853+0000 7f251d7ff700 1 -- start start 2023-11-27T15:20:38.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.853+0000 7f251d7ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2518100290 0x7f25181981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:38.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.853+0000 7f251d7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2518101480 0x7f2518198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:38.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.853+0000 7f251d7ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2518101e10 0x7f251819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:38.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.853+0000 7f251d7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251819fd40 con 0x7f2518101480 2023-11-27T15:20:38.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.853+0000 7f251d7ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251819fe80 con 0x7f2518101e10 2023-11-27T15:20:38.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.853+0000 7f251d7ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25181a0010 con 0x7f2518100290 2023-11-27T15:20:38.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.853+0000 7f25177fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2518101e10 0x7f251819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:38.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.854+0000 7f25167fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2518101480 0x7f2518198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:38.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.854+0000 7f25177fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2518101e10 0x7f251819f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:57122/0 (socket says 172.21.15.133:57122) 2023-11-27T15:20:38.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.854+0000 7f25177fe700 1 -- 172.21.15.133:0/1737593332 learned_addr learned my addr 172.21.15.133:0/1737593332 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:38.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.854+0000 7f2516ffd700 1 --2- 172.21.15.133:0/1737593332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2518100290 0x7f25181981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:38.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.854+0000 7f25167fc700 1 -- 172.21.15.133:0/1737593332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2518100290 msgr2=0x7f25181981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:38.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.854+0000 7f25167fc700 1 --2- 172.21.15.133:0/1737593332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2518100290 0x7f25181981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.854+0000 7f25167fc700 1 -- 172.21.15.133:0/1737593332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2518101e10 msgr2=0x7f251819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:38.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.854+0000 7f25167fc700 1 --2- 172.21.15.133:0/1737593332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2518101e10 0x7f251819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:38.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.854+0000 7f25167fc700 1 -- 172.21.15.133:0/1737593332 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250c0095f0 con 0x7f2518101480 2023-11-27T15:20:38.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.854+0000 7f25177fe700 1 --2- 172.21.15.133:0/1737593332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2518101e10 0x7f251819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:38.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.854+0000 7f25167fc700 1 --2- 172.21.15.133:0/1737593332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2518101480 0x7f2518198700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f250c000c00 tx=0x7f250c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:38.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.854+0000 7f24fffff700 1 -- 172.21.15.133:0/1737593332 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c01d070 con 0x7f2518101480 2023-11-27T15:20:38.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.854+0000 7f24fffff700 1 -- 172.21.15.133:0/1737593332 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f250c003f60 con 0x7f2518101480 2023-11-27T15:20:38.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.854+0000 7f251d7ff700 1 -- 172.21.15.133:0/1737593332 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251810bc50 con 0x7f2518101480 2023-11-27T15:20:38.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.854+0000 7f24fffff700 1 -- 172.21.15.133:0/1737593332 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c022b40 con 0x7f2518101480 2023-11-27T15:20:38.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.855+0000 7f251d7ff700 1 -- 172.21.15.133:0/1737593332 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251810bac0 con 0x7f2518101480 2023-11-27T15:20:38.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.856+0000 7f251d7ff700 1 -- 172.21.15.133:0/1737593332 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2518067a60 con 0x7f2518101480 2023-11-27T15:20:38.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.858+0000 7f24fffff700 1 -- 172.21.15.133:0/1737593332 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f250c003aa0 con 0x7f2518101480 2023-11-27T15:20:38.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.860+0000 7f24fffff700 1 --2- 172.21.15.133:0/1737593332 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f25000a1150 0x7f25000a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:38.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.860+0000 7f24fffff700 1 -- 172.21.15.133:0/1737593332 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f250c0ce800 con 0x7f2518101480 2023-11-27T15:20:38.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.860+0000 7f2516ffd700 1 --2- 172.21.15.133:0/1737593332 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f25000a1150 0x7f25000a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:38.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.861+0000 7f2516ffd700 1 --2- 172.21.15.133:0/1737593332 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f25000a1150 0x7f25000a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f25181012e0 tx=0x7f2508007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:38.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:38.862+0000 7f24fffff700 1 -- 172.21.15.133:0/1737593332 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f250c097d90 con 0x7f2518101480 2023-11-27T15:20:39.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:38 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1848760947' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-11-27T15:20:39.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:38 smithi133 ceph-mon[97860]: osdmap e87: 12 total, 12 up, 12 in 2023-11-27T15:20:39.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:38 smithi133 ceph-mon[97860]: pgmap v275: 129 pgs: 7 creating+peering, 29 unknown, 93 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2023-11-27T15:20:39.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:38 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3358885588' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-11-27T15:20:39.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:38 smithi133 ceph-mon[97860]: osdmap e88: 12 total, 12 up, 12 in 2023-11-27T15:20:39.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:38 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3358885588' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-11-27T15:20:39.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:38 smithi133 ceph-mon[97860]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:20:39.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:38 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/362419621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:39.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.063+0000 7f251d7ff700 1 -- 172.21.15.133:0/1737593332 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25181a0b80 con 0x7f2518101480 2023-11-27T15:20:39.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.064+0000 7f24fffff700 1 -- 172.21.15.133:0/1737593332 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23996 (secure 0 0 0) 0x7f250c027090 con 0x7f2518101480 2023-11-27T15:20:39.069 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:39.069 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:20:38.147036+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:20:39.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.067+0000 7f251d7ff700 1 -- 172.21.15.133:0/1737593332 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f25000a1150 msgr2=0x7f25000a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.067+0000 7f251d7ff700 1 --2- 172.21.15.133:0/1737593332 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f25000a1150 0x7f25000a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f25181012e0 tx=0x7f2508007400 comp rx=0 tx=0).stop 2023-11-27T15:20:39.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.067+0000 7f251d7ff700 1 -- 172.21.15.133:0/1737593332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2518101480 msgr2=0x7f2518198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.067+0000 7f251d7ff700 1 --2- 172.21.15.133:0/1737593332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2518101480 0x7f2518198700 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f250c000c00 tx=0x7f250c005740 comp rx=0 tx=0).stop 2023-11-27T15:20:39.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.067+0000 7f251d7ff700 1 -- 172.21.15.133:0/1737593332 shutdown_connections 2023-11-27T15:20:39.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.067+0000 7f251d7ff700 1 --2- 172.21.15.133:0/1737593332 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f25000a1150 0x7f25000a3600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.067+0000 7f251d7ff700 1 --2- 172.21.15.133:0/1737593332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2518101e10 0x7f251819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.067+0000 7f251d7ff700 1 --2- 172.21.15.133:0/1737593332 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2518100290 0x7f25181981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.067+0000 7f251d7ff700 1 --2- 172.21.15.133:0/1737593332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2518101480 0x7f2518198700 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.067+0000 7f251d7ff700 1 -- 172.21.15.133:0/1737593332 >> 172.21.15.133:0/1737593332 conn(0x7f25180fb870 msgr2=0x7f25180fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:39.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.067+0000 7f251d7ff700 1 -- 172.21.15.133:0/1737593332 shutdown_connections 2023-11-27T15:20:39.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.067+0000 7f251d7ff700 1 -- 172.21.15.133:0/1737593332 wait complete. 2023-11-27T15:20:39.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:38 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1848760947' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-11-27T15:20:39.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:38 smithi177 ceph-mon[131504]: osdmap e87: 12 total, 12 up, 12 in 2023-11-27T15:20:39.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:38 smithi177 ceph-mon[131504]: pgmap v275: 129 pgs: 7 creating+peering, 29 unknown, 93 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2023-11-27T15:20:39.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:38 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3358885588' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-11-27T15:20:39.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:38 smithi177 ceph-mon[131504]: osdmap e88: 12 total, 12 up, 12 in 2023-11-27T15:20:39.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:38 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3358885588' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-11-27T15:20:39.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:38 smithi177 ceph-mon[131504]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:20:39.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:38 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/362419621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:39.091 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:20:39.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:38 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1848760947' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-11-27T15:20:39.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:38 smithi174 ceph-mon[131110]: osdmap e87: 12 total, 12 up, 12 in 2023-11-27T15:20:39.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:38 smithi174 ceph-mon[131110]: pgmap v275: 129 pgs: 7 creating+peering, 29 unknown, 93 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s wr, 5 op/s 2023-11-27T15:20:39.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:38 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3358885588' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-11-27T15:20:39.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:38 smithi174 ceph-mon[131110]: osdmap e88: 12 total, 12 up, 12 in 2023-11-27T15:20:39.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:38 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3358885588' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-11-27T15:20:39.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:38 smithi174 ceph-mon[131110]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:20:39.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:38 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/362419621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:39.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.283+0000 7f4a021ff700 1 -- 172.21.15.133:0/564081950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49fc101e10 msgr2=0x7f49fc1086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.283+0000 7f4a021ff700 1 --2- 172.21.15.133:0/564081950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49fc101e10 0x7f49fc1086d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009f90 tx=0x7f49f400b330 comp rx=0 tx=0).stop 2023-11-27T15:20:39.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.283+0000 7f4a021ff700 1 -- 172.21.15.133:0/564081950 shutdown_connections 2023-11-27T15:20:39.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.283+0000 7f4a021ff700 1 --2- 172.21.15.133:0/564081950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49fc101e10 0x7f49fc1086d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.283+0000 7f4a021ff700 1 --2- 172.21.15.133:0/564081950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49fc100290 0x7f49fc1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.283+0000 7f4a021ff700 1 --2- 172.21.15.133:0/564081950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49fc101480 0x7f49fc1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.283+0000 7f4a021ff700 1 -- 172.21.15.133:0/564081950 >> 172.21.15.133:0/564081950 conn(0x7f49fc0fb870 msgr2=0x7f49fc0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:39.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.283+0000 7f4a021ff700 1 -- 172.21.15.133:0/564081950 shutdown_connections 2023-11-27T15:20:39.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.283+0000 7f4a021ff700 1 -- 172.21.15.133:0/564081950 wait complete. 2023-11-27T15:20:39.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.284+0000 7f4a021ff700 1 Processor -- start 2023-11-27T15:20:39.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.284+0000 7f4a021ff700 1 -- start start 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.284+0000 7f4a021ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49fc100290 0x7f49fc198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.284+0000 7f4a021ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49fc101480 0x7f49fc1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.284+0000 7f4a021ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49fc101e10 0x7f49fc19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.284+0000 7f4a021ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc19fe10 con 0x7f49fc100290 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.284+0000 7f4a021ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc19ff50 con 0x7f49fc101e10 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.284+0000 7f4a021ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49fc1a00e0 con 0x7f49fc101480 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.285+0000 7f49fb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49fc100290 0x7f49fc198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.285+0000 7f49fb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49fc100290 0x7f49fc198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44834/0 (socket says 172.21.15.133:44834) 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.285+0000 7f49fb7fe700 1 -- 172.21.15.133:0/312673007 learned_addr learned my addr 172.21.15.133:0/312673007 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.285+0000 7f49fbfff700 1 --2- 172.21.15.133:0/312673007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49fc101e10 0x7f49fc19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.285+0000 7f49f3fff700 1 --2- 172.21.15.133:0/312673007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49fc101480 0x7f49fc1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.285+0000 7f49fb7fe700 1 -- 172.21.15.133:0/312673007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49fc101480 msgr2=0x7f49fc1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.285+0000 7f49fb7fe700 1 --2- 172.21.15.133:0/312673007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49fc101480 0x7f49fc1987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.285+0000 7f49fb7fe700 1 -- 172.21.15.133:0/312673007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49fc101e10 msgr2=0x7f49fc19f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.285+0000 7f49fb7fe700 1 --2- 172.21.15.133:0/312673007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49fc101e10 0x7f49fc19f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.285+0000 7f49fb7fe700 1 -- 172.21.15.133:0/312673007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f4009c40 con 0x7f49fc100290 2023-11-27T15:20:39.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.285+0000 7f49fb7fe700 1 --2- 172.21.15.133:0/312673007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49fc100290 0x7f49fc198290 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00e9f0 tx=0x7f49ec00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:39.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.285+0000 7f49f97fa700 1 -- 172.21.15.133:0/312673007 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec004220 con 0x7f49fc100290 2023-11-27T15:20:39.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.285+0000 7f49f97fa700 1 -- 172.21.15.133:0/312673007 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49ec004380 con 0x7f49fc100290 2023-11-27T15:20:39.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.285+0000 7f4a021ff700 1 -- 172.21.15.133:0/312673007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49fc10bc70 con 0x7f49fc100290 2023-11-27T15:20:39.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.285+0000 7f49f97fa700 1 -- 172.21.15.133:0/312673007 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ec003be0 con 0x7f49fc100290 2023-11-27T15:20:39.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.286+0000 7f4a021ff700 1 -- 172.21.15.133:0/312673007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49fc10bae0 con 0x7f49fc100290 2023-11-27T15:20:39.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.287+0000 7f4a021ff700 1 -- 172.21.15.133:0/312673007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49fc1a0880 con 0x7f49fc100290 2023-11-27T15:20:39.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.289+0000 7f49f97fa700 1 -- 172.21.15.133:0/312673007 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f49ec003d40 con 0x7f49fc100290 2023-11-27T15:20:39.294 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.291+0000 7f49f97fa700 1 --2- 172.21.15.133:0/312673007 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f49e80a11d0 0x7f49e80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.294 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.291+0000 7f49f97fa700 1 -- 172.21.15.133:0/312673007 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f49ec014070 con 0x7f49fc100290 2023-11-27T15:20:39.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.291+0000 7f49f3fff700 1 --2- 172.21.15.133:0/312673007 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f49e80a11d0 0x7f49e80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.291+0000 7f49f3fff700 1 --2- 172.21.15.133:0/312673007 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f49e80a11d0 0x7f49e80a3680 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009c60 tx=0x7f49e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:39.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.295+0000 7f49f97fa700 1 -- 172.21.15.133:0/312673007 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f49ec0960a0 con 0x7f49fc100290 2023-11-27T15:20:39.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.496+0000 7f4a021ff700 1 -- 172.21.15.133:0/312673007 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49fc1a0e30 con 0x7f49fc100290 2023-11-27T15:20:39.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.498+0000 7f49f97fa700 1 -- 172.21.15.133:0/312673007 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23996 (secure 0 0 0) 0x7f49ec0996c0 con 0x7f49fc100290 2023-11-27T15:20:39.502 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:39.502 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:20:38.147036+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:20:39.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.500+0000 7f4a021ff700 1 -- 172.21.15.133:0/312673007 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f49e80a11d0 msgr2=0x7f49e80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.500+0000 7f4a021ff700 1 --2- 172.21.15.133:0/312673007 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f49e80a11d0 0x7f49e80a3680 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009c60 tx=0x7f49e4009260 comp rx=0 tx=0).stop 2023-11-27T15:20:39.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.500+0000 7f4a021ff700 1 -- 172.21.15.133:0/312673007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49fc100290 msgr2=0x7f49fc198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.500+0000 7f4a021ff700 1 --2- 172.21.15.133:0/312673007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49fc100290 0x7f49fc198290 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f49ec00e9f0 tx=0x7f49ec00ed00 comp rx=0 tx=0).stop 2023-11-27T15:20:39.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.500+0000 7f4a021ff700 1 -- 172.21.15.133:0/312673007 shutdown_connections 2023-11-27T15:20:39.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.500+0000 7f4a021ff700 1 --2- 172.21.15.133:0/312673007 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f49e80a11d0 0x7f49e80a3680 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.500+0000 7f4a021ff700 1 --2- 172.21.15.133:0/312673007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49fc101e10 0x7f49fc19f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.500+0000 7f4a021ff700 1 --2- 172.21.15.133:0/312673007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49fc100290 0x7f49fc198290 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.500+0000 7f4a021ff700 1 --2- 172.21.15.133:0/312673007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f49fc101480 0x7f49fc1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.500+0000 7f4a021ff700 1 -- 172.21.15.133:0/312673007 >> 172.21.15.133:0/312673007 conn(0x7f49fc0fb870 msgr2=0x7f49fc0fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:39.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.501+0000 7f4a021ff700 1 -- 172.21.15.133:0/312673007 shutdown_connections 2023-11-27T15:20:39.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.501+0000 7f4a021ff700 1 -- 172.21.15.133:0/312673007 wait complete. 2023-11-27T15:20:39.535 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:20:39.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.729+0000 7fcdecbff700 1 -- 172.21.15.133:0/2629270921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcde81036b0 msgr2=0x7fcde8105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.729+0000 7fcdecbff700 1 --2- 172.21.15.133:0/2629270921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcde81036b0 0x7fcde8105a90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009990 tx=0x7fcddc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.729+0000 7fcdecbff700 1 -- 172.21.15.133:0/2629270921 shutdown_connections 2023-11-27T15:20:39.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.729+0000 7fcdecbff700 1 --2- 172.21.15.133:0/2629270921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde8105fd0 0x7fcde81083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.729+0000 7fcdecbff700 1 --2- 172.21.15.133:0/2629270921 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcde8100d90 0x7fcde8103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.729+0000 7fcdecbff700 1 --2- 172.21.15.133:0/2629270921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcde81036b0 0x7fcde8105a90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.733 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.729+0000 7fcdecbff700 1 -- 172.21.15.133:0/2629270921 >> 172.21.15.133:0/2629270921 conn(0x7fcde80fa750 msgr2=0x7fcde80fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:39.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.729+0000 7fcdecbff700 1 -- 172.21.15.133:0/2629270921 shutdown_connections 2023-11-27T15:20:39.734 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.729+0000 7fcdecbff700 1 -- 172.21.15.133:0/2629270921 wait complete. 2023-11-27T15:20:39.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.730+0000 7fcdecbff700 1 Processor -- start 2023-11-27T15:20:39.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.730+0000 7fcdecbff700 1 -- start start 2023-11-27T15:20:39.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.730+0000 7fcdecbff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcde8100d90 0x7fcde8195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.730+0000 7fcdecbff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde81036b0 0x7fcde8196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.730+0000 7fcdecbff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcde8105fd0 0x7fcde819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.730+0000 7fcdecbff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde819db40 con 0x7fcde81036b0 2023-11-27T15:20:39.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.730+0000 7fcdecbff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde819dc80 con 0x7fcde8105fd0 2023-11-27T15:20:39.736 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.730+0000 7fcdecbff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde819de10 con 0x7fcde8100d90 2023-11-27T15:20:39.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.731+0000 7fcde5d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde81036b0 0x7fcde8196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.731+0000 7fcde5d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde81036b0 0x7fcde8196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44844/0 (socket says 172.21.15.133:44844) 2023-11-27T15:20:39.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.731+0000 7fcde5d9b700 1 -- 172.21.15.133:0/190220272 learned_addr learned my addr 172.21.15.133:0/190220272 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:39.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.731+0000 7fcde6d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcde8105fd0 0x7fcde819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.731+0000 7fcde659c700 1 --2- 172.21.15.133:0/190220272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcde8100d90 0x7fcde8195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.731+0000 7fcde5d9b700 1 -- 172.21.15.133:0/190220272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcde8100d90 msgr2=0x7fcde8195fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.731+0000 7fcde5d9b700 1 --2- 172.21.15.133:0/190220272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcde8100d90 0x7fcde8195fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.731+0000 7fcde5d9b700 1 -- 172.21.15.133:0/190220272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcde8105fd0 msgr2=0x7fcde819d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.731+0000 7fcde5d9b700 1 --2- 172.21.15.133:0/190220272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcde8105fd0 0x7fcde819d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.731+0000 7fcde5d9b700 1 -- 172.21.15.133:0/190220272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc0095f0 con 0x7fcde81036b0 2023-11-27T15:20:39.737 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.731+0000 7fcde5d9b700 1 --2- 172.21.15.133:0/190220272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde81036b0 0x7fcde8196500 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcddc004730 tx=0x7fcddc004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:39.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.731+0000 7fcdd77fe700 1 -- 172.21.15.133:0/190220272 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc01d070 con 0x7fcde81036b0 2023-11-27T15:20:39.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.731+0000 7fcdd77fe700 1 -- 172.21.15.133:0/190220272 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcddc022470 con 0x7fcde81036b0 2023-11-27T15:20:39.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.731+0000 7fcdecbff700 1 -- 172.21.15.133:0/190220272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde810bc60 con 0x7fcde81036b0 2023-11-27T15:20:39.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.731+0000 7fcdd77fe700 1 -- 172.21.15.133:0/190220272 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc017650 con 0x7fcde81036b0 2023-11-27T15:20:39.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.732+0000 7fcdecbff700 1 -- 172.21.15.133:0/190220272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde810bad0 con 0x7fcde81036b0 2023-11-27T15:20:39.738 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.732+0000 7fcdecbff700 1 -- 172.21.15.133:0/190220272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde819e410 con 0x7fcde81036b0 2023-11-27T15:20:39.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.734+0000 7fcdd77fe700 1 -- 172.21.15.133:0/190220272 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fcddc0225e0 con 0x7fcde81036b0 2023-11-27T15:20:39.739 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.736+0000 7fcdd77fe700 1 --2- 172.21.15.133:0/190220272 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcdd00a11d0 0x7fcdd00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:39.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.736+0000 7fcdd77fe700 1 -- 172.21.15.133:0/190220272 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcddc0cde50 con 0x7fcde81036b0 2023-11-27T15:20:39.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.736+0000 7fcde659c700 1 --2- 172.21.15.133:0/190220272 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcdd00a11d0 0x7fcdd00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:39.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.736+0000 7fcde659c700 1 --2- 172.21.15.133:0/190220272 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcdd00a11d0 0x7fcdd00a3680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcde8068890 tx=0x7fcdd8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:39.743 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.739+0000 7fcdd77fe700 1 -- 172.21.15.133:0/190220272 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcddc0973e0 con 0x7fcde81036b0 2023-11-27T15:20:39.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:39 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1737593332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:20:39.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:39 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/312673007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:20:39.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.987+0000 7fcdecbff700 1 -- 172.21.15.133:0/190220272 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcde819e9c0 con 0x7fcde81036b0 2023-11-27T15:20:39.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.988+0000 7fcdd77fe700 1 -- 172.21.15.133:0/190220272 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10768 (secure 0 0 0) 0x7fcddc027090 con 0x7fcde81036b0 2023-11-27T15:20:39.992 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:39.992 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14493,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2381290259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2381290259},{"type":"v1","addr":"172.21.15.133:6839","nonce":2381290259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1704875895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1704875895},{"type":"v1","addr":"172.21.15.133:6841","nonce":1704875895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24491,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24496,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6837/3492215689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3492215689},{"type":"v1","addr":"172.21.15.177:6837","nonce":3492215689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24520,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-27T15:20:33.201023+0000","modified":"2023-11-27T15:20:38.147285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24560},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6839/2781994447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2781994447},{"type":"v1","addr":"172.21.15.174:6839","nonce":2781994447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-27T15:20:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.990+0000 7fcdecbff700 1 -- 172.21.15.133:0/190220272 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcdd00a11d0 msgr2=0x7fcdd00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.990+0000 7fcdecbff700 1 --2- 172.21.15.133:0/190220272 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcdd00a11d0 0x7fcdd00a3680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcde8068890 tx=0x7fcdd8009260 comp rx=0 tx=0).stop 2023-11-27T15:20:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.990+0000 7fcdecbff700 1 -- 172.21.15.133:0/190220272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde81036b0 msgr2=0x7fcde8196500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.990+0000 7fcdecbff700 1 --2- 172.21.15.133:0/190220272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde81036b0 0x7fcde8196500 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcddc004730 tx=0x7fcddc004760 comp rx=0 tx=0).stop 2023-11-27T15:20:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.990+0000 7fcdecbff700 1 -- 172.21.15.133:0/190220272 shutdown_connections 2023-11-27T15:20:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.990+0000 7fcdecbff700 1 --2- 172.21.15.133:0/190220272 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcdd00a11d0 0x7fcdd00a3680 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.990+0000 7fcdecbff700 1 --2- 172.21.15.133:0/190220272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcde8105fd0 0x7fcde819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.990+0000 7fcdecbff700 1 --2- 172.21.15.133:0/190220272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcde8100d90 0x7fcde8195fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.990+0000 7fcdecbff700 1 --2- 172.21.15.133:0/190220272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcde81036b0 0x7fcde8196500 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.990+0000 7fcdecbff700 1 -- 172.21.15.133:0/190220272 >> 172.21.15.133:0/190220272 conn(0x7fcde80fa750 msgr2=0x7fcde8102270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:39.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.991+0000 7fcdecbff700 1 -- 172.21.15.133:0/190220272 shutdown_connections 2023-11-27T15:20:39.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:39.991+0000 7fcdecbff700 1 -- 172.21.15.133:0/190220272 wait complete. 2023-11-27T15:20:39.996 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 17 2023-11-27T15:20:40.013 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:20:40.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:39 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1737593332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:20:40.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:39 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/312673007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:20:40.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:39 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1737593332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:20:40.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:39 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/312673007' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:20:40.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.207+0000 7fa04ebff700 1 -- 172.21.15.133:0/2445571167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0481002b0 msgr2=0x7fa0481006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.207+0000 7fa04ebff700 1 --2- 172.21.15.133:0/2445571167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0481002b0 0x7fa0481006c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0098f0 tx=0x7fa03c009c00 comp rx=0 tx=0).stop 2023-11-27T15:20:40.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.207+0000 7fa04ebff700 1 -- 172.21.15.133:0/2445571167 shutdown_connections 2023-11-27T15:20:40.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.207+0000 7fa04ebff700 1 --2- 172.21.15.133:0/2445571167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa048101e30 0x7fa0481086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.207+0000 7fa04ebff700 1 --2- 172.21.15.133:0/2445571167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa0481002b0 0x7fa0481006c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.207+0000 7fa04ebff700 1 --2- 172.21.15.133:0/2445571167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0481014a0 0x7fa0481018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.207+0000 7fa04ebff700 1 -- 172.21.15.133:0/2445571167 >> 172.21.15.133:0/2445571167 conn(0x7fa0480fb890 msgr2=0x7fa0480fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:40.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.207+0000 7fa04ebff700 1 -- 172.21.15.133:0/2445571167 shutdown_connections 2023-11-27T15:20:40.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.207+0000 7fa04ebff700 1 -- 172.21.15.133:0/2445571167 wait complete. 2023-11-27T15:20:40.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.208+0000 7fa04ebff700 1 Processor -- start 2023-11-27T15:20:40.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.208+0000 7fa04ebff700 1 -- start start 2023-11-27T15:20:40.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.208+0000 7fa04ebff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0481002b0 0x7fa0480748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.208+0000 7fa04ebff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0481014a0 0x7fa048072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.208+0000 7fa04ebff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa048101e30 0x7fa048073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.208+0000 7fa04ebff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa048073bc0 con 0x7fa0481014a0 2023-11-27T15:20:40.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.208+0000 7fa04ebff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa048073d30 con 0x7fa0481002b0 2023-11-27T15:20:40.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.208+0000 7fa04ebff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0481a00b0 con 0x7fa048101e30 2023-11-27T15:20:40.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.208+0000 7fa047fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0481014a0 0x7fa048072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.208+0000 7fa047fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0481014a0 0x7fa048072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44860/0 (socket says 172.21.15.133:44860) 2023-11-27T15:20:40.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.209+0000 7fa047fff700 1 -- 172.21.15.133:0/173318620 learned_addr learned my addr 172.21.15.133:0/173318620 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:40.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.209+0000 7fa04c99b700 1 --2- 172.21.15.133:0/173318620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0481002b0 0x7fa0480748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.209+0000 7fa04d19c700 1 --2- 172.21.15.133:0/173318620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa048101e30 0x7fa048073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.209+0000 7fa047fff700 1 -- 172.21.15.133:0/173318620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa048101e30 msgr2=0x7fa048073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.209+0000 7fa047fff700 1 --2- 172.21.15.133:0/173318620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa048101e30 0x7fa048073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.209+0000 7fa047fff700 1 -- 172.21.15.133:0/173318620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0481002b0 msgr2=0x7fa0480748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.209+0000 7fa047fff700 1 --2- 172.21.15.133:0/173318620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0481002b0 0x7fa0480748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.209+0000 7fa047fff700 1 -- 172.21.15.133:0/173318620 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa03c009520 con 0x7fa0481014a0 2023-11-27T15:20:40.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.209+0000 7fa047fff700 1 --2- 172.21.15.133:0/173318620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0481014a0 0x7fa048072ee0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa03400e9c0 tx=0x7fa03400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.209+0000 7fa045ffb700 1 -- 172.21.15.133:0/173318620 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa034004a30 con 0x7fa0481014a0 2023-11-27T15:20:40.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.209+0000 7fa045ffb700 1 -- 172.21.15.133:0/173318620 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa03400cc80 con 0x7fa0481014a0 2023-11-27T15:20:40.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.209+0000 7fa04ebff700 1 -- 172.21.15.133:0/173318620 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa04810bc00 con 0x7fa0481014a0 2023-11-27T15:20:40.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.209+0000 7fa045ffb700 1 -- 172.21.15.133:0/173318620 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0340054e0 con 0x7fa0481014a0 2023-11-27T15:20:40.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.210+0000 7fa04ebff700 1 -- 172.21.15.133:0/173318620 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa04810ba70 con 0x7fa0481014a0 2023-11-27T15:20:40.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.211+0000 7fa04ebff700 1 -- 172.21.15.133:0/173318620 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa048107900 con 0x7fa0481014a0 2023-11-27T15:20:40.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.212+0000 7fa045ffb700 1 -- 172.21.15.133:0/173318620 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa034004b90 con 0x7fa0481014a0 2023-11-27T15:20:40.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.214+0000 7fa045ffb700 1 --2- 172.21.15.133:0/173318620 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa0380a1150 0x7fa0380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.214+0000 7fa045ffb700 1 -- 172.21.15.133:0/173318620 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa034014070 con 0x7fa0481014a0 2023-11-27T15:20:40.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.214+0000 7fa04c99b700 1 --2- 172.21.15.133:0/173318620 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa0380a1150 0x7fa0380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.215+0000 7fa04c99b700 1 --2- 172.21.15.133:0/173318620 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa0380a1150 0x7fa0380a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009fd0 tx=0x7fa03c019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.217+0000 7fa045ffb700 1 -- 172.21.15.133:0/173318620 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa034096130 con 0x7fa0481014a0 2023-11-27T15:20:40.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.415+0000 7fa04ebff700 1 -- 172.21.15.133:0/173318620 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa04802cfe0 con 0x7fa0481014a0 2023-11-27T15:20:40.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.417+0000 7fa045ffb700 1 -- 172.21.15.133:0/173318620 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23996 (secure 0 0 0) 0x7fa034099750 con 0x7fa0481014a0 2023-11-27T15:20:40.421 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:40.422 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:20:38.147036+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:20:40.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.419+0000 7fa04ebff700 1 -- 172.21.15.133:0/173318620 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa0380a1150 msgr2=0x7fa0380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.419+0000 7fa04ebff700 1 --2- 172.21.15.133:0/173318620 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa0380a1150 0x7fa0380a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009fd0 tx=0x7fa03c019040 comp rx=0 tx=0).stop 2023-11-27T15:20:40.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.420+0000 7fa04ebff700 1 -- 172.21.15.133:0/173318620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0481014a0 msgr2=0x7fa048072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.420+0000 7fa04ebff700 1 --2- 172.21.15.133:0/173318620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0481014a0 0x7fa048072ee0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa03400e9c0 tx=0x7fa03400ecd0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.420+0000 7fa04ebff700 1 -- 172.21.15.133:0/173318620 shutdown_connections 2023-11-27T15:20:40.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.420+0000 7fa04ebff700 1 --2- 172.21.15.133:0/173318620 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa0380a1150 0x7fa0380a3600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.420+0000 7fa04ebff700 1 --2- 172.21.15.133:0/173318620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa048101e30 0x7fa048073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.420+0000 7fa04ebff700 1 --2- 172.21.15.133:0/173318620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0481002b0 0x7fa0480748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.420+0000 7fa04ebff700 1 --2- 172.21.15.133:0/173318620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0481014a0 0x7fa048072ee0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.420+0000 7fa04ebff700 1 -- 172.21.15.133:0/173318620 >> 172.21.15.133:0/173318620 conn(0x7fa0480fb890 msgr2=0x7fa048104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:40.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.420+0000 7fa04ebff700 1 -- 172.21.15.133:0/173318620 shutdown_connections 2023-11-27T15:20:40.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.420+0000 7fa04ebff700 1 -- 172.21.15.133:0/173318620 wait complete. 2023-11-27T15:20:40.443 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-27T15:20:33.201023+0000', 'modified': '2023-11-27T15:20:38.147285+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24560}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24560': {'gid': 24560, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.174:6839/2781994447', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 2781994447}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 2781994447}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-11-27T15:20:40.443 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-27T15:20:40.443 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-27T15:20:40.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.637+0000 7ff7cd7ff700 1 -- 172.21.15.133:0/3309330897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7c8100230 msgr2=0x7ff7c8100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.637+0000 7ff7cd7ff700 1 --2- 172.21.15.133:0/3309330897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7c8100230 0x7ff7c8100640 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009990 tx=0x7ff7b8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.637+0000 7ff7cd7ff700 1 -- 172.21.15.133:0/3309330897 shutdown_connections 2023-11-27T15:20:40.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.637+0000 7ff7cd7ff700 1 --2- 172.21.15.133:0/3309330897 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7c8101db0 0x7ff7c8108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.637+0000 7ff7cd7ff700 1 --2- 172.21.15.133:0/3309330897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7c8100230 0x7ff7c8100640 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.637+0000 7ff7cd7ff700 1 --2- 172.21.15.133:0/3309330897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c8101420 0x7ff7c8101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.637+0000 7ff7cd7ff700 1 -- 172.21.15.133:0/3309330897 >> 172.21.15.133:0/3309330897 conn(0x7ff7c80fb810 msgr2=0x7ff7c80fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:40.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.637+0000 7ff7cd7ff700 1 -- 172.21.15.133:0/3309330897 shutdown_connections 2023-11-27T15:20:40.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.637+0000 7ff7cd7ff700 1 -- 172.21.15.133:0/3309330897 wait complete. 2023-11-27T15:20:40.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.638+0000 7ff7cd7ff700 1 Processor -- start 2023-11-27T15:20:40.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.638+0000 7ff7cd7ff700 1 -- start start 2023-11-27T15:20:40.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.639+0000 7ff7cd7ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7c8100230 0x7ff7c8195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.639+0000 7ff7cd7ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c8101420 0x7ff7c8196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.639+0000 7ff7cd7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7c8101db0 0x7ff7c819d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.639+0000 7ff7cd7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c819db60 con 0x7ff7c8101db0 2023-11-27T15:20:40.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.639+0000 7ff7cd7ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c819dca0 con 0x7ff7c8101420 2023-11-27T15:20:40.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.639+0000 7ff7cd7ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c819de30 con 0x7ff7c8100230 2023-11-27T15:20:40.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.639+0000 7ff7c77fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7c8101db0 0x7ff7c819d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.639+0000 7ff7c67fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c8101420 0x7ff7c8196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.639+0000 7ff7c77fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7c8101db0 0x7ff7c819d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44876/0 (socket says 172.21.15.133:44876) 2023-11-27T15:20:40.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.639+0000 7ff7c67fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c8101420 0x7ff7c8196520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:57206/0 (socket says 172.21.15.133:57206) 2023-11-27T15:20:40.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.639+0000 7ff7c77fe700 1 -- 172.21.15.133:0/39218613 learned_addr learned my addr 172.21.15.133:0/39218613 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:40.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.639+0000 7ff7c6ffd700 1 --2- 172.21.15.133:0/39218613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7c8100230 0x7ff7c8195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.639+0000 7ff7c77fe700 1 -- 172.21.15.133:0/39218613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7c8100230 msgr2=0x7ff7c8195fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.639+0000 7ff7c77fe700 1 --2- 172.21.15.133:0/39218613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7c8100230 0x7ff7c8195fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.639+0000 7ff7c77fe700 1 -- 172.21.15.133:0/39218613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c8101420 msgr2=0x7ff7c8196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.639+0000 7ff7c77fe700 1 --2- 172.21.15.133:0/39218613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c8101420 0x7ff7c8196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.639+0000 7ff7c77fe700 1 -- 172.21.15.133:0/39218613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b80095f0 con 0x7ff7c8101db0 2023-11-27T15:20:40.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.640+0000 7ff7c77fe700 1 --2- 172.21.15.133:0/39218613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7c8101db0 0x7ff7c819d590 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000ef30 tx=0x7ff7c000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.640+0000 7ff7affff700 1 -- 172.21.15.133:0/39218613 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0018070 con 0x7ff7c8101db0 2023-11-27T15:20:40.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.640+0000 7ff7affff700 1 -- 172.21.15.133:0/39218613 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7c000f040 con 0x7ff7c8101db0 2023-11-27T15:20:40.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.640+0000 7ff7affff700 1 -- 172.21.15.133:0/39218613 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c000cdb0 con 0x7ff7c8101db0 2023-11-27T15:20:40.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.640+0000 7ff7cd7ff700 1 -- 172.21.15.133:0/39218613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c810bc10 con 0x7ff7c8101db0 2023-11-27T15:20:40.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.640+0000 7ff7cd7ff700 1 -- 172.21.15.133:0/39218613 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c810ba80 con 0x7ff7c8101db0 2023-11-27T15:20:40.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.643+0000 7ff7affff700 1 -- 172.21.15.133:0/39218613 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff7c0003a00 con 0x7ff7c8101db0 2023-11-27T15:20:40.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.645+0000 7ff7e11aeb80 1 Processor -- start 2023-11-27T15:20:40.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.646+0000 7ff7e11aeb80 1 -- start start 2023-11-27T15:20:40.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.646+0000 7ff7affff700 1 --2- 172.21.15.133:0/39218613 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff7b00a11d0 0x7ff7b00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.646+0000 7ff7e11aeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537050 0x55cf48537460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.646+0000 7ff7e11aeb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537ee0 0x55cf48538350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.646+0000 7ff7e11aeb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 0x55cf4853f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.646+0000 7ff7e11aeb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf48438010 con 0x55cf48537ee0 2023-11-27T15:20:40.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.646+0000 7ff7e11aeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf4848a850 con 0x55cf48537050 2023-11-27T15:20:40.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.646+0000 7ff7e11aeb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf48483ce0 con 0x55cf48538890 2023-11-27T15:20:40.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.646+0000 7ff7affff700 1 -- 172.21.15.133:0/39218613 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff7c00cc710 con 0x7ff7c8101db0 2023-11-27T15:20:40.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.646+0000 7ff7c67fc700 1 --2- 172.21.15.133:0/39218613 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff7b00a11d0 0x7ff7b00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.646+0000 7ff7c6ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537050 0x55cf48537460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.646+0000 7ff7c6ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537050 0x55cf48537460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:57214/0 (socket says 172.21.15.133:57214) 2023-11-27T15:20:40.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.646+0000 7ff7c6ffd700 1 -- 172.21.15.133:0/1954932848 learned_addr learned my addr 172.21.15.133:0/1954932848 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:40.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.646+0000 7ff7c77fe700 1 --2- 172.21.15.133:0/1954932848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 0x55cf4853f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.646+0000 7ff7c67fc700 1 --2- 172.21.15.133:0/1954932848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537ee0 0x55cf48538350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.647+0000 7ff7c77fe700 1 -- 172.21.15.133:0/1954932848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537050 msgr2=0x55cf48537460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.647+0000 7ff7c77fe700 1 --2- 172.21.15.133:0/1954932848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537050 0x55cf48537460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.647+0000 7ff7c77fe700 1 -- 172.21.15.133:0/1954932848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537ee0 msgr2=0x55cf48538350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.647+0000 7ff7c77fe700 1 --2- 172.21.15.133:0/1954932848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537ee0 0x55cf48538350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.647+0000 7ff7c77fe700 1 -- 172.21.15.133:0/1954932848 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf483055e0 con 0x55cf48538890 2023-11-27T15:20:40.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.647+0000 7ff7c6ffd700 1 --2- 172.21.15.133:0/1954932848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537050 0x55cf48537460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:40.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.647+0000 7ff7c77fe700 1 --2- 172.21.15.133:0/1954932848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 0x55cf4853f130 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0099660 tx=0x7ff7c00997b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebabc37284adda79 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.647+0000 7ff7c67fc700 1 --2- 172.21.15.133:0/39218613 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff7b00a11d0 0x7ff7b00a3680 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc0119b0 tx=0x7ff7bc009bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.647+0000 7ff7ad7fa700 1 -- 172.21.15.133:0/1954932848 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0004ad0 con 0x55cf48538890 2023-11-27T15:20:40.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.647+0000 7ff7ad7fa700 1 -- 172.21.15.133:0/1954932848 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7c0099960 con 0x55cf48538890 2023-11-27T15:20:40.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.647+0000 7ff7ad7fa700 1 -- 172.21.15.133:0/1954932848 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c0098e50 con 0x55cf48538890 2023-11-27T15:20:40.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.648+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/1954932848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 msgr2=0x55cf4853f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.648+0000 7ff7e11aeb80 1 --2- 172.21.15.133:0/1954932848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 0x55cf4853f130 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0099660 tx=0x7ff7c00997b0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.648+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/1954932848 shutdown_connections 2023-11-27T15:20:40.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.648+0000 7ff7e11aeb80 1 --2- 172.21.15.133:0/1954932848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 0x55cf4853f130 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.648+0000 7ff7e11aeb80 1 --2- 172.21.15.133:0/1954932848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537050 0x55cf48537460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.648+0000 7ff7e11aeb80 1 --2- 172.21.15.133:0/1954932848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537ee0 0x55cf48538350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.648+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/1954932848 >> 172.21.15.133:0/1954932848 conn(0x55cf48528310 msgr2=0x55cf4852d620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:40.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.648+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/1954932848 shutdown_connections 2023-11-27T15:20:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.648+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/1954932848 wait complete. 2023-11-27T15:20:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.649+0000 7ff7e11aeb80 1 -- start start 2023-11-27T15:20:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.649+0000 7ff7e11aeb80 1 Processor -- start 2023-11-27T15:20:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.649+0000 7ff7e11aeb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537050 0x55cf485409f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.649+0000 7ff7e11aeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537ee0 0x55cf48540f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.649+0000 7ff7e11aeb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 0x55cf48547f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.649+0000 7ff7e11aeb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf48425890 con 0x55cf48537050 2023-11-27T15:20:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.649+0000 7ff7e11aeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf48425530 con 0x55cf48537ee0 2023-11-27T15:20:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.649+0000 7ff7e11aeb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf483078d0 con 0x55cf48538890 2023-11-27T15:20:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.649+0000 7ff7e11aeb80 10 client.0 _ll_register_callbacks cb 0x55cf48443730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:40.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.649+0000 7ff7c6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537050 0x55cf485409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.650+0000 7ff7c6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537050 0x55cf485409f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44898/0 (socket says 172.21.15.133:44898) 2023-11-27T15:20:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.650+0000 7ff7c6ffd700 1 -- 172.21.15.133:0/2343823658 learned_addr learned my addr 172.21.15.133:0/2343823658 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.650+0000 7ff7c77fe700 1 --2- 172.21.15.133:0/2343823658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 0x55cf48547f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.650+0000 7ff7c67fc700 1 --2- 172.21.15.133:0/2343823658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537ee0 0x55cf48540f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.650+0000 7ff7c6ffd700 1 -- 172.21.15.133:0/2343823658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 msgr2=0x55cf48547f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.650+0000 7ff7c6ffd700 1 --2- 172.21.15.133:0/2343823658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 0x55cf48547f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.650+0000 7ff7c6ffd700 1 -- 172.21.15.133:0/2343823658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537ee0 msgr2=0x55cf48540f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.650+0000 7ff7c6ffd700 1 --2- 172.21.15.133:0/2343823658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537ee0 0x55cf48540f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.650+0000 7ff7c6ffd700 1 -- 172.21.15.133:0/2343823658 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c810ba80 con 0x55cf48537050 2023-11-27T15:20:40.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.650+0000 7ff7c6ffd700 1 --2- 172.21.15.133:0/2343823658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537050 0x55cf485409f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009990 tx=0x7ff7b8005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de69c8a06c42dcd server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.650+0000 7ff7aaff5700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:20:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.650+0000 7ff7aaff5700 1 -- 172.21.15.133:0/2343823658 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8005920 con 0x55cf48537050 2023-11-27T15:20:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.650+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/2343823658 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf48525ad0 con 0x55cf48537050 2023-11-27T15:20:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.650+0000 7ff7aaff5700 1 -- 172.21.15.133:0/2343823658 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff7b8004500 con 0x55cf48537050 2023-11-27T15:20:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.651+0000 7ff7aaff5700 1 -- 172.21.15.133:0/2343823658 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7b80046c0 con 0x55cf48537050 2023-11-27T15:20:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.651+0000 7ff7e11aeb80 10 client.14622 fetch_fsmap learned FSMap version 17 2023-11-27T15:20:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.651+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/2343823658 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55cf4844a140 con 0x55cf48537050 2023-11-27T15:20:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.651+0000 7ff7aaff5700 1 -- 172.21.15.133:0/2343823658 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cf48525ad0 con 0x55cf48537050 2023-11-27T15:20:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.651+0000 7ff7aaff5700 1 -- 172.21.15.133:0/2343823658 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff7b802c070 con 0x55cf48537050 2023-11-27T15:20:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.651+0000 7ff7e11aeb80 10 client.14622 fetch_fsmap finished waiting for FSMap version 17 2023-11-27T15:20:40.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.651+0000 7ff7e11aeb80 10 client.14622 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-27T15:20:40.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.651+0000 7ff7e11aeb80 20 client.14622 populate_metadata read hostname 'smithi133' 2023-11-27T15:20:40.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.651+0000 7ff7e11aeb80 1 --2- 172.21.15.133:0/2343823658 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55cf4854bc40 0x55cf4854b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.651+0000 7ff7e11aeb80 4 client.14622 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:20:40.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.651+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/2343823658 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf48446050 con 0x55cf4854bc40 2023-11-27T15:20:40.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.652+0000 7ff7c67fc700 1 --2- 172.21.15.133:0/2343823658 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55cf4854bc40 0x55cf4854b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.653+0000 7ff7c67fc700 1 --2- 172.21.15.133:0/2343823658 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55cf4854bc40 0x55cf4854b070 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47ff0d00d3625417 server_cookie=94a5f7e0e5d42e0f in_seq=0 out_seq=0 2023-11-27T15:20:40.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.653+0000 7ff7aaff5700 10 client.14622 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:20:40.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.653+0000 7ff7aaff5700 1 -- 172.21.15.133:0/2343823658 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cf48446050 con 0x55cf4854bc40 2023-11-27T15:20:40.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.653+0000 7ff7aaff5700 10 client.14622 handle_command_reply: tid=0 2023-11-27T15:20:40.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.653+0000 7ff7e11aeb80 1 client.14622 shutdown 2023-11-27T15:20:40.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.654+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/2343823658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537050 msgr2=0x55cf485409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.654+0000 7ff7e11aeb80 1 --2- 172.21.15.133:0/2343823658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537050 0x55cf485409f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009990 tx=0x7ff7b8005750 comp rx=0 tx=0).stop 2023-11-27T15:20:40.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.654+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/2343823658 shutdown_connections 2023-11-27T15:20:40.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.654+0000 7ff7e11aeb80 1 --2- 172.21.15.133:0/2343823658 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55cf4854bc40 0x55cf4854b070 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.654+0000 7ff7e11aeb80 1 --2- 172.21.15.133:0/2343823658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 0x55cf48547f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.654+0000 7ff7e11aeb80 1 --2- 172.21.15.133:0/2343823658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537050 0x55cf485409f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.654+0000 7ff7e11aeb80 1 --2- 172.21.15.133:0/2343823658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537ee0 0x55cf48540f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.654+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/2343823658 >> 172.21.15.133:0/2343823658 conn(0x55cf48528310 msgr2=0x55cf4852cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:40.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.654+0000 7ff7aaff5700 0 client.14622 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:20:40.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.654+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/2343823658 shutdown_connections 2023-11-27T15:20:40.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.654+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/2343823658 wait complete. 2023-11-27T15:20:40.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.654+0000 7ff7e11aeb80 20 client.14622 trim_cache size 0 max 16384 2023-11-27T15:20:40.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.674+0000 7ff7e11aeb80 1 Processor -- start 2023-11-27T15:20:40.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.674+0000 7ff7e11aeb80 1 -- start start 2023-11-27T15:20:40.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.674+0000 7ff7e11aeb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537050 0x55cf4854b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.674+0000 7ff7e11aeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537ee0 0x55cf48547e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.674+0000 7ff7e11aeb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 0x55cf4853b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.674+0000 7ff7e11aeb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf48541a80 con 0x55cf48537050 2023-11-27T15:20:40.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.674+0000 7ff7e11aeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf4852ee50 con 0x55cf48537ee0 2023-11-27T15:20:40.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.675+0000 7ff7e11aeb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf48537d90 con 0x55cf48538890 2023-11-27T15:20:40.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.675+0000 7ff7c6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537050 0x55cf4854b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.675+0000 7ff7c6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537050 0x55cf4854b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44900/0 (socket says 172.21.15.133:44900) 2023-11-27T15:20:40.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.675+0000 7ff7c6ffd700 1 -- 172.21.15.133:0/2992529861 learned_addr learned my addr 172.21.15.133:0/2992529861 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:40.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.675+0000 7ff7c67fc700 1 --2- 172.21.15.133:0/2992529861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537ee0 0x55cf48547e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.675+0000 7ff7c77fe700 1 --2- 172.21.15.133:0/2992529861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 0x55cf4853b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.675+0000 7ff7c6ffd700 1 -- 172.21.15.133:0/2992529861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 msgr2=0x55cf4853b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.675+0000 7ff7c6ffd700 1 --2- 172.21.15.133:0/2992529861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 0x55cf4853b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.675+0000 7ff7c6ffd700 1 -- 172.21.15.133:0/2992529861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537ee0 msgr2=0x55cf48547e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.675+0000 7ff7c6ffd700 1 --2- 172.21.15.133:0/2992529861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537ee0 0x55cf48547e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.675+0000 7ff7c6ffd700 1 -- 172.21.15.133:0/2992529861 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf48449e90 con 0x55cf48537050 2023-11-27T15:20:40.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.675+0000 7ff7c67fc700 1 --2- 172.21.15.133:0/2992529861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537ee0 0x55cf48547e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:20:40.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.675+0000 7ff7c77fe700 1 --2- 172.21.15.133:0/2992529861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 0x55cf4853b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:20:40.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.675+0000 7ff7c6ffd700 1 --2- 172.21.15.133:0/2992529861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537050 0x55cf4854b070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800f9c0 tx=0x7ff7b800b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c26dfda07feb6d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.676+0000 7ff7adffb700 1 -- 172.21.15.133:0/2992529861 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8039030 con 0x55cf48537050 2023-11-27T15:20:40.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.676+0000 7ff7adffb700 1 -- 172.21.15.133:0/2992529861 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7b801e040 con 0x55cf48537050 2023-11-27T15:20:40.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.676+0000 7ff7adffb700 1 -- 172.21.15.133:0/2992529861 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8004760 con 0x55cf48537050 2023-11-27T15:20:40.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.676+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/2992529861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537050 msgr2=0x55cf4854b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.676+0000 7ff7e11aeb80 1 --2- 172.21.15.133:0/2992529861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537050 0x55cf4854b070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800f9c0 tx=0x7ff7b800b2e0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.676+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/2992529861 shutdown_connections 2023-11-27T15:20:40.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.676+0000 7ff7e11aeb80 1 --2- 172.21.15.133:0/2992529861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 0x55cf4853b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.676+0000 7ff7e11aeb80 1 --2- 172.21.15.133:0/2992529861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537050 0x55cf4854b070 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.676+0000 7ff7e11aeb80 1 --2- 172.21.15.133:0/2992529861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537ee0 0x55cf48547e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.676+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/2992529861 >> 172.21.15.133:0/2992529861 conn(0x55cf48528310 msgr2=0x55cf4853f700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:40.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.676+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/2992529861 shutdown_connections 2023-11-27T15:20:40.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.676+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/2992529861 wait complete. 2023-11-27T15:20:40.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.677+0000 7ff7e11aeb80 1 -- start start 2023-11-27T15:20:40.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.677+0000 7ff7e11aeb80 1 Processor -- start 2023-11-27T15:20:40.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.677+0000 7ff7e11aeb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537050 0x55cf485479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.677+0000 7ff7e11aeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537ee0 0x55cf48547f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.677+0000 7ff7e11aeb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 0x55cf485445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.677+0000 7ff7e11aeb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf48544ca0 con 0x55cf48537050 2023-11-27T15:20:40.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.677+0000 7ff7e11aeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf48544e10 con 0x55cf48537ee0 2023-11-27T15:20:40.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.677+0000 7ff7e11aeb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf48544fd0 con 0x55cf48538890 2023-11-27T15:20:40.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.678+0000 7ff7e11aeb80 10 client.0 _ll_register_callbacks cb 0x7ff7b8009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:20:40.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.678+0000 7ff7c6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537050 0x55cf485479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.678+0000 7ff7c6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537050 0x55cf485479e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44908/0 (socket says 172.21.15.133:44908) 2023-11-27T15:20:40.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.678+0000 7ff7c6ffd700 1 -- 172.21.15.133:0/1095057200 learned_addr learned my addr 172.21.15.133:0/1095057200 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:40.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.678+0000 7ff7c77fe700 1 --2- 172.21.15.133:0/1095057200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 0x55cf485445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.678+0000 7ff7c67fc700 1 --2- 172.21.15.133:0/1095057200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537ee0 0x55cf48547f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.678+0000 7ff7c6ffd700 1 -- 172.21.15.133:0/1095057200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 msgr2=0x55cf485445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.678+0000 7ff7c6ffd700 1 --2- 172.21.15.133:0/1095057200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 0x55cf485445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.678+0000 7ff7c6ffd700 1 -- 172.21.15.133:0/1095057200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537ee0 msgr2=0x55cf48547f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:20:40.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.678+0000 7ff7c6ffd700 1 --2- 172.21.15.133:0/1095057200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537ee0 0x55cf48547f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.678+0000 7ff7c6ffd700 1 -- 172.21.15.133:0/1095057200 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cf4844a140 con 0x55cf48537050 2023-11-27T15:20:40.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.678+0000 7ff7c67fc700 1 --2- 172.21.15.133:0/1095057200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537ee0 0x55cf48547f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:20:40.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.678+0000 7ff7c6ffd700 1 --2- 172.21.15.133:0/1095057200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537050 0x55cf485479e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80177c0 tx=0x7ff7b800fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4269b51c4a3cf8d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:40.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.678+0000 7ff7aaff5700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:20:40.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.678+0000 7ff7aaff5700 1 -- 172.21.15.133:0/1095057200 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8039040 con 0x55cf48537050 2023-11-27T15:20:40.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.679+0000 7ff7aaff5700 1 -- 172.21.15.133:0/1095057200 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff7b803a070 con 0x55cf48537050 2023-11-27T15:20:40.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.679+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/1095057200 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7c0099960 con 0x55cf48537050 2023-11-27T15:20:40.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.679+0000 7ff7aaff5700 1 -- 172.21.15.133:0/1095057200 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7b80059a0 con 0x55cf48537050 2023-11-27T15:20:40.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.679+0000 7ff7e11aeb80 10 client.14628 fetch_fsmap learned FSMap version 17 2023-11-27T15:20:40.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.679+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/1095057200 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x55cf4829ac10 con 0x55cf48537050 2023-11-27T15:20:40.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.679+0000 7ff7aaff5700 1 -- 172.21.15.133:0/1095057200 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7c0099960 con 0x55cf48537050 2023-11-27T15:20:40.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.679+0000 7ff7aaff5700 1 -- 172.21.15.133:0/1095057200 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7ff7b8049070 con 0x55cf48537050 2023-11-27T15:20:40.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.679+0000 7ff7e11aeb80 10 client.14628 fetch_fsmap finished waiting for FSMap version 17 2023-11-27T15:20:40.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.679+0000 7ff7e11aeb80 10 client.14628 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-27T15:20:40.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.679+0000 7ff7e11aeb80 20 client.14628 populate_metadata read hostname 'smithi133' 2023-11-27T15:20:40.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.679+0000 7ff7e11aeb80 1 --2- 172.21.15.133:0/1095057200 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55cf485a4140 0x55cf485a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:40.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.679+0000 7ff7e11aeb80 4 client.14628 mds_command: new command op to 24560 tid=0[{"prefix": "status"}] 2023-11-27T15:20:40.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.679+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/1095057200 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cf48452610 con 0x55cf485a4140 2023-11-27T15:20:40.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.680+0000 7ff7c67fc700 1 --2- 172.21.15.133:0/1095057200 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55cf485a4140 0x55cf485a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:40.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.680+0000 7ff7aaff5700 10 client.14628 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:20:40.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.680+0000 7ff7c67fc700 1 --2- 172.21.15.133:0/1095057200 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55cf485a4140 0x55cf485a0af0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=971fceb975fd14ae server_cookie=97e5ee9b9192f2cd in_seq=0 out_seq=0 2023-11-27T15:20:40.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.680+0000 7ff7aaff5700 1 -- 172.21.15.133:0/1095057200 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55cf48452610 con 0x55cf485a4140 2023-11-27T15:20:40.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.680+0000 7ff7aaff5700 10 client.14628 handle_command_reply: tid=0 2023-11-27T15:20:40.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.680+0000 7ff7e11aeb80 1 client.14628 shutdown 2023-11-27T15:20:40.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.680+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/1095057200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537050 msgr2=0x55cf485479e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.680+0000 7ff7e11aeb80 1 --2- 172.21.15.133:0/1095057200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537050 0x55cf485479e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80177c0 tx=0x7ff7b800fae0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.681+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/1095057200 shutdown_connections 2023-11-27T15:20:40.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.681+0000 7ff7e11aeb80 1 --2- 172.21.15.133:0/1095057200 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55cf485a4140 0x55cf485a0af0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.681+0000 7ff7e11aeb80 1 --2- 172.21.15.133:0/1095057200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55cf48538890 0x55cf485445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.681+0000 7ff7e11aeb80 1 --2- 172.21.15.133:0/1095057200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55cf48537050 0x55cf485479e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.681+0000 7ff7e11aeb80 1 --2- 172.21.15.133:0/1095057200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cf48537ee0 0x55cf48547f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.681+0000 7ff7aaff5700 0 client.14628 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:20:40.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.681+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/1095057200 >> 172.21.15.133:0/1095057200 conn(0x55cf48528310 msgr2=0x55cf4854b070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:40.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.681+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/1095057200 shutdown_connections 2023-11-27T15:20:40.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.681+0000 7ff7e11aeb80 1 -- 172.21.15.133:0/1095057200 wait complete. 2023-11-27T15:20:40.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.681+0000 7ff7e11aeb80 20 client.14628 trim_cache size 0 max 16384 2023-11-27T15:20:40.691 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:20:40.691 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "5227da4c-8d37-11ee-95a2-87774f69a715", 2023-11-27T15:20:40.691 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2023-11-27T15:20:40.691 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24560, 2023-11-27T15:20:40.691 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2023-11-27T15:20:40.692 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2023-11-27T15:20:40.692 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2023-11-27T15:20:40.692 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 7.4772609000000001, 2023-11-27T15:20:40.692 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 17, 2023-11-27T15:20:40.692 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 88, 2023-11-27T15:20:40.692 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 87, 2023-11-27T15:20:40.692 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 14.170328455 2023-11-27T15:20:40.692 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:20:40.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.681+0000 7ff7cd7ff700 1 -- 172.21.15.133:0/39218613 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff7b00a11d0 msgr2=0x7ff7b00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.681+0000 7ff7cd7ff700 1 --2- 172.21.15.133:0/39218613 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff7b00a11d0 0x7ff7b00a3680 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc0119b0 tx=0x7ff7bc009bf0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.681+0000 7ff7cd7ff700 1 -- 172.21.15.133:0/39218613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7c8101db0 msgr2=0x7ff7c819d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:40.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.681+0000 7ff7cd7ff700 1 --2- 172.21.15.133:0/39218613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7c8101db0 0x7ff7c819d590 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff7c000ef30 tx=0x7ff7c000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.681+0000 7ff7cd7ff700 1 -- 172.21.15.133:0/39218613 shutdown_connections 2023-11-27T15:20:40.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.682+0000 7ff7cd7ff700 1 --2- 172.21.15.133:0/39218613 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff7b00a11d0 0x7ff7b00a3680 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.682+0000 7ff7cd7ff700 1 --2- 172.21.15.133:0/39218613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff7c8101db0 0x7ff7c819d590 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.682+0000 7ff7cd7ff700 1 --2- 172.21.15.133:0/39218613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7c8100230 0x7ff7c8195fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.694 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.682+0000 7ff7cd7ff700 1 --2- 172.21.15.133:0/39218613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c8101420 0x7ff7c8196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:40.694 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.682+0000 7ff7cd7ff700 1 -- 172.21.15.133:0/39218613 >> 172.21.15.133:0/39218613 conn(0x7ff7c80fb810 msgr2=0x7ff7c80fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:40.694 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.682+0000 7ff7cd7ff700 1 -- 172.21.15.133:0/39218613 shutdown_connections 2023-11-27T15:20:40.694 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:40.682+0000 7ff7cd7ff700 1 -- 172.21.15.133:0/39218613 wait complete. 2023-11-27T15:20:40.701 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:40 smithi133 ceph-mon[97860]: pgmap v277: 129 pgs: 3 creating+peering, 9 unknown, 117 active+clean; 579 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 1006 B/s wr, 7 op/s 2023-11-27T15:20:40.701 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:40 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/190220272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:40.701 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:40 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/173318620' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:20:40.703 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-27T15:20:40.703 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-27T15:20:40.704 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph fs ls 2023-11-27T15:20:41.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:40 smithi177 ceph-mon[131504]: pgmap v277: 129 pgs: 3 creating+peering, 9 unknown, 117 active+clean; 579 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 1006 B/s wr, 7 op/s 2023-11-27T15:20:41.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:40 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/190220272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:41.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:40 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/173318620' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:20:41.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:40 smithi174 ceph-mon[131110]: pgmap v277: 129 pgs: 3 creating+peering, 9 unknown, 117 active+clean; 579 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 1006 B/s wr, 7 op/s 2023-11-27T15:20:41.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:40 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/190220272' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:20:41.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:40 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/173318620' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:20:41.544 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:20:43.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:42 smithi133 ceph-mon[97860]: pgmap v278: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 4 op/s 2023-11-27T15:20:43.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.030+0000 7f2a8b0c1700 1 -- 172.21.15.133:0/792746324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a841055e0 msgr2=0x7f2a8410d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:43.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.030+0000 7f2a8b0c1700 1 --2- 172.21.15.133:0/792746324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a841055e0 0x7f2a8410d160 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00b0d0 tx=0x7f2a7c00b3e0 comp rx=0 tx=0).stop 2023-11-27T15:20:43.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.030+0000 7f2a8b0c1700 1 -- 172.21.15.133:0/792746324 shutdown_connections 2023-11-27T15:20:43.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.030+0000 7f2a8b0c1700 1 --2- 172.21.15.133:0/792746324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a841055e0 0x7f2a8410d160 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:43.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.030+0000 7f2a8b0c1700 1 --2- 172.21.15.133:0/792746324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a84100540 0x7f2a84100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:43.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.030+0000 7f2a8b0c1700 1 --2- 172.21.15.133:0/792746324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a84100ee0 0x7f2a84104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:43.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.030+0000 7f2a8b0c1700 1 -- 172.21.15.133:0/792746324 >> 172.21.15.133:0/792746324 conn(0x7f2a840fbdc0 msgr2=0x7f2a840fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:43.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.030+0000 7f2a8b0c1700 1 -- 172.21.15.133:0/792746324 shutdown_connections 2023-11-27T15:20:43.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.030+0000 7f2a8b0c1700 1 -- 172.21.15.133:0/792746324 wait complete. 2023-11-27T15:20:43.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.031+0000 7f2a8b0c1700 1 Processor -- start 2023-11-27T15:20:43.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.031+0000 7f2a8b0c1700 1 -- start start 2023-11-27T15:20:43.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.032+0000 7f2a8b0c1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a84100540 0x7f2a8419e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:43.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.032+0000 7f2a8b0c1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a84100ee0 0x7f2a8419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:43.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.032+0000 7f2a8b0c1700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a841055e0 0x7f2a841985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:43.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.032+0000 7f2a8b0c1700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a84198c20 con 0x7f2a841055e0 2023-11-27T15:20:43.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.032+0000 7f2a8b0c1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a84198d90 con 0x7f2a84100540 2023-11-27T15:20:43.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.032+0000 7f2a8b0c1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a84198f50 con 0x7f2a84100ee0 2023-11-27T15:20:43.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.032+0000 7f2a8965e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a841055e0 0x7f2a841985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:43.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.032+0000 7f2a88e5d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a84100540 0x7f2a8419e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:43.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.032+0000 7f2a8965e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a841055e0 0x7f2a841985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32804/0 (socket says 172.21.15.133:32804) 2023-11-27T15:20:43.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.032+0000 7f2a83fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a84100ee0 0x7f2a8419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:43.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.032+0000 7f2a8965e700 1 -- 172.21.15.133:0/2214734832 learned_addr learned my addr 172.21.15.133:0/2214734832 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:43.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.032+0000 7f2a8965e700 1 -- 172.21.15.133:0/2214734832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a84100ee0 msgr2=0x7f2a8419ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:43.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.032+0000 7f2a8965e700 1 --2- 172.21.15.133:0/2214734832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a84100ee0 0x7f2a8419ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:43.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.032+0000 7f2a8965e700 1 -- 172.21.15.133:0/2214734832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a84100540 msgr2=0x7f2a8419e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:43.039 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.032+0000 7f2a8965e700 1 --2- 172.21.15.133:0/2214734832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a84100540 0x7f2a8419e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:43.039 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.032+0000 7f2a8965e700 1 -- 172.21.15.133:0/2214734832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7c009d00 con 0x7f2a841055e0 2023-11-27T15:20:43.039 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.033+0000 7f2a8965e700 1 --2- 172.21.15.133:0/2214734832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a841055e0 0x7f2a841985b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0077f0 tx=0x7f2a7c007820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:43.039 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.033+0000 7f2a81ffb700 1 -- 172.21.15.133:0/2214734832 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c00e040 con 0x7f2a841055e0 2023-11-27T15:20:43.039 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.033+0000 7f2a81ffb700 1 -- 172.21.15.133:0/2214734832 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a7c003e10 con 0x7f2a841055e0 2023-11-27T15:20:43.039 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.033+0000 7f2a8b0c1700 1 -- 172.21.15.133:0/2214734832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a84110370 con 0x7f2a841055e0 2023-11-27T15:20:43.039 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.033+0000 7f2a81ffb700 1 -- 172.21.15.133:0/2214734832 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c01b990 con 0x7f2a841055e0 2023-11-27T15:20:43.039 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.033+0000 7f2a8b0c1700 1 -- 172.21.15.133:0/2214734832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a841101e0 con 0x7f2a841055e0 2023-11-27T15:20:43.040 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.034+0000 7f2a8b0c1700 1 -- 172.21.15.133:0/2214734832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a84110500 con 0x7f2a841055e0 2023-11-27T15:20:43.040 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.036+0000 7f2a81ffb700 1 -- 172.21.15.133:0/2214734832 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2a7c0055c0 con 0x7f2a841055e0 2023-11-27T15:20:43.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.038+0000 7f2a81ffb700 1 --2- 172.21.15.133:0/2214734832 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2a740a11d0 0x7f2a740a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:43.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.039+0000 7f2a81ffb700 1 -- 172.21.15.133:0/2214734832 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2a7c0ce070 con 0x7f2a841055e0 2023-11-27T15:20:43.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.039+0000 7f2a88e5d700 1 --2- 172.21.15.133:0/2214734832 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2a740a11d0 0x7f2a740a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:43.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.039+0000 7f2a88e5d700 1 --2- 172.21.15.133:0/2214734832 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2a740a11d0 0x7f2a740a3680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2a78007be0 tx=0x7f2a780073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:43.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.043+0000 7f2a81ffb700 1 -- 172.21.15.133:0/2214734832 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2a7c100730 con 0x7f2a841055e0 2023-11-27T15:20:43.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:42 smithi177 ceph-mon[131504]: pgmap v278: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 4 op/s 2023-11-27T15:20:43.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:42 smithi174 ceph-mon[131110]: pgmap v278: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s wr, 4 op/s 2023-11-27T15:20:43.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.283+0000 7f2a8b0c1700 1 -- 172.21.15.133:0/2214734832 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2a84066ea0 con 0x7f2a841055e0 2023-11-27T15:20:43.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.283+0000 7f2a81ffb700 1 -- 172.21.15.133:0/2214734832 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f2a7c097600 con 0x7f2a841055e0 2023-11-27T15:20:43.287 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-27T15:20:43.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.285+0000 7f2a8b0c1700 1 -- 172.21.15.133:0/2214734832 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2a740a11d0 msgr2=0x7f2a740a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:43.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.285+0000 7f2a8b0c1700 1 --2- 172.21.15.133:0/2214734832 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2a740a11d0 0x7f2a740a3680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2a78007be0 tx=0x7f2a780073d0 comp rx=0 tx=0).stop 2023-11-27T15:20:43.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.285+0000 7f2a8b0c1700 1 -- 172.21.15.133:0/2214734832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a841055e0 msgr2=0x7f2a841985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:43.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.285+0000 7f2a8b0c1700 1 --2- 172.21.15.133:0/2214734832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a841055e0 0x7f2a841985b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0077f0 tx=0x7f2a7c007820 comp rx=0 tx=0).stop 2023-11-27T15:20:43.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.285+0000 7f2a8b0c1700 1 -- 172.21.15.133:0/2214734832 shutdown_connections 2023-11-27T15:20:43.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.285+0000 7f2a8b0c1700 1 --2- 172.21.15.133:0/2214734832 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2a740a11d0 0x7f2a740a3680 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:43.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.285+0000 7f2a8b0c1700 1 --2- 172.21.15.133:0/2214734832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2a841055e0 0x7f2a841985b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:43.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.285+0000 7f2a8b0c1700 1 --2- 172.21.15.133:0/2214734832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a84100540 0x7f2a8419e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:43.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.285+0000 7f2a8b0c1700 1 --2- 172.21.15.133:0/2214734832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2a84100ee0 0x7f2a8419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:43.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.285+0000 7f2a8b0c1700 1 -- 172.21.15.133:0/2214734832 >> 172.21.15.133:0/2214734832 conn(0x7f2a840fbdc0 msgr2=0x7f2a8410b660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:43.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.286+0000 7f2a8b0c1700 1 -- 172.21.15.133:0/2214734832 shutdown_connections 2023-11-27T15:20:43.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:43.286+0000 7f2a8b0c1700 1 -- 172.21.15.133:0/2214734832 wait complete. 2023-11-27T15:20:43.828 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-27T15:20:43.828 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-27T15:20:43.828 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2023-11-27T15:20:43.829 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-11-27T15:20:43.829 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-27T15:20:43.829 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-27T15:20:43.829 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-27T15:20:43.829 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-27T15:20:43.829 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-11-27T15:20:43.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:20:43.830 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2023-11-27T15:20:43.875 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-27T15:20:43.876 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-27T15:20:43.876 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2023-11-27T15:20:43.876 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-11-27T15:20:43.876 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2023-11-27T15:20:43.876 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-11-27T15:20:43.876 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-27T15:20:43.876 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2023-11-27T15:20:43.877 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2023-11-27T15:20:43.877 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-27T15:20:43.877 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2023-11-27T15:20:43.877 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2023-11-27T15:20:43.877 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-27T15:20:43.877 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2023-11-27T15:20:43.877 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-27T15:20:43.877 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2023-11-27T15:20:43.878 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-27T15:20:43.878 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 41647sec preferred_lft 41647sec 2023-11-27T15:20:43.878 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::89cb:f695:7bf5:3338/64 scope link noprefixroute 2023-11-27T15:20:43.878 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-11-27T15:20:43.879 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-27T15:20:43.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:20:43.879 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-11-27T15:20:43.879 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-11-27T15:20:43.879 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add name ceph-brx type bridge 2023-11-27T15:20:43.880 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr flush dev ceph-brx 2023-11-27T15:20:43.880 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set ceph-brx up 2023-11-27T15:20:43.880 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-27T15:20:43.880 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-11-27T15:20:44.021 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:43 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2214734832' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-27T15:20:44.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:43 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2214734832' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-27T15:20:44.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:43 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2214734832' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-27T15:20:44.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:20:44.345 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-27T15:20:44.417 INFO:teuthology.orchestra.run.smithi133.stdout:1 2023-11-27T15:20:44.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:20:44.421 DEBUG:teuthology.orchestra.run.smithi133:> route 2023-11-27T15:20:44.495 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2023-11-27T15:20:44.496 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-27T15:20:44.496 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-27T15:20:44.496 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-27T15:20:44.496 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-27T15:20:44.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:20:44.498 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-11-27T15:20:44.498 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-11-27T15:20:44.498 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-27T15:20:44.498 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-27T15:20:44.498 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-27T15:20:44.498 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-11-27T15:20:44.871 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:44 smithi133 ceph-mon[97860]: pgmap v279: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 144 B/s wr, 2 op/s 2023-11-27T15:20:45.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:44 smithi177 ceph-mon[131504]: pgmap v279: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 144 B/s wr, 2 op/s 2023-11-27T15:20:45.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:44 smithi174 ceph-mon[131110]: pgmap v279: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 144 B/s wr, 2 op/s 2023-11-27T15:20:45.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:20:45.222 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2023-11-27T15:20:45.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:20:45.267 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list-id 2023-11-27T15:20:45.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:20:45.352 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-11-27T15:20:45.352 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-11-27T15:20:45.352 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-27T15:20:45.352 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-11-27T15:20:45.352 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-11-27T15:20:45.535 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-11-27T15:20:45.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:20:45.536 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-11-27T15:20:45.536 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-11-27T15:20:45.536 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-11-27T15:20:45.536 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-27T15:20:45.536 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-11-27T15:20:45.537 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-11-27T15:20:45.537 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-11-27T15:20:45.537 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-11-27T15:20:45.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:20:45.816 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-11-27T15:20:45.816 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-11-27T15:20:45.816 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2023-11-27T15:20:45.817 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set dev brx.0 master ceph-brx 2023-11-27T15:20:45.817 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-11-27T15:20:45.990 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-11-27T15:20:45.990 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-27T15:20:45.990 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-11-27T15:20:46.034 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-11-27T15:20:46.035 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-27T15:20:46.035 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-11-27T15:20:46.117 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2023-11-27T15:20:46.227 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-27T15:20:46.227 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-27T15:20:46.295 INFO:teuthology.orchestra.run.smithi133.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-27T15:20:46.298 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-27T15:20:46.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:20:46.299 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2023-11-27T15:20:46.342 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-27T15:20:46.343 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-11-27T15:20:46.387 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2023-11-27T15:20:46.459 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-27T15:20:46.459 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-27T15:20:46.527 INFO:teuthology.orchestra.run.smithi133.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-27T15:20:46.531 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-27T15:20:46.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:20:46.532 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2023-11-27T15:20:46.584 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi133.stderr:2023-11-27T15:20:46.580+0000 7fc9110d35c0 -1 init, newargv = 0x559790a7a4e0 newargc=15 2023-11-27T15:20:46.585 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi133.stderr:ceph-fuse[134938]: starting ceph client 2023-11-27T15:20:46.594 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi133.stderr:ceph-fuse[134938]: starting fuse 2023-11-27T15:20:47.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:46 smithi133 ceph-mon[97860]: pgmap v280: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2023-11-27T15:20:47.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:46 smithi177 ceph-mon[131504]: pgmap v280: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2023-11-27T15:20:47.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:46 smithi174 ceph-mon[131110]: pgmap v280: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 2 op/s 2023-11-27T15:20:47.677 DEBUG:teuthology.orchestra.run.smithi133:> sudo modprobe fuse 2023-11-27T15:20:47.755 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-27T15:20:47.755 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-27T15:20:47.825 INFO:teuthology.orchestra.run.smithi133.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-27T15:20:47.827 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-27T15:20:47.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:20:47.828 DEBUG:teuthology.orchestra.run.smithi133:> ls /sys/fs/fuse/connections 2023-11-27T15:20:47.873 INFO:teuthology.orchestra.run.smithi133.stdout:121 2023-11-27T15:20:47.873 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-11-27T15:20:47.873 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-27T15:20:47.873 DEBUG:teuthology.orchestra.run.smithi133:> import glob 2023-11-27T15:20:47.874 DEBUG:teuthology.orchestra.run.smithi133:> import re 2023-11-27T15:20:47.874 DEBUG:teuthology.orchestra.run.smithi133:> import os 2023-11-27T15:20:47.874 DEBUG:teuthology.orchestra.run.smithi133:> import subprocess 2023-11-27T15:20:47.874 DEBUG:teuthology.orchestra.run.smithi133:> 2023-11-27T15:20:47.874 DEBUG:teuthology.orchestra.run.smithi133:> def _find_admin_socket(client_name): 2023-11-27T15:20:47.874 DEBUG:teuthology.orchestra.run.smithi133:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-11-27T15:20:47.874 DEBUG:teuthology.orchestra.run.smithi133:> files = glob.glob(asok_path) 2023-11-27T15:20:47.874 DEBUG:teuthology.orchestra.run.smithi133:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-11-27T15:20:47.875 DEBUG:teuthology.orchestra.run.smithi133:> 2023-11-27T15:20:47.875 DEBUG:teuthology.orchestra.run.smithi133:> # Given a non-glob path, it better be there 2023-11-27T15:20:47.875 DEBUG:teuthology.orchestra.run.smithi133:> if "*" not in asok_path: 2023-11-27T15:20:47.875 DEBUG:teuthology.orchestra.run.smithi133:> assert(len(files) == 1) 2023-11-27T15:20:47.875 DEBUG:teuthology.orchestra.run.smithi133:> return files[0] 2023-11-27T15:20:47.875 DEBUG:teuthology.orchestra.run.smithi133:> 2023-11-27T15:20:47.875 DEBUG:teuthology.orchestra.run.smithi133:> for f in files: 2023-11-27T15:20:47.875 DEBUG:teuthology.orchestra.run.smithi133:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-27T15:20:47.875 DEBUG:teuthology.orchestra.run.smithi133:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-27T15:20:47.875 DEBUG:teuthology.orchestra.run.smithi133:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-27T15:20:47.875 DEBUG:teuthology.orchestra.run.smithi133:> contents = proc_f.read() 2023-11-27T15:20:47.876 DEBUG:teuthology.orchestra.run.smithi133:> if mountpoint in contents: 2023-11-27T15:20:47.876 DEBUG:teuthology.orchestra.run.smithi133:> return f 2023-11-27T15:20:47.876 DEBUG:teuthology.orchestra.run.smithi133:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-27T15:20:47.876 DEBUG:teuthology.orchestra.run.smithi133:> 2023-11-27T15:20:47.876 DEBUG:teuthology.orchestra.run.smithi133:> print(_find_admin_socket("client.admin")) 2023-11-27T15:20:47.876 DEBUG:teuthology.orchestra.run.smithi133:> ' 2023-11-27T15:20:48.057 INFO:teuthology.orchestra.run.smithi133.stdout:/var/run/ceph/ceph-client.admin.134938.asok 2023-11-27T15:20:48.235 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.134938.asok 2023-11-27T15:20:48.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:20:48.235 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.134938.asok status 2023-11-27T15:20:48.461 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:20:48.461 INFO:teuthology.orchestra.run.smithi133.stdout: "metadata": { 2023-11-27T15:20:48.461 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_sha1": "1c9565146c8647d4e00eb6c7ae626dc192680e7b", 2023-11-27T15:20:48.462 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_version": "ceph version 18.0.0-3906-g1c956514 (1c9565146c8647d4e00eb6c7ae626dc192680e7b) reef (dev)", 2023-11-27T15:20:48.462 INFO:teuthology.orchestra.run.smithi133.stdout: "entity_id": "admin", 2023-11-27T15:20:48.462 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": "smithi133", 2023-11-27T15:20:48.462 INFO:teuthology.orchestra.run.smithi133.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-11-27T15:20:48.463 INFO:teuthology.orchestra.run.smithi133.stdout: "pid": "134938", 2023-11-27T15:20:48.463 INFO:teuthology.orchestra.run.smithi133.stdout: "root": "/" 2023-11-27T15:20:48.463 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:20:48.463 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_count": 0, 2023-11-27T15:20:48.463 INFO:teuthology.orchestra.run.smithi133.stdout: "dentry_pinned_count": 0, 2023-11-27T15:20:48.463 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 14640, 2023-11-27T15:20:48.463 INFO:teuthology.orchestra.run.smithi133.stdout: "inst": { 2023-11-27T15:20:48.463 INFO:teuthology.orchestra.run.smithi133.stdout: "name": { 2023-11-27T15:20:48.463 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "client", 2023-11-27T15:20:48.463 INFO:teuthology.orchestra.run.smithi133.stdout: "num": 14640 2023-11-27T15:20:48.464 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:20:48.464 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2023-11-27T15:20:48.464 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2023-11-27T15:20:48.464 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2023-11-27T15:20:48.464 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2196472565 2023-11-27T15:20:48.464 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:20:48.464 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:20:48.464 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": { 2023-11-27T15:20:48.464 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2023-11-27T15:20:48.464 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "192.168.0.1:0", 2023-11-27T15:20:48.464 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2196472565 2023-11-27T15:20:48.465 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:20:48.465 INFO:teuthology.orchestra.run.smithi133.stdout: "inst_str": "client.14640 192.168.0.1:0/2196472565", 2023-11-27T15:20:48.465 INFO:teuthology.orchestra.run.smithi133.stdout: "addr_str": "192.168.0.1:0/2196472565", 2023-11-27T15:20:48.465 INFO:teuthology.orchestra.run.smithi133.stdout: "inode_count": 1, 2023-11-27T15:20:48.465 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_epoch": 17, 2023-11-27T15:20:48.465 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch": 88, 2023-11-27T15:20:48.465 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_epoch_barrier": 0, 2023-11-27T15:20:48.465 INFO:teuthology.orchestra.run.smithi133.stdout: "blocklisted": false, 2023-11-27T15:20:48.465 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs" 2023-11-27T15:20:48.466 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:20:48.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:20:48.476 DEBUG:teuthology.orchestra.run.smithi133:> stat --file-system '--printf=%T 2023-11-27T15:20:48.477 DEBUG:teuthology.orchestra.run.smithi133:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-11-27T15:20:48.521 INFO:teuthology.orchestra.run.smithi133.stdout:fuseblk 2023-11-27T15:20:48.521 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-11-27T15:20:48.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:20:48.522 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-11-27T15:20:48.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:20:48.633 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-11-27T15:20:48.761 INFO:teuthology.orchestra.run.smithi133.stdout:# file: . 2023-11-27T15:20:48.761 INFO:teuthology.orchestra.run.smithi133.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-11-27T15:20:48.761 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:48.763 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-11-27T15:20:48.763 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-27T15:20:48.808 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:20:48.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:48 smithi174 ceph-mon[131110]: pgmap v281: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 2 op/s 2023-11-27T15:20:48.907 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:48 smithi133 ceph-mon[97860]: pgmap v281: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 2 op/s 2023-11-27T15:20:49.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.041+0000 7f85f0bff700 1 -- 172.21.15.133:0/3493072329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85ec074b30 msgr2=0x7f85ec072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:49.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.041+0000 7f85f0bff700 1 --2- 172.21.15.133:0/3493072329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85ec074b30 0x7f85ec072f80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009990 tx=0x7f85dc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.041+0000 7f85f0bff700 1 -- 172.21.15.133:0/3493072329 shutdown_connections 2023-11-27T15:20:49.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.041+0000 7f85f0bff700 1 --2- 172.21.15.133:0/3493072329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85ec10c860 0x7f85ec10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.041+0000 7f85f0bff700 1 --2- 172.21.15.133:0/3493072329 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85ec074b30 0x7f85ec072f80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.041+0000 7f85f0bff700 1 --2- 172.21.15.133:0/3493072329 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85ec073550 0x7f85ec0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.041+0000 7f85f0bff700 1 -- 172.21.15.133:0/3493072329 >> 172.21.15.133:0/3493072329 conn(0x7f85ec0fb870 msgr2=0x7f85ec0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:49.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.041+0000 7f85f0bff700 1 -- 172.21.15.133:0/3493072329 shutdown_connections 2023-11-27T15:20:49.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.041+0000 7f85f0bff700 1 -- 172.21.15.133:0/3493072329 wait complete. 2023-11-27T15:20:49.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.042+0000 7f85f0bff700 1 Processor -- start 2023-11-27T15:20:49.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.042+0000 7f85f0bff700 1 -- start start 2023-11-27T15:20:49.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.043+0000 7f85f0bff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85ec073550 0x7f85ec19c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:49.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.043+0000 7f85f0bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85ec074b30 0x7f85ec19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:49.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.043+0000 7f85f0bff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85ec10c860 0x7f85ec1a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:49.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.043+0000 7f85f0bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec1a4120 con 0x7f85ec074b30 2023-11-27T15:20:49.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.043+0000 7f85f0bff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec1a4260 con 0x7f85ec10c860 2023-11-27T15:20:49.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.043+0000 7f85f0bff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec1a43f0 con 0x7f85ec073550 2023-11-27T15:20:49.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.043+0000 7f85e9d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85ec074b30 0x7f85ec19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:49.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.043+0000 7f85ea59c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85ec073550 0x7f85ec19c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:49.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.043+0000 7f85e9d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85ec074b30 0x7f85ec19cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32818/0 (socket says 172.21.15.133:32818) 2023-11-27T15:20:49.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.043+0000 7f85ea59c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85ec073550 0x7f85ec19c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:36188/0 (socket says 172.21.15.133:36188) 2023-11-27T15:20:49.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.043+0000 7f85e9d9b700 1 -- 172.21.15.133:0/3052813317 learned_addr learned my addr 172.21.15.133:0/3052813317 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:49.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.043+0000 7f85ead9d700 1 --2- 172.21.15.133:0/3052813317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85ec10c860 0x7f85ec1a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:49.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.043+0000 7f85e9d9b700 1 -- 172.21.15.133:0/3052813317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85ec073550 msgr2=0x7f85ec19c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:49.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.043+0000 7f85e9d9b700 1 --2- 172.21.15.133:0/3052813317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85ec073550 0x7f85ec19c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.043+0000 7f85e9d9b700 1 -- 172.21.15.133:0/3052813317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85ec10c860 msgr2=0x7f85ec1a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:49.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.043+0000 7f85e9d9b700 1 --2- 172.21.15.133:0/3052813317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85ec10c860 0x7f85ec1a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.043+0000 7f85e9d9b700 1 -- 172.21.15.133:0/3052813317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc0095f0 con 0x7f85ec074b30 2023-11-27T15:20:49.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.043+0000 7f85ea59c700 1 --2- 172.21.15.133:0/3052813317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85ec073550 0x7f85ec19c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:49.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.044+0000 7f85e9d9b700 1 --2- 172.21.15.133:0/3052813317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85ec074b30 0x7f85ec19cae0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f85e000e970 tx=0x7f85e000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:49.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.044+0000 7f85db7fe700 1 -- 172.21.15.133:0/3052813317 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0009790 con 0x7f85ec074b30 2023-11-27T15:20:49.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.044+0000 7f85db7fe700 1 -- 172.21.15.133:0/3052813317 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85e000c7e0 con 0x7f85ec074b30 2023-11-27T15:20:49.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.044+0000 7f85f0bff700 1 -- 172.21.15.133:0/3052813317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ec112200 con 0x7f85ec074b30 2023-11-27T15:20:49.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.044+0000 7f85db7fe700 1 -- 172.21.15.133:0/3052813317 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0003c70 con 0x7f85ec074b30 2023-11-27T15:20:49.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.044+0000 7f85f0bff700 1 -- 172.21.15.133:0/3052813317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ec112070 con 0x7f85ec074b30 2023-11-27T15:20:49.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.045+0000 7f85f0bff700 1 -- 172.21.15.133:0/3052813317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85ec1023b0 con 0x7f85ec074b30 2023-11-27T15:20:49.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.047+0000 7f85db7fe700 1 -- 172.21.15.133:0/3052813317 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f85e000cc50 con 0x7f85ec074b30 2023-11-27T15:20:49.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.049+0000 7f85db7fe700 1 --2- 172.21.15.133:0/3052813317 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f85d40a11d0 0x7f85d40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:49.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.049+0000 7f85db7fe700 1 -- 172.21.15.133:0/3052813317 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f85e0014070 con 0x7f85ec074b30 2023-11-27T15:20:49.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.049+0000 7f85ea59c700 1 --2- 172.21.15.133:0/3052813317 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f85d40a11d0 0x7f85d40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:49.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.050+0000 7f85ea59c700 1 --2- 172.21.15.133:0/3052813317 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f85d40a11d0 0x7f85d40a3680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00b2e0 tx=0x7f85dc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:49.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.052+0000 7f85db7fe700 1 -- 172.21.15.133:0/3052813317 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f85e00962c0 con 0x7f85ec074b30 2023-11-27T15:20:49.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:48 smithi177 ceph-mon[131504]: pgmap v281: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 102 B/s wr, 2 op/s 2023-11-27T15:20:49.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.260+0000 7f85f0bff700 1 -- 172.21.15.133:0/3052813317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85ec02cfe0 con 0x7f85ec074b30 2023-11-27T15:20:49.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.261+0000 7f85db7fe700 1 -- 172.21.15.133:0/3052813317 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23996 (secure 0 0 0) 0x7f85e00998e0 con 0x7f85ec074b30 2023-11-27T15:20:49.266 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:49.266 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:20:38.147036+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:20:49.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.265+0000 7f85f0bff700 1 -- 172.21.15.133:0/3052813317 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f85d40a11d0 msgr2=0x7f85d40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:49.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.265+0000 7f85f0bff700 1 --2- 172.21.15.133:0/3052813317 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f85d40a11d0 0x7f85d40a3680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00b2e0 tx=0x7f85dc005fb0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.265+0000 7f85f0bff700 1 -- 172.21.15.133:0/3052813317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85ec074b30 msgr2=0x7f85ec19cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:49.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.265+0000 7f85f0bff700 1 --2- 172.21.15.133:0/3052813317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85ec074b30 0x7f85ec19cae0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f85e000e970 tx=0x7f85e000ec80 comp rx=0 tx=0).stop 2023-11-27T15:20:49.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.265+0000 7f85f0bff700 1 -- 172.21.15.133:0/3052813317 shutdown_connections 2023-11-27T15:20:49.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.265+0000 7f85f0bff700 1 --2- 172.21.15.133:0/3052813317 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f85d40a11d0 0x7f85d40a3680 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.265+0000 7f85f0bff700 1 --2- 172.21.15.133:0/3052813317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85ec10c860 0x7f85ec1a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.265+0000 7f85f0bff700 1 --2- 172.21.15.133:0/3052813317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f85ec073550 0x7f85ec19c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.265+0000 7f85f0bff700 1 --2- 172.21.15.133:0/3052813317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85ec074b30 0x7f85ec19cae0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.265+0000 7f85f0bff700 1 -- 172.21.15.133:0/3052813317 >> 172.21.15.133:0/3052813317 conn(0x7f85ec0fb870 msgr2=0x7f85ec0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:49.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.265+0000 7f85f0bff700 1 -- 172.21.15.133:0/3052813317 shutdown_connections 2023-11-27T15:20:49.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.265+0000 7f85f0bff700 1 -- 172.21.15.133:0/3052813317 wait complete. 2023-11-27T15:20:49.288 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:16:20.216150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:20:31.107146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:20:32.614089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-27T15:20:36.126040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-27T15:20:49.288 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:20:49.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.485+0000 7f240d9ff700 1 -- 172.21.15.133:0/2916097034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24081014b0 msgr2=0x7f2408101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:49.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.485+0000 7f240d9ff700 1 --2- 172.21.15.133:0/2916097034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24081014b0 0x7f2408101900 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f23fc009990 tx=0x7f23fc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.485+0000 7f240d9ff700 1 -- 172.21.15.133:0/2916097034 shutdown_connections 2023-11-27T15:20:49.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.485+0000 7f240d9ff700 1 --2- 172.21.15.133:0/2916097034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2408101e40 0x7f2408108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.485+0000 7f240d9ff700 1 --2- 172.21.15.133:0/2916097034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24081002c0 0x7f24081006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.485+0000 7f240d9ff700 1 --2- 172.21.15.133:0/2916097034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f24081014b0 0x7f2408101900 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.485+0000 7f240d9ff700 1 -- 172.21.15.133:0/2916097034 >> 172.21.15.133:0/2916097034 conn(0x7f24080fb8a0 msgr2=0x7f24080fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:49.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.485+0000 7f240d9ff700 1 -- 172.21.15.133:0/2916097034 shutdown_connections 2023-11-27T15:20:49.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.485+0000 7f240d9ff700 1 -- 172.21.15.133:0/2916097034 wait complete. 2023-11-27T15:20:49.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.486+0000 7f240d9ff700 1 Processor -- start 2023-11-27T15:20:49.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.486+0000 7f240d9ff700 1 -- start start 2023-11-27T15:20:49.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.487+0000 7f240d9ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24081002c0 0x7f24080748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:49.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.487+0000 7f240d9ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24081014b0 0x7f2408072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:49.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.487+0000 7f240d9ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2408101e40 0x7f2408073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:49.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.487+0000 7f240d9ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2408073bc0 con 0x7f2408101e40 2023-11-27T15:20:49.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.487+0000 7f240d9ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2408073d30 con 0x7f24081002c0 2023-11-27T15:20:49.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.487+0000 7f240d9ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24081a0120 con 0x7f24081014b0 2023-11-27T15:20:49.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.487+0000 7f24077fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2408101e40 0x7f2408073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:49.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.487+0000 7f24077fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2408101e40 0x7f2408073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32840/0 (socket says 172.21.15.133:32840) 2023-11-27T15:20:49.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.487+0000 7f24077fe700 1 -- 172.21.15.133:0/4091234408 learned_addr learned my addr 172.21.15.133:0/4091234408 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:49.493 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.487+0000 7f2406ffd700 1 --2- 172.21.15.133:0/4091234408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24081002c0 0x7f24080748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:49.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.487+0000 7f24067fc700 1 --2- 172.21.15.133:0/4091234408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24081014b0 0x7f2408072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:49.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.487+0000 7f24077fe700 1 -- 172.21.15.133:0/4091234408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24081014b0 msgr2=0x7f2408072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:49.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.487+0000 7f24077fe700 1 --2- 172.21.15.133:0/4091234408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24081014b0 0x7f2408072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.487+0000 7f24077fe700 1 -- 172.21.15.133:0/4091234408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24081002c0 msgr2=0x7f24080748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:49.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.487+0000 7f24077fe700 1 --2- 172.21.15.133:0/4091234408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24081002c0 0x7f24080748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.487+0000 7f24077fe700 1 -- 172.21.15.133:0/4091234408 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23fc0095f0 con 0x7f2408101e40 2023-11-27T15:20:49.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.488+0000 7f24077fe700 1 --2- 172.21.15.133:0/4091234408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2408101e40 0x7f2408073420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f240000ef30 tx=0x7f240000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:49.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.488+0000 7f240c9fd700 1 -- 172.21.15.133:0/4091234408 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2400018070 con 0x7f2408101e40 2023-11-27T15:20:49.494 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.488+0000 7f240c9fd700 1 -- 172.21.15.133:0/4091234408 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f240000f040 con 0x7f2408101e40 2023-11-27T15:20:49.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.488+0000 7f240d9ff700 1 -- 172.21.15.133:0/4091234408 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f240810bc70 con 0x7f2408101e40 2023-11-27T15:20:49.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.488+0000 7f240c9fd700 1 -- 172.21.15.133:0/4091234408 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240000cd00 con 0x7f2408101e40 2023-11-27T15:20:49.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.488+0000 7f240d9ff700 1 -- 172.21.15.133:0/4091234408 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240810bae0 con 0x7f2408101e40 2023-11-27T15:20:49.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.489+0000 7f240d9ff700 1 -- 172.21.15.133:0/4091234408 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2408107860 con 0x7f2408101e40 2023-11-27T15:20:49.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.491+0000 7f240c9fd700 1 -- 172.21.15.133:0/4091234408 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2400003a00 con 0x7f2408101e40 2023-11-27T15:20:49.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.492+0000 7f240c9fd700 1 --2- 172.21.15.133:0/4091234408 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f23f00a11d0 0x7f23f00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:49.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.492+0000 7f240c9fd700 1 -- 172.21.15.133:0/4091234408 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f24000cc570 con 0x7f2408101e40 2023-11-27T15:20:49.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.492+0000 7f2406ffd700 1 --2- 172.21.15.133:0/4091234408 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f23f00a11d0 0x7f23f00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:49.497 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.493+0000 7f2406ffd700 1 --2- 172.21.15.133:0/4091234408 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f23f00a11d0 0x7f23f00a3680 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2408101310 tx=0x7f23f8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:49.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.496+0000 7f240c9fd700 1 -- 172.21.15.133:0/4091234408 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2400095b00 con 0x7f2408101e40 2023-11-27T15:20:49.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.699+0000 7f240d9ff700 1 -- 172.21.15.133:0/4091234408 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2408061700 con 0x7f2408101e40 2023-11-27T15:20:49.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.699+0000 7f240c9fd700 1 -- 172.21.15.133:0/4091234408 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2400099120 con 0x7f2408101e40 2023-11-27T15:20:49.703 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-11-27T15:20:49.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.701+0000 7f240d9ff700 1 -- 172.21.15.133:0/4091234408 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f23f00a11d0 msgr2=0x7f23f00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:49.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.701+0000 7f240d9ff700 1 --2- 172.21.15.133:0/4091234408 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f23f00a11d0 0x7f23f00a3680 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2408101310 tx=0x7f23f8009260 comp rx=0 tx=0).stop 2023-11-27T15:20:49.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.701+0000 7f240d9ff700 1 -- 172.21.15.133:0/4091234408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2408101e40 msgr2=0x7f2408073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:49.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.701+0000 7f240d9ff700 1 --2- 172.21.15.133:0/4091234408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2408101e40 0x7f2408073420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f240000ef30 tx=0x7f240000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.702+0000 7f240d9ff700 1 -- 172.21.15.133:0/4091234408 shutdown_connections 2023-11-27T15:20:49.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.702+0000 7f240d9ff700 1 --2- 172.21.15.133:0/4091234408 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f23f00a11d0 0x7f23f00a3680 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.702+0000 7f240d9ff700 1 --2- 172.21.15.133:0/4091234408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2408101e40 0x7f2408073420 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.702+0000 7f240d9ff700 1 --2- 172.21.15.133:0/4091234408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24081002c0 0x7f24080748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.702+0000 7f240d9ff700 1 --2- 172.21.15.133:0/4091234408 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24081014b0 0x7f2408072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.702+0000 7f240d9ff700 1 -- 172.21.15.133:0/4091234408 >> 172.21.15.133:0/4091234408 conn(0x7f24080fb8a0 msgr2=0x7f24081049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:49.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.702+0000 7f240d9ff700 1 -- 172.21.15.133:0/4091234408 shutdown_connections 2023-11-27T15:20:49.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.702+0000 7f240d9ff700 1 -- 172.21.15.133:0/4091234408 wait complete. 2023-11-27T15:20:49.724 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:20:49.902 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:49 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3052813317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:20:49.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.921+0000 7f8324bff700 1 -- 172.21.15.133:0/3908905613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8320101e50 msgr2=0x7f8320108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:49.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.921+0000 7f8324bff700 1 --2- 172.21.15.133:0/3908905613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8320101e50 0x7f8320108710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f831800b0a0 tx=0x7f831800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.921+0000 7f8324bff700 1 -- 172.21.15.133:0/3908905613 shutdown_connections 2023-11-27T15:20:49.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.921+0000 7f8324bff700 1 --2- 172.21.15.133:0/3908905613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8320101e50 0x7f8320108710 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.921+0000 7f8324bff700 1 --2- 172.21.15.133:0/3908905613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83201002d0 0x7f83201006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.921+0000 7f8324bff700 1 --2- 172.21.15.133:0/3908905613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83201014c0 0x7f8320101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.921+0000 7f8324bff700 1 -- 172.21.15.133:0/3908905613 >> 172.21.15.133:0/3908905613 conn(0x7f83200fb8b0 msgr2=0x7f83200fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:49.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.921+0000 7f8324bff700 1 -- 172.21.15.133:0/3908905613 shutdown_connections 2023-11-27T15:20:49.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.921+0000 7f8324bff700 1 -- 172.21.15.133:0/3908905613 wait complete. 2023-11-27T15:20:49.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.922+0000 7f8324bff700 1 Processor -- start 2023-11-27T15:20:49.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.922+0000 7f8324bff700 1 -- start start 2023-11-27T15:20:49.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.922+0000 7f8324bff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83201002d0 0x7f8320198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:49.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.922+0000 7f8324bff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83201014c0 0x7f83201988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:49.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.922+0000 7f8324bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8320101e50 0x7f832019f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:49.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.922+0000 7f8324bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832019fee0 con 0x7f8320101e50 2023-11-27T15:20:49.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.922+0000 7f8324bff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83201a0020 con 0x7f83201002d0 2023-11-27T15:20:49.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.922+0000 7f8324bff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83201a01b0 con 0x7f83201014c0 2023-11-27T15:20:49.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.923+0000 7f831ed9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8320101e50 0x7f832019f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:49.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.923+0000 7f831e59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83201002d0 0x7f8320198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:49.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.923+0000 7f831ed9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8320101e50 0x7f832019f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32860/0 (socket says 172.21.15.133:32860) 2023-11-27T15:20:49.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.923+0000 7f831e59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83201002d0 0x7f8320198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:33782/0 (socket says 172.21.15.133:33782) 2023-11-27T15:20:49.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.923+0000 7f831ed9d700 1 -- 172.21.15.133:0/1755997889 learned_addr learned my addr 172.21.15.133:0/1755997889 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:49.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.923+0000 7f8317fff700 1 --2- 172.21.15.133:0/1755997889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83201014c0 0x7f83201988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:49.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.923+0000 7f831ed9d700 1 -- 172.21.15.133:0/1755997889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83201014c0 msgr2=0x7f83201988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:49.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.923+0000 7f831ed9d700 1 --2- 172.21.15.133:0/1755997889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83201014c0 0x7f83201988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.923+0000 7f831ed9d700 1 -- 172.21.15.133:0/1755997889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83201002d0 msgr2=0x7f8320198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:49.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.923+0000 7f831ed9d700 1 --2- 172.21.15.133:0/1755997889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83201002d0 0x7f8320198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:49.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.923+0000 7f831ed9d700 1 -- 172.21.15.133:0/1755997889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8308009520 con 0x7f8320101e50 2023-11-27T15:20:49.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.923+0000 7f831e59c700 1 --2- 172.21.15.133:0/1755997889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83201002d0 0x7f8320198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:49.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.923+0000 7f831ed9d700 1 --2- 172.21.15.133:0/1755997889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8320101e50 0x7f832019f910 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f831800d010 tx=0x7f831800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:49.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.923+0000 7f83177fe700 1 -- 172.21.15.133:0/1755997889 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831800e040 con 0x7f8320101e50 2023-11-27T15:20:49.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.923+0000 7f83177fe700 1 -- 172.21.15.133:0/1755997889 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8318003e10 con 0x7f8320101e50 2023-11-27T15:20:49.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.923+0000 7f8324bff700 1 -- 172.21.15.133:0/1755997889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8318009d00 con 0x7f8320101e50 2023-11-27T15:20:49.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.923+0000 7f83177fe700 1 -- 172.21.15.133:0/1755997889 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83180052a0 con 0x7f8320101e50 2023-11-27T15:20:49.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.924+0000 7f8324bff700 1 -- 172.21.15.133:0/1755997889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832010bca0 con 0x7f8320101e50 2023-11-27T15:20:49.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.925+0000 7f8324bff700 1 -- 172.21.15.133:0/1755997889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8320067b00 con 0x7f8320101e50 2023-11-27T15:20:49.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.926+0000 7f83177fe700 1 -- 172.21.15.133:0/1755997889 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f831801d480 con 0x7f8320101e50 2023-11-27T15:20:49.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.929+0000 7f83177fe700 1 --2- 172.21.15.133:0/1755997889 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f830c0a11d0 0x7f830c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:49.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.929+0000 7f83177fe700 1 -- 172.21.15.133:0/1755997889 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f83180cde30 con 0x7f8320101e50 2023-11-27T15:20:49.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.929+0000 7f831e59c700 1 --2- 172.21.15.133:0/1755997889 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f830c0a11d0 0x7f830c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:49.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.929+0000 7f831e59c700 1 --2- 172.21.15.133:0/1755997889 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f830c0a11d0 0x7f830c0a3680 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8320101320 tx=0x7f8310009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:49.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:49.932+0000 7f83177fe700 1 -- 172.21.15.133:0/1755997889 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f83180973c0 con 0x7f8320101e50 2023-11-27T15:20:50.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:49 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3052813317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:20:50.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:49 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3052813317' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:20:50.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.133+0000 7f8324bff700 1 -- 172.21.15.133:0/1755997889 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f83201a0c60 con 0x7f8320101e50 2023-11-27T15:20:50.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.133+0000 7f83177fe700 1 -- 172.21.15.133:0/1755997889 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f831809a9e0 con 0x7f8320101e50 2023-11-27T15:20:50.137 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:20:50.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.136+0000 7f8324bff700 1 -- 172.21.15.133:0/1755997889 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f830c0a11d0 msgr2=0x7f830c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:50.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.136+0000 7f8324bff700 1 --2- 172.21.15.133:0/1755997889 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f830c0a11d0 0x7f830c0a3680 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8320101320 tx=0x7f8310009260 comp rx=0 tx=0).stop 2023-11-27T15:20:50.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.136+0000 7f8324bff700 1 -- 172.21.15.133:0/1755997889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8320101e50 msgr2=0x7f832019f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:50.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.136+0000 7f8324bff700 1 --2- 172.21.15.133:0/1755997889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8320101e50 0x7f832019f910 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f831800d010 tx=0x7f831800bc80 comp rx=0 tx=0).stop 2023-11-27T15:20:50.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.136+0000 7f8324bff700 1 -- 172.21.15.133:0/1755997889 shutdown_connections 2023-11-27T15:20:50.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.136+0000 7f8324bff700 1 --2- 172.21.15.133:0/1755997889 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f830c0a11d0 0x7f830c0a3680 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.136+0000 7f8324bff700 1 --2- 172.21.15.133:0/1755997889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8320101e50 0x7f832019f910 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.136+0000 7f8324bff700 1 --2- 172.21.15.133:0/1755997889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83201002d0 0x7f8320198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.136+0000 7f8324bff700 1 --2- 172.21.15.133:0/1755997889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f83201014c0 0x7f83201988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.136+0000 7f8324bff700 1 -- 172.21.15.133:0/1755997889 >> 172.21.15.133:0/1755997889 conn(0x7f83200fb8b0 msgr2=0x7f83200fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:50.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.136+0000 7f8324bff700 1 -- 172.21.15.133:0/1755997889 shutdown_connections 2023-11-27T15:20:50.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.136+0000 7f8324bff700 1 -- 172.21.15.133:0/1755997889 wait complete. 2023-11-27T15:20:50.158 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:20:50.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.353+0000 7f7b6a7ff700 1 -- 172.21.15.133:0/322331166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b641014b0 msgr2=0x7f7b64101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:50.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.353+0000 7f7b6a7ff700 1 --2- 172.21.15.133:0/322331166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b641014b0 0x7f7b64101900 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7b58009990 tx=0x7f7b58009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.353+0000 7f7b6a7ff700 1 -- 172.21.15.133:0/322331166 shutdown_connections 2023-11-27T15:20:50.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.353+0000 7f7b6a7ff700 1 --2- 172.21.15.133:0/322331166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b64101e40 0x7f7b64108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.353+0000 7f7b6a7ff700 1 --2- 172.21.15.133:0/322331166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b641002c0 0x7f7b641006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.353+0000 7f7b6a7ff700 1 --2- 172.21.15.133:0/322331166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b641014b0 0x7f7b64101900 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.353+0000 7f7b6a7ff700 1 -- 172.21.15.133:0/322331166 >> 172.21.15.133:0/322331166 conn(0x7f7b640fb8a0 msgr2=0x7f7b640fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:50.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.353+0000 7f7b6a7ff700 1 -- 172.21.15.133:0/322331166 shutdown_connections 2023-11-27T15:20:50.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.353+0000 7f7b6a7ff700 1 -- 172.21.15.133:0/322331166 wait complete. 2023-11-27T15:20:50.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.354+0000 7f7b6a7ff700 1 Processor -- start 2023-11-27T15:20:50.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.354+0000 7f7b6a7ff700 1 -- start start 2023-11-27T15:20:50.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.354+0000 7f7b6a7ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b641002c0 0x7f7b640748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:50.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.355+0000 7f7b6a7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b641014b0 0x7f7b64072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:50.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.355+0000 7f7b6a7ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b64101e40 0x7f7b64073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:50.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.355+0000 7f7b6a7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b64073bc0 con 0x7f7b641014b0 2023-11-27T15:20:50.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.355+0000 7f7b6a7ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b64073d30 con 0x7f7b641002c0 2023-11-27T15:20:50.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.355+0000 7f7b6a7ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b641a0120 con 0x7f7b64101e40 2023-11-27T15:20:50.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.355+0000 7f7b637fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b641014b0 0x7f7b64072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:50.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.355+0000 7f7b63fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b641002c0 0x7f7b640748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:50.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.355+0000 7f7b63fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b641002c0 0x7f7b640748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:33798/0 (socket says 172.21.15.133:33798) 2023-11-27T15:20:50.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.355+0000 7f7b63fff700 1 -- 172.21.15.133:0/1537963630 learned_addr learned my addr 172.21.15.133:0/1537963630 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:50.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.355+0000 7f7b68d9c700 1 --2- 172.21.15.133:0/1537963630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b64101e40 0x7f7b64073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:50.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.355+0000 7f7b63fff700 1 -- 172.21.15.133:0/1537963630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b64101e40 msgr2=0x7f7b64073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:50.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.355+0000 7f7b63fff700 1 --2- 172.21.15.133:0/1537963630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b64101e40 0x7f7b64073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.355+0000 7f7b63fff700 1 -- 172.21.15.133:0/1537963630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b641014b0 msgr2=0x7f7b64072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:50.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.355+0000 7f7b63fff700 1 --2- 172.21.15.133:0/1537963630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b641014b0 0x7f7b64072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.355+0000 7f7b63fff700 1 -- 172.21.15.133:0/1537963630 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b54009520 con 0x7f7b641002c0 2023-11-27T15:20:50.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.355+0000 7f7b63fff700 1 --2- 172.21.15.133:0/1537963630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b641002c0 0x7f7b640748a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400ea50 tx=0x7f7b5400ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:50.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.356+0000 7f7b617fa700 1 -- 172.21.15.133:0/1537963630 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5400cbd0 con 0x7f7b641002c0 2023-11-27T15:20:50.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.356+0000 7f7b6a7ff700 1 -- 172.21.15.133:0/1537963630 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b580095f0 con 0x7f7b641002c0 2023-11-27T15:20:50.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.356+0000 7f7b617fa700 1 -- 172.21.15.133:0/1537963630 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b5400cd30 con 0x7f7b641002c0 2023-11-27T15:20:50.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.356+0000 7f7b617fa700 1 -- 172.21.15.133:0/1537963630 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b54018680 con 0x7f7b641002c0 2023-11-27T15:20:50.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.356+0000 7f7b6a7ff700 1 -- 172.21.15.133:0/1537963630 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6410bc70 con 0x7f7b641002c0 2023-11-27T15:20:50.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.357+0000 7f7b6a7ff700 1 -- 172.21.15.133:0/1537963630 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b64107860 con 0x7f7b641002c0 2023-11-27T15:20:50.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.359+0000 7f7b617fa700 1 -- 172.21.15.133:0/1537963630 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7b540188a0 con 0x7f7b641002c0 2023-11-27T15:20:50.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.360+0000 7f7b617fa700 1 --2- 172.21.15.133:0/1537963630 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7b4c0a1150 0x7f7b4c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:50.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.360+0000 7f7b617fa700 1 -- 172.21.15.133:0/1537963630 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7b54014070 con 0x7f7b641002c0 2023-11-27T15:20:50.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.360+0000 7f7b637fe700 1 --2- 172.21.15.133:0/1537963630 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7b4c0a1150 0x7f7b4c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:50.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.361+0000 7f7b637fe700 1 --2- 172.21.15.133:0/1537963630 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7b4c0a1150 0x7f7b4c0a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800b2e0 tx=0x7f7b58005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:50.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.362+0000 7f7b617fa700 1 -- 172.21.15.133:0/1537963630 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7b54096b70 con 0x7f7b641002c0 2023-11-27T15:20:50.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.561+0000 7f7b6a7ff700 1 -- 172.21.15.133:0/1537963630 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7b6402cfe0 con 0x7f7b641002c0 2023-11-27T15:20:50.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.562+0000 7f7b617fa700 1 -- 172.21.15.133:0/1537963630 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7b540b8e60 con 0x7f7b641002c0 2023-11-27T15:20:50.566 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-11-27T15:20:50.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.564+0000 7f7b6a7ff700 1 -- 172.21.15.133:0/1537963630 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7b4c0a1150 msgr2=0x7f7b4c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:50.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.564+0000 7f7b6a7ff700 1 --2- 172.21.15.133:0/1537963630 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7b4c0a1150 0x7f7b4c0a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7b5800b2e0 tx=0x7f7b58005fb0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.564+0000 7f7b6a7ff700 1 -- 172.21.15.133:0/1537963630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b641002c0 msgr2=0x7f7b640748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.564+0000 7f7b6a7ff700 1 --2- 172.21.15.133:0/1537963630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b641002c0 0x7f7b640748a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400ea50 tx=0x7f7b5400ed60 comp rx=0 tx=0).stop 2023-11-27T15:20:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.564+0000 7f7b6a7ff700 1 -- 172.21.15.133:0/1537963630 shutdown_connections 2023-11-27T15:20:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.564+0000 7f7b6a7ff700 1 --2- 172.21.15.133:0/1537963630 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7b4c0a1150 0x7f7b4c0a3600 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.564+0000 7f7b6a7ff700 1 --2- 172.21.15.133:0/1537963630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b64101e40 0x7f7b64073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.564+0000 7f7b6a7ff700 1 --2- 172.21.15.133:0/1537963630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b641002c0 0x7f7b640748a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.564+0000 7f7b6a7ff700 1 --2- 172.21.15.133:0/1537963630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b641014b0 0x7f7b64072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.564+0000 7f7b6a7ff700 1 -- 172.21.15.133:0/1537963630 >> 172.21.15.133:0/1537963630 conn(0x7f7b640fb8a0 msgr2=0x7f7b641049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.564+0000 7f7b6a7ff700 1 -- 172.21.15.133:0/1537963630 shutdown_connections 2023-11-27T15:20:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.564+0000 7f7b6a7ff700 1 -- 172.21.15.133:0/1537963630 wait complete. 2023-11-27T15:20:50.586 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-27T15:20:50.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.783+0000 7f1aba3ff700 1 -- 172.21.15.133:0/1609091111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ab41002c0 msgr2=0x7f1ab41006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:50.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.783+0000 7f1aba3ff700 1 --2- 172.21.15.133:0/1609091111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ab41002c0 0x7f1ab41006d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009990 tx=0x7f1aa4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.783+0000 7f1aba3ff700 1 -- 172.21.15.133:0/1609091111 shutdown_connections 2023-11-27T15:20:50.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.783+0000 7f1aba3ff700 1 --2- 172.21.15.133:0/1609091111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ab4101e40 0x7f1ab4108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.783+0000 7f1aba3ff700 1 --2- 172.21.15.133:0/1609091111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ab41002c0 0x7f1ab41006d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.783+0000 7f1aba3ff700 1 --2- 172.21.15.133:0/1609091111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ab41014b0 0x7f1ab4101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.783+0000 7f1aba3ff700 1 -- 172.21.15.133:0/1609091111 >> 172.21.15.133:0/1609091111 conn(0x7f1ab40fb8a0 msgr2=0x7f1ab40fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:50.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.783+0000 7f1aba3ff700 1 -- 172.21.15.133:0/1609091111 shutdown_connections 2023-11-27T15:20:50.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.783+0000 7f1aba3ff700 1 -- 172.21.15.133:0/1609091111 wait complete. 2023-11-27T15:20:50.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.784+0000 7f1aba3ff700 1 Processor -- start 2023-11-27T15:20:50.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.784+0000 7f1aba3ff700 1 -- start start 2023-11-27T15:20:50.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.784+0000 7f1aba3ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ab41002c0 0x7f1ab40748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:50.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.784+0000 7f1aba3ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ab41014b0 0x7f1ab4072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:50.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.784+0000 7f1aba3ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ab4101e40 0x7f1ab4073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:50.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.784+0000 7f1aba3ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab4073bc0 con 0x7f1ab41014b0 2023-11-27T15:20:50.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.784+0000 7f1aba3ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab4073d30 con 0x7f1ab41002c0 2023-11-27T15:20:50.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.784+0000 7f1aba3ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab41a00c0 con 0x7f1ab4101e40 2023-11-27T15:20:50.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.785+0000 7f1ab37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ab41014b0 0x7f1ab4072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:50.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.785+0000 7f1ab3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ab41002c0 0x7f1ab40748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:50.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.785+0000 7f1ab3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ab41002c0 0x7f1ab40748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:33818/0 (socket says 172.21.15.133:33818) 2023-11-27T15:20:50.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.785+0000 7f1ab3fff700 1 -- 172.21.15.133:0/73528006 learned_addr learned my addr 172.21.15.133:0/73528006 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:50.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.785+0000 7f1ab899c700 1 --2- 172.21.15.133:0/73528006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ab4101e40 0x7f1ab4073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:50.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.785+0000 7f1ab3fff700 1 -- 172.21.15.133:0/73528006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ab4101e40 msgr2=0x7f1ab4073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:50.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.785+0000 7f1ab3fff700 1 --2- 172.21.15.133:0/73528006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ab4101e40 0x7f1ab4073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.785+0000 7f1ab3fff700 1 -- 172.21.15.133:0/73528006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ab41014b0 msgr2=0x7f1ab4072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:50.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.785+0000 7f1ab3fff700 1 --2- 172.21.15.133:0/73528006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ab41014b0 0x7f1ab4072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.785+0000 7f1ab3fff700 1 -- 172.21.15.133:0/73528006 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa40095f0 con 0x7f1ab41002c0 2023-11-27T15:20:50.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.785+0000 7f1ab899c700 1 --2- 172.21.15.133:0/73528006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ab4101e40 0x7f1ab4073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:20:50.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.785+0000 7f1ab37fe700 1 --2- 172.21.15.133:0/73528006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ab41014b0 0x7f1ab4072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:50.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.785+0000 7f1ab3fff700 1 --2- 172.21.15.133:0/73528006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ab41002c0 0x7f1ab40748a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4005740 tx=0x7f1aa4005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:50.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.786+0000 7f1ab17fa700 1 -- 172.21.15.133:0/73528006 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa401d070 con 0x7f1ab41002c0 2023-11-27T15:20:50.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.786+0000 7f1aba3ff700 1 -- 172.21.15.133:0/73528006 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab410bc90 con 0x7f1ab41002c0 2023-11-27T15:20:50.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.786+0000 7f1ab17fa700 1 -- 172.21.15.133:0/73528006 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1aa4021410 con 0x7f1ab41002c0 2023-11-27T15:20:50.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.786+0000 7f1ab17fa700 1 -- 172.21.15.133:0/73528006 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4017650 con 0x7f1ab41002c0 2023-11-27T15:20:50.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.786+0000 7f1aba3ff700 1 -- 172.21.15.133:0/73528006 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab410bb00 con 0x7f1ab41002c0 2023-11-27T15:20:50.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.787+0000 7f1aba3ff700 1 -- 172.21.15.133:0/73528006 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab4067b00 con 0x7f1ab41002c0 2023-11-27T15:20:50.793 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.789+0000 7f1ab17fa700 1 -- 172.21.15.133:0/73528006 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1aa4021580 con 0x7f1ab41002c0 2023-11-27T15:20:50.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.790+0000 7f1ab17fa700 1 --2- 172.21.15.133:0/73528006 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1a9c0a1150 0x7f1a9c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:50.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.790+0000 7f1ab17fa700 1 -- 172.21.15.133:0/73528006 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1aa40ce480 con 0x7f1ab41002c0 2023-11-27T15:20:50.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.790+0000 7f1ab37fe700 1 --2- 172.21.15.133:0/73528006 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1a9c0a1150 0x7f1a9c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:50.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.791+0000 7f1ab37fe700 1 --2- 172.21.15.133:0/73528006 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1a9c0a1150 0x7f1a9c0a3600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8011490 tx=0x7f1aa8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:50.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.792+0000 7f1ab17fa700 1 -- 172.21.15.133:0/73528006 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1aa4097b00 con 0x7f1ab41002c0 2023-11-27T15:20:50.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:50 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/4091234408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:20:50.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:50 smithi174 ceph-mon[131110]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-11-27T15:20:50.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:50 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1755997889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:20:50.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:50 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1537963630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:20:50.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.989+0000 7f1aba3ff700 1 -- 172.21.15.133:0/73528006 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1ab4107170 con 0x7f1ab41002c0 2023-11-27T15:20:50.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.990+0000 7f1ab17fa700 1 -- 172.21.15.133:0/73528006 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1aa409b120 con 0x7f1ab41002c0 2023-11-27T15:20:50.994 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:20:50.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.992+0000 7f1aba3ff700 1 -- 172.21.15.133:0/73528006 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1a9c0a1150 msgr2=0x7f1a9c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:50.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.992+0000 7f1aba3ff700 1 --2- 172.21.15.133:0/73528006 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1a9c0a1150 0x7f1a9c0a3600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8011490 tx=0x7f1aa8009260 comp rx=0 tx=0).stop 2023-11-27T15:20:50.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.992+0000 7f1aba3ff700 1 -- 172.21.15.133:0/73528006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ab41002c0 msgr2=0x7f1ab40748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:50.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.992+0000 7f1aba3ff700 1 --2- 172.21.15.133:0/73528006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ab41002c0 0x7f1ab40748a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4005740 tx=0x7f1aa4005770 comp rx=0 tx=0).stop 2023-11-27T15:20:50.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.992+0000 7f1aba3ff700 1 -- 172.21.15.133:0/73528006 shutdown_connections 2023-11-27T15:20:50.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.992+0000 7f1aba3ff700 1 --2- 172.21.15.133:0/73528006 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1a9c0a1150 0x7f1a9c0a3600 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.992+0000 7f1aba3ff700 1 --2- 172.21.15.133:0/73528006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ab4101e40 0x7f1ab4073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.992+0000 7f1aba3ff700 1 --2- 172.21.15.133:0/73528006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ab41002c0 0x7f1ab40748a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.992+0000 7f1aba3ff700 1 --2- 172.21.15.133:0/73528006 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ab41014b0 0x7f1ab4072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:50.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.992+0000 7f1aba3ff700 1 -- 172.21.15.133:0/73528006 >> 172.21.15.133:0/73528006 conn(0x7f1ab40fb8a0 msgr2=0x7f1ab40fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:50.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.993+0000 7f1aba3ff700 1 -- 172.21.15.133:0/73528006 shutdown_connections 2023-11-27T15:20:50.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:50.993+0000 7f1aba3ff700 1 -- 172.21.15.133:0/73528006 wait complete. 2023-11-27T15:20:51.012 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:50 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/4091234408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:20:51.013 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:50 smithi133 ceph-mon[97860]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-11-27T15:20:51.013 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:50 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1755997889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:20:51.013 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:50 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1537963630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:20:51.015 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-27T15:20:51.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:50 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/4091234408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:20:51.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:50 smithi177 ceph-mon[131504]: pgmap v282: 129 pgs: 129 active+clean; 579 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-11-27T15:20:51.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:50 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1755997889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:20:51.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:50 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1537963630' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:20:51.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.209+0000 7f93131ff700 1 -- 172.21.15.133:0/1544020838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f930c100230 msgr2=0x7f930c100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:51.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.209+0000 7f93131ff700 1 --2- 172.21.15.133:0/1544020838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f930c100230 0x7f930c100640 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9300009990 tx=0x7f9300009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:51.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.209+0000 7f93131ff700 1 -- 172.21.15.133:0/1544020838 shutdown_connections 2023-11-27T15:20:51.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.209+0000 7f93131ff700 1 --2- 172.21.15.133:0/1544020838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f930c101db0 0x7f930c108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:51.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.209+0000 7f93131ff700 1 --2- 172.21.15.133:0/1544020838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f930c100230 0x7f930c100640 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:51.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.209+0000 7f93131ff700 1 --2- 172.21.15.133:0/1544020838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f930c101420 0x7f930c101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:51.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.209+0000 7f93131ff700 1 -- 172.21.15.133:0/1544020838 >> 172.21.15.133:0/1544020838 conn(0x7f930c0fb810 msgr2=0x7f930c0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:51.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.209+0000 7f93131ff700 1 -- 172.21.15.133:0/1544020838 shutdown_connections 2023-11-27T15:20:51.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.209+0000 7f93131ff700 1 -- 172.21.15.133:0/1544020838 wait complete. 2023-11-27T15:20:51.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.210+0000 7f93131ff700 1 Processor -- start 2023-11-27T15:20:51.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.210+0000 7f93131ff700 1 -- start start 2023-11-27T15:20:51.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.210+0000 7f93131ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f930c100230 0x7f930c0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:51.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.210+0000 7f93131ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f930c101420 0x7f930c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:51.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.210+0000 7f93131ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f930c101db0 0x7f930c073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:51.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.210+0000 7f93131ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c073bc0 con 0x7f930c100230 2023-11-27T15:20:51.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.210+0000 7f93131ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c073d30 con 0x7f930c101420 2023-11-27T15:20:51.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.210+0000 7f93131ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c1a0020 con 0x7f930c101db0 2023-11-27T15:20:51.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.211+0000 7f9310f9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f930c100230 0x7f930c0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:51.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.211+0000 7f9310f9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f930c100230 0x7f930c0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32900/0 (socket says 172.21.15.133:32900) 2023-11-27T15:20:51.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.211+0000 7f9310f9b700 1 -- 172.21.15.133:0/1653091595 learned_addr learned my addr 172.21.15.133:0/1653091595 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:51.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.211+0000 7f930bfff700 1 --2- 172.21.15.133:0/1653091595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f930c101420 0x7f930c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:51.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.211+0000 7f931179c700 1 --2- 172.21.15.133:0/1653091595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f930c101db0 0x7f930c073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:51.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.211+0000 7f9310f9b700 1 -- 172.21.15.133:0/1653091595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f930c101db0 msgr2=0x7f930c073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:51.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.211+0000 7f9310f9b700 1 --2- 172.21.15.133:0/1653091595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f930c101db0 0x7f930c073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:51.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.211+0000 7f9310f9b700 1 -- 172.21.15.133:0/1653091595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f930c101420 msgr2=0x7f930c072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:51.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.211+0000 7f9310f9b700 1 --2- 172.21.15.133:0/1653091595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f930c101420 0x7f930c072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:51.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.211+0000 7f9310f9b700 1 -- 172.21.15.133:0/1653091595 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9304009c40 con 0x7f930c100230 2023-11-27T15:20:51.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.211+0000 7f931179c700 1 --2- 172.21.15.133:0/1653091595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f930c101db0 0x7f930c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:20:51.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.211+0000 7f9310f9b700 1 --2- 172.21.15.133:0/1653091595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f930c100230 0x7f930c0748a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f930000b620 tx=0x7f9300004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:51.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.212+0000 7f9309ffb700 1 -- 172.21.15.133:0/1653091595 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930001d070 con 0x7f930c100230 2023-11-27T15:20:51.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.212+0000 7f93131ff700 1 -- 172.21.15.133:0/1653091595 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93000095f0 con 0x7f930c100230 2023-11-27T15:20:51.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.212+0000 7f9309ffb700 1 -- 172.21.15.133:0/1653091595 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9300005030 con 0x7f930c100230 2023-11-27T15:20:51.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.212+0000 7f9309ffb700 1 -- 172.21.15.133:0/1653091595 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9300004bf0 con 0x7f930c100230 2023-11-27T15:20:51.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.212+0000 7f93131ff700 1 -- 172.21.15.133:0/1653091595 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930c10bba0 con 0x7f930c100230 2023-11-27T15:20:51.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.213+0000 7f93131ff700 1 -- 172.21.15.133:0/1653091595 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f930c10bd30 con 0x7f930c100230 2023-11-27T15:20:51.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.215+0000 7f9309ffb700 1 -- 172.21.15.133:0/1653091595 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f9300004d50 con 0x7f930c100230 2023-11-27T15:20:51.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.216+0000 7f9309ffb700 1 --2- 172.21.15.133:0/1653091595 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92fc0a1150 0x7f92fc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:51.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.216+0000 7f9309ffb700 1 -- 172.21.15.133:0/1653091595 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f93000cdbd0 con 0x7f930c100230 2023-11-27T15:20:51.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.216+0000 7f930bfff700 1 --2- 172.21.15.133:0/1653091595 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92fc0a1150 0x7f92fc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:51.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.217+0000 7f930bfff700 1 --2- 172.21.15.133:0/1653091595 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92fc0a1150 0x7f92fc0a3600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009c60 tx=0x7f92f8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:51.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.219+0000 7f9309ffb700 1 -- 172.21.15.133:0/1653091595 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9300097160 con 0x7f930c100230 2023-11-27T15:20:51.438 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.434+0000 7f93131ff700 1 -- 172.21.15.133:0/1653091595 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f930c107130 con 0x7f930c100230 2023-11-27T15:20:51.438 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.435+0000 7f9309ffb700 1 -- 172.21.15.133:0/1653091595 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+602 (secure 0 0 0) 0x7f930009a780 con 0x7f930c100230 2023-11-27T15:20:51.439 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6800/2374615220 2023-11-28T15:14:01.628185+0000 2023-11-27T15:20:51.439 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/1594111147 2023-11-28T15:14:01.628185+0000 2023-11-27T15:20:51.439 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6801/3149426454 2023-11-28T15:13:35.034149+0000 2023-11-27T15:20:51.439 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6800/3149426454 2023-11-28T15:13:35.034149+0000 2023-11-27T15:20:51.439 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/2898896137 2023-11-28T15:14:01.628185+0000 2023-11-27T15:20:51.440 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/1816614695 2023-11-28T15:13:35.034149+0000 2023-11-27T15:20:51.440 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/2624811615 2023-11-28T15:13:35.034149+0000 2023-11-27T15:20:51.440 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6801/2374615220 2023-11-28T15:14:01.628185+0000 2023-11-27T15:20:51.440 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/4208867747 2023-11-28T15:14:01.628185+0000 2023-11-27T15:20:51.440 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/4192331415 2023-11-28T15:13:35.034149+0000 2023-11-27T15:20:51.441 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.437+0000 7f93131ff700 1 -- 172.21.15.133:0/1653091595 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92fc0a1150 msgr2=0x7f92fc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:51.441 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.437+0000 7f93131ff700 1 --2- 172.21.15.133:0/1653091595 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92fc0a1150 0x7f92fc0a3600 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009c60 tx=0x7f92f8009260 comp rx=0 tx=0).stop 2023-11-27T15:20:51.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.437+0000 7f93131ff700 1 -- 172.21.15.133:0/1653091595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f930c100230 msgr2=0x7f930c0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:51.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.437+0000 7f93131ff700 1 --2- 172.21.15.133:0/1653091595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f930c100230 0x7f930c0748a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f930000b620 tx=0x7f9300004730 comp rx=0 tx=0).stop 2023-11-27T15:20:51.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.437+0000 7f93131ff700 1 -- 172.21.15.133:0/1653091595 shutdown_connections 2023-11-27T15:20:51.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.437+0000 7f93131ff700 1 --2- 172.21.15.133:0/1653091595 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92fc0a1150 0x7f92fc0a3600 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:51.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.437+0000 7f93131ff700 1 --2- 172.21.15.133:0/1653091595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f930c101db0 0x7f930c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:51.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.437+0000 7f93131ff700 1 --2- 172.21.15.133:0/1653091595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f930c100230 0x7f930c0748a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:51.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.437+0000 7f93131ff700 1 --2- 172.21.15.133:0/1653091595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f930c101420 0x7f930c072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:51.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.437+0000 7f93131ff700 1 -- 172.21.15.133:0/1653091595 >> 172.21.15.133:0/1653091595 conn(0x7f930c0fb810 msgr2=0x7f930c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:51.443 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.437+0000 7f93131ff700 1 -- 172.21.15.133:0/1653091595 shutdown_connections 2023-11-27T15:20:51.443 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.437+0000 7f93131ff700 1 -- 172.21.15.133:0/1653091595 wait complete. 2023-11-27T15:20:51.443 INFO:teuthology.orchestra.run.smithi133.stderr:listed 10 entries 2023-11-27T15:20:51.459 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-11-27T15:20:51.459 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-27T15:20:51.504 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:20:51.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.740+0000 7f9600bff700 1 -- 172.21.15.133:0/4075534192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95fc100290 msgr2=0x7f95fc1006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:51.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.740+0000 7f9600bff700 1 --2- 172.21.15.133:0/4075534192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95fc100290 0x7f95fc1006a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009990 tx=0x7f95ec009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:51.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.740+0000 7f9600bff700 1 -- 172.21.15.133:0/4075534192 shutdown_connections 2023-11-27T15:20:51.744 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.740+0000 7f9600bff700 1 --2- 172.21.15.133:0/4075534192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95fc101e10 0x7f95fc1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:51.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.740+0000 7f9600bff700 1 --2- 172.21.15.133:0/4075534192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95fc100290 0x7f95fc1006a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:51.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.740+0000 7f9600bff700 1 --2- 172.21.15.133:0/4075534192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95fc101480 0x7f95fc1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:51.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.740+0000 7f9600bff700 1 -- 172.21.15.133:0/4075534192 >> 172.21.15.133:0/4075534192 conn(0x7f95fc0fb870 msgr2=0x7f95fc0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:51.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.740+0000 7f9600bff700 1 -- 172.21.15.133:0/4075534192 shutdown_connections 2023-11-27T15:20:51.745 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.740+0000 7f9600bff700 1 -- 172.21.15.133:0/4075534192 wait complete. 2023-11-27T15:20:51.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.741+0000 7f9600bff700 1 Processor -- start 2023-11-27T15:20:51.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.741+0000 7f9600bff700 1 -- start start 2023-11-27T15:20:51.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.741+0000 7f9600bff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95fc100290 0x7f95fc195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:51.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.741+0000 7f9600bff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95fc101480 0x7f95fc1964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:51.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.741+0000 7f9600bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95fc101e10 0x7f95fc19d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:51.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.741+0000 7f9600bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc19db20 con 0x7f95fc101e10 2023-11-27T15:20:51.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.741+0000 7f9600bff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc19dc60 con 0x7f95fc100290 2023-11-27T15:20:51.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.741+0000 7f9600bff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc19ddf0 con 0x7f95fc101480 2023-11-27T15:20:51.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.742+0000 7f95fa59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95fc100290 0x7f95fc195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:51.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.742+0000 7f95fad9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95fc101e10 0x7f95fc19d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:51.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.742+0000 7f95fa59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95fc100290 0x7f95fc195fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:33852/0 (socket says 172.21.15.133:33852) 2023-11-27T15:20:51.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.742+0000 7f95fa59c700 1 -- 172.21.15.133:0/4190547768 learned_addr learned my addr 172.21.15.133:0/4190547768 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:51.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.742+0000 7f95f9d9b700 1 --2- 172.21.15.133:0/4190547768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95fc101480 0x7f95fc1964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:51.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.742+0000 7f95fa59c700 1 -- 172.21.15.133:0/4190547768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95fc101480 msgr2=0x7f95fc1964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:51.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.742+0000 7f95fa59c700 1 --2- 172.21.15.133:0/4190547768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95fc101480 0x7f95fc1964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:51.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.742+0000 7f95fa59c700 1 -- 172.21.15.133:0/4190547768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95fc101e10 msgr2=0x7f95fc19d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:51.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.742+0000 7f95fa59c700 1 --2- 172.21.15.133:0/4190547768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95fc101e10 0x7f95fc19d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:51.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.742+0000 7f95fa59c700 1 -- 172.21.15.133:0/4190547768 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ec0095f0 con 0x7f95fc100290 2023-11-27T15:20:51.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.742+0000 7f95fad9d700 1 --2- 172.21.15.133:0/4190547768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95fc101e10 0x7f95fc19d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:51.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.742+0000 7f95f9d9b700 1 --2- 172.21.15.133:0/4190547768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95fc101480 0x7f95fc1964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:51.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.742+0000 7f95fa59c700 1 --2- 172.21.15.133:0/4190547768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95fc100290 0x7f95fc195fa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f95ec0098e0 tx=0x7f95ec004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:51.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.743+0000 7f95eb7fe700 1 -- 172.21.15.133:0/4190547768 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ec01d070 con 0x7f95fc100290 2023-11-27T15:20:51.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.743+0000 7f95eb7fe700 1 -- 172.21.15.133:0/4190547768 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95ec005030 con 0x7f95fc100290 2023-11-27T15:20:51.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.743+0000 7f9600bff700 1 -- 172.21.15.133:0/4190547768 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95fc10bc40 con 0x7f95fc100290 2023-11-27T15:20:51.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.743+0000 7f95eb7fe700 1 -- 172.21.15.133:0/4190547768 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ec004b50 con 0x7f95fc100290 2023-11-27T15:20:51.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.743+0000 7f9600bff700 1 -- 172.21.15.133:0/4190547768 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95fc10bab0 con 0x7f95fc100290 2023-11-27T15:20:51.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.744+0000 7f9600bff700 1 -- 172.21.15.133:0/4190547768 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95fc107210 con 0x7f95fc100290 2023-11-27T15:20:51.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.746+0000 7f95eb7fe700 1 -- 172.21.15.133:0/4190547768 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f95ec004d30 con 0x7f95fc100290 2023-11-27T15:20:51.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.747+0000 7f95eb7fe700 1 --2- 172.21.15.133:0/4190547768 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f95e40a1150 0x7f95e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:51.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.747+0000 7f95eb7fe700 1 -- 172.21.15.133:0/4190547768 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f95ec0cdde0 con 0x7f95fc100290 2023-11-27T15:20:51.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.747+0000 7f95f9d9b700 1 --2- 172.21.15.133:0/4190547768 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f95e40a1150 0x7f95e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:51.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.748+0000 7f95f9d9b700 1 --2- 172.21.15.133:0/4190547768 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f95e40a1150 0x7f95e40a3600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f95f0011490 tx=0x7f95f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:51.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.751+0000 7f95eb7fe700 1 -- 172.21.15.133:0/4190547768 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f95ec05a730 con 0x7f95fc100290 2023-11-27T15:20:51.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:51 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/73528006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:20:51.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:51 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1653091595' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-27T15:20:51.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:51 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/73528006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:20:51.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:51 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1653091595' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-27T15:20:51.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.952+0000 7f9600bff700 1 -- 172.21.15.133:0/4190547768 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95fc067a60 con 0x7f95fc100290 2023-11-27T15:20:51.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.954+0000 7f95eb7fe700 1 -- 172.21.15.133:0/4190547768 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23996 (secure 0 0 0) 0x7f95ec09ae00 con 0x7f95fc100290 2023-11-27T15:20:51.959 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:20:51.959 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:20:38.147036+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:20:51.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.957+0000 7f9600bff700 1 -- 172.21.15.133:0/4190547768 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f95e40a1150 msgr2=0x7f95e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:51.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.957+0000 7f9600bff700 1 --2- 172.21.15.133:0/4190547768 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f95e40a1150 0x7f95e40a3600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f95f0011490 tx=0x7f95f0009260 comp rx=0 tx=0).stop 2023-11-27T15:20:51.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.957+0000 7f9600bff700 1 -- 172.21.15.133:0/4190547768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95fc100290 msgr2=0x7f95fc195fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:51.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.957+0000 7f9600bff700 1 --2- 172.21.15.133:0/4190547768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95fc100290 0x7f95fc195fa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f95ec0098e0 tx=0x7f95ec004730 comp rx=0 tx=0).stop 2023-11-27T15:20:51.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.957+0000 7f9600bff700 1 -- 172.21.15.133:0/4190547768 shutdown_connections 2023-11-27T15:20:51.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.957+0000 7f9600bff700 1 --2- 172.21.15.133:0/4190547768 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f95e40a1150 0x7f95e40a3600 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:51.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.957+0000 7f9600bff700 1 --2- 172.21.15.133:0/4190547768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95fc101e10 0x7f95fc19d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:51.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.957+0000 7f9600bff700 1 --2- 172.21.15.133:0/4190547768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95fc100290 0x7f95fc195fa0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:51.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.957+0000 7f9600bff700 1 --2- 172.21.15.133:0/4190547768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95fc101480 0x7f95fc1964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:51.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.957+0000 7f9600bff700 1 -- 172.21.15.133:0/4190547768 >> 172.21.15.133:0/4190547768 conn(0x7f95fc0fb870 msgr2=0x7f95fc0fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:51.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.957+0000 7f9600bff700 1 -- 172.21.15.133:0/4190547768 shutdown_connections 2023-11-27T15:20:51.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:51.957+0000 7f9600bff700 1 -- 172.21.15.133:0/4190547768 wait complete. 2023-11-27T15:20:51.979 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:16:20.216150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:20:31.107146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:20:32.614089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-27T15:20:36.126040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-27T15:20:51.980 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:20:52.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:51 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/73528006' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:20:52.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:51 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1653091595' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-27T15:20:52.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.174+0000 7f11fcdff700 1 -- 172.21.15.133:0/696003628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11f81002c0 msgr2=0x7f11f81006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:52.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.174+0000 7f11fcdff700 1 --2- 172.21.15.133:0/696003628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11f81002c0 0x7f11f81006d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009990 tx=0x7f11e8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.174+0000 7f11fcdff700 1 -- 172.21.15.133:0/696003628 shutdown_connections 2023-11-27T15:20:52.178 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.174+0000 7f11fcdff700 1 --2- 172.21.15.133:0/696003628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11f8101e40 0x7f11f8108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.174+0000 7f11fcdff700 1 --2- 172.21.15.133:0/696003628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11f81002c0 0x7f11f81006d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.174+0000 7f11fcdff700 1 --2- 172.21.15.133:0/696003628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11f81014b0 0x7f11f8101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.174+0000 7f11fcdff700 1 -- 172.21.15.133:0/696003628 >> 172.21.15.133:0/696003628 conn(0x7f11f80fb8a0 msgr2=0x7f11f80fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:52.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.174+0000 7f11fcdff700 1 -- 172.21.15.133:0/696003628 shutdown_connections 2023-11-27T15:20:52.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.174+0000 7f11fcdff700 1 -- 172.21.15.133:0/696003628 wait complete. 2023-11-27T15:20:52.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.175+0000 7f11fcdff700 1 Processor -- start 2023-11-27T15:20:52.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.175+0000 7f11fcdff700 1 -- start start 2023-11-27T15:20:52.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.176+0000 7f11fcdff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11f81002c0 0x7f11f8198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:52.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.176+0000 7f11fcdff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11f81014b0 0x7f11f8198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:52.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.176+0000 7f11fcdff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11f8101e40 0x7f11f819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:52.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.176+0000 7f11fcdff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f819fdd0 con 0x7f11f8101e40 2023-11-27T15:20:52.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.176+0000 7f11fcdff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f819ff10 con 0x7f11f81002c0 2023-11-27T15:20:52.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.176+0000 7f11fcdff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f81a00a0 con 0x7f11f81014b0 2023-11-27T15:20:52.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.176+0000 7f11f6d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11f8101e40 0x7f11f819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:52.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.176+0000 7f11f6d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11f8101e40 0x7f11f819f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32930/0 (socket says 172.21.15.133:32930) 2023-11-27T15:20:52.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.176+0000 7f11f6d9d700 1 -- 172.21.15.133:0/536381762 learned_addr learned my addr 172.21.15.133:0/536381762 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:52.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.176+0000 7f11f5d9b700 1 --2- 172.21.15.133:0/536381762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11f81014b0 0x7f11f8198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:52.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.176+0000 7f11f659c700 1 --2- 172.21.15.133:0/536381762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11f81002c0 0x7f11f8198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:52.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.176+0000 7f11f6d9d700 1 -- 172.21.15.133:0/536381762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11f81014b0 msgr2=0x7f11f8198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:52.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.176+0000 7f11f6d9d700 1 --2- 172.21.15.133:0/536381762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11f81014b0 0x7f11f8198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.176+0000 7f11f6d9d700 1 -- 172.21.15.133:0/536381762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11f81002c0 msgr2=0x7f11f8198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:52.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.176+0000 7f11f6d9d700 1 --2- 172.21.15.133:0/536381762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11f81002c0 0x7f11f8198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.176+0000 7f11f6d9d700 1 -- 172.21.15.133:0/536381762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e80095f0 con 0x7f11f8101e40 2023-11-27T15:20:52.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.177+0000 7f11f6d9d700 1 --2- 172.21.15.133:0/536381762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11f8101e40 0x7f11f819f800 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f11f000ef30 tx=0x7f11f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:52.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.177+0000 7f11e77fe700 1 -- 172.21.15.133:0/536381762 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0018070 con 0x7f11f8101e40 2023-11-27T15:20:52.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.177+0000 7f11e77fe700 1 -- 172.21.15.133:0/536381762 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11f000f040 con 0x7f11f8101e40 2023-11-27T15:20:52.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.177+0000 7f11fcdff700 1 -- 172.21.15.133:0/536381762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f810bbc0 con 0x7f11f8101e40 2023-11-27T15:20:52.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.177+0000 7f11e77fe700 1 -- 172.21.15.133:0/536381762 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f000cd00 con 0x7f11f8101e40 2023-11-27T15:20:52.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.177+0000 7f11fcdff700 1 -- 172.21.15.133:0/536381762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f810ba30 con 0x7f11f8101e40 2023-11-27T15:20:52.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.178+0000 7f11fcdff700 1 -- 172.21.15.133:0/536381762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f8192920 con 0x7f11f8101e40 2023-11-27T15:20:52.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.180+0000 7f11e77fe700 1 -- 172.21.15.133:0/536381762 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f11f0003a00 con 0x7f11f8101e40 2023-11-27T15:20:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.181+0000 7f11e77fe700 1 --2- 172.21.15.133:0/536381762 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f11e00a1150 0x7f11e00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.181+0000 7f11e77fe700 1 -- 172.21.15.133:0/536381762 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f11f00cc5d0 con 0x7f11f8101e40 2023-11-27T15:20:52.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.182+0000 7f11f659c700 1 --2- 172.21.15.133:0/536381762 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f11e00a1150 0x7f11e00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:52.186 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.182+0000 7f11f659c700 1 --2- 172.21.15.133:0/536381762 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f11e00a1150 0x7f11e00a3600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f11e8006010 tx=0x7f11e8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:52.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.185+0000 7f11e77fe700 1 -- 172.21.15.133:0/536381762 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f11f0095b60 con 0x7f11f8101e40 2023-11-27T15:20:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.384+0000 7f11fcdff700 1 -- 172.21.15.133:0/536381762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11f8067b00 con 0x7f11f8101e40 2023-11-27T15:20:52.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.384+0000 7f11e77fe700 1 -- 172.21.15.133:0/536381762 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11f0099180 con 0x7f11f8101e40 2023-11-27T15:20:52.388 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-11-27T15:20:52.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.387+0000 7f11fcdff700 1 -- 172.21.15.133:0/536381762 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f11e00a1150 msgr2=0x7f11e00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:52.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.387+0000 7f11fcdff700 1 --2- 172.21.15.133:0/536381762 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f11e00a1150 0x7f11e00a3600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f11e8006010 tx=0x7f11e8005960 comp rx=0 tx=0).stop 2023-11-27T15:20:52.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.387+0000 7f11fcdff700 1 -- 172.21.15.133:0/536381762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11f8101e40 msgr2=0x7f11f819f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:52.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.387+0000 7f11fcdff700 1 --2- 172.21.15.133:0/536381762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11f8101e40 0x7f11f819f800 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f11f000ef30 tx=0x7f11f000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.387+0000 7f11fcdff700 1 -- 172.21.15.133:0/536381762 shutdown_connections 2023-11-27T15:20:52.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.387+0000 7f11fcdff700 1 --2- 172.21.15.133:0/536381762 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f11e00a1150 0x7f11e00a3600 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.387+0000 7f11fcdff700 1 --2- 172.21.15.133:0/536381762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11f8101e40 0x7f11f819f800 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.387+0000 7f11fcdff700 1 --2- 172.21.15.133:0/536381762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11f81002c0 0x7f11f8198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.387+0000 7f11fcdff700 1 --2- 172.21.15.133:0/536381762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11f81014b0 0x7f11f8198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.387+0000 7f11fcdff700 1 -- 172.21.15.133:0/536381762 >> 172.21.15.133:0/536381762 conn(0x7f11f80fb8a0 msgr2=0x7f11f8106e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:52.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.387+0000 7f11fcdff700 1 -- 172.21.15.133:0/536381762 shutdown_connections 2023-11-27T15:20:52.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.387+0000 7f11fcdff700 1 -- 172.21.15.133:0/536381762 wait complete. 2023-11-27T15:20:52.409 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:20:52.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.606+0000 7f233a3ff700 1 -- 172.21.15.133:0/461124013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2334101e50 msgr2=0x7f2334108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:52.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.606+0000 7f233a3ff700 1 --2- 172.21.15.133:0/461124013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2334101e50 0x7f2334108710 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f232c00b0a0 tx=0x7f232c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.606+0000 7f233a3ff700 1 -- 172.21.15.133:0/461124013 shutdown_connections 2023-11-27T15:20:52.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.606+0000 7f233a3ff700 1 --2- 172.21.15.133:0/461124013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2334101e50 0x7f2334108710 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.606+0000 7f233a3ff700 1 --2- 172.21.15.133:0/461124013 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23341002d0 0x7f23341006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.606+0000 7f233a3ff700 1 --2- 172.21.15.133:0/461124013 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23341014c0 0x7f2334101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.606+0000 7f233a3ff700 1 -- 172.21.15.133:0/461124013 >> 172.21.15.133:0/461124013 conn(0x7f23340fb8b0 msgr2=0x7f23340fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:52.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.606+0000 7f233a3ff700 1 -- 172.21.15.133:0/461124013 shutdown_connections 2023-11-27T15:20:52.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.606+0000 7f233a3ff700 1 -- 172.21.15.133:0/461124013 wait complete. 2023-11-27T15:20:52.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.607+0000 7f233a3ff700 1 Processor -- start 2023-11-27T15:20:52.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.607+0000 7f233a3ff700 1 -- start start 2023-11-27T15:20:52.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.607+0000 7f233a3ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23341002d0 0x7f2334198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:52.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.607+0000 7f233a3ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23341014c0 0x7f23341988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:52.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.607+0000 7f233a3ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2334101e50 0x7f233419f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:52.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.607+0000 7f233a3ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233419fee0 con 0x7f23341002d0 2023-11-27T15:20:52.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.607+0000 7f233a3ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23341a0020 con 0x7f2334101e50 2023-11-27T15:20:52.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.607+0000 7f233a3ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23341a01b0 con 0x7f23341014c0 2023-11-27T15:20:52.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.608+0000 7f23337fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23341014c0 0x7f23341988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:52.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.608+0000 7f23337fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23341014c0 0x7f23341988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:36312/0 (socket says 172.21.15.133:36312) 2023-11-27T15:20:52.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.608+0000 7f23337fe700 1 -- 172.21.15.133:0/3466648048 learned_addr learned my addr 172.21.15.133:0/3466648048 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:52.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.608+0000 7f2333fff700 1 --2- 172.21.15.133:0/3466648048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23341002d0 0x7f2334198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:52.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.608+0000 7f233899c700 1 --2- 172.21.15.133:0/3466648048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2334101e50 0x7f233419f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:52.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.608+0000 7f23337fe700 1 -- 172.21.15.133:0/3466648048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2334101e50 msgr2=0x7f233419f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:52.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.608+0000 7f23337fe700 1 --2- 172.21.15.133:0/3466648048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2334101e50 0x7f233419f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.608+0000 7f23337fe700 1 -- 172.21.15.133:0/3466648048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23341002d0 msgr2=0x7f2334198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:52.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.608+0000 7f23337fe700 1 --2- 172.21.15.133:0/3466648048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23341002d0 0x7f2334198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.608+0000 7f23337fe700 1 -- 172.21.15.133:0/3466648048 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232c009d00 con 0x7f23341014c0 2023-11-27T15:20:52.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.608+0000 7f23337fe700 1 --2- 172.21.15.133:0/3466648048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23341014c0 0x7f23341988a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f232800e9a0 tx=0x7f232800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:52.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.608+0000 7f23317fa700 1 -- 172.21.15.133:0/3466648048 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328009790 con 0x7f23341014c0 2023-11-27T15:20:52.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.609+0000 7f23317fa700 1 -- 172.21.15.133:0/3466648048 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2328010430 con 0x7f23341014c0 2023-11-27T15:20:52.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.609+0000 7f233a3ff700 1 -- 172.21.15.133:0/3466648048 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f233410bca0 con 0x7f23341014c0 2023-11-27T15:20:52.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.609+0000 7f23317fa700 1 -- 172.21.15.133:0/3466648048 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328018680 con 0x7f23341014c0 2023-11-27T15:20:52.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.609+0000 7f233a3ff700 1 -- 172.21.15.133:0/3466648048 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233410bb10 con 0x7f23341014c0 2023-11-27T15:20:52.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.610+0000 7f233a3ff700 1 -- 172.21.15.133:0/3466648048 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23341a08a0 con 0x7f23341014c0 2023-11-27T15:20:52.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.612+0000 7f23317fa700 1 -- 172.21.15.133:0/3466648048 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f23280105a0 con 0x7f23341014c0 2023-11-27T15:20:52.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.613+0000 7f23317fa700 1 --2- 172.21.15.133:0/3466648048 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f231c0a1150 0x7f231c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:52.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.613+0000 7f23317fa700 1 -- 172.21.15.133:0/3466648048 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2328014070 con 0x7f23341014c0 2023-11-27T15:20:52.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.613+0000 7f2333fff700 1 --2- 172.21.15.133:0/3466648048 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f231c0a1150 0x7f231c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:52.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.614+0000 7f2333fff700 1 --2- 172.21.15.133:0/3466648048 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f231c0a1150 0x7f231c0a3600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2334101320 tx=0x7f2324009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:52.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.615+0000 7f23317fa700 1 -- 172.21.15.133:0/3466648048 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2328096410 con 0x7f23341014c0 2023-11-27T15:20:52.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:52 smithi133 ceph-mon[97860]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-27T15:20:52.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:52 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/4190547768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:20:52.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:52 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/536381762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:20:52.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.817+0000 7f233a3ff700 1 -- 172.21.15.133:0/3466648048 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f23341a0e50 con 0x7f23341014c0 2023-11-27T15:20:52.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.818+0000 7f23317fa700 1 -- 172.21.15.133:0/3466648048 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2328099a30 con 0x7f23341014c0 2023-11-27T15:20:52.822 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:20:52.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.821+0000 7f233a3ff700 1 -- 172.21.15.133:0/3466648048 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f231c0a1150 msgr2=0x7f231c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:52.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.821+0000 7f233a3ff700 1 --2- 172.21.15.133:0/3466648048 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f231c0a1150 0x7f231c0a3600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2334101320 tx=0x7f2324009260 comp rx=0 tx=0).stop 2023-11-27T15:20:52.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.821+0000 7f233a3ff700 1 -- 172.21.15.133:0/3466648048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23341014c0 msgr2=0x7f23341988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:52.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.821+0000 7f233a3ff700 1 --2- 172.21.15.133:0/3466648048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23341014c0 0x7f23341988a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f232800e9a0 tx=0x7f232800ecb0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.821+0000 7f233a3ff700 1 -- 172.21.15.133:0/3466648048 shutdown_connections 2023-11-27T15:20:52.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.821+0000 7f233a3ff700 1 --2- 172.21.15.133:0/3466648048 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f231c0a1150 0x7f231c0a3600 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.821+0000 7f233a3ff700 1 --2- 172.21.15.133:0/3466648048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2334101e50 0x7f233419f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.821+0000 7f233a3ff700 1 --2- 172.21.15.133:0/3466648048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23341002d0 0x7f2334198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.821+0000 7f233a3ff700 1 --2- 172.21.15.133:0/3466648048 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23341014c0 0x7f23341988a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:52.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.821+0000 7f233a3ff700 1 -- 172.21.15.133:0/3466648048 >> 172.21.15.133:0/3466648048 conn(0x7f23340fb8b0 msgr2=0x7f23340fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:52.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.821+0000 7f233a3ff700 1 -- 172.21.15.133:0/3466648048 shutdown_connections 2023-11-27T15:20:52.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:52.821+0000 7f233a3ff700 1 -- 172.21.15.133:0/3466648048 wait complete. 2023-11-27T15:20:52.843 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:20:52.902 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:52 smithi174 ceph-mon[131110]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-27T15:20:52.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:52 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/4190547768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:20:52.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:52 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/536381762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:20:53.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.039+0000 7f1d379ff700 1 -- 172.21.15.133:0/1815751603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d30074b30 msgr2=0x7f1d30072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:53.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.039+0000 7f1d379ff700 1 --2- 172.21.15.133:0/1815751603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d30074b30 0x7f1d30072f80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009990 tx=0x7f1d2c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.039+0000 7f1d379ff700 1 -- 172.21.15.133:0/1815751603 shutdown_connections 2023-11-27T15:20:53.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.039+0000 7f1d379ff700 1 --2- 172.21.15.133:0/1815751603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d3010c890 0x7f1d3010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.039+0000 7f1d379ff700 1 --2- 172.21.15.133:0/1815751603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d30074b30 0x7f1d30072f80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.039+0000 7f1d379ff700 1 --2- 172.21.15.133:0/1815751603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d30073550 0x7f1d300739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.039+0000 7f1d379ff700 1 -- 172.21.15.133:0/1815751603 >> 172.21.15.133:0/1815751603 conn(0x7f1d300fb8a0 msgr2=0x7f1d300fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:53.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.039+0000 7f1d379ff700 1 -- 172.21.15.133:0/1815751603 shutdown_connections 2023-11-27T15:20:53.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.039+0000 7f1d379ff700 1 -- 172.21.15.133:0/1815751603 wait complete. 2023-11-27T15:20:53.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.040+0000 7f1d379ff700 1 Processor -- start 2023-11-27T15:20:53.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.040+0000 7f1d379ff700 1 -- start start 2023-11-27T15:20:53.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.040+0000 7f1d379ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d30073550 0x7f1d3019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:53.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.040+0000 7f1d379ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d30074b30 0x7f1d3019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:53.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.040+0000 7f1d379ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d3010c890 0x7f1d301a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:53.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.040+0000 7f1d379ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d301a4250 con 0x7f1d30074b30 2023-11-27T15:20:53.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.040+0000 7f1d379ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d301a4390 con 0x7f1d3010c890 2023-11-27T15:20:53.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.040+0000 7f1d379ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d301a4520 con 0x7f1d30073550 2023-11-27T15:20:53.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.041+0000 7f1d3579b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d30073550 0x7f1d3019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:53.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.041+0000 7f1d34f9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d30074b30 0x7f1d3019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:53.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.041+0000 7f1d3579b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d30073550 0x7f1d3019c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:51082/0 (socket says 172.21.15.133:51082) 2023-11-27T15:20:53.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.041+0000 7f1d3579b700 1 -- 172.21.15.133:0/1501745016 learned_addr learned my addr 172.21.15.133:0/1501745016 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:53.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.041+0000 7f1d35f9c700 1 --2- 172.21.15.133:0/1501745016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d3010c890 0x7f1d301a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:53.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.041+0000 7f1d3579b700 1 -- 172.21.15.133:0/1501745016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d3010c890 msgr2=0x7f1d301a3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:53.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.041+0000 7f1d3579b700 1 --2- 172.21.15.133:0/1501745016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d3010c890 0x7f1d301a3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.041+0000 7f1d3579b700 1 -- 172.21.15.133:0/1501745016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d30074b30 msgr2=0x7f1d3019cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:53.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.041+0000 7f1d3579b700 1 --2- 172.21.15.133:0/1501745016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d30074b30 0x7f1d3019cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.041+0000 7f1d3579b700 1 -- 172.21.15.133:0/1501745016 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2c0095f0 con 0x7f1d30073550 2023-11-27T15:20:53.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.041+0000 7f1d34f9a700 1 --2- 172.21.15.133:0/1501745016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d30074b30 0x7f1d3019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:53.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.041+0000 7f1d3579b700 1 --2- 172.21.15.133:0/1501745016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d30073550 0x7f1d3019c6d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c000c00 tx=0x7f1d2c00f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:53.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.042+0000 7f1d267fc700 1 -- 172.21.15.133:0/1501745016 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c01d070 con 0x7f1d30073550 2023-11-27T15:20:53.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.042+0000 7f1d267fc700 1 -- 172.21.15.133:0/1501745016 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d2c005050 con 0x7f1d30073550 2023-11-27T15:20:53.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.042+0000 7f1d379ff700 1 -- 172.21.15.133:0/1501745016 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d30112280 con 0x7f1d30073550 2023-11-27T15:20:53.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.042+0000 7f1d267fc700 1 -- 172.21.15.133:0/1501745016 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c004ac0 con 0x7f1d30073550 2023-11-27T15:20:53.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.042+0000 7f1d379ff700 1 -- 172.21.15.133:0/1501745016 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d301120f0 con 0x7f1d30073550 2023-11-27T15:20:53.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.043+0000 7f1d379ff700 1 -- 172.21.15.133:0/1501745016 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d301023e0 con 0x7f1d30073550 2023-11-27T15:20:53.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.045+0000 7f1d267fc700 1 -- 172.21.15.133:0/1501745016 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1d2c0051c0 con 0x7f1d30073550 2023-11-27T15:20:53.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.046+0000 7f1d267fc700 1 --2- 172.21.15.133:0/1501745016 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1d1c0a11d0 0x7f1d1c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:53.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.046+0000 7f1d267fc700 1 -- 172.21.15.133:0/1501745016 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1d2c0cde80 con 0x7f1d30073550 2023-11-27T15:20:53.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.046+0000 7f1d34f9a700 1 --2- 172.21.15.133:0/1501745016 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1d1c0a11d0 0x7f1d1c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:53.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.047+0000 7f1d34f9a700 1 --2- 172.21.15.133:0/1501745016 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1d1c0a11d0 0x7f1d1c0a3680 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1d30074850 tx=0x7f1d20007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:53.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.048+0000 7f1d267fc700 1 -- 172.21.15.133:0/1501745016 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1d2c097410 con 0x7f1d30073550 2023-11-27T15:20:53.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:52 smithi177 ceph-mon[131504]: pgmap v283: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-27T15:20:53.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:52 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/4190547768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:20:53.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:52 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/536381762' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:20:53.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.246+0000 7f1d379ff700 1 -- 172.21.15.133:0/1501745016 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1d30061700 con 0x7f1d30073550 2023-11-27T15:20:53.251 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.247+0000 7f1d267fc700 1 -- 172.21.15.133:0/1501745016 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1d2c026020 con 0x7f1d30073550 2023-11-27T15:20:53.251 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-11-27T15:20:53.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.249+0000 7f1d379ff700 1 -- 172.21.15.133:0/1501745016 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1d1c0a11d0 msgr2=0x7f1d1c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:53.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.249+0000 7f1d379ff700 1 --2- 172.21.15.133:0/1501745016 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1d1c0a11d0 0x7f1d1c0a3680 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1d30074850 tx=0x7f1d20007400 comp rx=0 tx=0).stop 2023-11-27T15:20:53.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.249+0000 7f1d379ff700 1 -- 172.21.15.133:0/1501745016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d30073550 msgr2=0x7f1d3019c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:53.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.249+0000 7f1d379ff700 1 --2- 172.21.15.133:0/1501745016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d30073550 0x7f1d3019c6d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c000c00 tx=0x7f1d2c00f670 comp rx=0 tx=0).stop 2023-11-27T15:20:53.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.249+0000 7f1d379ff700 1 -- 172.21.15.133:0/1501745016 shutdown_connections 2023-11-27T15:20:53.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.249+0000 7f1d379ff700 1 --2- 172.21.15.133:0/1501745016 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1d1c0a11d0 0x7f1d1c0a3680 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.249+0000 7f1d379ff700 1 --2- 172.21.15.133:0/1501745016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d3010c890 0x7f1d301a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.249+0000 7f1d379ff700 1 --2- 172.21.15.133:0/1501745016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d30073550 0x7f1d3019c6d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.249+0000 7f1d379ff700 1 --2- 172.21.15.133:0/1501745016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d30074b30 0x7f1d3019cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.249+0000 7f1d379ff700 1 -- 172.21.15.133:0/1501745016 >> 172.21.15.133:0/1501745016 conn(0x7f1d300fb8a0 msgr2=0x7f1d300fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:53.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.250+0000 7f1d379ff700 1 -- 172.21.15.133:0/1501745016 shutdown_connections 2023-11-27T15:20:53.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.250+0000 7f1d379ff700 1 -- 172.21.15.133:0/1501745016 wait complete. 2023-11-27T15:20:53.272 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-27T15:20:53.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.467+0000 7f9f5d3ff700 1 -- 172.21.15.133:0/1989971695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f58108180 msgr2=0x7f9f5810a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:53.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.467+0000 7f9f5d3ff700 1 --2- 172.21.15.133:0/1989971695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f58108180 0x7f9f5810a600 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009f90 tx=0x7f9f5000b330 comp rx=0 tx=0).stop 2023-11-27T15:20:53.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.467+0000 7f9f5d3ff700 1 -- 172.21.15.133:0/1989971695 shutdown_connections 2023-11-27T15:20:53.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.467+0000 7f9f5d3ff700 1 --2- 172.21.15.133:0/1989971695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f58108180 0x7f9f5810a600 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.467+0000 7f9f5d3ff700 1 --2- 172.21.15.133:0/1989971695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f58074b30 0x7f9f58072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.467+0000 7f9f5d3ff700 1 --2- 172.21.15.133:0/1989971695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f58073550 0x7f9f580739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.467+0000 7f9f5d3ff700 1 -- 172.21.15.133:0/1989971695 >> 172.21.15.133:0/1989971695 conn(0x7f9f580fb8a0 msgr2=0x7f9f580fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:53.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.467+0000 7f9f5d3ff700 1 -- 172.21.15.133:0/1989971695 shutdown_connections 2023-11-27T15:20:53.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.467+0000 7f9f5d3ff700 1 -- 172.21.15.133:0/1989971695 wait complete. 2023-11-27T15:20:53.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.468+0000 7f9f5d3ff700 1 Processor -- start 2023-11-27T15:20:53.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.469+0000 7f9f5d3ff700 1 -- start start 2023-11-27T15:20:53.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.469+0000 7f9f5d3ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f58073550 0x7f9f58198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:53.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.469+0000 7f9f5d3ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f58074b30 0x7f9f581988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:53.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.469+0000 7f9f5d3ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f58108180 0x7f9f5819f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:53.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.469+0000 7f9f5d3ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5819ff10 con 0x7f9f58074b30 2023-11-27T15:20:53.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.469+0000 7f9f5d3ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f581a0050 con 0x7f9f58108180 2023-11-27T15:20:53.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.469+0000 7f9f5d3ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f581a01e0 con 0x7f9f58073550 2023-11-27T15:20:53.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.469+0000 7f9f56ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f58073550 0x7f9f58198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:53.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.469+0000 7f9f4ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f58074b30 0x7f9f581988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.469+0000 7f9f56ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f58073550 0x7f9f58198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:51100/0 (socket says 172.21.15.133:51100) 2023-11-27T15:20:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.469+0000 7f9f56ffd700 1 -- 172.21.15.133:0/3157303776 learned_addr learned my addr 172.21.15.133:0/3157303776 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.469+0000 7f9f577fe700 1 --2- 172.21.15.133:0/3157303776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f58108180 0x7f9f5819f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.470+0000 7f9f4ffff700 1 -- 172.21.15.133:0/3157303776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f58073550 msgr2=0x7f9f58198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.470+0000 7f9f4ffff700 1 --2- 172.21.15.133:0/3157303776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f58073550 0x7f9f58198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.470+0000 7f9f4ffff700 1 -- 172.21.15.133:0/3157303776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f58108180 msgr2=0x7f9f5819f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.470+0000 7f9f4ffff700 1 --2- 172.21.15.133:0/3157303776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f58108180 0x7f9f5819f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.470+0000 7f9f4ffff700 1 -- 172.21.15.133:0/3157303776 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f50009c40 con 0x7f9f58074b30 2023-11-27T15:20:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.470+0000 7f9f4ffff700 1 --2- 172.21.15.133:0/3157303776 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f58074b30 0x7f9f581988d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9f40005fd0 tx=0x7f9f4000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.470+0000 7f9f54ff9700 1 -- 172.21.15.133:0/3157303776 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40004220 con 0x7f9f58074b30 2023-11-27T15:20:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.470+0000 7f9f5d3ff700 1 -- 172.21.15.133:0/3157303776 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5810dea0 con 0x7f9f58074b30 2023-11-27T15:20:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.470+0000 7f9f54ff9700 1 -- 172.21.15.133:0/3157303776 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f40004380 con 0x7f9f58074b30 2023-11-27T15:20:53.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.470+0000 7f9f54ff9700 1 -- 172.21.15.133:0/3157303776 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40003be0 con 0x7f9f58074b30 2023-11-27T15:20:53.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.470+0000 7f9f5d3ff700 1 -- 172.21.15.133:0/3157303776 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5810dd10 con 0x7f9f58074b30 2023-11-27T15:20:53.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.471+0000 7f9f5d3ff700 1 -- 172.21.15.133:0/3157303776 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f58106f40 con 0x7f9f58074b30 2023-11-27T15:20:53.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.473+0000 7f9f54ff9700 1 -- 172.21.15.133:0/3157303776 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f9f40003d40 con 0x7f9f58074b30 2023-11-27T15:20:53.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.474+0000 7f9f54ff9700 1 --2- 172.21.15.133:0/3157303776 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9f440a1150 0x7f9f440a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:53.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.475+0000 7f9f54ff9700 1 -- 172.21.15.133:0/3157303776 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9f40014070 con 0x7f9f58074b30 2023-11-27T15:20:53.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.475+0000 7f9f56ffd700 1 --2- 172.21.15.133:0/3157303776 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9f440a1150 0x7f9f440a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:53.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.475+0000 7f9f56ffd700 1 --2- 172.21.15.133:0/3157303776 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9f440a1150 0x7f9f440a3600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9f48005950 tx=0x7f9f48009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:53.480 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.476+0000 7f9f54ff9700 1 -- 172.21.15.133:0/3157303776 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9f40096360 con 0x7f9f58074b30 2023-11-27T15:20:53.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.682+0000 7f9f5d3ff700 1 -- 172.21.15.133:0/3157303776 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9f58061700 con 0x7f9f58074b30 2023-11-27T15:20:53.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.683+0000 7f9f54ff9700 1 -- 172.21.15.133:0/3157303776 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9f40099980 con 0x7f9f58074b30 2023-11-27T15:20:53.687 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:20:53.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.685+0000 7f9f5d3ff700 1 -- 172.21.15.133:0/3157303776 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9f440a1150 msgr2=0x7f9f440a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:53.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.685+0000 7f9f5d3ff700 1 --2- 172.21.15.133:0/3157303776 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9f440a1150 0x7f9f440a3600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9f48005950 tx=0x7f9f48009260 comp rx=0 tx=0).stop 2023-11-27T15:20:53.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.685+0000 7f9f5d3ff700 1 -- 172.21.15.133:0/3157303776 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f58074b30 msgr2=0x7f9f581988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:53.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.685+0000 7f9f5d3ff700 1 --2- 172.21.15.133:0/3157303776 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f58074b30 0x7f9f581988d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9f40005fd0 tx=0x7f9f4000eca0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.685+0000 7f9f5d3ff700 1 -- 172.21.15.133:0/3157303776 shutdown_connections 2023-11-27T15:20:53.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.685+0000 7f9f5d3ff700 1 --2- 172.21.15.133:0/3157303776 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9f440a1150 0x7f9f440a3600 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.685+0000 7f9f5d3ff700 1 --2- 172.21.15.133:0/3157303776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f58108180 0x7f9f5819f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.685+0000 7f9f5d3ff700 1 --2- 172.21.15.133:0/3157303776 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f58073550 0x7f9f58198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.685+0000 7f9f5d3ff700 1 --2- 172.21.15.133:0/3157303776 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f58074b30 0x7f9f581988d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.685+0000 7f9f5d3ff700 1 -- 172.21.15.133:0/3157303776 >> 172.21.15.133:0/3157303776 conn(0x7f9f580fb8a0 msgr2=0x7f9f58104080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:53.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.685+0000 7f9f5d3ff700 1 -- 172.21.15.133:0/3157303776 shutdown_connections 2023-11-27T15:20:53.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.685+0000 7f9f5d3ff700 1 -- 172.21.15.133:0/3157303776 wait complete. 2023-11-27T15:20:53.707 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-27T15:20:53.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:53 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3466648048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:20:53.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:53 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1501745016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:20:53.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.904+0000 7f11ef3ff700 1 -- 172.21.15.133:0/2670141362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e80734f0 msgr2=0x7f11e8073960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:53.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.904+0000 7f11ef3ff700 1 --2- 172.21.15.133:0/2670141362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e80734f0 0x7f11e8073960 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f11d8009990 tx=0x7f11d8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.904+0000 7f11ef3ff700 1 -- 172.21.15.133:0/2670141362 shutdown_connections 2023-11-27T15:20:53.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.904+0000 7f11ef3ff700 1 --2- 172.21.15.133:0/2670141362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11e8108100 0x7f11e810a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.904+0000 7f11ef3ff700 1 --2- 172.21.15.133:0/2670141362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11e8074ad0 0x7f11e8072f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.904+0000 7f11ef3ff700 1 --2- 172.21.15.133:0/2670141362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e80734f0 0x7f11e8073960 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.904+0000 7f11ef3ff700 1 -- 172.21.15.133:0/2670141362 >> 172.21.15.133:0/2670141362 conn(0x7f11e80fb8b0 msgr2=0x7f11e80fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:53.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.904+0000 7f11ef3ff700 1 -- 172.21.15.133:0/2670141362 shutdown_connections 2023-11-27T15:20:53.909 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.904+0000 7f11ef3ff700 1 -- 172.21.15.133:0/2670141362 wait complete. 2023-11-27T15:20:53.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.905+0000 7f11ef3ff700 1 Processor -- start 2023-11-27T15:20:53.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.905+0000 7f11ef3ff700 1 -- start start 2023-11-27T15:20:53.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.905+0000 7f11ef3ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e80734f0 0x7f11e81981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:53.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.905+0000 7f11ef3ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11e8074ad0 0x7f11e8198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:53.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.905+0000 7f11ef3ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11e8108100 0x7f11e819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:53.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.905+0000 7f11ef3ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e819fd40 con 0x7f11e80734f0 2023-11-27T15:20:53.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.905+0000 7f11ef3ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e819fe80 con 0x7f11e8074ad0 2023-11-27T15:20:53.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.906+0000 7f11ef3ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e81a0010 con 0x7f11e8108100 2023-11-27T15:20:53.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.906+0000 7f11ed19b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e80734f0 0x7f11e81981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:53.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.906+0000 7f11ed19b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e80734f0 0x7f11e81981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39744/0 (socket says 172.21.15.133:39744) 2023-11-27T15:20:53.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.906+0000 7f11ed19b700 1 -- 172.21.15.133:0/3926966576 learned_addr learned my addr 172.21.15.133:0/3926966576 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:20:53.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.906+0000 7f11ec99a700 1 --2- 172.21.15.133:0/3926966576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11e8074ad0 0x7f11e8198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:53.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.906+0000 7f11ed99c700 1 --2- 172.21.15.133:0/3926966576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11e8108100 0x7f11e819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:53.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.906+0000 7f11ed19b700 1 -- 172.21.15.133:0/3926966576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11e8108100 msgr2=0x7f11e819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:53.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.906+0000 7f11ed19b700 1 --2- 172.21.15.133:0/3926966576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11e8108100 0x7f11e819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.906+0000 7f11ed19b700 1 -- 172.21.15.133:0/3926966576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11e8074ad0 msgr2=0x7f11e8198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:53.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.906+0000 7f11ed19b700 1 --2- 172.21.15.133:0/3926966576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11e8074ad0 0x7f11e8198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:53.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.906+0000 7f11ed19b700 1 -- 172.21.15.133:0/3926966576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e4009520 con 0x7f11e80734f0 2023-11-27T15:20:53.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.906+0000 7f11ec99a700 1 --2- 172.21.15.133:0/3926966576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11e8074ad0 0x7f11e8198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:20:53.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.906+0000 7f11ed19b700 1 --2- 172.21.15.133:0/3926966576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e80734f0 0x7f11e81981c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f11e400eb30 tx=0x7f11e400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:53.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.907+0000 7f11de7fc700 1 -- 172.21.15.133:0/3926966576 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e4009880 con 0x7f11e80734f0 2023-11-27T15:20:53.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.907+0000 7f11de7fc700 1 -- 172.21.15.133:0/3926966576 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f11e40044e0 con 0x7f11e80734f0 2023-11-27T15:20:53.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.907+0000 7f11ef3ff700 1 -- 172.21.15.133:0/3926966576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d80095f0 con 0x7f11e80734f0 2023-11-27T15:20:53.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.907+0000 7f11de7fc700 1 -- 172.21.15.133:0/3926966576 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e4003e90 con 0x7f11e80734f0 2023-11-27T15:20:53.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.907+0000 7f11ef3ff700 1 -- 172.21.15.133:0/3926966576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e810de10 con 0x7f11e80734f0 2023-11-27T15:20:53.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.908+0000 7f11ef3ff700 1 -- 172.21.15.133:0/3926966576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e8106840 con 0x7f11e80734f0 2023-11-27T15:20:53.913 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.910+0000 7f11de7fc700 1 -- 172.21.15.133:0/3926966576 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f11e4004000 con 0x7f11e80734f0 2023-11-27T15:20:53.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.911+0000 7f11de7fc700 1 --2- 172.21.15.133:0/3926966576 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f11d40a11d0 0x7f11d40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:20:53.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.911+0000 7f11de7fc700 1 -- 172.21.15.133:0/3926966576 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f11e4014070 con 0x7f11e80734f0 2023-11-27T15:20:53.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.911+0000 7f11ec99a700 1 --2- 172.21.15.133:0/3926966576 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f11d40a11d0 0x7f11d40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:20:53.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.912+0000 7f11ec99a700 1 --2- 172.21.15.133:0/3926966576 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f11d40a11d0 0x7f11d40a3680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f11d800b2e0 tx=0x7f11d8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:20:53.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:53.915+0000 7f11de7fc700 1 -- 172.21.15.133:0/3926966576 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f11e4096490 con 0x7f11e80734f0 2023-11-27T15:20:54.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:53 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3466648048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:20:54.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:53 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1501745016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:20:54.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:53 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3466648048' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:20:54.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:53 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1501745016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:20:54.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:54.126+0000 7f11ef3ff700 1 -- 172.21.15.133:0/3926966576 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f11e802cfe0 con 0x7f11e80734f0 2023-11-27T15:20:54.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:54.127+0000 7f11de7fc700 1 -- 172.21.15.133:0/3926966576 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+602 (secure 0 0 0) 0x7f11e4099ab0 con 0x7f11e80734f0 2023-11-27T15:20:54.130 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6800/2374615220 2023-11-28T15:14:01.628185+0000 2023-11-27T15:20:54.131 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/1594111147 2023-11-28T15:14:01.628185+0000 2023-11-27T15:20:54.131 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6801/3149426454 2023-11-28T15:13:35.034149+0000 2023-11-27T15:20:54.131 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6800/3149426454 2023-11-28T15:13:35.034149+0000 2023-11-27T15:20:54.131 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/2898896137 2023-11-28T15:14:01.628185+0000 2023-11-27T15:20:54.131 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/1816614695 2023-11-28T15:13:35.034149+0000 2023-11-27T15:20:54.131 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/2624811615 2023-11-28T15:13:35.034149+0000 2023-11-27T15:20:54.131 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6801/2374615220 2023-11-28T15:14:01.628185+0000 2023-11-27T15:20:54.131 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/4208867747 2023-11-28T15:14:01.628185+0000 2023-11-27T15:20:54.131 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/4192331415 2023-11-28T15:13:35.034149+0000 2023-11-27T15:20:54.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:54.129+0000 7f11ef3ff700 1 -- 172.21.15.133:0/3926966576 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f11d40a11d0 msgr2=0x7f11d40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:54.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:54.129+0000 7f11ef3ff700 1 --2- 172.21.15.133:0/3926966576 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f11d40a11d0 0x7f11d40a3680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f11d800b2e0 tx=0x7f11d8005fb0 comp rx=0 tx=0).stop 2023-11-27T15:20:54.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:54.129+0000 7f11ef3ff700 1 -- 172.21.15.133:0/3926966576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e80734f0 msgr2=0x7f11e81981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:20:54.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:54.129+0000 7f11ef3ff700 1 --2- 172.21.15.133:0/3926966576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e80734f0 0x7f11e81981c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f11e400eb30 tx=0x7f11e400ee40 comp rx=0 tx=0).stop 2023-11-27T15:20:54.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:54.129+0000 7f11ef3ff700 1 -- 172.21.15.133:0/3926966576 shutdown_connections 2023-11-27T15:20:54.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:54.129+0000 7f11ef3ff700 1 --2- 172.21.15.133:0/3926966576 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f11d40a11d0 0x7f11d40a3680 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:54.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:54.129+0000 7f11ef3ff700 1 --2- 172.21.15.133:0/3926966576 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f11e8108100 0x7f11e819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:54.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:54.129+0000 7f11ef3ff700 1 --2- 172.21.15.133:0/3926966576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11e80734f0 0x7f11e81981c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:54.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:54.129+0000 7f11ef3ff700 1 --2- 172.21.15.133:0/3926966576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11e8074ad0 0x7f11e8198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:20:54.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:54.129+0000 7f11ef3ff700 1 -- 172.21.15.133:0/3926966576 >> 172.21.15.133:0/3926966576 conn(0x7f11e80fb8b0 msgr2=0x7f11e80fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:20:54.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:54.130+0000 7f11ef3ff700 1 -- 172.21.15.133:0/3926966576 shutdown_connections 2023-11-27T15:20:54.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:20:54.130+0000 7f11ef3ff700 1 -- 172.21.15.133:0/3926966576 wait complete. 2023-11-27T15:20:54.135 INFO:teuthology.orchestra.run.smithi133.stderr:listed 10 entries 2023-11-27T15:20:54.151 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi133.front.sepia.ceph.com... 2023-11-27T15:20:54.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:20:54.152 DEBUG:teuthology.orchestra.run.smithi133:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-11-27T15:20:54.217 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi133.stderr:ceph-fuse[134938]: fuse finished with error 0 and tester_r 0 2023-11-27T15:20:54.221 INFO:teuthology.orchestra.run:waiting for 300 2023-11-27T15:20:54.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:54 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3157303776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:20:54.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:54 smithi133 ceph-mon[97860]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-27T15:20:54.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:54 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3926966576' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-27T15:20:55.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:54 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3157303776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:20:55.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:54 smithi177 ceph-mon[131504]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-27T15:20:55.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:54 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3926966576' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-27T15:20:55.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:54 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3157303776' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:20:55.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:54 smithi174 ceph-mon[131110]: pgmap v284: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-27T15:20:55.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:54 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3926966576' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-27T15:20:57.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:56 smithi133 ceph-mon[97860]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-27T15:20:57.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:56 smithi177 ceph-mon[131504]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-27T15:20:57.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:56 smithi174 ceph-mon[131110]: pgmap v285: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-11-27T15:20:58.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:20:58 smithi174 ceph-mon[131110]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-11-27T15:20:59.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:20:58 smithi133 ceph-mon[97860]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-11-27T15:20:59.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:20:58 smithi177 ceph-mon[131504]: pgmap v286: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-11-27T15:21:00.322 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2023-11-27T15:21:00.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:21:00.323 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-11-27T15:21:00.388 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:21:00.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.585+0000 7f7ae49ff700 1 -- 172.21.15.133:0/2359877975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ae01002b0 msgr2=0x7f7ae01006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:00.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.585+0000 7f7ae49ff700 1 --2- 172.21.15.133:0/2359877975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ae01002b0 0x7f7ae01006c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0009990 tx=0x7f7ad0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:21:00.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.585+0000 7f7ae49ff700 1 -- 172.21.15.133:0/2359877975 shutdown_connections 2023-11-27T15:21:00.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.585+0000 7f7ae49ff700 1 --2- 172.21.15.133:0/2359877975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ae0101e30 0x7f7ae01086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:00.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.585+0000 7f7ae49ff700 1 --2- 172.21.15.133:0/2359877975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ae01002b0 0x7f7ae01006c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:00.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.585+0000 7f7ae49ff700 1 --2- 172.21.15.133:0/2359877975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ae01014a0 0x7f7ae01018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:00.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.585+0000 7f7ae49ff700 1 -- 172.21.15.133:0/2359877975 >> 172.21.15.133:0/2359877975 conn(0x7f7ae00fb890 msgr2=0x7f7ae00fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:00.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.585+0000 7f7ae49ff700 1 -- 172.21.15.133:0/2359877975 shutdown_connections 2023-11-27T15:21:00.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.585+0000 7f7ae49ff700 1 -- 172.21.15.133:0/2359877975 wait complete. 2023-11-27T15:21:00.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.586+0000 7f7ae49ff700 1 Processor -- start 2023-11-27T15:21:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.586+0000 7f7ae49ff700 1 -- start start 2023-11-27T15:21:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.586+0000 7f7ae49ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ae01002b0 0x7f7ae0196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.586+0000 7f7ae49ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ae01014a0 0x7f7ae0196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.587+0000 7f7ae49ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ae0101e30 0x7f7ae019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.587+0000 7f7ae49ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae019dba0 con 0x7f7ae01002b0 2023-11-27T15:21:00.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.587+0000 7f7ae49ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae019dce0 con 0x7f7ae0101e30 2023-11-27T15:21:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.587+0000 7f7ae49ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae019de70 con 0x7f7ae01014a0 2023-11-27T15:21:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.587+0000 7f7ade59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ae01002b0 0x7f7ae0196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.587+0000 7f7ade59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ae01002b0 0x7f7ae0196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39766/0 (socket says 172.21.15.133:39766) 2023-11-27T15:21:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.587+0000 7f7ade59c700 1 -- 172.21.15.133:0/2377706000 learned_addr learned my addr 172.21.15.133:0/2377706000 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.587+0000 7f7ad7fff700 1 --2- 172.21.15.133:0/2377706000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ae01014a0 0x7f7ae0196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.587+0000 7f7aded9d700 1 --2- 172.21.15.133:0/2377706000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ae0101e30 0x7f7ae019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.587+0000 7f7ade59c700 1 -- 172.21.15.133:0/2377706000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ae01014a0 msgr2=0x7f7ae0196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:00.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.587+0000 7f7ade59c700 1 --2- 172.21.15.133:0/2377706000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ae01014a0 0x7f7ae0196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.587+0000 7f7ade59c700 1 -- 172.21.15.133:0/2377706000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ae0101e30 msgr2=0x7f7ae019d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.587+0000 7f7ade59c700 1 --2- 172.21.15.133:0/2377706000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ae0101e30 0x7f7ae019d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.587+0000 7f7ade59c700 1 -- 172.21.15.133:0/2377706000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac8009520 con 0x7f7ae01002b0 2023-11-27T15:21:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.587+0000 7f7ade59c700 1 --2- 172.21.15.133:0/2377706000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ae01002b0 0x7f7ae0196020 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7ad000b620 tx=0x7f7ad0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.588+0000 7f7ad77fe700 1 -- 172.21.15.133:0/2377706000 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad001d070 con 0x7f7ae01002b0 2023-11-27T15:21:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.588+0000 7f7ae49ff700 1 -- 172.21.15.133:0/2377706000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad00095f0 con 0x7f7ae01002b0 2023-11-27T15:21:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.588+0000 7f7ad77fe700 1 -- 172.21.15.133:0/2377706000 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ad0005030 con 0x7f7ae01002b0 2023-11-27T15:21:00.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.588+0000 7f7ad77fe700 1 -- 172.21.15.133:0/2377706000 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0004b80 con 0x7f7ae01002b0 2023-11-27T15:21:00.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.588+0000 7f7ae49ff700 1 -- 172.21.15.133:0/2377706000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae010bc10 con 0x7f7ae01002b0 2023-11-27T15:21:00.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.589+0000 7f7ae49ff700 1 -- 172.21.15.133:0/2377706000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae010bda0 con 0x7f7ae01002b0 2023-11-27T15:21:00.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.591+0000 7f7ad77fe700 1 -- 172.21.15.133:0/2377706000 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7ad00051a0 con 0x7f7ae01002b0 2023-11-27T15:21:00.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.592+0000 7f7ad77fe700 1 --2- 172.21.15.133:0/2377706000 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7acc0a1150 0x7f7acc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:00.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.592+0000 7f7ad77fe700 1 -- 172.21.15.133:0/2377706000 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7ad00cdb60 con 0x7f7ae01002b0 2023-11-27T15:21:00.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.592+0000 7f7ad7fff700 1 --2- 172.21.15.133:0/2377706000 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7acc0a1150 0x7f7acc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:00.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.593+0000 7f7ad7fff700 1 --2- 172.21.15.133:0/2377706000 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7acc0a1150 0x7f7acc0a3600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009df0 tx=0x7f7ac8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:00.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.596+0000 7f7ad77fe700 1 -- 172.21.15.133:0/2377706000 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7ad00970f0 con 0x7f7ae01002b0 2023-11-27T15:21:00.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.795+0000 7f7ae49ff700 1 -- 172.21.15.133:0/2377706000 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ae0061700 con 0x7f7ae01002b0 2023-11-27T15:21:00.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.797+0000 7f7ad77fe700 1 -- 172.21.15.133:0/2377706000 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23996 (secure 0 0 0) 0x7f7ad0004ce0 con 0x7f7ae01002b0 2023-11-27T15:21:00.802 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:21:00.802 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:20:38.147036+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:21:00.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.799+0000 7f7ae49ff700 1 -- 172.21.15.133:0/2377706000 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7acc0a1150 msgr2=0x7f7acc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:00.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.799+0000 7f7ae49ff700 1 --2- 172.21.15.133:0/2377706000 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7acc0a1150 0x7f7acc0a3600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009df0 tx=0x7f7ac8009260 comp rx=0 tx=0).stop 2023-11-27T15:21:00.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.799+0000 7f7ae49ff700 1 -- 172.21.15.133:0/2377706000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ae01002b0 msgr2=0x7f7ae0196020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:00.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.799+0000 7f7ae49ff700 1 --2- 172.21.15.133:0/2377706000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ae01002b0 0x7f7ae0196020 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7ad000b620 tx=0x7f7ad0004730 comp rx=0 tx=0).stop 2023-11-27T15:21:00.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.799+0000 7f7ae49ff700 1 -- 172.21.15.133:0/2377706000 shutdown_connections 2023-11-27T15:21:00.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.799+0000 7f7ae49ff700 1 --2- 172.21.15.133:0/2377706000 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7acc0a1150 0x7f7acc0a3600 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:00.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.800+0000 7f7ae49ff700 1 --2- 172.21.15.133:0/2377706000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ae0101e30 0x7f7ae019d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:00.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.800+0000 7f7ae49ff700 1 --2- 172.21.15.133:0/2377706000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ae01002b0 0x7f7ae0196020 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:00.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.800+0000 7f7ae49ff700 1 --2- 172.21.15.133:0/2377706000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ae01014a0 0x7f7ae0196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:00.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.800+0000 7f7ae49ff700 1 -- 172.21.15.133:0/2377706000 >> 172.21.15.133:0/2377706000 conn(0x7f7ae00fb890 msgr2=0x7f7ae0104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:00.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.800+0000 7f7ae49ff700 1 -- 172.21.15.133:0/2377706000 shutdown_connections 2023-11-27T15:21:00.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:00.800+0000 7f7ae49ff700 1 -- 172.21.15.133:0/2377706000 wait complete. 2023-11-27T15:21:00.823 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:21:01.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.017+0000 7fefbebff700 1 -- 172.21.15.133:0/1585395290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefb8101480 msgr2=0x7fefb81018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:01.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.017+0000 7fefbebff700 1 --2- 172.21.15.133:0/1585395290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefb8101480 0x7fefb81018d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fefa40098f0 tx=0x7fefa4009c00 comp rx=0 tx=0).stop 2023-11-27T15:21:01.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.017+0000 7fefbebff700 1 -- 172.21.15.133:0/1585395290 shutdown_connections 2023-11-27T15:21:01.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.017+0000 7fefbebff700 1 --2- 172.21.15.133:0/1585395290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb8101e10 0x7fefb81086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:01.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.017+0000 7fefbebff700 1 --2- 172.21.15.133:0/1585395290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefb8100290 0x7fefb81006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:01.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.017+0000 7fefbebff700 1 --2- 172.21.15.133:0/1585395290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefb8101480 0x7fefb81018d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:01.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.017+0000 7fefbebff700 1 -- 172.21.15.133:0/1585395290 >> 172.21.15.133:0/1585395290 conn(0x7fefb80fb870 msgr2=0x7fefb80fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:01.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.017+0000 7fefbebff700 1 -- 172.21.15.133:0/1585395290 shutdown_connections 2023-11-27T15:21:01.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.017+0000 7fefbebff700 1 -- 172.21.15.133:0/1585395290 wait complete. 2023-11-27T15:21:01.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.018+0000 7fefbebff700 1 Processor -- start 2023-11-27T15:21:01.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.018+0000 7fefbebff700 1 -- start start 2023-11-27T15:21:01.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.018+0000 7fefbebff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefb8100290 0x7fefb81981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:01.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.018+0000 7fefbebff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefb8101480 0x7fefb8198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:01.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.018+0000 7fefbebff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb8101e10 0x7fefb819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:01.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.018+0000 7fefbebff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb819fd60 con 0x7fefb8101e10 2023-11-27T15:21:01.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.018+0000 7fefbebff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb819fea0 con 0x7fefb8100290 2023-11-27T15:21:01.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.019+0000 7fefbebff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb81a0030 con 0x7fefb8101480 2023-11-27T15:21:01.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.019+0000 7fefbd19c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb8101e10 0x7fefb819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:01.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.019+0000 7fefbc99b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefb8100290 0x7fefb81981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:01.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.019+0000 7fefbc99b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefb8100290 0x7fefb81981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:56576/0 (socket says 172.21.15.133:56576) 2023-11-27T15:21:01.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.019+0000 7fefbc99b700 1 -- 172.21.15.133:0/2472468409 learned_addr learned my addr 172.21.15.133:0/2472468409 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:01.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.019+0000 7fefb7fff700 1 --2- 172.21.15.133:0/2472468409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefb8101480 0x7fefb8198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:01.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.019+0000 7fefbc99b700 1 -- 172.21.15.133:0/2472468409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefb8101480 msgr2=0x7fefb8198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:01.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.019+0000 7fefbc99b700 1 --2- 172.21.15.133:0/2472468409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefb8101480 0x7fefb8198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:01.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.019+0000 7fefbc99b700 1 -- 172.21.15.133:0/2472468409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb8101e10 msgr2=0x7fefb819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:01.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.019+0000 7fefbc99b700 1 --2- 172.21.15.133:0/2472468409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb8101e10 0x7fefb819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:01.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.019+0000 7fefbc99b700 1 -- 172.21.15.133:0/2472468409 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa4009520 con 0x7fefb8100290 2023-11-27T15:21:01.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.019+0000 7fefbc99b700 1 --2- 172.21.15.133:0/2472468409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefb8100290 0x7fefb81981e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fefac00e9f0 tx=0x7fefac00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:01.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.020+0000 7fefb5ffb700 1 -- 172.21.15.133:0/2472468409 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac004220 con 0x7fefb8100290 2023-11-27T15:21:01.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.020+0000 7fefb5ffb700 1 -- 172.21.15.133:0/2472468409 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefac004380 con 0x7fefb8100290 2023-11-27T15:21:01.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.020+0000 7fefbebff700 1 -- 172.21.15.133:0/2472468409 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb810bc50 con 0x7fefb8100290 2023-11-27T15:21:01.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.020+0000 7fefb5ffb700 1 -- 172.21.15.133:0/2472468409 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac003be0 con 0x7fefb8100290 2023-11-27T15:21:01.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.020+0000 7fefbebff700 1 -- 172.21.15.133:0/2472468409 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb810bac0 con 0x7fefb8100290 2023-11-27T15:21:01.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.021+0000 7fefbebff700 1 -- 172.21.15.133:0/2472468409 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb81928e0 con 0x7fefb8100290 2023-11-27T15:21:01.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.023+0000 7fefb5ffb700 1 -- 172.21.15.133:0/2472468409 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fefac003d40 con 0x7fefb8100290 2023-11-27T15:21:01.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:00 smithi133 ceph-mon[97860]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-11-27T15:21:01.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.024+0000 7fefb5ffb700 1 --2- 172.21.15.133:0/2472468409 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fefa80a1150 0x7fefa80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:01.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.024+0000 7fefb5ffb700 1 -- 172.21.15.133:0/2472468409 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fefac014070 con 0x7fefb8100290 2023-11-27T15:21:01.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.024+0000 7fefb7fff700 1 --2- 172.21.15.133:0/2472468409 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fefa80a1150 0x7fefa80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:01.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.025+0000 7fefb7fff700 1 --2- 172.21.15.133:0/2472468409 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fefa80a1150 0x7fefa80a3600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fefa4005e90 tx=0x7fefa4019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:01.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.026+0000 7fefb5ffb700 1 -- 172.21.15.133:0/2472468409 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fefac096460 con 0x7fefb8100290 2023-11-27T15:21:01.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:00 smithi177 ceph-mon[131504]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-11-27T15:21:01.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:00 smithi174 ceph-mon[131110]: pgmap v287: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-11-27T15:21:01.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.223+0000 7fefbebff700 1 -- 172.21.15.133:0/2472468409 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefb802cfe0 con 0x7fefb8100290 2023-11-27T15:21:01.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.226+0000 7fefb5ffb700 1 -- 172.21.15.133:0/2472468409 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+23996 (secure 0 0 0) 0x7fefac099a80 con 0x7fefb8100290 2023-11-27T15:21:01.229 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:21:01.230 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":88,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:20:38.147036+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:21:01.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.228+0000 7fefbebff700 1 -- 172.21.15.133:0/2472468409 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fefa80a1150 msgr2=0x7fefa80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:01.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.228+0000 7fefbebff700 1 --2- 172.21.15.133:0/2472468409 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fefa80a1150 0x7fefa80a3600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fefa4005e90 tx=0x7fefa4019040 comp rx=0 tx=0).stop 2023-11-27T15:21:01.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.228+0000 7fefbebff700 1 -- 172.21.15.133:0/2472468409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefb8100290 msgr2=0x7fefb81981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:01.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.228+0000 7fefbebff700 1 --2- 172.21.15.133:0/2472468409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefb8100290 0x7fefb81981e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fefac00e9f0 tx=0x7fefac00ed00 comp rx=0 tx=0).stop 2023-11-27T15:21:01.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.228+0000 7fefbebff700 1 -- 172.21.15.133:0/2472468409 shutdown_connections 2023-11-27T15:21:01.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.228+0000 7fefbebff700 1 --2- 172.21.15.133:0/2472468409 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fefa80a1150 0x7fefa80a3600 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:01.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.228+0000 7fefbebff700 1 --2- 172.21.15.133:0/2472468409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fefb8101e10 0x7fefb819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:01.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.228+0000 7fefbebff700 1 --2- 172.21.15.133:0/2472468409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fefb8100290 0x7fefb81981e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:01.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.228+0000 7fefbebff700 1 --2- 172.21.15.133:0/2472468409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fefb8101480 0x7fefb8198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:01.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.228+0000 7fefbebff700 1 -- 172.21.15.133:0/2472468409 >> 172.21.15.133:0/2472468409 conn(0x7fefb80fb870 msgr2=0x7fefb8106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:01.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.228+0000 7fefbebff700 1 -- 172.21.15.133:0/2472468409 shutdown_connections 2023-11-27T15:21:01.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.228+0000 7fefbebff700 1 -- 172.21.15.133:0/2472468409 wait complete. 2023-11-27T15:21:01.251 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-27T15:21:01.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.446+0000 7f347adff700 1 -- 172.21.15.133:0/3562275771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34741014b0 msgr2=0x7f3474101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:01.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.446+0000 7f347adff700 1 --2- 172.21.15.133:0/3562275771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34741014b0 0x7f3474101900 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3460009990 tx=0x7f3460009ca0 comp rx=0 tx=0).stop 2023-11-27T15:21:01.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.446+0000 7f347adff700 1 -- 172.21.15.133:0/3562275771 shutdown_connections 2023-11-27T15:21:01.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.446+0000 7f347adff700 1 --2- 172.21.15.133:0/3562275771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3474101e40 0x7f3474108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:01.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.446+0000 7f347adff700 1 --2- 172.21.15.133:0/3562275771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34741002c0 0x7f34741006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:01.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.446+0000 7f347adff700 1 --2- 172.21.15.133:0/3562275771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34741014b0 0x7f3474101900 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:01.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.446+0000 7f347adff700 1 -- 172.21.15.133:0/3562275771 >> 172.21.15.133:0/3562275771 conn(0x7f34740fb8a0 msgr2=0x7f34740fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:01.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.446+0000 7f347adff700 1 -- 172.21.15.133:0/3562275771 shutdown_connections 2023-11-27T15:21:01.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.446+0000 7f347adff700 1 -- 172.21.15.133:0/3562275771 wait complete. 2023-11-27T15:21:01.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.447+0000 7f347adff700 1 Processor -- start 2023-11-27T15:21:01.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.447+0000 7f347adff700 1 -- start start 2023-11-27T15:21:01.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.447+0000 7f347adff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34741002c0 0x7f3474198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:01.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.447+0000 7f347adff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34741014b0 0x7f3474198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:01.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.447+0000 7f347adff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3474101e40 0x7f347419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:01.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.447+0000 7f347adff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347419fd90 con 0x7f34741014b0 2023-11-27T15:21:01.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.447+0000 7f347adff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347419fed0 con 0x7f34741002c0 2023-11-27T15:21:01.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.447+0000 7f347adff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34741a0060 con 0x7f3474101e40 2023-11-27T15:21:01.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.448+0000 7f3473fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34741014b0 0x7f3474198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:01.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.448+0000 7f3473fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34741014b0 0x7f3474198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39816/0 (socket says 172.21.15.133:39816) 2023-11-27T15:21:01.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.448+0000 7f3473fff700 1 -- 172.21.15.133:0/2968588761 learned_addr learned my addr 172.21.15.133:0/2968588761 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:01.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.448+0000 7f3478b9b700 1 --2- 172.21.15.133:0/2968588761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34741002c0 0x7f3474198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:01.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.448+0000 7f347939c700 1 --2- 172.21.15.133:0/2968588761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3474101e40 0x7f347419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:01.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.448+0000 7f3473fff700 1 -- 172.21.15.133:0/2968588761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3474101e40 msgr2=0x7f347419f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:01.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.448+0000 7f3473fff700 1 --2- 172.21.15.133:0/2968588761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3474101e40 0x7f347419f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:01.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.448+0000 7f3473fff700 1 -- 172.21.15.133:0/2968588761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34741002c0 msgr2=0x7f3474198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:01.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.448+0000 7f3473fff700 1 --2- 172.21.15.133:0/2968588761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34741002c0 0x7f3474198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:01.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.448+0000 7f3473fff700 1 -- 172.21.15.133:0/2968588761 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34600095f0 con 0x7f34741014b0 2023-11-27T15:21:01.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.448+0000 7f3473fff700 1 --2- 172.21.15.133:0/2968588761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34741014b0 0x7f3474198750 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f346000b680 tx=0x7f34600057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:01.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.448+0000 7f3471ffb700 1 -- 172.21.15.133:0/2968588761 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346001d070 con 0x7f34741014b0 2023-11-27T15:21:01.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.449+0000 7f347adff700 1 -- 172.21.15.133:0/2968588761 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f347410bc20 con 0x7f34741014b0 2023-11-27T15:21:01.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.449+0000 7f3471ffb700 1 -- 172.21.15.133:0/2968588761 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3460022470 con 0x7f34741014b0 2023-11-27T15:21:01.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.449+0000 7f3471ffb700 1 -- 172.21.15.133:0/2968588761 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3460017650 con 0x7f34741014b0 2023-11-27T15:21:01.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.449+0000 7f347adff700 1 -- 172.21.15.133:0/2968588761 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f347410ba90 con 0x7f34741014b0 2023-11-27T15:21:01.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.450+0000 7f347adff700 1 -- 172.21.15.133:0/2968588761 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3474192980 con 0x7f34741014b0 2023-11-27T15:21:01.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.452+0000 7f3471ffb700 1 -- 172.21.15.133:0/2968588761 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f34600225e0 con 0x7f34741014b0 2023-11-27T15:21:01.456 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.453+0000 7f3471ffb700 1 --2- 172.21.15.133:0/2968588761 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f34640a1150 0x7f34640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:01.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.453+0000 7f3471ffb700 1 -- 172.21.15.133:0/2968588761 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f34600cdfe0 con 0x7f34741014b0 2023-11-27T15:21:01.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.453+0000 7f3478b9b700 1 --2- 172.21.15.133:0/2968588761 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f34640a1150 0x7f34640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:01.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.454+0000 7f3478b9b700 1 --2- 172.21.15.133:0/2968588761 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f34640a1150 0x7f34640a3600 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3474101310 tx=0x7f3468009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:01.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.457+0000 7f3471ffb700 1 -- 172.21.15.133:0/2968588761 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f34600975d0 con 0x7f34741014b0 2023-11-27T15:21:01.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.708+0000 7f347adff700 1 -- 172.21.15.133:0/2968588761 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f347402cfe0 con 0x7f34741014b0 2023-11-27T15:21:01.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.780+0000 7f3471ffb700 1 -- 172.21.15.133:0/2968588761 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f346009abf0 con 0x7f34741014b0 2023-11-27T15:21:01.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.784+0000 7f347adff700 1 -- 172.21.15.133:0/2968588761 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f34640a1150 msgr2=0x7f34640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:01.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.784+0000 7f347adff700 1 --2- 172.21.15.133:0/2968588761 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f34640a1150 0x7f34640a3600 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3474101310 tx=0x7f3468009260 comp rx=0 tx=0).stop 2023-11-27T15:21:01.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.784+0000 7f347adff700 1 -- 172.21.15.133:0/2968588761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34741014b0 msgr2=0x7f3474198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:01.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.784+0000 7f347adff700 1 --2- 172.21.15.133:0/2968588761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34741014b0 0x7f3474198750 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f346000b680 tx=0x7f34600057b0 comp rx=0 tx=0).stop 2023-11-27T15:21:01.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.785+0000 7f347adff700 1 -- 172.21.15.133:0/2968588761 shutdown_connections 2023-11-27T15:21:01.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.785+0000 7f347adff700 1 --2- 172.21.15.133:0/2968588761 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f34640a1150 0x7f34640a3600 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:01.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.785+0000 7f347adff700 1 --2- 172.21.15.133:0/2968588761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3474101e40 0x7f347419f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:01.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.785+0000 7f347adff700 1 --2- 172.21.15.133:0/2968588761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34741002c0 0x7f3474198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:01.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.785+0000 7f347adff700 1 --2- 172.21.15.133:0/2968588761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f34741014b0 0x7f3474198750 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:01.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.785+0000 7f347adff700 1 -- 172.21.15.133:0/2968588761 >> 172.21.15.133:0/2968588761 conn(0x7f34740fb8a0 msgr2=0x7f3474106d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:01.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.785+0000 7f347adff700 1 -- 172.21.15.133:0/2968588761 shutdown_connections 2023-11-27T15:21:01.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:01.785+0000 7f347adff700 1 -- 172.21.15.133:0/2968588761 wait complete. 2023-11-27T15:21:01.810 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2023-11-27T15:21:01.811 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-11-27T15:21:02.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.007+0000 7fedcdfff700 1 -- 172.21.15.133:0/783835289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedc81002c0 msgr2=0x7fedc81006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:02.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.007+0000 7fedcdfff700 1 --2- 172.21.15.133:0/783835289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedc81002c0 0x7fedc81006d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fedb80098f0 tx=0x7fedb8009c00 comp rx=0 tx=0).stop 2023-11-27T15:21:02.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.007+0000 7fedcdfff700 1 -- 172.21.15.133:0/783835289 shutdown_connections 2023-11-27T15:21:02.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.007+0000 7fedcdfff700 1 --2- 172.21.15.133:0/783835289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedc8101e40 0x7fedc8108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.007+0000 7fedcdfff700 1 --2- 172.21.15.133:0/783835289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedc81002c0 0x7fedc81006d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.007+0000 7fedcdfff700 1 --2- 172.21.15.133:0/783835289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedc81014b0 0x7fedc8101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.007+0000 7fedcdfff700 1 -- 172.21.15.133:0/783835289 >> 172.21.15.133:0/783835289 conn(0x7fedc80fb8a0 msgr2=0x7fedc80fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:02.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.007+0000 7fedcdfff700 1 -- 172.21.15.133:0/783835289 shutdown_connections 2023-11-27T15:21:02.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.008+0000 7fedcdfff700 1 -- 172.21.15.133:0/783835289 wait complete. 2023-11-27T15:21:02.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.008+0000 7fedcdfff700 1 Processor -- start 2023-11-27T15:21:02.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.009+0000 7fedcdfff700 1 -- start start 2023-11-27T15:21:02.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.009+0000 7fedcdfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedc81002c0 0x7fedc81982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:02.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.009+0000 7fedcdfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedc81014b0 0x7fedc8198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:02.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.009+0000 7fedcdfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedc8101e40 0x7fedc819f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:02.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.009+0000 7fedcdfff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc819fe40 con 0x7fedc81014b0 2023-11-27T15:21:02.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.009+0000 7fedcdfff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc819ff80 con 0x7fedc81002c0 2023-11-27T15:21:02.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.009+0000 7fedcdfff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc81a0110 con 0x7fedc8101e40 2023-11-27T15:21:02.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.009+0000 7fedc6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedc81014b0 0x7fedc8198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:02.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.009+0000 7fedc77fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedc81002c0 0x7fedc81982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:02.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.009+0000 7fedc7fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedc8101e40 0x7fedc819f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:02.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.009+0000 7fedc6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedc81014b0 0x7fedc8198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39828/0 (socket says 172.21.15.133:39828) 2023-11-27T15:21:02.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.009+0000 7fedc6ffd700 1 -- 172.21.15.133:0/1677235066 learned_addr learned my addr 172.21.15.133:0/1677235066 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:02.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.010+0000 7fedc6ffd700 1 -- 172.21.15.133:0/1677235066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedc8101e40 msgr2=0x7fedc819f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:02.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.010+0000 7fedc6ffd700 1 --2- 172.21.15.133:0/1677235066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedc8101e40 0x7fedc819f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.010+0000 7fedc6ffd700 1 -- 172.21.15.133:0/1677235066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedc81002c0 msgr2=0x7fedc81982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:02.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.010+0000 7fedc6ffd700 1 --2- 172.21.15.133:0/1677235066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedc81002c0 0x7fedc81982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.010+0000 7fedc6ffd700 1 -- 172.21.15.133:0/1677235066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb8009520 con 0x7fedc81014b0 2023-11-27T15:21:02.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.010+0000 7fedc6ffd700 1 --2- 172.21.15.133:0/1677235066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedc81014b0 0x7fedc8198800 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00e9a0 tx=0x7fedbc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:02.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.010+0000 7fedc4ff9700 1 -- 172.21.15.133:0/1677235066 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc009790 con 0x7fedc81014b0 2023-11-27T15:21:02.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.010+0000 7fedc4ff9700 1 -- 172.21.15.133:0/1677235066 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedbc010430 con 0x7fedc81014b0 2023-11-27T15:21:02.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.010+0000 7fedcdfff700 1 -- 172.21.15.133:0/1677235066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc810bc90 con 0x7fedc81014b0 2023-11-27T15:21:02.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.010+0000 7fedc4ff9700 1 -- 172.21.15.133:0/1677235066 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc018680 con 0x7fedc81014b0 2023-11-27T15:21:02.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.010+0000 7fedcdfff700 1 -- 172.21.15.133:0/1677235066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc810bb00 con 0x7fedc81014b0 2023-11-27T15:21:02.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.011+0000 7fedcdfff700 1 -- 172.21.15.133:0/1677235066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc81929f0 con 0x7fedc81014b0 2023-11-27T15:21:02.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.013+0000 7fedc4ff9700 1 -- 172.21.15.133:0/1677235066 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fedbc0105a0 con 0x7fedc81014b0 2023-11-27T15:21:02.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.014+0000 7fedc4ff9700 1 --2- 172.21.15.133:0/1677235066 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fedb00a1150 0x7fedb00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:02.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.015+0000 7fedc4ff9700 1 -- 172.21.15.133:0/1677235066 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fedbc014070 con 0x7fedc81014b0 2023-11-27T15:21:02.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.015+0000 7fedc77fe700 1 --2- 172.21.15.133:0/1677235066 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fedb00a1150 0x7fedb00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:02.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.015+0000 7fedc77fe700 1 --2- 172.21.15.133:0/1677235066 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fedb00a1150 0x7fedb00a3600 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009fd0 tx=0x7fedb8019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:02.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.018+0000 7fedc4ff9700 1 -- 172.21.15.133:0/1677235066 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fedbc051730 con 0x7fedc81014b0 2023-11-27T15:21:02.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:01 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2377706000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:21:02.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:01 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2472468409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:21:02.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-11-27T15:21:02.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2023-11-27T15:21:02.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2023-11-27T15:21:02.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2023-11-27T15:21:02.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2023-11-27T15:21:02.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2023-11-27T15:21:02.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]: dispatch 2023-11-27T15:21:02.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2023-11-27T15:21:02.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2023-11-27T15:21:02.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2023-11-27T15:21:02.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:01 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2968588761' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-27T15:21:02.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:01 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2377706000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:21:02.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:01 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2472468409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:21:02.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-11-27T15:21:02.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2023-11-27T15:21:02.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2023-11-27T15:21:02.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2023-11-27T15:21:02.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2023-11-27T15:21:02.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2023-11-27T15:21:02.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]: dispatch 2023-11-27T15:21:02.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2023-11-27T15:21:02.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2023-11-27T15:21:02.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2023-11-27T15:21:02.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:01 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2968588761' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-27T15:21:02.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:01 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2377706000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:21:02.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:01 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2472468409' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:21:02.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2023-11-27T15:21:02.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2023-11-27T15:21:02.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2023-11-27T15:21:02.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2023-11-27T15:21:02.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2023-11-27T15:21:02.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2023-11-27T15:21:02.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]: dispatch 2023-11-27T15:21:02.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2023-11-27T15:21:02.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2023-11-27T15:21:02.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2023-11-27T15:21:02.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:01 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2968588761' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-11-27T15:21:02.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.266+0000 7fedcdfff700 1 -- 172.21.15.133:0/1677235066 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7fedc8061950 con 0x7fedc81014b0 2023-11-27T15:21:02.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.629+0000 7fedc4ff9700 1 -- 172.21.15.133:0/1677235066 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7fedbc096220 con 0x7fedc81014b0 2023-11-27T15:21:02.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.632+0000 7fedcdfff700 1 -- 172.21.15.133:0/1677235066 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fedb00a1150 msgr2=0x7fedb00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:02.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.632+0000 7fedcdfff700 1 --2- 172.21.15.133:0/1677235066 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fedb00a1150 0x7fedb00a3600 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009fd0 tx=0x7fedb8019040 comp rx=0 tx=0).stop 2023-11-27T15:21:02.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.632+0000 7fedcdfff700 1 -- 172.21.15.133:0/1677235066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedc81014b0 msgr2=0x7fedc8198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:02.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.632+0000 7fedcdfff700 1 --2- 172.21.15.133:0/1677235066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedc81014b0 0x7fedc8198800 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00e9a0 tx=0x7fedbc00ecb0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.632+0000 7fedcdfff700 1 -- 172.21.15.133:0/1677235066 shutdown_connections 2023-11-27T15:21:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.632+0000 7fedcdfff700 1 --2- 172.21.15.133:0/1677235066 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fedb00a1150 0x7fedb00a3600 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.632+0000 7fedcdfff700 1 --2- 172.21.15.133:0/1677235066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedc8101e40 0x7fedc819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.632+0000 7fedcdfff700 1 --2- 172.21.15.133:0/1677235066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedc81002c0 0x7fedc81982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.632+0000 7fedcdfff700 1 --2- 172.21.15.133:0/1677235066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fedc81014b0 0x7fedc8198800 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.632+0000 7fedcdfff700 1 -- 172.21.15.133:0/1677235066 >> 172.21.15.133:0/1677235066 conn(0x7fedc80fb8a0 msgr2=0x7fedc8106df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.632+0000 7fedcdfff700 1 -- 172.21.15.133:0/1677235066 shutdown_connections 2023-11-27T15:21:02.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.632+0000 7fedcdfff700 1 -- 172.21.15.133:0/1677235066 wait complete. 2023-11-27T15:21:02.658 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-11-27T15:21:02.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.861+0000 7f39c35ff700 1 -- 172.21.15.133:0/1831105111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc101e50 msgr2=0x7f39bc108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:02.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.861+0000 7f39c35ff700 1 --2- 172.21.15.133:0/1831105111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc101e50 0x7f39bc108710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f39b400b0a0 tx=0x7f39b400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.861+0000 7f39c35ff700 1 -- 172.21.15.133:0/1831105111 shutdown_connections 2023-11-27T15:21:02.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.861+0000 7f39c35ff700 1 --2- 172.21.15.133:0/1831105111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc101e50 0x7f39bc108710 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.861+0000 7f39c35ff700 1 --2- 172.21.15.133:0/1831105111 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39bc1002d0 0x7f39bc1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.861+0000 7f39c35ff700 1 --2- 172.21.15.133:0/1831105111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39bc1014c0 0x7f39bc101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.861+0000 7f39c35ff700 1 -- 172.21.15.133:0/1831105111 >> 172.21.15.133:0/1831105111 conn(0x7f39bc0fb8b0 msgr2=0x7f39bc0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:02.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.861+0000 7f39c35ff700 1 -- 172.21.15.133:0/1831105111 shutdown_connections 2023-11-27T15:21:02.866 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.861+0000 7f39c35ff700 1 -- 172.21.15.133:0/1831105111 wait complete. 2023-11-27T15:21:02.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.862+0000 7f39c35ff700 1 Processor -- start 2023-11-27T15:21:02.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.862+0000 7f39c35ff700 1 -- start start 2023-11-27T15:21:02.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.862+0000 7f39c35ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc1002d0 0x7f39bc0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:02.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.862+0000 7f39c35ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39bc1014c0 0x7f39bc072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:02.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.862+0000 7f39c35ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39bc101e50 0x7f39bc073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:02.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.862+0000 7f39c35ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc073bc0 con 0x7f39bc1002d0 2023-11-27T15:21:02.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.862+0000 7f39c35ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc073d30 con 0x7f39bc1014c0 2023-11-27T15:21:02.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.862+0000 7f39c35ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc1a01e0 con 0x7f39bc101e50 2023-11-27T15:21:02.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.863+0000 7f39c139b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc1002d0 0x7f39bc0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:02.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.863+0000 7f39c139b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc1002d0 0x7f39bc0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35910/0 (socket says 172.21.15.133:35910) 2023-11-27T15:21:02.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.863+0000 7f39c139b700 1 -- 172.21.15.133:0/1844509009 learned_addr learned my addr 172.21.15.133:0/1844509009 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:02.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.863+0000 7f39c0b9a700 1 --2- 172.21.15.133:0/1844509009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39bc1014c0 0x7f39bc072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:02.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.863+0000 7f39c1b9c700 1 --2- 172.21.15.133:0/1844509009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39bc101e50 0x7f39bc073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:02.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.863+0000 7f39c139b700 1 -- 172.21.15.133:0/1844509009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39bc101e50 msgr2=0x7f39bc073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:02.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.863+0000 7f39c139b700 1 --2- 172.21.15.133:0/1844509009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39bc101e50 0x7f39bc073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.863+0000 7f39c139b700 1 -- 172.21.15.133:0/1844509009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39bc1014c0 msgr2=0x7f39bc072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:02.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.863+0000 7f39c139b700 1 --2- 172.21.15.133:0/1844509009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39bc1014c0 0x7f39bc072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:02.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.863+0000 7f39c139b700 1 -- 172.21.15.133:0/1844509009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b4009d00 con 0x7f39bc1002d0 2023-11-27T15:21:02.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.863+0000 7f39c1b9c700 1 --2- 172.21.15.133:0/1844509009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39bc101e50 0x7f39bc073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:21:02.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.863+0000 7f39c139b700 1 --2- 172.21.15.133:0/1844509009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc1002d0 0x7f39bc0748a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f39b800e970 tx=0x7f39b800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:02.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.863+0000 7f39b27fc700 1 -- 172.21.15.133:0/1844509009 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8009790 con 0x7f39bc1002d0 2023-11-27T15:21:02.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.863+0000 7f39b27fc700 1 -- 172.21.15.133:0/1844509009 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39b800eec0 con 0x7f39bc1002d0 2023-11-27T15:21:02.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.863+0000 7f39b27fc700 1 -- 172.21.15.133:0/1844509009 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b8003cf0 con 0x7f39bc1002d0 2023-11-27T15:21:02.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.863+0000 7f39c35ff700 1 -- 172.21.15.133:0/1844509009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39bc10bca0 con 0x7f39bc1002d0 2023-11-27T15:21:02.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.863+0000 7f39c35ff700 1 -- 172.21.15.133:0/1844509009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39bc10bb10 con 0x7f39bc1002d0 2023-11-27T15:21:02.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.865+0000 7f39c35ff700 1 -- 172.21.15.133:0/1844509009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39bc1077f0 con 0x7f39bc1002d0 2023-11-27T15:21:02.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.867+0000 7f39b27fc700 1 -- 172.21.15.133:0/1844509009 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f39b800ccd0 con 0x7f39bc1002d0 2023-11-27T15:21:02.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.868+0000 7f39b27fc700 1 --2- 172.21.15.133:0/1844509009 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f39a80a11d0 0x7f39a80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:02.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.868+0000 7f39b27fc700 1 -- 172.21.15.133:0/1844509009 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f39b8014070 con 0x7f39bc1002d0 2023-11-27T15:21:02.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.868+0000 7f39c0b9a700 1 --2- 172.21.15.133:0/1844509009 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f39a80a11d0 0x7f39a80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:02.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.869+0000 7f39c0b9a700 1 --2- 172.21.15.133:0/1844509009 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f39a80a11d0 0x7f39a80a3680 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009c60 tx=0x7f39ac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:02.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:02.871+0000 7f39b27fc700 1 -- 172.21.15.133:0/1844509009 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f39b80963f0 con 0x7f39bc1002d0 2023-11-27T15:21:03.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:02 smithi133 ceph-mon[97860]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-11-27T15:21:03.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-11-27T15:21:03.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2023-11-27T15:21:03.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2023-11-27T15:21:03.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2023-11-27T15:21:03.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2023-11-27T15:21:03.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2023-11-27T15:21:03.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]': finished 2023-11-27T15:21:03.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2023-11-27T15:21:03.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2023-11-27T15:21:03.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2023-11-27T15:21:03.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:02 smithi133 ceph-mon[97860]: osdmap e89: 12 total, 12 up, 12 in 2023-11-27T15:21:03.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:02 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2968588761' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-11-27T15:21:03.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:02 smithi133 ceph-mon[97860]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:21:03.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:02 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1677235066' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-11-27T15:21:03.029 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:02 smithi133 ceph-mon[97860]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-11-27T15:21:03.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:02 smithi133 ceph-mon[97860]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-11-27T15:21:03.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:02 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1677235066' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-11-27T15:21:03.030 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:02 smithi133 ceph-mon[97860]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-11-27T15:21:03.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:02 smithi177 ceph-mon[131504]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-11-27T15:21:03.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-11-27T15:21:03.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2023-11-27T15:21:03.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2023-11-27T15:21:03.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2023-11-27T15:21:03.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2023-11-27T15:21:03.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2023-11-27T15:21:03.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]': finished 2023-11-27T15:21:03.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2023-11-27T15:21:03.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2023-11-27T15:21:03.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2023-11-27T15:21:03.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:02 smithi177 ceph-mon[131504]: osdmap e89: 12 total, 12 up, 12 in 2023-11-27T15:21:03.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:02 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2968588761' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-11-27T15:21:03.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:02 smithi177 ceph-mon[131504]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:21:03.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:02 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1677235066' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-11-27T15:21:03.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:02 smithi177 ceph-mon[131504]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-11-27T15:21:03.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:02 smithi177 ceph-mon[131504]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-11-27T15:21:03.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:02 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1677235066' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-11-27T15:21:03.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:02 smithi177 ceph-mon[131504]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-11-27T15:21:03.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:03.123+0000 7f39c35ff700 1 -- 172.21.15.133:0/1844509009 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f39bc061930 con 0x7f39bc1002d0 2023-11-27T15:21:03.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:02 smithi174 ceph-mon[131110]: pgmap v288: 129 pgs: 129 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-11-27T15:21:03.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2023-11-27T15:21:03.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2023-11-27T15:21:03.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2023-11-27T15:21:03.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2023-11-27T15:21:03.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2023-11-27T15:21:03.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2023-11-27T15:21:03.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]': finished 2023-11-27T15:21:03.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2023-11-27T15:21:03.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2023-11-27T15:21:03.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2023-11-27T15:21:03.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:02 smithi174 ceph-mon[131110]: osdmap e89: 12 total, 12 up, 12 in 2023-11-27T15:21:03.155 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:02 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2968588761' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-11-27T15:21:03.155 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:02 smithi174 ceph-mon[131110]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-11-27T15:21:03.155 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:02 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1677235066' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-11-27T15:21:03.155 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:02 smithi174 ceph-mon[131110]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-11-27T15:21:03.155 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:02 smithi174 ceph-mon[131110]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-11-27T15:21:03.155 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:02 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1677235066' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-11-27T15:21:03.157 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:02 smithi174 ceph-mon[131110]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-11-27T15:21:03.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:03.786+0000 7f39b27fc700 1 -- 172.21.15.133:0/1844509009 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f39b8099a10 con 0x7f39bc1002d0 2023-11-27T15:21:03.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:03.790+0000 7f39c35ff700 1 -- 172.21.15.133:0/1844509009 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f39a80a11d0 msgr2=0x7f39a80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:03.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:03.790+0000 7f39c35ff700 1 --2- 172.21.15.133:0/1844509009 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f39a80a11d0 0x7f39a80a3680 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009c60 tx=0x7f39ac009260 comp rx=0 tx=0).stop 2023-11-27T15:21:03.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:03.790+0000 7f39c35ff700 1 -- 172.21.15.133:0/1844509009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc1002d0 msgr2=0x7f39bc0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:03.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:03.790+0000 7f39c35ff700 1 --2- 172.21.15.133:0/1844509009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc1002d0 0x7f39bc0748a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f39b800e970 tx=0x7f39b800ec80 comp rx=0 tx=0).stop 2023-11-27T15:21:03.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:03.790+0000 7f39c35ff700 1 -- 172.21.15.133:0/1844509009 shutdown_connections 2023-11-27T15:21:03.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:03.790+0000 7f39c35ff700 1 --2- 172.21.15.133:0/1844509009 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f39a80a11d0 0x7f39a80a3680 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:03.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:03.790+0000 7f39c35ff700 1 --2- 172.21.15.133:0/1844509009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f39bc101e50 0x7f39bc073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:03.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:03.790+0000 7f39c35ff700 1 --2- 172.21.15.133:0/1844509009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39bc1002d0 0x7f39bc0748a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:03.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:03.790+0000 7f39c35ff700 1 --2- 172.21.15.133:0/1844509009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39bc1014c0 0x7f39bc072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:03.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:03.790+0000 7f39c35ff700 1 -- 172.21.15.133:0/1844509009 >> 172.21.15.133:0/1844509009 conn(0x7f39bc0fb8b0 msgr2=0x7f39bc104930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:03.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:03.790+0000 7f39c35ff700 1 -- 172.21.15.133:0/1844509009 shutdown_connections 2023-11-27T15:21:03.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:03.790+0000 7f39c35ff700 1 -- 172.21.15.133:0/1844509009 wait complete. 2023-11-27T15:21:03.819 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-11-27T15:21:04.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.017+0000 7fc2f7bff700 1 -- 172.21.15.133:0/640380522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2f0074b30 msgr2=0x7fc2f0072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:04.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.017+0000 7fc2f7bff700 1 --2- 172.21.15.133:0/640380522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2f0074b30 0x7fc2f0072f80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0098f0 tx=0x7fc2ec009c00 comp rx=0 tx=0).stop 2023-11-27T15:21:04.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.017+0000 7fc2f7bff700 1 -- 172.21.15.133:0/640380522 shutdown_connections 2023-11-27T15:21:04.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.017+0000 7fc2f7bff700 1 --2- 172.21.15.133:0/640380522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f010c890 0x7fc2f010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:04.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.017+0000 7fc2f7bff700 1 --2- 172.21.15.133:0/640380522 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2f0074b30 0x7fc2f0072f80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:04.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.017+0000 7fc2f7bff700 1 --2- 172.21.15.133:0/640380522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2f0073550 0x7fc2f00739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:04.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.017+0000 7fc2f7bff700 1 -- 172.21.15.133:0/640380522 >> 172.21.15.133:0/640380522 conn(0x7fc2f00fb8a0 msgr2=0x7fc2f00fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:04.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.017+0000 7fc2f7bff700 1 -- 172.21.15.133:0/640380522 shutdown_connections 2023-11-27T15:21:04.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.017+0000 7fc2f7bff700 1 -- 172.21.15.133:0/640380522 wait complete. 2023-11-27T15:21:04.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.018+0000 7fc2f7bff700 1 Processor -- start 2023-11-27T15:21:04.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.018+0000 7fc2f7bff700 1 -- start start 2023-11-27T15:21:04.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.018+0000 7fc2f7bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f0073550 0x7fc2f019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:04.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.018+0000 7fc2f7bff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2f0074b30 0x7fc2f019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:04.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.019+0000 7fc2f7bff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2f010c890 0x7fc2f01a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:04.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.019+0000 7fc2f7bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f01a4290 con 0x7fc2f0073550 2023-11-27T15:21:04.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.019+0000 7fc2f7bff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f01a43d0 con 0x7fc2f0074b30 2023-11-27T15:21:04.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.019+0000 7fc2f7bff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f01a4560 con 0x7fc2f010c890 2023-11-27T15:21:04.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.019+0000 7fc2f599b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f0073550 0x7fc2f019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:04.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.019+0000 7fc2f599b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f0073550 0x7fc2f019c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35932/0 (socket says 172.21.15.133:35932) 2023-11-27T15:21:04.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.019+0000 7fc2f599b700 1 -- 172.21.15.133:0/3978826931 learned_addr learned my addr 172.21.15.133:0/3978826931 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:04.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.019+0000 7fc2f519a700 1 --2- 172.21.15.133:0/3978826931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2f0074b30 0x7fc2f019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:04.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.019+0000 7fc2f619c700 1 --2- 172.21.15.133:0/3978826931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2f010c890 0x7fc2f01a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:04.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.019+0000 7fc2f599b700 1 -- 172.21.15.133:0/3978826931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2f010c890 msgr2=0x7fc2f01a3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:04.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.019+0000 7fc2f599b700 1 --2- 172.21.15.133:0/3978826931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2f010c890 0x7fc2f01a3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:04.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.019+0000 7fc2f599b700 1 -- 172.21.15.133:0/3978826931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2f0074b30 msgr2=0x7fc2f019cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:04.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.019+0000 7fc2f599b700 1 --2- 172.21.15.133:0/3978826931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2f0074b30 0x7fc2f019cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:04.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.019+0000 7fc2f599b700 1 -- 172.21.15.133:0/3978826931 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e00095f0 con 0x7fc2f0073550 2023-11-27T15:21:04.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.019+0000 7fc2f619c700 1 --2- 172.21.15.133:0/3978826931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2f010c890 0x7fc2f01a3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:21:04.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.019+0000 7fc2f599b700 1 --2- 172.21.15.133:0/3978826931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f0073550 0x7fc2f019c710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009fd0 tx=0x7fc2ec00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:04.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.020+0000 7fc2e6ffd700 1 -- 172.21.15.133:0/3978826931 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec01c070 con 0x7fc2f0073550 2023-11-27T15:21:04.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.020+0000 7fc2e6ffd700 1 -- 172.21.15.133:0/3978826931 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc2ec00f7d0 con 0x7fc2f0073550 2023-11-27T15:21:04.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.020+0000 7fc2e6ffd700 1 -- 172.21.15.133:0/3978826931 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec003c40 con 0x7fc2f0073550 2023-11-27T15:21:04.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.020+0000 7fc2f7bff700 1 -- 172.21.15.133:0/3978826931 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2ec009520 con 0x7fc2f0073550 2023-11-27T15:21:04.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.020+0000 7fc2f7bff700 1 -- 172.21.15.133:0/3978826931 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f0112280 con 0x7fc2f0073550 2023-11-27T15:21:04.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.021+0000 7fc2f7bff700 1 -- 172.21.15.133:0/3978826931 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f01023e0 con 0x7fc2f0073550 2023-11-27T15:21:04.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.023+0000 7fc2e6ffd700 1 -- 172.21.15.133:0/3978826931 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fc2ec020030 con 0x7fc2f0073550 2023-11-27T15:21:04.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.024+0000 7fc2e6ffd700 1 --2- 172.21.15.133:0/3978826931 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc2dc0a11d0 0x7fc2dc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:04.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.024+0000 7fc2e6ffd700 1 -- 172.21.15.133:0/3978826931 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc2ec0ce2a0 con 0x7fc2f0073550 2023-11-27T15:21:04.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.024+0000 7fc2f519a700 1 --2- 172.21.15.133:0/3978826931 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc2dc0a11d0 0x7fc2dc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:04.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.025+0000 7fc2f519a700 1 --2- 172.21.15.133:0/3978826931 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc2dc0a11d0 0x7fc2dc0a3680 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0074850 tx=0x7fc2e0009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:04.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.026+0000 7fc2e6ffd700 1 -- 172.21.15.133:0/3978826931 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc2ec097710 con 0x7fc2f0073550 2023-11-27T15:21:04.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:03 smithi177 ceph-mon[131504]: osdmap e90: 12 total, 12 up, 12 in 2023-11-27T15:21:04.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:03 smithi177 ceph-mon[131504]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-11-27T15:21:04.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:03 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1844509009' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-27T15:21:04.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:03 smithi174 ceph-mon[131110]: osdmap e90: 12 total, 12 up, 12 in 2023-11-27T15:21:04.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:03 smithi174 ceph-mon[131110]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-11-27T15:21:04.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:03 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1844509009' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-27T15:21:04.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.269+0000 7fc2f7bff700 1 -- 172.21.15.133:0/3978826931 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fc2f0067b00 con 0x7fc2f0073550 2023-11-27T15:21:04.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:03 smithi133 ceph-mon[97860]: osdmap e90: 12 total, 12 up, 12 in 2023-11-27T15:21:04.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:03 smithi133 ceph-mon[97860]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-11-27T15:21:04.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:03 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1844509009' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-11-27T15:21:04.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.793+0000 7fc2e6ffd700 1 -- 172.21.15.133:0/3978826931 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7fc2ec09ad30 con 0x7fc2f0073550 2023-11-27T15:21:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.796+0000 7fc2f7bff700 1 -- 172.21.15.133:0/3978826931 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc2dc0a11d0 msgr2=0x7fc2dc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.796+0000 7fc2f7bff700 1 --2- 172.21.15.133:0/3978826931 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc2dc0a11d0 0x7fc2dc0a3680 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0074850 tx=0x7fc2e0009310 comp rx=0 tx=0).stop 2023-11-27T15:21:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.796+0000 7fc2f7bff700 1 -- 172.21.15.133:0/3978826931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f0073550 msgr2=0x7fc2f019c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.796+0000 7fc2f7bff700 1 --2- 172.21.15.133:0/3978826931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f0073550 0x7fc2f019c710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009fd0 tx=0x7fc2ec00b840 comp rx=0 tx=0).stop 2023-11-27T15:21:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.796+0000 7fc2f7bff700 1 -- 172.21.15.133:0/3978826931 shutdown_connections 2023-11-27T15:21:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.796+0000 7fc2f7bff700 1 --2- 172.21.15.133:0/3978826931 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc2dc0a11d0 0x7fc2dc0a3680 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:04.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.797+0000 7fc2f7bff700 1 --2- 172.21.15.133:0/3978826931 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2f010c890 0x7fc2f01a3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.797+0000 7fc2f7bff700 1 --2- 172.21.15.133:0/3978826931 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f0073550 0x7fc2f019c710 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.797+0000 7fc2f7bff700 1 --2- 172.21.15.133:0/3978826931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2f0074b30 0x7fc2f019cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.797+0000 7fc2f7bff700 1 -- 172.21.15.133:0/3978826931 >> 172.21.15.133:0/3978826931 conn(0x7fc2f00fb8a0 msgr2=0x7fc2f00fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.797+0000 7fc2f7bff700 1 -- 172.21.15.133:0/3978826931 shutdown_connections 2023-11-27T15:21:04.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:04.797+0000 7fc2f7bff700 1 -- 172.21.15.133:0/3978826931 wait complete. 2023-11-27T15:21:04.822 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2023-11-27T15:21:04.822 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-11-27T15:21:05.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.022+0000 7f8e1f3ff700 1 -- 172.21.15.133:0/931114009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e18101e50 msgr2=0x7f8e18108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:05.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.022+0000 7f8e1f3ff700 1 --2- 172.21.15.133:0/931114009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e18101e50 0x7f8e18108710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000b0a0 tx=0x7f8e1000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.022+0000 7f8e1f3ff700 1 -- 172.21.15.133:0/931114009 shutdown_connections 2023-11-27T15:21:05.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.022+0000 7f8e1f3ff700 1 --2- 172.21.15.133:0/931114009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e18101e50 0x7f8e18108710 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.022+0000 7f8e1f3ff700 1 --2- 172.21.15.133:0/931114009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e181002d0 0x7f8e181006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.022+0000 7f8e1f3ff700 1 --2- 172.21.15.133:0/931114009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e181014c0 0x7f8e18101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.022+0000 7f8e1f3ff700 1 -- 172.21.15.133:0/931114009 >> 172.21.15.133:0/931114009 conn(0x7f8e180fb8b0 msgr2=0x7f8e180fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:05.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.022+0000 7f8e1f3ff700 1 -- 172.21.15.133:0/931114009 shutdown_connections 2023-11-27T15:21:05.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.022+0000 7f8e1f3ff700 1 -- 172.21.15.133:0/931114009 wait complete. 2023-11-27T15:21:05.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.023+0000 7f8e1f3ff700 1 Processor -- start 2023-11-27T15:21:05.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.023+0000 7f8e1f3ff700 1 -- start start 2023-11-27T15:21:05.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.023+0000 7f8e1f3ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e181002d0 0x7f8e180748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:05.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.023+0000 7f8e1f3ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e181014c0 0x7f8e18072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:05.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.023+0000 7f8e1f3ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e18101e50 0x7f8e18073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:05.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.023+0000 7f8e1f3ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e18073bc0 con 0x7f8e181014c0 2023-11-27T15:21:05.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.023+0000 7f8e1f3ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e18073d30 con 0x7f8e181002d0 2023-11-27T15:21:05.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.023+0000 7f8e1f3ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e181a0240 con 0x7f8e18101e50 2023-11-27T15:21:05.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.023+0000 7f8e1c99a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e181014c0 0x7f8e18072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:05.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.023+0000 7f8e1c99a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e181014c0 0x7f8e18072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35952/0 (socket says 172.21.15.133:35952) 2023-11-27T15:21:05.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.023+0000 7f8e1c99a700 1 -- 172.21.15.133:0/1034049496 learned_addr learned my addr 172.21.15.133:0/1034049496 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:05.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.023+0000 7f8e1d99c700 1 --2- 172.21.15.133:0/1034049496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e18101e50 0x7f8e18073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:05.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.023+0000 7f8e1d19b700 1 --2- 172.21.15.133:0/1034049496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e181002d0 0x7f8e180748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:05.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.024+0000 7f8e1c99a700 1 -- 172.21.15.133:0/1034049496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e18101e50 msgr2=0x7f8e18073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:05.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.024+0000 7f8e1c99a700 1 --2- 172.21.15.133:0/1034049496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e18101e50 0x7f8e18073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.024+0000 7f8e1c99a700 1 -- 172.21.15.133:0/1034049496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e181002d0 msgr2=0x7f8e180748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:05.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.024+0000 7f8e1c99a700 1 --2- 172.21.15.133:0/1034049496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e181002d0 0x7f8e180748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.024+0000 7f8e1c99a700 1 -- 172.21.15.133:0/1034049496 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e10009d00 con 0x7f8e181014c0 2023-11-27T15:21:05.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.024+0000 7f8e1d99c700 1 --2- 172.21.15.133:0/1034049496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e18101e50 0x7f8e18073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:05.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.024+0000 7f8e1c99a700 1 --2- 172.21.15.133:0/1034049496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e181014c0 0x7f8e18072ee0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800e9a0 tx=0x7f8e0800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:05.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.024+0000 7f8e0e7fc700 1 -- 172.21.15.133:0/1034049496 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08009790 con 0x7f8e181014c0 2023-11-27T15:21:05.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.024+0000 7f8e0e7fc700 1 -- 172.21.15.133:0/1034049496 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e08010430 con 0x7f8e181014c0 2023-11-27T15:21:05.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.024+0000 7f8e0e7fc700 1 -- 172.21.15.133:0/1034049496 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08018680 con 0x7f8e181014c0 2023-11-27T15:21:05.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.024+0000 7f8e1f3ff700 1 -- 172.21.15.133:0/1034049496 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e1810bca0 con 0x7f8e181014c0 2023-11-27T15:21:05.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.024+0000 7f8e1f3ff700 1 -- 172.21.15.133:0/1034049496 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1810bb10 con 0x7f8e181014c0 2023-11-27T15:21:05.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.025+0000 7f8e1f3ff700 1 -- 172.21.15.133:0/1034049496 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e181077c0 con 0x7f8e181014c0 2023-11-27T15:21:05.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.027+0000 7f8e0e7fc700 1 -- 172.21.15.133:0/1034049496 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f8e080105a0 con 0x7f8e181014c0 2023-11-27T15:21:05.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.029+0000 7f8e0e7fc700 1 --2- 172.21.15.133:0/1034049496 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8e040a11d0 0x7f8e040a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:05.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.030+0000 7f8e0e7fc700 1 -- 172.21.15.133:0/1034049496 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8e08014070 con 0x7f8e181014c0 2023-11-27T15:21:05.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.030+0000 7f8e1d19b700 1 --2- 172.21.15.133:0/1034049496 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8e040a11d0 0x7f8e040a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:05.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.030+0000 7f8e1d19b700 1 --2- 172.21.15.133:0/1034049496 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8e040a11d0 0x7f8e040a3680 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8e18101320 tx=0x7f8e14009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:05.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.033+0000 7f8e0e7fc700 1 -- 172.21.15.133:0/1034049496 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8e08051730 con 0x7f8e181014c0 2023-11-27T15:21:05.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:04 smithi177 ceph-mon[131504]: pgmap v291: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-11-27T15:21:05.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:04 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] up:active 2023-11-27T15:21:05.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:04 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1844509009' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-11-27T15:21:05.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:04 smithi177 ceph-mon[131504]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-11-27T15:21:05.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:04 smithi177 ceph-mon[131504]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-11-27T15:21:05.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:04 smithi177 ceph-mon[131504]: Cluster is now healthy 2023-11-27T15:21:05.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:04 smithi177 ceph-mon[131504]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2023-11-27T15:21:05.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:04 smithi177 ceph-mon[131504]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2023-11-27T15:21:05.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:04 smithi177 ceph-mon[131504]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-11-27T15:21:05.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:04 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3978826931' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-27T15:21:05.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:04 smithi174 ceph-mon[131110]: pgmap v291: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-11-27T15:21:05.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:04 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] up:active 2023-11-27T15:21:05.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:04 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1844509009' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-11-27T15:21:05.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:04 smithi174 ceph-mon[131110]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-11-27T15:21:05.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:04 smithi174 ceph-mon[131110]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-11-27T15:21:05.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:04 smithi174 ceph-mon[131110]: Cluster is now healthy 2023-11-27T15:21:05.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:04 smithi174 ceph-mon[131110]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2023-11-27T15:21:05.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:04 smithi174 ceph-mon[131110]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2023-11-27T15:21:05.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:04 smithi174 ceph-mon[131110]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-11-27T15:21:05.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:04 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3978826931' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-27T15:21:05.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.272+0000 7f8e1f3ff700 1 -- 172.21.15.133:0/1034049496 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e18061c40 con 0x7f8e040a11d0 2023-11-27T15:21:05.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:04 smithi133 ceph-mon[97860]: pgmap v291: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-11-27T15:21:05.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:04 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] up:active 2023-11-27T15:21:05.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:04 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1844509009' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-11-27T15:21:05.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:04 smithi133 ceph-mon[97860]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-11-27T15:21:05.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:04 smithi133 ceph-mon[97860]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-11-27T15:21:05.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:04 smithi133 ceph-mon[97860]: Cluster is now healthy 2023-11-27T15:21:05.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:04 smithi133 ceph-mon[97860]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2023-11-27T15:21:05.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:04 smithi133 ceph-mon[97860]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2023-11-27T15:21:05.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:04 smithi133 ceph-mon[97860]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-11-27T15:21:05.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:04 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3978826931' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-11-27T15:21:05.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.316+0000 7f8e0e7fc700 1 -- 172.21.15.133:0/1034049496 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8e18061c40 con 0x7f8e040a11d0 2023-11-27T15:21:05.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.318+0000 7f8e1f3ff700 1 -- 172.21.15.133:0/1034049496 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8e040a11d0 msgr2=0x7f8e040a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:05.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.318+0000 7f8e1f3ff700 1 --2- 172.21.15.133:0/1034049496 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8e040a11d0 0x7f8e040a3680 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8e18101320 tx=0x7f8e14009260 comp rx=0 tx=0).stop 2023-11-27T15:21:05.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.318+0000 7f8e1f3ff700 1 -- 172.21.15.133:0/1034049496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e181014c0 msgr2=0x7f8e18072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:05.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.318+0000 7f8e1f3ff700 1 --2- 172.21.15.133:0/1034049496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e181014c0 0x7f8e18072ee0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800e9a0 tx=0x7f8e0800ecb0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.318+0000 7f8e1f3ff700 1 -- 172.21.15.133:0/1034049496 shutdown_connections 2023-11-27T15:21:05.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.318+0000 7f8e1f3ff700 1 --2- 172.21.15.133:0/1034049496 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8e040a11d0 0x7f8e040a3680 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.318+0000 7f8e1f3ff700 1 --2- 172.21.15.133:0/1034049496 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8e18101e50 0x7f8e18073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.318+0000 7f8e1f3ff700 1 --2- 172.21.15.133:0/1034049496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e181002d0 0x7f8e180748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.318+0000 7f8e1f3ff700 1 --2- 172.21.15.133:0/1034049496 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e181014c0 0x7f8e18072ee0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.318+0000 7f8e1f3ff700 1 -- 172.21.15.133:0/1034049496 >> 172.21.15.133:0/1034049496 conn(0x7f8e180fb8b0 msgr2=0x7f8e18104900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.319+0000 7f8e1f3ff700 1 -- 172.21.15.133:0/1034049496 shutdown_connections 2023-11-27T15:21:05.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.319+0000 7f8e1f3ff700 1 -- 172.21.15.133:0/1034049496 wait complete. 2023-11-27T15:21:05.342 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-11-27T15:21:05.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.539+0000 7f5c3d1ff700 1 -- 172.21.15.133:0/3382904599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c38073550 msgr2=0x7f5c380739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:05.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.539+0000 7f5c3d1ff700 1 --2- 172.21.15.133:0/3382904599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c38073550 0x7f5c380739c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009990 tx=0x7f5c20009ca0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.539+0000 7f5c3d1ff700 1 -- 172.21.15.133:0/3382904599 shutdown_connections 2023-11-27T15:21:05.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.539+0000 7f5c3d1ff700 1 --2- 172.21.15.133:0/3382904599 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c3810c8a0 0x7f5c3810ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.539+0000 7f5c3d1ff700 1 --2- 172.21.15.133:0/3382904599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c38074b30 0x7f5c38072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.539+0000 7f5c3d1ff700 1 --2- 172.21.15.133:0/3382904599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c38073550 0x7f5c380739c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.539+0000 7f5c3d1ff700 1 -- 172.21.15.133:0/3382904599 >> 172.21.15.133:0/3382904599 conn(0x7f5c380fb8b0 msgr2=0x7f5c380fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:05.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.539+0000 7f5c3d1ff700 1 -- 172.21.15.133:0/3382904599 shutdown_connections 2023-11-27T15:21:05.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.539+0000 7f5c3d1ff700 1 -- 172.21.15.133:0/3382904599 wait complete. 2023-11-27T15:21:05.547 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.540+0000 7f5c3d1ff700 1 Processor -- start 2023-11-27T15:21:05.547 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.540+0000 7f5c3d1ff700 1 -- start start 2023-11-27T15:21:05.547 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.540+0000 7f5c3d1ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c38073550 0x7f5c3819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:05.548 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.540+0000 7f5c3d1ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c38074b30 0x7f5c3819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:05.548 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.540+0000 7f5c3d1ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c3810c8a0 0x7f5c381a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:05.548 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.540+0000 7f5c3d1ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c381a4210 con 0x7f5c3810c8a0 2023-11-27T15:21:05.548 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.540+0000 7f5c3d1ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c381a4350 con 0x7f5c38074b30 2023-11-27T15:21:05.548 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.540+0000 7f5c3d1ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c381a44e0 con 0x7f5c38073550 2023-11-27T15:21:05.548 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.540+0000 7f5c3759e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c3810c8a0 0x7f5c381a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:05.548 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.540+0000 7f5c36d9d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c38073550 0x7f5c3819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:05.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.540+0000 7f5c3759e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c3810c8a0 0x7f5c381a3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35982/0 (socket says 172.21.15.133:35982) 2023-11-27T15:21:05.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.540+0000 7f5c3759e700 1 -- 172.21.15.133:0/4185064985 learned_addr learned my addr 172.21.15.133:0/4185064985 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:05.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.541+0000 7f5c2ffff700 1 --2- 172.21.15.133:0/4185064985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c38074b30 0x7f5c3819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:05.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.540+0000 7f5c36d9d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c38073550 0x7f5c3819c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:49592/0 (socket says 172.21.15.133:49592) 2023-11-27T15:21:05.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.541+0000 7f5c3759e700 1 -- 172.21.15.133:0/4185064985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c38073550 msgr2=0x7f5c3819c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:05.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.541+0000 7f5c3759e700 1 --2- 172.21.15.133:0/4185064985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c38073550 0x7f5c3819c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.541+0000 7f5c3759e700 1 -- 172.21.15.133:0/4185064985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c38074b30 msgr2=0x7f5c3819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:05.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.541+0000 7f5c3759e700 1 --2- 172.21.15.133:0/4185064985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c38074b30 0x7f5c3819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.550 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.541+0000 7f5c3759e700 1 -- 172.21.15.133:0/4185064985 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c200095f0 con 0x7f5c3810c8a0 2023-11-27T15:21:05.550 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.541+0000 7f5c3759e700 1 --2- 172.21.15.133:0/4185064985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c3810c8a0 0x7f5c381a3c40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5c30007ee0 tx=0x7f5c3000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:05.550 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.541+0000 7f5c34d99700 1 -- 172.21.15.133:0/4185064985 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c30017070 con 0x7f5c3810c8a0 2023-11-27T15:21:05.550 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.541+0000 7f5c34d99700 1 -- 172.21.15.133:0/4185064985 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c3000f040 con 0x7f5c3810c8a0 2023-11-27T15:21:05.550 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.541+0000 7f5c3d1ff700 1 -- 172.21.15.133:0/4185064985 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c381122b0 con 0x7f5c3810c8a0 2023-11-27T15:21:05.550 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.541+0000 7f5c34d99700 1 -- 172.21.15.133:0/4185064985 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c30005580 con 0x7f5c3810c8a0 2023-11-27T15:21:05.550 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.542+0000 7f5c3d1ff700 1 -- 172.21.15.133:0/4185064985 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c38112120 con 0x7f5c3810c8a0 2023-11-27T15:21:05.551 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.543+0000 7f5c3d1ff700 1 -- 172.21.15.133:0/4185064985 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c381023f0 con 0x7f5c3810c8a0 2023-11-27T15:21:05.551 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.545+0000 7f5c34d99700 1 -- 172.21.15.133:0/4185064985 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 16) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5c3000db20 con 0x7f5c3810c8a0 2023-11-27T15:21:05.552 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.546+0000 7f5c34d99700 1 --2- 172.21.15.133:0/4185064985 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5c240a1150 0x7f5c240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:05.552 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.546+0000 7f5c34d99700 1 -- 172.21.15.133:0/4185064985 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5c300cc390 con 0x7f5c3810c8a0 2023-11-27T15:21:05.552 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.546+0000 7f5c36d9d700 1 --2- 172.21.15.133:0/4185064985 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5c240a1150 0x7f5c240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:05.552 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.547+0000 7f5c36d9d700 1 --2- 172.21.15.133:0/4185064985 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5c240a1150 0x7f5c240a3600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5c38074f00 tx=0x7f5c28009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:05.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.550+0000 7f5c34d99700 1 -- 172.21.15.133:0/4185064985 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5c30095770 con 0x7f5c3810c8a0 2023-11-27T15:21:05.777 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:21:05 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:21:05.279+0000 7f982ebcb700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:21:05.777 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:21:05 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:21:05.279+0000 7f982ebcb700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:21:05.777 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:21:05 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:21:05.279+0000 7f982ebcb700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:21:05.778 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:21:05 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:21:05.279+0000 7f982ebcb700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:21:05.778 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:21:05 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:21:05.279+0000 7f982ebcb700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-27T15:21:05.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.785+0000 7f5c3d1ff700 1 -- 172.21.15.133:0/4185064985 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c38061c10 con 0x7f5c240a1150 2023-11-27T15:21:05.793 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.789+0000 7f5c34d99700 1 -- 172.21.15.133:0/4185064985 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f5c38061c10 con 0x7f5c240a1150 2023-11-27T15:21:05.794 INFO:teuthology.orchestra.run.smithi133.stdout:/volumes/_nogroup/sv_0/7504ff4f-c948-41ff-b61a-c25ffc59449c 2023-11-27T15:21:05.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.791+0000 7f5c3d1ff700 1 -- 172.21.15.133:0/4185064985 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5c240a1150 msgr2=0x7f5c240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:05.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.791+0000 7f5c3d1ff700 1 --2- 172.21.15.133:0/4185064985 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5c240a1150 0x7f5c240a3600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5c38074f00 tx=0x7f5c28009260 comp rx=0 tx=0).stop 2023-11-27T15:21:05.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.791+0000 7f5c3d1ff700 1 -- 172.21.15.133:0/4185064985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c3810c8a0 msgr2=0x7f5c381a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:05.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.791+0000 7f5c3d1ff700 1 --2- 172.21.15.133:0/4185064985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c3810c8a0 0x7f5c381a3c40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5c30007ee0 tx=0x7f5c3000d3b0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.792+0000 7f5c3d1ff700 1 -- 172.21.15.133:0/4185064985 shutdown_connections 2023-11-27T15:21:05.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.792+0000 7f5c3d1ff700 1 --2- 172.21.15.133:0/4185064985 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5c240a1150 0x7f5c240a3600 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.792+0000 7f5c3d1ff700 1 --2- 172.21.15.133:0/4185064985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c3810c8a0 0x7f5c381a3c40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.792+0000 7f5c3d1ff700 1 --2- 172.21.15.133:0/4185064985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c38073550 0x7f5c3819c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.792+0000 7f5c3d1ff700 1 --2- 172.21.15.133:0/4185064985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c38074b30 0x7f5c3819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:05.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.792+0000 7f5c3d1ff700 1 -- 172.21.15.133:0/4185064985 >> 172.21.15.133:0/4185064985 conn(0x7f5c380fb8b0 msgr2=0x7f5c380fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:05.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.792+0000 7f5c3d1ff700 1 -- 172.21.15.133:0/4185064985 shutdown_connections 2023-11-27T15:21:05.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:05.792+0000 7f5c3d1ff700 1 -- 172.21.15.133:0/4185064985 wait complete. 2023-11-27T15:21:05.820 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-11-27T15:21:06.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.019+0000 7f72899ff700 1 -- 172.21.15.133:0/3208700439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72841014b0 msgr2=0x7f7284101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:06.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.019+0000 7f72899ff700 1 --2- 172.21.15.133:0/3208700439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72841014b0 0x7f7284101900 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f726c009990 tx=0x7f726c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.019+0000 7f72899ff700 1 -- 172.21.15.133:0/3208700439 shutdown_connections 2023-11-27T15:21:06.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.019+0000 7f72899ff700 1 --2- 172.21.15.133:0/3208700439 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7284101e40 0x7f7284108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.019+0000 7f72899ff700 1 --2- 172.21.15.133:0/3208700439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72841002c0 0x7f72841006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.019+0000 7f72899ff700 1 --2- 172.21.15.133:0/3208700439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72841014b0 0x7f7284101900 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.019+0000 7f72899ff700 1 -- 172.21.15.133:0/3208700439 >> 172.21.15.133:0/3208700439 conn(0x7f72840fb8a0 msgr2=0x7f72840fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:06.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.019+0000 7f72899ff700 1 -- 172.21.15.133:0/3208700439 shutdown_connections 2023-11-27T15:21:06.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.019+0000 7f72899ff700 1 -- 172.21.15.133:0/3208700439 wait complete. 2023-11-27T15:21:06.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.020+0000 7f72899ff700 1 Processor -- start 2023-11-27T15:21:06.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.020+0000 7f72899ff700 1 -- start start 2023-11-27T15:21:06.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.020+0000 7f72899ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72841002c0 0x7f7284198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:06.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.020+0000 7f72899ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72841014b0 0x7f72841987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:06.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.020+0000 7f72899ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7284101e40 0x7f728419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:06.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.020+0000 7f72899ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728419fde0 con 0x7f72841002c0 2023-11-27T15:21:06.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.020+0000 7f72899ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728419ff20 con 0x7f72841014b0 2023-11-27T15:21:06.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.020+0000 7f72899ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72841a00b0 con 0x7f7284101e40 2023-11-27T15:21:06.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.021+0000 7f7282ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72841002c0 0x7f7284198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:06.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.021+0000 7f727bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72841014b0 0x7f72841987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:06.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.021+0000 7f7282ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72841002c0 0x7f7284198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35990/0 (socket says 172.21.15.133:35990) 2023-11-27T15:21:06.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.021+0000 7f727bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72841014b0 0x7f72841987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:53548/0 (socket says 172.21.15.133:53548) 2023-11-27T15:21:06.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.021+0000 7f7282ffd700 1 -- 172.21.15.133:0/4280143187 learned_addr learned my addr 172.21.15.133:0/4280143187 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:06.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.021+0000 7f72837fe700 1 --2- 172.21.15.133:0/4280143187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7284101e40 0x7f728419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:06.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.021+0000 7f7282ffd700 1 -- 172.21.15.133:0/4280143187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7284101e40 msgr2=0x7f728419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:06.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.021+0000 7f7282ffd700 1 --2- 172.21.15.133:0/4280143187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7284101e40 0x7f728419f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.021+0000 7f7282ffd700 1 -- 172.21.15.133:0/4280143187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72841014b0 msgr2=0x7f72841987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:06.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.021+0000 7f7282ffd700 1 --2- 172.21.15.133:0/4280143187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72841014b0 0x7f72841987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.021+0000 7f7282ffd700 1 -- 172.21.15.133:0/4280143187 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726c0095f0 con 0x7f72841002c0 2023-11-27T15:21:06.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.021+0000 7f727bfff700 1 --2- 172.21.15.133:0/4280143187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72841014b0 0x7f72841987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:06.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.021+0000 7f7282ffd700 1 --2- 172.21.15.133:0/4280143187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72841002c0 0x7f7284198260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f727400e970 tx=0x7f727400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:06.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.021+0000 7f72889fd700 1 -- 172.21.15.133:0/4280143187 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7274009790 con 0x7f72841002c0 2023-11-27T15:21:06.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.022+0000 7f72889fd700 1 -- 172.21.15.133:0/4280143187 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f727400eec0 con 0x7f72841002c0 2023-11-27T15:21:06.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.022+0000 7f72899ff700 1 -- 172.21.15.133:0/4280143187 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728410bc70 con 0x7f72841002c0 2023-11-27T15:21:06.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.022+0000 7f72889fd700 1 -- 172.21.15.133:0/4280143187 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7274003cf0 con 0x7f72841002c0 2023-11-27T15:21:06.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.022+0000 7f72899ff700 1 -- 172.21.15.133:0/4280143187 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728410bae0 con 0x7f72841002c0 2023-11-27T15:21:06.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.023+0000 7f72899ff700 1 -- 172.21.15.133:0/4280143187 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72841a0850 con 0x7f72841002c0 2023-11-27T15:21:06.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.025+0000 7f72889fd700 1 -- 172.21.15.133:0/4280143187 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f727400ccd0 con 0x7f72841002c0 2023-11-27T15:21:06.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.027+0000 7f72889fd700 1 --2- 172.21.15.133:0/4280143187 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f72700a1260 0x7f72700a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:06.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.027+0000 7f72889fd700 1 -- 172.21.15.133:0/4280143187 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7274014070 con 0x7f72841002c0 2023-11-27T15:21:06.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.027+0000 7f727bfff700 1 --2- 172.21.15.133:0/4280143187 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f72700a1260 0x7f72700a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:06.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.028+0000 7f727bfff700 1 --2- 172.21.15.133:0/4280143187 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f72700a1260 0x7f72700a3710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f726c00b2e0 tx=0x7f726c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:06.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.030+0000 7f72889fd700 1 -- 172.21.15.133:0/4280143187 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7274096240 con 0x7f72841002c0 2023-11-27T15:21:06.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:05 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] up:active 2023-11-27T15:21:06.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:05 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3978826931' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-11-27T15:21:06.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:05 smithi177 ceph-mon[131504]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2023-11-27T15:21:06.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:05 smithi177 ceph-mon[131504]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:21:06.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:05 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] up:active 2023-11-27T15:21:06.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:05 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3978826931' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-11-27T15:21:06.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:05 smithi174 ceph-mon[131110]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2023-11-27T15:21:06.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:05 smithi174 ceph-mon[131110]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:21:06.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.258+0000 7f72899ff700 1 -- 172.21.15.133:0/4280143187 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f72841a0e90 con 0x7f72700a1260 2023-11-27T15:21:06.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:05 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] up:active 2023-11-27T15:21:06.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:05 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3978826931' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-11-27T15:21:06.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:05 smithi133 ceph-mon[97860]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2023-11-27T15:21:06.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:05 smithi133 ceph-mon[97860]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:21:06.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.284+0000 7f72889fd700 1 -- 172.21.15.133:0/4280143187 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f72841a0e90 con 0x7f72700a1260 2023-11-27T15:21:06.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.287+0000 7f72899ff700 1 -- 172.21.15.133:0/4280143187 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f72700a1260 msgr2=0x7f72700a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:06.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.287+0000 7f72899ff700 1 --2- 172.21.15.133:0/4280143187 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f72700a1260 0x7f72700a3710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f726c00b2e0 tx=0x7f726c005fb0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.287+0000 7f72899ff700 1 -- 172.21.15.133:0/4280143187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72841002c0 msgr2=0x7f7284198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:06.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.287+0000 7f72899ff700 1 --2- 172.21.15.133:0/4280143187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72841002c0 0x7f7284198260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f727400e970 tx=0x7f727400ec80 comp rx=0 tx=0).stop 2023-11-27T15:21:06.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.287+0000 7f72899ff700 1 -- 172.21.15.133:0/4280143187 shutdown_connections 2023-11-27T15:21:06.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.287+0000 7f72899ff700 1 --2- 172.21.15.133:0/4280143187 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f72700a1260 0x7f72700a3710 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.287+0000 7f72899ff700 1 --2- 172.21.15.133:0/4280143187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7284101e40 0x7f728419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.287+0000 7f72899ff700 1 --2- 172.21.15.133:0/4280143187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72841002c0 0x7f7284198260 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.287+0000 7f72899ff700 1 --2- 172.21.15.133:0/4280143187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72841014b0 0x7f72841987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.287+0000 7f72899ff700 1 -- 172.21.15.133:0/4280143187 >> 172.21.15.133:0/4280143187 conn(0x7f72840fb8a0 msgr2=0x7f72840fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:06.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.287+0000 7f72899ff700 1 -- 172.21.15.133:0/4280143187 shutdown_connections 2023-11-27T15:21:06.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.287+0000 7f72899ff700 1 -- 172.21.15.133:0/4280143187 wait complete. 2023-11-27T15:21:06.310 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-11-27T15:21:06.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.507+0000 7f9166f9d700 1 -- 172.21.15.133:0/2221262264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9160100050 msgr2=0x7f9160100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:06.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.507+0000 7f9166f9d700 1 --2- 172.21.15.133:0/2221262264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9160100050 0x7f9160100460 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f915c009990 tx=0x7f915c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.507+0000 7f9166f9d700 1 -- 172.21.15.133:0/2221262264 shutdown_connections 2023-11-27T15:21:06.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.507+0000 7f9166f9d700 1 --2- 172.21.15.133:0/2221262264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9160101c90 0x7f9160108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.507+0000 7f9166f9d700 1 --2- 172.21.15.133:0/2221262264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9160100050 0x7f9160100460 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.507+0000 7f9166f9d700 1 --2- 172.21.15.133:0/2221262264 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91601012e0 0x7f9160101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.507+0000 7f9166f9d700 1 -- 172.21.15.133:0/2221262264 >> 172.21.15.133:0/2221262264 conn(0x7f91600fb650 msgr2=0x7f91600fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:06.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.507+0000 7f9166f9d700 1 -- 172.21.15.133:0/2221262264 shutdown_connections 2023-11-27T15:21:06.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.507+0000 7f9166f9d700 1 -- 172.21.15.133:0/2221262264 wait complete. 2023-11-27T15:21:06.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.508+0000 7f9166f9d700 1 Processor -- start 2023-11-27T15:21:06.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.508+0000 7f9166f9d700 1 -- start start 2023-11-27T15:21:06.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.508+0000 7f9166f9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9160100050 0x7f9160197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:06.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.508+0000 7f9166f9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91601012e0 0x7f9160198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:06.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.508+0000 7f9166f9d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9160101c90 0x7f916019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:06.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.509+0000 7f9166f9d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916019fb40 con 0x7f9160100050 2023-11-27T15:21:06.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.509+0000 7f9166f9d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916019fc80 con 0x7f91601012e0 2023-11-27T15:21:06.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.509+0000 7f9166f9d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916019fe10 con 0x7f9160101c90 2023-11-27T15:21:06.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.509+0000 7f9165f9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9160100050 0x7f9160197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.509+0000 7f9165f9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9160100050 0x7f9160197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36014/0 (socket says 172.21.15.133:36014) 2023-11-27T15:21:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.509+0000 7f9165f9b700 1 -- 172.21.15.133:0/2390870284 learned_addr learned my addr 172.21.15.133:0/2390870284 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.509+0000 7f916579a700 1 --2- 172.21.15.133:0/2390870284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91601012e0 0x7f9160198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.509+0000 7f916679c700 1 --2- 172.21.15.133:0/2390870284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9160101c90 0x7f916019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.509+0000 7f9165f9b700 1 -- 172.21.15.133:0/2390870284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9160101c90 msgr2=0x7f916019f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.509+0000 7f9165f9b700 1 --2- 172.21.15.133:0/2390870284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9160101c90 0x7f916019f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.509+0000 7f9165f9b700 1 -- 172.21.15.133:0/2390870284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91601012e0 msgr2=0x7f9160198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.509+0000 7f9165f9b700 1 --2- 172.21.15.133:0/2390870284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91601012e0 0x7f9160198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.509+0000 7f9165f9b700 1 -- 172.21.15.133:0/2390870284 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f915c0095f0 con 0x7f9160100050 2023-11-27T15:21:06.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.509+0000 7f9165f9b700 1 --2- 172.21.15.133:0/2390870284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9160100050 0x7f9160197fc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f915c005910 tx=0x7f915c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.510+0000 7f9156ffd700 1 -- 172.21.15.133:0/2390870284 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c01d070 con 0x7f9160100050 2023-11-27T15:21:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.510+0000 7f9156ffd700 1 -- 172.21.15.133:0/2390870284 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f915c00fb80 con 0x7f9160100050 2023-11-27T15:21:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.510+0000 7f9166f9d700 1 -- 172.21.15.133:0/2390870284 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916010ba00 con 0x7f9160100050 2023-11-27T15:21:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.510+0000 7f9156ffd700 1 -- 172.21.15.133:0/2390870284 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c017650 con 0x7f9160100050 2023-11-27T15:21:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.510+0000 7f9166f9d700 1 -- 172.21.15.133:0/2390870284 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916010b870 con 0x7f9160100050 2023-11-27T15:21:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.511+0000 7f9166f9d700 1 -- 172.21.15.133:0/2390870284 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9160067980 con 0x7f9160100050 2023-11-27T15:21:06.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.513+0000 7f9156ffd700 1 -- 172.21.15.133:0/2390870284 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f915c00fcf0 con 0x7f9160100050 2023-11-27T15:21:06.519 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.515+0000 7f9156ffd700 1 --2- 172.21.15.133:0/2390870284 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f914c0a1260 0x7f914c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:06.519 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.515+0000 7f9156ffd700 1 -- 172.21.15.133:0/2390870284 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f915c0ce060 con 0x7f9160100050 2023-11-27T15:21:06.519 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.515+0000 7f916579a700 1 --2- 172.21.15.133:0/2390870284 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f914c0a1260 0x7f914c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:06.519 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.516+0000 7f916579a700 1 --2- 172.21.15.133:0/2390870284 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f914c0a1260 0x7f914c0a3710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9150005950 tx=0x7f915000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:06.523 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.519+0000 7f9156ffd700 1 -- 172.21.15.133:0/2390870284 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f915c0973d0 con 0x7f9160100050 2023-11-27T15:21:06.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.753+0000 7f9166f9d700 1 -- 172.21.15.133:0/2390870284 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f91601a09b0 con 0x7f914c0a1260 2023-11-27T15:21:06.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.757+0000 7f9156ffd700 1 -- 172.21.15.133:0/2390870284 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f91601a09b0 con 0x7f914c0a1260 2023-11-27T15:21:06.761 INFO:teuthology.orchestra.run.smithi133.stdout:/volumes/_nogroup/sv_1/5deb6751-ee17-4f54-a826-7cf3bc242b61 2023-11-27T15:21:06.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.760+0000 7f9166f9d700 1 -- 172.21.15.133:0/2390870284 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f914c0a1260 msgr2=0x7f914c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:06.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.760+0000 7f9166f9d700 1 --2- 172.21.15.133:0/2390870284 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f914c0a1260 0x7f914c0a3710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9150005950 tx=0x7f915000b410 comp rx=0 tx=0).stop 2023-11-27T15:21:06.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.760+0000 7f9166f9d700 1 -- 172.21.15.133:0/2390870284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9160100050 msgr2=0x7f9160197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:06.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.760+0000 7f9166f9d700 1 --2- 172.21.15.133:0/2390870284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9160100050 0x7f9160197fc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f915c005910 tx=0x7f915c00f690 comp rx=0 tx=0).stop 2023-11-27T15:21:06.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.760+0000 7f9166f9d700 1 -- 172.21.15.133:0/2390870284 shutdown_connections 2023-11-27T15:21:06.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.760+0000 7f9166f9d700 1 --2- 172.21.15.133:0/2390870284 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f914c0a1260 0x7f914c0a3710 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.760+0000 7f9166f9d700 1 --2- 172.21.15.133:0/2390870284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9160101c90 0x7f916019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.760+0000 7f9166f9d700 1 --2- 172.21.15.133:0/2390870284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9160100050 0x7f9160197fc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.760+0000 7f9166f9d700 1 --2- 172.21.15.133:0/2390870284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91601012e0 0x7f9160198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.760+0000 7f9166f9d700 1 -- 172.21.15.133:0/2390870284 >> 172.21.15.133:0/2390870284 conn(0x7f91600fb650 msgr2=0x7f91600fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:06.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.760+0000 7f9166f9d700 1 -- 172.21.15.133:0/2390870284 shutdown_connections 2023-11-27T15:21:06.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.760+0000 7f9166f9d700 1 -- 172.21.15.133:0/2390870284 wait complete. 2023-11-27T15:21:06.783 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:21:06.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.978+0000 7fb6171ff700 1 -- 172.21.15.133:0/2343104532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610100290 msgr2=0x7fb6101006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:06.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.978+0000 7fb6171ff700 1 --2- 172.21.15.133:0/2343104532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610100290 0x7fb6101006a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb604009990 tx=0x7fb604009ca0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.978+0000 7fb6171ff700 1 -- 172.21.15.133:0/2343104532 shutdown_connections 2023-11-27T15:21:06.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.978+0000 7fb6171ff700 1 --2- 172.21.15.133:0/2343104532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb610101e10 0x7fb6101086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.978+0000 7fb6171ff700 1 --2- 172.21.15.133:0/2343104532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610100290 0x7fb6101006a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.978+0000 7fb6171ff700 1 --2- 172.21.15.133:0/2343104532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb610101480 0x7fb6101018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.978+0000 7fb6171ff700 1 -- 172.21.15.133:0/2343104532 >> 172.21.15.133:0/2343104532 conn(0x7fb6100fb870 msgr2=0x7fb6100fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:06.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.978+0000 7fb6171ff700 1 -- 172.21.15.133:0/2343104532 shutdown_connections 2023-11-27T15:21:06.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.978+0000 7fb6171ff700 1 -- 172.21.15.133:0/2343104532 wait complete. 2023-11-27T15:21:06.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.979+0000 7fb6171ff700 1 Processor -- start 2023-11-27T15:21:06.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.979+0000 7fb6171ff700 1 -- start start 2023-11-27T15:21:06.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.979+0000 7fb6171ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb610100290 0x7fb6100748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:06.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.979+0000 7fb6171ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb610101480 0x7fb610072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:06.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.979+0000 7fb6171ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610101e10 0x7fb610073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.979+0000 7fb6171ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb610073bc0 con 0x7fb610101e10 2023-11-27T15:21:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.979+0000 7fb6171ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb610073d30 con 0x7fb610101480 2023-11-27T15:21:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.979+0000 7fb6171ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6101a0020 con 0x7fb610100290 2023-11-27T15:21:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.980+0000 7fb61579c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610101e10 0x7fb610073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.980+0000 7fb60ffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb610101480 0x7fb610072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.980+0000 7fb60ffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb610101480 0x7fb610072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:53586/0 (socket says 172.21.15.133:53586) 2023-11-27T15:21:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.980+0000 7fb60ffff700 1 -- 172.21.15.133:0/558405952 learned_addr learned my addr 172.21.15.133:0/558405952 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.980+0000 7fb61579c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610101e10 0x7fb610073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36034/0 (socket says 172.21.15.133:36034) 2023-11-27T15:21:06.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.980+0000 7fb614f9b700 1 --2- 172.21.15.133:0/558405952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb610100290 0x7fb6100748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:06.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.980+0000 7fb60ffff700 1 -- 172.21.15.133:0/558405952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb610100290 msgr2=0x7fb6100748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:06.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.980+0000 7fb60ffff700 1 --2- 172.21.15.133:0/558405952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb610100290 0x7fb6100748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.980+0000 7fb60ffff700 1 -- 172.21.15.133:0/558405952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610101e10 msgr2=0x7fb610073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:06.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.980+0000 7fb60ffff700 1 --2- 172.21.15.133:0/558405952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610101e10 0x7fb610073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:06.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.980+0000 7fb60ffff700 1 -- 172.21.15.133:0/558405952 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6040095f0 con 0x7fb610101480 2023-11-27T15:21:06.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.980+0000 7fb61579c700 1 --2- 172.21.15.133:0/558405952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610101e10 0x7fb610073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:06.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.980+0000 7fb60ffff700 1 --2- 172.21.15.133:0/558405952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb610101480 0x7fb610072ee0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00d950 tx=0x7fb5fc00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:06.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.981+0000 7fb60dffb700 1 -- 172.21.15.133:0/558405952 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc00f890 con 0x7fb610101480 2023-11-27T15:21:06.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.981+0000 7fb6171ff700 1 -- 172.21.15.133:0/558405952 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61010bc40 con 0x7fb610101480 2023-11-27T15:21:06.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.981+0000 7fb60dffb700 1 -- 172.21.15.133:0/558405952 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5fc010460 con 0x7fb610101480 2023-11-27T15:21:06.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.981+0000 7fb60dffb700 1 -- 172.21.15.133:0/558405952 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc00e610 con 0x7fb610101480 2023-11-27T15:21:06.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.981+0000 7fb6171ff700 1 -- 172.21.15.133:0/558405952 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb61010bab0 con 0x7fb610101480 2023-11-27T15:21:06.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.982+0000 7fb6171ff700 1 -- 172.21.15.133:0/558405952 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb610107210 con 0x7fb610101480 2023-11-27T15:21:06.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.984+0000 7fb60dffb700 1 -- 172.21.15.133:0/558405952 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fb5fc0108d0 con 0x7fb610101480 2023-11-27T15:21:06.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.985+0000 7fb60dffb700 1 --2- 172.21.15.133:0/558405952 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb6000a11e0 0x7fb6000a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:06.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.985+0000 7fb60dffb700 1 -- 172.21.15.133:0/558405952 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb5fc0ccbf0 con 0x7fb610101480 2023-11-27T15:21:06.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.985+0000 7fb614f9b700 1 --2- 172.21.15.133:0/558405952 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb6000a11e0 0x7fb6000a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:06.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.986+0000 7fb614f9b700 1 --2- 172.21.15.133:0/558405952 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb6000a11e0 0x7fb6000a3690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb60400b5c0 tx=0x7fb604005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:06.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:06.989+0000 7fb60dffb700 1 -- 172.21.15.133:0/558405952 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb5fc095e70 con 0x7fb610101480 2023-11-27T15:21:07.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:06 smithi177 ceph-mon[131504]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:21:07.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:06 smithi177 ceph-mon[131504]: pgmap v292: 129 pgs: 2 peering, 127 active+clean; 583 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 1.6 KiB/s wr, 6 op/s 2023-11-27T15:21:07.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:06 smithi177 ceph-mon[131504]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:21:07.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:06 smithi177 ceph-mon[131504]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-11-27T15:21:07.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:06 smithi174 ceph-mon[131110]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:21:07.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:06 smithi174 ceph-mon[131110]: pgmap v292: 129 pgs: 2 peering, 127 active+clean; 583 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 1.6 KiB/s wr, 6 op/s 2023-11-27T15:21:07.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:06 smithi174 ceph-mon[131110]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:21:07.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:06 smithi174 ceph-mon[131110]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-11-27T15:21:07.238 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.233+0000 7fb6171ff700 1 -- 172.21.15.133:0/558405952 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb610061700 con 0x7fb610101480 2023-11-27T15:21:07.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.234+0000 7fb60dffb700 1 -- 172.21.15.133:0/558405952 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10829 (secure 0 0 0) 0x7fb5fc016090 con 0x7fb610101480 2023-11-27T15:21:07.239 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:21:07.239 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14493,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2381290259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2381290259},{"type":"v1","addr":"172.21.15.133:6839","nonce":2381290259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1704875895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1704875895},{"type":"v1","addr":"172.21.15.133:6841","nonce":1704875895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:20:33.201023+0000","modified":"2023-11-27T15:21:04.794526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24521,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3492215689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3492215689},{"type":"v1","addr":"172.21.15.177:6837","nonce":3492215689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6839/2781994447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2781994447},{"type":"v1","addr":"172.21.15.174:6839","nonce":2781994447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:21:07.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.236+0000 7fb6171ff700 1 -- 172.21.15.133:0/558405952 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb6000a11e0 msgr2=0x7fb6000a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:07.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.236+0000 7fb6171ff700 1 --2- 172.21.15.133:0/558405952 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb6000a11e0 0x7fb6000a3690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb60400b5c0 tx=0x7fb604005fd0 comp rx=0 tx=0).stop 2023-11-27T15:21:07.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.236+0000 7fb6171ff700 1 -- 172.21.15.133:0/558405952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb610101480 msgr2=0x7fb610072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:07.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.236+0000 7fb6171ff700 1 --2- 172.21.15.133:0/558405952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb610101480 0x7fb610072ee0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00d950 tx=0x7fb5fc00dc60 comp rx=0 tx=0).stop 2023-11-27T15:21:07.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.237+0000 7fb6171ff700 1 -- 172.21.15.133:0/558405952 shutdown_connections 2023-11-27T15:21:07.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.237+0000 7fb6171ff700 1 --2- 172.21.15.133:0/558405952 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb6000a11e0 0x7fb6000a3690 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:07.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.237+0000 7fb6171ff700 1 --2- 172.21.15.133:0/558405952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb610101e10 0x7fb610073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:07.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.237+0000 7fb6171ff700 1 --2- 172.21.15.133:0/558405952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb610100290 0x7fb6100748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:07.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.237+0000 7fb6171ff700 1 --2- 172.21.15.133:0/558405952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb610101480 0x7fb610072ee0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:07.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.237+0000 7fb6171ff700 1 -- 172.21.15.133:0/558405952 >> 172.21.15.133:0/558405952 conn(0x7fb6100fb870 msgr2=0x7fb6100fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:07.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.237+0000 7fb6171ff700 1 -- 172.21.15.133:0/558405952 shutdown_connections 2023-11-27T15:21:07.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.237+0000 7fb6171ff700 1 -- 172.21.15.133:0/558405952 wait complete. 2023-11-27T15:21:07.243 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 23 2023-11-27T15:21:07.258 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:06 smithi133 ceph-mon[97860]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:21:07.258 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:06 smithi133 ceph-mon[97860]: pgmap v292: 129 pgs: 2 peering, 127 active+clean; 583 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 1.6 KiB/s wr, 6 op/s 2023-11-27T15:21:07.258 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:06 smithi133 ceph-mon[97860]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:21:07.258 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:06 smithi133 ceph-mon[97860]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-11-27T15:21:07.260 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:21:07.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.452+0000 7f0e33fff700 1 -- 172.21.15.133:0/3015234823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c1002b0 msgr2=0x7f0e2c1006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:07.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.452+0000 7f0e33fff700 1 --2- 172.21.15.133:0/3015234823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c1002b0 0x7f0e2c1006c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009990 tx=0x7f0e28009ca0 comp rx=0 tx=0).stop 2023-11-27T15:21:07.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.452+0000 7f0e33fff700 1 -- 172.21.15.133:0/3015234823 shutdown_connections 2023-11-27T15:21:07.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.452+0000 7f0e33fff700 1 --2- 172.21.15.133:0/3015234823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e2c101e30 0x7f0e2c1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:07.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.452+0000 7f0e33fff700 1 --2- 172.21.15.133:0/3015234823 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c1002b0 0x7f0e2c1006c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:07.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.452+0000 7f0e33fff700 1 --2- 172.21.15.133:0/3015234823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e2c1014a0 0x7f0e2c1018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:07.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.452+0000 7f0e33fff700 1 -- 172.21.15.133:0/3015234823 >> 172.21.15.133:0/3015234823 conn(0x7f0e2c0fb890 msgr2=0x7f0e2c0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:07.458 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.452+0000 7f0e33fff700 1 -- 172.21.15.133:0/3015234823 shutdown_connections 2023-11-27T15:21:07.458 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.452+0000 7f0e33fff700 1 -- 172.21.15.133:0/3015234823 wait complete. 2023-11-27T15:21:07.459 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.453+0000 7f0e33fff700 1 Processor -- start 2023-11-27T15:21:07.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.453+0000 7f0e33fff700 1 -- start start 2023-11-27T15:21:07.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.453+0000 7f0e33fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c1002b0 0x7f0e2c1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:07.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.453+0000 7f0e33fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e2c1014a0 0x7f0e2c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:07.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.454+0000 7f0e33fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e2c101e30 0x7f0e2c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:07.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.454+0000 7f0e33fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c19fd60 con 0x7f0e2c1002b0 2023-11-27T15:21:07.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.454+0000 7f0e33fff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c19fea0 con 0x7f0e2c1014a0 2023-11-27T15:21:07.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.454+0000 7f0e33fff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c1a0030 con 0x7f0e2c101e30 2023-11-27T15:21:07.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.454+0000 7f0e31d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c1002b0 0x7f0e2c1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:07.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.454+0000 7f0e31d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c1002b0 0x7f0e2c1981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36050/0 (socket says 172.21.15.133:36050) 2023-11-27T15:21:07.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.454+0000 7f0e31d9b700 1 -- 172.21.15.133:0/1368723892 learned_addr learned my addr 172.21.15.133:0/1368723892 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:07.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.454+0000 7f0e3159a700 1 --2- 172.21.15.133:0/1368723892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e2c1014a0 0x7f0e2c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:07.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.454+0000 7f0e3259c700 1 --2- 172.21.15.133:0/1368723892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e2c101e30 0x7f0e2c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:07.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.454+0000 7f0e31d9b700 1 -- 172.21.15.133:0/1368723892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e2c101e30 msgr2=0x7f0e2c19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:07.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.454+0000 7f0e31d9b700 1 --2- 172.21.15.133:0/1368723892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e2c101e30 0x7f0e2c19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:07.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.454+0000 7f0e31d9b700 1 -- 172.21.15.133:0/1368723892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e2c1014a0 msgr2=0x7f0e2c198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:07.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.454+0000 7f0e31d9b700 1 --2- 172.21.15.133:0/1368723892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e2c1014a0 0x7f0e2c198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:07.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.454+0000 7f0e31d9b700 1 -- 172.21.15.133:0/1368723892 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e280095f0 con 0x7f0e2c1002b0 2023-11-27T15:21:07.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.455+0000 7f0e31d9b700 1 --2- 172.21.15.133:0/1368723892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c1002b0 0x7f0e2c1981e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0e28004730 tx=0x7f0e28004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:07.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.455+0000 7f0e22ffd700 1 -- 172.21.15.133:0/1368723892 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2801d070 con 0x7f0e2c1002b0 2023-11-27T15:21:07.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.455+0000 7f0e22ffd700 1 -- 172.21.15.133:0/1368723892 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e28021410 con 0x7f0e2c1002b0 2023-11-27T15:21:07.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.455+0000 7f0e22ffd700 1 -- 172.21.15.133:0/1368723892 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28017650 con 0x7f0e2c1002b0 2023-11-27T15:21:07.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.455+0000 7f0e33fff700 1 -- 172.21.15.133:0/1368723892 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e2c10bc80 con 0x7f0e2c1002b0 2023-11-27T15:21:07.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.455+0000 7f0e33fff700 1 -- 172.21.15.133:0/1368723892 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2c10baf0 con 0x7f0e2c1002b0 2023-11-27T15:21:07.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.456+0000 7f0e33fff700 1 -- 172.21.15.133:0/1368723892 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2c192910 con 0x7f0e2c1002b0 2023-11-27T15:21:07.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.458+0000 7f0e22ffd700 1 -- 172.21.15.133:0/1368723892 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f0e28021580 con 0x7f0e2c1002b0 2023-11-27T15:21:07.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.459+0000 7f0e22ffd700 1 --2- 172.21.15.133:0/1368723892 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0e180a1260 0x7f0e180a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:07.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.459+0000 7f0e22ffd700 1 -- 172.21.15.133:0/1368723892 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0e280ce170 con 0x7f0e2c1002b0 2023-11-27T15:21:07.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.460+0000 7f0e3159a700 1 --2- 172.21.15.133:0/1368723892 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0e180a1260 0x7f0e180a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:07.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.460+0000 7f0e3159a700 1 --2- 172.21.15.133:0/1368723892 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0e180a1260 0x7f0e180a3710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009c60 tx=0x7f0e1c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:07.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.463+0000 7f0e22ffd700 1 -- 172.21.15.133:0/1368723892 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0e28097480 con 0x7f0e2c1002b0 2023-11-27T15:21:07.652 INFO:journalctl@ceph.mds.b.smithi174.stdout:Nov 27 15:21:07 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-b[156420]: 2023-11-27T15:21:07.151+0000 7ff29549d700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-11-27T15:21:07.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.663+0000 7f0e33fff700 1 -- 172.21.15.133:0/1368723892 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e2c02cfe0 con 0x7f0e2c1002b0 2023-11-27T15:21:07.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.665+0000 7f0e22ffd700 1 -- 172.21.15.133:0/1368723892 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24465 (secure 0 0 0) 0x7f0e2809aaa0 con 0x7f0e2c1002b0 2023-11-27T15:21:07.670 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:21:07.670 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":90,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:21:02.774978+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:21:07.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.667+0000 7f0e33fff700 1 -- 172.21.15.133:0/1368723892 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0e180a1260 msgr2=0x7f0e180a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:07.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.667+0000 7f0e33fff700 1 --2- 172.21.15.133:0/1368723892 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0e180a1260 0x7f0e180a3710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009c60 tx=0x7f0e1c009260 comp rx=0 tx=0).stop 2023-11-27T15:21:07.673 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.667+0000 7f0e33fff700 1 -- 172.21.15.133:0/1368723892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c1002b0 msgr2=0x7f0e2c1981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:07.673 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.667+0000 7f0e33fff700 1 --2- 172.21.15.133:0/1368723892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c1002b0 0x7f0e2c1981e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0e28004730 tx=0x7f0e28004760 comp rx=0 tx=0).stop 2023-11-27T15:21:07.673 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.667+0000 7f0e33fff700 1 -- 172.21.15.133:0/1368723892 shutdown_connections 2023-11-27T15:21:07.673 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.667+0000 7f0e33fff700 1 --2- 172.21.15.133:0/1368723892 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0e180a1260 0x7f0e180a3710 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:07.673 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.667+0000 7f0e33fff700 1 --2- 172.21.15.133:0/1368723892 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e2c101e30 0x7f0e2c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:07.673 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.667+0000 7f0e33fff700 1 --2- 172.21.15.133:0/1368723892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e2c1002b0 0x7f0e2c1981e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:07.673 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.667+0000 7f0e33fff700 1 --2- 172.21.15.133:0/1368723892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e2c1014a0 0x7f0e2c198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:07.673 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.667+0000 7f0e33fff700 1 -- 172.21.15.133:0/1368723892 >> 172.21.15.133:0/1368723892 conn(0x7f0e2c0fb890 msgr2=0x7f0e2c106e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:07.673 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.668+0000 7f0e33fff700 1 -- 172.21.15.133:0/1368723892 shutdown_connections 2023-11-27T15:21:07.673 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.668+0000 7f0e33fff700 1 -- 172.21.15.133:0/1368723892 wait complete. 2023-11-27T15:21:07.691 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-27T15:21:07.691 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-27T15:21:07.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.886+0000 7f93f45ff700 1 -- 172.21.15.133:0/3332686107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93ec0ff170 msgr2=0x7f93ec0ff580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:07.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.886+0000 7f93f45ff700 1 --2- 172.21.15.133:0/3332686107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93ec0ff170 0x7f93ec0ff580 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f93e800b0a0 tx=0x7f93e800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:21:07.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.886+0000 7f93f45ff700 1 -- 172.21.15.133:0/3332686107 shutdown_connections 2023-11-27T15:21:07.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.886+0000 7f93f45ff700 1 --2- 172.21.15.133:0/3332686107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93ec100db0 0x7f93ec107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:07.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.886+0000 7f93f45ff700 1 --2- 172.21.15.133:0/3332686107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93ec0ff170 0x7f93ec0ff580 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:07.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.886+0000 7f93f45ff700 1 --2- 172.21.15.133:0/3332686107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93ec100400 0x7f93ec100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:07.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.886+0000 7f93f45ff700 1 -- 172.21.15.133:0/3332686107 >> 172.21.15.133:0/3332686107 conn(0x7f93ec0fa750 msgr2=0x7f93ec0fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:07.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.886+0000 7f93f45ff700 1 -- 172.21.15.133:0/3332686107 shutdown_connections 2023-11-27T15:21:07.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.886+0000 7f93f45ff700 1 -- 172.21.15.133:0/3332686107 wait complete. 2023-11-27T15:21:07.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.887+0000 7f93f45ff700 1 Processor -- start 2023-11-27T15:21:07.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.887+0000 7f93f45ff700 1 -- start start 2023-11-27T15:21:07.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.887+0000 7f93f45ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93ec0ff170 0x7f93ec198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:07.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.887+0000 7f93f45ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93ec100400 0x7f93ec198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:07.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.887+0000 7f93f45ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93ec100db0 0x7f93ec19f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:07.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.887+0000 7f93f45ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec19fe90 con 0x7f93ec100400 2023-11-27T15:21:07.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.887+0000 7f93f45ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec19ffd0 con 0x7f93ec0ff170 2023-11-27T15:21:07.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.887+0000 7f93f45ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec1a0160 con 0x7f93ec100db0 2023-11-27T15:21:07.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.888+0000 7f93f1b9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93ec100400 0x7f93ec198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:07.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.888+0000 7f93f1b9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93ec100400 0x7f93ec198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36070/0 (socket says 172.21.15.133:36070) 2023-11-27T15:21:07.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.888+0000 7f93f1b9a700 1 -- 172.21.15.133:0/1309542847 learned_addr learned my addr 172.21.15.133:0/1309542847 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:07.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.888+0000 7f93f239b700 1 --2- 172.21.15.133:0/1309542847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93ec0ff170 0x7f93ec198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:07.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.888+0000 7f93f2b9c700 1 --2- 172.21.15.133:0/1309542847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93ec100db0 0x7f93ec19f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:07.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.888+0000 7f93f1b9a700 1 -- 172.21.15.133:0/1309542847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93ec100db0 msgr2=0x7f93ec19f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:07.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.888+0000 7f93f1b9a700 1 --2- 172.21.15.133:0/1309542847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93ec100db0 0x7f93ec19f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.888+0000 7f93f1b9a700 1 -- 172.21.15.133:0/1309542847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93ec0ff170 msgr2=0x7f93ec198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.888+0000 7f93f1b9a700 1 --2- 172.21.15.133:0/1309542847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93ec0ff170 0x7f93ec198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.888+0000 7f93f1b9a700 1 -- 172.21.15.133:0/1309542847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e8009d00 con 0x7f93ec100400 2023-11-27T15:21:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.888+0000 7f93f2b9c700 1 --2- 172.21.15.133:0/1309542847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93ec100db0 0x7f93ec19f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:21:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.888+0000 7f93f239b700 1 --2- 172.21.15.133:0/1309542847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93ec0ff170 0x7f93ec198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.888+0000 7f93f1b9a700 1 --2- 172.21.15.133:0/1309542847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93ec100400 0x7f93ec198850 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00e9a0 tx=0x7f93dc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.888+0000 7f93e37fe700 1 -- 172.21.15.133:0/1309542847 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc009790 con 0x7f93ec100400 2023-11-27T15:21:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.888+0000 7f93e37fe700 1 -- 172.21.15.133:0/1309542847 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93dc00cbd0 con 0x7f93ec100400 2023-11-27T15:21:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.888+0000 7f93f45ff700 1 -- 172.21.15.133:0/1309542847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ec10ab60 con 0x7f93ec100400 2023-11-27T15:21:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.888+0000 7f93e37fe700 1 -- 172.21.15.133:0/1309542847 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc018680 con 0x7f93ec100400 2023-11-27T15:21:07.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.889+0000 7f93f45ff700 1 -- 172.21.15.133:0/1309542847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ec10a9d0 con 0x7f93ec100400 2023-11-27T15:21:07.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.890+0000 7f93f45ff700 1 -- 172.21.15.133:0/1309542847 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93ec105100 con 0x7f93ec100400 2023-11-27T15:21:07.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.891+0000 7f93e37fe700 1 -- 172.21.15.133:0/1309542847 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f93dc00cd40 con 0x7f93ec100400 2023-11-27T15:21:07.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.893+0000 7f93e37fe700 1 --2- 172.21.15.133:0/1309542847 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f93d80a1260 0x7f93d80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:07.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.894+0000 7f93e37fe700 1 -- 172.21.15.133:0/1309542847 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f93dc014070 con 0x7f93ec100400 2023-11-27T15:21:07.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.894+0000 7f93f239b700 1 --2- 172.21.15.133:0/1309542847 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f93d80a1260 0x7f93d80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:07.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.894+0000 7f93f239b700 1 --2- 172.21.15.133:0/1309542847 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f93d80a1260 0x7f93d80a3710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f93e800b780 tx=0x7f93e8006210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:07.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:07.898+0000 7f93e37fe700 1 -- 172.21.15.133:0/1309542847 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f93dc051730 con 0x7f93ec100400 2023-11-27T15:21:08.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:07 smithi133 ceph-mon[97860]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:21:08.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:07 smithi133 ceph-mon[97860]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:21:08.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:07 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/558405952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:21:08.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:07 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1368723892' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:21:08.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:07 smithi177 ceph-mon[131504]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:21:08.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:07 smithi177 ceph-mon[131504]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:21:08.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:07 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/558405952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:21:08.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:07 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1368723892' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:21:08.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.093+0000 7f93f45ff700 1 -- 172.21.15.133:0/1309542847 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f93ec061c60 con 0x7f93d80a1260 2023-11-27T15:21:08.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.108+0000 7f93e37fe700 1 -- 172.21.15.133:0/1309542847 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422572 (secure 0 0 0) 0x7f93ec061c60 con 0x7f93d80a1260 2023-11-27T15:21:08.119 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:21:08.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.114+0000 7f93f45ff700 1 -- 172.21.15.133:0/1309542847 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f93d80a1260 msgr2=0x7f93d80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:08.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.114+0000 7f93f45ff700 1 --2- 172.21.15.133:0/1309542847 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f93d80a1260 0x7f93d80a3710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f93e800b780 tx=0x7f93e8006210 comp rx=0 tx=0).stop 2023-11-27T15:21:08.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.114+0000 7f93f45ff700 1 -- 172.21.15.133:0/1309542847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93ec100400 msgr2=0x7f93ec198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:08.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.114+0000 7f93f45ff700 1 --2- 172.21.15.133:0/1309542847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93ec100400 0x7f93ec198850 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00e9a0 tx=0x7f93dc00ecb0 comp rx=0 tx=0).stop 2023-11-27T15:21:08.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.114+0000 7f93f45ff700 1 -- 172.21.15.133:0/1309542847 shutdown_connections 2023-11-27T15:21:08.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.114+0000 7f93f45ff700 1 --2- 172.21.15.133:0/1309542847 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f93d80a1260 0x7f93d80a3710 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:08.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.114+0000 7f93f45ff700 1 --2- 172.21.15.133:0/1309542847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f93ec100db0 0x7f93ec19f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:08.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.114+0000 7f93f45ff700 1 --2- 172.21.15.133:0/1309542847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93ec0ff170 0x7f93ec198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:08.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.114+0000 7f93f45ff700 1 --2- 172.21.15.133:0/1309542847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93ec100400 0x7f93ec198850 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:08.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.114+0000 7f93f45ff700 1 -- 172.21.15.133:0/1309542847 >> 172.21.15.133:0/1309542847 conn(0x7f93ec0fa750 msgr2=0x7f93ec0fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:08.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.114+0000 7f93f45ff700 1 -- 172.21.15.133:0/1309542847 shutdown_connections 2023-11-27T15:21:08.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.114+0000 7f93f45ff700 1 -- 172.21.15.133:0/1309542847 wait complete. 2023-11-27T15:21:08.124 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2023-11-27T15:21:08.138 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2023-11-27T15:21:07.716069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632408,"num_objects":57,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":142,"num_read_kb":141,"num_write":313,"num_write_kb":3012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":253,"ondisk_log_size":253,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":82424,"kb_used_data":13656,"kb_used_omap":0,"kb_used_meta":68672,"kb_avail":1124859400,"statfs":{"total":1151940427776,"available":1151856025600,"internally_reserved":0,"allocated":13983744,"data_stored":7687004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":70320128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":37448,"num_objects":33,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":16,"num_read_kb":32,"num_write":83,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002256"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.933701+0000","last_change":"2023-11-27T15:20:33.236352+0000","last_active":"2023-11-27T15:21:02.933701+0000","last_peered":"2023-11-27T15:21:02.933701+0000","last_clean":"2023-11-27T15:21:02.933701+0000","last_became_active":"2023-11-27T15:20:33.235974+0000","last_became_peered":"2023-11-27T15:20:33.235974+0000","last_unstale":"2023-11-27T15:21:02.933701+0000","last_undegraded":"2023-11-27T15:21:02.933701+0000","last_fullsized":"2023-11-27T15:21:02.933701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:42:55.142804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786021+0000","last_change":"2023-11-27T15:20:33.234882+0000","last_active":"2023-11-27T15:21:02.786021+0000","last_peered":"2023-11-27T15:21:02.786021+0000","last_clean":"2023-11-27T15:21:02.786021+0000","last_became_active":"2023-11-27T15:20:33.231062+0000","last_became_peered":"2023-11-27T15:20:33.231062+0000","last_unstale":"2023-11-27T15:21:02.786021+0000","last_undegraded":"2023-11-27T15:21:02.786021+0000","last_fullsized":"2023-11-27T15:21:02.786021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:18:27.469196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785364+0000","last_change":"2023-11-27T15:20:33.238280+0000","last_active":"2023-11-27T15:21:02.785364+0000","last_peered":"2023-11-27T15:21:02.785364+0000","last_clean":"2023-11-27T15:21:02.785364+0000","last_became_active":"2023-11-27T15:20:33.237602+0000","last_became_peered":"2023-11-27T15:20:33.237602+0000","last_unstale":"2023-11-27T15:21:02.785364+0000","last_undegraded":"2023-11-27T15:21:02.785364+0000","last_fullsized":"2023-11-27T15:21:02.785364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:28:48.211669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784641+0000","last_change":"2023-11-27T15:20:33.217909+0000","last_active":"2023-11-27T15:21:02.784641+0000","last_peered":"2023-11-27T15:21:02.784641+0000","last_clean":"2023-11-27T15:21:02.784641+0000","last_became_active":"2023-11-27T15:20:33.217511+0000","last_became_peered":"2023-11-27T15:20:33.217511+0000","last_unstale":"2023-11-27T15:21:02.784641+0000","last_undegraded":"2023-11-27T15:21:02.784641+0000","last_fullsized":"2023-11-27T15:21:02.784641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:37:24.759082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782641+0000","last_change":"2023-11-27T15:20:33.246630+0000","last_active":"2023-11-27T15:21:02.782641+0000","last_peered":"2023-11-27T15:21:02.782641+0000","last_clean":"2023-11-27T15:21:02.782641+0000","last_became_active":"2023-11-27T15:20:33.246249+0000","last_became_peered":"2023-11-27T15:20:33.246249+0000","last_unstale":"2023-11-27T15:21:02.782641+0000","last_undegraded":"2023-11-27T15:21:02.782641+0000","last_fullsized":"2023-11-27T15:21:02.782641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:14:48.110743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932766+0000","last_change":"2023-11-27T15:20:33.236280+0000","last_active":"2023-11-27T15:21:02.932766+0000","last_peered":"2023-11-27T15:21:02.932766+0000","last_clean":"2023-11-27T15:21:02.932766+0000","last_became_active":"2023-11-27T15:20:33.235983+0000","last_became_peered":"2023-11-27T15:20:33.235983+0000","last_unstale":"2023-11-27T15:21:02.932766+0000","last_undegraded":"2023-11-27T15:21:02.932766+0000","last_fullsized":"2023-11-27T15:21:02.932766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:32:27.538426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779828+0000","last_change":"2023-11-27T15:20:33.223085+0000","last_active":"2023-11-27T15:21:02.779828+0000","last_peered":"2023-11-27T15:21:02.779828+0000","last_clean":"2023-11-27T15:21:02.779828+0000","last_became_active":"2023-11-27T15:20:33.222809+0000","last_became_peered":"2023-11-27T15:20:33.222809+0000","last_unstale":"2023-11-27T15:21:02.779828+0000","last_undegraded":"2023-11-27T15:21:02.779828+0000","last_fullsized":"2023-11-27T15:21:02.779828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:46:23.567184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783786+0000","last_change":"2023-11-27T15:20:33.226217+0000","last_active":"2023-11-27T15:21:02.783786+0000","last_peered":"2023-11-27T15:21:02.783786+0000","last_clean":"2023-11-27T15:21:02.783786+0000","last_became_active":"2023-11-27T15:20:33.225856+0000","last_became_peered":"2023-11-27T15:20:33.225856+0000","last_unstale":"2023-11-27T15:21:02.783786+0000","last_undegraded":"2023-11-27T15:21:02.783786+0000","last_fullsized":"2023-11-27T15:21:02.783786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:41:31.773939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.795641+0000","last_change":"2023-11-27T15:20:33.227554+0000","last_active":"2023-11-27T15:21:02.795641+0000","last_peered":"2023-11-27T15:21:02.795641+0000","last_clean":"2023-11-27T15:21:02.795641+0000","last_became_active":"2023-11-27T15:20:33.227222+0000","last_became_peered":"2023-11-27T15:20:33.227222+0000","last_unstale":"2023-11-27T15:21:02.795641+0000","last_undegraded":"2023-11-27T15:21:02.795641+0000","last_fullsized":"2023-11-27T15:21:02.795641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:34:15.927242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.933040+0000","last_change":"2023-11-27T15:20:33.230528+0000","last_active":"2023-11-27T15:21:02.933040+0000","last_peered":"2023-11-27T15:21:02.933040+0000","last_clean":"2023-11-27T15:21:02.933040+0000","last_became_active":"2023-11-27T15:20:33.230148+0000","last_became_peered":"2023-11-27T15:20:33.230148+0000","last_unstale":"2023-11-27T15:21:02.933040+0000","last_undegraded":"2023-11-27T15:21:02.933040+0000","last_fullsized":"2023-11-27T15:21:02.933040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:56:46.117728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780265+0000","last_change":"2023-11-27T15:20:33.228098+0000","last_active":"2023-11-27T15:21:02.780265+0000","last_peered":"2023-11-27T15:21:02.780265+0000","last_clean":"2023-11-27T15:21:02.780265+0000","last_became_active":"2023-11-27T15:20:33.227760+0000","last_became_peered":"2023-11-27T15:20:33.227760+0000","last_unstale":"2023-11-27T15:21:02.780265+0000","last_undegraded":"2023-11-27T15:21:02.780265+0000","last_fullsized":"2023-11-27T15:21:02.780265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:27:54.894445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.935458+0000","last_change":"2023-11-27T15:21:02.934583+0000","last_active":"2023-11-27T15:21:02.935458+0000","last_peered":"2023-11-27T15:21:02.935458+0000","last_clean":"2023-11-27T15:21:02.935458+0000","last_became_active":"2023-11-27T15:21:02.933631+0000","last_became_peered":"2023-11-27T15:21:02.933631+0000","last_unstale":"2023-11-27T15:21:02.935458+0000","last_undegraded":"2023-11-27T15:21:02.935458+0000","last_fullsized":"2023-11-27T15:21:02.935458+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:45:15.679810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783578+0000","last_change":"2023-11-27T15:20:33.231255+0000","last_active":"2023-11-27T15:21:02.783578+0000","last_peered":"2023-11-27T15:21:02.783578+0000","last_clean":"2023-11-27T15:21:02.783578+0000","last_became_active":"2023-11-27T15:20:33.230867+0000","last_became_peered":"2023-11-27T15:20:33.230867+0000","last_unstale":"2023-11-27T15:21:02.783578+0000","last_undegraded":"2023-11-27T15:21:02.783578+0000","last_fullsized":"2023-11-27T15:21:02.783578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:58:03.899781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785072+0000","last_change":"2023-11-27T15:20:33.230306+0000","last_active":"2023-11-27T15:21:02.785072+0000","last_peered":"2023-11-27T15:21:02.785072+0000","last_clean":"2023-11-27T15:21:02.785072+0000","last_became_active":"2023-11-27T15:20:33.229892+0000","last_became_peered":"2023-11-27T15:20:33.229892+0000","last_unstale":"2023-11-27T15:21:02.785072+0000","last_undegraded":"2023-11-27T15:21:02.785072+0000","last_fullsized":"2023-11-27T15:21:02.785072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:24:55.365713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785143+0000","last_change":"2023-11-27T15:20:33.220592+0000","last_active":"2023-11-27T15:21:02.785143+0000","last_peered":"2023-11-27T15:21:02.785143+0000","last_clean":"2023-11-27T15:21:02.785143+0000","last_became_active":"2023-11-27T15:20:33.220258+0000","last_became_peered":"2023-11-27T15:20:33.220258+0000","last_unstale":"2023-11-27T15:21:02.785143+0000","last_undegraded":"2023-11-27T15:21:02.785143+0000","last_fullsized":"2023-11-27T15:21:02.785143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:48:48.921465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.794878+0000","last_change":"2023-11-27T15:20:33.245041+0000","last_active":"2023-11-27T15:21:02.794878+0000","last_peered":"2023-11-27T15:21:02.794878+0000","last_clean":"2023-11-27T15:21:02.794878+0000","last_became_active":"2023-11-27T15:20:33.244759+0000","last_became_peered":"2023-11-27T15:20:33.244759+0000","last_unstale":"2023-11-27T15:21:02.794878+0000","last_undegraded":"2023-11-27T15:21:02.794878+0000","last_fullsized":"2023-11-27T15:21:02.794878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:31:00.164868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.934087+0000","last_change":"2023-11-27T15:20:33.243288+0000","last_active":"2023-11-27T15:21:02.934087+0000","last_peered":"2023-11-27T15:21:02.934087+0000","last_clean":"2023-11-27T15:21:02.934087+0000","last_became_active":"2023-11-27T15:20:33.242989+0000","last_became_peered":"2023-11-27T15:20:33.242989+0000","last_unstale":"2023-11-27T15:21:02.934087+0000","last_undegraded":"2023-11-27T15:21:02.934087+0000","last_fullsized":"2023-11-27T15:21:02.934087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:25:53.302075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779978+0000","last_change":"2023-11-27T15:20:33.229613+0000","last_active":"2023-11-27T15:21:02.779978+0000","last_peered":"2023-11-27T15:21:02.779978+0000","last_clean":"2023-11-27T15:21:02.779978+0000","last_became_active":"2023-11-27T15:20:33.227747+0000","last_became_peered":"2023-11-27T15:20:33.227747+0000","last_unstale":"2023-11-27T15:21:02.779978+0000","last_undegraded":"2023-11-27T15:21:02.779978+0000","last_fullsized":"2023-11-27T15:21:02.779978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:18:24.422450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780242+0000","last_change":"2023-11-27T15:20:33.227004+0000","last_active":"2023-11-27T15:21:02.780242+0000","last_peered":"2023-11-27T15:21:02.780242+0000","last_clean":"2023-11-27T15:21:02.780242+0000","last_became_active":"2023-11-27T15:20:33.226703+0000","last_became_peered":"2023-11-27T15:20:33.226703+0000","last_unstale":"2023-11-27T15:21:02.780242+0000","last_undegraded":"2023-11-27T15:21:02.780242+0000","last_fullsized":"2023-11-27T15:21:02.780242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:18:04.531443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779865+0000","last_change":"2023-11-27T15:20:37.682021+0000","last_active":"2023-11-27T15:21:02.779865+0000","last_peered":"2023-11-27T15:21:02.779865+0000","last_clean":"2023-11-27T15:21:02.779865+0000","last_became_active":"2023-11-27T15:20:37.681600+0000","last_became_peered":"2023-11-27T15:20:37.681600+0000","last_unstale":"2023-11-27T15:21:02.779865+0000","last_undegraded":"2023-11-27T15:21:02.779865+0000","last_fullsized":"2023-11-27T15:21:02.779865+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:27:41.540353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.801130+0000","last_change":"2023-11-27T15:20:32.657542+0000","last_active":"2023-11-27T15:21:03.801130+0000","last_peered":"2023-11-27T15:21:03.801130+0000","last_clean":"2023-11-27T15:21:03.801130+0000","last_became_active":"2023-11-27T15:20:32.657158+0000","last_became_peered":"2023-11-27T15:20:32.657158+0000","last_unstale":"2023-11-27T15:21:03.801130+0000","last_undegraded":"2023-11-27T15:21:03.801130+0000","last_fullsized":"2023-11-27T15:21:03.801130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:01:39.181861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.935289+0000","last_change":"2023-11-27T15:21:02.933684+0000","last_active":"2023-11-27T15:21:02.935289+0000","last_peered":"2023-11-27T15:21:02.935289+0000","last_clean":"2023-11-27T15:21:02.935289+0000","last_became_active":"2023-11-27T15:21:02.933215+0000","last_became_peered":"2023-11-27T15:21:02.933215+0000","last_unstale":"2023-11-27T15:21:02.935289+0000","last_undegraded":"2023-11-27T15:21:02.935289+0000","last_fullsized":"2023-11-27T15:21:02.935289+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:59:39.553465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783397+0000","last_change":"2023-11-27T15:20:37.666705+0000","last_active":"2023-11-27T15:21:02.783397+0000","last_peered":"2023-11-27T15:21:02.783397+0000","last_clean":"2023-11-27T15:21:02.783397+0000","last_became_active":"2023-11-27T15:20:37.666236+0000","last_became_peered":"2023-11-27T15:20:37.666236+0000","last_unstale":"2023-11-27T15:21:02.783397+0000","last_undegraded":"2023-11-27T15:21:02.783397+0000","last_fullsized":"2023-11-27T15:21:02.783397+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:49:31.365357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785700+0000","last_change":"2023-11-27T15:20:32.663915+0000","last_active":"2023-11-27T15:21:02.785700+0000","last_peered":"2023-11-27T15:21:02.785700+0000","last_clean":"2023-11-27T15:21:02.785700+0000","last_became_active":"2023-11-27T15:20:32.663069+0000","last_became_peered":"2023-11-27T15:20:32.663069+0000","last_unstale":"2023-11-27T15:21:02.785700+0000","last_undegraded":"2023-11-27T15:21:02.785700+0000","last_fullsized":"2023-11-27T15:21:02.785700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:15:25.522191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785737+0000","last_change":"2023-11-27T15:20:33.238150+0000","last_active":"2023-11-27T15:21:02.785737+0000","last_peered":"2023-11-27T15:21:02.785737+0000","last_clean":"2023-11-27T15:21:02.785737+0000","last_became_active":"2023-11-27T15:20:33.236972+0000","last_became_peered":"2023-11-27T15:20:33.236972+0000","last_unstale":"2023-11-27T15:21:02.785737+0000","last_undegraded":"2023-11-27T15:21:02.785737+0000","last_fullsized":"2023-11-27T15:21:02.785737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:32:03.275170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783667+0000","last_change":"2023-11-27T15:20:37.668031+0000","last_active":"2023-11-27T15:21:02.783667+0000","last_peered":"2023-11-27T15:21:02.783667+0000","last_clean":"2023-11-27T15:21:02.783667+0000","last_became_active":"2023-11-27T15:20:37.667640+0000","last_became_peered":"2023-11-27T15:20:37.667640+0000","last_unstale":"2023-11-27T15:21:02.783667+0000","last_undegraded":"2023-11-27T15:21:02.783667+0000","last_fullsized":"2023-11-27T15:21:02.783667+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:45:18.290503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783029+0000","last_change":"2023-11-27T15:20:32.655738+0000","last_active":"2023-11-27T15:21:02.783029+0000","last_peered":"2023-11-27T15:21:02.783029+0000","last_clean":"2023-11-27T15:21:02.783029+0000","last_became_active":"2023-11-27T15:20:32.655439+0000","last_became_peered":"2023-11-27T15:20:32.655439+0000","last_unstale":"2023-11-27T15:21:02.783029+0000","last_undegraded":"2023-11-27T15:21:02.783029+0000","last_fullsized":"2023-11-27T15:21:02.783029+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:24:02.616850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.795136+0000","last_change":"2023-11-27T15:20:33.229267+0000","last_active":"2023-11-27T15:21:02.795136+0000","last_peered":"2023-11-27T15:21:02.795136+0000","last_clean":"2023-11-27T15:21:02.795136+0000","last_became_active":"2023-11-27T15:20:33.228746+0000","last_became_peered":"2023-11-27T15:20:33.228746+0000","last_unstale":"2023-11-27T15:21:02.795136+0000","last_undegraded":"2023-11-27T15:21:02.795136+0000","last_fullsized":"2023-11-27T15:21:02.795136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:55:36.391108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783571+0000","last_change":"2023-11-27T15:20:37.671893+0000","last_active":"2023-11-27T15:21:02.783571+0000","last_peered":"2023-11-27T15:21:02.783571+0000","last_clean":"2023-11-27T15:21:02.783571+0000","last_became_active":"2023-11-27T15:20:37.671033+0000","last_became_peered":"2023-11-27T15:20:37.671033+0000","last_unstale":"2023-11-27T15:21:02.783571+0000","last_undegraded":"2023-11-27T15:21:02.783571+0000","last_fullsized":"2023-11-27T15:21:02.783571+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:43:21.793592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782934+0000","last_change":"2023-11-27T15:20:32.650342+0000","last_active":"2023-11-27T15:21:02.782934+0000","last_peered":"2023-11-27T15:21:02.782934+0000","last_clean":"2023-11-27T15:21:02.782934+0000","last_became_active":"2023-11-27T15:20:32.650034+0000","last_became_peered":"2023-11-27T15:20:32.650034+0000","last_unstale":"2023-11-27T15:21:02.782934+0000","last_undegraded":"2023-11-27T15:21:02.782934+0000","last_fullsized":"2023-11-27T15:21:02.782934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:29:04.356130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932717+0000","last_change":"2023-11-27T15:20:33.226005+0000","last_active":"2023-11-27T15:21:02.932717+0000","last_peered":"2023-11-27T15:21:02.932717+0000","last_clean":"2023-11-27T15:21:02.932717+0000","last_became_active":"2023-11-27T15:20:33.225685+0000","last_became_peered":"2023-11-27T15:20:33.225685+0000","last_unstale":"2023-11-27T15:21:02.932717+0000","last_undegraded":"2023-11-27T15:21:02.932717+0000","last_fullsized":"2023-11-27T15:21:02.932717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:20:11.054422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779958+0000","last_change":"2023-11-27T15:20:37.673168+0000","last_active":"2023-11-27T15:21:02.779958+0000","last_peered":"2023-11-27T15:21:02.779958+0000","last_clean":"2023-11-27T15:21:02.779958+0000","last_became_active":"2023-11-27T15:20:37.672762+0000","last_became_peered":"2023-11-27T15:20:37.672762+0000","last_unstale":"2023-11-27T15:21:02.779958+0000","last_undegraded":"2023-11-27T15:21:02.779958+0000","last_fullsized":"2023-11-27T15:21:02.779958+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:17:17.977179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:04.806488+0000","last_change":"2023-11-27T15:20:32.656402+0000","last_active":"2023-11-27T15:21:04.806488+0000","last_peered":"2023-11-27T15:21:04.806488+0000","last_clean":"2023-11-27T15:21:04.806488+0000","last_became_active":"2023-11-27T15:20:32.656112+0000","last_became_peered":"2023-11-27T15:20:32.656112+0000","last_unstale":"2023-11-27T15:21:04.806488+0000","last_undegraded":"2023-11-27T15:21:04.806488+0000","last_fullsized":"2023-11-27T15:21:04.806488+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:10:24.829148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782376+0000","last_change":"2023-11-27T15:20:33.223073+0000","last_active":"2023-11-27T15:21:02.782376+0000","last_peered":"2023-11-27T15:21:02.782376+0000","last_clean":"2023-11-27T15:21:02.782376+0000","last_became_active":"2023-11-27T15:20:33.222655+0000","last_became_peered":"2023-11-27T15:20:33.222655+0000","last_unstale":"2023-11-27T15:21:02.782376+0000","last_undegraded":"2023-11-27T15:21:02.782376+0000","last_fullsized":"2023-11-27T15:21:02.782376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:48:43.125499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779818+0000","last_change":"2023-11-27T15:20:37.678722+0000","last_active":"2023-11-27T15:21:02.779818+0000","last_peered":"2023-11-27T15:21:02.779818+0000","last_clean":"2023-11-27T15:21:02.779818+0000","last_became_active":"2023-11-27T15:20:37.678242+0000","last_became_peered":"2023-11-27T15:20:37.678242+0000","last_unstale":"2023-11-27T15:21:02.779818+0000","last_undegraded":"2023-11-27T15:21:02.779818+0000","last_fullsized":"2023-11-27T15:21:02.779818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:36:10.661192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.800170+0000","last_change":"2023-11-27T15:20:32.646951+0000","last_active":"2023-11-27T15:21:03.800170+0000","last_peered":"2023-11-27T15:21:03.800170+0000","last_clean":"2023-11-27T15:21:03.800170+0000","last_became_active":"2023-11-27T15:20:32.646620+0000","last_became_peered":"2023-11-27T15:20:32.646620+0000","last_unstale":"2023-11-27T15:21:03.800170+0000","last_undegraded":"2023-11-27T15:21:03.800170+0000","last_fullsized":"2023-11-27T15:21:03.800170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:13:29.369691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932503+0000","last_change":"2023-11-27T15:20:33.226576+0000","last_active":"2023-11-27T15:21:02.932503+0000","last_peered":"2023-11-27T15:21:02.932503+0000","last_clean":"2023-11-27T15:21:02.932503+0000","last_became_active":"2023-11-27T15:20:33.226181+0000","last_became_peered":"2023-11-27T15:20:33.226181+0000","last_unstale":"2023-11-27T15:21:02.932503+0000","last_undegraded":"2023-11-27T15:21:02.932503+0000","last_fullsized":"2023-11-27T15:21:02.932503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:27:44.284870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932740+0000","last_change":"2023-11-27T15:20:37.681668+0000","last_active":"2023-11-27T15:21:02.932740+0000","last_peered":"2023-11-27T15:21:02.932740+0000","last_clean":"2023-11-27T15:21:02.932740+0000","last_became_active":"2023-11-27T15:20:37.681236+0000","last_became_peered":"2023-11-27T15:20:37.681236+0000","last_unstale":"2023-11-27T15:21:02.932740+0000","last_undegraded":"2023-11-27T15:21:02.932740+0000","last_fullsized":"2023-11-27T15:21:02.932740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:29:12.216944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.800787+0000","last_change":"2023-11-27T15:20:32.659421+0000","last_active":"2023-11-27T15:21:03.800787+0000","last_peered":"2023-11-27T15:21:03.800787+0000","last_clean":"2023-11-27T15:21:03.800787+0000","last_became_active":"2023-11-27T15:20:32.658882+0000","last_became_peered":"2023-11-27T15:20:32.658882+0000","last_unstale":"2023-11-27T15:21:03.800787+0000","last_undegraded":"2023-11-27T15:21:03.800787+0000","last_fullsized":"2023-11-27T15:21:03.800787+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:08:52.781635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783921+0000","last_change":"2023-11-27T15:20:33.227635+0000","last_active":"2023-11-27T15:21:02.783921+0000","last_peered":"2023-11-27T15:21:02.783921+0000","last_clean":"2023-11-27T15:21:02.783921+0000","last_became_active":"2023-11-27T15:20:33.225217+0000","last_became_peered":"2023-11-27T15:20:33.225217+0000","last_unstale":"2023-11-27T15:21:02.783921+0000","last_undegraded":"2023-11-27T15:21:02.783921+0000","last_fullsized":"2023-11-27T15:21:02.783921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:53:42.037169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785814+0000","last_change":"2023-11-27T15:20:37.679575+0000","last_active":"2023-11-27T15:21:02.785814+0000","last_peered":"2023-11-27T15:21:02.785814+0000","last_clean":"2023-11-27T15:21:02.785814+0000","last_became_active":"2023-11-27T15:20:37.679131+0000","last_became_peered":"2023-11-27T15:20:37.679131+0000","last_unstale":"2023-11-27T15:21:02.785814+0000","last_undegraded":"2023-11-27T15:21:02.785814+0000","last_fullsized":"2023-11-27T15:21:02.785814+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:43:21.129782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.931632+0000","last_change":"2023-11-27T15:20:32.657155+0000","last_active":"2023-11-27T15:21:02.931632+0000","last_peered":"2023-11-27T15:21:02.931632+0000","last_clean":"2023-11-27T15:21:02.931632+0000","last_became_active":"2023-11-27T15:20:32.656832+0000","last_became_peered":"2023-11-27T15:20:32.656832+0000","last_unstale":"2023-11-27T15:21:02.931632+0000","last_undegraded":"2023-11-27T15:21:02.931632+0000","last_fullsized":"2023-11-27T15:21:02.931632+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:03:07.478635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779387+0000","last_change":"2023-11-27T15:20:33.229215+0000","last_active":"2023-11-27T15:21:02.779387+0000","last_peered":"2023-11-27T15:21:02.779387+0000","last_clean":"2023-11-27T15:21:02.779387+0000","last_became_active":"2023-11-27T15:20:33.228616+0000","last_became_peered":"2023-11-27T15:20:33.228616+0000","last_unstale":"2023-11-27T15:21:02.779387+0000","last_undegraded":"2023-11-27T15:21:02.779387+0000","last_fullsized":"2023-11-27T15:21:02.779387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:30:10.600160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784114+0000","last_change":"2023-11-27T15:20:37.672582+0000","last_active":"2023-11-27T15:21:02.784114+0000","last_peered":"2023-11-27T15:21:02.784114+0000","last_clean":"2023-11-27T15:21:02.784114+0000","last_became_active":"2023-11-27T15:20:37.672193+0000","last_became_peered":"2023-11-27T15:20:37.672193+0000","last_unstale":"2023-11-27T15:21:02.784114+0000","last_undegraded":"2023-11-27T15:21:02.784114+0000","last_fullsized":"2023-11-27T15:21:02.784114+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:53:51.137569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:04.805924+0000","last_change":"2023-11-27T15:20:32.647894+0000","last_active":"2023-11-27T15:21:04.805924+0000","last_peered":"2023-11-27T15:21:04.805924+0000","last_clean":"2023-11-27T15:21:04.805924+0000","last_became_active":"2023-11-27T15:20:32.647597+0000","last_became_peered":"2023-11-27T15:20:32.647597+0000","last_unstale":"2023-11-27T15:21:04.805924+0000","last_undegraded":"2023-11-27T15:21:04.805924+0000","last_fullsized":"2023-11-27T15:21:04.805924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:00:17.488397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.795490+0000","last_change":"2023-11-27T15:20:33.229382+0000","last_active":"2023-11-27T15:21:02.795490+0000","last_peered":"2023-11-27T15:21:02.795490+0000","last_clean":"2023-11-27T15:21:02.795490+0000","last_became_active":"2023-11-27T15:20:33.229031+0000","last_became_peered":"2023-11-27T15:20:33.229031+0000","last_unstale":"2023-11-27T15:21:02.795490+0000","last_undegraded":"2023-11-27T15:21:02.795490+0000","last_fullsized":"2023-11-27T15:21:02.795490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:43:48.637904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779387+0000","last_change":"2023-11-27T15:20:37.679131+0000","last_active":"2023-11-27T15:21:02.779387+0000","last_peered":"2023-11-27T15:21:02.779387+0000","last_clean":"2023-11-27T15:21:02.779387+0000","last_became_active":"2023-11-27T15:20:37.678290+0000","last_became_peered":"2023-11-27T15:20:37.678290+0000","last_unstale":"2023-11-27T15:21:02.779387+0000","last_undegraded":"2023-11-27T15:21:02.779387+0000","last_fullsized":"2023-11-27T15:21:02.779387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:55:28.895714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.798466+0000","last_change":"2023-11-27T15:21:02.797374+0000","last_active":"2023-11-27T15:21:02.798466+0000","last_peered":"2023-11-27T15:21:02.798466+0000","last_clean":"2023-11-27T15:21:02.798466+0000","last_became_active":"2023-11-27T15:21:02.796076+0000","last_became_peered":"2023-11-27T15:21:02.796076+0000","last_unstale":"2023-11-27T15:21:02.798466+0000","last_undegraded":"2023-11-27T15:21:02.798466+0000","last_fullsized":"2023-11-27T15:21:02.798466+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:07:58.898194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.935104+0000","last_change":"2023-11-27T15:21:02.935104+0000","last_active":"2023-11-27T15:21:02.935104+0000","last_peered":"2023-11-27T15:21:02.935104+0000","last_clean":"2023-11-27T15:21:02.935104+0000","last_became_active":"2023-11-27T15:21:02.934486+0000","last_became_peered":"2023-11-27T15:21:02.934486+0000","last_unstale":"2023-11-27T15:21:02.935104+0000","last_undegraded":"2023-11-27T15:21:02.935104+0000","last_fullsized":"2023-11-27T15:21:02.935104+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:06:11.768697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783910+0000","last_change":"2023-11-27T15:20:37.670389+0000","last_active":"2023-11-27T15:21:02.783910+0000","last_peered":"2023-11-27T15:21:02.783910+0000","last_clean":"2023-11-27T15:21:02.783910+0000","last_became_active":"2023-11-27T15:20:37.669512+0000","last_became_peered":"2023-11-27T15:20:37.669512+0000","last_unstale":"2023-11-27T15:21:02.783910+0000","last_undegraded":"2023-11-27T15:21:02.783910+0000","last_fullsized":"2023-11-27T15:21:02.783910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:47:38.447258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932618+0000","last_change":"2023-11-27T15:20:32.658287+0000","last_active":"2023-11-27T15:21:02.932618+0000","last_peered":"2023-11-27T15:21:02.932618+0000","last_clean":"2023-11-27T15:21:02.932618+0000","last_became_active":"2023-11-27T15:20:32.657925+0000","last_became_peered":"2023-11-27T15:20:32.657925+0000","last_unstale":"2023-11-27T15:21:02.932618+0000","last_undegraded":"2023-11-27T15:21:02.932618+0000","last_fullsized":"2023-11-27T15:21:02.932618+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:49:45.401491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.795421+0000","last_change":"2023-11-27T15:20:33.220240+0000","last_active":"2023-11-27T15:21:02.795421+0000","last_peered":"2023-11-27T15:21:02.795421+0000","last_clean":"2023-11-27T15:21:02.795421+0000","last_became_active":"2023-11-27T15:20:33.219962+0000","last_became_peered":"2023-11-27T15:20:33.219962+0000","last_unstale":"2023-11-27T15:21:02.795421+0000","last_undegraded":"2023-11-27T15:21:02.795421+0000","last_fullsized":"2023-11-27T15:21:02.795421+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:22:09.271036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779503+0000","last_change":"2023-11-27T15:20:37.672203+0000","last_active":"2023-11-27T15:21:02.779503+0000","last_peered":"2023-11-27T15:21:02.779503+0000","last_clean":"2023-11-27T15:21:02.779503+0000","last_became_active":"2023-11-27T15:20:37.671149+0000","last_became_peered":"2023-11-27T15:20:37.671149+0000","last_unstale":"2023-11-27T15:21:02.779503+0000","last_undegraded":"2023-11-27T15:21:02.779503+0000","last_fullsized":"2023-11-27T15:21:02.779503+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:14:24.426741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'23","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:06.276319+0000","last_change":"2023-11-27T15:20:32.646637+0000","last_active":"2023-11-27T15:21:06.276319+0000","last_peered":"2023-11-27T15:21:06.276319+0000","last_clean":"2023-11-27T15:21:06.276319+0000","last_became_active":"2023-11-27T15:20:32.646292+0000","last_became_peered":"2023-11-27T15:20:32.646292+0000","last_unstale":"2023-11-27T15:21:06.276319+0000","last_undegraded":"2023-11-27T15:21:06.276319+0000","last_fullsized":"2023-11-27T15:21:06.276319+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:01:39.494636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38515,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":24,"num_write":24,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782815+0000","last_change":"2023-11-27T15:20:33.217078+0000","last_active":"2023-11-27T15:21:02.782815+0000","last_peered":"2023-11-27T15:21:02.782815+0000","last_clean":"2023-11-27T15:21:02.782815+0000","last_became_active":"2023-11-27T15:20:33.216787+0000","last_became_peered":"2023-11-27T15:20:33.216787+0000","last_unstale":"2023-11-27T15:21:02.782815+0000","last_undegraded":"2023-11-27T15:21:02.782815+0000","last_fullsized":"2023-11-27T15:21:02.782815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:33:46.415082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.933136+0000","last_change":"2023-11-27T15:20:37.671800+0000","last_active":"2023-11-27T15:21:02.933136+0000","last_peered":"2023-11-27T15:21:02.933136+0000","last_clean":"2023-11-27T15:21:02.933136+0000","last_became_active":"2023-11-27T15:20:37.671374+0000","last_became_peered":"2023-11-27T15:20:37.671374+0000","last_unstale":"2023-11-27T15:21:02.933136+0000","last_undegraded":"2023-11-27T15:21:02.933136+0000","last_fullsized":"2023-11-27T15:21:02.933136+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:04:07.644987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.801478+0000","last_change":"2023-11-27T15:20:32.659508+0000","last_active":"2023-11-27T15:21:03.801478+0000","last_peered":"2023-11-27T15:21:03.801478+0000","last_clean":"2023-11-27T15:21:03.801478+0000","last_became_active":"2023-11-27T15:20:32.659226+0000","last_became_peered":"2023-11-27T15:20:32.659226+0000","last_unstale":"2023-11-27T15:21:03.801478+0000","last_undegraded":"2023-11-27T15:21:03.801478+0000","last_fullsized":"2023-11-27T15:21:03.801478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:42:20.386331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786639+0000","last_change":"2023-11-27T15:20:33.230012+0000","last_active":"2023-11-27T15:21:02.786639+0000","last_peered":"2023-11-27T15:21:02.786639+0000","last_clean":"2023-11-27T15:21:02.786639+0000","last_became_active":"2023-11-27T15:20:33.225193+0000","last_became_peered":"2023-11-27T15:20:33.225193+0000","last_unstale":"2023-11-27T15:21:02.786639+0000","last_undegraded":"2023-11-27T15:21:02.786639+0000","last_fullsized":"2023-11-27T15:21:02.786639+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:31:18.056712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932755+0000","last_change":"2023-11-27T15:20:37.672430+0000","last_active":"2023-11-27T15:21:02.932755+0000","last_peered":"2023-11-27T15:21:02.932755+0000","last_clean":"2023-11-27T15:21:02.932755+0000","last_became_active":"2023-11-27T15:20:37.671982+0000","last_became_peered":"2023-11-27T15:20:37.671982+0000","last_unstale":"2023-11-27T15:21:02.932755+0000","last_undegraded":"2023-11-27T15:21:02.932755+0000","last_fullsized":"2023-11-27T15:21:02.932755+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:53:27.660441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932051+0000","last_change":"2023-11-27T15:20:32.655015+0000","last_active":"2023-11-27T15:21:02.932051+0000","last_peered":"2023-11-27T15:21:02.932051+0000","last_clean":"2023-11-27T15:21:02.932051+0000","last_became_active":"2023-11-27T15:20:32.654662+0000","last_became_peered":"2023-11-27T15:20:32.654662+0000","last_unstale":"2023-11-27T15:21:02.932051+0000","last_undegraded":"2023-11-27T15:21:02.932051+0000","last_fullsized":"2023-11-27T15:21:02.932051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:10:34.935987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780064+0000","last_change":"2023-11-27T15:20:33.233980+0000","last_active":"2023-11-27T15:21:02.780064+0000","last_peered":"2023-11-27T15:21:02.780064+0000","last_clean":"2023-11-27T15:21:02.780064+0000","last_became_active":"2023-11-27T15:20:33.233581+0000","last_became_peered":"2023-11-27T15:20:33.233581+0000","last_unstale":"2023-11-27T15:21:02.780064+0000","last_undegraded":"2023-11-27T15:21:02.780064+0000","last_fullsized":"2023-11-27T15:21:02.780064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:21:09.061874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779860+0000","last_change":"2023-11-27T15:20:37.682359+0000","last_active":"2023-11-27T15:21:02.779860+0000","last_peered":"2023-11-27T15:21:02.779860+0000","last_clean":"2023-11-27T15:21:02.779860+0000","last_became_active":"2023-11-27T15:20:37.681888+0000","last_became_peered":"2023-11-27T15:20:37.681888+0000","last_unstale":"2023-11-27T15:21:02.779860+0000","last_undegraded":"2023-11-27T15:21:02.779860+0000","last_fullsized":"2023-11-27T15:21:02.779860+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:22:58.040147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.799258+0000","last_change":"2023-11-27T15:20:32.653388+0000","last_active":"2023-11-27T15:21:03.799258+0000","last_peered":"2023-11-27T15:21:03.799258+0000","last_clean":"2023-11-27T15:21:03.799258+0000","last_became_active":"2023-11-27T15:20:32.653007+0000","last_became_peered":"2023-11-27T15:20:32.653007+0000","last_unstale":"2023-11-27T15:21:03.799258+0000","last_undegraded":"2023-11-27T15:21:03.799258+0000","last_fullsized":"2023-11-27T15:21:03.799258+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:02:11.654541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779871+0000","last_change":"2023-11-27T15:20:33.216539+0000","last_active":"2023-11-27T15:21:02.779871+0000","last_peered":"2023-11-27T15:21:02.779871+0000","last_clean":"2023-11-27T15:21:02.779871+0000","last_became_active":"2023-11-27T15:20:33.216166+0000","last_became_peered":"2023-11-27T15:20:33.216166+0000","last_unstale":"2023-11-27T15:21:02.779871+0000","last_undegraded":"2023-11-27T15:21:02.779871+0000","last_fullsized":"2023-11-27T15:21:02.779871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:12:37.881354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786643+0000","last_change":"2023-11-27T15:20:37.675735+0000","last_active":"2023-11-27T15:21:02.786643+0000","last_peered":"2023-11-27T15:21:02.786643+0000","last_clean":"2023-11-27T15:21:02.786643+0000","last_became_active":"2023-11-27T15:20:37.669685+0000","last_became_peered":"2023-11-27T15:20:37.669685+0000","last_unstale":"2023-11-27T15:21:02.786643+0000","last_undegraded":"2023-11-27T15:21:02.786643+0000","last_fullsized":"2023-11-27T15:21:02.786643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:47:13.695674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'10","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782372+0000","last_change":"2023-11-27T15:20:32.657567+0000","last_active":"2023-11-27T15:21:02.782372+0000","last_peered":"2023-11-27T15:21:02.782372+0000","last_clean":"2023-11-27T15:21:02.782372+0000","last_became_active":"2023-11-27T15:20:32.657251+0000","last_became_peered":"2023-11-27T15:20:32.657251+0000","last_unstale":"2023-11-27T15:21:02.782372+0000","last_undegraded":"2023-11-27T15:21:02.782372+0000","last_fullsized":"2023-11-27T15:21:02.782372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:23:00.166520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932029+0000","last_change":"2023-11-27T15:20:33.225019+0000","last_active":"2023-11-27T15:21:02.932029+0000","last_peered":"2023-11-27T15:21:02.932029+0000","last_clean":"2023-11-27T15:21:02.932029+0000","last_became_active":"2023-11-27T15:20:33.224622+0000","last_became_peered":"2023-11-27T15:20:33.224622+0000","last_unstale":"2023-11-27T15:21:02.932029+0000","last_undegraded":"2023-11-27T15:21:02.932029+0000","last_fullsized":"2023-11-27T15:21:02.932029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:44:13.508838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.931675+0000","last_change":"2023-11-27T15:20:37.680808+0000","last_active":"2023-11-27T15:21:02.931675+0000","last_peered":"2023-11-27T15:21:02.931675+0000","last_clean":"2023-11-27T15:21:02.931675+0000","last_became_active":"2023-11-27T15:20:37.680348+0000","last_became_peered":"2023-11-27T15:20:37.680348+0000","last_unstale":"2023-11-27T15:21:02.931675+0000","last_undegraded":"2023-11-27T15:21:02.931675+0000","last_fullsized":"2023-11-27T15:21:02.931675+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:48:00.800370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783285+0000","last_change":"2023-11-27T15:20:32.651093+0000","last_active":"2023-11-27T15:21:02.783285+0000","last_peered":"2023-11-27T15:21:02.783285+0000","last_clean":"2023-11-27T15:21:02.783285+0000","last_became_active":"2023-11-27T15:20:32.650657+0000","last_became_peered":"2023-11-27T15:20:32.650657+0000","last_unstale":"2023-11-27T15:21:02.783285+0000","last_undegraded":"2023-11-27T15:21:02.783285+0000","last_fullsized":"2023-11-27T15:21:02.783285+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:28:31.347970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.933913+0000","last_change":"2023-11-27T15:20:33.224806+0000","last_active":"2023-11-27T15:21:02.933913+0000","last_peered":"2023-11-27T15:21:02.933913+0000","last_clean":"2023-11-27T15:21:02.933913+0000","last_became_active":"2023-11-27T15:20:33.224362+0000","last_became_peered":"2023-11-27T15:20:33.224362+0000","last_unstale":"2023-11-27T15:21:02.933913+0000","last_undegraded":"2023-11-27T15:21:02.933913+0000","last_fullsized":"2023-11-27T15:21:02.933913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:34:11.704495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782260+0000","last_change":"2023-11-27T15:20:37.849274+0000","last_active":"2023-11-27T15:21:02.782260+0000","last_peered":"2023-11-27T15:21:02.782260+0000","last_clean":"2023-11-27T15:21:02.782260+0000","last_became_active":"2023-11-27T15:20:37.848839+0000","last_became_peered":"2023-11-27T15:20:37.848839+0000","last_unstale":"2023-11-27T15:21:02.782260+0000","last_undegraded":"2023-11-27T15:21:02.782260+0000","last_fullsized":"2023-11-27T15:21:02.782260+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:46:25.321810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779849+0000","last_change":"2023-11-27T15:20:32.660835+0000","last_active":"2023-11-27T15:21:02.779849+0000","last_peered":"2023-11-27T15:21:02.779849+0000","last_clean":"2023-11-27T15:21:02.779849+0000","last_became_active":"2023-11-27T15:20:32.660431+0000","last_became_peered":"2023-11-27T15:20:32.660431+0000","last_unstale":"2023-11-27T15:21:02.779849+0000","last_undegraded":"2023-11-27T15:21:02.779849+0000","last_fullsized":"2023-11-27T15:21:02.779849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:52:11.617105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932764+0000","last_change":"2023-11-27T15:20:33.245772+0000","last_active":"2023-11-27T15:21:02.932764+0000","last_peered":"2023-11-27T15:21:02.932764+0000","last_clean":"2023-11-27T15:21:02.932764+0000","last_became_active":"2023-11-27T15:20:33.245468+0000","last_became_peered":"2023-11-27T15:20:33.245468+0000","last_unstale":"2023-11-27T15:21:02.932764+0000","last_undegraded":"2023-11-27T15:21:02.932764+0000","last_fullsized":"2023-11-27T15:21:02.932764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:24:42.963520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784658+0000","last_change":"2023-11-27T15:20:37.678972+0000","last_active":"2023-11-27T15:21:02.784658+0000","last_peered":"2023-11-27T15:21:02.784658+0000","last_clean":"2023-11-27T15:21:02.784658+0000","last_became_active":"2023-11-27T15:20:37.678304+0000","last_became_peered":"2023-11-27T15:20:37.678304+0000","last_unstale":"2023-11-27T15:21:02.784658+0000","last_undegraded":"2023-11-27T15:21:02.784658+0000","last_fullsized":"2023-11-27T15:21:02.784658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:32:12.064148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780000+0000","last_change":"2023-11-27T15:20:32.661739+0000","last_active":"2023-11-27T15:21:02.780000+0000","last_peered":"2023-11-27T15:21:02.780000+0000","last_clean":"2023-11-27T15:21:02.780000+0000","last_became_active":"2023-11-27T15:20:32.661401+0000","last_became_peered":"2023-11-27T15:20:32.661401+0000","last_unstale":"2023-11-27T15:21:02.780000+0000","last_undegraded":"2023-11-27T15:21:02.780000+0000","last_fullsized":"2023-11-27T15:21:02.780000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:47:15.469916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932830+0000","last_change":"2023-11-27T15:20:33.235595+0000","last_active":"2023-11-27T15:21:02.932830+0000","last_peered":"2023-11-27T15:21:02.932830+0000","last_clean":"2023-11-27T15:21:02.932830+0000","last_became_active":"2023-11-27T15:20:33.235277+0000","last_became_peered":"2023-11-27T15:20:33.235277+0000","last_unstale":"2023-11-27T15:21:02.932830+0000","last_undegraded":"2023-11-27T15:21:02.932830+0000","last_fullsized":"2023-11-27T15:21:02.932830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:23:31.896222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.934194+0000","last_change":"2023-11-27T15:21:02.934194+0000","last_active":"2023-11-27T15:21:02.934194+0000","last_peered":"2023-11-27T15:21:02.934194+0000","last_clean":"2023-11-27T15:21:02.934194+0000","last_became_active":"2023-11-27T15:21:02.933555+0000","last_became_peered":"2023-11-27T15:21:02.933555+0000","last_unstale":"2023-11-27T15:21:02.934194+0000","last_undegraded":"2023-11-27T15:21:02.934194+0000","last_fullsized":"2023-11-27T15:21:02.934194+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:49:59.949602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785630+0000","last_change":"2023-11-27T15:21:02.784351+0000","last_active":"2023-11-27T15:21:02.785630+0000","last_peered":"2023-11-27T15:21:02.785630+0000","last_clean":"2023-11-27T15:21:02.785630+0000","last_became_active":"2023-11-27T15:21:02.783996+0000","last_became_peered":"2023-11-27T15:21:02.783996+0000","last_unstale":"2023-11-27T15:21:02.785630+0000","last_undegraded":"2023-11-27T15:21:02.785630+0000","last_fullsized":"2023-11-27T15:21:02.785630+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:30:13.050830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:04.807056+0000","last_change":"2023-11-27T15:20:32.659823+0000","last_active":"2023-11-27T15:21:04.807056+0000","last_peered":"2023-11-27T15:21:04.807056+0000","last_clean":"2023-11-27T15:21:04.807056+0000","last_became_active":"2023-11-27T15:20:32.659472+0000","last_became_peered":"2023-11-27T15:20:32.659472+0000","last_unstale":"2023-11-27T15:21:04.807056+0000","last_undegraded":"2023-11-27T15:21:04.807056+0000","last_fullsized":"2023-11-27T15:21:04.807056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:19:47.161135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780173+0000","last_change":"2023-11-27T15:20:37.671494+0000","last_active":"2023-11-27T15:21:02.780173+0000","last_peered":"2023-11-27T15:21:02.780173+0000","last_clean":"2023-11-27T15:21:02.780173+0000","last_became_active":"2023-11-27T15:20:37.671010+0000","last_became_peered":"2023-11-27T15:20:37.671010+0000","last_unstale":"2023-11-27T15:21:02.780173+0000","last_undegraded":"2023-11-27T15:21:02.780173+0000","last_fullsized":"2023-11-27T15:21:02.780173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:09:00.036788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784810+0000","last_change":"2023-11-27T15:20:33.241669+0000","last_active":"2023-11-27T15:21:02.784810+0000","last_peered":"2023-11-27T15:21:02.784810+0000","last_clean":"2023-11-27T15:21:02.784810+0000","last_became_active":"2023-11-27T15:20:33.241373+0000","last_became_peered":"2023-11-27T15:20:33.241373+0000","last_unstale":"2023-11-27T15:21:02.784810+0000","last_undegraded":"2023-11-27T15:21:02.784810+0000","last_fullsized":"2023-11-27T15:21:02.784810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:21:19.709967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783900+0000","last_change":"2023-11-27T15:20:32.647596+0000","last_active":"2023-11-27T15:21:02.783900+0000","last_peered":"2023-11-27T15:21:02.783900+0000","last_clean":"2023-11-27T15:21:02.783900+0000","last_became_active":"2023-11-27T15:20:32.647228+0000","last_became_peered":"2023-11-27T15:20:32.647228+0000","last_unstale":"2023-11-27T15:21:02.783900+0000","last_undegraded":"2023-11-27T15:21:02.783900+0000","last_fullsized":"2023-11-27T15:21:02.783900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:40:49.354792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779663+0000","last_change":"2023-11-27T15:19:30.114855+0000","last_active":"2023-11-27T15:21:02.779663+0000","last_peered":"2023-11-27T15:21:02.779663+0000","last_clean":"2023-11-27T15:21:02.779663+0000","last_became_active":"2023-11-27T15:19:30.114419+0000","last_became_peered":"2023-11-27T15:19:30.114419+0000","last_unstale":"2023-11-27T15:21:02.779663+0000","last_undegraded":"2023-11-27T15:21:02.779663+0000","last_fullsized":"2023-11-27T15:21:02.779663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:20.216695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:20.216695+0000","last_clean_scrub_stamp":"2023-11-27T15:16:20.216695+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:14:23.715586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932257+0000","last_change":"2023-11-27T15:20:37.850539+0000","last_active":"2023-11-27T15:21:02.932257+0000","last_peered":"2023-11-27T15:21:02.932257+0000","last_clean":"2023-11-27T15:21:02.932257+0000","last_became_active":"2023-11-27T15:20:37.850044+0000","last_became_peered":"2023-11-27T15:20:37.850044+0000","last_unstale":"2023-11-27T15:21:02.932257+0000","last_undegraded":"2023-11-27T15:21:02.932257+0000","last_fullsized":"2023-11-27T15:21:02.932257+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:29:43.906707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783098+0000","last_change":"2023-11-27T15:20:33.211319+0000","last_active":"2023-11-27T15:21:02.783098+0000","last_peered":"2023-11-27T15:21:02.783098+0000","last_clean":"2023-11-27T15:21:02.783098+0000","last_became_active":"2023-11-27T15:20:33.211004+0000","last_became_peered":"2023-11-27T15:20:33.211004+0000","last_unstale":"2023-11-27T15:21:02.783098+0000","last_undegraded":"2023-11-27T15:21:02.783098+0000","last_fullsized":"2023-11-27T15:21:02.783098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:57:04.622964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.799863+0000","last_change":"2023-11-27T15:20:32.655003+0000","last_active":"2023-11-27T15:21:03.799863+0000","last_peered":"2023-11-27T15:21:03.799863+0000","last_clean":"2023-11-27T15:21:03.799863+0000","last_became_active":"2023-11-27T15:20:32.654517+0000","last_became_peered":"2023-11-27T15:20:32.654517+0000","last_unstale":"2023-11-27T15:21:03.799863+0000","last_undegraded":"2023-11-27T15:21:03.799863+0000","last_fullsized":"2023-11-27T15:21:03.799863+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:12:26.090416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932725+0000","last_change":"2023-11-27T15:20:37.673183+0000","last_active":"2023-11-27T15:21:02.932725+0000","last_peered":"2023-11-27T15:21:02.932725+0000","last_clean":"2023-11-27T15:21:02.932725+0000","last_became_active":"2023-11-27T15:20:37.672838+0000","last_became_peered":"2023-11-27T15:20:37.672838+0000","last_unstale":"2023-11-27T15:21:02.932725+0000","last_undegraded":"2023-11-27T15:21:02.932725+0000","last_fullsized":"2023-11-27T15:21:02.932725+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:47:56.458647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782687+0000","last_change":"2023-11-27T15:20:32.659540+0000","last_active":"2023-11-27T15:21:02.782687+0000","last_peered":"2023-11-27T15:21:02.782687+0000","last_clean":"2023-11-27T15:21:02.782687+0000","last_became_active":"2023-11-27T15:20:32.659211+0000","last_became_peered":"2023-11-27T15:20:32.659211+0000","last_unstale":"2023-11-27T15:21:02.782687+0000","last_undegraded":"2023-11-27T15:21:02.782687+0000","last_fullsized":"2023-11-27T15:21:02.782687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:02:50.140093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784853+0000","last_change":"2023-11-27T15:20:33.216405+0000","last_active":"2023-11-27T15:21:02.784853+0000","last_peered":"2023-11-27T15:21:02.784853+0000","last_clean":"2023-11-27T15:21:02.784853+0000","last_became_active":"2023-11-27T15:20:33.216072+0000","last_became_peered":"2023-11-27T15:20:33.216072+0000","last_unstale":"2023-11-27T15:21:02.784853+0000","last_undegraded":"2023-11-27T15:21:02.784853+0000","last_fullsized":"2023-11-27T15:21:02.784853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:09:28.211261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785517+0000","last_change":"2023-11-27T15:20:37.849753+0000","last_active":"2023-11-27T15:21:02.785517+0000","last_peered":"2023-11-27T15:21:02.785517+0000","last_clean":"2023-11-27T15:21:02.785517+0000","last_became_active":"2023-11-27T15:20:37.848576+0000","last_became_peered":"2023-11-27T15:20:37.848576+0000","last_unstale":"2023-11-27T15:21:02.785517+0000","last_undegraded":"2023-11-27T15:21:02.785517+0000","last_fullsized":"2023-11-27T15:21:02.785517+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:59:24.696190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.802128+0000","last_change":"2023-11-27T15:20:32.652739+0000","last_active":"2023-11-27T15:21:03.802128+0000","last_peered":"2023-11-27T15:21:03.802128+0000","last_clean":"2023-11-27T15:21:03.802128+0000","last_became_active":"2023-11-27T15:20:32.652314+0000","last_became_peered":"2023-11-27T15:20:32.652314+0000","last_unstale":"2023-11-27T15:21:03.802128+0000","last_undegraded":"2023-11-27T15:21:03.802128+0000","last_fullsized":"2023-11-27T15:21:03.802128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:44:25.188063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.787449+0000","last_change":"2023-11-27T15:20:33.240552+0000","last_active":"2023-11-27T15:21:02.787449+0000","last_peered":"2023-11-27T15:21:02.787449+0000","last_clean":"2023-11-27T15:21:02.787449+0000","last_became_active":"2023-11-27T15:20:33.239885+0000","last_became_peered":"2023-11-27T15:20:33.239885+0000","last_unstale":"2023-11-27T15:21:02.787449+0000","last_undegraded":"2023-11-27T15:21:02.787449+0000","last_fullsized":"2023-11-27T15:21:02.787449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:36:51.366222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779611+0000","last_change":"2023-11-27T15:20:37.669895+0000","last_active":"2023-11-27T15:21:02.779611+0000","last_peered":"2023-11-27T15:21:02.779611+0000","last_clean":"2023-11-27T15:21:02.779611+0000","last_became_active":"2023-11-27T15:20:37.669489+0000","last_became_peered":"2023-11-27T15:20:37.669489+0000","last_unstale":"2023-11-27T15:21:02.779611+0000","last_undegraded":"2023-11-27T15:21:02.779611+0000","last_fullsized":"2023-11-27T15:21:02.779611+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:18:44.798721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784573+0000","last_change":"2023-11-27T15:20:32.656773+0000","last_active":"2023-11-27T15:21:02.784573+0000","last_peered":"2023-11-27T15:21:02.784573+0000","last_clean":"2023-11-27T15:21:02.784573+0000","last_became_active":"2023-11-27T15:20:32.656451+0000","last_became_peered":"2023-11-27T15:20:32.656451+0000","last_unstale":"2023-11-27T15:21:02.784573+0000","last_undegraded":"2023-11-27T15:21:02.784573+0000","last_fullsized":"2023-11-27T15:21:02.784573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:34:42.007222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786148+0000","last_change":"2023-11-27T15:20:33.231685+0000","last_active":"2023-11-27T15:21:02.786148+0000","last_peered":"2023-11-27T15:21:02.786148+0000","last_clean":"2023-11-27T15:21:02.786148+0000","last_became_active":"2023-11-27T15:20:33.228265+0000","last_became_peered":"2023-11-27T15:20:33.228265+0000","last_unstale":"2023-11-27T15:21:02.786148+0000","last_undegraded":"2023-11-27T15:21:02.786148+0000","last_fullsized":"2023-11-27T15:21:02.786148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:18:19.094289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.935830+0000","last_change":"2023-11-27T15:21:02.935830+0000","last_active":"2023-11-27T15:21:02.935830+0000","last_peered":"2023-11-27T15:21:02.935830+0000","last_clean":"2023-11-27T15:21:02.935830+0000","last_became_active":"2023-11-27T15:21:02.935412+0000","last_became_peered":"2023-11-27T15:21:02.935412+0000","last_unstale":"2023-11-27T15:21:02.935830+0000","last_undegraded":"2023-11-27T15:21:02.935830+0000","last_fullsized":"2023-11-27T15:21:02.935830+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:26:59.988473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.794970+0000","last_change":"2023-11-27T15:20:32.660424+0000","last_active":"2023-11-27T15:21:02.794970+0000","last_peered":"2023-11-27T15:21:02.794970+0000","last_clean":"2023-11-27T15:21:02.794970+0000","last_became_active":"2023-11-27T15:20:32.660112+0000","last_became_peered":"2023-11-27T15:20:32.660112+0000","last_unstale":"2023-11-27T15:21:02.794970+0000","last_undegraded":"2023-11-27T15:21:02.794970+0000","last_fullsized":"2023-11-27T15:21:02.794970+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:59:35.405721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782221+0000","last_change":"2023-11-27T15:20:33.227677+0000","last_active":"2023-11-27T15:21:02.782221+0000","last_peered":"2023-11-27T15:21:02.782221+0000","last_clean":"2023-11-27T15:21:02.782221+0000","last_became_active":"2023-11-27T15:20:33.227334+0000","last_became_peered":"2023-11-27T15:20:33.227334+0000","last_unstale":"2023-11-27T15:21:02.782221+0000","last_undegraded":"2023-11-27T15:21:02.782221+0000","last_fullsized":"2023-11-27T15:21:02.782221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:45:49.719897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.935341+0000","last_change":"2023-11-27T15:21:02.934516+0000","last_active":"2023-11-27T15:21:02.935341+0000","last_peered":"2023-11-27T15:21:02.935341+0000","last_clean":"2023-11-27T15:21:02.935341+0000","last_became_active":"2023-11-27T15:21:02.934012+0000","last_became_peered":"2023-11-27T15:21:02.934012+0000","last_unstale":"2023-11-27T15:21:02.935341+0000","last_undegraded":"2023-11-27T15:21:02.935341+0000","last_fullsized":"2023-11-27T15:21:02.935341+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:07:11.301253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784148+0000","last_change":"2023-11-27T15:20:32.657935+0000","last_active":"2023-11-27T15:21:02.784148+0000","last_peered":"2023-11-27T15:21:02.784148+0000","last_clean":"2023-11-27T15:21:02.784148+0000","last_became_active":"2023-11-27T15:20:32.657579+0000","last_became_peered":"2023-11-27T15:20:32.657579+0000","last_unstale":"2023-11-27T15:21:02.784148+0000","last_undegraded":"2023-11-27T15:21:02.784148+0000","last_fullsized":"2023-11-27T15:21:02.784148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:30:23.087862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932840+0000","last_change":"2023-11-27T15:20:33.225817+0000","last_active":"2023-11-27T15:21:02.932840+0000","last_peered":"2023-11-27T15:21:02.932840+0000","last_clean":"2023-11-27T15:21:02.932840+0000","last_became_active":"2023-11-27T15:20:33.225438+0000","last_became_peered":"2023-11-27T15:20:33.225438+0000","last_unstale":"2023-11-27T15:21:02.932840+0000","last_undegraded":"2023-11-27T15:21:02.932840+0000","last_fullsized":"2023-11-27T15:21:02.932840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:04:27.668653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.934304+0000","last_change":"2023-11-27T15:21:02.934304+0000","last_active":"2023-11-27T15:21:02.934304+0000","last_peered":"2023-11-27T15:21:02.934304+0000","last_clean":"2023-11-27T15:21:02.934304+0000","last_became_active":"2023-11-27T15:21:02.933709+0000","last_became_peered":"2023-11-27T15:21:02.933709+0000","last_unstale":"2023-11-27T15:21:02.934304+0000","last_undegraded":"2023-11-27T15:21:02.934304+0000","last_fullsized":"2023-11-27T15:21:02.934304+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:43:38.974368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780192+0000","last_change":"2023-11-27T15:20:32.659484+0000","last_active":"2023-11-27T15:21:02.780192+0000","last_peered":"2023-11-27T15:21:02.780192+0000","last_clean":"2023-11-27T15:21:02.780192+0000","last_became_active":"2023-11-27T15:20:32.659134+0000","last_became_peered":"2023-11-27T15:20:32.659134+0000","last_unstale":"2023-11-27T15:21:02.780192+0000","last_undegraded":"2023-11-27T15:21:02.780192+0000","last_fullsized":"2023-11-27T15:21:02.780192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:04:03.053379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784616+0000","last_change":"2023-11-27T15:20:33.235755+0000","last_active":"2023-11-27T15:21:02.784616+0000","last_peered":"2023-11-27T15:21:02.784616+0000","last_clean":"2023-11-27T15:21:02.784616+0000","last_became_active":"2023-11-27T15:20:33.234576+0000","last_became_peered":"2023-11-27T15:20:33.234576+0000","last_unstale":"2023-11-27T15:21:02.784616+0000","last_undegraded":"2023-11-27T15:21:02.784616+0000","last_fullsized":"2023-11-27T15:21:02.784616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:25:25.181797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784302+0000","last_change":"2023-11-27T15:20:37.849121+0000","last_active":"2023-11-27T15:21:02.784302+0000","last_peered":"2023-11-27T15:21:02.784302+0000","last_clean":"2023-11-27T15:21:02.784302+0000","last_became_active":"2023-11-27T15:20:37.848683+0000","last_became_peered":"2023-11-27T15:20:37.848683+0000","last_unstale":"2023-11-27T15:21:02.784302+0000","last_undegraded":"2023-11-27T15:21:02.784302+0000","last_fullsized":"2023-11-27T15:21:02.784302+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:05:39.554898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782466+0000","last_change":"2023-11-27T15:20:32.655896+0000","last_active":"2023-11-27T15:21:02.782466+0000","last_peered":"2023-11-27T15:21:02.782466+0000","last_clean":"2023-11-27T15:21:02.782466+0000","last_became_active":"2023-11-27T15:20:32.655623+0000","last_became_peered":"2023-11-27T15:20:32.655623+0000","last_unstale":"2023-11-27T15:21:02.782466+0000","last_undegraded":"2023-11-27T15:21:02.782466+0000","last_fullsized":"2023-11-27T15:21:02.782466+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:01:57.075773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779947+0000","last_change":"2023-11-27T15:20:33.227318+0000","last_active":"2023-11-27T15:21:02.779947+0000","last_peered":"2023-11-27T15:21:02.779947+0000","last_clean":"2023-11-27T15:21:02.779947+0000","last_became_active":"2023-11-27T15:20:33.226944+0000","last_became_peered":"2023-11-27T15:20:33.226944+0000","last_unstale":"2023-11-27T15:21:02.779947+0000","last_undegraded":"2023-11-27T15:21:02.779947+0000","last_fullsized":"2023-11-27T15:21:02.779947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:47:26.971182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.794990+0000","last_change":"2023-11-27T15:20:37.669098+0000","last_active":"2023-11-27T15:21:02.794990+0000","last_peered":"2023-11-27T15:21:02.794990+0000","last_clean":"2023-11-27T15:21:02.794990+0000","last_became_active":"2023-11-27T15:20:37.668693+0000","last_became_peered":"2023-11-27T15:20:37.668693+0000","last_unstale":"2023-11-27T15:21:02.794990+0000","last_undegraded":"2023-11-27T15:21:02.794990+0000","last_fullsized":"2023-11-27T15:21:02.794990+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:38:51.699084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.800964+0000","last_change":"2023-11-27T15:20:32.660285+0000","last_active":"2023-11-27T15:21:03.800964+0000","last_peered":"2023-11-27T15:21:03.800964+0000","last_clean":"2023-11-27T15:21:03.800964+0000","last_became_active":"2023-11-27T15:20:32.659982+0000","last_became_peered":"2023-11-27T15:20:32.659982+0000","last_unstale":"2023-11-27T15:21:03.800964+0000","last_undegraded":"2023-11-27T15:21:03.800964+0000","last_fullsized":"2023-11-27T15:21:03.800964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:42:36.081550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780414+0000","last_change":"2023-11-27T15:20:33.226629+0000","last_active":"2023-11-27T15:21:02.780414+0000","last_peered":"2023-11-27T15:21:02.780414+0000","last_clean":"2023-11-27T15:21:02.780414+0000","last_became_active":"2023-11-27T15:20:33.226183+0000","last_became_peered":"2023-11-27T15:20:33.226183+0000","last_unstale":"2023-11-27T15:21:02.780414+0000","last_undegraded":"2023-11-27T15:21:02.780414+0000","last_fullsized":"2023-11-27T15:21:02.780414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:33:24.895609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785284+0000","last_change":"2023-11-27T15:20:37.680987+0000","last_active":"2023-11-27T15:21:02.785284+0000","last_peered":"2023-11-27T15:21:02.785284+0000","last_clean":"2023-11-27T15:21:02.785284+0000","last_became_active":"2023-11-27T15:20:37.680219+0000","last_became_peered":"2023-11-27T15:20:37.680219+0000","last_unstale":"2023-11-27T15:21:02.785284+0000","last_undegraded":"2023-11-27T15:21:02.785284+0000","last_fullsized":"2023-11-27T15:21:02.785284+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:27:01.990771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932894+0000","last_change":"2023-11-27T15:20:32.655035+0000","last_active":"2023-11-27T15:21:02.932894+0000","last_peered":"2023-11-27T15:21:02.932894+0000","last_clean":"2023-11-27T15:21:02.932894+0000","last_became_active":"2023-11-27T15:20:32.654687+0000","last_became_peered":"2023-11-27T15:20:32.654687+0000","last_unstale":"2023-11-27T15:21:02.932894+0000","last_undegraded":"2023-11-27T15:21:02.932894+0000","last_fullsized":"2023-11-27T15:21:02.932894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:54:01.028543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780460+0000","last_change":"2023-11-27T15:20:33.245908+0000","last_active":"2023-11-27T15:21:02.780460+0000","last_peered":"2023-11-27T15:21:02.780460+0000","last_clean":"2023-11-27T15:21:02.780460+0000","last_became_active":"2023-11-27T15:20:33.245576+0000","last_became_peered":"2023-11-27T15:20:33.245576+0000","last_unstale":"2023-11-27T15:21:02.780460+0000","last_undegraded":"2023-11-27T15:21:02.780460+0000","last_fullsized":"2023-11-27T15:21:02.780460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:41:20.860779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.931498+0000","last_change":"2023-11-27T15:20:37.674253+0000","last_active":"2023-11-27T15:21:02.931498+0000","last_peered":"2023-11-27T15:21:02.931498+0000","last_clean":"2023-11-27T15:21:02.931498+0000","last_became_active":"2023-11-27T15:20:37.673708+0000","last_became_peered":"2023-11-27T15:20:37.673708+0000","last_unstale":"2023-11-27T15:21:02.931498+0000","last_undegraded":"2023-11-27T15:21:02.931498+0000","last_fullsized":"2023-11-27T15:21:02.931498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:25:32.219577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786521+0000","last_change":"2023-11-27T15:20:32.659605+0000","last_active":"2023-11-27T15:21:02.786521+0000","last_peered":"2023-11-27T15:21:02.786521+0000","last_clean":"2023-11-27T15:21:02.786521+0000","last_became_active":"2023-11-27T15:20:32.659186+0000","last_became_peered":"2023-11-27T15:20:32.659186+0000","last_unstale":"2023-11-27T15:21:02.786521+0000","last_undegraded":"2023-11-27T15:21:02.786521+0000","last_fullsized":"2023-11-27T15:21:02.786521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:19:33.727199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780674+0000","last_change":"2023-11-27T15:20:33.227914+0000","last_active":"2023-11-27T15:21:02.780674+0000","last_peered":"2023-11-27T15:21:02.780674+0000","last_clean":"2023-11-27T15:21:02.780674+0000","last_became_active":"2023-11-27T15:20:33.227572+0000","last_became_peered":"2023-11-27T15:20:33.227572+0000","last_unstale":"2023-11-27T15:21:02.780674+0000","last_undegraded":"2023-11-27T15:21:02.780674+0000","last_fullsized":"2023-11-27T15:21:02.780674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:16:00.098620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932874+0000","last_change":"2023-11-27T15:20:33.215605+0000","last_active":"2023-11-27T15:21:02.932874+0000","last_peered":"2023-11-27T15:21:02.932874+0000","last_clean":"2023-11-27T15:21:02.932874+0000","last_became_active":"2023-11-27T15:20:33.215295+0000","last_became_peered":"2023-11-27T15:20:33.215295+0000","last_unstale":"2023-11-27T15:21:02.932874+0000","last_undegraded":"2023-11-27T15:21:02.932874+0000","last_fullsized":"2023-11-27T15:21:02.932874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:46:08.937950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780509+0000","last_change":"2023-11-27T15:20:33.214014+0000","last_active":"2023-11-27T15:21:02.780509+0000","last_peered":"2023-11-27T15:21:02.780509+0000","last_clean":"2023-11-27T15:21:02.780509+0000","last_became_active":"2023-11-27T15:20:33.213685+0000","last_became_peered":"2023-11-27T15:20:33.213685+0000","last_unstale":"2023-11-27T15:21:02.780509+0000","last_undegraded":"2023-11-27T15:21:02.780509+0000","last_fullsized":"2023-11-27T15:21:02.780509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:06:53.889016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779696+0000","last_change":"2023-11-27T15:20:33.229722+0000","last_active":"2023-11-27T15:21:02.779696+0000","last_peered":"2023-11-27T15:21:02.779696+0000","last_clean":"2023-11-27T15:21:02.779696+0000","last_became_active":"2023-11-27T15:20:33.229372+0000","last_became_peered":"2023-11-27T15:20:33.229372+0000","last_unstale":"2023-11-27T15:21:02.779696+0000","last_undegraded":"2023-11-27T15:21:02.779696+0000","last_fullsized":"2023-11-27T15:21:02.779696+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:48:26.454003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785590+0000","last_change":"2023-11-27T15:20:33.235047+0000","last_active":"2023-11-27T15:21:02.785590+0000","last_peered":"2023-11-27T15:21:02.785590+0000","last_clean":"2023-11-27T15:21:02.785590+0000","last_became_active":"2023-11-27T15:20:33.233078+0000","last_became_peered":"2023-11-27T15:20:33.233078+0000","last_unstale":"2023-11-27T15:21:02.785590+0000","last_undegraded":"2023-11-27T15:21:02.785590+0000","last_fullsized":"2023-11-27T15:21:02.785590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:56:34.043375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784969+0000","last_change":"2023-11-27T15:20:33.229118+0000","last_active":"2023-11-27T15:21:02.784969+0000","last_peered":"2023-11-27T15:21:02.784969+0000","last_clean":"2023-11-27T15:21:02.784969+0000","last_became_active":"2023-11-27T15:20:33.228748+0000","last_became_peered":"2023-11-27T15:20:33.228748+0000","last_unstale":"2023-11-27T15:21:02.784969+0000","last_undegraded":"2023-11-27T15:21:02.784969+0000","last_fullsized":"2023-11-27T15:21:02.784969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:44:41.459719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.933214+0000","last_change":"2023-11-27T15:20:33.222714+0000","last_active":"2023-11-27T15:21:02.933214+0000","last_peered":"2023-11-27T15:21:02.933214+0000","last_clean":"2023-11-27T15:21:02.933214+0000","last_became_active":"2023-11-27T15:20:33.222396+0000","last_became_peered":"2023-11-27T15:20:33.222396+0000","last_unstale":"2023-11-27T15:21:02.933214+0000","last_undegraded":"2023-11-27T15:21:02.933214+0000","last_fullsized":"2023-11-27T15:21:02.933214+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:13:29.493579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780269+0000","last_change":"2023-11-27T15:20:33.223568+0000","last_active":"2023-11-27T15:21:02.780269+0000","last_peered":"2023-11-27T15:21:02.780269+0000","last_clean":"2023-11-27T15:21:02.780269+0000","last_became_active":"2023-11-27T15:20:33.220687+0000","last_became_peered":"2023-11-27T15:20:33.220687+0000","last_unstale":"2023-11-27T15:21:02.780269+0000","last_undegraded":"2023-11-27T15:21:02.780269+0000","last_fullsized":"2023-11-27T15:21:02.780269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:56:29.431345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786992+0000","last_change":"2023-11-27T15:21:02.786992+0000","last_active":"2023-11-27T15:21:02.786992+0000","last_peered":"2023-11-27T15:21:02.786992+0000","last_clean":"2023-11-27T15:21:02.786992+0000","last_became_active":"2023-11-27T15:21:02.786644+0000","last_became_peered":"2023-11-27T15:21:02.786644+0000","last_unstale":"2023-11-27T15:21:02.786992+0000","last_undegraded":"2023-11-27T15:21:02.786992+0000","last_fullsized":"2023-11-27T15:21:02.786992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:18:27.697514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.931958+0000","last_change":"2023-11-27T15:20:33.245815+0000","last_active":"2023-11-27T15:21:02.931958+0000","last_peered":"2023-11-27T15:21:02.931958+0000","last_clean":"2023-11-27T15:21:02.931958+0000","last_became_active":"2023-11-27T15:20:33.245243+0000","last_became_peered":"2023-11-27T15:20:33.245243+0000","last_unstale":"2023-11-27T15:21:02.931958+0000","last_undegraded":"2023-11-27T15:21:02.931958+0000","last_fullsized":"2023-11-27T15:21:02.931958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:26:43.029408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784226+0000","last_change":"2023-11-27T15:20:33.245595+0000","last_active":"2023-11-27T15:21:02.784226+0000","last_peered":"2023-11-27T15:21:02.784226+0000","last_clean":"2023-11-27T15:21:02.784226+0000","last_became_active":"2023-11-27T15:20:33.245294+0000","last_became_peered":"2023-11-27T15:20:33.245294+0000","last_unstale":"2023-11-27T15:21:02.784226+0000","last_undegraded":"2023-11-27T15:21:02.784226+0000","last_fullsized":"2023-11-27T15:21:02.784226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:38:31.036319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.795538+0000","last_change":"2023-11-27T15:20:33.232976+0000","last_active":"2023-11-27T15:21:02.795538+0000","last_peered":"2023-11-27T15:21:02.795538+0000","last_clean":"2023-11-27T15:21:02.795538+0000","last_became_active":"2023-11-27T15:20:33.232648+0000","last_became_peered":"2023-11-27T15:20:33.232648+0000","last_unstale":"2023-11-27T15:21:02.795538+0000","last_undegraded":"2023-11-27T15:21:02.795538+0000","last_fullsized":"2023-11-27T15:21:02.795538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:45:02.656102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783702+0000","last_change":"2023-11-27T15:20:33.226332+0000","last_active":"2023-11-27T15:21:02.783702+0000","last_peered":"2023-11-27T15:21:02.783702+0000","last_clean":"2023-11-27T15:21:02.783702+0000","last_became_active":"2023-11-27T15:20:33.226000+0000","last_became_peered":"2023-11-27T15:20:33.226000+0000","last_unstale":"2023-11-27T15:21:02.783702+0000","last_undegraded":"2023-11-27T15:21:02.783702+0000","last_fullsized":"2023-11-27T15:21:02.783702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:24:34.018543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932933+0000","last_change":"2023-11-27T15:20:33.217927+0000","last_active":"2023-11-27T15:21:02.932933+0000","last_peered":"2023-11-27T15:21:02.932933+0000","last_clean":"2023-11-27T15:21:02.932933+0000","last_became_active":"2023-11-27T15:20:33.217003+0000","last_became_peered":"2023-11-27T15:20:33.217003+0000","last_unstale":"2023-11-27T15:21:02.932933+0000","last_undegraded":"2023-11-27T15:21:02.932933+0000","last_fullsized":"2023-11-27T15:21:02.932933+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:27:50.580759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":41917,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":16,"num_read_kb":32,"num_write":131,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":274432,"data_stored":57313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6168,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93738984,"statfs":{"total":95995035648,"available":95988719616,"internally_reserved":0,"allocated":999424,"data_stored":490042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.57999999999999996}]},{"osd":1,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.72399999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":4,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.76800000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.747}]},{"osd":6,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.68400000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.79700000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":9,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.89200000000000002}]},{"osd":10,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.877}]}]},{"osd":10,"up_from":71,"seq":304942678038,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7068,"kb_used_data":1620,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738084,"statfs":{"total":95995035648,"available":95987798016,"internally_reserved":0,"allocated":1658880,"data_stored":1103505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.36599999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.64600000000000002}]},{"osd":3,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.51000000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.41199999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.57199999999999995}]},{"osd":6,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.55700000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.56799999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.67300000000000004}]},{"osd":9,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.58699999999999997}]},{"osd":11,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.70399999999999996}]}]},{"osd":9,"up_from":66,"seq":283467841561,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6380,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738772,"statfs":{"total":95995035648,"available":95988502528,"internally_reserved":0,"allocated":1019904,"data_stored":490860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.63600000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.71099999999999997}]},{"osd":2,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.97999999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.73599999999999999}]},{"osd":4,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.90100000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.996},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.88600000000000001}]},{"osd":6,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.80400000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.85199999999999998}]},{"osd":8,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.0649999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.91700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]}]},{"osd":8,"up_from":58,"seq":249108103199,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6412,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738740,"statfs":{"total":95995035648,"available":95988469760,"internally_reserved":0,"allocated":987136,"data_stored":480749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.58899999999999997,"15min":0.60099999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71199999999999997,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.47099999999999997}]},{"osd":1,"last update":"Mon Nov 27 15:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78500000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.746}]},{"osd":2,"last update":"Mon Nov 27 15:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.442},{"interface":"front","average":{"1min":0.495,"5min":0.45100000000000001,"15min":0.443},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.55500000000000005}]},{"osd":3,"last update":"Mon Nov 27 15:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.40899999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59099999999999997,"15min":0.59199999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.57599999999999996}]},{"osd":4,"last update":"Mon Nov 27 15:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80000000000000004,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.47399999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.88400000000000001,"5min":1.377,"15min":1.377},"last":0.59899999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58099999999999996,"15min":0.57999999999999996},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80900000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.71299999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.52000000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.91000000000000003}]},{"osd":7,"last update":"Mon Nov 27 15:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95799999999999996,"5min":1.323,"15min":1.323},"last":0.83699999999999997}]},{"osd":9,"last update":"Mon Nov 27 15:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.93100000000000005}]},{"osd":10,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.88400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]}]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7156,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93737996,"statfs":{"total":95995035648,"available":95987707904,"internally_reserved":0,"allocated":1617920,"data_stored":1090688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.436,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84699999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.44},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77100000000000002,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.46999999999999997}]},{"osd":1,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.39800000000000002,"15min":0.39000000000000001},"min":{"1min":0.27900000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.40000000000000002,"15min":0.39600000000000002},"min":{"1min":0.32100000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.39800000000000002}]},{"osd":2,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.622,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.63700000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61799999999999999,"15min":0.62},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85699999999999998,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64300000000000002,"15min":0.63700000000000001},"min":{"1min":0.51500000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.61299999999999999}]},{"osd":4,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.504,"15min":0.49099999999999999},"min":{"1min":0.34699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.40100000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67800000000000005,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.38200000000000001}]},{"osd":5,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.46999999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.76200000000000001,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.495,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.753,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.59699999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.68400000000000005,"15min":0.67200000000000004},"min":{"1min":0.48499999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.70099999999999996,"15min":0.69799999999999995},"min":{"1min":0.48599999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.93600000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.58499999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64900000000000002,"15min":0.64400000000000002},"min":{"1min":0.46100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.48599999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.85699999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.623}]},{"osd":9,"last update":"Mon Nov 27 15:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.747}]},{"osd":10,"last update":"Mon Nov 27 15:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.73399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]}]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6428,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738724,"statfs":{"total":95995035648,"available":95988453376,"internally_reserved":0,"allocated":1003520,"data_stored":490132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47699999999999998,"15min":0.46999999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.50700000000000001,"15min":0.48499999999999999},"min":{"1min":0.36599999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.54300000000000004}]},{"osd":1,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65600000000000003,"15min":0.63600000000000001},"min":{"1min":0.45300000000000001,"5min":0.441,"15min":0.441},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60899999999999999,"15min":0.57899999999999996},"min":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.55500000000000005}]},{"osd":2,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.64800000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68200000000000005,"15min":0.67400000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.70999999999999996}]},{"osd":3,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55700000000000005,"15min":0.54400000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.61299999999999999,"15min":0.59599999999999997},"min":{"1min":0.38900000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.498}]},{"osd":4,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69399999999999995,"15min":0.68600000000000005},"min":{"1min":0.55700000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.63},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68500000000000005,"15min":0.66900000000000004},"min":{"1min":0.48799999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.61199999999999999}]},{"osd":5,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71899999999999997,"15min":0.70799999999999996},"min":{"1min":0.505,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76300000000000001,"15min":0.753},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.67700000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.753,"15min":0.747},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.72499999999999998,"15min":0.70799999999999996},"min":{"1min":0.54700000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.64600000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.86099999999999999,"15min":0.86499999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.84799999999999998,"15min":0.85799999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.82399999999999995}]},{"osd":9,"last update":"Mon Nov 27 15:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.81799999999999995}]},{"osd":10,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.76800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7332,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737820,"statfs":{"total":95995035648,"available":95987527680,"internally_reserved":0,"allocated":1011712,"data_stored":493836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.501,"15min":0.48299999999999998},"min":{"1min":0.38500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73499999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.51300000000000001,"15min":0.48199999999999998},"min":{"1min":0.36699999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.67900000000000005}]},{"osd":2,"last update":"Mon Nov 27 15:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.501,"15min":0.47999999999999998},"min":{"1min":0.38200000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.52300000000000002,"15min":0.48599999999999999},"min":{"1min":0.39800000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.90300000000000002,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.77100000000000002}]},{"osd":3,"last update":"Mon Nov 27 15:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.55500000000000005,"15min":0.52800000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54700000000000004,"15min":0.53400000000000003},"min":{"1min":0.39700000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84899999999999998,"5min":0.88,"15min":0.88},"last":0.58199999999999996}]},{"osd":4,"last update":"Mon Nov 27 15:20:15 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.40500000000000003,"15min":0.38800000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.42799999999999999,"15min":0.40500000000000003},"min":{"1min":0.29899999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.505}]},{"osd":5,"last update":"Mon Nov 27 15:20:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54900000000000004,"15min":0.53900000000000003},"min":{"1min":0.38100000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.57599999999999996,"15min":0.55600000000000005},"min":{"1min":0.41899999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.80900000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.64700000000000002,"15min":0.624},"min":{"1min":0.45900000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.747},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.64400000000000002,"15min":0.62},"min":{"1min":0.53400000000000003,"5min":0.38,"15min":0.38},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.83599999999999997}]},{"osd":7,"last update":"Mon Nov 27 15:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.36599999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.59999999999999998,"15min":0.58699999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.878}]},{"osd":8,"last update":"Mon Nov 27 15:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64500000000000002,"15min":0.63700000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64500000000000002,"15min":0.63800000000000001},"min":{"1min":0.47999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.76400000000000001}]},{"osd":9,"last update":"Mon Nov 27 15:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.93400000000000005}]},{"osd":10,"last update":"Mon Nov 27 15:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.86099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7412,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737740,"statfs":{"total":95995035648,"available":95987445760,"internally_reserved":0,"allocated":1028096,"data_stored":491520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.53700000000000003,"15min":0.499},"min":{"1min":0.433,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.56299999999999994,"15min":0.51700000000000002},"min":{"1min":0.375,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.77000000000000002}]},{"osd":2,"last update":"Mon Nov 27 15:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.59899999999999998,"15min":0.57299999999999995},"min":{"1min":0.41599999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.59999999999999998,"15min":0.55700000000000005},"min":{"1min":0.45700000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.57099999999999995}]},{"osd":3,"last update":"Mon Nov 27 15:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.50800000000000001,"15min":0.49399999999999999},"min":{"1min":0.35499999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.497},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53200000000000003,"15min":0.52400000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.84499999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.58699999999999997}]},{"osd":4,"last update":"Mon Nov 27 15:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.622,"15min":0.60999999999999999},"min":{"1min":0.45400000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.63,"15min":0.59999999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.72199999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.45200000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.69699999999999995,"15min":0.67900000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.625}]},{"osd":6,"last update":"Mon Nov 27 15:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65500000000000003,"15min":0.627},"min":{"1min":0.41499999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.121,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63600000000000001,"15min":0.61799999999999999},"min":{"1min":0.437,"5min":0.373,"15min":0.373},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.437}]},{"osd":7,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.71799999999999997,"15min":0.71099999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.77500000000000002,"15min":0.76800000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.61499999999999999}]},{"osd":8,"last update":"Mon Nov 27 15:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.748,"15min":0.752},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.78000000000000003,"15min":0.78600000000000003},"min":{"1min":0.58999999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.60399999999999998}]},{"osd":9,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.84399999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.76100000000000001}]},{"osd":11,"last update":"Mon Nov 27 15:20:50 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.79100000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7404,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737748,"statfs":{"total":95995035648,"available":95987453952,"internally_reserved":0,"allocated":1019904,"data_stored":493775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51300000000000001,"15min":0.5},"min":{"1min":0.38100000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.50800000000000001,"15min":0.51500000000000001},"min":{"1min":0.33900000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.77900000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.83799999999999997}]},{"osd":1,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51200000000000001,"15min":0.48499999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.51400000000000001,"15min":0.49199999999999999},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.82799999999999996}]},{"osd":3,"last update":"Mon Nov 27 15:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58199999999999996,"15min":0.56299999999999994},"min":{"1min":0.443,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.56100000000000005},"min":{"1min":0.42599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77000000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.92300000000000004}]},{"osd":4,"last update":"Mon Nov 27 15:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52500000000000002,"15min":0.503},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.51500000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.76400000000000001}]},{"osd":5,"last update":"Mon Nov 27 15:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.434,"15min":0.42299999999999999},"min":{"1min":0.28899999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.70399999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.45900000000000002,"15min":0.44700000000000001},"min":{"1min":0.28899999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.61199999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.62,"15min":0.60299999999999998},"min":{"1min":0.42099999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.80000000000000004}]},{"osd":7,"last update":"Mon Nov 27 15:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.65100000000000002,"15min":0.63900000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":4.3499999999999996,"5min":4.3499999999999996,"15min":4.3499999999999996},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.73699999999999999}]},{"osd":8,"last update":"Mon Nov 27 15:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.35299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.71499999999999997}]},{"osd":9,"last update":"Mon Nov 27 15:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.91200000000000003}]},{"osd":10,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":3.2999999999999998,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.875},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.88800000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6760,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738392,"statfs":{"total":95995035648,"available":95988113408,"internally_reserved":0,"allocated":1015808,"data_stored":490326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.42499999999999999,"15min":0.40000000000000002},"min":{"1min":0.33400000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72099999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.45600000000000002,"15min":0.41699999999999998},"min":{"1min":0.29999999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.58399999999999996,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.60399999999999998}]},{"osd":1,"last update":"Mon Nov 27 15:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61099999999999999,"15min":0.56499999999999995},"min":{"1min":0.42999999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.872,"5min":1.056,"15min":1.056},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63500000000000001,"15min":0.57099999999999995},"min":{"1min":0.40699999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90300000000000002,"5min":5.6529999999999996,"15min":5.6529999999999996},"last":0.58999999999999997}]},{"osd":2,"last update":"Mon Nov 27 15:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64000000000000001,"15min":0.59399999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85099999999999998,"5min":1.208,"15min":1.208},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.63200000000000001,"15min":0.58799999999999997},"min":{"1min":0.49099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78900000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.65300000000000002}]},{"osd":4,"last update":"Mon Nov 27 15:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64000000000000001,"15min":0.626},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91900000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.626},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.69299999999999995,"15min":0.66800000000000004},"min":{"1min":0.52500000000000002,"5min":0.502,"15min":0.502},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.86199999999999999}]},{"osd":5,"last update":"Mon Nov 27 15:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67000000000000004,"15min":0.65000000000000002},"min":{"1min":0.48399999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94099999999999995,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66900000000000004,"15min":0.65300000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.97099999999999997,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.73399999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.65600000000000003,"15min":0.63800000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0269999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.41399999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94399999999999995,"5min":1.018,"15min":1.018},"last":0.55200000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:20:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.01,"15min":0.99399999999999999},"min":{"1min":0.51800000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":6.9619999999999997,"5min":6.9619999999999997,"15min":6.9619999999999997},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.78400000000000003,"15min":0.77800000000000002},"min":{"1min":0.58199999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.71399999999999997}]},{"osd":8,"last update":"Mon Nov 27 15:19:38 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":5.6100000000000003,"5min":5.6100000000000003,"15min":5.6100000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.64400000000000002}]},{"osd":9,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.94399999999999995}]},{"osd":10,"last update":"Mon Nov 27 15:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.75},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.83999999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]}]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6620,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738532,"statfs":{"total":95995035648,"available":95988256768,"internally_reserved":0,"allocated":1003520,"data_stored":490064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.54400000000000004,"15min":0.53700000000000003},"min":{"1min":0.38300000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53500000000000003,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77900000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.54900000000000004}]},{"osd":1,"last update":"Mon Nov 27 15:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.36399999999999999,"15min":0.34499999999999997},"min":{"1min":0.28999999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.56499999999999995,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.372},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.374,"15min":0.35699999999999998},"min":{"1min":0.26600000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.35299999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56299999999999994,"15min":0.54000000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85199999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.67700000000000005}]},{"osd":3,"last update":"Mon Nov 27 15:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56299999999999994,"15min":0.54700000000000004},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.41099999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.48699999999999999}]},{"osd":5,"last update":"Mon Nov 27 15:20:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.62,"15min":0.60599999999999998},"min":{"1min":0.45000000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.62,"15min":0.60299999999999998},"min":{"1min":0.39500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.57399999999999995}]},{"osd":6,"last update":"Mon Nov 27 15:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.45800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.66900000000000004,"15min":0.65700000000000003},"min":{"1min":0.53700000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.58399999999999996}]},{"osd":7,"last update":"Mon Nov 27 15:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.39300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.93400000000000005,"5min":1.054,"15min":1.054},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.35299999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.46899999999999997}]},{"osd":8,"last update":"Mon Nov 27 15:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70699999999999996,"15min":0.70299999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.499,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.61699999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.55900000000000005}]},{"osd":10,"last update":"Mon Nov 27 15:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.435}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7284,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93737868,"statfs":{"total":95995035648,"available":95987576832,"internally_reserved":0,"allocated":1617920,"data_stored":1081507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.51800000000000002,"15min":0.52100000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71899999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.64900000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.64600000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:20:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.40500000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70399999999999996,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55300000000000005,"15min":0.53500000000000003},"min":{"1min":0.443,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76400000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.38400000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:20:30 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.42099999999999999,"15min":0.42299999999999999},"min":{"1min":0.26400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.49199999999999999}]},{"osd":3,"last update":"Mon Nov 27 15:20:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56000000000000005,"15min":0.56299999999999994},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.58899999999999997}]},{"osd":4,"last update":"Mon Nov 27 15:20:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.46100000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.437,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80100000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.432}]},{"osd":6,"last update":"Mon Nov 27 15:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.625,"15min":0.63500000000000001},"min":{"1min":0.41899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.44400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.93500000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.69399999999999995}]},{"osd":7,"last update":"Mon Nov 27 15:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.39900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85099999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.55600000000000005}]},{"osd":8,"last update":"Mon Nov 27 15:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.61399999999999999,"15min":0.622},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.621},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.40799999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67900000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.71999999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.73099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":40435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-27T15:21:08.145 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-27T15:21:08.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:07 smithi174 ceph-mon[131110]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:21:08.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:07 smithi174 ceph-mon[131110]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:21:08.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:07 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/558405952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:21:08.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:07 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1368723892' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:21:08.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.370+0000 7f77a33ff700 1 -- 172.21.15.133:0/350237054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f779c105f60 msgr2=0x7f779c1083c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:08.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.370+0000 7f77a33ff700 1 --2- 172.21.15.133:0/350237054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f779c105f60 0x7f779c1083c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f779400b0a0 tx=0x7f779400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:21:08.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.371+0000 7f77a33ff700 1 -- 172.21.15.133:0/350237054 shutdown_connections 2023-11-27T15:21:08.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.371+0000 7f77a33ff700 1 --2- 172.21.15.133:0/350237054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f779c105f60 0x7f779c1083c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:08.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.371+0000 7f77a33ff700 1 --2- 172.21.15.133:0/350237054 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779c100d20 0x7f779c103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:08.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.371+0000 7f77a33ff700 1 --2- 172.21.15.133:0/350237054 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f779c103640 0x7f779c105a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:08.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.371+0000 7f77a33ff700 1 -- 172.21.15.133:0/350237054 >> 172.21.15.133:0/350237054 conn(0x7f779c0fa7a0 msgr2=0x7f779c0fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:08.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.371+0000 7f77a33ff700 1 -- 172.21.15.133:0/350237054 shutdown_connections 2023-11-27T15:21:08.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.371+0000 7f77a33ff700 1 -- 172.21.15.133:0/350237054 wait complete. 2023-11-27T15:21:08.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.372+0000 7f77a33ff700 1 Processor -- start 2023-11-27T15:21:08.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.372+0000 7f77a33ff700 1 -- start start 2023-11-27T15:21:08.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.372+0000 7f77a33ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f779c100d20 0x7f779c100690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:08.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.372+0000 7f77a33ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f779c103640 0x7f779c0fecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:08.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.372+0000 7f77a33ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779c105f60 0x7f779c0ff210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.372+0000 7f77a33ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c0ff9b0 con 0x7f779c105f60 2023-11-27T15:21:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.372+0000 7f77a33ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c0ffb20 con 0x7f779c103640 2023-11-27T15:21:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.372+0000 7f77a33ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c1a0100 con 0x7f779c100d20 2023-11-27T15:21:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.372+0000 7f77a199c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779c105f60 0x7f779c0ff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.373+0000 7f77a199c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779c105f60 0x7f779c0ff210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36088/0 (socket says 172.21.15.133:36088) 2023-11-27T15:21:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.373+0000 7f77a199c700 1 -- 172.21.15.133:0/3646919908 learned_addr learned my addr 172.21.15.133:0/3646919908 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.373+0000 7f77a119b700 1 --2- 172.21.15.133:0/3646919908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f779c100d20 0x7f779c100690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.373+0000 7f77a099a700 1 --2- 172.21.15.133:0/3646919908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f779c103640 0x7f779c0fecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.373+0000 7f77a199c700 1 -- 172.21.15.133:0/3646919908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f779c100d20 msgr2=0x7f779c100690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:08.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.373+0000 7f77a199c700 1 --2- 172.21.15.133:0/3646919908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f779c100d20 0x7f779c100690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:08.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.373+0000 7f77a199c700 1 -- 172.21.15.133:0/3646919908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f779c103640 msgr2=0x7f779c0fecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:08.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.373+0000 7f77a199c700 1 --2- 172.21.15.133:0/3646919908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f779c103640 0x7f779c0fecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:08.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.373+0000 7f77a199c700 1 -- 172.21.15.133:0/3646919908 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7794009d00 con 0x7f779c105f60 2023-11-27T15:21:08.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.373+0000 7f77a199c700 1 --2- 172.21.15.133:0/3646919908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779c105f60 0x7f779c0ff210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f779400b780 tx=0x7f779400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:08.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.373+0000 7f77927fc700 1 -- 172.21.15.133:0/3646919908 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779400e040 con 0x7f779c105f60 2023-11-27T15:21:08.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.373+0000 7f77927fc700 1 -- 172.21.15.133:0/3646919908 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77940094a0 con 0x7f779c105f60 2023-11-27T15:21:08.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.373+0000 7f77a33ff700 1 -- 172.21.15.133:0/3646919908 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f779c10bc00 con 0x7f779c105f60 2023-11-27T15:21:08.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.373+0000 7f77927fc700 1 -- 172.21.15.133:0/3646919908 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77940052a0 con 0x7f779c105f60 2023-11-27T15:21:08.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.373+0000 7f77a33ff700 1 -- 172.21.15.133:0/3646919908 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779c10ba70 con 0x7f779c105f60 2023-11-27T15:21:08.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.375+0000 7f77a33ff700 1 -- 172.21.15.133:0/3646919908 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f779c0fc310 con 0x7f779c105f60 2023-11-27T15:21:08.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.376+0000 7f77927fc700 1 -- 172.21.15.133:0/3646919908 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f779401d470 con 0x7f779c105f60 2023-11-27T15:21:08.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.378+0000 7f77927fc700 1 --2- 172.21.15.133:0/3646919908 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f77880a1260 0x7f77880a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:08.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.379+0000 7f77927fc700 1 -- 172.21.15.133:0/3646919908 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f77940ce3c0 con 0x7f779c105f60 2023-11-27T15:21:08.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.379+0000 7f77a119b700 1 --2- 172.21.15.133:0/3646919908 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f77880a1260 0x7f77880a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:08.384 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.379+0000 7f77a119b700 1 --2- 172.21.15.133:0/3646919908 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f77880a1260 0x7f77880a3710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7798009c60 tx=0x7f7798009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:08.387 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.382+0000 7f77927fc700 1 -- 172.21.15.133:0/3646919908 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7794097730 con 0x7f779c105f60 2023-11-27T15:21:08.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.576+0000 7f77a33ff700 1 -- 172.21.15.133:0/3646919908 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f779c061a50 con 0x7f77880a1260 2023-11-27T15:21:08.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.589+0000 7f77927fc700 1 -- 172.21.15.133:0/3646919908 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+422572 (secure 0 0 0) 0x7f779c061a50 con 0x7f77880a1260 2023-11-27T15:21:08.599 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:21:08.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.594+0000 7f77a33ff700 1 -- 172.21.15.133:0/3646919908 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f77880a1260 msgr2=0x7f77880a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:08.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.594+0000 7f77a33ff700 1 --2- 172.21.15.133:0/3646919908 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f77880a1260 0x7f77880a3710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7798009c60 tx=0x7f7798009260 comp rx=0 tx=0).stop 2023-11-27T15:21:08.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.594+0000 7f77a33ff700 1 -- 172.21.15.133:0/3646919908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779c105f60 msgr2=0x7f779c0ff210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:08.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.594+0000 7f77a33ff700 1 --2- 172.21.15.133:0/3646919908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779c105f60 0x7f779c0ff210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f779400b780 tx=0x7f779400bd60 comp rx=0 tx=0).stop 2023-11-27T15:21:08.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.594+0000 7f77a33ff700 1 -- 172.21.15.133:0/3646919908 shutdown_connections 2023-11-27T15:21:08.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.594+0000 7f77a33ff700 1 --2- 172.21.15.133:0/3646919908 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f77880a1260 0x7f77880a3710 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:08.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.594+0000 7f77a33ff700 1 --2- 172.21.15.133:0/3646919908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779c105f60 0x7f779c0ff210 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:08.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.594+0000 7f77a33ff700 1 --2- 172.21.15.133:0/3646919908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f779c100d20 0x7f779c100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:08.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.594+0000 7f77a33ff700 1 --2- 172.21.15.133:0/3646919908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f779c103640 0x7f779c0fecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:08.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.594+0000 7f77a33ff700 1 -- 172.21.15.133:0/3646919908 >> 172.21.15.133:0/3646919908 conn(0x7f779c0fa7a0 msgr2=0x7f779c101980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:08.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.594+0000 7f77a33ff700 1 -- 172.21.15.133:0/3646919908 shutdown_connections 2023-11-27T15:21:08.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:08.594+0000 7f77a33ff700 1 -- 172.21.15.133:0/3646919908 wait complete. 2023-11-27T15:21:08.604 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2023-11-27T15:21:08.618 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2023-11-27T15:21:07.716069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632408,"num_objects":57,"num_object_clones":0,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":142,"num_read_kb":141,"num_write":313,"num_write_kb":3012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":253,"ondisk_log_size":253,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":82424,"kb_used_data":13656,"kb_used_omap":0,"kb_used_meta":68672,"kb_avail":1124859400,"statfs":{"total":1151940427776,"available":1151856025600,"internally_reserved":0,"allocated":13983744,"data_stored":7687004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":70320128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":37448,"num_objects":33,"num_object_clones":0,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":16,"num_read_kb":32,"num_write":83,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002256"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.933701+0000","last_change":"2023-11-27T15:20:33.236352+0000","last_active":"2023-11-27T15:21:02.933701+0000","last_peered":"2023-11-27T15:21:02.933701+0000","last_clean":"2023-11-27T15:21:02.933701+0000","last_became_active":"2023-11-27T15:20:33.235974+0000","last_became_peered":"2023-11-27T15:20:33.235974+0000","last_unstale":"2023-11-27T15:21:02.933701+0000","last_undegraded":"2023-11-27T15:21:02.933701+0000","last_fullsized":"2023-11-27T15:21:02.933701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:42:55.142804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786021+0000","last_change":"2023-11-27T15:20:33.234882+0000","last_active":"2023-11-27T15:21:02.786021+0000","last_peered":"2023-11-27T15:21:02.786021+0000","last_clean":"2023-11-27T15:21:02.786021+0000","last_became_active":"2023-11-27T15:20:33.231062+0000","last_became_peered":"2023-11-27T15:20:33.231062+0000","last_unstale":"2023-11-27T15:21:02.786021+0000","last_undegraded":"2023-11-27T15:21:02.786021+0000","last_fullsized":"2023-11-27T15:21:02.786021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:18:27.469196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785364+0000","last_change":"2023-11-27T15:20:33.238280+0000","last_active":"2023-11-27T15:21:02.785364+0000","last_peered":"2023-11-27T15:21:02.785364+0000","last_clean":"2023-11-27T15:21:02.785364+0000","last_became_active":"2023-11-27T15:20:33.237602+0000","last_became_peered":"2023-11-27T15:20:33.237602+0000","last_unstale":"2023-11-27T15:21:02.785364+0000","last_undegraded":"2023-11-27T15:21:02.785364+0000","last_fullsized":"2023-11-27T15:21:02.785364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:28:48.211669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784641+0000","last_change":"2023-11-27T15:20:33.217909+0000","last_active":"2023-11-27T15:21:02.784641+0000","last_peered":"2023-11-27T15:21:02.784641+0000","last_clean":"2023-11-27T15:21:02.784641+0000","last_became_active":"2023-11-27T15:20:33.217511+0000","last_became_peered":"2023-11-27T15:20:33.217511+0000","last_unstale":"2023-11-27T15:21:02.784641+0000","last_undegraded":"2023-11-27T15:21:02.784641+0000","last_fullsized":"2023-11-27T15:21:02.784641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:37:24.759082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782641+0000","last_change":"2023-11-27T15:20:33.246630+0000","last_active":"2023-11-27T15:21:02.782641+0000","last_peered":"2023-11-27T15:21:02.782641+0000","last_clean":"2023-11-27T15:21:02.782641+0000","last_became_active":"2023-11-27T15:20:33.246249+0000","last_became_peered":"2023-11-27T15:20:33.246249+0000","last_unstale":"2023-11-27T15:21:02.782641+0000","last_undegraded":"2023-11-27T15:21:02.782641+0000","last_fullsized":"2023-11-27T15:21:02.782641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:14:48.110743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932766+0000","last_change":"2023-11-27T15:20:33.236280+0000","last_active":"2023-11-27T15:21:02.932766+0000","last_peered":"2023-11-27T15:21:02.932766+0000","last_clean":"2023-11-27T15:21:02.932766+0000","last_became_active":"2023-11-27T15:20:33.235983+0000","last_became_peered":"2023-11-27T15:20:33.235983+0000","last_unstale":"2023-11-27T15:21:02.932766+0000","last_undegraded":"2023-11-27T15:21:02.932766+0000","last_fullsized":"2023-11-27T15:21:02.932766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:32:27.538426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779828+0000","last_change":"2023-11-27T15:20:33.223085+0000","last_active":"2023-11-27T15:21:02.779828+0000","last_peered":"2023-11-27T15:21:02.779828+0000","last_clean":"2023-11-27T15:21:02.779828+0000","last_became_active":"2023-11-27T15:20:33.222809+0000","last_became_peered":"2023-11-27T15:20:33.222809+0000","last_unstale":"2023-11-27T15:21:02.779828+0000","last_undegraded":"2023-11-27T15:21:02.779828+0000","last_fullsized":"2023-11-27T15:21:02.779828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:46:23.567184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783786+0000","last_change":"2023-11-27T15:20:33.226217+0000","last_active":"2023-11-27T15:21:02.783786+0000","last_peered":"2023-11-27T15:21:02.783786+0000","last_clean":"2023-11-27T15:21:02.783786+0000","last_became_active":"2023-11-27T15:20:33.225856+0000","last_became_peered":"2023-11-27T15:20:33.225856+0000","last_unstale":"2023-11-27T15:21:02.783786+0000","last_undegraded":"2023-11-27T15:21:02.783786+0000","last_fullsized":"2023-11-27T15:21:02.783786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:41:31.773939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.795641+0000","last_change":"2023-11-27T15:20:33.227554+0000","last_active":"2023-11-27T15:21:02.795641+0000","last_peered":"2023-11-27T15:21:02.795641+0000","last_clean":"2023-11-27T15:21:02.795641+0000","last_became_active":"2023-11-27T15:20:33.227222+0000","last_became_peered":"2023-11-27T15:20:33.227222+0000","last_unstale":"2023-11-27T15:21:02.795641+0000","last_undegraded":"2023-11-27T15:21:02.795641+0000","last_fullsized":"2023-11-27T15:21:02.795641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:34:15.927242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.933040+0000","last_change":"2023-11-27T15:20:33.230528+0000","last_active":"2023-11-27T15:21:02.933040+0000","last_peered":"2023-11-27T15:21:02.933040+0000","last_clean":"2023-11-27T15:21:02.933040+0000","last_became_active":"2023-11-27T15:20:33.230148+0000","last_became_peered":"2023-11-27T15:20:33.230148+0000","last_unstale":"2023-11-27T15:21:02.933040+0000","last_undegraded":"2023-11-27T15:21:02.933040+0000","last_fullsized":"2023-11-27T15:21:02.933040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:56:46.117728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780265+0000","last_change":"2023-11-27T15:20:33.228098+0000","last_active":"2023-11-27T15:21:02.780265+0000","last_peered":"2023-11-27T15:21:02.780265+0000","last_clean":"2023-11-27T15:21:02.780265+0000","last_became_active":"2023-11-27T15:20:33.227760+0000","last_became_peered":"2023-11-27T15:20:33.227760+0000","last_unstale":"2023-11-27T15:21:02.780265+0000","last_undegraded":"2023-11-27T15:21:02.780265+0000","last_fullsized":"2023-11-27T15:21:02.780265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:27:54.894445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.935458+0000","last_change":"2023-11-27T15:21:02.934583+0000","last_active":"2023-11-27T15:21:02.935458+0000","last_peered":"2023-11-27T15:21:02.935458+0000","last_clean":"2023-11-27T15:21:02.935458+0000","last_became_active":"2023-11-27T15:21:02.933631+0000","last_became_peered":"2023-11-27T15:21:02.933631+0000","last_unstale":"2023-11-27T15:21:02.935458+0000","last_undegraded":"2023-11-27T15:21:02.935458+0000","last_fullsized":"2023-11-27T15:21:02.935458+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:45:15.679810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783578+0000","last_change":"2023-11-27T15:20:33.231255+0000","last_active":"2023-11-27T15:21:02.783578+0000","last_peered":"2023-11-27T15:21:02.783578+0000","last_clean":"2023-11-27T15:21:02.783578+0000","last_became_active":"2023-11-27T15:20:33.230867+0000","last_became_peered":"2023-11-27T15:20:33.230867+0000","last_unstale":"2023-11-27T15:21:02.783578+0000","last_undegraded":"2023-11-27T15:21:02.783578+0000","last_fullsized":"2023-11-27T15:21:02.783578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:58:03.899781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785072+0000","last_change":"2023-11-27T15:20:33.230306+0000","last_active":"2023-11-27T15:21:02.785072+0000","last_peered":"2023-11-27T15:21:02.785072+0000","last_clean":"2023-11-27T15:21:02.785072+0000","last_became_active":"2023-11-27T15:20:33.229892+0000","last_became_peered":"2023-11-27T15:20:33.229892+0000","last_unstale":"2023-11-27T15:21:02.785072+0000","last_undegraded":"2023-11-27T15:21:02.785072+0000","last_fullsized":"2023-11-27T15:21:02.785072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:24:55.365713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785143+0000","last_change":"2023-11-27T15:20:33.220592+0000","last_active":"2023-11-27T15:21:02.785143+0000","last_peered":"2023-11-27T15:21:02.785143+0000","last_clean":"2023-11-27T15:21:02.785143+0000","last_became_active":"2023-11-27T15:20:33.220258+0000","last_became_peered":"2023-11-27T15:20:33.220258+0000","last_unstale":"2023-11-27T15:21:02.785143+0000","last_undegraded":"2023-11-27T15:21:02.785143+0000","last_fullsized":"2023-11-27T15:21:02.785143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:48:48.921465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.794878+0000","last_change":"2023-11-27T15:20:33.245041+0000","last_active":"2023-11-27T15:21:02.794878+0000","last_peered":"2023-11-27T15:21:02.794878+0000","last_clean":"2023-11-27T15:21:02.794878+0000","last_became_active":"2023-11-27T15:20:33.244759+0000","last_became_peered":"2023-11-27T15:20:33.244759+0000","last_unstale":"2023-11-27T15:21:02.794878+0000","last_undegraded":"2023-11-27T15:21:02.794878+0000","last_fullsized":"2023-11-27T15:21:02.794878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:31:00.164868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.934087+0000","last_change":"2023-11-27T15:20:33.243288+0000","last_active":"2023-11-27T15:21:02.934087+0000","last_peered":"2023-11-27T15:21:02.934087+0000","last_clean":"2023-11-27T15:21:02.934087+0000","last_became_active":"2023-11-27T15:20:33.242989+0000","last_became_peered":"2023-11-27T15:20:33.242989+0000","last_unstale":"2023-11-27T15:21:02.934087+0000","last_undegraded":"2023-11-27T15:21:02.934087+0000","last_fullsized":"2023-11-27T15:21:02.934087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:25:53.302075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779978+0000","last_change":"2023-11-27T15:20:33.229613+0000","last_active":"2023-11-27T15:21:02.779978+0000","last_peered":"2023-11-27T15:21:02.779978+0000","last_clean":"2023-11-27T15:21:02.779978+0000","last_became_active":"2023-11-27T15:20:33.227747+0000","last_became_peered":"2023-11-27T15:20:33.227747+0000","last_unstale":"2023-11-27T15:21:02.779978+0000","last_undegraded":"2023-11-27T15:21:02.779978+0000","last_fullsized":"2023-11-27T15:21:02.779978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:18:24.422450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780242+0000","last_change":"2023-11-27T15:20:33.227004+0000","last_active":"2023-11-27T15:21:02.780242+0000","last_peered":"2023-11-27T15:21:02.780242+0000","last_clean":"2023-11-27T15:21:02.780242+0000","last_became_active":"2023-11-27T15:20:33.226703+0000","last_became_peered":"2023-11-27T15:20:33.226703+0000","last_unstale":"2023-11-27T15:21:02.780242+0000","last_undegraded":"2023-11-27T15:21:02.780242+0000","last_fullsized":"2023-11-27T15:21:02.780242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:18:04.531443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779865+0000","last_change":"2023-11-27T15:20:37.682021+0000","last_active":"2023-11-27T15:21:02.779865+0000","last_peered":"2023-11-27T15:21:02.779865+0000","last_clean":"2023-11-27T15:21:02.779865+0000","last_became_active":"2023-11-27T15:20:37.681600+0000","last_became_peered":"2023-11-27T15:20:37.681600+0000","last_unstale":"2023-11-27T15:21:02.779865+0000","last_undegraded":"2023-11-27T15:21:02.779865+0000","last_fullsized":"2023-11-27T15:21:02.779865+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:27:41.540353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.801130+0000","last_change":"2023-11-27T15:20:32.657542+0000","last_active":"2023-11-27T15:21:03.801130+0000","last_peered":"2023-11-27T15:21:03.801130+0000","last_clean":"2023-11-27T15:21:03.801130+0000","last_became_active":"2023-11-27T15:20:32.657158+0000","last_became_peered":"2023-11-27T15:20:32.657158+0000","last_unstale":"2023-11-27T15:21:03.801130+0000","last_undegraded":"2023-11-27T15:21:03.801130+0000","last_fullsized":"2023-11-27T15:21:03.801130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:01:39.181861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.935289+0000","last_change":"2023-11-27T15:21:02.933684+0000","last_active":"2023-11-27T15:21:02.935289+0000","last_peered":"2023-11-27T15:21:02.935289+0000","last_clean":"2023-11-27T15:21:02.935289+0000","last_became_active":"2023-11-27T15:21:02.933215+0000","last_became_peered":"2023-11-27T15:21:02.933215+0000","last_unstale":"2023-11-27T15:21:02.935289+0000","last_undegraded":"2023-11-27T15:21:02.935289+0000","last_fullsized":"2023-11-27T15:21:02.935289+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:59:39.553465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783397+0000","last_change":"2023-11-27T15:20:37.666705+0000","last_active":"2023-11-27T15:21:02.783397+0000","last_peered":"2023-11-27T15:21:02.783397+0000","last_clean":"2023-11-27T15:21:02.783397+0000","last_became_active":"2023-11-27T15:20:37.666236+0000","last_became_peered":"2023-11-27T15:20:37.666236+0000","last_unstale":"2023-11-27T15:21:02.783397+0000","last_undegraded":"2023-11-27T15:21:02.783397+0000","last_fullsized":"2023-11-27T15:21:02.783397+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:49:31.365357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785700+0000","last_change":"2023-11-27T15:20:32.663915+0000","last_active":"2023-11-27T15:21:02.785700+0000","last_peered":"2023-11-27T15:21:02.785700+0000","last_clean":"2023-11-27T15:21:02.785700+0000","last_became_active":"2023-11-27T15:20:32.663069+0000","last_became_peered":"2023-11-27T15:20:32.663069+0000","last_unstale":"2023-11-27T15:21:02.785700+0000","last_undegraded":"2023-11-27T15:21:02.785700+0000","last_fullsized":"2023-11-27T15:21:02.785700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:15:25.522191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785737+0000","last_change":"2023-11-27T15:20:33.238150+0000","last_active":"2023-11-27T15:21:02.785737+0000","last_peered":"2023-11-27T15:21:02.785737+0000","last_clean":"2023-11-27T15:21:02.785737+0000","last_became_active":"2023-11-27T15:20:33.236972+0000","last_became_peered":"2023-11-27T15:20:33.236972+0000","last_unstale":"2023-11-27T15:21:02.785737+0000","last_undegraded":"2023-11-27T15:21:02.785737+0000","last_fullsized":"2023-11-27T15:21:02.785737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:32:03.275170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783667+0000","last_change":"2023-11-27T15:20:37.668031+0000","last_active":"2023-11-27T15:21:02.783667+0000","last_peered":"2023-11-27T15:21:02.783667+0000","last_clean":"2023-11-27T15:21:02.783667+0000","last_became_active":"2023-11-27T15:20:37.667640+0000","last_became_peered":"2023-11-27T15:20:37.667640+0000","last_unstale":"2023-11-27T15:21:02.783667+0000","last_undegraded":"2023-11-27T15:21:02.783667+0000","last_fullsized":"2023-11-27T15:21:02.783667+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:45:18.290503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783029+0000","last_change":"2023-11-27T15:20:32.655738+0000","last_active":"2023-11-27T15:21:02.783029+0000","last_peered":"2023-11-27T15:21:02.783029+0000","last_clean":"2023-11-27T15:21:02.783029+0000","last_became_active":"2023-11-27T15:20:32.655439+0000","last_became_peered":"2023-11-27T15:20:32.655439+0000","last_unstale":"2023-11-27T15:21:02.783029+0000","last_undegraded":"2023-11-27T15:21:02.783029+0000","last_fullsized":"2023-11-27T15:21:02.783029+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:24:02.616850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.795136+0000","last_change":"2023-11-27T15:20:33.229267+0000","last_active":"2023-11-27T15:21:02.795136+0000","last_peered":"2023-11-27T15:21:02.795136+0000","last_clean":"2023-11-27T15:21:02.795136+0000","last_became_active":"2023-11-27T15:20:33.228746+0000","last_became_peered":"2023-11-27T15:20:33.228746+0000","last_unstale":"2023-11-27T15:21:02.795136+0000","last_undegraded":"2023-11-27T15:21:02.795136+0000","last_fullsized":"2023-11-27T15:21:02.795136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:55:36.391108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783571+0000","last_change":"2023-11-27T15:20:37.671893+0000","last_active":"2023-11-27T15:21:02.783571+0000","last_peered":"2023-11-27T15:21:02.783571+0000","last_clean":"2023-11-27T15:21:02.783571+0000","last_became_active":"2023-11-27T15:20:37.671033+0000","last_became_peered":"2023-11-27T15:20:37.671033+0000","last_unstale":"2023-11-27T15:21:02.783571+0000","last_undegraded":"2023-11-27T15:21:02.783571+0000","last_fullsized":"2023-11-27T15:21:02.783571+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:43:21.793592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782934+0000","last_change":"2023-11-27T15:20:32.650342+0000","last_active":"2023-11-27T15:21:02.782934+0000","last_peered":"2023-11-27T15:21:02.782934+0000","last_clean":"2023-11-27T15:21:02.782934+0000","last_became_active":"2023-11-27T15:20:32.650034+0000","last_became_peered":"2023-11-27T15:20:32.650034+0000","last_unstale":"2023-11-27T15:21:02.782934+0000","last_undegraded":"2023-11-27T15:21:02.782934+0000","last_fullsized":"2023-11-27T15:21:02.782934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:29:04.356130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932717+0000","last_change":"2023-11-27T15:20:33.226005+0000","last_active":"2023-11-27T15:21:02.932717+0000","last_peered":"2023-11-27T15:21:02.932717+0000","last_clean":"2023-11-27T15:21:02.932717+0000","last_became_active":"2023-11-27T15:20:33.225685+0000","last_became_peered":"2023-11-27T15:20:33.225685+0000","last_unstale":"2023-11-27T15:21:02.932717+0000","last_undegraded":"2023-11-27T15:21:02.932717+0000","last_fullsized":"2023-11-27T15:21:02.932717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:20:11.054422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779958+0000","last_change":"2023-11-27T15:20:37.673168+0000","last_active":"2023-11-27T15:21:02.779958+0000","last_peered":"2023-11-27T15:21:02.779958+0000","last_clean":"2023-11-27T15:21:02.779958+0000","last_became_active":"2023-11-27T15:20:37.672762+0000","last_became_peered":"2023-11-27T15:20:37.672762+0000","last_unstale":"2023-11-27T15:21:02.779958+0000","last_undegraded":"2023-11-27T15:21:02.779958+0000","last_fullsized":"2023-11-27T15:21:02.779958+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:17:17.977179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:04.806488+0000","last_change":"2023-11-27T15:20:32.656402+0000","last_active":"2023-11-27T15:21:04.806488+0000","last_peered":"2023-11-27T15:21:04.806488+0000","last_clean":"2023-11-27T15:21:04.806488+0000","last_became_active":"2023-11-27T15:20:32.656112+0000","last_became_peered":"2023-11-27T15:20:32.656112+0000","last_unstale":"2023-11-27T15:21:04.806488+0000","last_undegraded":"2023-11-27T15:21:04.806488+0000","last_fullsized":"2023-11-27T15:21:04.806488+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:10:24.829148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782376+0000","last_change":"2023-11-27T15:20:33.223073+0000","last_active":"2023-11-27T15:21:02.782376+0000","last_peered":"2023-11-27T15:21:02.782376+0000","last_clean":"2023-11-27T15:21:02.782376+0000","last_became_active":"2023-11-27T15:20:33.222655+0000","last_became_peered":"2023-11-27T15:20:33.222655+0000","last_unstale":"2023-11-27T15:21:02.782376+0000","last_undegraded":"2023-11-27T15:21:02.782376+0000","last_fullsized":"2023-11-27T15:21:02.782376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:48:43.125499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779818+0000","last_change":"2023-11-27T15:20:37.678722+0000","last_active":"2023-11-27T15:21:02.779818+0000","last_peered":"2023-11-27T15:21:02.779818+0000","last_clean":"2023-11-27T15:21:02.779818+0000","last_became_active":"2023-11-27T15:20:37.678242+0000","last_became_peered":"2023-11-27T15:20:37.678242+0000","last_unstale":"2023-11-27T15:21:02.779818+0000","last_undegraded":"2023-11-27T15:21:02.779818+0000","last_fullsized":"2023-11-27T15:21:02.779818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:36:10.661192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.800170+0000","last_change":"2023-11-27T15:20:32.646951+0000","last_active":"2023-11-27T15:21:03.800170+0000","last_peered":"2023-11-27T15:21:03.800170+0000","last_clean":"2023-11-27T15:21:03.800170+0000","last_became_active":"2023-11-27T15:20:32.646620+0000","last_became_peered":"2023-11-27T15:20:32.646620+0000","last_unstale":"2023-11-27T15:21:03.800170+0000","last_undegraded":"2023-11-27T15:21:03.800170+0000","last_fullsized":"2023-11-27T15:21:03.800170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:13:29.369691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932503+0000","last_change":"2023-11-27T15:20:33.226576+0000","last_active":"2023-11-27T15:21:02.932503+0000","last_peered":"2023-11-27T15:21:02.932503+0000","last_clean":"2023-11-27T15:21:02.932503+0000","last_became_active":"2023-11-27T15:20:33.226181+0000","last_became_peered":"2023-11-27T15:20:33.226181+0000","last_unstale":"2023-11-27T15:21:02.932503+0000","last_undegraded":"2023-11-27T15:21:02.932503+0000","last_fullsized":"2023-11-27T15:21:02.932503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:27:44.284870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932740+0000","last_change":"2023-11-27T15:20:37.681668+0000","last_active":"2023-11-27T15:21:02.932740+0000","last_peered":"2023-11-27T15:21:02.932740+0000","last_clean":"2023-11-27T15:21:02.932740+0000","last_became_active":"2023-11-27T15:20:37.681236+0000","last_became_peered":"2023-11-27T15:20:37.681236+0000","last_unstale":"2023-11-27T15:21:02.932740+0000","last_undegraded":"2023-11-27T15:21:02.932740+0000","last_fullsized":"2023-11-27T15:21:02.932740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:29:12.216944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.800787+0000","last_change":"2023-11-27T15:20:32.659421+0000","last_active":"2023-11-27T15:21:03.800787+0000","last_peered":"2023-11-27T15:21:03.800787+0000","last_clean":"2023-11-27T15:21:03.800787+0000","last_became_active":"2023-11-27T15:20:32.658882+0000","last_became_peered":"2023-11-27T15:20:32.658882+0000","last_unstale":"2023-11-27T15:21:03.800787+0000","last_undegraded":"2023-11-27T15:21:03.800787+0000","last_fullsized":"2023-11-27T15:21:03.800787+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:08:52.781635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783921+0000","last_change":"2023-11-27T15:20:33.227635+0000","last_active":"2023-11-27T15:21:02.783921+0000","last_peered":"2023-11-27T15:21:02.783921+0000","last_clean":"2023-11-27T15:21:02.783921+0000","last_became_active":"2023-11-27T15:20:33.225217+0000","last_became_peered":"2023-11-27T15:20:33.225217+0000","last_unstale":"2023-11-27T15:21:02.783921+0000","last_undegraded":"2023-11-27T15:21:02.783921+0000","last_fullsized":"2023-11-27T15:21:02.783921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:53:42.037169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785814+0000","last_change":"2023-11-27T15:20:37.679575+0000","last_active":"2023-11-27T15:21:02.785814+0000","last_peered":"2023-11-27T15:21:02.785814+0000","last_clean":"2023-11-27T15:21:02.785814+0000","last_became_active":"2023-11-27T15:20:37.679131+0000","last_became_peered":"2023-11-27T15:20:37.679131+0000","last_unstale":"2023-11-27T15:21:02.785814+0000","last_undegraded":"2023-11-27T15:21:02.785814+0000","last_fullsized":"2023-11-27T15:21:02.785814+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:43:21.129782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.931632+0000","last_change":"2023-11-27T15:20:32.657155+0000","last_active":"2023-11-27T15:21:02.931632+0000","last_peered":"2023-11-27T15:21:02.931632+0000","last_clean":"2023-11-27T15:21:02.931632+0000","last_became_active":"2023-11-27T15:20:32.656832+0000","last_became_peered":"2023-11-27T15:20:32.656832+0000","last_unstale":"2023-11-27T15:21:02.931632+0000","last_undegraded":"2023-11-27T15:21:02.931632+0000","last_fullsized":"2023-11-27T15:21:02.931632+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:03:07.478635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779387+0000","last_change":"2023-11-27T15:20:33.229215+0000","last_active":"2023-11-27T15:21:02.779387+0000","last_peered":"2023-11-27T15:21:02.779387+0000","last_clean":"2023-11-27T15:21:02.779387+0000","last_became_active":"2023-11-27T15:20:33.228616+0000","last_became_peered":"2023-11-27T15:20:33.228616+0000","last_unstale":"2023-11-27T15:21:02.779387+0000","last_undegraded":"2023-11-27T15:21:02.779387+0000","last_fullsized":"2023-11-27T15:21:02.779387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:30:10.600160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784114+0000","last_change":"2023-11-27T15:20:37.672582+0000","last_active":"2023-11-27T15:21:02.784114+0000","last_peered":"2023-11-27T15:21:02.784114+0000","last_clean":"2023-11-27T15:21:02.784114+0000","last_became_active":"2023-11-27T15:20:37.672193+0000","last_became_peered":"2023-11-27T15:20:37.672193+0000","last_unstale":"2023-11-27T15:21:02.784114+0000","last_undegraded":"2023-11-27T15:21:02.784114+0000","last_fullsized":"2023-11-27T15:21:02.784114+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:53:51.137569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:04.805924+0000","last_change":"2023-11-27T15:20:32.647894+0000","last_active":"2023-11-27T15:21:04.805924+0000","last_peered":"2023-11-27T15:21:04.805924+0000","last_clean":"2023-11-27T15:21:04.805924+0000","last_became_active":"2023-11-27T15:20:32.647597+0000","last_became_peered":"2023-11-27T15:20:32.647597+0000","last_unstale":"2023-11-27T15:21:04.805924+0000","last_undegraded":"2023-11-27T15:21:04.805924+0000","last_fullsized":"2023-11-27T15:21:04.805924+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:00:17.488397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.795490+0000","last_change":"2023-11-27T15:20:33.229382+0000","last_active":"2023-11-27T15:21:02.795490+0000","last_peered":"2023-11-27T15:21:02.795490+0000","last_clean":"2023-11-27T15:21:02.795490+0000","last_became_active":"2023-11-27T15:20:33.229031+0000","last_became_peered":"2023-11-27T15:20:33.229031+0000","last_unstale":"2023-11-27T15:21:02.795490+0000","last_undegraded":"2023-11-27T15:21:02.795490+0000","last_fullsized":"2023-11-27T15:21:02.795490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:43:48.637904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779387+0000","last_change":"2023-11-27T15:20:37.679131+0000","last_active":"2023-11-27T15:21:02.779387+0000","last_peered":"2023-11-27T15:21:02.779387+0000","last_clean":"2023-11-27T15:21:02.779387+0000","last_became_active":"2023-11-27T15:20:37.678290+0000","last_became_peered":"2023-11-27T15:20:37.678290+0000","last_unstale":"2023-11-27T15:21:02.779387+0000","last_undegraded":"2023-11-27T15:21:02.779387+0000","last_fullsized":"2023-11-27T15:21:02.779387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:55:28.895714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.798466+0000","last_change":"2023-11-27T15:21:02.797374+0000","last_active":"2023-11-27T15:21:02.798466+0000","last_peered":"2023-11-27T15:21:02.798466+0000","last_clean":"2023-11-27T15:21:02.798466+0000","last_became_active":"2023-11-27T15:21:02.796076+0000","last_became_peered":"2023-11-27T15:21:02.796076+0000","last_unstale":"2023-11-27T15:21:02.798466+0000","last_undegraded":"2023-11-27T15:21:02.798466+0000","last_fullsized":"2023-11-27T15:21:02.798466+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:07:58.898194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.935104+0000","last_change":"2023-11-27T15:21:02.935104+0000","last_active":"2023-11-27T15:21:02.935104+0000","last_peered":"2023-11-27T15:21:02.935104+0000","last_clean":"2023-11-27T15:21:02.935104+0000","last_became_active":"2023-11-27T15:21:02.934486+0000","last_became_peered":"2023-11-27T15:21:02.934486+0000","last_unstale":"2023-11-27T15:21:02.935104+0000","last_undegraded":"2023-11-27T15:21:02.935104+0000","last_fullsized":"2023-11-27T15:21:02.935104+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:06:11.768697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783910+0000","last_change":"2023-11-27T15:20:37.670389+0000","last_active":"2023-11-27T15:21:02.783910+0000","last_peered":"2023-11-27T15:21:02.783910+0000","last_clean":"2023-11-27T15:21:02.783910+0000","last_became_active":"2023-11-27T15:20:37.669512+0000","last_became_peered":"2023-11-27T15:20:37.669512+0000","last_unstale":"2023-11-27T15:21:02.783910+0000","last_undegraded":"2023-11-27T15:21:02.783910+0000","last_fullsized":"2023-11-27T15:21:02.783910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:47:38.447258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932618+0000","last_change":"2023-11-27T15:20:32.658287+0000","last_active":"2023-11-27T15:21:02.932618+0000","last_peered":"2023-11-27T15:21:02.932618+0000","last_clean":"2023-11-27T15:21:02.932618+0000","last_became_active":"2023-11-27T15:20:32.657925+0000","last_became_peered":"2023-11-27T15:20:32.657925+0000","last_unstale":"2023-11-27T15:21:02.932618+0000","last_undegraded":"2023-11-27T15:21:02.932618+0000","last_fullsized":"2023-11-27T15:21:02.932618+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:49:45.401491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.795421+0000","last_change":"2023-11-27T15:20:33.220240+0000","last_active":"2023-11-27T15:21:02.795421+0000","last_peered":"2023-11-27T15:21:02.795421+0000","last_clean":"2023-11-27T15:21:02.795421+0000","last_became_active":"2023-11-27T15:20:33.219962+0000","last_became_peered":"2023-11-27T15:20:33.219962+0000","last_unstale":"2023-11-27T15:21:02.795421+0000","last_undegraded":"2023-11-27T15:21:02.795421+0000","last_fullsized":"2023-11-27T15:21:02.795421+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:22:09.271036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779503+0000","last_change":"2023-11-27T15:20:37.672203+0000","last_active":"2023-11-27T15:21:02.779503+0000","last_peered":"2023-11-27T15:21:02.779503+0000","last_clean":"2023-11-27T15:21:02.779503+0000","last_became_active":"2023-11-27T15:20:37.671149+0000","last_became_peered":"2023-11-27T15:20:37.671149+0000","last_unstale":"2023-11-27T15:21:02.779503+0000","last_undegraded":"2023-11-27T15:21:02.779503+0000","last_fullsized":"2023-11-27T15:21:02.779503+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:14:24.426741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'23","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:06.276319+0000","last_change":"2023-11-27T15:20:32.646637+0000","last_active":"2023-11-27T15:21:06.276319+0000","last_peered":"2023-11-27T15:21:06.276319+0000","last_clean":"2023-11-27T15:21:06.276319+0000","last_became_active":"2023-11-27T15:20:32.646292+0000","last_became_peered":"2023-11-27T15:20:32.646292+0000","last_unstale":"2023-11-27T15:21:06.276319+0000","last_undegraded":"2023-11-27T15:21:06.276319+0000","last_fullsized":"2023-11-27T15:21:06.276319+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:01:39.494636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38515,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":24,"num_write":24,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782815+0000","last_change":"2023-11-27T15:20:33.217078+0000","last_active":"2023-11-27T15:21:02.782815+0000","last_peered":"2023-11-27T15:21:02.782815+0000","last_clean":"2023-11-27T15:21:02.782815+0000","last_became_active":"2023-11-27T15:20:33.216787+0000","last_became_peered":"2023-11-27T15:20:33.216787+0000","last_unstale":"2023-11-27T15:21:02.782815+0000","last_undegraded":"2023-11-27T15:21:02.782815+0000","last_fullsized":"2023-11-27T15:21:02.782815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:33:46.415082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.933136+0000","last_change":"2023-11-27T15:20:37.671800+0000","last_active":"2023-11-27T15:21:02.933136+0000","last_peered":"2023-11-27T15:21:02.933136+0000","last_clean":"2023-11-27T15:21:02.933136+0000","last_became_active":"2023-11-27T15:20:37.671374+0000","last_became_peered":"2023-11-27T15:20:37.671374+0000","last_unstale":"2023-11-27T15:21:02.933136+0000","last_undegraded":"2023-11-27T15:21:02.933136+0000","last_fullsized":"2023-11-27T15:21:02.933136+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:04:07.644987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.801478+0000","last_change":"2023-11-27T15:20:32.659508+0000","last_active":"2023-11-27T15:21:03.801478+0000","last_peered":"2023-11-27T15:21:03.801478+0000","last_clean":"2023-11-27T15:21:03.801478+0000","last_became_active":"2023-11-27T15:20:32.659226+0000","last_became_peered":"2023-11-27T15:20:32.659226+0000","last_unstale":"2023-11-27T15:21:03.801478+0000","last_undegraded":"2023-11-27T15:21:03.801478+0000","last_fullsized":"2023-11-27T15:21:03.801478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:42:20.386331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786639+0000","last_change":"2023-11-27T15:20:33.230012+0000","last_active":"2023-11-27T15:21:02.786639+0000","last_peered":"2023-11-27T15:21:02.786639+0000","last_clean":"2023-11-27T15:21:02.786639+0000","last_became_active":"2023-11-27T15:20:33.225193+0000","last_became_peered":"2023-11-27T15:20:33.225193+0000","last_unstale":"2023-11-27T15:21:02.786639+0000","last_undegraded":"2023-11-27T15:21:02.786639+0000","last_fullsized":"2023-11-27T15:21:02.786639+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:31:18.056712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932755+0000","last_change":"2023-11-27T15:20:37.672430+0000","last_active":"2023-11-27T15:21:02.932755+0000","last_peered":"2023-11-27T15:21:02.932755+0000","last_clean":"2023-11-27T15:21:02.932755+0000","last_became_active":"2023-11-27T15:20:37.671982+0000","last_became_peered":"2023-11-27T15:20:37.671982+0000","last_unstale":"2023-11-27T15:21:02.932755+0000","last_undegraded":"2023-11-27T15:21:02.932755+0000","last_fullsized":"2023-11-27T15:21:02.932755+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:53:27.660441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932051+0000","last_change":"2023-11-27T15:20:32.655015+0000","last_active":"2023-11-27T15:21:02.932051+0000","last_peered":"2023-11-27T15:21:02.932051+0000","last_clean":"2023-11-27T15:21:02.932051+0000","last_became_active":"2023-11-27T15:20:32.654662+0000","last_became_peered":"2023-11-27T15:20:32.654662+0000","last_unstale":"2023-11-27T15:21:02.932051+0000","last_undegraded":"2023-11-27T15:21:02.932051+0000","last_fullsized":"2023-11-27T15:21:02.932051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:10:34.935987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780064+0000","last_change":"2023-11-27T15:20:33.233980+0000","last_active":"2023-11-27T15:21:02.780064+0000","last_peered":"2023-11-27T15:21:02.780064+0000","last_clean":"2023-11-27T15:21:02.780064+0000","last_became_active":"2023-11-27T15:20:33.233581+0000","last_became_peered":"2023-11-27T15:20:33.233581+0000","last_unstale":"2023-11-27T15:21:02.780064+0000","last_undegraded":"2023-11-27T15:21:02.780064+0000","last_fullsized":"2023-11-27T15:21:02.780064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:21:09.061874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779860+0000","last_change":"2023-11-27T15:20:37.682359+0000","last_active":"2023-11-27T15:21:02.779860+0000","last_peered":"2023-11-27T15:21:02.779860+0000","last_clean":"2023-11-27T15:21:02.779860+0000","last_became_active":"2023-11-27T15:20:37.681888+0000","last_became_peered":"2023-11-27T15:20:37.681888+0000","last_unstale":"2023-11-27T15:21:02.779860+0000","last_undegraded":"2023-11-27T15:21:02.779860+0000","last_fullsized":"2023-11-27T15:21:02.779860+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:22:58.040147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.799258+0000","last_change":"2023-11-27T15:20:32.653388+0000","last_active":"2023-11-27T15:21:03.799258+0000","last_peered":"2023-11-27T15:21:03.799258+0000","last_clean":"2023-11-27T15:21:03.799258+0000","last_became_active":"2023-11-27T15:20:32.653007+0000","last_became_peered":"2023-11-27T15:20:32.653007+0000","last_unstale":"2023-11-27T15:21:03.799258+0000","last_undegraded":"2023-11-27T15:21:03.799258+0000","last_fullsized":"2023-11-27T15:21:03.799258+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:02:11.654541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779871+0000","last_change":"2023-11-27T15:20:33.216539+0000","last_active":"2023-11-27T15:21:02.779871+0000","last_peered":"2023-11-27T15:21:02.779871+0000","last_clean":"2023-11-27T15:21:02.779871+0000","last_became_active":"2023-11-27T15:20:33.216166+0000","last_became_peered":"2023-11-27T15:20:33.216166+0000","last_unstale":"2023-11-27T15:21:02.779871+0000","last_undegraded":"2023-11-27T15:21:02.779871+0000","last_fullsized":"2023-11-27T15:21:02.779871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:12:37.881354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786643+0000","last_change":"2023-11-27T15:20:37.675735+0000","last_active":"2023-11-27T15:21:02.786643+0000","last_peered":"2023-11-27T15:21:02.786643+0000","last_clean":"2023-11-27T15:21:02.786643+0000","last_became_active":"2023-11-27T15:20:37.669685+0000","last_became_peered":"2023-11-27T15:20:37.669685+0000","last_unstale":"2023-11-27T15:21:02.786643+0000","last_undegraded":"2023-11-27T15:21:02.786643+0000","last_fullsized":"2023-11-27T15:21:02.786643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:47:13.695674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'10","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782372+0000","last_change":"2023-11-27T15:20:32.657567+0000","last_active":"2023-11-27T15:21:02.782372+0000","last_peered":"2023-11-27T15:21:02.782372+0000","last_clean":"2023-11-27T15:21:02.782372+0000","last_became_active":"2023-11-27T15:20:32.657251+0000","last_became_peered":"2023-11-27T15:20:32.657251+0000","last_unstale":"2023-11-27T15:21:02.782372+0000","last_undegraded":"2023-11-27T15:21:02.782372+0000","last_fullsized":"2023-11-27T15:21:02.782372+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:23:00.166520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932029+0000","last_change":"2023-11-27T15:20:33.225019+0000","last_active":"2023-11-27T15:21:02.932029+0000","last_peered":"2023-11-27T15:21:02.932029+0000","last_clean":"2023-11-27T15:21:02.932029+0000","last_became_active":"2023-11-27T15:20:33.224622+0000","last_became_peered":"2023-11-27T15:20:33.224622+0000","last_unstale":"2023-11-27T15:21:02.932029+0000","last_undegraded":"2023-11-27T15:21:02.932029+0000","last_fullsized":"2023-11-27T15:21:02.932029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:44:13.508838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.931675+0000","last_change":"2023-11-27T15:20:37.680808+0000","last_active":"2023-11-27T15:21:02.931675+0000","last_peered":"2023-11-27T15:21:02.931675+0000","last_clean":"2023-11-27T15:21:02.931675+0000","last_became_active":"2023-11-27T15:20:37.680348+0000","last_became_peered":"2023-11-27T15:20:37.680348+0000","last_unstale":"2023-11-27T15:21:02.931675+0000","last_undegraded":"2023-11-27T15:21:02.931675+0000","last_fullsized":"2023-11-27T15:21:02.931675+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:48:00.800370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"83'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783285+0000","last_change":"2023-11-27T15:20:32.651093+0000","last_active":"2023-11-27T15:21:02.783285+0000","last_peered":"2023-11-27T15:21:02.783285+0000","last_clean":"2023-11-27T15:21:02.783285+0000","last_became_active":"2023-11-27T15:20:32.650657+0000","last_became_peered":"2023-11-27T15:20:32.650657+0000","last_unstale":"2023-11-27T15:21:02.783285+0000","last_undegraded":"2023-11-27T15:21:02.783285+0000","last_fullsized":"2023-11-27T15:21:02.783285+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:28:31.347970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.933913+0000","last_change":"2023-11-27T15:20:33.224806+0000","last_active":"2023-11-27T15:21:02.933913+0000","last_peered":"2023-11-27T15:21:02.933913+0000","last_clean":"2023-11-27T15:21:02.933913+0000","last_became_active":"2023-11-27T15:20:33.224362+0000","last_became_peered":"2023-11-27T15:20:33.224362+0000","last_unstale":"2023-11-27T15:21:02.933913+0000","last_undegraded":"2023-11-27T15:21:02.933913+0000","last_fullsized":"2023-11-27T15:21:02.933913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:34:11.704495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782260+0000","last_change":"2023-11-27T15:20:37.849274+0000","last_active":"2023-11-27T15:21:02.782260+0000","last_peered":"2023-11-27T15:21:02.782260+0000","last_clean":"2023-11-27T15:21:02.782260+0000","last_became_active":"2023-11-27T15:20:37.848839+0000","last_became_peered":"2023-11-27T15:20:37.848839+0000","last_unstale":"2023-11-27T15:21:02.782260+0000","last_undegraded":"2023-11-27T15:21:02.782260+0000","last_fullsized":"2023-11-27T15:21:02.782260+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:46:25.321810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779849+0000","last_change":"2023-11-27T15:20:32.660835+0000","last_active":"2023-11-27T15:21:02.779849+0000","last_peered":"2023-11-27T15:21:02.779849+0000","last_clean":"2023-11-27T15:21:02.779849+0000","last_became_active":"2023-11-27T15:20:32.660431+0000","last_became_peered":"2023-11-27T15:20:32.660431+0000","last_unstale":"2023-11-27T15:21:02.779849+0000","last_undegraded":"2023-11-27T15:21:02.779849+0000","last_fullsized":"2023-11-27T15:21:02.779849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:52:11.617105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932764+0000","last_change":"2023-11-27T15:20:33.245772+0000","last_active":"2023-11-27T15:21:02.932764+0000","last_peered":"2023-11-27T15:21:02.932764+0000","last_clean":"2023-11-27T15:21:02.932764+0000","last_became_active":"2023-11-27T15:20:33.245468+0000","last_became_peered":"2023-11-27T15:20:33.245468+0000","last_unstale":"2023-11-27T15:21:02.932764+0000","last_undegraded":"2023-11-27T15:21:02.932764+0000","last_fullsized":"2023-11-27T15:21:02.932764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:24:42.963520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784658+0000","last_change":"2023-11-27T15:20:37.678972+0000","last_active":"2023-11-27T15:21:02.784658+0000","last_peered":"2023-11-27T15:21:02.784658+0000","last_clean":"2023-11-27T15:21:02.784658+0000","last_became_active":"2023-11-27T15:20:37.678304+0000","last_became_peered":"2023-11-27T15:20:37.678304+0000","last_unstale":"2023-11-27T15:21:02.784658+0000","last_undegraded":"2023-11-27T15:21:02.784658+0000","last_fullsized":"2023-11-27T15:21:02.784658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:32:12.064148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780000+0000","last_change":"2023-11-27T15:20:32.661739+0000","last_active":"2023-11-27T15:21:02.780000+0000","last_peered":"2023-11-27T15:21:02.780000+0000","last_clean":"2023-11-27T15:21:02.780000+0000","last_became_active":"2023-11-27T15:20:32.661401+0000","last_became_peered":"2023-11-27T15:20:32.661401+0000","last_unstale":"2023-11-27T15:21:02.780000+0000","last_undegraded":"2023-11-27T15:21:02.780000+0000","last_fullsized":"2023-11-27T15:21:02.780000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:47:15.469916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932830+0000","last_change":"2023-11-27T15:20:33.235595+0000","last_active":"2023-11-27T15:21:02.932830+0000","last_peered":"2023-11-27T15:21:02.932830+0000","last_clean":"2023-11-27T15:21:02.932830+0000","last_became_active":"2023-11-27T15:20:33.235277+0000","last_became_peered":"2023-11-27T15:20:33.235277+0000","last_unstale":"2023-11-27T15:21:02.932830+0000","last_undegraded":"2023-11-27T15:21:02.932830+0000","last_fullsized":"2023-11-27T15:21:02.932830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:23:31.896222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.934194+0000","last_change":"2023-11-27T15:21:02.934194+0000","last_active":"2023-11-27T15:21:02.934194+0000","last_peered":"2023-11-27T15:21:02.934194+0000","last_clean":"2023-11-27T15:21:02.934194+0000","last_became_active":"2023-11-27T15:21:02.933555+0000","last_became_peered":"2023-11-27T15:21:02.933555+0000","last_unstale":"2023-11-27T15:21:02.934194+0000","last_undegraded":"2023-11-27T15:21:02.934194+0000","last_fullsized":"2023-11-27T15:21:02.934194+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:49:59.949602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785630+0000","last_change":"2023-11-27T15:21:02.784351+0000","last_active":"2023-11-27T15:21:02.785630+0000","last_peered":"2023-11-27T15:21:02.785630+0000","last_clean":"2023-11-27T15:21:02.785630+0000","last_became_active":"2023-11-27T15:21:02.783996+0000","last_became_peered":"2023-11-27T15:21:02.783996+0000","last_unstale":"2023-11-27T15:21:02.785630+0000","last_undegraded":"2023-11-27T15:21:02.785630+0000","last_fullsized":"2023-11-27T15:21:02.785630+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:30:13.050830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:04.807056+0000","last_change":"2023-11-27T15:20:32.659823+0000","last_active":"2023-11-27T15:21:04.807056+0000","last_peered":"2023-11-27T15:21:04.807056+0000","last_clean":"2023-11-27T15:21:04.807056+0000","last_became_active":"2023-11-27T15:20:32.659472+0000","last_became_peered":"2023-11-27T15:20:32.659472+0000","last_unstale":"2023-11-27T15:21:04.807056+0000","last_undegraded":"2023-11-27T15:21:04.807056+0000","last_fullsized":"2023-11-27T15:21:04.807056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:19:47.161135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780173+0000","last_change":"2023-11-27T15:20:37.671494+0000","last_active":"2023-11-27T15:21:02.780173+0000","last_peered":"2023-11-27T15:21:02.780173+0000","last_clean":"2023-11-27T15:21:02.780173+0000","last_became_active":"2023-11-27T15:20:37.671010+0000","last_became_peered":"2023-11-27T15:20:37.671010+0000","last_unstale":"2023-11-27T15:21:02.780173+0000","last_undegraded":"2023-11-27T15:21:02.780173+0000","last_fullsized":"2023-11-27T15:21:02.780173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:09:00.036788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784810+0000","last_change":"2023-11-27T15:20:33.241669+0000","last_active":"2023-11-27T15:21:02.784810+0000","last_peered":"2023-11-27T15:21:02.784810+0000","last_clean":"2023-11-27T15:21:02.784810+0000","last_became_active":"2023-11-27T15:20:33.241373+0000","last_became_peered":"2023-11-27T15:20:33.241373+0000","last_unstale":"2023-11-27T15:21:02.784810+0000","last_undegraded":"2023-11-27T15:21:02.784810+0000","last_fullsized":"2023-11-27T15:21:02.784810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:21:19.709967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783900+0000","last_change":"2023-11-27T15:20:32.647596+0000","last_active":"2023-11-27T15:21:02.783900+0000","last_peered":"2023-11-27T15:21:02.783900+0000","last_clean":"2023-11-27T15:21:02.783900+0000","last_became_active":"2023-11-27T15:20:32.647228+0000","last_became_peered":"2023-11-27T15:20:32.647228+0000","last_unstale":"2023-11-27T15:21:02.783900+0000","last_undegraded":"2023-11-27T15:21:02.783900+0000","last_fullsized":"2023-11-27T15:21:02.783900+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:40:49.354792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779663+0000","last_change":"2023-11-27T15:19:30.114855+0000","last_active":"2023-11-27T15:21:02.779663+0000","last_peered":"2023-11-27T15:21:02.779663+0000","last_clean":"2023-11-27T15:21:02.779663+0000","last_became_active":"2023-11-27T15:19:30.114419+0000","last_became_peered":"2023-11-27T15:19:30.114419+0000","last_unstale":"2023-11-27T15:21:02.779663+0000","last_undegraded":"2023-11-27T15:21:02.779663+0000","last_fullsized":"2023-11-27T15:21:02.779663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:20.216695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:20.216695+0000","last_clean_scrub_stamp":"2023-11-27T15:16:20.216695+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:14:23.715586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932257+0000","last_change":"2023-11-27T15:20:37.850539+0000","last_active":"2023-11-27T15:21:02.932257+0000","last_peered":"2023-11-27T15:21:02.932257+0000","last_clean":"2023-11-27T15:21:02.932257+0000","last_became_active":"2023-11-27T15:20:37.850044+0000","last_became_peered":"2023-11-27T15:20:37.850044+0000","last_unstale":"2023-11-27T15:21:02.932257+0000","last_undegraded":"2023-11-27T15:21:02.932257+0000","last_fullsized":"2023-11-27T15:21:02.932257+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:29:43.906707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783098+0000","last_change":"2023-11-27T15:20:33.211319+0000","last_active":"2023-11-27T15:21:02.783098+0000","last_peered":"2023-11-27T15:21:02.783098+0000","last_clean":"2023-11-27T15:21:02.783098+0000","last_became_active":"2023-11-27T15:20:33.211004+0000","last_became_peered":"2023-11-27T15:20:33.211004+0000","last_unstale":"2023-11-27T15:21:02.783098+0000","last_undegraded":"2023-11-27T15:21:02.783098+0000","last_fullsized":"2023-11-27T15:21:02.783098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:57:04.622964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.799863+0000","last_change":"2023-11-27T15:20:32.655003+0000","last_active":"2023-11-27T15:21:03.799863+0000","last_peered":"2023-11-27T15:21:03.799863+0000","last_clean":"2023-11-27T15:21:03.799863+0000","last_became_active":"2023-11-27T15:20:32.654517+0000","last_became_peered":"2023-11-27T15:20:32.654517+0000","last_unstale":"2023-11-27T15:21:03.799863+0000","last_undegraded":"2023-11-27T15:21:03.799863+0000","last_fullsized":"2023-11-27T15:21:03.799863+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:12:26.090416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932725+0000","last_change":"2023-11-27T15:20:37.673183+0000","last_active":"2023-11-27T15:21:02.932725+0000","last_peered":"2023-11-27T15:21:02.932725+0000","last_clean":"2023-11-27T15:21:02.932725+0000","last_became_active":"2023-11-27T15:20:37.672838+0000","last_became_peered":"2023-11-27T15:20:37.672838+0000","last_unstale":"2023-11-27T15:21:02.932725+0000","last_undegraded":"2023-11-27T15:21:02.932725+0000","last_fullsized":"2023-11-27T15:21:02.932725+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:47:56.458647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782687+0000","last_change":"2023-11-27T15:20:32.659540+0000","last_active":"2023-11-27T15:21:02.782687+0000","last_peered":"2023-11-27T15:21:02.782687+0000","last_clean":"2023-11-27T15:21:02.782687+0000","last_became_active":"2023-11-27T15:20:32.659211+0000","last_became_peered":"2023-11-27T15:20:32.659211+0000","last_unstale":"2023-11-27T15:21:02.782687+0000","last_undegraded":"2023-11-27T15:21:02.782687+0000","last_fullsized":"2023-11-27T15:21:02.782687+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:02:50.140093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784853+0000","last_change":"2023-11-27T15:20:33.216405+0000","last_active":"2023-11-27T15:21:02.784853+0000","last_peered":"2023-11-27T15:21:02.784853+0000","last_clean":"2023-11-27T15:21:02.784853+0000","last_became_active":"2023-11-27T15:20:33.216072+0000","last_became_peered":"2023-11-27T15:20:33.216072+0000","last_unstale":"2023-11-27T15:21:02.784853+0000","last_undegraded":"2023-11-27T15:21:02.784853+0000","last_fullsized":"2023-11-27T15:21:02.784853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:09:28.211261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785517+0000","last_change":"2023-11-27T15:20:37.849753+0000","last_active":"2023-11-27T15:21:02.785517+0000","last_peered":"2023-11-27T15:21:02.785517+0000","last_clean":"2023-11-27T15:21:02.785517+0000","last_became_active":"2023-11-27T15:20:37.848576+0000","last_became_peered":"2023-11-27T15:20:37.848576+0000","last_unstale":"2023-11-27T15:21:02.785517+0000","last_undegraded":"2023-11-27T15:21:02.785517+0000","last_fullsized":"2023-11-27T15:21:02.785517+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:59:24.696190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.802128+0000","last_change":"2023-11-27T15:20:32.652739+0000","last_active":"2023-11-27T15:21:03.802128+0000","last_peered":"2023-11-27T15:21:03.802128+0000","last_clean":"2023-11-27T15:21:03.802128+0000","last_became_active":"2023-11-27T15:20:32.652314+0000","last_became_peered":"2023-11-27T15:20:32.652314+0000","last_unstale":"2023-11-27T15:21:03.802128+0000","last_undegraded":"2023-11-27T15:21:03.802128+0000","last_fullsized":"2023-11-27T15:21:03.802128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:44:25.188063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.787449+0000","last_change":"2023-11-27T15:20:33.240552+0000","last_active":"2023-11-27T15:21:02.787449+0000","last_peered":"2023-11-27T15:21:02.787449+0000","last_clean":"2023-11-27T15:21:02.787449+0000","last_became_active":"2023-11-27T15:20:33.239885+0000","last_became_peered":"2023-11-27T15:20:33.239885+0000","last_unstale":"2023-11-27T15:21:02.787449+0000","last_undegraded":"2023-11-27T15:21:02.787449+0000","last_fullsized":"2023-11-27T15:21:02.787449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:36:51.366222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779611+0000","last_change":"2023-11-27T15:20:37.669895+0000","last_active":"2023-11-27T15:21:02.779611+0000","last_peered":"2023-11-27T15:21:02.779611+0000","last_clean":"2023-11-27T15:21:02.779611+0000","last_became_active":"2023-11-27T15:20:37.669489+0000","last_became_peered":"2023-11-27T15:20:37.669489+0000","last_unstale":"2023-11-27T15:21:02.779611+0000","last_undegraded":"2023-11-27T15:21:02.779611+0000","last_fullsized":"2023-11-27T15:21:02.779611+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:18:44.798721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784573+0000","last_change":"2023-11-27T15:20:32.656773+0000","last_active":"2023-11-27T15:21:02.784573+0000","last_peered":"2023-11-27T15:21:02.784573+0000","last_clean":"2023-11-27T15:21:02.784573+0000","last_became_active":"2023-11-27T15:20:32.656451+0000","last_became_peered":"2023-11-27T15:20:32.656451+0000","last_unstale":"2023-11-27T15:21:02.784573+0000","last_undegraded":"2023-11-27T15:21:02.784573+0000","last_fullsized":"2023-11-27T15:21:02.784573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:34:42.007222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786148+0000","last_change":"2023-11-27T15:20:33.231685+0000","last_active":"2023-11-27T15:21:02.786148+0000","last_peered":"2023-11-27T15:21:02.786148+0000","last_clean":"2023-11-27T15:21:02.786148+0000","last_became_active":"2023-11-27T15:20:33.228265+0000","last_became_peered":"2023-11-27T15:20:33.228265+0000","last_unstale":"2023-11-27T15:21:02.786148+0000","last_undegraded":"2023-11-27T15:21:02.786148+0000","last_fullsized":"2023-11-27T15:21:02.786148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:18:19.094289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.935830+0000","last_change":"2023-11-27T15:21:02.935830+0000","last_active":"2023-11-27T15:21:02.935830+0000","last_peered":"2023-11-27T15:21:02.935830+0000","last_clean":"2023-11-27T15:21:02.935830+0000","last_became_active":"2023-11-27T15:21:02.935412+0000","last_became_peered":"2023-11-27T15:21:02.935412+0000","last_unstale":"2023-11-27T15:21:02.935830+0000","last_undegraded":"2023-11-27T15:21:02.935830+0000","last_fullsized":"2023-11-27T15:21:02.935830+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:26:59.988473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.794970+0000","last_change":"2023-11-27T15:20:32.660424+0000","last_active":"2023-11-27T15:21:02.794970+0000","last_peered":"2023-11-27T15:21:02.794970+0000","last_clean":"2023-11-27T15:21:02.794970+0000","last_became_active":"2023-11-27T15:20:32.660112+0000","last_became_peered":"2023-11-27T15:20:32.660112+0000","last_unstale":"2023-11-27T15:21:02.794970+0000","last_undegraded":"2023-11-27T15:21:02.794970+0000","last_fullsized":"2023-11-27T15:21:02.794970+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:59:35.405721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782221+0000","last_change":"2023-11-27T15:20:33.227677+0000","last_active":"2023-11-27T15:21:02.782221+0000","last_peered":"2023-11-27T15:21:02.782221+0000","last_clean":"2023-11-27T15:21:02.782221+0000","last_became_active":"2023-11-27T15:20:33.227334+0000","last_became_peered":"2023-11-27T15:20:33.227334+0000","last_unstale":"2023-11-27T15:21:02.782221+0000","last_undegraded":"2023-11-27T15:21:02.782221+0000","last_fullsized":"2023-11-27T15:21:02.782221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:45:49.719897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.935341+0000","last_change":"2023-11-27T15:21:02.934516+0000","last_active":"2023-11-27T15:21:02.935341+0000","last_peered":"2023-11-27T15:21:02.935341+0000","last_clean":"2023-11-27T15:21:02.935341+0000","last_became_active":"2023-11-27T15:21:02.934012+0000","last_became_peered":"2023-11-27T15:21:02.934012+0000","last_unstale":"2023-11-27T15:21:02.935341+0000","last_undegraded":"2023-11-27T15:21:02.935341+0000","last_fullsized":"2023-11-27T15:21:02.935341+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:07:11.301253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784148+0000","last_change":"2023-11-27T15:20:32.657935+0000","last_active":"2023-11-27T15:21:02.784148+0000","last_peered":"2023-11-27T15:21:02.784148+0000","last_clean":"2023-11-27T15:21:02.784148+0000","last_became_active":"2023-11-27T15:20:32.657579+0000","last_became_peered":"2023-11-27T15:20:32.657579+0000","last_unstale":"2023-11-27T15:21:02.784148+0000","last_undegraded":"2023-11-27T15:21:02.784148+0000","last_fullsized":"2023-11-27T15:21:02.784148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:30:23.087862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932840+0000","last_change":"2023-11-27T15:20:33.225817+0000","last_active":"2023-11-27T15:21:02.932840+0000","last_peered":"2023-11-27T15:21:02.932840+0000","last_clean":"2023-11-27T15:21:02.932840+0000","last_became_active":"2023-11-27T15:20:33.225438+0000","last_became_peered":"2023-11-27T15:20:33.225438+0000","last_unstale":"2023-11-27T15:21:02.932840+0000","last_undegraded":"2023-11-27T15:21:02.932840+0000","last_fullsized":"2023-11-27T15:21:02.932840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:04:27.668653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.934304+0000","last_change":"2023-11-27T15:21:02.934304+0000","last_active":"2023-11-27T15:21:02.934304+0000","last_peered":"2023-11-27T15:21:02.934304+0000","last_clean":"2023-11-27T15:21:02.934304+0000","last_became_active":"2023-11-27T15:21:02.933709+0000","last_became_peered":"2023-11-27T15:21:02.933709+0000","last_unstale":"2023-11-27T15:21:02.934304+0000","last_undegraded":"2023-11-27T15:21:02.934304+0000","last_fullsized":"2023-11-27T15:21:02.934304+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:43:38.974368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780192+0000","last_change":"2023-11-27T15:20:32.659484+0000","last_active":"2023-11-27T15:21:02.780192+0000","last_peered":"2023-11-27T15:21:02.780192+0000","last_clean":"2023-11-27T15:21:02.780192+0000","last_became_active":"2023-11-27T15:20:32.659134+0000","last_became_peered":"2023-11-27T15:20:32.659134+0000","last_unstale":"2023-11-27T15:21:02.780192+0000","last_undegraded":"2023-11-27T15:21:02.780192+0000","last_fullsized":"2023-11-27T15:21:02.780192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:04:03.053379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784616+0000","last_change":"2023-11-27T15:20:33.235755+0000","last_active":"2023-11-27T15:21:02.784616+0000","last_peered":"2023-11-27T15:21:02.784616+0000","last_clean":"2023-11-27T15:21:02.784616+0000","last_became_active":"2023-11-27T15:20:33.234576+0000","last_became_peered":"2023-11-27T15:20:33.234576+0000","last_unstale":"2023-11-27T15:21:02.784616+0000","last_undegraded":"2023-11-27T15:21:02.784616+0000","last_fullsized":"2023-11-27T15:21:02.784616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:25:25.181797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784302+0000","last_change":"2023-11-27T15:20:37.849121+0000","last_active":"2023-11-27T15:21:02.784302+0000","last_peered":"2023-11-27T15:21:02.784302+0000","last_clean":"2023-11-27T15:21:02.784302+0000","last_became_active":"2023-11-27T15:20:37.848683+0000","last_became_peered":"2023-11-27T15:20:37.848683+0000","last_unstale":"2023-11-27T15:21:02.784302+0000","last_undegraded":"2023-11-27T15:21:02.784302+0000","last_fullsized":"2023-11-27T15:21:02.784302+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:05:39.554898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782466+0000","last_change":"2023-11-27T15:20:32.655896+0000","last_active":"2023-11-27T15:21:02.782466+0000","last_peered":"2023-11-27T15:21:02.782466+0000","last_clean":"2023-11-27T15:21:02.782466+0000","last_became_active":"2023-11-27T15:20:32.655623+0000","last_became_peered":"2023-11-27T15:20:32.655623+0000","last_unstale":"2023-11-27T15:21:02.782466+0000","last_undegraded":"2023-11-27T15:21:02.782466+0000","last_fullsized":"2023-11-27T15:21:02.782466+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:01:57.075773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779947+0000","last_change":"2023-11-27T15:20:33.227318+0000","last_active":"2023-11-27T15:21:02.779947+0000","last_peered":"2023-11-27T15:21:02.779947+0000","last_clean":"2023-11-27T15:21:02.779947+0000","last_became_active":"2023-11-27T15:20:33.226944+0000","last_became_peered":"2023-11-27T15:20:33.226944+0000","last_unstale":"2023-11-27T15:21:02.779947+0000","last_undegraded":"2023-11-27T15:21:02.779947+0000","last_fullsized":"2023-11-27T15:21:02.779947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:47:26.971182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.794990+0000","last_change":"2023-11-27T15:20:37.669098+0000","last_active":"2023-11-27T15:21:02.794990+0000","last_peered":"2023-11-27T15:21:02.794990+0000","last_clean":"2023-11-27T15:21:02.794990+0000","last_became_active":"2023-11-27T15:20:37.668693+0000","last_became_peered":"2023-11-27T15:20:37.668693+0000","last_unstale":"2023-11-27T15:21:02.794990+0000","last_undegraded":"2023-11-27T15:21:02.794990+0000","last_fullsized":"2023-11-27T15:21:02.794990+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:38:51.699084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.800964+0000","last_change":"2023-11-27T15:20:32.660285+0000","last_active":"2023-11-27T15:21:03.800964+0000","last_peered":"2023-11-27T15:21:03.800964+0000","last_clean":"2023-11-27T15:21:03.800964+0000","last_became_active":"2023-11-27T15:20:32.659982+0000","last_became_peered":"2023-11-27T15:20:32.659982+0000","last_unstale":"2023-11-27T15:21:03.800964+0000","last_undegraded":"2023-11-27T15:21:03.800964+0000","last_fullsized":"2023-11-27T15:21:03.800964+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:42:36.081550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780414+0000","last_change":"2023-11-27T15:20:33.226629+0000","last_active":"2023-11-27T15:21:02.780414+0000","last_peered":"2023-11-27T15:21:02.780414+0000","last_clean":"2023-11-27T15:21:02.780414+0000","last_became_active":"2023-11-27T15:20:33.226183+0000","last_became_peered":"2023-11-27T15:20:33.226183+0000","last_unstale":"2023-11-27T15:21:02.780414+0000","last_undegraded":"2023-11-27T15:21:02.780414+0000","last_fullsized":"2023-11-27T15:21:02.780414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:33:24.895609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785284+0000","last_change":"2023-11-27T15:20:37.680987+0000","last_active":"2023-11-27T15:21:02.785284+0000","last_peered":"2023-11-27T15:21:02.785284+0000","last_clean":"2023-11-27T15:21:02.785284+0000","last_became_active":"2023-11-27T15:20:37.680219+0000","last_became_peered":"2023-11-27T15:20:37.680219+0000","last_unstale":"2023-11-27T15:21:02.785284+0000","last_undegraded":"2023-11-27T15:21:02.785284+0000","last_fullsized":"2023-11-27T15:21:02.785284+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:27:01.990771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932894+0000","last_change":"2023-11-27T15:20:32.655035+0000","last_active":"2023-11-27T15:21:02.932894+0000","last_peered":"2023-11-27T15:21:02.932894+0000","last_clean":"2023-11-27T15:21:02.932894+0000","last_became_active":"2023-11-27T15:20:32.654687+0000","last_became_peered":"2023-11-27T15:20:32.654687+0000","last_unstale":"2023-11-27T15:21:02.932894+0000","last_undegraded":"2023-11-27T15:21:02.932894+0000","last_fullsized":"2023-11-27T15:21:02.932894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:54:01.028543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780460+0000","last_change":"2023-11-27T15:20:33.245908+0000","last_active":"2023-11-27T15:21:02.780460+0000","last_peered":"2023-11-27T15:21:02.780460+0000","last_clean":"2023-11-27T15:21:02.780460+0000","last_became_active":"2023-11-27T15:20:33.245576+0000","last_became_peered":"2023-11-27T15:20:33.245576+0000","last_unstale":"2023-11-27T15:21:02.780460+0000","last_undegraded":"2023-11-27T15:21:02.780460+0000","last_fullsized":"2023-11-27T15:21:02.780460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:41:20.860779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.931498+0000","last_change":"2023-11-27T15:20:37.674253+0000","last_active":"2023-11-27T15:21:02.931498+0000","last_peered":"2023-11-27T15:21:02.931498+0000","last_clean":"2023-11-27T15:21:02.931498+0000","last_became_active":"2023-11-27T15:20:37.673708+0000","last_became_peered":"2023-11-27T15:20:37.673708+0000","last_unstale":"2023-11-27T15:21:02.931498+0000","last_undegraded":"2023-11-27T15:21:02.931498+0000","last_fullsized":"2023-11-27T15:21:02.931498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:25:32.219577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786521+0000","last_change":"2023-11-27T15:20:32.659605+0000","last_active":"2023-11-27T15:21:02.786521+0000","last_peered":"2023-11-27T15:21:02.786521+0000","last_clean":"2023-11-27T15:21:02.786521+0000","last_became_active":"2023-11-27T15:20:32.659186+0000","last_became_peered":"2023-11-27T15:20:32.659186+0000","last_unstale":"2023-11-27T15:21:02.786521+0000","last_undegraded":"2023-11-27T15:21:02.786521+0000","last_fullsized":"2023-11-27T15:21:02.786521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:19:33.727199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780674+0000","last_change":"2023-11-27T15:20:33.227914+0000","last_active":"2023-11-27T15:21:02.780674+0000","last_peered":"2023-11-27T15:21:02.780674+0000","last_clean":"2023-11-27T15:21:02.780674+0000","last_became_active":"2023-11-27T15:20:33.227572+0000","last_became_peered":"2023-11-27T15:20:33.227572+0000","last_unstale":"2023-11-27T15:21:02.780674+0000","last_undegraded":"2023-11-27T15:21:02.780674+0000","last_fullsized":"2023-11-27T15:21:02.780674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:16:00.098620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932874+0000","last_change":"2023-11-27T15:20:33.215605+0000","last_active":"2023-11-27T15:21:02.932874+0000","last_peered":"2023-11-27T15:21:02.932874+0000","last_clean":"2023-11-27T15:21:02.932874+0000","last_became_active":"2023-11-27T15:20:33.215295+0000","last_became_peered":"2023-11-27T15:20:33.215295+0000","last_unstale":"2023-11-27T15:21:02.932874+0000","last_undegraded":"2023-11-27T15:21:02.932874+0000","last_fullsized":"2023-11-27T15:21:02.932874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:46:08.937950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780509+0000","last_change":"2023-11-27T15:20:33.214014+0000","last_active":"2023-11-27T15:21:02.780509+0000","last_peered":"2023-11-27T15:21:02.780509+0000","last_clean":"2023-11-27T15:21:02.780509+0000","last_became_active":"2023-11-27T15:20:33.213685+0000","last_became_peered":"2023-11-27T15:20:33.213685+0000","last_unstale":"2023-11-27T15:21:02.780509+0000","last_undegraded":"2023-11-27T15:21:02.780509+0000","last_fullsized":"2023-11-27T15:21:02.780509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:06:53.889016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779696+0000","last_change":"2023-11-27T15:20:33.229722+0000","last_active":"2023-11-27T15:21:02.779696+0000","last_peered":"2023-11-27T15:21:02.779696+0000","last_clean":"2023-11-27T15:21:02.779696+0000","last_became_active":"2023-11-27T15:20:33.229372+0000","last_became_peered":"2023-11-27T15:20:33.229372+0000","last_unstale":"2023-11-27T15:21:02.779696+0000","last_undegraded":"2023-11-27T15:21:02.779696+0000","last_fullsized":"2023-11-27T15:21:02.779696+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:48:26.454003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785590+0000","last_change":"2023-11-27T15:20:33.235047+0000","last_active":"2023-11-27T15:21:02.785590+0000","last_peered":"2023-11-27T15:21:02.785590+0000","last_clean":"2023-11-27T15:21:02.785590+0000","last_became_active":"2023-11-27T15:20:33.233078+0000","last_became_peered":"2023-11-27T15:20:33.233078+0000","last_unstale":"2023-11-27T15:21:02.785590+0000","last_undegraded":"2023-11-27T15:21:02.785590+0000","last_fullsized":"2023-11-27T15:21:02.785590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:56:34.043375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784969+0000","last_change":"2023-11-27T15:20:33.229118+0000","last_active":"2023-11-27T15:21:02.784969+0000","last_peered":"2023-11-27T15:21:02.784969+0000","last_clean":"2023-11-27T15:21:02.784969+0000","last_became_active":"2023-11-27T15:20:33.228748+0000","last_became_peered":"2023-11-27T15:20:33.228748+0000","last_unstale":"2023-11-27T15:21:02.784969+0000","last_undegraded":"2023-11-27T15:21:02.784969+0000","last_fullsized":"2023-11-27T15:21:02.784969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:44:41.459719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.933214+0000","last_change":"2023-11-27T15:20:33.222714+0000","last_active":"2023-11-27T15:21:02.933214+0000","last_peered":"2023-11-27T15:21:02.933214+0000","last_clean":"2023-11-27T15:21:02.933214+0000","last_became_active":"2023-11-27T15:20:33.222396+0000","last_became_peered":"2023-11-27T15:20:33.222396+0000","last_unstale":"2023-11-27T15:21:02.933214+0000","last_undegraded":"2023-11-27T15:21:02.933214+0000","last_fullsized":"2023-11-27T15:21:02.933214+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:13:29.493579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780269+0000","last_change":"2023-11-27T15:20:33.223568+0000","last_active":"2023-11-27T15:21:02.780269+0000","last_peered":"2023-11-27T15:21:02.780269+0000","last_clean":"2023-11-27T15:21:02.780269+0000","last_became_active":"2023-11-27T15:20:33.220687+0000","last_became_peered":"2023-11-27T15:20:33.220687+0000","last_unstale":"2023-11-27T15:21:02.780269+0000","last_undegraded":"2023-11-27T15:21:02.780269+0000","last_fullsized":"2023-11-27T15:21:02.780269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:56:29.431345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786992+0000","last_change":"2023-11-27T15:21:02.786992+0000","last_active":"2023-11-27T15:21:02.786992+0000","last_peered":"2023-11-27T15:21:02.786992+0000","last_clean":"2023-11-27T15:21:02.786992+0000","last_became_active":"2023-11-27T15:21:02.786644+0000","last_became_peered":"2023-11-27T15:21:02.786644+0000","last_unstale":"2023-11-27T15:21:02.786992+0000","last_undegraded":"2023-11-27T15:21:02.786992+0000","last_fullsized":"2023-11-27T15:21:02.786992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:18:27.697514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.931958+0000","last_change":"2023-11-27T15:20:33.245815+0000","last_active":"2023-11-27T15:21:02.931958+0000","last_peered":"2023-11-27T15:21:02.931958+0000","last_clean":"2023-11-27T15:21:02.931958+0000","last_became_active":"2023-11-27T15:20:33.245243+0000","last_became_peered":"2023-11-27T15:20:33.245243+0000","last_unstale":"2023-11-27T15:21:02.931958+0000","last_undegraded":"2023-11-27T15:21:02.931958+0000","last_fullsized":"2023-11-27T15:21:02.931958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:26:43.029408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784226+0000","last_change":"2023-11-27T15:20:33.245595+0000","last_active":"2023-11-27T15:21:02.784226+0000","last_peered":"2023-11-27T15:21:02.784226+0000","last_clean":"2023-11-27T15:21:02.784226+0000","last_became_active":"2023-11-27T15:20:33.245294+0000","last_became_peered":"2023-11-27T15:20:33.245294+0000","last_unstale":"2023-11-27T15:21:02.784226+0000","last_undegraded":"2023-11-27T15:21:02.784226+0000","last_fullsized":"2023-11-27T15:21:02.784226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:38:31.036319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.795538+0000","last_change":"2023-11-27T15:20:33.232976+0000","last_active":"2023-11-27T15:21:02.795538+0000","last_peered":"2023-11-27T15:21:02.795538+0000","last_clean":"2023-11-27T15:21:02.795538+0000","last_became_active":"2023-11-27T15:20:33.232648+0000","last_became_peered":"2023-11-27T15:20:33.232648+0000","last_unstale":"2023-11-27T15:21:02.795538+0000","last_undegraded":"2023-11-27T15:21:02.795538+0000","last_fullsized":"2023-11-27T15:21:02.795538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:45:02.656102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783702+0000","last_change":"2023-11-27T15:20:33.226332+0000","last_active":"2023-11-27T15:21:02.783702+0000","last_peered":"2023-11-27T15:21:02.783702+0000","last_clean":"2023-11-27T15:21:02.783702+0000","last_became_active":"2023-11-27T15:20:33.226000+0000","last_became_peered":"2023-11-27T15:20:33.226000+0000","last_unstale":"2023-11-27T15:21:02.783702+0000","last_undegraded":"2023-11-27T15:21:02.783702+0000","last_fullsized":"2023-11-27T15:21:02.783702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:24:34.018543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932933+0000","last_change":"2023-11-27T15:20:33.217927+0000","last_active":"2023-11-27T15:21:02.932933+0000","last_peered":"2023-11-27T15:21:02.932933+0000","last_clean":"2023-11-27T15:21:02.932933+0000","last_became_active":"2023-11-27T15:20:33.217003+0000","last_became_peered":"2023-11-27T15:20:33.217003+0000","last_unstale":"2023-11-27T15:21:02.932933+0000","last_undegraded":"2023-11-27T15:21:02.932933+0000","last_fullsized":"2023-11-27T15:21:02.932933+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:27:50.580759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":41917,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":16,"num_read_kb":32,"num_write":131,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":274432,"data_stored":57313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115,"ondisk_log_size":115,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416402,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6168,"kb_used_data":976,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93738984,"statfs":{"total":95995035648,"available":95988719616,"internally_reserved":0,"allocated":999424,"data_stored":490042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.57999999999999996}]},{"osd":1,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.72399999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":4,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.76800000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.747}]},{"osd":6,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.68400000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.79700000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":9,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.89200000000000002}]},{"osd":10,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.877}]}]},{"osd":10,"up_from":71,"seq":304942678038,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7068,"kb_used_data":1620,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738084,"statfs":{"total":95995035648,"available":95987798016,"internally_reserved":0,"allocated":1658880,"data_stored":1103505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.36599999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.64600000000000002}]},{"osd":3,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.51000000000000001}]},{"osd":4,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.41199999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.57199999999999995}]},{"osd":6,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.55700000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.56799999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.67300000000000004}]},{"osd":9,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.58699999999999997}]},{"osd":11,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.70399999999999996}]}]},{"osd":9,"up_from":66,"seq":283467841561,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6380,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738772,"statfs":{"total":95995035648,"available":95988502528,"internally_reserved":0,"allocated":1019904,"data_stored":490860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.63600000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.71099999999999997}]},{"osd":2,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.97999999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.73599999999999999}]},{"osd":4,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.90100000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.996},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.88600000000000001}]},{"osd":6,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.80400000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.85199999999999998}]},{"osd":8,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.0649999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.91700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0349999999999999}]}]},{"osd":8,"up_from":58,"seq":249108103199,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6412,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738740,"statfs":{"total":95995035648,"available":95988469760,"internally_reserved":0,"allocated":987136,"data_stored":480749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58199999999999996,"15min":0.58599999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.58899999999999997,"15min":0.60099999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71199999999999997,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.47099999999999997}]},{"osd":1,"last update":"Mon Nov 27 15:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78500000000000003,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.746}]},{"osd":2,"last update":"Mon Nov 27 15:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.442},{"interface":"front","average":{"1min":0.495,"5min":0.45100000000000001,"15min":0.443},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.55500000000000005}]},{"osd":3,"last update":"Mon Nov 27 15:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.40899999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.59099999999999997,"15min":0.59199999999999997},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.57599999999999996}]},{"osd":4,"last update":"Mon Nov 27 15:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.62,"15min":0.625},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80000000000000004,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.47399999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.88400000000000001,"5min":1.377,"15min":1.377},"last":0.59899999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58099999999999996,"15min":0.57999999999999996},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55200000000000005,"15min":0.55000000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80900000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.71299999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.52000000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.91000000000000003}]},{"osd":7,"last update":"Mon Nov 27 15:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95799999999999996,"5min":1.323,"15min":1.323},"last":0.83699999999999997}]},{"osd":9,"last update":"Mon Nov 27 15:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.93100000000000005}]},{"osd":10,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.88400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]}]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7156,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93737996,"statfs":{"total":95995035648,"available":95987707904,"internally_reserved":0,"allocated":1617920,"data_stored":1090688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.436,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84699999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.44},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77100000000000002,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.46999999999999997}]},{"osd":1,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.39800000000000002,"15min":0.39000000000000001},"min":{"1min":0.27900000000000003,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.40000000000000002,"15min":0.39600000000000002},"min":{"1min":0.32100000000000001,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.39800000000000002}]},{"osd":2,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.622,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.63700000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.61799999999999999,"15min":0.62},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85699999999999998,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.64300000000000002,"15min":0.63700000000000001},"min":{"1min":0.51500000000000001,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.61299999999999999}]},{"osd":4,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.504,"15min":0.49099999999999999},"min":{"1min":0.34699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.40100000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.67800000000000005,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.38200000000000001}]},{"osd":5,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.46999999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.76200000000000001,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.495,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.753,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.59699999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.68400000000000005,"15min":0.67200000000000004},"min":{"1min":0.48499999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.70099999999999996,"15min":0.69799999999999995},"min":{"1min":0.48599999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.93600000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.58499999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64900000000000002,"15min":0.64400000000000002},"min":{"1min":0.46100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.65800000000000003},"min":{"1min":0.48599999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.85699999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.623}]},{"osd":9,"last update":"Mon Nov 27 15:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.747}]},{"osd":10,"last update":"Mon Nov 27 15:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.73399999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]}]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6428,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738724,"statfs":{"total":95995035648,"available":95988453376,"internally_reserved":0,"allocated":1003520,"data_stored":490132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47699999999999998,"15min":0.46999999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.50700000000000001,"15min":0.48499999999999999},"min":{"1min":0.36599999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.54300000000000004}]},{"osd":1,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65600000000000003,"15min":0.63600000000000001},"min":{"1min":0.45300000000000001,"5min":0.441,"15min":0.441},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60899999999999999,"15min":0.57899999999999996},"min":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.55500000000000005}]},{"osd":2,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.64800000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68200000000000005,"15min":0.67400000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.70999999999999996}]},{"osd":3,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55700000000000005,"15min":0.54400000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.61299999999999999,"15min":0.59599999999999997},"min":{"1min":0.38900000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.498}]},{"osd":4,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69399999999999995,"15min":0.68600000000000005},"min":{"1min":0.55700000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.63},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68500000000000005,"15min":0.66900000000000004},"min":{"1min":0.48799999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.61199999999999999}]},{"osd":5,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71899999999999997,"15min":0.70799999999999996},"min":{"1min":0.505,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76300000000000001,"15min":0.753},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.67700000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.753,"15min":0.747},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.72499999999999998,"15min":0.70799999999999996},"min":{"1min":0.54700000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.64600000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.86099999999999999,"15min":0.86499999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.84799999999999998,"15min":0.85799999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.82399999999999995}]},{"osd":9,"last update":"Mon Nov 27 15:20:06 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":0.81799999999999995}]},{"osd":10,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.76800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7332,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":6336,"kb_avail":93737820,"statfs":{"total":95995035648,"available":95987527680,"internally_reserved":0,"allocated":1011712,"data_stored":493836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6488064},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:02 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.501,"15min":0.48299999999999998},"min":{"1min":0.38500000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.73499999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.51300000000000001,"15min":0.48199999999999998},"min":{"1min":0.36699999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.67900000000000005}]},{"osd":2,"last update":"Mon Nov 27 15:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.501,"15min":0.47999999999999998},"min":{"1min":0.38200000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.52300000000000002,"15min":0.48599999999999999},"min":{"1min":0.39800000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.90300000000000002,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.77100000000000002}]},{"osd":3,"last update":"Mon Nov 27 15:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.55500000000000005,"15min":0.52800000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54700000000000004,"15min":0.53400000000000003},"min":{"1min":0.39700000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84899999999999998,"5min":0.88,"15min":0.88},"last":0.58199999999999996}]},{"osd":4,"last update":"Mon Nov 27 15:20:15 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.40500000000000003,"15min":0.38800000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.42799999999999999,"15min":0.40500000000000003},"min":{"1min":0.29899999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.505}]},{"osd":5,"last update":"Mon Nov 27 15:20:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.54900000000000004,"15min":0.53900000000000003},"min":{"1min":0.38100000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.57599999999999996,"15min":0.55600000000000005},"min":{"1min":0.41899999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.80900000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.64700000000000002,"15min":0.624},"min":{"1min":0.45900000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.747},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.64400000000000002,"15min":0.62},"min":{"1min":0.53400000000000003,"5min":0.38,"15min":0.38},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.83599999999999997}]},{"osd":7,"last update":"Mon Nov 27 15:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.36599999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.59999999999999998,"15min":0.58699999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.878}]},{"osd":8,"last update":"Mon Nov 27 15:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64500000000000002,"15min":0.63700000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64500000000000002,"15min":0.63800000000000001},"min":{"1min":0.47999999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.76400000000000001}]},{"osd":9,"last update":"Mon Nov 27 15:20:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.93400000000000005}]},{"osd":10,"last update":"Mon Nov 27 15:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.86099999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7412,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737740,"statfs":{"total":95995035648,"available":95987445760,"internally_reserved":0,"allocated":1028096,"data_stored":491520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.53700000000000003,"15min":0.499},"min":{"1min":0.433,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.56299999999999994,"15min":0.51700000000000002},"min":{"1min":0.375,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.77000000000000002}]},{"osd":2,"last update":"Mon Nov 27 15:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.59899999999999998,"15min":0.57299999999999995},"min":{"1min":0.41599999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.59999999999999998,"15min":0.55700000000000005},"min":{"1min":0.45700000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.57099999999999995}]},{"osd":3,"last update":"Mon Nov 27 15:18:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.50800000000000001,"15min":0.49399999999999999},"min":{"1min":0.35499999999999998,"5min":0.309,"15min":0.309},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.497},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53200000000000003,"15min":0.52400000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.84499999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.58699999999999997}]},{"osd":4,"last update":"Mon Nov 27 15:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.622,"15min":0.60999999999999999},"min":{"1min":0.45400000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.63,"15min":0.59999999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.72199999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.45200000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.69699999999999995,"15min":0.67900000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.625}]},{"osd":6,"last update":"Mon Nov 27 15:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65500000000000003,"15min":0.627},"min":{"1min":0.41499999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.121,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63600000000000001,"15min":0.61799999999999999},"min":{"1min":0.437,"5min":0.373,"15min":0.373},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.437}]},{"osd":7,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.71799999999999997,"15min":0.71099999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.77500000000000002,"15min":0.76800000000000002},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.61499999999999999}]},{"osd":8,"last update":"Mon Nov 27 15:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.748,"15min":0.752},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.78000000000000003,"15min":0.78600000000000003},"min":{"1min":0.58999999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.60399999999999998}]},{"osd":9,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.84399999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.76100000000000001}]},{"osd":11,"last update":"Mon Nov 27 15:20:50 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.79100000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7404,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737748,"statfs":{"total":95995035648,"available":95987453952,"internally_reserved":0,"allocated":1019904,"data_stored":493775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51300000000000001,"15min":0.5},"min":{"1min":0.38100000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.505,"5min":0.50800000000000001,"15min":0.51500000000000001},"min":{"1min":0.33900000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.77900000000000003,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.83799999999999997}]},{"osd":1,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51200000000000001,"15min":0.48499999999999999},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.51400000000000001,"15min":0.49199999999999999},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.82799999999999996}]},{"osd":3,"last update":"Mon Nov 27 15:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58199999999999996,"15min":0.56299999999999994},"min":{"1min":0.443,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.56100000000000005},"min":{"1min":0.42599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77000000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.92300000000000004}]},{"osd":4,"last update":"Mon Nov 27 15:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52500000000000002,"15min":0.503},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53600000000000003,"15min":0.51500000000000001},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.76400000000000001}]},{"osd":5,"last update":"Mon Nov 27 15:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.434,"15min":0.42299999999999999},"min":{"1min":0.28899999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.70399999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.45900000000000002,"15min":0.44700000000000001},"min":{"1min":0.28899999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.61199999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.62,"15min":0.60299999999999998},"min":{"1min":0.42099999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.80000000000000004}]},{"osd":7,"last update":"Mon Nov 27 15:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.65100000000000002,"15min":0.63900000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":4.3499999999999996,"5min":4.3499999999999996,"15min":4.3499999999999996},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.73699999999999999}]},{"osd":8,"last update":"Mon Nov 27 15:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56599999999999995,"15min":0.56799999999999995},"min":{"1min":0.35299999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.59499999999999997,"15min":0.59799999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.71499999999999997}]},{"osd":9,"last update":"Mon Nov 27 15:20:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.91200000000000003}]},{"osd":10,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":3.2999999999999998,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.875},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.88800000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6760,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738392,"statfs":{"total":95995035648,"available":95988113408,"internally_reserved":0,"allocated":1015808,"data_stored":490326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.42499999999999999,"15min":0.40000000000000002},"min":{"1min":0.33400000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72099999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.45600000000000002,"15min":0.41699999999999998},"min":{"1min":0.29999999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.58399999999999996,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.60399999999999998}]},{"osd":1,"last update":"Mon Nov 27 15:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61099999999999999,"15min":0.56499999999999995},"min":{"1min":0.42999999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.872,"5min":1.056,"15min":1.056},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63500000000000001,"15min":0.57099999999999995},"min":{"1min":0.40699999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90300000000000002,"5min":5.6529999999999996,"15min":5.6529999999999996},"last":0.58999999999999997}]},{"osd":2,"last update":"Mon Nov 27 15:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64000000000000001,"15min":0.59399999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85099999999999998,"5min":1.208,"15min":1.208},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.63200000000000001,"15min":0.58799999999999997},"min":{"1min":0.49099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78900000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.65300000000000002}]},{"osd":4,"last update":"Mon Nov 27 15:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.64000000000000001,"15min":0.626},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91900000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.626},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.69299999999999995,"15min":0.66800000000000004},"min":{"1min":0.52500000000000002,"5min":0.502,"15min":0.502},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.86199999999999999}]},{"osd":5,"last update":"Mon Nov 27 15:20:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67000000000000004,"15min":0.65000000000000002},"min":{"1min":0.48399999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94099999999999995,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66900000000000004,"15min":0.65300000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.97099999999999997,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.73399999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.65600000000000003,"15min":0.63800000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0269999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.41399999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94399999999999995,"5min":1.018,"15min":1.018},"last":0.55200000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:20:22 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.01,"15min":0.99399999999999999},"min":{"1min":0.51800000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":6.9619999999999997,"5min":6.9619999999999997,"15min":6.9619999999999997},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.78400000000000003,"15min":0.77800000000000002},"min":{"1min":0.58199999999999996,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.71399999999999997}]},{"osd":8,"last update":"Mon Nov 27 15:19:38 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":5.6100000000000003,"5min":5.6100000000000003,"15min":5.6100000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.64400000000000002}]},{"osd":9,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.94399999999999995}]},{"osd":10,"last update":"Mon Nov 27 15:20:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.75},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.83999999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]}]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6620,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738532,"statfs":{"total":95995035648,"available":95988256768,"internally_reserved":0,"allocated":1003520,"data_stored":490064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.54400000000000004,"15min":0.53700000000000003},"min":{"1min":0.38300000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53500000000000003,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77900000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.54900000000000004}]},{"osd":1,"last update":"Mon Nov 27 15:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.36399999999999999,"15min":0.34499999999999997},"min":{"1min":0.28999999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.56499999999999995,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.372},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.374,"15min":0.35699999999999998},"min":{"1min":0.26600000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.35299999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.56299999999999994,"15min":0.54000000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85199999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.67700000000000005}]},{"osd":3,"last update":"Mon Nov 27 15:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56299999999999994,"15min":0.54700000000000004},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.41099999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.48699999999999999}]},{"osd":5,"last update":"Mon Nov 27 15:20:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.62,"15min":0.60599999999999998},"min":{"1min":0.45000000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.62,"15min":0.60299999999999998},"min":{"1min":0.39500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.57399999999999995}]},{"osd":6,"last update":"Mon Nov 27 15:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.45800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.66900000000000004,"15min":0.65700000000000003},"min":{"1min":0.53700000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.58399999999999996}]},{"osd":7,"last update":"Mon Nov 27 15:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57099999999999995,"15min":0.56200000000000006},"min":{"1min":0.39300000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.93400000000000005,"5min":1.054,"15min":1.054},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.35299999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.46899999999999997}]},{"osd":8,"last update":"Mon Nov 27 15:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70699999999999996,"15min":0.70299999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.499,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.61699999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.55900000000000005}]},{"osd":10,"last update":"Mon Nov 27 15:20:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.435}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7284,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93737868,"statfs":{"total":95995035648,"available":95987576832,"internally_reserved":0,"allocated":1617920,"data_stored":1081507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.51800000000000002,"15min":0.52100000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71899999999999997,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.64900000000000002,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.64600000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:20:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.40500000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.70399999999999996,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55300000000000005,"15min":0.53500000000000003},"min":{"1min":0.443,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76400000000000001,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.38400000000000001}]},{"osd":2,"last update":"Mon Nov 27 15:20:30 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.42099999999999999,"15min":0.42299999999999999},"min":{"1min":0.26400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.49199999999999999}]},{"osd":3,"last update":"Mon Nov 27 15:20:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56000000000000005,"15min":0.56299999999999994},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.58899999999999997}]},{"osd":4,"last update":"Mon Nov 27 15:20:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.46100000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58499999999999996,"15min":0.57899999999999996},"min":{"1min":0.437,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80100000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.432}]},{"osd":6,"last update":"Mon Nov 27 15:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.625,"15min":0.63500000000000001},"min":{"1min":0.41899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.44400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.93500000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.69399999999999995}]},{"osd":7,"last update":"Mon Nov 27 15:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.64100000000000001,"15min":0.64700000000000002},"min":{"1min":0.39900000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85099999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.63600000000000001,"15min":0.63800000000000001},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.55600000000000005}]},{"osd":8,"last update":"Mon Nov 27 15:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.61399999999999999,"15min":0.622},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.621},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.60199999999999998,"15min":0.60399999999999998},"min":{"1min":0.40799999999999997,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67900000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.71999999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.73099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":5096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":40435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-27T15:21:08.626 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-27T15:21:08.626 INFO:tasks.cephadm:Setting up client nodes... 2023-11-27T15:21:08.626 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-11-27T15:21:09.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:08 smithi133 ceph-mon[97860]: pgmap v293: 129 pgs: 129 active+clean; 618 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6.9 KiB/s wr, 12 op/s 2023-11-27T15:21:09.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:08 smithi133 ceph-mon[97860]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:21:09.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:08 smithi177 ceph-mon[131504]: pgmap v293: 129 pgs: 129 active+clean; 618 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6.9 KiB/s wr, 12 op/s 2023-11-27T15:21:09.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:08 smithi177 ceph-mon[131504]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:21:09.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:08 smithi174 ceph-mon[131110]: pgmap v293: 129 pgs: 129 active+clean; 618 KiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 6.9 KiB/s wr, 12 op/s 2023-11-27T15:21:09.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:08 smithi174 ceph-mon[131110]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:21:09.505 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:10.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:09 smithi133 ceph-mon[97860]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:21:10.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:09 smithi177 ceph-mon[131504]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:21:10.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:09 smithi174 ceph-mon[131110]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:21:11.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.048+0000 7f7832f00700 1 -- 172.21.15.133:0/991892847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f782c074b20 msgr2=0x7f782c072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:11.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.048+0000 7f7832f00700 1 --2- 172.21.15.133:0/991892847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f782c074b20 0x7f782c072f80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7820009990 tx=0x7f7820009ca0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.048+0000 7f7832f00700 1 -- 172.21.15.133:0/991892847 shutdown_connections 2023-11-27T15:21:11.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.048+0000 7f7832f00700 1 --2- 172.21.15.133:0/991892847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782c1081c0 0x7f782c10a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.048+0000 7f7832f00700 1 --2- 172.21.15.133:0/991892847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f782c074b20 0x7f782c072f80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.048+0000 7f7832f00700 1 --2- 172.21.15.133:0/991892847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f782c073550 0x7f782c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.048+0000 7f7832f00700 1 -- 172.21.15.133:0/991892847 >> 172.21.15.133:0/991892847 conn(0x7f782c0fb900 msgr2=0x7f782c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:11.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.048+0000 7f7832f00700 1 -- 172.21.15.133:0/991892847 shutdown_connections 2023-11-27T15:21:11.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.048+0000 7f7832f00700 1 -- 172.21.15.133:0/991892847 wait complete. 2023-11-27T15:21:11.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.049+0000 7f7832f00700 1 Processor -- start 2023-11-27T15:21:11.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.050+0000 7f7832f00700 1 -- start start 2023-11-27T15:21:11.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.050+0000 7f7832f00700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f782c073550 0x7f782c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.050+0000 7f7832f00700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782c074b20 0x7f782c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.050+0000 7f7832f00700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f782c1081c0 0x7f782c19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.050+0000 7f7832f00700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c19fdc0 con 0x7f782c074b20 2023-11-27T15:21:11.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.050+0000 7f7832f00700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c19ff00 con 0x7f782c073550 2023-11-27T15:21:11.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.050+0000 7f7832f00700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c1a0090 con 0x7f782c1081c0 2023-11-27T15:21:11.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.050+0000 7f782bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782c074b20 0x7f782c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.050+0000 7f782bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782c074b20 0x7f782c198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36092/0 (socket says 172.21.15.133:36092) 2023-11-27T15:21:11.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.050+0000 7f782bfff700 1 -- 172.21.15.133:0/4190055302 learned_addr learned my addr 172.21.15.133:0/4190055302 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:11.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.050+0000 7f7830c9c700 1 --2- 172.21.15.133:0/4190055302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f782c073550 0x7f782c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.050+0000 7f783149d700 1 --2- 172.21.15.133:0/4190055302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f782c1081c0 0x7f782c19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.051+0000 7f782bfff700 1 -- 172.21.15.133:0/4190055302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f782c1081c0 msgr2=0x7f782c19f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:11.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.051+0000 7f782bfff700 1 --2- 172.21.15.133:0/4190055302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f782c1081c0 0x7f782c19f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.051+0000 7f782bfff700 1 -- 172.21.15.133:0/4190055302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f782c073550 msgr2=0x7f782c198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:11.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.051+0000 7f782bfff700 1 --2- 172.21.15.133:0/4190055302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f782c073550 0x7f782c198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.051+0000 7f782bfff700 1 -- 172.21.15.133:0/4190055302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78200095f0 con 0x7f782c074b20 2023-11-27T15:21:11.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.051+0000 7f7830c9c700 1 --2- 172.21.15.133:0/4190055302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f782c073550 0x7f782c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:11.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.051+0000 7f782bfff700 1 --2- 172.21.15.133:0/4190055302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782c074b20 0x7f782c198770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f781800e9d0 tx=0x7f781800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:11.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.051+0000 7f7829ffb700 1 -- 172.21.15.133:0/4190055302 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7818009790 con 0x7f782c074b20 2023-11-27T15:21:11.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.051+0000 7f7829ffb700 1 -- 172.21.15.133:0/4190055302 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f781800ee70 con 0x7f782c074b20 2023-11-27T15:21:11.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.051+0000 7f7832f00700 1 -- 172.21.15.133:0/4190055302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782c10de80 con 0x7f782c074b20 2023-11-27T15:21:11.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.051+0000 7f7829ffb700 1 -- 172.21.15.133:0/4190055302 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7818010ea0 con 0x7f782c074b20 2023-11-27T15:21:11.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.052+0000 7f7832f00700 1 -- 172.21.15.133:0/4190055302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782c10dcf0 con 0x7f782c074b20 2023-11-27T15:21:11.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.053+0000 7f7832f00700 1 -- 172.21.15.133:0/4190055302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f782c106870 con 0x7f782c074b20 2023-11-27T15:21:11.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.054+0000 7f7829ffb700 1 -- 172.21.15.133:0/4190055302 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f781800cbd0 con 0x7f782c074b20 2023-11-27T15:21:11.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.056+0000 7f7829ffb700 1 --2- 172.21.15.133:0/4190055302 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f781c0a11e0 0x7f781c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:11.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.056+0000 7f7829ffb700 1 -- 172.21.15.133:0/4190055302 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7818014070 con 0x7f782c074b20 2023-11-27T15:21:11.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.056+0000 7f7830c9c700 1 --2- 172.21.15.133:0/4190055302 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f781c0a11e0 0x7f781c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:11.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.056+0000 7f7830c9c700 1 --2- 172.21.15.133:0/4190055302 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f781c0a11e0 0x7f781c0a3690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f782000b2e0 tx=0x7f7820005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:11.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.059+0000 7f7829ffb700 1 -- 172.21.15.133:0/4190055302 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7818051730 con 0x7f782c074b20 2023-11-27T15:21:11.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:10 smithi177 ceph-mon[131504]: pgmap v294: 129 pgs: 129 active+clean; 618 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 8.2 KiB/s wr, 16 op/s 2023-11-27T15:21:11.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:10 smithi174 ceph-mon[131110]: pgmap v294: 129 pgs: 129 active+clean; 618 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 8.2 KiB/s wr, 16 op/s 2023-11-27T15:21:11.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:10 smithi133 ceph-mon[97860]: pgmap v294: 129 pgs: 129 active+clean; 618 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 8.2 KiB/s wr, 16 op/s 2023-11-27T15:21:11.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.335+0000 7f7832f00700 1 -- 172.21.15.133:0/4190055302 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f782c02cfe0 con 0x7f782c074b20 2023-11-27T15:21:11.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.340+0000 7f7829ffb700 1 -- 172.21.15.133:0/4190055302 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f7818096240 con 0x7f782c074b20 2023-11-27T15:21:11.345 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2023-11-27T15:21:11.345 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDns2Rl9y0iFBAAxn6/pqQWLgRkURncH5253A== 2023-11-27T15:21:11.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.342+0000 7f7832f00700 1 -- 172.21.15.133:0/4190055302 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f781c0a11e0 msgr2=0x7f781c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:11.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.342+0000 7f7832f00700 1 --2- 172.21.15.133:0/4190055302 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f781c0a11e0 0x7f781c0a3690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f782000b2e0 tx=0x7f7820005fb0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.342+0000 7f7832f00700 1 -- 172.21.15.133:0/4190055302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782c074b20 msgr2=0x7f782c198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:11.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.342+0000 7f7832f00700 1 --2- 172.21.15.133:0/4190055302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782c074b20 0x7f782c198770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f781800e9d0 tx=0x7f781800ece0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.342+0000 7f7832f00700 1 -- 172.21.15.133:0/4190055302 shutdown_connections 2023-11-27T15:21:11.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.342+0000 7f7832f00700 1 --2- 172.21.15.133:0/4190055302 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f781c0a11e0 0x7f781c0a3690 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.342+0000 7f7832f00700 1 --2- 172.21.15.133:0/4190055302 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f782c1081c0 0x7f782c19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.342+0000 7f7832f00700 1 --2- 172.21.15.133:0/4190055302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f782c073550 0x7f782c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.342+0000 7f7832f00700 1 --2- 172.21.15.133:0/4190055302 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782c074b20 0x7f782c198770 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:11.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.342+0000 7f7832f00700 1 -- 172.21.15.133:0/4190055302 >> 172.21.15.133:0/4190055302 conn(0x7f782c0fb900 msgr2=0x7f782c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:11.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.342+0000 7f7832f00700 1 -- 172.21.15.133:0/4190055302 shutdown_connections 2023-11-27T15:21:11.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:11.342+0000 7f7832f00700 1 -- 172.21.15.133:0/4190055302 wait complete. 2023-11-27T15:21:11.905 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-11-27T15:21:11.905 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-11-27T15:21:11.905 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-27T15:21:12.004 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-27T15:21:12.004 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-11-27T15:21:12.004 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph mgr dump --format=json 2023-11-27T15:21:12.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:11 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/4190055302' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-11-27T15:21:12.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:11 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/4190055302' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-11-27T15:21:12.104 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:11 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/4190055302' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-11-27T15:21:12.104 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:11 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/4190055302' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-11-27T15:21:12.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:11 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/4190055302' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-11-27T15:21:12.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:11 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/4190055302' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-11-27T15:21:12.890 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:13.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:12 smithi177 ceph-mon[131504]: pgmap v295: 129 pgs: 129 active+clean; 623 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2023-11-27T15:21:13.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:12 smithi174 ceph-mon[131110]: pgmap v295: 129 pgs: 129 active+clean; 623 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2023-11-27T15:21:13.155 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:12 smithi133 ceph-mon[97860]: pgmap v295: 129 pgs: 129 active+clean; 623 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2023-11-27T15:21:14.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.331+0000 7f2717848700 1 -- 172.21.15.133:0/1013209291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2710100540 msgr2=0x7f2710100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:14.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.331+0000 7f2717848700 1 --2- 172.21.15.133:0/1013209291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2710100540 0x7f2710100910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f270c0099c0 tx=0x7f270c009cd0 comp rx=0 tx=0).stop 2023-11-27T15:21:14.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.331+0000 7f2717848700 1 -- 172.21.15.133:0/1013209291 shutdown_connections 2023-11-27T15:21:14.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.331+0000 7f2717848700 1 --2- 172.21.15.133:0/1013209291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27101055e0 0x7f271010d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:14.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.331+0000 7f2717848700 1 --2- 172.21.15.133:0/1013209291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2710100540 0x7f2710100910 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:14.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.331+0000 7f2717848700 1 --2- 172.21.15.133:0/1013209291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2710100ee0 0x7f2710104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:14.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.331+0000 7f2717848700 1 -- 172.21.15.133:0/1013209291 >> 172.21.15.133:0/1013209291 conn(0x7f27100fbdc0 msgr2=0x7f27100fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:14.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.331+0000 7f2717848700 1 -- 172.21.15.133:0/1013209291 shutdown_connections 2023-11-27T15:21:14.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.331+0000 7f2717848700 1 -- 172.21.15.133:0/1013209291 wait complete. 2023-11-27T15:21:14.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.333+0000 7f2717848700 1 Processor -- start 2023-11-27T15:21:14.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.333+0000 7f2717848700 1 -- start start 2023-11-27T15:21:14.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.333+0000 7f2717848700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2710100540 0x7f27101985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:14.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.333+0000 7f2717848700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2710100ee0 0x7f2710198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:14.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.333+0000 7f2717848700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27101055e0 0x7f271019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:14.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.333+0000 7f2717848700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271019d580 con 0x7f27101055e0 2023-11-27T15:21:14.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.333+0000 7f2717848700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271019d6f0 con 0x7f2710100540 2023-11-27T15:21:14.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.333+0000 7f2717848700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271019d8b0 con 0x7f2710100ee0 2023-11-27T15:21:14.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.333+0000 7f27155e4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2710100540 0x7f27101985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:14.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.333+0000 7f2715de5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27101055e0 0x7f271019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:14.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.333+0000 7f2714de3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2710100ee0 0x7f2710198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:14.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.333+0000 7f27155e4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2710100540 0x7f27101985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:46832/0 (socket says 172.21.15.133:46832) 2023-11-27T15:21:14.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.333+0000 7f27155e4700 1 -- 172.21.15.133:0/2718382593 learned_addr learned my addr 172.21.15.133:0/2718382593 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:14.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.334+0000 7f2715de5700 1 -- 172.21.15.133:0/2718382593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2710100ee0 msgr2=0x7f2710198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:14.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.334+0000 7f2715de5700 1 --2- 172.21.15.133:0/2718382593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2710100ee0 0x7f2710198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:14.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.334+0000 7f2715de5700 1 -- 172.21.15.133:0/2718382593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2710100540 msgr2=0x7f27101985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:14.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.334+0000 7f2715de5700 1 --2- 172.21.15.133:0/2718382593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2710100540 0x7f27101985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:14.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.334+0000 7f2715de5700 1 -- 172.21.15.133:0/2718382593 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270c0095f0 con 0x7f27101055e0 2023-11-27T15:21:14.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.334+0000 7f27155e4700 1 --2- 172.21.15.133:0/2718382593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2710100540 0x7f27101985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:14.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.334+0000 7f2715de5700 1 --2- 172.21.15.133:0/2718382593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27101055e0 0x7f271019cf70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f270800efc0 tx=0x7f270800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:14.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.334+0000 7f27067fc700 1 -- 172.21.15.133:0/2718382593 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2708018070 con 0x7f27101055e0 2023-11-27T15:21:14.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.334+0000 7f27067fc700 1 -- 172.21.15.133:0/2718382593 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f270800f040 con 0x7f27101055e0 2023-11-27T15:21:14.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.334+0000 7f27067fc700 1 -- 172.21.15.133:0/2718382593 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270800cd40 con 0x7f27101055e0 2023-11-27T15:21:14.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.334+0000 7f2717848700 1 -- 172.21.15.133:0/2718382593 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2710110370 con 0x7f27101055e0 2023-11-27T15:21:14.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.334+0000 7f2717848700 1 -- 172.21.15.133:0/2718382593 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27101101e0 con 0x7f27101055e0 2023-11-27T15:21:14.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.336+0000 7f2717848700 1 -- 172.21.15.133:0/2718382593 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2710110500 con 0x7f27101055e0 2023-11-27T15:21:14.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.337+0000 7f27067fc700 1 -- 172.21.15.133:0/2718382593 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f2708003a00 con 0x7f27101055e0 2023-11-27T15:21:14.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.339+0000 7f27067fc700 1 --2- 172.21.15.133:0/2718382593 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f26fc0a1260 0x7f26fc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:14.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.339+0000 7f27067fc700 1 -- 172.21.15.133:0/2718382593 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f27080cc820 con 0x7f27101055e0 2023-11-27T15:21:14.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.339+0000 7f27155e4700 1 --2- 172.21.15.133:0/2718382593 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f26fc0a1260 0x7f26fc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:14.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.340+0000 7f27155e4700 1 --2- 172.21.15.133:0/2718382593 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f26fc0a1260 0x7f26fc0a3710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f270c004800 tx=0x7f270c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:14.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.343+0000 7f27067fc700 1 -- 172.21.15.133:0/2718382593 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2708095b90 con 0x7f27101055e0 2023-11-27T15:21:14.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.594+0000 7f2717848700 1 -- 172.21.15.133:0/2718382593 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f2710199920 con 0x7f27101055e0 2023-11-27T15:21:14.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.599+0000 7f27067fc700 1 -- 172.21.15.133:0/2718382593 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260132 (secure 0 0 0) 0x7f27080991b0 con 0x7f27101055e0 2023-11-27T15:21:14.608 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:21:14.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.603+0000 7f2717848700 1 -- 172.21.15.133:0/2718382593 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f26fc0a1260 msgr2=0x7f26fc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:14.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.603+0000 7f2717848700 1 --2- 172.21.15.133:0/2718382593 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f26fc0a1260 0x7f26fc0a3710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f270c004800 tx=0x7f270c01a040 comp rx=0 tx=0).stop 2023-11-27T15:21:14.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.603+0000 7f2717848700 1 -- 172.21.15.133:0/2718382593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27101055e0 msgr2=0x7f271019cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:14.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.603+0000 7f2717848700 1 --2- 172.21.15.133:0/2718382593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27101055e0 0x7f271019cf70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f270800efc0 tx=0x7f270800c3a0 comp rx=0 tx=0).stop 2023-11-27T15:21:14.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.603+0000 7f2717848700 1 -- 172.21.15.133:0/2718382593 shutdown_connections 2023-11-27T15:21:14.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.603+0000 7f2717848700 1 --2- 172.21.15.133:0/2718382593 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f26fc0a1260 0x7f26fc0a3710 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:14.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.603+0000 7f2717848700 1 --2- 172.21.15.133:0/2718382593 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27101055e0 0x7f271019cf70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:14.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.603+0000 7f2717848700 1 --2- 172.21.15.133:0/2718382593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2710100540 0x7f27101985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:14.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.603+0000 7f2717848700 1 --2- 172.21.15.133:0/2718382593 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2710100ee0 0x7f2710198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:14.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.603+0000 7f2717848700 1 -- 172.21.15.133:0/2718382593 >> 172.21.15.133:0/2718382593 conn(0x7f27100fbdc0 msgr2=0x7f27100fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:14.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.603+0000 7f2717848700 1 -- 172.21.15.133:0/2718382593 shutdown_connections 2023-11-27T15:21:14.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:14.603+0000 7f2717848700 1 -- 172.21.15.133:0/2718382593 wait complete. 2023-11-27T15:21:15.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:14 smithi133 ceph-mon[97860]: pgmap v296: 129 pgs: 129 active+clean; 623 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2023-11-27T15:21:15.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:14 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2718382593' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-11-27T15:21:15.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:14 smithi177 ceph-mon[131504]: pgmap v296: 129 pgs: 129 active+clean; 623 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2023-11-27T15:21:15.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:14 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2718382593' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-11-27T15:21:15.090 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":17,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":4209056668},{"type":"v1","addr":"172.21.15.133:6801","nonce":4209056668}]},"active_addr":"172.21.15.133:6801/4209056668","active_change":"2023-11-27T15:14:01.628299+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.133:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":2764902438}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":3716770505}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":22185990}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":992274730}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":1683942748}]}]} 2023-11-27T15:21:15.094 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-11-27T15:21:15.094 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-11-27T15:21:15.095 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph osd dump --format=json 2023-11-27T15:21:15.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:14 smithi174 ceph-mon[131110]: pgmap v296: 129 pgs: 129 active+clean; 623 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.5 KiB/s wr, 14 op/s 2023-11-27T15:21:15.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:14 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2718382593' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-11-27T15:21:15.941 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:17.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:16 smithi177 ceph-mon[131504]: pgmap v297: 129 pgs: 129 active+clean; 623 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2023-11-27T15:21:17.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:16 smithi174 ceph-mon[131110]: pgmap v297: 129 pgs: 129 active+clean; 623 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2023-11-27T15:21:17.194 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:16 smithi133 ceph-mon[97860]: pgmap v297: 129 pgs: 129 active+clean; 623 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.9 KiB/s wr, 13 op/s 2023-11-27T15:21:17.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.463+0000 7f1ce26b6700 1 -- 172.21.15.133:0/468470074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cdc1055e0 msgr2=0x7f1cdc10d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:17.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.463+0000 7f1ce26b6700 1 --2- 172.21.15.133:0/468470074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cdc1055e0 0x7f1cdc10d160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400b0a0 tx=0x7f1cd400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:21:17.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.463+0000 7f1ce26b6700 1 -- 172.21.15.133:0/468470074 shutdown_connections 2023-11-27T15:21:17.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.463+0000 7f1ce26b6700 1 --2- 172.21.15.133:0/468470074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cdc1055e0 0x7f1cdc10d160 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:17.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.463+0000 7f1ce26b6700 1 --2- 172.21.15.133:0/468470074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cdc100540 0x7f1cdc100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:17.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.463+0000 7f1ce26b6700 1 --2- 172.21.15.133:0/468470074 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cdc100ee0 0x7f1cdc104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:17.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.463+0000 7f1ce26b6700 1 -- 172.21.15.133:0/468470074 >> 172.21.15.133:0/468470074 conn(0x7f1cdc0fbdc0 msgr2=0x7f1cdc0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:17.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.463+0000 7f1ce26b6700 1 -- 172.21.15.133:0/468470074 shutdown_connections 2023-11-27T15:21:17.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.463+0000 7f1ce26b6700 1 -- 172.21.15.133:0/468470074 wait complete. 2023-11-27T15:21:17.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.464+0000 7f1ce26b6700 1 Processor -- start 2023-11-27T15:21:17.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.464+0000 7f1ce26b6700 1 -- start start 2023-11-27T15:21:17.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.465+0000 7f1ce26b6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cdc100540 0x7f1cdc1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:17.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.465+0000 7f1ce26b6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cdc100ee0 0x7f1cdc1969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:17.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.465+0000 7f1ce26b6700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cdc1055e0 0x7f1cdc19ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:17.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.465+0000 7f1ce26b6700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc19b470 con 0x7f1cdc1055e0 2023-11-27T15:21:17.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.465+0000 7f1ce26b6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc19b5e0 con 0x7f1cdc100ee0 2023-11-27T15:21:17.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.465+0000 7f1ce26b6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc19b7a0 con 0x7f1cdc100540 2023-11-27T15:21:17.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.465+0000 7f1ce0c53700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cdc1055e0 0x7f1cdc19ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:17.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.465+0000 7f1ce0c53700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cdc1055e0 0x7f1cdc19ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39314/0 (socket says 172.21.15.133:39314) 2023-11-27T15:21:17.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.465+0000 7f1ce0c53700 1 -- 172.21.15.133:0/1390383650 learned_addr learned my addr 172.21.15.133:0/1390383650 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:17.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.465+0000 7f1cdb7fe700 1 --2- 172.21.15.133:0/1390383650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cdc100ee0 0x7f1cdc1969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:17.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.465+0000 7f1cdbfff700 1 --2- 172.21.15.133:0/1390383650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cdc100540 0x7f1cdc1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:17.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.465+0000 7f1ce0c53700 1 -- 172.21.15.133:0/1390383650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cdc100540 msgr2=0x7f1cdc1964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:17.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.466+0000 7f1ce0c53700 1 --2- 172.21.15.133:0/1390383650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cdc100540 0x7f1cdc1964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:17.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.466+0000 7f1ce0c53700 1 -- 172.21.15.133:0/1390383650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cdc100ee0 msgr2=0x7f1cdc1969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:17.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.466+0000 7f1ce0c53700 1 --2- 172.21.15.133:0/1390383650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cdc100ee0 0x7f1cdc1969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:17.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.466+0000 7f1ce0c53700 1 -- 172.21.15.133:0/1390383650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd0009520 con 0x7f1cdc1055e0 2023-11-27T15:21:17.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.466+0000 7f1ce0c53700 1 --2- 172.21.15.133:0/1390383650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cdc1055e0 0x7f1cdc19ae60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4009fd0 tx=0x7f1cd40077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:17.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.466+0000 7f1cd97fa700 1 -- 172.21.15.133:0/1390383650 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd400e040 con 0x7f1cdc1055e0 2023-11-27T15:21:17.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.466+0000 7f1cd97fa700 1 -- 172.21.15.133:0/1390383650 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cd40094c0 con 0x7f1cdc1055e0 2023-11-27T15:21:17.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.466+0000 7f1ce26b6700 1 -- 172.21.15.133:0/1390383650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd4009d00 con 0x7f1cdc1055e0 2023-11-27T15:21:17.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.466+0000 7f1cd97fa700 1 -- 172.21.15.133:0/1390383650 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd40052a0 con 0x7f1cdc1055e0 2023-11-27T15:21:17.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.466+0000 7f1ce26b6700 1 -- 172.21.15.133:0/1390383650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cdc110370 con 0x7f1cdc1055e0 2023-11-27T15:21:17.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.467+0000 7f1ce26b6700 1 -- 172.21.15.133:0/1390383650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cdc1101e0 con 0x7f1cdc1055e0 2023-11-27T15:21:17.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.469+0000 7f1cd97fa700 1 -- 172.21.15.133:0/1390383650 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f1cd401d470 con 0x7f1cdc1055e0 2023-11-27T15:21:17.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.470+0000 7f1cd97fa700 1 --2- 172.21.15.133:0/1390383650 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1cc40a1260 0x7f1cc40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:17.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.471+0000 7f1cd97fa700 1 -- 172.21.15.133:0/1390383650 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1cd40ce170 con 0x7f1cdc1055e0 2023-11-27T15:21:17.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.471+0000 7f1cdbfff700 1 --2- 172.21.15.133:0/1390383650 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1cc40a1260 0x7f1cc40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:17.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.471+0000 7f1cdbfff700 1 --2- 172.21.15.133:0/1390383650 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1cc40a1260 0x7f1cc40a3710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009c60 tx=0x7f1ccc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:17.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.474+0000 7f1cd97fa700 1 -- 172.21.15.133:0/1390383650 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1cd4097480 con 0x7f1cdc1055e0 2023-11-27T15:21:17.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.675+0000 7f1ce26b6700 1 -- 172.21.15.133:0/1390383650 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1cdc110500 con 0x7f1cdc1055e0 2023-11-27T15:21:17.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.676+0000 7f1cd97fa700 1 -- 172.21.15.133:0/1390383650 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24465 (secure 0 0 0) 0x7f1cd40bae60 con 0x7f1cdc1055e0 2023-11-27T15:21:17.682 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:21:17.682 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":90,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:21:02.774978+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:21:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.679+0000 7f1ce26b6700 1 -- 172.21.15.133:0/1390383650 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1cc40a1260 msgr2=0x7f1cc40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.679+0000 7f1ce26b6700 1 --2- 172.21.15.133:0/1390383650 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1cc40a1260 0x7f1cc40a3710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009c60 tx=0x7f1ccc009260 comp rx=0 tx=0).stop 2023-11-27T15:21:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.679+0000 7f1ce26b6700 1 -- 172.21.15.133:0/1390383650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cdc1055e0 msgr2=0x7f1cdc19ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.679+0000 7f1ce26b6700 1 --2- 172.21.15.133:0/1390383650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cdc1055e0 0x7f1cdc19ae60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4009fd0 tx=0x7f1cd40077f0 comp rx=0 tx=0).stop 2023-11-27T15:21:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.679+0000 7f1ce26b6700 1 -- 172.21.15.133:0/1390383650 shutdown_connections 2023-11-27T15:21:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.679+0000 7f1ce26b6700 1 --2- 172.21.15.133:0/1390383650 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1cc40a1260 0x7f1cc40a3710 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.679+0000 7f1ce26b6700 1 --2- 172.21.15.133:0/1390383650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cdc1055e0 0x7f1cdc19ae60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.679+0000 7f1ce26b6700 1 --2- 172.21.15.133:0/1390383650 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cdc100540 0x7f1cdc1964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.679+0000 7f1ce26b6700 1 --2- 172.21.15.133:0/1390383650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1cdc100ee0 0x7f1cdc1969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.680+0000 7f1ce26b6700 1 -- 172.21.15.133:0/1390383650 >> 172.21.15.133:0/1390383650 conn(0x7f1cdc0fbdc0 msgr2=0x7f1cdc0fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.680+0000 7f1ce26b6700 1 -- 172.21.15.133:0/1390383650 shutdown_connections 2023-11-27T15:21:17.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:17.680+0000 7f1ce26b6700 1 -- 172.21.15.133:0/1390383650 wait complete. 2023-11-27T15:21:18.136 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-11-27T15:21:18.137 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph osd dump --format=json 2023-11-27T15:21:18.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:18 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1390383650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:21:18.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:18 smithi174 ceph-mon[131110]: pgmap v298: 129 pgs: 129 active+clean; 631 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.8 KiB/s wr, 12 op/s 2023-11-27T15:21:18.987 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:19.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:18 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1390383650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:21:19.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:18 smithi133 ceph-mon[97860]: pgmap v298: 129 pgs: 129 active+clean; 631 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.8 KiB/s wr, 12 op/s 2023-11-27T15:21:19.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:18 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1390383650' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:21:19.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:18 smithi177 ceph-mon[131504]: pgmap v298: 129 pgs: 129 active+clean; 631 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.8 KiB/s wr, 12 op/s 2023-11-27T15:21:20.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.397+0000 7f881f431700 1 -- 172.21.15.133:0/4074219847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8818100540 msgr2=0x7f8818100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:20.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.397+0000 7f881f431700 1 --2- 172.21.15.133:0/4074219847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8818100540 0x7f8818100910 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f88140099c0 tx=0x7f8814009cd0 comp rx=0 tx=0).stop 2023-11-27T15:21:20.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.398+0000 7f881f431700 1 -- 172.21.15.133:0/4074219847 shutdown_connections 2023-11-27T15:21:20.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.398+0000 7f881f431700 1 --2- 172.21.15.133:0/4074219847 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88181055e0 0x7f881810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:20.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.398+0000 7f881f431700 1 --2- 172.21.15.133:0/4074219847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8818100540 0x7f8818100910 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:20.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.398+0000 7f881f431700 1 --2- 172.21.15.133:0/4074219847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8818100ee0 0x7f8818104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:20.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.398+0000 7f881f431700 1 -- 172.21.15.133:0/4074219847 >> 172.21.15.133:0/4074219847 conn(0x7f88180fbdc0 msgr2=0x7f88180fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:20.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.398+0000 7f881f431700 1 -- 172.21.15.133:0/4074219847 shutdown_connections 2023-11-27T15:21:20.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.398+0000 7f881f431700 1 -- 172.21.15.133:0/4074219847 wait complete. 2023-11-27T15:21:20.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.399+0000 7f881f431700 1 Processor -- start 2023-11-27T15:21:20.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.399+0000 7f881f431700 1 -- start start 2023-11-27T15:21:20.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.399+0000 7f881f431700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8818100540 0x7f8818072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:20.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.399+0000 7f881f431700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8818100ee0 0x7f8818073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:20.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.399+0000 7f881f431700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88181055e0 0x7f88180738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:20.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.399+0000 7f881f431700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88181a4810 con 0x7f88181055e0 2023-11-27T15:21:20.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.399+0000 7f881f431700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88181a4980 con 0x7f8818100540 2023-11-27T15:21:20.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.399+0000 7f881f431700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88181a4b40 con 0x7f8818100ee0 2023-11-27T15:21:20.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.399+0000 7f881c9cc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8818100ee0 0x7f8818073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:20.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.399+0000 7f881d1cd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8818100540 0x7f8818072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:20.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.399+0000 7f881c9cc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8818100ee0 0x7f8818073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:37620/0 (socket says 172.21.15.133:37620) 2023-11-27T15:21:20.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.399+0000 7f881c9cc700 1 -- 172.21.15.133:0/2368274846 learned_addr learned my addr 172.21.15.133:0/2368274846 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:20.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.399+0000 7f881d9ce700 1 --2- 172.21.15.133:0/2368274846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88181055e0 0x7f88180738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:20.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.400+0000 7f881c9cc700 1 -- 172.21.15.133:0/2368274846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8818100540 msgr2=0x7f8818072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:20.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.400+0000 7f881c9cc700 1 --2- 172.21.15.133:0/2368274846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8818100540 0x7f8818072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:20.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.400+0000 7f881c9cc700 1 -- 172.21.15.133:0/2368274846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88181055e0 msgr2=0x7f88180738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:20.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.400+0000 7f881c9cc700 1 --2- 172.21.15.133:0/2368274846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88181055e0 0x7f88180738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:20.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.400+0000 7f881c9cc700 1 -- 172.21.15.133:0/2368274846 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88140095f0 con 0x7f8818100ee0 2023-11-27T15:21:20.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.400+0000 7f881c9cc700 1 --2- 172.21.15.133:0/2368274846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8818100ee0 0x7f8818073330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f880800e970 tx=0x7f880800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:20.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.400+0000 7f880e7fc700 1 -- 172.21.15.133:0/2368274846 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8808009790 con 0x7f8818100ee0 2023-11-27T15:21:20.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.400+0000 7f881f431700 1 -- 172.21.15.133:0/2368274846 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8818110320 con 0x7f8818100ee0 2023-11-27T15:21:20.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.400+0000 7f880e7fc700 1 -- 172.21.15.133:0/2368274846 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f880800c7e0 con 0x7f8818100ee0 2023-11-27T15:21:20.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.400+0000 7f880e7fc700 1 -- 172.21.15.133:0/2368274846 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88080054e0 con 0x7f8818100ee0 2023-11-27T15:21:20.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.400+0000 7f881f431700 1 -- 172.21.15.133:0/2368274846 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8818110190 con 0x7f8818100ee0 2023-11-27T15:21:20.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.402+0000 7f881f431700 1 -- 172.21.15.133:0/2368274846 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88181104b0 con 0x7f8818100ee0 2023-11-27T15:21:20.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.403+0000 7f880e7fc700 1 -- 172.21.15.133:0/2368274846 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f880800c950 con 0x7f8818100ee0 2023-11-27T15:21:20.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.405+0000 7f880e7fc700 1 --2- 172.21.15.133:0/2368274846 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f88040a11e0 0x7f88040a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:20.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.405+0000 7f880e7fc700 1 -- 172.21.15.133:0/2368274846 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8808014070 con 0x7f8818100ee0 2023-11-27T15:21:20.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.405+0000 7f881d1cd700 1 --2- 172.21.15.133:0/2368274846 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f88040a11e0 0x7f88040a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:20.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.405+0000 7f881d1cd700 1 --2- 172.21.15.133:0/2368274846 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f88040a11e0 0x7f88040a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8814009990 tx=0x7f8814005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:20.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.407+0000 7f880e7fc700 1 -- 172.21.15.133:0/2368274846 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f88080969b0 con 0x7f8818100ee0 2023-11-27T15:21:20.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.604+0000 7f881f431700 1 -- 172.21.15.133:0/2368274846 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8818066ea0 con 0x7f8818100ee0 2023-11-27T15:21:20.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.606+0000 7f880e7fc700 1 -- 172.21.15.133:0/2368274846 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24465 (secure 0 0 0) 0x7f8808099fd0 con 0x7f8818100ee0 2023-11-27T15:21:20.612 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:21:20.612 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":90,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:21:02.774978+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:21:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.609+0000 7f881f431700 1 -- 172.21.15.133:0/2368274846 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f88040a11e0 msgr2=0x7f88040a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.609+0000 7f881f431700 1 --2- 172.21.15.133:0/2368274846 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f88040a11e0 0x7f88040a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8814009990 tx=0x7f8814005fb0 comp rx=0 tx=0).stop 2023-11-27T15:21:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.609+0000 7f881f431700 1 -- 172.21.15.133:0/2368274846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8818100ee0 msgr2=0x7f8818073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:20.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.609+0000 7f881f431700 1 --2- 172.21.15.133:0/2368274846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8818100ee0 0x7f8818073330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f880800e970 tx=0x7f880800ec80 comp rx=0 tx=0).stop 2023-11-27T15:21:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.609+0000 7f881f431700 1 -- 172.21.15.133:0/2368274846 shutdown_connections 2023-11-27T15:21:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.609+0000 7f881f431700 1 --2- 172.21.15.133:0/2368274846 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f88040a11e0 0x7f88040a3690 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.609+0000 7f881f431700 1 --2- 172.21.15.133:0/2368274846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88181055e0 0x7f88180738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.609+0000 7f881f431700 1 --2- 172.21.15.133:0/2368274846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8818100540 0x7f8818072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.609+0000 7f881f431700 1 --2- 172.21.15.133:0/2368274846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8818100ee0 0x7f8818073330 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.609+0000 7f881f431700 1 -- 172.21.15.133:0/2368274846 >> 172.21.15.133:0/2368274846 conn(0x7f88180fbdc0 msgr2=0x7f88180fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.609+0000 7f881f431700 1 -- 172.21.15.133:0/2368274846 shutdown_connections 2023-11-27T15:21:20.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:20.609+0000 7f881f431700 1 -- 172.21.15.133:0/2368274846 wait complete. 2023-11-27T15:21:20.997 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:20 smithi133 ceph-mon[97860]: pgmap v299: 129 pgs: 129 active+clean; 631 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2023-11-27T15:21:20.997 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:20 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2368274846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:21:21.078 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2023-11-27T15:21:21.079 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2023-11-27T15:21:21.080 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2023-11-27T15:21:21.080 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2023-11-27T15:21:21.081 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2023-11-27T15:21:21.081 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2023-11-27T15:21:21.082 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2023-11-27T15:21:21.082 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2023-11-27T15:21:21.083 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2023-11-27T15:21:21.084 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2023-11-27T15:21:21.084 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2023-11-27T15:21:21.085 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2023-11-27T15:21:21.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:20 smithi177 ceph-mon[131504]: pgmap v299: 129 pgs: 129 active+clean; 631 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2023-11-27T15:21:21.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:20 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2368274846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:21:21.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:20 smithi174 ceph-mon[131110]: pgmap v299: 129 pgs: 129 active+clean; 631 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2023-11-27T15:21:21.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:20 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2368274846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:21:23.031 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:23.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:22 smithi177 ceph-mon[131504]: pgmap v300: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2023-11-27T15:21:23.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:22 smithi174 ceph-mon[131110]: pgmap v300: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2023-11-27T15:21:23.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:22 smithi133 ceph-mon[97860]: pgmap v300: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2023-11-27T15:21:23.377 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:23.966 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:23.969 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:24.060 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:24.206 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:24.212 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:24.213 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:24.603 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:24.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.632+0000 7f1bd585a700 1 -- 172.21.15.133:0/2484111781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bd0100540 msgr2=0x7f1bd0100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:24.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.632+0000 7f1bd585a700 1 --2- 172.21.15.133:0/2484111781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bd0100540 0x7f1bd0100910 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1bc00098f0 tx=0x7f1bc0009c00 comp rx=0 tx=0).stop 2023-11-27T15:21:24.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.632+0000 7f1bd585a700 1 -- 172.21.15.133:0/2484111781 shutdown_connections 2023-11-27T15:21:24.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.632+0000 7f1bd585a700 1 --2- 172.21.15.133:0/2484111781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bd01055e0 0x7f1bd010d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:24.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.632+0000 7f1bd585a700 1 --2- 172.21.15.133:0/2484111781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bd0100540 0x7f1bd0100910 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:24.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.632+0000 7f1bd585a700 1 --2- 172.21.15.133:0/2484111781 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bd0100ee0 0x7f1bd0104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:24.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.632+0000 7f1bd585a700 1 -- 172.21.15.133:0/2484111781 >> 172.21.15.133:0/2484111781 conn(0x7f1bd00fbdc0 msgr2=0x7f1bd00fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:24.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.632+0000 7f1bd585a700 1 -- 172.21.15.133:0/2484111781 shutdown_connections 2023-11-27T15:21:24.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.632+0000 7f1bd585a700 1 -- 172.21.15.133:0/2484111781 wait complete. 2023-11-27T15:21:24.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.633+0000 7f1bd585a700 1 Processor -- start 2023-11-27T15:21:24.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.633+0000 7f1bd585a700 1 -- start start 2023-11-27T15:21:24.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.634+0000 7f1bd585a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bd0100540 0x7f1bd01986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:24.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.634+0000 7f1bd585a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bd0100ee0 0x7f1bd0198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:24.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.634+0000 7f1bd585a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bd01055e0 0x7f1bd019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:24.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.634+0000 7f1bd585a700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd019d5d0 con 0x7f1bd0100540 2023-11-27T15:21:24.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.634+0000 7f1bd585a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd019d740 con 0x7f1bd01055e0 2023-11-27T15:21:24.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.634+0000 7f1bd585a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bd019d900 con 0x7f1bd0100ee0 2023-11-27T15:21:24.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.634+0000 7f1bceffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bd0100540 0x7f1bd01986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:24.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.634+0000 7f1bceffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bd0100540 0x7f1bd01986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60136/0 (socket says 172.21.15.133:60136) 2023-11-27T15:21:24.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.634+0000 7f1bcf7fe700 1 --2- 172.21.15.133:0/2136499173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bd01055e0 0x7f1bd019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:24.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.634+0000 7f1bceffd700 1 -- 172.21.15.133:0/2136499173 learned_addr learned my addr 172.21.15.133:0/2136499173 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:24.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.634+0000 7f1bce7fc700 1 --2- 172.21.15.133:0/2136499173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bd0100ee0 0x7f1bd0198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:24.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.635+0000 7f1bcf7fe700 1 -- 172.21.15.133:0/2136499173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bd0100ee0 msgr2=0x7f1bd0198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:24.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.635+0000 7f1bcf7fe700 1 --2- 172.21.15.133:0/2136499173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bd0100ee0 0x7f1bd0198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:24.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.635+0000 7f1bcf7fe700 1 -- 172.21.15.133:0/2136499173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bd0100540 msgr2=0x7f1bd01986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:24.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.635+0000 7f1bcf7fe700 1 --2- 172.21.15.133:0/2136499173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bd0100540 0x7f1bd01986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:24.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.635+0000 7f1bcf7fe700 1 -- 172.21.15.133:0/2136499173 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc0009520 con 0x7f1bd01055e0 2023-11-27T15:21:24.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.635+0000 7f1bce7fc700 1 --2- 172.21.15.133:0/2136499173 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1bd0100ee0 0x7f1bd0198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:24.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.635+0000 7f1bceffd700 1 --2- 172.21.15.133:0/2136499173 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1bd0100540 0x7f1bd01986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:24.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.635+0000 7f1bcf7fe700 1 --2- 172.21.15.133:0/2136499173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bd01055e0 0x7f1bd019cfc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800efc0 tx=0x7f1bc800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:24.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.636+0000 7f1bd4858700 1 -- 172.21.15.133:0/2136499173 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8018070 con 0x7f1bd01055e0 2023-11-27T15:21:24.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.636+0000 7f1bd4858700 1 -- 172.21.15.133:0/2136499173 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bc800f040 con 0x7f1bd01055e0 2023-11-27T15:21:24.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.636+0000 7f1bd4858700 1 -- 172.21.15.133:0/2136499173 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc800ccb0 con 0x7f1bd01055e0 2023-11-27T15:21:24.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.636+0000 7f1bd585a700 1 -- 172.21.15.133:0/2136499173 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bd0110370 con 0x7f1bd01055e0 2023-11-27T15:21:24.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.636+0000 7f1bd585a700 1 -- 172.21.15.133:0/2136499173 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bd01101e0 con 0x7f1bd01055e0 2023-11-27T15:21:24.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.637+0000 7f1bd585a700 1 -- 172.21.15.133:0/2136499173 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1bd00fd2e0 con 0x7f1bd01055e0 2023-11-27T15:21:24.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.641+0000 7f1bd4858700 1 -- 172.21.15.133:0/2136499173 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f1bc8003a00 con 0x7f1bd01055e0 2023-11-27T15:21:24.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.643+0000 7f1bd4858700 1 --2- 172.21.15.133:0/2136499173 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1bb80a12e0 0x7f1bb80a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:24.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.643+0000 7f1bd4858700 1 -- 172.21.15.133:0/2136499173 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1bc80cc730 con 0x7f1bd01055e0 2023-11-27T15:21:24.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.643+0000 7f1bceffd700 1 --2- 172.21.15.133:0/2136499173 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1bb80a12e0 0x7f1bb80a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:24.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.643+0000 7f1bd4858700 1 --2- 172.21.15.133:0/2136499173 >> [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592] conn(0x7f1bb80ac2f0 0x7f1bb80ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:24.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.643+0000 7f1bd4858700 1 -- 172.21.15.133:0/2136499173 --> [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1bb80aee50 con 0x7f1bb80ac2f0 2023-11-27T15:21:24.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.643+0000 7f1bd4858700 1 -- 172.21.15.133:0/2136499173 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1bc80ced50 con 0x7f1bd01055e0 2023-11-27T15:21:24.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.644+0000 7f1bce7fc700 1 --2- 172.21.15.133:0/2136499173 >> [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592] conn(0x7f1bb80ac2f0 0x7f1bb80ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:24.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.644+0000 7f1bceffd700 1 --2- 172.21.15.133:0/2136499173 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1bb80a12e0 0x7f1bb80a3790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0011040 tx=0x7f1bc0019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:24.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.644+0000 7f1bce7fc700 1 --2- 172.21.15.133:0/2136499173 >> [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592] conn(0x7f1bb80ac2f0 0x7f1bb80ae780 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:24.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.645+0000 7f1bd4858700 1 -- 172.21.15.133:0/2136499173 <== osd.6 v2:172.21.15.133:6818/3664128592 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1bb80aee50 con 0x7f1bb80ac2f0 2023-11-27T15:21:24.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.672+0000 7f1bd585a700 1 -- 172.21.15.133:0/2136499173 --> [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1bd0110500 con 0x7f1bb80ac2f0 2023-11-27T15:21:24.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.672+0000 7f1bd4858700 1 -- 172.21.15.133:0/2136499173 <== osd.6 v2:172.21.15.133:6818/3664128592 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1bd0110500 con 0x7f1bb80ac2f0 2023-11-27T15:21:24.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.673+0000 7f1bd585a700 1 -- 172.21.15.133:0/2136499173 >> [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592] conn(0x7f1bb80ac2f0 msgr2=0x7f1bb80ae780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:24.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.673+0000 7f1bd585a700 1 --2- 172.21.15.133:0/2136499173 >> [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592] conn(0x7f1bb80ac2f0 0x7f1bb80ae780 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:24.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.673+0000 7f1bd585a700 1 -- 172.21.15.133:0/2136499173 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1bb80a12e0 msgr2=0x7f1bb80a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:24.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.673+0000 7f1bd585a700 1 --2- 172.21.15.133:0/2136499173 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1bb80a12e0 0x7f1bb80a3790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0011040 tx=0x7f1bc0019040 comp rx=0 tx=0).stop 2023-11-27T15:21:24.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.673+0000 7f1bd585a700 1 -- 172.21.15.133:0/2136499173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bd01055e0 msgr2=0x7f1bd019cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:24.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.673+0000 7f1bd585a700 1 --2- 172.21.15.133:0/2136499173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bd01055e0 0x7f1bd019cfc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800efc0 tx=0x7f1bc800c3a0 comp rx=0 tx=0).stop 2023-11-27T15:21:24.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.673+0000 7f1bcf7fe700 1 -- 172.21.15.133:0/2136499173 reap_dead start 2023-11-27T15:21:24.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.673+0000 7f1bd585a700 1 -- 172.21.15.133:0/2136499173 shutdown_connections 2023-11-27T15:21:24.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.673+0000 7f1bd585a700 1 -- 172.21.15.133:0/2136499173 >> 172.21.15.133:0/2136499173 conn(0x7f1bd00fbdc0 msgr2=0x7f1bd00fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:24.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.673+0000 7f1bd585a700 1 -- 172.21.15.133:0/2136499173 shutdown_connections 2023-11-27T15:21:24.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:24.673+0000 7f1bd585a700 1 -- 172.21.15.133:0/2136499173 wait complete. 2023-11-27T15:21:24.999 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:25.020 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:24 smithi133 ceph-mon[97860]: pgmap v301: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-11-27T15:21:25.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:24 smithi177 ceph-mon[131504]: pgmap v301: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-11-27T15:21:25.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:24 smithi174 ceph-mon[131110]: pgmap v301: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2023-11-27T15:21:25.608 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:25.638 INFO:teuthology.orchestra.run.smithi133.stdout:197568495660 2023-11-27T15:21:25.639 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.6 2023-11-27T15:21:26.195 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:26.860 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:26 smithi133 ceph-mon[97860]: pgmap v302: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-11-27T15:21:27.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:26 smithi177 ceph-mon[131504]: pgmap v302: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-11-27T15:21:27.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:26 smithi174 ceph-mon[131110]: pgmap v302: 129 pgs: 129 active+clean; 635 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2023-11-27T15:21:28.903 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:28 smithi174 ceph-mon[131110]: pgmap v303: 129 pgs: 129 active+clean; 637 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-27T15:21:28.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.909+0000 7f2072b3e700 1 -- 172.21.15.133:0/2189015967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f206c100ee0 msgr2=0x7f206c104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:28.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.909+0000 7f2072b3e700 1 --2- 172.21.15.133:0/2189015967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f206c100ee0 0x7f206c104eb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f20580099c0 tx=0x7f2058009cd0 comp rx=0 tx=0).stop 2023-11-27T15:21:28.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.909+0000 7f2072b3e700 1 -- 172.21.15.133:0/2189015967 shutdown_connections 2023-11-27T15:21:28.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.909+0000 7f2072b3e700 1 --2- 172.21.15.133:0/2189015967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f206c1055e0 0x7f206c10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:28.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.909+0000 7f2072b3e700 1 --2- 172.21.15.133:0/2189015967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f206c100540 0x7f206c100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:28.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.909+0000 7f2072b3e700 1 --2- 172.21.15.133:0/2189015967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f206c100ee0 0x7f206c104eb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:28.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.909+0000 7f2072b3e700 1 -- 172.21.15.133:0/2189015967 >> 172.21.15.133:0/2189015967 conn(0x7f206c0fbdc0 msgr2=0x7f206c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:28.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.909+0000 7f2072b3e700 1 -- 172.21.15.133:0/2189015967 shutdown_connections 2023-11-27T15:21:28.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.909+0000 7f2072b3e700 1 -- 172.21.15.133:0/2189015967 wait complete. 2023-11-27T15:21:28.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.910+0000 7f2072b3e700 1 Processor -- start 2023-11-27T15:21:28.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.911+0000 7f2072b3e700 1 -- start start 2023-11-27T15:21:28.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.911+0000 7f2072b3e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f206c100540 0x7f206c072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:28.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.911+0000 7f2072b3e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f206c100ee0 0x7f206c073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:28.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.911+0000 7f2072b3e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f206c1055e0 0x7f206c0738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:28.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.911+0000 7f2072b3e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c1a4840 con 0x7f206c100540 2023-11-27T15:21:28.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.911+0000 7f2072b3e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c1a49b0 con 0x7f206c100ee0 2023-11-27T15:21:28.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.911+0000 7f2072b3e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c1a4b70 con 0x7f206c1055e0 2023-11-27T15:21:28.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.911+0000 7f20708da700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f206c100540 0x7f206c072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:28.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.911+0000 7f20708da700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f206c100540 0x7f206c072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60158/0 (socket says 172.21.15.133:60158) 2023-11-27T15:21:28.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.911+0000 7f20708da700 1 -- 172.21.15.133:0/1505444870 learned_addr learned my addr 172.21.15.133:0/1505444870 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:28.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.911+0000 7f20710db700 1 --2- 172.21.15.133:0/1505444870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f206c1055e0 0x7f206c0738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:28.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.911+0000 7f206bfff700 1 --2- 172.21.15.133:0/1505444870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f206c100ee0 0x7f206c073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:28.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.912+0000 7f20708da700 1 -- 172.21.15.133:0/1505444870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f206c1055e0 msgr2=0x7f206c0738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:28.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.912+0000 7f20708da700 1 --2- 172.21.15.133:0/1505444870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f206c1055e0 0x7f206c0738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:28.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.912+0000 7f20708da700 1 -- 172.21.15.133:0/1505444870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f206c100ee0 msgr2=0x7f206c073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:28.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.912+0000 7f20708da700 1 --2- 172.21.15.133:0/1505444870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f206c100ee0 0x7f206c073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:28.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.912+0000 7f20708da700 1 -- 172.21.15.133:0/1505444870 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20580095f0 con 0x7f206c100540 2023-11-27T15:21:28.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.912+0000 7f20710db700 1 --2- 172.21.15.133:0/1505444870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f206c1055e0 0x7f206c0738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:28.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.912+0000 7f20708da700 1 --2- 172.21.15.133:0/1505444870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f206c100540 0x7f206c072df0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f206000e9a0 tx=0x7f206000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:28.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.912+0000 7f2069ffb700 1 -- 172.21.15.133:0/1505444870 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2060009790 con 0x7f206c100540 2023-11-27T15:21:28.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.912+0000 7f2069ffb700 1 -- 172.21.15.133:0/1505444870 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2060010430 con 0x7f206c100540 2023-11-27T15:21:28.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.912+0000 7f2069ffb700 1 -- 172.21.15.133:0/1505444870 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2060018680 con 0x7f206c100540 2023-11-27T15:21:28.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.912+0000 7f2072b3e700 1 -- 172.21.15.133:0/1505444870 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206c110360 con 0x7f206c100540 2023-11-27T15:21:28.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.912+0000 7f2072b3e700 1 -- 172.21.15.133:0/1505444870 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206c1101d0 con 0x7f206c100540 2023-11-27T15:21:28.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.913+0000 7f2072b3e700 1 -- 172.21.15.133:0/1505444870 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f206c0fd1a0 con 0x7f206c100540 2023-11-27T15:21:28.920 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:28 smithi133 ceph-mon[97860]: pgmap v303: 129 pgs: 129 active+clean; 637 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-27T15:21:28.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.916+0000 7f2069ffb700 1 -- 172.21.15.133:0/1505444870 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f20600105a0 con 0x7f206c100540 2023-11-27T15:21:28.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.918+0000 7f2069ffb700 1 --2- 172.21.15.133:0/1505444870 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f205c0a1260 0x7f205c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:28.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.918+0000 7f2069ffb700 1 -- 172.21.15.133:0/1505444870 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2060014070 con 0x7f206c100540 2023-11-27T15:21:28.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.918+0000 7f206bfff700 1 --2- 172.21.15.133:0/1505444870 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f205c0a1260 0x7f205c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:28.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.919+0000 7f2069ffb700 1 --2- 172.21.15.133:0/1505444870 >> [v2:172.21.15.133:6802/3850463495,v1:172.21.15.133:6803/3850463495] conn(0x7f205c0ac270 0x7f205c0ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:28.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.919+0000 7f2069ffb700 1 -- 172.21.15.133:0/1505444870 --> [v2:172.21.15.133:6802/3850463495,v1:172.21.15.133:6803/3850463495] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f205c0aedd0 con 0x7f205c0ac270 2023-11-27T15:21:28.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.919+0000 7f2069ffb700 1 -- 172.21.15.133:0/1505444870 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20600cd5b0 con 0x7f206c100540 2023-11-27T15:21:28.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.919+0000 7f20710db700 1 --2- 172.21.15.133:0/1505444870 >> [v2:172.21.15.133:6802/3850463495,v1:172.21.15.133:6803/3850463495] conn(0x7f205c0ac270 0x7f205c0ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:28.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.919+0000 7f206bfff700 1 --2- 172.21.15.133:0/1505444870 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f205c0a1260 0x7f205c0a3710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f20580098e0 tx=0x7f2058000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:28.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.920+0000 7f20710db700 1 --2- 172.21.15.133:0/1505444870 >> [v2:172.21.15.133:6802/3850463495,v1:172.21.15.133:6803/3850463495] conn(0x7f205c0ac270 0x7f205c0ae700 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:28.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.920+0000 7f2069ffb700 1 -- 172.21.15.133:0/1505444870 <== osd.0 v2:172.21.15.133:6802/3850463495 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f205c0aedd0 con 0x7f205c0ac270 2023-11-27T15:21:28.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.943+0000 7f2072b3e700 1 -- 172.21.15.133:0/1505444870 --> [v2:172.21.15.133:6802/3850463495,v1:172.21.15.133:6803/3850463495] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f206c1104f0 con 0x7f205c0ac270 2023-11-27T15:21:28.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.944+0000 7f2069ffb700 1 -- 172.21.15.133:0/1505444870 <== osd.0 v2:172.21.15.133:6802/3850463495 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f206c1104f0 con 0x7f205c0ac270 2023-11-27T15:21:28.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.945+0000 7f2072b3e700 1 -- 172.21.15.133:0/1505444870 >> [v2:172.21.15.133:6802/3850463495,v1:172.21.15.133:6803/3850463495] conn(0x7f205c0ac270 msgr2=0x7f205c0ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:28.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.945+0000 7f2072b3e700 1 --2- 172.21.15.133:0/1505444870 >> [v2:172.21.15.133:6802/3850463495,v1:172.21.15.133:6803/3850463495] conn(0x7f205c0ac270 0x7f205c0ae700 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:28.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.945+0000 7f2072b3e700 1 -- 172.21.15.133:0/1505444870 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f205c0a1260 msgr2=0x7f205c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:28.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.945+0000 7f2072b3e700 1 --2- 172.21.15.133:0/1505444870 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f205c0a1260 0x7f205c0a3710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f20580098e0 tx=0x7f2058000bc0 comp rx=0 tx=0).stop 2023-11-27T15:21:28.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.945+0000 7f2072b3e700 1 -- 172.21.15.133:0/1505444870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f206c100540 msgr2=0x7f206c072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:28.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.945+0000 7f2072b3e700 1 --2- 172.21.15.133:0/1505444870 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f206c100540 0x7f206c072df0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f206000e9a0 tx=0x7f206000ecb0 comp rx=0 tx=0).stop 2023-11-27T15:21:28.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.945+0000 7f20710db700 1 -- 172.21.15.133:0/1505444870 reap_dead start 2023-11-27T15:21:28.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.945+0000 7f2072b3e700 1 -- 172.21.15.133:0/1505444870 shutdown_connections 2023-11-27T15:21:28.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.945+0000 7f2072b3e700 1 -- 172.21.15.133:0/1505444870 >> 172.21.15.133:0/1505444870 conn(0x7f206c0fbdc0 msgr2=0x7f206c0fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:28.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.945+0000 7f2072b3e700 1 -- 172.21.15.133:0/1505444870 shutdown_connections 2023-11-27T15:21:28.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.945+0000 7f2072b3e700 1 -- 172.21.15.133:0/1505444870 wait complete. 2023-11-27T15:21:29.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:28 smithi177 ceph-mon[131504]: pgmap v303: 129 pgs: 129 active+clean; 637 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-27T15:21:29.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.564+0000 7f6f5bc2d700 1 -- 172.21.15.133:0/4292383580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f54100ee0 msgr2=0x7f6f54104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:29.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.564+0000 7f6f5bc2d700 1 --2- 172.21.15.133:0/4292383580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f54100ee0 0x7f6f54104eb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6f440099c0 tx=0x7f6f44009cd0 comp rx=0 tx=0).stop 2023-11-27T15:21:29.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.564+0000 7f6f5bc2d700 1 -- 172.21.15.133:0/4292383580 shutdown_connections 2023-11-27T15:21:29.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.564+0000 7f6f5bc2d700 1 --2- 172.21.15.133:0/4292383580 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f541055e0 0x7f6f5410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:29.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.564+0000 7f6f5bc2d700 1 --2- 172.21.15.133:0/4292383580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f54100540 0x7f6f54100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:29.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.564+0000 7f6f5bc2d700 1 --2- 172.21.15.133:0/4292383580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f54100ee0 0x7f6f54104eb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:29.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.564+0000 7f6f5bc2d700 1 -- 172.21.15.133:0/4292383580 >> 172.21.15.133:0/4292383580 conn(0x7f6f540fbdc0 msgr2=0x7f6f540fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:29.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.565+0000 7f6f5bc2d700 1 -- 172.21.15.133:0/4292383580 shutdown_connections 2023-11-27T15:21:29.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.565+0000 7f6f5bc2d700 1 -- 172.21.15.133:0/4292383580 wait complete. 2023-11-27T15:21:29.572 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.566+0000 7f6f5bc2d700 1 Processor -- start 2023-11-27T15:21:29.572 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.566+0000 7f6f5bc2d700 1 -- start start 2023-11-27T15:21:29.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.567+0000 7f6f5bc2d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f54100540 0x7f6f541963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:29.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.567+0000 7f6f5bc2d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f54100ee0 0x7f6f541968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:29.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.567+0000 7f6f5bc2d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f541055e0 0x7f6f5419ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:29.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.567+0000 7f6f5bc2d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5419b360 con 0x7f6f54100ee0 2023-11-27T15:21:29.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.567+0000 7f6f5bc2d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5419b4d0 con 0x7f6f54100540 2023-11-27T15:21:29.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.567+0000 7f6f5bc2d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5419b690 con 0x7f6f541055e0 2023-11-27T15:21:29.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.567+0000 7f6f591c8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f54100ee0 0x7f6f541968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:29.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.567+0000 7f6f591c8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f54100ee0 0x7f6f541968e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60182/0 (socket says 172.21.15.133:60182) 2023-11-27T15:21:29.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.567+0000 7f6f591c8700 1 -- 172.21.15.133:0/1896885883 learned_addr learned my addr 172.21.15.133:0/1896885883 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:29.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.567+0000 7f6f599c9700 1 --2- 172.21.15.133:0/1896885883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f54100540 0x7f6f541963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:29.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.567+0000 7f6f5a1ca700 1 --2- 172.21.15.133:0/1896885883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f541055e0 0x7f6f5419ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:29.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.567+0000 7f6f591c8700 1 -- 172.21.15.133:0/1896885883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f541055e0 msgr2=0x7f6f5419ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:29.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.567+0000 7f6f591c8700 1 --2- 172.21.15.133:0/1896885883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f541055e0 0x7f6f5419ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:29.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.567+0000 7f6f591c8700 1 -- 172.21.15.133:0/1896885883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f54100540 msgr2=0x7f6f541963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:29.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.567+0000 7f6f591c8700 1 --2- 172.21.15.133:0/1896885883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f54100540 0x7f6f541963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:29.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.567+0000 7f6f591c8700 1 -- 172.21.15.133:0/1896885883 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f440095f0 con 0x7f6f54100ee0 2023-11-27T15:21:29.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.567+0000 7f6f5a1ca700 1 --2- 172.21.15.133:0/1896885883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6f541055e0 0x7f6f5419ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:21:29.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.568+0000 7f6f591c8700 1 --2- 172.21.15.133:0/1896885883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f54100ee0 0x7f6f541968e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6f44006010 tx=0x7f6f44005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:29.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.568+0000 7f6f4affd700 1 -- 172.21.15.133:0/1896885883 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4401d070 con 0x7f6f54100ee0 2023-11-27T15:21:29.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.568+0000 7f6f4affd700 1 -- 172.21.15.133:0/1896885883 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f44004050 con 0x7f6f54100ee0 2023-11-27T15:21:29.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.568+0000 7f6f5bc2d700 1 -- 172.21.15.133:0/1896885883 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f541103b0 con 0x7f6f54100ee0 2023-11-27T15:21:29.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.568+0000 7f6f4affd700 1 -- 172.21.15.133:0/1896885883 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44005290 con 0x7f6f54100ee0 2023-11-27T15:21:29.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.568+0000 7f6f5bc2d700 1 -- 172.21.15.133:0/1896885883 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f54110220 con 0x7f6f54100ee0 2023-11-27T15:21:29.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.569+0000 7f6f5bc2d700 1 -- 172.21.15.133:0/1896885883 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6f540fd150 con 0x7f6f54100ee0 2023-11-27T15:21:29.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.572+0000 7f6f4affd700 1 -- 172.21.15.133:0/1896885883 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f6f440047e0 con 0x7f6f54100ee0 2023-11-27T15:21:29.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.573+0000 7f6f4affd700 1 --2- 172.21.15.133:0/1896885883 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6f400a12a0 0x7f6f400a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:29.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.573+0000 7f6f4affd700 1 -- 172.21.15.133:0/1896885883 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6f440ce4e0 con 0x7f6f54100ee0 2023-11-27T15:21:29.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.573+0000 7f6f599c9700 1 --2- 172.21.15.133:0/1896885883 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6f400a12a0 0x7f6f400a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:29.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.573+0000 7f6f4affd700 1 --2- 172.21.15.133:0/1896885883 >> [v2:172.21.15.174:6824/880237758,v1:172.21.15.174:6825/880237758] conn(0x7f6f400ac2b0 0x7f6f400ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:29.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.574+0000 7f6f4affd700 1 -- 172.21.15.133:0/1896885883 --> [v2:172.21.15.174:6824/880237758,v1:172.21.15.174:6825/880237758] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f400aee10 con 0x7f6f400ac2b0 2023-11-27T15:21:29.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.574+0000 7f6f4affd700 1 -- 172.21.15.133:0/1896885883 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f440ce850 con 0x7f6f54100ee0 2023-11-27T15:21:29.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.574+0000 7f6f5a1ca700 1 --2- 172.21.15.133:0/1896885883 >> [v2:172.21.15.174:6824/880237758,v1:172.21.15.174:6825/880237758] conn(0x7f6f400ac2b0 0x7f6f400ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:29.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.574+0000 7f6f599c9700 1 --2- 172.21.15.133:0/1896885883 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6f400a12a0 0x7f6f400a3750 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6f50007be0 tx=0x7f6f500073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:29.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.574+0000 7f6f5a1ca700 1 --2- 172.21.15.133:0/1896885883 >> [v2:172.21.15.174:6824/880237758,v1:172.21.15.174:6825/880237758] conn(0x7f6f400ac2b0 0x7f6f400ae740 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:29.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.575+0000 7f6f4affd700 1 -- 172.21.15.133:0/1896885883 <== osd.10 v2:172.21.15.174:6824/880237758 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6f400aee10 con 0x7f6f400ac2b0 2023-11-27T15:21:29.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.591+0000 7f6f5bc2d700 1 -- 172.21.15.133:0/1896885883 --> [v2:172.21.15.174:6824/880237758,v1:172.21.15.174:6825/880237758] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6f54110540 con 0x7f6f400ac2b0 2023-11-27T15:21:29.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.592+0000 7f6f4affd700 1 -- 172.21.15.133:0/1896885883 <== osd.10 v2:172.21.15.174:6824/880237758 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6f54110540 con 0x7f6f400ac2b0 2023-11-27T15:21:29.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.592+0000 7f6f5bc2d700 1 -- 172.21.15.133:0/1896885883 >> [v2:172.21.15.174:6824/880237758,v1:172.21.15.174:6825/880237758] conn(0x7f6f400ac2b0 msgr2=0x7f6f400ae740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:29.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.592+0000 7f6f5bc2d700 1 --2- 172.21.15.133:0/1896885883 >> [v2:172.21.15.174:6824/880237758,v1:172.21.15.174:6825/880237758] conn(0x7f6f400ac2b0 0x7f6f400ae740 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:29.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.592+0000 7f6f5bc2d700 1 -- 172.21.15.133:0/1896885883 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6f400a12a0 msgr2=0x7f6f400a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:29.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.592+0000 7f6f5bc2d700 1 --2- 172.21.15.133:0/1896885883 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6f400a12a0 0x7f6f400a3750 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6f50007be0 tx=0x7f6f500073d0 comp rx=0 tx=0).stop 2023-11-27T15:21:29.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.592+0000 7f6f5bc2d700 1 -- 172.21.15.133:0/1896885883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f54100ee0 msgr2=0x7f6f541968e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:29.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.592+0000 7f6f5bc2d700 1 --2- 172.21.15.133:0/1896885883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6f54100ee0 0x7f6f541968e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6f44006010 tx=0x7f6f44005b00 comp rx=0 tx=0).stop 2023-11-27T15:21:29.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.593+0000 7f6f5a1ca700 1 -- 172.21.15.133:0/1896885883 reap_dead start 2023-11-27T15:21:29.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.593+0000 7f6f5bc2d700 1 -- 172.21.15.133:0/1896885883 shutdown_connections 2023-11-27T15:21:29.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.593+0000 7f6f5bc2d700 1 -- 172.21.15.133:0/1896885883 >> 172.21.15.133:0/1896885883 conn(0x7f6f540fbdc0 msgr2=0x7f6f540fc780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:29.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.593+0000 7f6f5bc2d700 1 -- 172.21.15.133:0/1896885883 shutdown_connections 2023-11-27T15:21:29.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.593+0000 7f6f5bc2d700 1 -- 172.21.15.133:0/1896885883 wait complete. 2023-11-27T15:21:29.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.708+0000 7fbad5844700 1 -- 172.21.15.133:0/2026654235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad0102f50 msgr2=0x7fbad0106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:29.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.708+0000 7fbad5844700 1 --2- 172.21.15.133:0/2026654235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad0102f50 0x7fbad0106f20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbabc0099c0 tx=0x7fbabc009cd0 comp rx=0 tx=0).stop 2023-11-27T15:21:29.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.708+0000 7fbad5844700 1 -- 172.21.15.133:0/2026654235 shutdown_connections 2023-11-27T15:21:29.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.708+0000 7fbad5844700 1 --2- 172.21.15.133:0/2026654235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbad0107650 0x7fbad010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:29.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.708+0000 7fbad5844700 1 --2- 172.21.15.133:0/2026654235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbad01025b0 0x7fbad0102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:29.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.708+0000 7fbad5844700 1 --2- 172.21.15.133:0/2026654235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad0102f50 0x7fbad0106f20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:29.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.708+0000 7fbad5844700 1 -- 172.21.15.133:0/2026654235 >> 172.21.15.133:0/2026654235 conn(0x7fbad00fde50 msgr2=0x7fbad0100260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:29.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.708+0000 7fbad5844700 1 -- 172.21.15.133:0/2026654235 shutdown_connections 2023-11-27T15:21:29.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.708+0000 7fbad5844700 1 -- 172.21.15.133:0/2026654235 wait complete. 2023-11-27T15:21:29.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.709+0000 7fbad5844700 1 Processor -- start 2023-11-27T15:21:29.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.709+0000 7fbad5844700 1 -- start start 2023-11-27T15:21:29.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.710+0000 7fbad5844700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad01025b0 0x7fbad0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:29.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.710+0000 7fbad5844700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbad0102f50 0x7fbad0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:29.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.710+0000 7fbad5844700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbad0107650 0x7fbad019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:29.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.710+0000 7fbad5844700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad019d400 con 0x7fbad01025b0 2023-11-27T15:21:29.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.710+0000 7fbad5844700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad019d570 con 0x7fbad0107650 2023-11-27T15:21:29.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.710+0000 7fbad5844700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad019d730 con 0x7fbad0102f50 2023-11-27T15:21:29.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.710+0000 7fbad4842700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad01025b0 0x7fbad0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:29.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.710+0000 7fbad4842700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad01025b0 0x7fbad0198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60192/0 (socket says 172.21.15.133:60192) 2023-11-27T15:21:29.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.710+0000 7fbad4842700 1 -- 172.21.15.133:0/4087617102 learned_addr learned my addr 172.21.15.133:0/4087617102 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:29.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.710+0000 7fbad5043700 1 --2- 172.21.15.133:0/4087617102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbad0107650 0x7fbad019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:29.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.711+0000 7fbacffff700 1 --2- 172.21.15.133:0/4087617102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbad0102f50 0x7fbad0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:29.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.711+0000 7fbad4842700 1 -- 172.21.15.133:0/4087617102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbad0102f50 msgr2=0x7fbad0198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:29.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.711+0000 7fbad4842700 1 --2- 172.21.15.133:0/4087617102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbad0102f50 0x7fbad0198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:29.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.711+0000 7fbad4842700 1 -- 172.21.15.133:0/4087617102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbad0107650 msgr2=0x7fbad019cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:29.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.711+0000 7fbad4842700 1 --2- 172.21.15.133:0/4087617102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbad0107650 0x7fbad019cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:29.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.711+0000 7fbad4842700 1 -- 172.21.15.133:0/4087617102 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbabc0095f0 con 0x7fbad01025b0 2023-11-27T15:21:29.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.711+0000 7fbacffff700 1 --2- 172.21.15.133:0/4087617102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbad0102f50 0x7fbad0198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:21:29.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.711+0000 7fbad4842700 1 --2- 172.21.15.133:0/4087617102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad01025b0 0x7fbad0198440 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbac400e9a0 tx=0x7fbac400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:29.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.711+0000 7fbacdffb700 1 -- 172.21.15.133:0/4087617102 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac4009790 con 0x7fbad01025b0 2023-11-27T15:21:29.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.711+0000 7fbacdffb700 1 -- 172.21.15.133:0/4087617102 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbac400cbd0 con 0x7fbad01025b0 2023-11-27T15:21:29.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.711+0000 7fbacdffb700 1 -- 172.21.15.133:0/4087617102 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac4018680 con 0x7fbad01025b0 2023-11-27T15:21:29.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.711+0000 7fbad5844700 1 -- 172.21.15.133:0/4087617102 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad01123d0 con 0x7fbad01025b0 2023-11-27T15:21:29.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.711+0000 7fbad5844700 1 -- 172.21.15.133:0/4087617102 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad0112240 con 0x7fbad01025b0 2023-11-27T15:21:29.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.712+0000 7fbad5844700 1 -- 172.21.15.133:0/4087617102 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbad00ff270 con 0x7fbad01025b0 2023-11-27T15:21:29.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.716+0000 7fbacdffb700 1 -- 172.21.15.133:0/4087617102 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fbac400cd40 con 0x7fbad01025b0 2023-11-27T15:21:29.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.717+0000 7fbacdffb700 1 --2- 172.21.15.133:0/4087617102 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbac00a1260 0x7fbac00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:29.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.717+0000 7fbacdffb700 1 -- 172.21.15.133:0/4087617102 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbac4014070 con 0x7fbad01025b0 2023-11-27T15:21:29.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.718+0000 7fbacffff700 1 --2- 172.21.15.133:0/4087617102 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbac00a1260 0x7fbac00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:29.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.718+0000 7fbacdffb700 1 --2- 172.21.15.133:0/4087617102 >> [v2:172.21.15.177:6800/2043769283,v1:172.21.15.177:6801/2043769283] conn(0x7fbac00ac270 0x7fbac00ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:29.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.718+0000 7fbacdffb700 1 -- 172.21.15.133:0/4087617102 --> [v2:172.21.15.177:6800/2043769283,v1:172.21.15.177:6801/2043769283] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbac00aedd0 con 0x7fbac00ac270 2023-11-27T15:21:29.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.718+0000 7fbacdffb700 1 -- 172.21.15.133:0/4087617102 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbac40cd730 con 0x7fbad01025b0 2023-11-27T15:21:29.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.718+0000 7fbacffff700 1 --2- 172.21.15.133:0/4087617102 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbac00a1260 0x7fbac00a3710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbabc006010 tx=0x7fbabc003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:29.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.718+0000 7fbad5043700 1 --2- 172.21.15.133:0/4087617102 >> [v2:172.21.15.177:6800/2043769283,v1:172.21.15.177:6801/2043769283] conn(0x7fbac00ac270 0x7fbac00ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:29.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.719+0000 7fbad5043700 1 --2- 172.21.15.133:0/4087617102 >> [v2:172.21.15.177:6800/2043769283,v1:172.21.15.177:6801/2043769283] conn(0x7fbac00ac270 0x7fbac00ae700 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:29.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.720+0000 7fbacdffb700 1 -- 172.21.15.133:0/4087617102 <== osd.2 v2:172.21.15.177:6800/2043769283 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbac00aedd0 con 0x7fbac00ac270 2023-11-27T15:21:29.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.737+0000 7fbad5844700 1 -- 172.21.15.133:0/4087617102 --> [v2:172.21.15.177:6800/2043769283,v1:172.21.15.177:6801/2043769283] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbad0112560 con 0x7fbac00ac270 2023-11-27T15:21:29.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.737+0000 7fbacdffb700 1 -- 172.21.15.133:0/4087617102 <== osd.2 v2:172.21.15.177:6800/2043769283 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbad0112560 con 0x7fbac00ac270 2023-11-27T15:21:29.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.738+0000 7fbad5844700 1 -- 172.21.15.133:0/4087617102 >> [v2:172.21.15.177:6800/2043769283,v1:172.21.15.177:6801/2043769283] conn(0x7fbac00ac270 msgr2=0x7fbac00ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:29.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.738+0000 7fbad5844700 1 --2- 172.21.15.133:0/4087617102 >> [v2:172.21.15.177:6800/2043769283,v1:172.21.15.177:6801/2043769283] conn(0x7fbac00ac270 0x7fbac00ae700 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:29.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.738+0000 7fbad5844700 1 -- 172.21.15.133:0/4087617102 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbac00a1260 msgr2=0x7fbac00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:29.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.738+0000 7fbad5844700 1 --2- 172.21.15.133:0/4087617102 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbac00a1260 0x7fbac00a3710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbabc006010 tx=0x7fbabc003680 comp rx=0 tx=0).stop 2023-11-27T15:21:29.763 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.738+0000 7fbad5844700 1 -- 172.21.15.133:0/4087617102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad01025b0 msgr2=0x7fbad0198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:29.763 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.738+0000 7fbad5844700 1 --2- 172.21.15.133:0/4087617102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbad01025b0 0x7fbad0198440 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbac400e9a0 tx=0x7fbac400ecb0 comp rx=0 tx=0).stop 2023-11-27T15:21:29.763 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.738+0000 7fbad5043700 1 -- 172.21.15.133:0/4087617102 reap_dead start 2023-11-27T15:21:29.763 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.738+0000 7fbad5844700 1 -- 172.21.15.133:0/4087617102 shutdown_connections 2023-11-27T15:21:29.763 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.738+0000 7fbad5844700 1 -- 172.21.15.133:0/4087617102 >> 172.21.15.133:0/4087617102 conn(0x7fbad00fde50 msgr2=0x7fbad00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:29.763 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.738+0000 7fbad5844700 1 -- 172.21.15.133:0/4087617102 shutdown_connections 2023-11-27T15:21:29.763 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:29.738+0000 7fbad5844700 1 -- 172.21.15.133:0/4087617102 wait complete. 2023-11-27T15:21:30.807 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:30 smithi133 ceph-mon[97860]: pgmap v304: 129 pgs: 129 active+clean; 637 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 6 op/s 2023-11-27T15:21:30.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.862+0000 7f0bb1ccd700 1 -- 172.21.15.133:0/203734942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bac100ee0 msgr2=0x7f0bac104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:30.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.862+0000 7f0bb1ccd700 1 --2- 172.21.15.133:0/203734942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bac100ee0 0x7f0bac104eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0009990 tx=0x7f0ba0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:21:30.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.862+0000 7f0bb1ccd700 1 -- 172.21.15.133:0/203734942 shutdown_connections 2023-11-27T15:21:30.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.862+0000 7f0bb1ccd700 1 --2- 172.21.15.133:0/203734942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bac1055e0 0x7f0bac10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:30.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.862+0000 7f0bb1ccd700 1 --2- 172.21.15.133:0/203734942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bac100540 0x7f0bac100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:30.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.862+0000 7f0bb1ccd700 1 --2- 172.21.15.133:0/203734942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bac100ee0 0x7f0bac104eb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:30.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.862+0000 7f0bb1ccd700 1 -- 172.21.15.133:0/203734942 >> 172.21.15.133:0/203734942 conn(0x7f0bac0fbdc0 msgr2=0x7f0bac0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:30.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.862+0000 7f0bb1ccd700 1 -- 172.21.15.133:0/203734942 shutdown_connections 2023-11-27T15:21:30.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.862+0000 7f0bb1ccd700 1 -- 172.21.15.133:0/203734942 wait complete. 2023-11-27T15:21:30.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.864+0000 7f0bb1ccd700 1 Processor -- start 2023-11-27T15:21:30.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.864+0000 7f0bb1ccd700 1 -- start start 2023-11-27T15:21:30.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.864+0000 7f0bb1ccd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bac100540 0x7f0bac072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:30.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.864+0000 7f0bb1ccd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bac100ee0 0x7f0bac073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:30.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.864+0000 7f0bb1ccd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bac1055e0 0x7f0bac0738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:30.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.864+0000 7f0bb1ccd700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac1a4850 con 0x7f0bac100540 2023-11-27T15:21:30.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.864+0000 7f0bb1ccd700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac1a49c0 con 0x7f0bac1055e0 2023-11-27T15:21:30.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.864+0000 7f0bb1ccd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac1a4b80 con 0x7f0bac100ee0 2023-11-27T15:21:30.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.865+0000 7f0bab7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bac100540 0x7f0bac072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:30.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.865+0000 7f0bab7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bac100540 0x7f0bac072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60204/0 (socket says 172.21.15.133:60204) 2023-11-27T15:21:30.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.865+0000 7f0bab7fe700 1 -- 172.21.15.133:0/2106865639 learned_addr learned my addr 172.21.15.133:0/2106865639 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:30.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.865+0000 7f0babfff700 1 --2- 172.21.15.133:0/2106865639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bac1055e0 0x7f0bac0738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:30.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.865+0000 7f0baaffd700 1 --2- 172.21.15.133:0/2106865639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bac100ee0 0x7f0bac073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:30.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.865+0000 7f0bab7fe700 1 -- 172.21.15.133:0/2106865639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bac100ee0 msgr2=0x7f0bac073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:30.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.865+0000 7f0bab7fe700 1 --2- 172.21.15.133:0/2106865639 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0bac100ee0 0x7f0bac073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:30.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.865+0000 7f0bab7fe700 1 -- 172.21.15.133:0/2106865639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bac1055e0 msgr2=0x7f0bac0738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:30.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.865+0000 7f0bab7fe700 1 --2- 172.21.15.133:0/2106865639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bac1055e0 0x7f0bac0738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:30.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.865+0000 7f0bab7fe700 1 -- 172.21.15.133:0/2106865639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba00095f0 con 0x7f0bac100540 2023-11-27T15:21:30.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.865+0000 7f0bab7fe700 1 --2- 172.21.15.133:0/2106865639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bac100540 0x7f0bac072df0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00e9a0 tx=0x7f0b9c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:30.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.866+0000 7f0ba8ff9700 1 -- 172.21.15.133:0/2106865639 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c009790 con 0x7f0bac100540 2023-11-27T15:21:30.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.866+0000 7f0bb1ccd700 1 -- 172.21.15.133:0/2106865639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bac110360 con 0x7f0bac100540 2023-11-27T15:21:30.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.866+0000 7f0ba8ff9700 1 -- 172.21.15.133:0/2106865639 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b9c010430 con 0x7f0bac100540 2023-11-27T15:21:30.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.866+0000 7f0ba8ff9700 1 -- 172.21.15.133:0/2106865639 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c018680 con 0x7f0bac100540 2023-11-27T15:21:30.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.866+0000 7f0bb1ccd700 1 -- 172.21.15.133:0/2106865639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bac1101d0 con 0x7f0bac100540 2023-11-27T15:21:30.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.867+0000 7f0bb1ccd700 1 -- 172.21.15.133:0/2106865639 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0bac0fd210 con 0x7f0bac100540 2023-11-27T15:21:30.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.869+0000 7f0ba8ff9700 1 -- 172.21.15.133:0/2106865639 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f0b9c0105a0 con 0x7f0bac100540 2023-11-27T15:21:30.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.870+0000 7f0ba8ff9700 1 --2- 172.21.15.133:0/2106865639 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0b940a11e0 0x7f0b940a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:30.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.870+0000 7f0ba8ff9700 1 -- 172.21.15.133:0/2106865639 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0b9c014070 con 0x7f0bac100540 2023-11-27T15:21:30.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.870+0000 7f0baaffd700 1 --2- 172.21.15.133:0/2106865639 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0b940a11e0 0x7f0b940a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:30.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.871+0000 7f0ba8ff9700 1 --2- 172.21.15.133:0/2106865639 >> [v2:172.21.15.177:6824/1870203795,v1:172.21.15.177:6825/1870203795] conn(0x7f0b940ac1f0 0x7f0b940ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:30.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.871+0000 7f0ba8ff9700 1 -- 172.21.15.133:0/2106865639 --> [v2:172.21.15.177:6824/1870203795,v1:172.21.15.177:6825/1870203795] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b940aed50 con 0x7f0b940ac1f0 2023-11-27T15:21:30.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.871+0000 7f0ba8ff9700 1 -- 172.21.15.133:0/2106865639 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b9c0cd610 con 0x7f0bac100540 2023-11-27T15:21:30.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.871+0000 7f0babfff700 1 --2- 172.21.15.133:0/2106865639 >> [v2:172.21.15.177:6824/1870203795,v1:172.21.15.177:6825/1870203795] conn(0x7f0b940ac1f0 0x7f0b940ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:30.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.871+0000 7f0baaffd700 1 --2- 172.21.15.133:0/2106865639 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0b940a11e0 0x7f0b940a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0bac0745e0 tx=0x7f0ba0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:30.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.872+0000 7f0babfff700 1 --2- 172.21.15.133:0/2106865639 >> [v2:172.21.15.177:6824/1870203795,v1:172.21.15.177:6825/1870203795] conn(0x7f0b940ac1f0 0x7f0b940ae680 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:30.878 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.872+0000 7f0ba8ff9700 1 -- 172.21.15.133:0/2106865639 <== osd.11 v2:172.21.15.177:6824/1870203795 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0b940aed50 con 0x7f0b940ac1f0 2023-11-27T15:21:30.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.889+0000 7f0bb1ccd700 1 -- 172.21.15.133:0/2106865639 --> [v2:172.21.15.177:6824/1870203795,v1:172.21.15.177:6825/1870203795] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0bac1104f0 con 0x7f0b940ac1f0 2023-11-27T15:21:30.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.889+0000 7f0ba8ff9700 1 -- 172.21.15.133:0/2106865639 <== osd.11 v2:172.21.15.177:6824/1870203795 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0bac1104f0 con 0x7f0b940ac1f0 2023-11-27T15:21:30.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.890+0000 7f0bb1ccd700 1 -- 172.21.15.133:0/2106865639 >> [v2:172.21.15.177:6824/1870203795,v1:172.21.15.177:6825/1870203795] conn(0x7f0b940ac1f0 msgr2=0x7f0b940ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:30.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.890+0000 7f0bb1ccd700 1 --2- 172.21.15.133:0/2106865639 >> [v2:172.21.15.177:6824/1870203795,v1:172.21.15.177:6825/1870203795] conn(0x7f0b940ac1f0 0x7f0b940ae680 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:30.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.890+0000 7f0bb1ccd700 1 -- 172.21.15.133:0/2106865639 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0b940a11e0 msgr2=0x7f0b940a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:30.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.890+0000 7f0bb1ccd700 1 --2- 172.21.15.133:0/2106865639 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0b940a11e0 0x7f0b940a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0bac0745e0 tx=0x7f0ba0005fd0 comp rx=0 tx=0).stop 2023-11-27T15:21:30.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.890+0000 7f0bb1ccd700 1 -- 172.21.15.133:0/2106865639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bac100540 msgr2=0x7f0bac072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:30.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.890+0000 7f0bb1ccd700 1 --2- 172.21.15.133:0/2106865639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0bac100540 0x7f0bac072df0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00e9a0 tx=0x7f0b9c00ecb0 comp rx=0 tx=0).stop 2023-11-27T15:21:30.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.890+0000 7f0babfff700 1 -- 172.21.15.133:0/2106865639 reap_dead start 2023-11-27T15:21:30.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.890+0000 7f0bb1ccd700 1 -- 172.21.15.133:0/2106865639 shutdown_connections 2023-11-27T15:21:30.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.890+0000 7f0bb1ccd700 1 -- 172.21.15.133:0/2106865639 >> 172.21.15.133:0/2106865639 conn(0x7f0bac0fbdc0 msgr2=0x7f0bac0fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:30.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.890+0000 7f0bb1ccd700 1 -- 172.21.15.133:0/2106865639 shutdown_connections 2023-11-27T15:21:30.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:30.890+0000 7f0bb1ccd700 1 -- 172.21.15.133:0/2106865639 wait complete. 2023-11-27T15:21:31.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.582+0000 7fbe32fb4700 1 -- 172.21.15.133:0/111916762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe2c10a890 msgr2=0x7fbe2c10ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:31.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.582+0000 7fbe32fb4700 1 --2- 172.21.15.133:0/111916762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe2c10a890 0x7fbe2c10ccd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400b0a0 tx=0x7fbe2400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:21:31.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.582+0000 7fbe32fb4700 1 -- 172.21.15.133:0/111916762 shutdown_connections 2023-11-27T15:21:31.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.582+0000 7fbe32fb4700 1 --2- 172.21.15.133:0/111916762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe2c10a890 0x7fbe2c10ccd0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:31.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.582+0000 7fbe32fb4700 1 --2- 172.21.15.133:0/111916762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe2c101540 0x7fbe2c101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:31.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.582+0000 7fbe32fb4700 1 --2- 172.21.15.133:0/111916762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe2c101e50 0x7fbe2c10a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:31.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.582+0000 7fbe32fb4700 1 -- 172.21.15.133:0/111916762 >> 172.21.15.133:0/111916762 conn(0x7fbe2c0770a0 msgr2=0x7fbe2c0774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:31.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.582+0000 7fbe32fb4700 1 -- 172.21.15.133:0/111916762 shutdown_connections 2023-11-27T15:21:31.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.583+0000 7fbe32fb4700 1 -- 172.21.15.133:0/111916762 wait complete. 2023-11-27T15:21:31.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.583+0000 7fbe32fb4700 1 Processor -- start 2023-11-27T15:21:31.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.584+0000 7fbe32fb4700 1 -- start start 2023-11-27T15:21:31.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.584+0000 7fbe32fb4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe2c101540 0x7fbe2c1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:31.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.584+0000 7fbe32fb4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe2c101e50 0x7fbe2c196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:31.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.584+0000 7fbe32fb4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe2c10a890 0x7fbe2c19ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:31.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.584+0000 7fbe32fb4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c19b490 con 0x7fbe2c101e50 2023-11-27T15:21:31.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.584+0000 7fbe32fb4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c19b600 con 0x7fbe2c101540 2023-11-27T15:21:31.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.584+0000 7fbe32fb4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c19b7c0 con 0x7fbe2c10a890 2023-11-27T15:21:31.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.584+0000 7fbe2bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe2c101e50 0x7fbe2c196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:31.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.584+0000 7fbe2bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe2c101e50 0x7fbe2c196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60148/0 (socket says 172.21.15.133:60148) 2023-11-27T15:21:31.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.584+0000 7fbe2bfff700 1 -- 172.21.15.133:0/2597568201 learned_addr learned my addr 172.21.15.133:0/2597568201 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:31.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.584+0000 7fbe30d50700 1 --2- 172.21.15.133:0/2597568201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe2c101540 0x7fbe2c1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:31.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.584+0000 7fbe31551700 1 --2- 172.21.15.133:0/2597568201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe2c10a890 0x7fbe2c19ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:31.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.585+0000 7fbe2bfff700 1 -- 172.21.15.133:0/2597568201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe2c10a890 msgr2=0x7fbe2c19ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:31.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.585+0000 7fbe2bfff700 1 --2- 172.21.15.133:0/2597568201 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbe2c10a890 0x7fbe2c19ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:31.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.585+0000 7fbe2bfff700 1 -- 172.21.15.133:0/2597568201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe2c101540 msgr2=0x7fbe2c1964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:31.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.585+0000 7fbe2bfff700 1 --2- 172.21.15.133:0/2597568201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe2c101540 0x7fbe2c1964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:31.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.585+0000 7fbe2bfff700 1 -- 172.21.15.133:0/2597568201 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe24009d00 con 0x7fbe2c101e50 2023-11-27T15:21:31.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.585+0000 7fbe2bfff700 1 --2- 172.21.15.133:0/2597568201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe2c101e50 0x7fbe2c196a10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800e9a0 tx=0x7fbe1800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:31.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.585+0000 7fbe29ffb700 1 -- 172.21.15.133:0/2597568201 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe18009790 con 0x7fbe2c101e50 2023-11-27T15:21:31.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.585+0000 7fbe32fb4700 1 -- 172.21.15.133:0/2597568201 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe2c110300 con 0x7fbe2c101e50 2023-11-27T15:21:31.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.585+0000 7fbe29ffb700 1 -- 172.21.15.133:0/2597568201 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe18010430 con 0x7fbe2c101e50 2023-11-27T15:21:31.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.585+0000 7fbe29ffb700 1 -- 172.21.15.133:0/2597568201 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe18018680 con 0x7fbe2c101e50 2023-11-27T15:21:31.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.585+0000 7fbe32fb4700 1 -- 172.21.15.133:0/2597568201 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe2c110170 con 0x7fbe2c101e50 2023-11-27T15:21:31.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.586+0000 7fbe32fb4700 1 -- 172.21.15.133:0/2597568201 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbe2c0fc760 con 0x7fbe2c101e50 2023-11-27T15:21:31.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.588+0000 7fbe29ffb700 1 -- 172.21.15.133:0/2597568201 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fbe180105a0 con 0x7fbe2c101e50 2023-11-27T15:21:31.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.589+0000 7fbe29ffb700 1 --2- 172.21.15.133:0/2597568201 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbe1c0a11e0 0x7fbe1c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:31.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.589+0000 7fbe29ffb700 1 -- 172.21.15.133:0/2597568201 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbe18014070 con 0x7fbe2c101e50 2023-11-27T15:21:31.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.590+0000 7fbe30d50700 1 --2- 172.21.15.133:0/2597568201 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbe1c0a11e0 0x7fbe1c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:31.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.590+0000 7fbe29ffb700 1 --2- 172.21.15.133:0/2597568201 >> [v2:172.21.15.174:6808/959536963,v1:172.21.15.174:6809/959536963] conn(0x7fbe1c0ac1f0 0x7fbe1c0ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:31.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.590+0000 7fbe29ffb700 1 -- 172.21.15.133:0/2597568201 --> [v2:172.21.15.174:6808/959536963,v1:172.21.15.174:6809/959536963] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe1c0aed50 con 0x7fbe1c0ac1f0 2023-11-27T15:21:31.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.590+0000 7fbe29ffb700 1 -- 172.21.15.133:0/2597568201 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe180cd600 con 0x7fbe2c101e50 2023-11-27T15:21:31.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.590+0000 7fbe30d50700 1 --2- 172.21.15.133:0/2597568201 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbe1c0a11e0 0x7fbe1c0a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009c60 tx=0x7fbe20009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:31.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.590+0000 7fbe31551700 1 --2- 172.21.15.133:0/2597568201 >> [v2:172.21.15.174:6808/959536963,v1:172.21.15.174:6809/959536963] conn(0x7fbe1c0ac1f0 0x7fbe1c0ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:31.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.591+0000 7fbe31551700 1 --2- 172.21.15.133:0/2597568201 >> [v2:172.21.15.174:6808/959536963,v1:172.21.15.174:6809/959536963] conn(0x7fbe1c0ac1f0 0x7fbe1c0ae680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:31.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.592+0000 7fbe29ffb700 1 -- 172.21.15.133:0/2597568201 <== osd.4 v2:172.21.15.174:6808/959536963 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbe1c0aed50 con 0x7fbe1c0ac1f0 2023-11-27T15:21:31.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.608+0000 7fbe32fb4700 1 -- 172.21.15.133:0/2597568201 --> [v2:172.21.15.174:6808/959536963,v1:172.21.15.174:6809/959536963] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbe2c110490 con 0x7fbe1c0ac1f0 2023-11-27T15:21:31.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.609+0000 7fbe29ffb700 1 -- 172.21.15.133:0/2597568201 <== osd.4 v2:172.21.15.174:6808/959536963 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbe2c110490 con 0x7fbe1c0ac1f0 2023-11-27T15:21:31.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.609+0000 7fbe32fb4700 1 -- 172.21.15.133:0/2597568201 >> [v2:172.21.15.174:6808/959536963,v1:172.21.15.174:6809/959536963] conn(0x7fbe1c0ac1f0 msgr2=0x7fbe1c0ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:31.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.609+0000 7fbe32fb4700 1 --2- 172.21.15.133:0/2597568201 >> [v2:172.21.15.174:6808/959536963,v1:172.21.15.174:6809/959536963] conn(0x7fbe1c0ac1f0 0x7fbe1c0ae680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:31.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.609+0000 7fbe32fb4700 1 -- 172.21.15.133:0/2597568201 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbe1c0a11e0 msgr2=0x7fbe1c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:31.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.609+0000 7fbe32fb4700 1 --2- 172.21.15.133:0/2597568201 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbe1c0a11e0 0x7fbe1c0a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009c60 tx=0x7fbe20009260 comp rx=0 tx=0).stop 2023-11-27T15:21:31.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.609+0000 7fbe32fb4700 1 -- 172.21.15.133:0/2597568201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe2c101e50 msgr2=0x7fbe2c196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:31.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.609+0000 7fbe32fb4700 1 --2- 172.21.15.133:0/2597568201 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe2c101e50 0x7fbe2c196a10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbe1800e9a0 tx=0x7fbe1800ecb0 comp rx=0 tx=0).stop 2023-11-27T15:21:31.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.609+0000 7fbe31551700 1 -- 172.21.15.133:0/2597568201 reap_dead start 2023-11-27T15:21:31.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.609+0000 7fbe32fb4700 1 -- 172.21.15.133:0/2597568201 shutdown_connections 2023-11-27T15:21:31.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.609+0000 7fbe32fb4700 1 -- 172.21.15.133:0/2597568201 >> 172.21.15.133:0/2597568201 conn(0x7fbe2c0770a0 msgr2=0x7fbe2c0fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:31.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.609+0000 7fbe32fb4700 1 -- 172.21.15.133:0/2597568201 shutdown_connections 2023-11-27T15:21:31.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:28.609+0000 7fbe32fb4700 1 -- 172.21.15.133:0/2597568201 wait complete. 2023-11-27T15:21:31.094 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:30 smithi177 ceph-mon[131504]: pgmap v304: 129 pgs: 129 active+clean; 637 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 6 op/s 2023-11-27T15:21:31.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:30 smithi174 ceph-mon[131110]: pgmap v304: 129 pgs: 129 active+clean; 637 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 853 B/s wr, 6 op/s 2023-11-27T15:21:31.865 INFO:teuthology.orchestra.run.smithi133.stdout:34359738441 2023-11-27T15:21:31.866 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.0 2023-11-27T15:21:31.956 INFO:teuthology.orchestra.run.smithi133.stdout:77309411392 2023-11-27T15:21:31.956 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.2 2023-11-27T15:21:32.547 INFO:teuthology.orchestra.run.smithi133.stdout:304942678043 2023-11-27T15:21:32.547 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.10 2023-11-27T15:21:32.794 INFO:teuthology.orchestra.run.smithi133.stdout:339302416408 2023-11-27T15:21:32.795 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.11 2023-11-27T15:21:33.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:32 smithi133 ceph-mon[97860]: pgmap v305: 129 pgs: 129 active+clean; 637 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 853 B/s wr, 7 op/s 2023-11-27T15:21:33.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:32 smithi177 ceph-mon[131504]: pgmap v305: 129 pgs: 129 active+clean; 637 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 853 B/s wr, 7 op/s 2023-11-27T15:21:33.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.094+0000 7f01a763e700 1 -- 172.21.15.133:0/2902427253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01a0100540 msgr2=0x7f01a0100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:33.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.094+0000 7f01a763e700 1 --2- 172.21.15.133:0/2902427253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01a0100540 0x7f01a0100910 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f019c0099c0 tx=0x7f019c009cd0 comp rx=0 tx=0).stop 2023-11-27T15:21:33.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.094+0000 7f01a763e700 1 -- 172.21.15.133:0/2902427253 shutdown_connections 2023-11-27T15:21:33.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.094+0000 7f01a763e700 1 --2- 172.21.15.133:0/2902427253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01a01055e0 0x7f01a010d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:33.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.094+0000 7f01a763e700 1 --2- 172.21.15.133:0/2902427253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01a0100540 0x7f01a0100910 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:33.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.094+0000 7f01a763e700 1 --2- 172.21.15.133:0/2902427253 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01a0100ee0 0x7f01a0104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:33.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.094+0000 7f01a763e700 1 -- 172.21.15.133:0/2902427253 >> 172.21.15.133:0/2902427253 conn(0x7f01a00fbdc0 msgr2=0x7f01a00fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:33.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.094+0000 7f01a763e700 1 -- 172.21.15.133:0/2902427253 shutdown_connections 2023-11-27T15:21:33.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.094+0000 7f01a763e700 1 -- 172.21.15.133:0/2902427253 wait complete. 2023-11-27T15:21:33.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.096+0000 7f01a763e700 1 Processor -- start 2023-11-27T15:21:33.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.096+0000 7f01a763e700 1 -- start start 2023-11-27T15:21:33.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.097+0000 7f01a763e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01a0100540 0x7f01a0072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:33.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.097+0000 7f01a763e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01a0100ee0 0x7f01a0073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:33.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.097+0000 7f01a763e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01a01055e0 0x7f01a00738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:33.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.097+0000 7f01a763e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a01a4860 con 0x7f01a0100540 2023-11-27T15:21:33.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.097+0000 7f01a763e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a01a49d0 con 0x7f01a0100ee0 2023-11-27T15:21:33.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.097+0000 7f01a763e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a01a4b90 con 0x7f01a01055e0 2023-11-27T15:21:33.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.097+0000 7f01a53da700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01a0100540 0x7f01a0072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:33.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.097+0000 7f01a53da700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01a0100540 0x7f01a0072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57758/0 (socket says 172.21.15.133:57758) 2023-11-27T15:21:33.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.097+0000 7f01a53da700 1 -- 172.21.15.133:0/2961578762 learned_addr learned my addr 172.21.15.133:0/2961578762 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:33.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.097+0000 7f01a5bdb700 1 --2- 172.21.15.133:0/2961578762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01a01055e0 0x7f01a00738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:33.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.098+0000 7f01a4bd9700 1 --2- 172.21.15.133:0/2961578762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01a0100ee0 0x7f01a0073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:33.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.098+0000 7f01a53da700 1 -- 172.21.15.133:0/2961578762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01a01055e0 msgr2=0x7f01a00738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:33.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.098+0000 7f01a53da700 1 --2- 172.21.15.133:0/2961578762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01a01055e0 0x7f01a00738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:33.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.098+0000 7f01a53da700 1 -- 172.21.15.133:0/2961578762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01a0100ee0 msgr2=0x7f01a0073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:33.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.098+0000 7f01a53da700 1 --2- 172.21.15.133:0/2961578762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01a0100ee0 0x7f01a0073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:33.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.098+0000 7f01a53da700 1 -- 172.21.15.133:0/2961578762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0190009520 con 0x7f01a0100540 2023-11-27T15:21:33.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.098+0000 7f01a5bdb700 1 --2- 172.21.15.133:0/2961578762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01a01055e0 0x7f01a00738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:33.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.098+0000 7f01a4bd9700 1 --2- 172.21.15.133:0/2961578762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01a0100ee0 0x7f01a0073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:21:33.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.098+0000 7f01a53da700 1 --2- 172.21.15.133:0/2961578762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01a0100540 0x7f01a0072df0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f019c00b580 tx=0x7f019c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:33.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.098+0000 7f01967fc700 1 -- 172.21.15.133:0/2961578762 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c01d070 con 0x7f01a0100540 2023-11-27T15:21:33.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.098+0000 7f01967fc700 1 -- 172.21.15.133:0/2961578762 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f019c005050 con 0x7f01a0100540 2023-11-27T15:21:33.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.098+0000 7f01a763e700 1 -- 172.21.15.133:0/2961578762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019c0095f0 con 0x7f01a0100540 2023-11-27T15:21:33.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.098+0000 7f01967fc700 1 -- 172.21.15.133:0/2961578762 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c004a40 con 0x7f01a0100540 2023-11-27T15:21:33.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.098+0000 7f01a763e700 1 -- 172.21.15.133:0/2961578762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a0110370 con 0x7f01a0100540 2023-11-27T15:21:33.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.100+0000 7f01a763e700 1 -- 172.21.15.133:0/2961578762 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f01a00fd1e0 con 0x7f01a0100540 2023-11-27T15:21:33.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.104+0000 7f01967fc700 1 -- 172.21.15.133:0/2961578762 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f019c0051c0 con 0x7f01a0100540 2023-11-27T15:21:33.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.106+0000 7f01967fc700 1 --2- 172.21.15.133:0/2961578762 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f018c0a1260 0x7f018c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:33.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.106+0000 7f01967fc700 1 -- 172.21.15.133:0/2961578762 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f019c0ce290 con 0x7f01a0100540 2023-11-27T15:21:33.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.106+0000 7f01a4bd9700 1 --2- 172.21.15.133:0/2961578762 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f018c0a1260 0x7f018c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:33.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.106+0000 7f01967fc700 1 --2- 172.21.15.133:0/2961578762 >> [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790] conn(0x7f018c0ac270 0x7f018c0ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:33.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.106+0000 7f01967fc700 1 -- 172.21.15.133:0/2961578762 --> [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f018c0aedd0 con 0x7f018c0ac270 2023-11-27T15:21:33.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.106+0000 7f01967fc700 1 -- 172.21.15.133:0/2961578762 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f019c0ce6a0 con 0x7f01a0100540 2023-11-27T15:21:33.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.107+0000 7f01a5bdb700 1 --2- 172.21.15.133:0/2961578762 >> [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790] conn(0x7f018c0ac270 0x7f018c0ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:33.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.107+0000 7f01a5bdb700 1 --2- 172.21.15.133:0/2961578762 >> [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790] conn(0x7f018c0ac270 0x7f018c0ae700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:33.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.108+0000 7f01a4bd9700 1 --2- 172.21.15.133:0/2961578762 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f018c0a1260 0x7f018c0a3710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f01a00745e0 tx=0x7f0190009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:33.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.108+0000 7f01967fc700 1 -- 172.21.15.133:0/2961578762 <== osd.9 v2:172.21.15.133:6826/34943790 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f018c0aedd0 con 0x7f018c0ac270 2023-11-27T15:21:33.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.125+0000 7f01a763e700 1 -- 172.21.15.133:0/2961578762 --> [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f01a01101e0 con 0x7f018c0ac270 2023-11-27T15:21:33.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.126+0000 7f01967fc700 1 -- 172.21.15.133:0/2961578762 <== osd.9 v2:172.21.15.133:6826/34943790 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f01a01101e0 con 0x7f018c0ac270 2023-11-27T15:21:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.127+0000 7f01a763e700 1 -- 172.21.15.133:0/2961578762 >> [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790] conn(0x7f018c0ac270 msgr2=0x7f018c0ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.127+0000 7f01a763e700 1 --2- 172.21.15.133:0/2961578762 >> [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790] conn(0x7f018c0ac270 0x7f018c0ae700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.127+0000 7f01a763e700 1 -- 172.21.15.133:0/2961578762 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f018c0a1260 msgr2=0x7f018c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.127+0000 7f01a763e700 1 --2- 172.21.15.133:0/2961578762 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f018c0a1260 0x7f018c0a3710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f01a00745e0 tx=0x7f0190009260 comp rx=0 tx=0).stop 2023-11-27T15:21:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.127+0000 7f01a763e700 1 -- 172.21.15.133:0/2961578762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01a0100540 msgr2=0x7f01a0072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.127+0000 7f01a763e700 1 --2- 172.21.15.133:0/2961578762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01a0100540 0x7f01a0072df0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f019c00b580 tx=0x7f019c00bfd0 comp rx=0 tx=0).stop 2023-11-27T15:21:33.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.127+0000 7f01a5bdb700 1 -- 172.21.15.133:0/2961578762 reap_dead start 2023-11-27T15:21:33.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.127+0000 7f01a763e700 1 -- 172.21.15.133:0/2961578762 shutdown_connections 2023-11-27T15:21:33.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.127+0000 7f01a763e700 1 -- 172.21.15.133:0/2961578762 >> 172.21.15.133:0/2961578762 conn(0x7f01a00fbdc0 msgr2=0x7f01a00fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:33.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.128+0000 7f01a763e700 1 -- 172.21.15.133:0/2961578762 shutdown_connections 2023-11-27T15:21:33.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.128+0000 7f01a763e700 1 -- 172.21.15.133:0/2961578762 wait complete. 2023-11-27T15:21:33.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:32 smithi174 ceph-mon[131110]: pgmap v305: 129 pgs: 129 active+clean; 637 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 853 B/s wr, 7 op/s 2023-11-27T15:21:33.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.970+0000 7f597dbbd700 1 -- 172.21.15.133:0/2763956224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59781133b0 msgr2=0x7f5978115790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:33.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.970+0000 7f597dbbd700 1 --2- 172.21.15.133:0/2763956224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59781133b0 0x7f5978115790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f597000b0a0 tx=0x7f597000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:21:33.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.971+0000 7f597dbbd700 1 -- 172.21.15.133:0/2763956224 shutdown_connections 2023-11-27T15:21:33.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.971+0000 7f597dbbd700 1 --2- 172.21.15.133:0/2763956224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59781133b0 0x7f5978115790 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:33.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.971+0000 7f597dbbd700 1 --2- 172.21.15.133:0/2763956224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5978107af0 0x7f5978107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:33.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.971+0000 7f597dbbd700 1 --2- 172.21.15.133:0/2763956224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5978108400 0x7f5978112e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:33.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.971+0000 7f597dbbd700 1 -- 172.21.15.133:0/2763956224 >> 172.21.15.133:0/2763956224 conn(0x7f597806c600 msgr2=0x7f59780717b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:33.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.971+0000 7f597dbbd700 1 -- 172.21.15.133:0/2763956224 shutdown_connections 2023-11-27T15:21:33.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.971+0000 7f597dbbd700 1 -- 172.21.15.133:0/2763956224 wait complete. 2023-11-27T15:21:33.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.972+0000 7f597dbbd700 1 Processor -- start 2023-11-27T15:21:33.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.972+0000 7f597dbbd700 1 -- start start 2023-11-27T15:21:33.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.973+0000 7f597dbbd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5978107af0 0x7f59781a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:33.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.973+0000 7f597dbbd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5978108400 0x7f59781a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:33.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.973+0000 7f597dbbd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59781133b0 0x7f597819c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:33.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.973+0000 7f597dbbd700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597819cfc0 con 0x7f5978107af0 2023-11-27T15:21:33.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.973+0000 7f597dbbd700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597819d130 con 0x7f59781133b0 2023-11-27T15:21:33.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.973+0000 7f597dbbd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597819d2f0 con 0x7f5978108400 2023-11-27T15:21:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.973+0000 7f5976ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5978108400 0x7f59781a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.973+0000 7f5976ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5978108400 0x7f59781a2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:39936/0 (socket says 172.21.15.133:39936) 2023-11-27T15:21:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.973+0000 7f5976ffd700 1 -- 172.21.15.133:0/1791876680 learned_addr learned my addr 172.21.15.133:0/1791876680 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.973+0000 7f59777fe700 1 --2- 172.21.15.133:0/1791876680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5978107af0 0x7f59781a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.973+0000 7f5977fff700 1 --2- 172.21.15.133:0/1791876680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59781133b0 0x7f597819c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.974+0000 7f5976ffd700 1 -- 172.21.15.133:0/1791876680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59781133b0 msgr2=0x7f597819c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.974+0000 7f5976ffd700 1 --2- 172.21.15.133:0/1791876680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59781133b0 0x7f597819c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.974+0000 7f5976ffd700 1 -- 172.21.15.133:0/1791876680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5978107af0 msgr2=0x7f59781a28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:33.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.974+0000 7f5976ffd700 1 --2- 172.21.15.133:0/1791876680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5978107af0 0x7f59781a28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:33.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.974+0000 7f5976ffd700 1 -- 172.21.15.133:0/1791876680 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5970009d00 con 0x7f5978108400 2023-11-27T15:21:33.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.974+0000 7f59777fe700 1 --2- 172.21.15.133:0/1791876680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5978107af0 0x7f59781a28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:33.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.975+0000 7f5976ffd700 1 --2- 172.21.15.133:0/1791876680 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5978108400 0x7f59781a2e10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f596c011b90 tx=0x7f596c011ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:33.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.976+0000 7f5974ff9700 1 -- 172.21.15.133:0/1791876680 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c004220 con 0x7f5978108400 2023-11-27T15:21:33.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.976+0000 7f597dbbd700 1 -- 172.21.15.133:0/1791876680 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59781169d0 con 0x7f5978108400 2023-11-27T15:21:33.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.976+0000 7f5974ff9700 1 -- 172.21.15.133:0/1791876680 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f596c013040 con 0x7f5978108400 2023-11-27T15:21:33.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.976+0000 7f5974ff9700 1 -- 172.21.15.133:0/1791876680 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c00fc70 con 0x7f5978108400 2023-11-27T15:21:33.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.976+0000 7f597dbbd700 1 -- 172.21.15.133:0/1791876680 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5978116840 con 0x7f5978108400 2023-11-27T15:21:34.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.977+0000 7f597dbbd700 1 -- 172.21.15.133:0/1791876680 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5978061270 con 0x7f5978108400 2023-11-27T15:21:34.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.979+0000 7f5974ff9700 1 -- 172.21.15.133:0/1791876680 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f596c004750 con 0x7f5978108400 2023-11-27T15:21:34.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.980+0000 7f5974ff9700 1 --2- 172.21.15.133:0/1791876680 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f59600a1260 0x7f59600a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:34.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.981+0000 7f5974ff9700 1 -- 172.21.15.133:0/1791876680 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f596c0d0530 con 0x7f5978108400 2023-11-27T15:21:34.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.981+0000 7f5974ff9700 1 --2- 172.21.15.133:0/1791876680 >> [v2:172.21.15.174:6816/2275667196,v1:172.21.15.174:6817/2275667196] conn(0x7f59600ac270 0x7f59600ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:34.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.981+0000 7f59777fe700 1 --2- 172.21.15.133:0/1791876680 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f59600a1260 0x7f59600a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:34.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.981+0000 7f5974ff9700 1 -- 172.21.15.133:0/1791876680 --> [v2:172.21.15.174:6816/2275667196,v1:172.21.15.174:6817/2275667196] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f59600aedd0 con 0x7f59600ac270 2023-11-27T15:21:34.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.981+0000 7f5974ff9700 1 -- 172.21.15.133:0/1791876680 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f596c0d08a0 con 0x7f5978108400 2023-11-27T15:21:34.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.981+0000 7f5977fff700 1 --2- 172.21.15.133:0/1791876680 >> [v2:172.21.15.174:6816/2275667196,v1:172.21.15.174:6817/2275667196] conn(0x7f59600ac270 0x7f59600ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:34.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.982+0000 7f5977fff700 1 --2- 172.21.15.133:0/1791876680 >> [v2:172.21.15.174:6816/2275667196,v1:172.21.15.174:6817/2275667196] conn(0x7f59600ac270 0x7f59600ae700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:34.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.983+0000 7f5974ff9700 1 -- 172.21.15.133:0/1791876680 <== osd.7 v2:172.21.15.174:6816/2275667196 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f59600aedd0 con 0x7f59600ac270 2023-11-27T15:21:34.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:32.984+0000 7f59777fe700 1 --2- 172.21.15.133:0/1791876680 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f59600a1260 0x7f59600a3710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5968005950 tx=0x7f596800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:34.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.004+0000 7f597dbbd700 1 -- 172.21.15.133:0/1791876680 --> [v2:172.21.15.174:6816/2275667196,v1:172.21.15.174:6817/2275667196] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5978061620 con 0x7f59600ac270 2023-11-27T15:21:34.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.006+0000 7f5974ff9700 1 -- 172.21.15.133:0/1791876680 <== osd.7 v2:172.21.15.174:6816/2275667196 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5978061620 con 0x7f59600ac270 2023-11-27T15:21:34.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.007+0000 7f597dbbd700 1 -- 172.21.15.133:0/1791876680 >> [v2:172.21.15.174:6816/2275667196,v1:172.21.15.174:6817/2275667196] conn(0x7f59600ac270 msgr2=0x7f59600ae700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:34.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.007+0000 7f597dbbd700 1 --2- 172.21.15.133:0/1791876680 >> [v2:172.21.15.174:6816/2275667196,v1:172.21.15.174:6817/2275667196] conn(0x7f59600ac270 0x7f59600ae700 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:34.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.007+0000 7f597dbbd700 1 -- 172.21.15.133:0/1791876680 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f59600a1260 msgr2=0x7f59600a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:34.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.007+0000 7f597dbbd700 1 --2- 172.21.15.133:0/1791876680 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f59600a1260 0x7f59600a3710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5968005950 tx=0x7f596800b410 comp rx=0 tx=0).stop 2023-11-27T15:21:34.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.007+0000 7f597dbbd700 1 -- 172.21.15.133:0/1791876680 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5978108400 msgr2=0x7f59781a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:34.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.007+0000 7f597dbbd700 1 --2- 172.21.15.133:0/1791876680 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5978108400 0x7f59781a2e10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f596c011b90 tx=0x7f596c011ea0 comp rx=0 tx=0).stop 2023-11-27T15:21:34.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.007+0000 7f5977fff700 1 -- 172.21.15.133:0/1791876680 reap_dead start 2023-11-27T15:21:34.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.007+0000 7f597dbbd700 1 -- 172.21.15.133:0/1791876680 shutdown_connections 2023-11-27T15:21:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.007+0000 7f597dbbd700 1 -- 172.21.15.133:0/1791876680 >> 172.21.15.133:0/1791876680 conn(0x7f597806c600 msgr2=0x7f597810ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.007+0000 7f597dbbd700 1 -- 172.21.15.133:0/1791876680 shutdown_connections 2023-11-27T15:21:34.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:33.007+0000 7f597dbbd700 1 -- 172.21.15.133:0/1791876680 wait complete. 2023-11-27T15:21:34.069 INFO:teuthology.orchestra.run.smithi133.stdout:141733920823 2023-11-27T15:21:34.070 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.4 2023-11-27T15:21:34.490 INFO:teuthology.orchestra.run.smithi133.stdout:283467841567 2023-11-27T15:21:34.490 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.9 2023-11-27T15:21:34.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.840+0000 7fb77ad9c700 1 -- 172.21.15.133:0/1515368002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb774100ee0 msgr2=0x7fb774104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:34.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.840+0000 7fb77ad9c700 1 --2- 172.21.15.133:0/1515368002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb774100ee0 0x7fb774104eb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb7600099c0 tx=0x7fb760009cd0 comp rx=0 tx=0).stop 2023-11-27T15:21:34.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.840+0000 7fb77ad9c700 1 -- 172.21.15.133:0/1515368002 shutdown_connections 2023-11-27T15:21:34.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.840+0000 7fb77ad9c700 1 --2- 172.21.15.133:0/1515368002 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7741055e0 0x7fb77410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:34.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.840+0000 7fb77ad9c700 1 --2- 172.21.15.133:0/1515368002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb774100540 0x7fb774100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:34.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.840+0000 7fb77ad9c700 1 --2- 172.21.15.133:0/1515368002 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb774100ee0 0x7fb774104eb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:34.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.840+0000 7fb77ad9c700 1 -- 172.21.15.133:0/1515368002 >> 172.21.15.133:0/1515368002 conn(0x7fb7740fbdc0 msgr2=0x7fb7740fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:34.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.841+0000 7fb77ad9c700 1 -- 172.21.15.133:0/1515368002 shutdown_connections 2023-11-27T15:21:34.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.841+0000 7fb77ad9c700 1 -- 172.21.15.133:0/1515368002 wait complete. 2023-11-27T15:21:34.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.842+0000 7fb77ad9c700 1 Processor -- start 2023-11-27T15:21:34.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.842+0000 7fb77ad9c700 1 -- start start 2023-11-27T15:21:34.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.843+0000 7fb77ad9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb774100540 0x7fb7741966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:34.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.843+0000 7fb77ad9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7741055e0 0x7fb774196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:34.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.843+0000 7fb77ad9c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb774199fa0 0x7fb77419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:34.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.843+0000 7fb77ad9c700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77419a980 con 0x7fb7741055e0 2023-11-27T15:21:34.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.843+0000 7fb77ad9c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77419aaf0 con 0x7fb774100540 2023-11-27T15:21:34.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.843+0000 7fb77ad9c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77419acb0 con 0x7fb774199fa0 2023-11-27T15:21:34.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.843+0000 7fb778b38700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb774100540 0x7fb7741966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:34.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.843+0000 7fb773fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7741055e0 0x7fb774196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:34.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.843+0000 7fb778b38700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb774100540 0x7fb7741966a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:45398/0 (socket says 172.21.15.133:45398) 2023-11-27T15:21:34.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.843+0000 7fb778b38700 1 -- 172.21.15.133:0/3949653072 learned_addr learned my addr 172.21.15.133:0/3949653072 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:34.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.843+0000 7fb779339700 1 --2- 172.21.15.133:0/3949653072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb774199fa0 0x7fb77419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:34.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.844+0000 7fb773fff700 1 -- 172.21.15.133:0/3949653072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb774199fa0 msgr2=0x7fb77419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:34.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.844+0000 7fb773fff700 1 --2- 172.21.15.133:0/3949653072 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb774199fa0 0x7fb77419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:34.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.844+0000 7fb773fff700 1 -- 172.21.15.133:0/3949653072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb774100540 msgr2=0x7fb7741966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:34.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.844+0000 7fb773fff700 1 --2- 172.21.15.133:0/3949653072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb774100540 0x7fb7741966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:34.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.844+0000 7fb773fff700 1 -- 172.21.15.133:0/3949653072 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7600095f0 con 0x7fb7741055e0 2023-11-27T15:21:34.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.844+0000 7fb778b38700 1 --2- 172.21.15.133:0/3949653072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb774100540 0x7fb7741966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:21:34.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.844+0000 7fb773fff700 1 --2- 172.21.15.133:0/3949653072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7741055e0 0x7fb774196be0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb774101450 tx=0x7fb76000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:34.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.844+0000 7fb771ffb700 1 -- 172.21.15.133:0/3949653072 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76001d070 con 0x7fb7741055e0 2023-11-27T15:21:34.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.844+0000 7fb77ad9c700 1 -- 172.21.15.133:0/3949653072 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb774110360 con 0x7fb7741055e0 2023-11-27T15:21:34.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.844+0000 7fb771ffb700 1 -- 172.21.15.133:0/3949653072 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7600054e0 con 0x7fb7741055e0 2023-11-27T15:21:34.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.844+0000 7fb771ffb700 1 -- 172.21.15.133:0/3949653072 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760004ab0 con 0x7fb7741055e0 2023-11-27T15:21:34.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.844+0000 7fb77ad9c700 1 -- 172.21.15.133:0/3949653072 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7741101d0 con 0x7fb7741055e0 2023-11-27T15:21:34.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.846+0000 7fb77ad9c700 1 -- 172.21.15.133:0/3949653072 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb7740fd280 con 0x7fb7741055e0 2023-11-27T15:21:34.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.848+0000 7fb771ffb700 1 -- 172.21.15.133:0/3949653072 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fb76000fbb0 con 0x7fb7741055e0 2023-11-27T15:21:34.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.849+0000 7fb771ffb700 1 --2- 172.21.15.133:0/3949653072 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb7640a12e0 0x7fb7640a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:34.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.850+0000 7fb771ffb700 1 -- 172.21.15.133:0/3949653072 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb7600cd730 con 0x7fb7741055e0 2023-11-27T15:21:34.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.850+0000 7fb778b38700 1 --2- 172.21.15.133:0/3949653072 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb7640a12e0 0x7fb7640a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:34.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.850+0000 7fb771ffb700 1 --2- 172.21.15.133:0/3949653072 >> [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009] conn(0x7fb7640ac2f0 0x7fb7640ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:34.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.850+0000 7fb771ffb700 1 -- 172.21.15.133:0/3949653072 --> [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb7640aee50 con 0x7fb7640ac2f0 2023-11-27T15:21:34.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.850+0000 7fb771ffb700 1 -- 172.21.15.133:0/3949653072 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7600cfd50 con 0x7fb7741055e0 2023-11-27T15:21:34.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.850+0000 7fb779339700 1 --2- 172.21.15.133:0/3949653072 >> [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009] conn(0x7fb7640ac2f0 0x7fb7640ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:34.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.851+0000 7fb778b38700 1 --2- 172.21.15.133:0/3949653072 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb7640a12e0 0x7fb7640a3790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb768009d70 tx=0x7fb768009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:34.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.851+0000 7fb779339700 1 --2- 172.21.15.133:0/3949653072 >> [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009] conn(0x7fb7640ac2f0 0x7fb7640ae780 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:34.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.853+0000 7fb771ffb700 1 -- 172.21.15.133:0/3949653072 <== osd.3 v2:172.21.15.133:6810/673797009 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb7640aee50 con 0x7fb7640ac2f0 2023-11-27T15:21:34.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.871+0000 7fb77ad9c700 1 -- 172.21.15.133:0/3949653072 --> [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb7741104f0 con 0x7fb7640ac2f0 2023-11-27T15:21:34.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.871+0000 7fb771ffb700 1 -- 172.21.15.133:0/3949653072 <== osd.3 v2:172.21.15.133:6810/673797009 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb7741104f0 con 0x7fb7640ac2f0 2023-11-27T15:21:34.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.872+0000 7fb77ad9c700 1 -- 172.21.15.133:0/3949653072 >> [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009] conn(0x7fb7640ac2f0 msgr2=0x7fb7640ae780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:34.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.872+0000 7fb77ad9c700 1 --2- 172.21.15.133:0/3949653072 >> [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009] conn(0x7fb7640ac2f0 0x7fb7640ae780 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:34.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.872+0000 7fb77ad9c700 1 -- 172.21.15.133:0/3949653072 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb7640a12e0 msgr2=0x7fb7640a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:34.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.872+0000 7fb77ad9c700 1 --2- 172.21.15.133:0/3949653072 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb7640a12e0 0x7fb7640a3790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb768009d70 tx=0x7fb768009260 comp rx=0 tx=0).stop 2023-11-27T15:21:34.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.872+0000 7fb77ad9c700 1 -- 172.21.15.133:0/3949653072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7741055e0 msgr2=0x7fb774196be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:34.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.872+0000 7fb77ad9c700 1 --2- 172.21.15.133:0/3949653072 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7741055e0 0x7fb774196be0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb774101450 tx=0x7fb76000f740 comp rx=0 tx=0).stop 2023-11-27T15:21:34.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.872+0000 7fb779339700 1 -- 172.21.15.133:0/3949653072 reap_dead start 2023-11-27T15:21:34.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.873+0000 7fb77ad9c700 1 -- 172.21.15.133:0/3949653072 shutdown_connections 2023-11-27T15:21:34.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.873+0000 7fb77ad9c700 1 -- 172.21.15.133:0/3949653072 >> 172.21.15.133:0/3949653072 conn(0x7fb7740fbdc0 msgr2=0x7fb7740fc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:34.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.873+0000 7fb77ad9c700 1 -- 172.21.15.133:0/3949653072 shutdown_connections 2023-11-27T15:21:34.877 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.873+0000 7fb77ad9c700 1 -- 172.21.15.133:0/3949653072 wait complete. 2023-11-27T15:21:34.909 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:34 smithi133 ceph-mon[97860]: pgmap v306: 129 pgs: 129 active+clean; 637 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-11-27T15:21:34.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.989+0000 7f80b4a92700 1 -- 172.21.15.133:0/3707953843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80b0100540 msgr2=0x7f80b0100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:34.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.989+0000 7f80b4a92700 1 --2- 172.21.15.133:0/3707953843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80b0100540 0x7f80b0100910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f80a0009990 tx=0x7f80a0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:21:34.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.990+0000 7f80b4a92700 1 -- 172.21.15.133:0/3707953843 shutdown_connections 2023-11-27T15:21:34.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.990+0000 7f80b4a92700 1 --2- 172.21.15.133:0/3707953843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80b01055e0 0x7f80b010d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:34.994 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.990+0000 7f80b4a92700 1 --2- 172.21.15.133:0/3707953843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80b0100540 0x7f80b0100910 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:34.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.990+0000 7f80b4a92700 1 --2- 172.21.15.133:0/3707953843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80b0100ee0 0x7f80b0104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:34.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.990+0000 7f80b4a92700 1 -- 172.21.15.133:0/3707953843 >> 172.21.15.133:0/3707953843 conn(0x7f80b00fbdc0 msgr2=0x7f80b00fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:34.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.990+0000 7f80b4a92700 1 -- 172.21.15.133:0/3707953843 shutdown_connections 2023-11-27T15:21:34.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.990+0000 7f80b4a92700 1 -- 172.21.15.133:0/3707953843 wait complete. 2023-11-27T15:21:34.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.991+0000 7f80b4a92700 1 Processor -- start 2023-11-27T15:21:34.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.992+0000 7f80b4a92700 1 -- start start 2023-11-27T15:21:34.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.992+0000 7f80b4a92700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80b0100540 0x7f80b0194200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:34.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.992+0000 7f80b4a92700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80b0100ee0 0x7f80b0194740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:34.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.992+0000 7f80b4a92700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80b01055e0 0x7f80b0198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:34.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.992+0000 7f80b4a92700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b01991c0 con 0x7f80b0100ee0 2023-11-27T15:21:34.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.992+0000 7f80b4a92700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b0199330 con 0x7f80b0100540 2023-11-27T15:21:34.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.992+0000 7f80b4a92700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b01994f0 con 0x7f80b01055e0 2023-11-27T15:21:34.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.993+0000 7f80ae59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80b0100540 0x7f80b0194200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:34.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.993+0000 7f80add9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80b0100ee0 0x7f80b0194740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:34.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.993+0000 7f80ae59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80b0100540 0x7f80b0194200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:45412/0 (socket says 172.21.15.133:45412) 2023-11-27T15:21:34.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.993+0000 7f80ae59c700 1 -- 172.21.15.133:0/385439112 learned_addr learned my addr 172.21.15.133:0/385439112 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:34.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.993+0000 7f80ae59c700 1 -- 172.21.15.133:0/385439112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80b01055e0 msgr2=0x7f80b0198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:34.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.993+0000 7f80ae59c700 1 --2- 172.21.15.133:0/385439112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80b01055e0 0x7f80b0198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:34.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.993+0000 7f80ae59c700 1 -- 172.21.15.133:0/385439112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80b0100ee0 msgr2=0x7f80b0194740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:34.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.993+0000 7f80ae59c700 1 --2- 172.21.15.133:0/385439112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80b0100ee0 0x7f80b0194740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:34.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.993+0000 7f80ae59c700 1 -- 172.21.15.133:0/385439112 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a4009520 con 0x7f80b0100540 2023-11-27T15:21:34.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.994+0000 7f80add9b700 1 --2- 172.21.15.133:0/385439112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80b0100ee0 0x7f80b0194740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:34.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.994+0000 7f80ae59c700 1 --2- 172.21.15.133:0/385439112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80b0100540 0x7f80b0194200 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f80a0000c00 tx=0x7f80a0005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:34.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.994+0000 7f809f7fe700 1 -- 172.21.15.133:0/385439112 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a001d070 con 0x7f80b0100540 2023-11-27T15:21:34.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.994+0000 7f809f7fe700 1 -- 172.21.15.133:0/385439112 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80a0022470 con 0x7f80b0100540 2023-11-27T15:21:34.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.994+0000 7f809f7fe700 1 -- 172.21.15.133:0/385439112 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a0017650 con 0x7f80b0100540 2023-11-27T15:21:35.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.995+0000 7f80b4a92700 1 -- 172.21.15.133:0/385439112 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a00095f0 con 0x7f80b0100540 2023-11-27T15:21:35.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.996+0000 7f80b4a92700 1 -- 172.21.15.133:0/385439112 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b010e1d0 con 0x7f80b0100540 2023-11-27T15:21:35.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:34.997+0000 7f80b4a92700 1 -- 172.21.15.133:0/385439112 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f80b00fd260 con 0x7f80b0100540 2023-11-27T15:21:35.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.000+0000 7f809f7fe700 1 -- 172.21.15.133:0/385439112 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f80a000fa50 con 0x7f80b0100540 2023-11-27T15:21:35.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.002+0000 7f809f7fe700 1 --2- 172.21.15.133:0/385439112 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f80980a5740 0x7f80980a7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:35.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.002+0000 7f809f7fe700 1 -- 172.21.15.133:0/385439112 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f80a00cea30 con 0x7f80b0100540 2023-11-27T15:21:35.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.002+0000 7f80add9b700 1 --2- 172.21.15.133:0/385439112 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f80980a5740 0x7f80980a7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:35.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.002+0000 7f809f7fe700 1 --2- 172.21.15.133:0/385439112 >> [v2:172.21.15.174:6800/3890663971,v1:172.21.15.174:6801/3890663971] conn(0x7f80980b0750 0x7f80980b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:35.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.002+0000 7f809f7fe700 1 -- 172.21.15.133:0/385439112 --> [v2:172.21.15.174:6800/3890663971,v1:172.21.15.174:6801/3890663971] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80980b32b0 con 0x7f80980b0750 2023-11-27T15:21:35.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.003+0000 7f809f7fe700 1 -- 172.21.15.133:0/385439112 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80a00cee40 con 0x7f80b0100540 2023-11-27T15:21:35.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.003+0000 7f80aed9d700 1 --2- 172.21.15.133:0/385439112 >> [v2:172.21.15.174:6800/3890663971,v1:172.21.15.174:6801/3890663971] conn(0x7f80980b0750 0x7f80980b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:35.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.003+0000 7f80aed9d700 1 --2- 172.21.15.133:0/385439112 >> [v2:172.21.15.174:6800/3890663971,v1:172.21.15.174:6801/3890663971] conn(0x7f80980b0750 0x7f80980b2be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:35.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.003+0000 7f80add9b700 1 --2- 172.21.15.133:0/385439112 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f80980a5740 0x7f80980a7bf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009fd0 tx=0x7f80a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:35.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.004+0000 7f809f7fe700 1 -- 172.21.15.133:0/385439112 <== osd.1 v2:172.21.15.174:6800/3890663971 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f80980b32b0 con 0x7f80980b0750 2023-11-27T15:21:35.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.024+0000 7f79c3b75700 1 -- 172.21.15.133:0/1339899737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79bc100ee0 msgr2=0x7f79bc104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:35.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.025+0000 7f79c3b75700 1 --2- 172.21.15.133:0/1339899737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79bc100ee0 0x7f79bc104eb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0099c0 tx=0x7f79ac009cd0 comp rx=0 tx=0).stop 2023-11-27T15:21:35.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.025+0000 7f79c3b75700 1 -- 172.21.15.133:0/1339899737 shutdown_connections 2023-11-27T15:21:35.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.025+0000 7f79c3b75700 1 --2- 172.21.15.133:0/1339899737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79bc1055e0 0x7f79bc10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:35.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.025+0000 7f79c3b75700 1 --2- 172.21.15.133:0/1339899737 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79bc100540 0x7f79bc100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:35.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.025+0000 7f79c3b75700 1 --2- 172.21.15.133:0/1339899737 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79bc100ee0 0x7f79bc104eb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:35.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.025+0000 7f79c3b75700 1 -- 172.21.15.133:0/1339899737 >> 172.21.15.133:0/1339899737 conn(0x7f79bc0fbdc0 msgr2=0x7f79bc0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:35.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.026+0000 7f79c3b75700 1 -- 172.21.15.133:0/1339899737 shutdown_connections 2023-11-27T15:21:35.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.026+0000 7f79c3b75700 1 -- 172.21.15.133:0/1339899737 wait complete. 2023-11-27T15:21:35.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.027+0000 7f79c3b75700 1 Processor -- start 2023-11-27T15:21:35.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.027+0000 7f79c3b75700 1 -- start start 2023-11-27T15:21:35.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.028+0000 7f79c3b75700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79bc100540 0x7f79bc1071e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:35.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.028+0000 7f79c3b75700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79bc100ee0 0x7f79bc107720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:35.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.028+0000 7f79c3b75700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79bc1055e0 0x7f79bc107c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:35.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.028+0000 7f79c3b75700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc1a8cb0 con 0x7f79bc100540 2023-11-27T15:21:35.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.028+0000 7f79c3b75700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc1a8e20 con 0x7f79bc100ee0 2023-11-27T15:21:35.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.028+0000 7f79c2112700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79bc1055e0 0x7f79bc107c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:35.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.028+0000 7f79c2112700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79bc1055e0 0x7f79bc107c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:40016/0 (socket says 172.21.15.133:40016) 2023-11-27T15:21:35.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.028+0000 7f79c2112700 1 -- 172.21.15.133:0/1421754647 learned_addr learned my addr 172.21.15.133:0/1421754647 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:35.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.029+0000 7f79c1911700 1 --2- 172.21.15.133:0/1421754647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79bc100540 0x7f79bc1071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:35.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.028+0000 7f79c3b75700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc1a8fe0 con 0x7f79bc1055e0 2023-11-27T15:21:35.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.029+0000 7f79c1110700 1 --2- 172.21.15.133:0/1421754647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79bc100ee0 0x7f79bc107720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:35.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.029+0000 7f79c2112700 1 -- 172.21.15.133:0/1421754647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79bc100ee0 msgr2=0x7f79bc107720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:35.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.029+0000 7f79c2112700 1 --2- 172.21.15.133:0/1421754647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79bc100ee0 0x7f79bc107720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:35.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.029+0000 7f79c2112700 1 -- 172.21.15.133:0/1421754647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79bc100540 msgr2=0x7f79bc1071e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:35.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.029+0000 7f79c2112700 1 --2- 172.21.15.133:0/1421754647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79bc100540 0x7f79bc1071e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:35.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.029+0000 7f79c2112700 1 -- 172.21.15.133:0/1421754647 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ac0095f0 con 0x7f79bc1055e0 2023-11-27T15:21:35.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.029+0000 7f79c1911700 1 --2- 172.21.15.133:0/1421754647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79bc100540 0x7f79bc1071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:21:35.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.030+0000 7f79c1110700 1 --2- 172.21.15.133:0/1421754647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79bc100ee0 0x7f79bc107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:21:35.034 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.030+0000 7f79c2112700 1 --2- 172.21.15.133:0/1421754647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79bc1055e0 0x7f79bc107c60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f79b4007ee0 tx=0x7f79b400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:35.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.030+0000 7f79b2ffd700 1 -- 172.21.15.133:0/1421754647 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4017070 con 0x7f79bc1055e0 2023-11-27T15:21:35.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.030+0000 7f79c3b75700 1 -- 172.21.15.133:0/1421754647 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79bc110360 con 0x7f79bc1055e0 2023-11-27T15:21:35.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.031+0000 7f79b2ffd700 1 -- 172.21.15.133:0/1421754647 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79b400f040 con 0x7f79bc1055e0 2023-11-27T15:21:35.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.031+0000 7f79b2ffd700 1 -- 172.21.15.133:0/1421754647 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4005740 con 0x7f79bc1055e0 2023-11-27T15:21:35.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.031+0000 7f79c3b75700 1 -- 172.21.15.133:0/1421754647 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79bc1101d0 con 0x7f79bc1055e0 2023-11-27T15:21:35.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.033+0000 7f79c3b75700 1 -- 172.21.15.133:0/1421754647 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f79bc074070 con 0x7f79bc1055e0 2023-11-27T15:21:35.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.034+0000 7f80b4a92700 1 -- 172.21.15.133:0/385439112 --> [v2:172.21.15.174:6800/3890663971,v1:172.21.15.174:6801/3890663971] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f80b010e040 con 0x7f80980b0750 2023-11-27T15:21:35.038 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.034+0000 7f79b2ffd700 1 -- 172.21.15.133:0/1421754647 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f79b40058a0 con 0x7f79bc1055e0 2023-11-27T15:21:35.039 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.035+0000 7f809f7fe700 1 -- 172.21.15.133:0/385439112 <== osd.1 v2:172.21.15.174:6800/3890663971 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f80b010e040 con 0x7f80980b0750 2023-11-27T15:21:35.039 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.035+0000 7f80b4a92700 1 -- 172.21.15.133:0/385439112 >> [v2:172.21.15.174:6800/3890663971,v1:172.21.15.174:6801/3890663971] conn(0x7f80980b0750 msgr2=0x7f80980b2be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:35.039 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.035+0000 7f80b4a92700 1 --2- 172.21.15.133:0/385439112 >> [v2:172.21.15.174:6800/3890663971,v1:172.21.15.174:6801/3890663971] conn(0x7f80980b0750 0x7f80980b2be0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:35.040 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.035+0000 7f80b4a92700 1 -- 172.21.15.133:0/385439112 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f80980a5740 msgr2=0x7f80980a7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:35.040 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.035+0000 7f80b4a92700 1 --2- 172.21.15.133:0/385439112 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f80980a5740 0x7f80980a7bf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009fd0 tx=0x7f80a4009260 comp rx=0 tx=0).stop 2023-11-27T15:21:35.040 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.035+0000 7f80b4a92700 1 -- 172.21.15.133:0/385439112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80b0100540 msgr2=0x7f80b0194200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:35.040 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.036+0000 7f80b4a92700 1 --2- 172.21.15.133:0/385439112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80b0100540 0x7f80b0194200 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f80a0000c00 tx=0x7f80a0005740 comp rx=0 tx=0).stop 2023-11-27T15:21:35.040 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.036+0000 7f80aed9d700 1 -- 172.21.15.133:0/385439112 reap_dead start 2023-11-27T15:21:35.040 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.036+0000 7f79b2ffd700 1 --2- 172.21.15.133:0/1421754647 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f79a80a1260 0x7f79a80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:35.040 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.036+0000 7f80b4a92700 1 -- 172.21.15.133:0/385439112 shutdown_connections 2023-11-27T15:21:35.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.036+0000 7f80b4a92700 1 -- 172.21.15.133:0/385439112 >> 172.21.15.133:0/385439112 conn(0x7f80b00fbdc0 msgr2=0x7f80b00fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:35.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.036+0000 7f79b2ffd700 1 -- 172.21.15.133:0/1421754647 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f79b40cc770 con 0x7f79bc1055e0 2023-11-27T15:21:35.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.036+0000 7f80b4a92700 1 -- 172.21.15.133:0/385439112 shutdown_connections 2023-11-27T15:21:35.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.036+0000 7f80b4a92700 1 -- 172.21.15.133:0/385439112 wait complete. 2023-11-27T15:21:35.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.036+0000 7f79b2ffd700 1 --2- 172.21.15.133:0/1421754647 >> [v2:172.21.15.177:6808/3863044388,v1:172.21.15.177:6809/3863044388] conn(0x7f79a80ac1f0 0x7f79a80ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:35.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.036+0000 7f79b2ffd700 1 -- 172.21.15.133:0/1421754647 --> [v2:172.21.15.177:6808/3863044388,v1:172.21.15.177:6809/3863044388] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f79a80aed50 con 0x7f79a80ac1f0 2023-11-27T15:21:35.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.037+0000 7f79c1911700 1 --2- 172.21.15.133:0/1421754647 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f79a80a1260 0x7f79a80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:35.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.037+0000 7f79c1110700 1 --2- 172.21.15.133:0/1421754647 >> [v2:172.21.15.177:6808/3863044388,v1:172.21.15.177:6809/3863044388] conn(0x7f79a80ac1f0 0x7f79a80ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:35.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.037+0000 7f79c1911700 1 --2- 172.21.15.133:0/1421754647 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f79a80a1260 0x7f79a80a3710 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009cb0 tx=0x7f79b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:35.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.037+0000 7f79c1110700 1 --2- 172.21.15.133:0/1421754647 >> [v2:172.21.15.177:6808/3863044388,v1:172.21.15.177:6809/3863044388] conn(0x7f79a80ac1f0 0x7f79a80ae680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:35.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.038+0000 7f79b2ffd700 1 -- 172.21.15.133:0/1421754647 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79b4095a50 con 0x7f79bc1055e0 2023-11-27T15:21:35.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.038+0000 7f79b2ffd700 1 -- 172.21.15.133:0/1421754647 <== osd.5 v2:172.21.15.177:6808/3863044388 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f79a80aed50 con 0x7f79a80ac1f0 2023-11-27T15:21:35.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.065+0000 7f79c3b75700 1 -- 172.21.15.133:0/1421754647 --> [v2:172.21.15.177:6808/3863044388,v1:172.21.15.177:6809/3863044388] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f79bc1104f0 con 0x7f79a80ac1f0 2023-11-27T15:21:35.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.066+0000 7f79b2ffd700 1 -- 172.21.15.133:0/1421754647 <== osd.5 v2:172.21.15.177:6808/3863044388 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f79bc1104f0 con 0x7f79a80ac1f0 2023-11-27T15:21:35.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.067+0000 7f79c3b75700 1 -- 172.21.15.133:0/1421754647 >> [v2:172.21.15.177:6808/3863044388,v1:172.21.15.177:6809/3863044388] conn(0x7f79a80ac1f0 msgr2=0x7f79a80ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:35.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.067+0000 7f79c3b75700 1 --2- 172.21.15.133:0/1421754647 >> [v2:172.21.15.177:6808/3863044388,v1:172.21.15.177:6809/3863044388] conn(0x7f79a80ac1f0 0x7f79a80ae680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:35.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.067+0000 7f79c3b75700 1 -- 172.21.15.133:0/1421754647 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f79a80a1260 msgr2=0x7f79a80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:35.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.067+0000 7f79c3b75700 1 --2- 172.21.15.133:0/1421754647 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f79a80a1260 0x7f79a80a3710 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f79b8009cb0 tx=0x7f79b8009260 comp rx=0 tx=0).stop 2023-11-27T15:21:35.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.067+0000 7f79c3b75700 1 -- 172.21.15.133:0/1421754647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79bc1055e0 msgr2=0x7f79bc107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:35.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.067+0000 7f79c3b75700 1 --2- 172.21.15.133:0/1421754647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79bc1055e0 0x7f79bc107c60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f79b4007ee0 tx=0x7f79b400d3b0 comp rx=0 tx=0).stop 2023-11-27T15:21:35.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.067+0000 7f79c2112700 1 -- 172.21.15.133:0/1421754647 reap_dead start 2023-11-27T15:21:35.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.067+0000 7f79c3b75700 1 -- 172.21.15.133:0/1421754647 shutdown_connections 2023-11-27T15:21:35.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.067+0000 7f79c3b75700 1 -- 172.21.15.133:0/1421754647 >> 172.21.15.133:0/1421754647 conn(0x7f79bc0fbdc0 msgr2=0x7f79bc0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:35.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.067+0000 7f79c3b75700 1 -- 172.21.15.133:0/1421754647 shutdown_connections 2023-11-27T15:21:35.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:35.067+0000 7f79c3b75700 1 -- 172.21.15.133:0/1421754647 wait complete. 2023-11-27T15:21:35.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:34 smithi177 ceph-mon[131504]: pgmap v306: 129 pgs: 129 active+clean; 637 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-11-27T15:21:35.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:34 smithi174 ceph-mon[131110]: pgmap v306: 129 pgs: 129 active+clean; 637 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2023-11-27T15:21:35.739 INFO:teuthology.orchestra.run.smithi133.stdout:107374182459 2023-11-27T15:21:35.739 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.3 2023-11-27T15:21:35.915 INFO:teuthology.orchestra.run.smithi133.stdout:163208757300 2023-11-27T15:21:35.915 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.5 2023-11-27T15:21:35.976 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:35 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:21:36.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:35 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:21:36.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:35 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:21:36.253 INFO:teuthology.orchestra.run.smithi133.stdout:219043332137 2023-11-27T15:21:36.253 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.7 2023-11-27T15:21:36.282 INFO:teuthology.orchestra.run.smithi133.stdout:55834574917 2023-11-27T15:21:36.282 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.1 2023-11-27T15:21:36.644 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:36.740 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:36.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.892+0000 7fd9dbdc4700 1 -- 172.21.15.133:0/1750995090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9d40ff0d0 msgr2=0x7fd9d40ff4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:36.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.892+0000 7fd9dbdc4700 1 --2- 172.21.15.133:0/1750995090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9d40ff0d0 0x7fd9d40ff4a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009990 tx=0x7fd9cc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:21:36.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.892+0000 7fd9dbdc4700 1 -- 172.21.15.133:0/1750995090 shutdown_connections 2023-11-27T15:21:36.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.892+0000 7fd9dbdc4700 1 --2- 172.21.15.133:0/1750995090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9d410cc80 0x7fd9d410f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:36.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.892+0000 7fd9dbdc4700 1 --2- 172.21.15.133:0/1750995090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9d40ff0d0 0x7fd9d40ff4a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:36.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.892+0000 7fd9dbdc4700 1 --2- 172.21.15.133:0/1750995090 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9d40ff9e0 0x7fd9d410c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:36.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.892+0000 7fd9dbdc4700 1 -- 172.21.15.133:0/1750995090 >> 172.21.15.133:0/1750995090 conn(0x7fd9d40770a0 msgr2=0x7fd9d40774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:36.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.893+0000 7fd9dbdc4700 1 -- 172.21.15.133:0/1750995090 shutdown_connections 2023-11-27T15:21:36.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.893+0000 7fd9dbdc4700 1 -- 172.21.15.133:0/1750995090 wait complete. 2023-11-27T15:21:36.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.894+0000 7fd9dbdc4700 1 Processor -- start 2023-11-27T15:21:36.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.896+0000 7fd9dbdc4700 1 -- start start 2023-11-27T15:21:36.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.896+0000 7fd9dbdc4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9d40ff0d0 0x7fd9d41014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:36.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.896+0000 7fd9dbdc4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9d40ff9e0 0x7fd9d41019f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:36.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.896+0000 7fd9dbdc4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9d410cc80 0x7fd9d41069c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:36.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.896+0000 7fd9dbdc4700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d4107000 con 0x7fd9d40ff9e0 2023-11-27T15:21:36.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.896+0000 7fd9dbdc4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d4107170 con 0x7fd9d40ff0d0 2023-11-27T15:21:36.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.896+0000 7fd9dbdc4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d4107330 con 0x7fd9d410cc80 2023-11-27T15:21:36.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.897+0000 7fd9da361700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9d410cc80 0x7fd9d41069c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:36.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.897+0000 7fd9da361700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9d410cc80 0x7fd9d41069c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:40024/0 (socket says 172.21.15.133:40024) 2023-11-27T15:21:36.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.897+0000 7fd9da361700 1 -- 172.21.15.133:0/3452593802 learned_addr learned my addr 172.21.15.133:0/3452593802 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:36.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.897+0000 7fd9d9b60700 1 --2- 172.21.15.133:0/3452593802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9d40ff0d0 0x7fd9d41014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.897+0000 7fd9da361700 1 -- 172.21.15.133:0/3452593802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9d40ff0d0 msgr2=0x7fd9d41014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.897+0000 7fd9da361700 1 --2- 172.21.15.133:0/3452593802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9d40ff0d0 0x7fd9d41014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.897+0000 7fd9da361700 1 -- 172.21.15.133:0/3452593802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9d40ff9e0 msgr2=0x7fd9d41019f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:21:36.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.897+0000 7fd9d935f700 1 --2- 172.21.15.133:0/3452593802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9d40ff9e0 0x7fd9d41019f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:36.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.898+0000 7fd9da361700 1 --2- 172.21.15.133:0/3452593802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9d40ff9e0 0x7fd9d41019f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:36.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.898+0000 7fd9da361700 1 -- 172.21.15.133:0/3452593802 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc0095f0 con 0x7fd9d410cc80 2023-11-27T15:21:36.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.898+0000 7fd9d935f700 1 --2- 172.21.15.133:0/3452593802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd9d40ff9e0 0x7fd9d41019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:21:36.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.898+0000 7fd9da361700 1 --2- 172.21.15.133:0/3452593802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9d410cc80 0x7fd9d41069c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009fd0 tx=0x7fd9c400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:36.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.899+0000 7fd9d2ffd700 1 -- 172.21.15.133:0/3452593802 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4018070 con 0x7fd9d410cc80 2023-11-27T15:21:36.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.899+0000 7fd9d2ffd700 1 -- 172.21.15.133:0/3452593802 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9c400f040 con 0x7fd9d410cc80 2023-11-27T15:21:36.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.899+0000 7fd9d2ffd700 1 -- 172.21.15.133:0/3452593802 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c400cdb0 con 0x7fd9d410cc80 2023-11-27T15:21:36.904 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:36.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.899+0000 7fd9dbdc4700 1 -- 172.21.15.133:0/3452593802 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d4112650 con 0x7fd9d410cc80 2023-11-27T15:21:36.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.899+0000 7fd9dbdc4700 1 -- 172.21.15.133:0/3452593802 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d41124c0 con 0x7fd9d410cc80 2023-11-27T15:21:36.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.901+0000 7fd9dbdc4700 1 -- 172.21.15.133:0/3452593802 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd9d40611f0 con 0x7fd9d410cc80 2023-11-27T15:21:36.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.904+0000 7fd9d2ffd700 1 -- 172.21.15.133:0/3452593802 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fd9c4004ad0 con 0x7fd9d410cc80 2023-11-27T15:21:36.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.906+0000 7fd9d2ffd700 1 --2- 172.21.15.133:0/3452593802 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9bc0a13b0 0x7fd9bc0a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:36.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.906+0000 7fd9d2ffd700 1 -- 172.21.15.133:0/3452593802 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd9c40cc810 con 0x7fd9d410cc80 2023-11-27T15:21:36.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.907+0000 7fd9d9b60700 1 --2- 172.21.15.133:0/3452593802 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9bc0a13b0 0x7fd9bc0a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:36.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.908+0000 7fd9d2ffd700 1 --2- 172.21.15.133:0/3452593802 >> [v2:172.21.15.177:6816/3783894564,v1:172.21.15.177:6817/3783894564] conn(0x7fd9bc0ac3c0 0x7fd9bc0ae850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:36.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.908+0000 7fd9d2ffd700 1 -- 172.21.15.133:0/3452593802 --> [v2:172.21.15.177:6816/3783894564,v1:172.21.15.177:6817/3783894564] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9bc0aef20 con 0x7fd9bc0ac3c0 2023-11-27T15:21:36.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.908+0000 7fd9d2ffd700 1 -- 172.21.15.133:0/3452593802 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9c40ced90 con 0x7fd9d410cc80 2023-11-27T15:21:36.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.909+0000 7fd9d9b60700 1 --2- 172.21.15.133:0/3452593802 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9bc0a13b0 0x7fd9bc0a3860 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0098e0 tx=0x7fd9cc005ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:36.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.911+0000 7fd9d935f700 1 --2- 172.21.15.133:0/3452593802 >> [v2:172.21.15.177:6816/3783894564,v1:172.21.15.177:6817/3783894564] conn(0x7fd9bc0ac3c0 0x7fd9bc0ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:36.915 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.912+0000 7fd9d935f700 1 --2- 172.21.15.133:0/3452593802 >> [v2:172.21.15.177:6816/3783894564,v1:172.21.15.177:6817/3783894564] conn(0x7fd9bc0ac3c0 0x7fd9bc0ae850 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:36.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.913+0000 7fd9d2ffd700 1 -- 172.21.15.133:0/3452593802 <== osd.8 v2:172.21.15.177:6816/3783894564 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd9bc0aef20 con 0x7fd9bc0ac3c0 2023-11-27T15:21:36.939 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:36.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.943+0000 7fd9dbdc4700 1 -- 172.21.15.133:0/3452593802 --> [v2:172.21.15.177:6816/3783894564,v1:172.21.15.177:6817/3783894564] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd9d41127e0 con 0x7fd9bc0ac3c0 2023-11-27T15:21:36.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.944+0000 7fd9d2ffd700 1 -- 172.21.15.133:0/3452593802 <== osd.8 v2:172.21.15.177:6816/3783894564 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd9d41127e0 con 0x7fd9bc0ac3c0 2023-11-27T15:21:36.950 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:36.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.948+0000 7fd9dbdc4700 1 -- 172.21.15.133:0/3452593802 >> [v2:172.21.15.177:6816/3783894564,v1:172.21.15.177:6817/3783894564] conn(0x7fd9bc0ac3c0 msgr2=0x7fd9bc0ae850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:36.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.948+0000 7fd9dbdc4700 1 --2- 172.21.15.133:0/3452593802 >> [v2:172.21.15.177:6816/3783894564,v1:172.21.15.177:6817/3783894564] conn(0x7fd9bc0ac3c0 0x7fd9bc0ae850 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:36.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.949+0000 7fd9dbdc4700 1 -- 172.21.15.133:0/3452593802 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9bc0a13b0 msgr2=0x7fd9bc0a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:36.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.949+0000 7fd9dbdc4700 1 --2- 172.21.15.133:0/3452593802 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9bc0a13b0 0x7fd9bc0a3860 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc0098e0 tx=0x7fd9cc005ab0 comp rx=0 tx=0).stop 2023-11-27T15:21:36.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.949+0000 7fd9dbdc4700 1 -- 172.21.15.133:0/3452593802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9d410cc80 msgr2=0x7fd9d41069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:36.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.949+0000 7fd9dbdc4700 1 --2- 172.21.15.133:0/3452593802 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9d410cc80 0x7fd9d41069c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009fd0 tx=0x7fd9c400ef90 comp rx=0 tx=0).stop 2023-11-27T15:21:36.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.949+0000 7fd9da361700 1 -- 172.21.15.133:0/3452593802 reap_dead start 2023-11-27T15:21:36.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.949+0000 7fd9dbdc4700 1 -- 172.21.15.133:0/3452593802 shutdown_connections 2023-11-27T15:21:36.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.949+0000 7fd9dbdc4700 1 -- 172.21.15.133:0/3452593802 >> 172.21.15.133:0/3452593802 conn(0x7fd9d40770a0 msgr2=0x7fd9d410e580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:36.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.952+0000 7fd9dbdc4700 1 -- 172.21.15.133:0/3452593802 shutdown_connections 2023-11-27T15:21:36.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:36.953+0000 7fd9dbdc4700 1 -- 172.21.15.133:0/3452593802 wait complete. 2023-11-27T15:21:37.011 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:36 smithi133 ceph-mon[97860]: pgmap v307: 129 pgs: 129 active+clean; 637 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-27T15:21:37.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:36 smithi177 ceph-mon[131504]: pgmap v307: 129 pgs: 129 active+clean; 637 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-27T15:21:37.110 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:37.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:36 smithi174 ceph-mon[131110]: pgmap v307: 129 pgs: 129 active+clean; 637 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-27T15:21:38.006 INFO:teuthology.orchestra.run.smithi133.stdout:249108103206 2023-11-27T15:21:38.007 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph osd last-stat-seq osd.8 2023-11-27T15:21:39.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:39 smithi174 ceph-mon[131110]: pgmap v308: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 6 op/s 2023-11-27T15:21:39.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:39 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:21:39.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:39 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:21:39.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:39 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:21:39.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:39 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:21:39.410 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:39 smithi133 ceph-mon[97860]: pgmap v308: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 6 op/s 2023-11-27T15:21:39.410 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:39 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:21:39.410 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:39 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:21:39.410 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:39 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:21:39.410 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:39 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:21:39.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:39 smithi177 ceph-mon[131504]: pgmap v308: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 597 B/s wr, 6 op/s 2023-11-27T15:21:39.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:39 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:21:39.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:39 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:21:39.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:39 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:21:39.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:39 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:21:40.477 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:40.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.884+0000 7f64289a7700 1 -- 172.21.15.133:0/846403760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6424100d10 msgr2=0x7f6424104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:40.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.884+0000 7f64289a7700 1 --2- 172.21.15.133:0/846403760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6424100d10 0x7f6424104ce0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f64180099c0 tx=0x7f6418009cd0 comp rx=0 tx=0).stop 2023-11-27T15:21:40.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.884+0000 7f64289a7700 1 -- 172.21.15.133:0/846403760 shutdown_connections 2023-11-27T15:21:40.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.884+0000 7f64289a7700 1 --2- 172.21.15.133:0/846403760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6424105410 0x7f642410cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:40.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.884+0000 7f64289a7700 1 --2- 172.21.15.133:0/846403760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6424100370 0x7f6424100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:40.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.884+0000 7f64289a7700 1 --2- 172.21.15.133:0/846403760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6424100d10 0x7f6424104ce0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:40.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.884+0000 7f64289a7700 1 -- 172.21.15.133:0/846403760 >> 172.21.15.133:0/846403760 conn(0x7f64240fbbd0 msgr2=0x7f64240fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:40.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.884+0000 7f64289a7700 1 -- 172.21.15.133:0/846403760 shutdown_connections 2023-11-27T15:21:40.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.884+0000 7f64289a7700 1 -- 172.21.15.133:0/846403760 wait complete. 2023-11-27T15:21:40.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.885+0000 7f64289a7700 1 Processor -- start 2023-11-27T15:21:40.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.885+0000 7f64289a7700 1 -- start start 2023-11-27T15:21:40.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.885+0000 7f64289a7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6424100370 0x7f6424196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:40.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.885+0000 7f64289a7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6424100d10 0x7f6424196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:40.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.886+0000 7f64289a7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6424105410 0x7f642419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:40.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.886+0000 7f64289a7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642419b1c0 con 0x7f6424100370 2023-11-27T15:21:40.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.886+0000 7f64289a7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642419b330 con 0x7f6424105410 2023-11-27T15:21:40.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.886+0000 7f64289a7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642419b4f0 con 0x7f6424100d10 2023-11-27T15:21:40.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.886+0000 7f6422d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6424100370 0x7f6424196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:40.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.886+0000 7f6422d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6424100370 0x7f6424196200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57874/0 (socket says 172.21.15.133:57874) 2023-11-27T15:21:40.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.886+0000 7f6422d9d700 1 -- 172.21.15.133:0/2203924362 learned_addr learned my addr 172.21.15.133:0/2203924362 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:40.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.886+0000 7f642359e700 1 --2- 172.21.15.133:0/2203924362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6424105410 0x7f642419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:40.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.886+0000 7f642259c700 1 --2- 172.21.15.133:0/2203924362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6424100d10 0x7f6424196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:40.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.886+0000 7f6422d9d700 1 -- 172.21.15.133:0/2203924362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6424100d10 msgr2=0x7f6424196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:40.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.886+0000 7f6422d9d700 1 --2- 172.21.15.133:0/2203924362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6424100d10 0x7f6424196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:40.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.886+0000 7f6422d9d700 1 -- 172.21.15.133:0/2203924362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6424105410 msgr2=0x7f642419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:40.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.886+0000 7f6422d9d700 1 --2- 172.21.15.133:0/2203924362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6424105410 0x7f642419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:40.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.886+0000 7f6422d9d700 1 -- 172.21.15.133:0/2203924362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64180095f0 con 0x7f6424100370 2023-11-27T15:21:40.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.887+0000 7f6422d9d700 1 --2- 172.21.15.133:0/2203924362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6424100370 0x7f6424196200 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f641400d980 tx=0x7f641400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:40.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.887+0000 7f640bfff700 1 -- 172.21.15.133:0/2203924362 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6414004220 con 0x7f6424100370 2023-11-27T15:21:40.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.887+0000 7f640bfff700 1 -- 172.21.15.133:0/2203924362 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6414007640 con 0x7f6424100370 2023-11-27T15:21:40.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.887+0000 7f64289a7700 1 -- 172.21.15.133:0/2203924362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6424110140 con 0x7f6424100370 2023-11-27T15:21:40.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.887+0000 7f640bfff700 1 -- 172.21.15.133:0/2203924362 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6414005690 con 0x7f6424100370 2023-11-27T15:21:40.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.887+0000 7f64289a7700 1 -- 172.21.15.133:0/2203924362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f642410ffb0 con 0x7f6424100370 2023-11-27T15:21:40.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.888+0000 7f64289a7700 1 -- 172.21.15.133:0/2203924362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64241102d0 con 0x7f6424100370 2023-11-27T15:21:40.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.890+0000 7f640bfff700 1 -- 172.21.15.133:0/2203924362 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f6414004750 con 0x7f6424100370 2023-11-27T15:21:40.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.891+0000 7f640bfff700 1 --2- 172.21.15.133:0/2203924362 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f640c0a11e0 0x7f640c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:40.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.891+0000 7f640bfff700 1 -- 172.21.15.133:0/2203924362 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f64140ccbe0 con 0x7f6424100370 2023-11-27T15:21:40.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.892+0000 7f642259c700 1 --2- 172.21.15.133:0/2203924362 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f640c0a11e0 0x7f640c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:40.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.892+0000 7f642259c700 1 --2- 172.21.15.133:0/2203924362 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f640c0a11e0 0x7f640c0a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6418006010 tx=0x7f6418003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:40.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:40.896+0000 7f640bfff700 1 -- 172.21.15.133:0/2203924362 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6414095e60 con 0x7f6424100370 2023-11-27T15:21:41.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:41.120+0000 7f64289a7700 1 -- 172.21.15.133:0/2203924362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f64241a3280 con 0x7f6424100370 2023-11-27T15:21:41.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:41.120+0000 7f640bfff700 1 -- 172.21.15.133:0/2203924362 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6414016020 con 0x7f6424100370 2023-11-27T15:21:41.124 INFO:teuthology.orchestra.run.smithi133.stdout:77309411394 2023-11-27T15:21:41.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:41.123+0000 7f64289a7700 1 -- 172.21.15.133:0/2203924362 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f640c0a11e0 msgr2=0x7f640c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:41.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:41.123+0000 7f64289a7700 1 --2- 172.21.15.133:0/2203924362 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f640c0a11e0 0x7f640c0a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6418006010 tx=0x7f6418003680 comp rx=0 tx=0).stop 2023-11-27T15:21:41.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:41.123+0000 7f64289a7700 1 -- 172.21.15.133:0/2203924362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6424100370 msgr2=0x7f6424196200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:41.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:41.123+0000 7f64289a7700 1 --2- 172.21.15.133:0/2203924362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6424100370 0x7f6424196200 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f641400d980 tx=0x7f641400dc90 comp rx=0 tx=0).stop 2023-11-27T15:21:41.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:41.123+0000 7f64289a7700 1 -- 172.21.15.133:0/2203924362 shutdown_connections 2023-11-27T15:21:41.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:41.123+0000 7f64289a7700 1 --2- 172.21.15.133:0/2203924362 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f640c0a11e0 0x7f640c0a3690 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:41.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:41.123+0000 7f64289a7700 1 --2- 172.21.15.133:0/2203924362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6424105410 0x7f642419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:41.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:41.123+0000 7f64289a7700 1 --2- 172.21.15.133:0/2203924362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6424100370 0x7f6424196200 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:41.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:41.123+0000 7f64289a7700 1 --2- 172.21.15.133:0/2203924362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6424100d10 0x7f6424196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:41.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:41.123+0000 7f64289a7700 1 -- 172.21.15.133:0/2203924362 >> 172.21.15.133:0/2203924362 conn(0x7f64240fbbd0 msgr2=0x7f64240fc680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:41.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:41.123+0000 7f64289a7700 1 -- 172.21.15.133:0/2203924362 shutdown_connections 2023-11-27T15:21:41.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:41.123+0000 7f64289a7700 1 -- 172.21.15.133:0/2203924362 wait complete. 2023-11-27T15:21:41.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:41 smithi133 ceph-mon[97860]: pgmap v309: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:21:41.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:41 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2203924362' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-11-27T15:21:41.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:41 smithi174 ceph-mon[131110]: pgmap v309: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:21:41.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:41 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2203924362' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-11-27T15:21:41.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:41 smithi177 ceph-mon[131504]: pgmap v309: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:21:41.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:41 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2203924362' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-11-27T15:21:42.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.036+0000 7f8c134f0700 1 -- 172.21.15.133:0/1352901569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c0c100ee0 msgr2=0x7f8c0c104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.037+0000 7f8c134f0700 1 --2- 172.21.15.133:0/1352901569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c0c100ee0 0x7f8c0c104eb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0099c0 tx=0x7f8bfc009cd0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.037+0000 7f8c134f0700 1 -- 172.21.15.133:0/1352901569 shutdown_connections 2023-11-27T15:21:42.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.037+0000 7f8c134f0700 1 --2- 172.21.15.133:0/1352901569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c0c1055e0 0x7f8c0c10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.037+0000 7f8c134f0700 1 --2- 172.21.15.133:0/1352901569 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c0c100540 0x7f8c0c100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.037+0000 7f8c134f0700 1 --2- 172.21.15.133:0/1352901569 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c0c100ee0 0x7f8c0c104eb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.037+0000 7f8c134f0700 1 -- 172.21.15.133:0/1352901569 >> 172.21.15.133:0/1352901569 conn(0x7f8c0c0fbdc0 msgr2=0x7f8c0c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:42.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.037+0000 7f8c134f0700 1 -- 172.21.15.133:0/1352901569 shutdown_connections 2023-11-27T15:21:42.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.037+0000 7f8c134f0700 1 -- 172.21.15.133:0/1352901569 wait complete. 2023-11-27T15:21:42.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.038+0000 7f8c134f0700 1 Processor -- start 2023-11-27T15:21:42.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.038+0000 7f8c134f0700 1 -- start start 2023-11-27T15:21:42.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.039+0000 7f8c134f0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c0c100540 0x7f8c0c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.039+0000 7f8c134f0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c0c100ee0 0x7f8c0c198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.039+0000 7f8c134f0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c0c1055e0 0x7f8c0c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.039+0000 7f8c134f0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c19d590 con 0x7f8c0c100ee0 2023-11-27T15:21:42.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.039+0000 7f8c134f0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c19d700 con 0x7f8c0c1055e0 2023-11-27T15:21:42.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.039+0000 7f8c134f0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0c19d8c0 con 0x7f8c0c100540 2023-11-27T15:21:42.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.040+0000 7f8c10a8b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c0c100ee0 0x7f8c0c198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.040+0000 7f8c11a8d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c0c1055e0 0x7f8c0c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.040+0000 7f8c1128c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c0c100540 0x7f8c0c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.040+0000 7f8c11a8d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c0c1055e0 0x7f8c0c19cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:45496/0 (socket says 172.21.15.133:45496) 2023-11-27T15:21:42.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.040+0000 7f8c11a8d700 1 -- 172.21.15.133:0/3502014193 learned_addr learned my addr 172.21.15.133:0/3502014193 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:42.044 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.040+0000 7f8c1128c700 1 -- 172.21.15.133:0/3502014193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c0c1055e0 msgr2=0x7f8c0c19cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.040+0000 7f8c1128c700 1 --2- 172.21.15.133:0/3502014193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c0c1055e0 0x7f8c0c19cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.040+0000 7f8c1128c700 1 -- 172.21.15.133:0/3502014193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c0c100ee0 msgr2=0x7f8c0c198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.040+0000 7f8c1128c700 1 --2- 172.21.15.133:0/3502014193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c0c100ee0 0x7f8c0c198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.040+0000 7f8c1128c700 1 -- 172.21.15.133:0/3502014193 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bfc0095f0 con 0x7f8c0c100540 2023-11-27T15:21:42.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.040+0000 7f8c11a8d700 1 --2- 172.21.15.133:0/3502014193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c0c1055e0 0x7f8c0c19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:42.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.041+0000 7f8c1128c700 1 --2- 172.21.15.133:0/3502014193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c0c100540 0x7f8c0c1985d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800d950 tx=0x7f8c0800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:42.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.041+0000 7f8c027fc700 1 -- 172.21.15.133:0/3502014193 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0800f890 con 0x7f8c0c100540 2023-11-27T15:21:42.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.041+0000 7f8c027fc700 1 -- 172.21.15.133:0/3502014193 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c08010460 con 0x7f8c0c100540 2023-11-27T15:21:42.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.041+0000 7f8c027fc700 1 -- 172.21.15.133:0/3502014193 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0800e610 con 0x7f8c0c100540 2023-11-27T15:21:42.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.041+0000 7f8c134f0700 1 -- 172.21.15.133:0/3502014193 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0c110360 con 0x7f8c0c100540 2023-11-27T15:21:42.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.041+0000 7f8c134f0700 1 -- 172.21.15.133:0/3502014193 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0c1101d0 con 0x7f8c0c100540 2023-11-27T15:21:42.046 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.043+0000 7f8c134f0700 1 -- 172.21.15.133:0/3502014193 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c0c1104f0 con 0x7f8c0c100540 2023-11-27T15:21:42.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.044+0000 7f8c027fc700 1 -- 172.21.15.133:0/3502014193 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f8c080108d0 con 0x7f8c0c100540 2023-11-27T15:21:42.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.046+0000 7f8c027fc700 1 --2- 172.21.15.133:0/3502014193 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8bf80a11e0 0x7f8bf80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.046+0000 7f8c027fc700 1 -- 172.21.15.133:0/3502014193 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8c080cca80 con 0x7f8c0c100540 2023-11-27T15:21:42.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.046+0000 7f8c10a8b700 1 --2- 172.21.15.133:0/3502014193 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8bf80a11e0 0x7f8bf80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.046+0000 7f8c10a8b700 1 --2- 172.21.15.133:0/3502014193 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8bf80a11e0 0x7f8bf80a3690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0098e0 tx=0x7f8bfc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:42.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.050+0000 7f8c027fc700 1 -- 172.21.15.133:0/3502014193 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8c08095d00 con 0x7f8c0c100540 2023-11-27T15:21:42.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.281+0000 7f8c134f0700 1 -- 172.21.15.133:0/3502014193 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f8c0c1a5650 con 0x7f8c0c100540 2023-11-27T15:21:42.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.282+0000 7f8c027fc700 1 -- 172.21.15.133:0/3502014193 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8c08016090 con 0x7f8c0c100540 2023-11-27T15:21:42.286 INFO:teuthology.orchestra.run.smithi133.stdout:197568495664 2023-11-27T15:21:42.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.285+0000 7f8c134f0700 1 -- 172.21.15.133:0/3502014193 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8bf80a11e0 msgr2=0x7f8bf80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.285+0000 7f8c134f0700 1 --2- 172.21.15.133:0/3502014193 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8bf80a11e0 0x7f8bf80a3690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0098e0 tx=0x7f8bfc000bc0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.285+0000 7f8c134f0700 1 -- 172.21.15.133:0/3502014193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c0c100540 msgr2=0x7f8c0c1985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.285+0000 7f8c134f0700 1 --2- 172.21.15.133:0/3502014193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c0c100540 0x7f8c0c1985d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800d950 tx=0x7f8c0800dc60 comp rx=0 tx=0).stop 2023-11-27T15:21:42.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.285+0000 7f8c134f0700 1 -- 172.21.15.133:0/3502014193 shutdown_connections 2023-11-27T15:21:42.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.285+0000 7f8c134f0700 1 --2- 172.21.15.133:0/3502014193 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8bf80a11e0 0x7f8bf80a3690 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.285+0000 7f8c134f0700 1 --2- 172.21.15.133:0/3502014193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c0c1055e0 0x7f8c0c19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.285+0000 7f8c134f0700 1 --2- 172.21.15.133:0/3502014193 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8c0c100540 0x7f8c0c1985d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.285+0000 7f8c134f0700 1 --2- 172.21.15.133:0/3502014193 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c0c100ee0 0x7f8c0c198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.285+0000 7f8c134f0700 1 -- 172.21.15.133:0/3502014193 >> 172.21.15.133:0/3502014193 conn(0x7f8c0c0fbdc0 msgr2=0x7f8c0c0fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:42.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.285+0000 7f8c134f0700 1 -- 172.21.15.133:0/3502014193 shutdown_connections 2023-11-27T15:21:42.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.285+0000 7f8c134f0700 1 -- 172.21.15.133:0/3502014193 wait complete. 2023-11-27T15:21:42.491 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411392 got 77309411394 for osd.2 2023-11-27T15:21:42.491 DEBUG:teuthology.parallel:result is None 2023-11-27T15:21:42.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.355+0000 7f3a60e6b700 1 -- 172.21.15.133:0/2252239424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a5c100540 msgr2=0x7f3a5c100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.355+0000 7f3a60e6b700 1 --2- 172.21.15.133:0/2252239424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a5c100540 0x7f3a5c100910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c009990 tx=0x7f3a4c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.355+0000 7f3a60e6b700 1 -- 172.21.15.133:0/2252239424 shutdown_connections 2023-11-27T15:21:42.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.355+0000 7f3a60e6b700 1 --2- 172.21.15.133:0/2252239424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a5c1055e0 0x7f3a5c10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.355+0000 7f3a60e6b700 1 --2- 172.21.15.133:0/2252239424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a5c100540 0x7f3a5c100910 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.355+0000 7f3a60e6b700 1 --2- 172.21.15.133:0/2252239424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a5c100ee0 0x7f3a5c104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.355+0000 7f3a60e6b700 1 -- 172.21.15.133:0/2252239424 >> 172.21.15.133:0/2252239424 conn(0x7f3a5c0fbdc0 msgr2=0x7f3a5c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.355+0000 7f3a60e6b700 1 -- 172.21.15.133:0/2252239424 shutdown_connections 2023-11-27T15:21:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.355+0000 7f3a60e6b700 1 -- 172.21.15.133:0/2252239424 wait complete. 2023-11-27T15:21:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.356+0000 7f3a60e6b700 1 Processor -- start 2023-11-27T15:21:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.356+0000 7f3a60e6b700 1 -- start start 2023-11-27T15:21:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.357+0000 7f3a60e6b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a5c100540 0x7f3a5c072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.357+0000 7f3a60e6b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a5c100ee0 0x7f3a5c073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.357+0000 7f3a60e6b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a5c1055e0 0x7f3a5c0738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.357+0000 7f3a60e6b700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c1a4840 con 0x7f3a5c1055e0 2023-11-27T15:21:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.357+0000 7f3a60e6b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c1a49b0 con 0x7f3a5c100540 2023-11-27T15:21:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.357+0000 7f3a60e6b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5c1a4b70 con 0x7f3a5c100ee0 2023-11-27T15:21:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.357+0000 7f3a5ad9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a5c1055e0 0x7f3a5c0738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.357+0000 7f3a5a59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a5c100540 0x7f3a5c072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.357+0000 7f3a5ad9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a5c1055e0 0x7f3a5c0738b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57902/0 (socket says 172.21.15.133:57902) 2023-11-27T15:21:42.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.357+0000 7f3a5ad9d700 1 -- 172.21.15.133:0/2931444220 learned_addr learned my addr 172.21.15.133:0/2931444220 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:42.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.357+0000 7f3a59d9b700 1 --2- 172.21.15.133:0/2931444220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a5c100ee0 0x7f3a5c073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.357+0000 7f3a5ad9d700 1 -- 172.21.15.133:0/2931444220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a5c100ee0 msgr2=0x7f3a5c073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.357+0000 7f3a5ad9d700 1 --2- 172.21.15.133:0/2931444220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a5c100ee0 0x7f3a5c073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.357+0000 7f3a5ad9d700 1 -- 172.21.15.133:0/2931444220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a5c100540 msgr2=0x7f3a5c072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.357+0000 7f3a5ad9d700 1 --2- 172.21.15.133:0/2931444220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a5c100540 0x7f3a5c072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.357+0000 7f3a5ad9d700 1 -- 172.21.15.133:0/2931444220 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4c0095f0 con 0x7f3a5c1055e0 2023-11-27T15:21:42.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.358+0000 7f3a5ad9d700 1 --2- 172.21.15.133:0/2931444220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a5c1055e0 0x7f3a5c0738b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400ef30 tx=0x7f3a5400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:42.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.358+0000 7f3a4b7fe700 1 -- 172.21.15.133:0/2931444220 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a54018070 con 0x7f3a5c1055e0 2023-11-27T15:21:42.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.358+0000 7f3a4b7fe700 1 -- 172.21.15.133:0/2931444220 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a5400f040 con 0x7f3a5c1055e0 2023-11-27T15:21:42.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.358+0000 7f3a60e6b700 1 -- 172.21.15.133:0/2931444220 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a5c110370 con 0x7f3a5c1055e0 2023-11-27T15:21:42.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.358+0000 7f3a4b7fe700 1 -- 172.21.15.133:0/2931444220 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5400cd00 con 0x7f3a5c1055e0 2023-11-27T15:21:42.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.358+0000 7f3a60e6b700 1 -- 172.21.15.133:0/2931444220 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a5c1101e0 con 0x7f3a5c1055e0 2023-11-27T15:21:42.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.359+0000 7f3a60e6b700 1 -- 172.21.15.133:0/2931444220 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a5c110500 con 0x7f3a5c1055e0 2023-11-27T15:21:42.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.361+0000 7f3a4b7fe700 1 -- 172.21.15.133:0/2931444220 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f3a54003a00 con 0x7f3a5c1055e0 2023-11-27T15:21:42.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.362+0000 7f3a4b7fe700 1 --2- 172.21.15.133:0/2931444220 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3a440a11e0 0x7f3a440a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.363+0000 7f3a4b7fe700 1 -- 172.21.15.133:0/2931444220 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3a540cc8c0 con 0x7f3a5c1055e0 2023-11-27T15:21:42.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.363+0000 7f3a5a59c700 1 --2- 172.21.15.133:0/2931444220 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3a440a11e0 0x7f3a440a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.363+0000 7f3a5a59c700 1 --2- 172.21.15.133:0/2931444220 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3a440a11e0 0x7f3a440a3690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c00b2e0 tx=0x7f3a4c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:42.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.366+0000 7f3a4b7fe700 1 -- 172.21.15.133:0/2931444220 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3a54095bd0 con 0x7f3a5c1055e0 2023-11-27T15:21:42.555 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:42.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.624+0000 7f3a60e6b700 1 -- 172.21.15.133:0/2931444220 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f3a5c074320 con 0x7f3a5c1055e0 2023-11-27T15:21:42.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.624+0000 7f3a4b7fe700 1 -- 172.21.15.133:0/2931444220 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3a540991f0 con 0x7f3a5c1055e0 2023-11-27T15:21:42.628 INFO:teuthology.orchestra.run.smithi133.stdout:141733920826 2023-11-27T15:21:42.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.627+0000 7f3a60e6b700 1 -- 172.21.15.133:0/2931444220 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3a440a11e0 msgr2=0x7f3a440a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.627+0000 7f3a60e6b700 1 --2- 172.21.15.133:0/2931444220 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3a440a11e0 0x7f3a440a3690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c00b2e0 tx=0x7f3a4c005fd0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.627+0000 7f3a60e6b700 1 -- 172.21.15.133:0/2931444220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a5c1055e0 msgr2=0x7f3a5c0738b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.627+0000 7f3a60e6b700 1 --2- 172.21.15.133:0/2931444220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a5c1055e0 0x7f3a5c0738b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400ef30 tx=0x7f3a5400c3a0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.627+0000 7f3a60e6b700 1 -- 172.21.15.133:0/2931444220 shutdown_connections 2023-11-27T15:21:42.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.627+0000 7f3a60e6b700 1 --2- 172.21.15.133:0/2931444220 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3a440a11e0 0x7f3a440a3690 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.627+0000 7f3a60e6b700 1 --2- 172.21.15.133:0/2931444220 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a5c1055e0 0x7f3a5c0738b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.627+0000 7f3a60e6b700 1 --2- 172.21.15.133:0/2931444220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a5c100540 0x7f3a5c072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.627+0000 7f3a60e6b700 1 --2- 172.21.15.133:0/2931444220 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a5c100ee0 0x7f3a5c073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.627+0000 7f3a60e6b700 1 -- 172.21.15.133:0/2931444220 >> 172.21.15.133:0/2931444220 conn(0x7f3a5c0fbdc0 msgr2=0x7f3a5c103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:42.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.627+0000 7f3a60e6b700 1 -- 172.21.15.133:0/2931444220 shutdown_connections 2023-11-27T15:21:42.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.627+0000 7f3a60e6b700 1 -- 172.21.15.133:0/2931444220 wait complete. 2023-11-27T15:21:42.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.912+0000 7f9f38bc5700 1 -- 172.21.15.133:0/3296802428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f34102f50 msgr2=0x7f9f3410f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.912+0000 7f9f38bc5700 1 --2- 172.21.15.133:0/3296802428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f34102f50 0x7f9f3410f3d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00b0a0 tx=0x7f9f2c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.912+0000 7f9f38bc5700 1 -- 172.21.15.133:0/3296802428 shutdown_connections 2023-11-27T15:21:42.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.912+0000 7f9f38bc5700 1 --2- 172.21.15.133:0/3296802428 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f34102f50 0x7f9f3410f3d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.912+0000 7f9f38bc5700 1 --2- 172.21.15.133:0/3296802428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f341086b0 0x7f9f34108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.912+0000 7f9f38bc5700 1 --2- 172.21.15.133:0/3296802428 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f341025c0 0x7f9f34102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.912+0000 7f9f38bc5700 1 -- 172.21.15.133:0/3296802428 >> 172.21.15.133:0/3296802428 conn(0x7f9f340fe100 msgr2=0x7f9f34100510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:42.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.912+0000 7f9f38bc5700 1 -- 172.21.15.133:0/3296802428 shutdown_connections 2023-11-27T15:21:42.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.912+0000 7f9f38bc5700 1 -- 172.21.15.133:0/3296802428 wait complete. 2023-11-27T15:21:42.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.913+0000 7f9f38bc5700 1 Processor -- start 2023-11-27T15:21:42.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.914+0000 7f9f38bc5700 1 -- start start 2023-11-27T15:21:42.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.914+0000 7f9f38bc5700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f341025c0 0x7f9f3406fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.914+0000 7f9f38bc5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f34102f50 0x7f9f34070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.914+0000 7f9f38bc5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f341086b0 0x7f9f3406aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.914+0000 7f9f38bc5700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3406b4e0 con 0x7f9f341025c0 2023-11-27T15:21:42.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.914+0000 7f9f38bc5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3406b620 con 0x7f9f34102f50 2023-11-27T15:21:42.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.914+0000 7f9f38bc5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3406b7e0 con 0x7f9f341086b0 2023-11-27T15:21:42.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.915+0000 7f9f3259c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f341025c0 0x7f9f3406fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.915+0000 7f9f3259c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f341025c0 0x7f9f3406fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60174/0 (socket says 172.21.15.133:60174) 2023-11-27T15:21:42.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.915+0000 7f9f3259c700 1 -- 172.21.15.133:0/3817077577 learned_addr learned my addr 172.21.15.133:0/3817077577 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:42.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.915+0000 7f9f31d9b700 1 --2- 172.21.15.133:0/3817077577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f34102f50 0x7f9f34070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.915+0000 7f9f32d9d700 1 --2- 172.21.15.133:0/3817077577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f341086b0 0x7f9f3406aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.915+0000 7f9f31d9b700 1 -- 172.21.15.133:0/3817077577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f341086b0 msgr2=0x7f9f3406aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.915+0000 7f9f31d9b700 1 --2- 172.21.15.133:0/3817077577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f341086b0 0x7f9f3406aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.915+0000 7f9f31d9b700 1 -- 172.21.15.133:0/3817077577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f341025c0 msgr2=0x7f9f3406fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:42.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.915+0000 7f9f31d9b700 1 --2- 172.21.15.133:0/3817077577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f341025c0 0x7f9f3406fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:42.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.915+0000 7f9f31d9b700 1 -- 172.21.15.133:0/3817077577 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c009d00 con 0x7f9f34102f50 2023-11-27T15:21:42.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.915+0000 7f9f3259c700 1 --2- 172.21.15.133:0/3817077577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f341025c0 0x7f9f3406fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:42.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.915+0000 7f9f32d9d700 1 --2- 172.21.15.133:0/3817077577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f341086b0 0x7f9f3406aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:21:42.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.915+0000 7f9f31d9b700 1 --2- 172.21.15.133:0/3817077577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f34102f50 0x7f9f34070210 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800e9f0 tx=0x7f9f2800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:42.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.916+0000 7f9f237fe700 1 -- 172.21.15.133:0/3817077577 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f280096d0 con 0x7f9f34102f50 2023-11-27T15:21:42.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.916+0000 7f9f237fe700 1 -- 172.21.15.133:0/3817077577 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9f2800cbd0 con 0x7f9f34102f50 2023-11-27T15:21:42.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.916+0000 7f9f38bc5700 1 -- 172.21.15.133:0/3817077577 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f341125d0 con 0x7f9f34102f50 2023-11-27T15:21:42.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.916+0000 7f9f237fe700 1 -- 172.21.15.133:0/3817077577 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f28018680 con 0x7f9f34102f50 2023-11-27T15:21:42.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.916+0000 7f9f38bc5700 1 -- 172.21.15.133:0/3817077577 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f34112440 con 0x7f9f34102f50 2023-11-27T15:21:42.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.917+0000 7f9f38bc5700 1 -- 172.21.15.133:0/3817077577 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f34112760 con 0x7f9f34102f50 2023-11-27T15:21:42.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.919+0000 7f9f237fe700 1 -- 172.21.15.133:0/3817077577 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f9f2800cd40 con 0x7f9f34102f50 2023-11-27T15:21:42.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.921+0000 7f9f237fe700 1 --2- 172.21.15.133:0/3817077577 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9f1c0a1260 0x7f9f1c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:42.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.921+0000 7f9f237fe700 1 -- 172.21.15.133:0/3817077577 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9f28014070 con 0x7f9f34102f50 2023-11-27T15:21:42.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.922+0000 7f9f3259c700 1 --2- 172.21.15.133:0/3817077577 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9f1c0a1260 0x7f9f1c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:42.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.922+0000 7f9f3259c700 1 --2- 172.21.15.133:0/3817077577 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9f1c0a1260 0x7f9f1c0a3710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9f24011550 tx=0x7f9f24009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:42.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:42.925+0000 7f9f237fe700 1 -- 172.21.15.133:0/3817077577 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9f28051730 con 0x7f9f34102f50 2023-11-27T15:21:43.030 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495660 got 197568495664 for osd.6 2023-11-27T15:21:43.030 DEBUG:teuthology.parallel:result is None 2023-11-27T15:21:43.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.084+0000 7f8a87052700 1 -- 172.21.15.133:0/2269645560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a801086b0 msgr2=0x7f8a80108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:43.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.084+0000 7f8a87052700 1 --2- 172.21.15.133:0/2269645560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a801086b0 0x7f8a80108a80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8a74009990 tx=0x7f8a74009ca0 comp rx=0 tx=0).stop 2023-11-27T15:21:43.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.084+0000 7f8a87052700 1 -- 172.21.15.133:0/2269645560 shutdown_connections 2023-11-27T15:21:43.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.084+0000 7f8a87052700 1 --2- 172.21.15.133:0/2269645560 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a80102f50 0x7f8a8010f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:43.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.084+0000 7f8a87052700 1 --2- 172.21.15.133:0/2269645560 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a801086b0 0x7f8a80108a80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:43.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.084+0000 7f8a87052700 1 --2- 172.21.15.133:0/2269645560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a801025c0 0x7f8a80102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:43.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.084+0000 7f8a87052700 1 -- 172.21.15.133:0/2269645560 >> 172.21.15.133:0/2269645560 conn(0x7f8a800fe100 msgr2=0x7f8a80100510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:43.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.084+0000 7f8a87052700 1 -- 172.21.15.133:0/2269645560 shutdown_connections 2023-11-27T15:21:43.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.085+0000 7f8a87052700 1 -- 172.21.15.133:0/2269645560 wait complete. 2023-11-27T15:21:43.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.085+0000 7f8a87052700 1 Processor -- start 2023-11-27T15:21:43.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.086+0000 7f8a87052700 1 -- start start 2023-11-27T15:21:43.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.086+0000 7f8a87052700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a801025c0 0x7f8a801985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:43.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.086+0000 7f8a87052700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a80102f50 0x7f8a80198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:43.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.086+0000 7f8a87052700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a801086b0 0x7f8a8019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:43.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.086+0000 7f8a87052700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8019d580 con 0x7f8a801025c0 2023-11-27T15:21:43.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.086+0000 7f8a87052700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8019d6f0 con 0x7f8a801086b0 2023-11-27T15:21:43.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.086+0000 7f8a87052700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8019d8b0 con 0x7f8a80102f50 2023-11-27T15:21:43.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.086+0000 7f8a84dee700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a801025c0 0x7f8a801985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:43.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.086+0000 7f8a84dee700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a801025c0 0x7f8a801985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60200/0 (socket says 172.21.15.133:60200) 2023-11-27T15:21:43.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.086+0000 7f8a84dee700 1 -- 172.21.15.133:0/1309761723 learned_addr learned my addr 172.21.15.133:0/1309761723 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:43.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.086+0000 7f8a855ef700 1 --2- 172.21.15.133:0/1309761723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a801086b0 0x7f8a8019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:43.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.086+0000 7f8a7ffff700 1 --2- 172.21.15.133:0/1309761723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a80102f50 0x7f8a80198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:43.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.087+0000 7f8a84dee700 1 -- 172.21.15.133:0/1309761723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a80102f50 msgr2=0x7f8a80198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:43.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.087+0000 7f8a84dee700 1 --2- 172.21.15.133:0/1309761723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a80102f50 0x7f8a80198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:43.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.087+0000 7f8a84dee700 1 -- 172.21.15.133:0/1309761723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a801086b0 msgr2=0x7f8a8019cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:43.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.087+0000 7f8a84dee700 1 --2- 172.21.15.133:0/1309761723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a801086b0 0x7f8a8019cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:43.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.087+0000 7f8a84dee700 1 -- 172.21.15.133:0/1309761723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a740095f0 con 0x7f8a801025c0 2023-11-27T15:21:43.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.087+0000 7f8a855ef700 1 --2- 172.21.15.133:0/1309761723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a801086b0 0x7f8a8019cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:43.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.087+0000 7f8a84dee700 1 --2- 172.21.15.133:0/1309761723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a801025c0 0x7f8a801985c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8a74005910 tx=0x7f8a74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:43.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.087+0000 7f8a7dffb700 1 -- 172.21.15.133:0/1309761723 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7401d070 con 0x7f8a801025c0 2023-11-27T15:21:43.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.088+0000 7f8a87052700 1 -- 172.21.15.133:0/1309761723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a80112600 con 0x7f8a801025c0 2023-11-27T15:21:43.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.087+0000 7f8a7dffb700 1 -- 172.21.15.133:0/1309761723 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a74022470 con 0x7f8a801025c0 2023-11-27T15:21:43.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.088+0000 7f8a7dffb700 1 -- 172.21.15.133:0/1309761723 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a74017650 con 0x7f8a801025c0 2023-11-27T15:21:43.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.088+0000 7f8a87052700 1 -- 172.21.15.133:0/1309761723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a80112470 con 0x7f8a801025c0 2023-11-27T15:21:43.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.089+0000 7f8a87052700 1 -- 172.21.15.133:0/1309761723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a80112790 con 0x7f8a801025c0 2023-11-27T15:21:43.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.091+0000 7f8a7dffb700 1 -- 172.21.15.133:0/1309761723 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f8a740225e0 con 0x7f8a801025c0 2023-11-27T15:21:43.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.092+0000 7f8a7dffb700 1 --2- 172.21.15.133:0/1309761723 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8a700a1260 0x7f8a700a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:43.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.092+0000 7f8a7dffb700 1 -- 172.21.15.133:0/1309761723 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8a740cdec0 con 0x7f8a801025c0 2023-11-27T15:21:43.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.093+0000 7f8a7ffff700 1 --2- 172.21.15.133:0/1309761723 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8a700a1260 0x7f8a700a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:43.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.093+0000 7f8a7ffff700 1 --2- 172.21.15.133:0/1309761723 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8a700a1260 0x7f8a700a3710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8a80199be0 tx=0x7f8a6c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:43.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.096+0000 7f8a7dffb700 1 -- 172.21.15.133:0/1309761723 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8a7405c730 con 0x7f8a801025c0 2023-11-27T15:21:43.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.175+0000 7f9f38bc5700 1 -- 172.21.15.133:0/3817077577 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f9f3406eec0 con 0x7f9f34102f50 2023-11-27T15:21:43.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.176+0000 7f9f237fe700 1 -- 172.21.15.133:0/3817077577 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f9f28096210 con 0x7f9f34102f50 2023-11-27T15:21:43.180 INFO:teuthology.orchestra.run.smithi133.stdout:339302416410 2023-11-27T15:21:43.182 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.179+0000 7f9f38bc5700 1 -- 172.21.15.133:0/3817077577 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9f1c0a1260 msgr2=0x7f9f1c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:43.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.179+0000 7f9f38bc5700 1 --2- 172.21.15.133:0/3817077577 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9f1c0a1260 0x7f9f1c0a3710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9f24011550 tx=0x7f9f24009260 comp rx=0 tx=0).stop 2023-11-27T15:21:43.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.179+0000 7f9f38bc5700 1 -- 172.21.15.133:0/3817077577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f34102f50 msgr2=0x7f9f34070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:43.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.179+0000 7f9f38bc5700 1 --2- 172.21.15.133:0/3817077577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f34102f50 0x7f9f34070210 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9f2800e9f0 tx=0x7f9f2800ed00 comp rx=0 tx=0).stop 2023-11-27T15:21:43.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.179+0000 7f9f38bc5700 1 -- 172.21.15.133:0/3817077577 shutdown_connections 2023-11-27T15:21:43.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.179+0000 7f9f38bc5700 1 --2- 172.21.15.133:0/3817077577 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9f1c0a1260 0x7f9f1c0a3710 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:43.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.179+0000 7f9f38bc5700 1 --2- 172.21.15.133:0/3817077577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9f341086b0 0x7f9f3406aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:43.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.179+0000 7f9f38bc5700 1 --2- 172.21.15.133:0/3817077577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f341025c0 0x7f9f3406fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:43.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.179+0000 7f9f38bc5700 1 --2- 172.21.15.133:0/3817077577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f34102f50 0x7f9f34070210 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:43.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.179+0000 7f9f38bc5700 1 -- 172.21.15.133:0/3817077577 >> 172.21.15.133:0/3817077577 conn(0x7f9f340fe100 msgr2=0x7f9f34106250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:43.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.179+0000 7f9f38bc5700 1 -- 172.21.15.133:0/3817077577 shutdown_connections 2023-11-27T15:21:43.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.179+0000 7f9f38bc5700 1 -- 172.21.15.133:0/3817077577 wait complete. 2023-11-27T15:21:43.206 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:43 smithi133 ceph-mon[97860]: pgmap v310: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-27T15:21:43.206 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:43 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3502014193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-11-27T15:21:43.206 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:43 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2931444220' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-11-27T15:21:43.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.313+0000 7f8a87052700 1 -- 172.21.15.133:0/1309761723 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8a801a5640 con 0x7f8a801025c0 2023-11-27T15:21:43.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.314+0000 7f8a7dffb700 1 -- 172.21.15.133:0/1309761723 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8a74097230 con 0x7f8a801025c0 2023-11-27T15:21:43.318 INFO:teuthology.orchestra.run.smithi133.stdout:34359738443 2023-11-27T15:21:43.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.317+0000 7f8a87052700 1 -- 172.21.15.133:0/1309761723 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8a700a1260 msgr2=0x7f8a700a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:43.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.317+0000 7f8a87052700 1 --2- 172.21.15.133:0/1309761723 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8a700a1260 0x7f8a700a3710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8a80199be0 tx=0x7f8a6c009260 comp rx=0 tx=0).stop 2023-11-27T15:21:43.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.317+0000 7f8a87052700 1 -- 172.21.15.133:0/1309761723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a801025c0 msgr2=0x7f8a801985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:43.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.317+0000 7f8a87052700 1 --2- 172.21.15.133:0/1309761723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a801025c0 0x7f8a801985c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8a74005910 tx=0x7f8a74004730 comp rx=0 tx=0).stop 2023-11-27T15:21:43.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.317+0000 7f8a87052700 1 -- 172.21.15.133:0/1309761723 shutdown_connections 2023-11-27T15:21:43.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.317+0000 7f8a87052700 1 --2- 172.21.15.133:0/1309761723 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8a700a1260 0x7f8a700a3710 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:43.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.317+0000 7f8a87052700 1 --2- 172.21.15.133:0/1309761723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a801086b0 0x7f8a8019cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:43.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.317+0000 7f8a87052700 1 --2- 172.21.15.133:0/1309761723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a801025c0 0x7f8a801985c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:43.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.317+0000 7f8a87052700 1 --2- 172.21.15.133:0/1309761723 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a80102f50 0x7f8a80198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:43.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.317+0000 7f8a87052700 1 -- 172.21.15.133:0/1309761723 >> 172.21.15.133:0/1309761723 conn(0x7f8a800fe100 msgr2=0x7f8a800ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:43.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.317+0000 7f8a87052700 1 -- 172.21.15.133:0/1309761723 shutdown_connections 2023-11-27T15:21:43.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:43.317+0000 7f8a87052700 1 -- 172.21.15.133:0/1309761723 wait complete. 2023-11-27T15:21:43.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:43 smithi174 ceph-mon[131110]: pgmap v310: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-27T15:21:43.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:43 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3502014193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-11-27T15:21:43.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:43 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2931444220' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-11-27T15:21:43.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:43 smithi177 ceph-mon[131504]: pgmap v310: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-27T15:21:43.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:43 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3502014193' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-11-27T15:21:43.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:43 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2931444220' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-11-27T15:21:43.662 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:43.850 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:44.302 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738441 got 34359738443 for osd.0 2023-11-27T15:21:44.302 DEBUG:teuthology.parallel:result is None 2023-11-27T15:21:44.303 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920823 got 141733920826 for osd.4 2023-11-27T15:21:44.303 DEBUG:teuthology.parallel:result is None 2023-11-27T15:21:44.508 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:44 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3817077577' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-11-27T15:21:44.508 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:44 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1309761723' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-11-27T15:21:44.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:44 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3817077577' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-11-27T15:21:44.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:44 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1309761723' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-11-27T15:21:44.652 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:44 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3817077577' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-11-27T15:21:44.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:44 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1309761723' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-11-27T15:21:44.661 INFO:tasks.cephadm.ceph_manager.ceph:need seq 339302416408 got 339302416410 for osd.11 2023-11-27T15:21:44.661 DEBUG:teuthology.parallel:result is None 2023-11-27T15:21:45.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.238+0000 7f8563464700 1 -- 172.21.15.133:0/2369834683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f855c102f50 msgr2=0x7f855c10f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:45.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.238+0000 7f8563464700 1 --2- 172.21.15.133:0/2369834683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f855c102f50 0x7f855c10f3d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8554009f90 tx=0x7f855400b330 comp rx=0 tx=0).stop 2023-11-27T15:21:45.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.238+0000 7f8563464700 1 -- 172.21.15.133:0/2369834683 shutdown_connections 2023-11-27T15:21:45.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.239+0000 7f8563464700 1 --2- 172.21.15.133:0/2369834683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f855c102f50 0x7f855c10f3d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:45.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.239+0000 7f8563464700 1 --2- 172.21.15.133:0/2369834683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f855c1086b0 0x7f855c108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:45.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.239+0000 7f8563464700 1 --2- 172.21.15.133:0/2369834683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f855c1025c0 0x7f855c102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:45.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.239+0000 7f8563464700 1 -- 172.21.15.133:0/2369834683 >> 172.21.15.133:0/2369834683 conn(0x7f855c0fe100 msgr2=0x7f855c100510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:45.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.239+0000 7f8563464700 1 -- 172.21.15.133:0/2369834683 shutdown_connections 2023-11-27T15:21:45.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.239+0000 7f8563464700 1 -- 172.21.15.133:0/2369834683 wait complete. 2023-11-27T15:21:45.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.241+0000 7f8563464700 1 Processor -- start 2023-11-27T15:21:45.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.242+0000 7f8563464700 1 -- start start 2023-11-27T15:21:45.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.243+0000 7f8563464700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f855c1025c0 0x7f855c071f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:45.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.243+0000 7f8563464700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f855c102f50 0x7f855c0724e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:45.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.243+0000 7f8563464700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f855c1086b0 0x7f855c06af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:45.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.243+0000 7f8563464700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c06b690 con 0x7f855c1025c0 2023-11-27T15:21:45.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.243+0000 7f8563464700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c06b800 con 0x7f855c1086b0 2023-11-27T15:21:45.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.243+0000 7f8563464700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c06b9c0 con 0x7f855c102f50 2023-11-27T15:21:45.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.243+0000 7f8561200700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f855c1025c0 0x7f855c071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:45.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.243+0000 7f8561a01700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f855c1086b0 0x7f855c06af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:45.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.243+0000 7f8561a01700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f855c1086b0 0x7f855c06af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:55216/0 (socket says 172.21.15.133:55216) 2023-11-27T15:21:45.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.243+0000 7f8561a01700 1 -- 172.21.15.133:0/2563977854 learned_addr learned my addr 172.21.15.133:0/2563977854 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:45.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.243+0000 7f85609ff700 1 --2- 172.21.15.133:0/2563977854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f855c102f50 0x7f855c0724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:45.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.244+0000 7f8561a01700 1 -- 172.21.15.133:0/2563977854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f855c102f50 msgr2=0x7f855c0724e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:45.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.244+0000 7f8561a01700 1 --2- 172.21.15.133:0/2563977854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f855c102f50 0x7f855c0724e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:45.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.244+0000 7f8561a01700 1 -- 172.21.15.133:0/2563977854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f855c1025c0 msgr2=0x7f855c071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:45.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.244+0000 7f8561a01700 1 --2- 172.21.15.133:0/2563977854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f855c1025c0 0x7f855c071f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:45.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.244+0000 7f8561a01700 1 -- 172.21.15.133:0/2563977854 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854c0095f0 con 0x7f855c1086b0 2023-11-27T15:21:45.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.244+0000 7f85609ff700 1 --2- 172.21.15.133:0/2563977854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f855c102f50 0x7f855c0724e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:45.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.244+0000 7f8561a01700 1 --2- 172.21.15.133:0/2563977854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f855c1086b0 0x7f855c06af80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f85540094b0 tx=0x7f8554004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:45.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.244+0000 7f85527fc700 1 -- 172.21.15.133:0/2563977854 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855400e040 con 0x7f855c1086b0 2023-11-27T15:21:45.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.245+0000 7f85527fc700 1 -- 172.21.15.133:0/2563977854 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8554004e20 con 0x7f855c1086b0 2023-11-27T15:21:45.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.245+0000 7f8563464700 1 -- 172.21.15.133:0/2563977854 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8558009520 con 0x7f855c1086b0 2023-11-27T15:21:45.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.245+0000 7f85527fc700 1 -- 172.21.15.133:0/2563977854 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85540052a0 con 0x7f855c1086b0 2023-11-27T15:21:45.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.245+0000 7f8563464700 1 -- 172.21.15.133:0/2563977854 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8554009c40 con 0x7f855c1086b0 2023-11-27T15:21:45.251 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.247+0000 7f8563464700 1 -- 172.21.15.133:0/2563977854 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855c066ea0 con 0x7f855c1086b0 2023-11-27T15:21:45.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.248+0000 7f85527fc700 1 -- 172.21.15.133:0/2563977854 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f855401d470 con 0x7f855c1086b0 2023-11-27T15:21:45.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.250+0000 7f85527fc700 1 --2- 172.21.15.133:0/2563977854 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f85480a1220 0x7f85480a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:45.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.250+0000 7f85527fc700 1 -- 172.21.15.133:0/2563977854 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f85540ce2d0 con 0x7f855c1086b0 2023-11-27T15:21:45.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.250+0000 7f8561200700 1 --2- 172.21.15.133:0/2563977854 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f85480a1220 0x7f85480a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:45.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.251+0000 7f8561200700 1 --2- 172.21.15.133:0/2563977854 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f85480a1220 0x7f85480a36d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8558009d70 tx=0x7f8558009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:45.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.254+0000 7f85527fc700 1 -- 172.21.15.133:0/2563977854 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8554097640 con 0x7f855c1086b0 2023-11-27T15:21:45.325 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:45 smithi133 ceph-mon[97860]: pgmap v311: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:21:45.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.498+0000 7f8563464700 1 -- 172.21.15.133:0/2563977854 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f855c06fb00 con 0x7f855c1086b0 2023-11-27T15:21:45.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.499+0000 7f85527fc700 1 -- 172.21.15.133:0/2563977854 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f855409ac60 con 0x7f855c1086b0 2023-11-27T15:21:45.503 INFO:teuthology.orchestra.run.smithi133.stdout:304942678046 2023-11-27T15:21:45.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.501+0000 7f8563464700 1 -- 172.21.15.133:0/2563977854 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f85480a1220 msgr2=0x7f85480a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:45.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.501+0000 7f8563464700 1 --2- 172.21.15.133:0/2563977854 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f85480a1220 0x7f85480a36d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8558009d70 tx=0x7f8558009260 comp rx=0 tx=0).stop 2023-11-27T15:21:45.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.501+0000 7f8563464700 1 -- 172.21.15.133:0/2563977854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f855c1086b0 msgr2=0x7f855c06af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:45.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.501+0000 7f8563464700 1 --2- 172.21.15.133:0/2563977854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f855c1086b0 0x7f855c06af80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f85540094b0 tx=0x7f8554004c80 comp rx=0 tx=0).stop 2023-11-27T15:21:45.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.502+0000 7f8563464700 1 -- 172.21.15.133:0/2563977854 shutdown_connections 2023-11-27T15:21:45.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.502+0000 7f8563464700 1 --2- 172.21.15.133:0/2563977854 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f85480a1220 0x7f85480a36d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:45.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.502+0000 7f8563464700 1 --2- 172.21.15.133:0/2563977854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f855c1086b0 0x7f855c06af80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:45.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.502+0000 7f8563464700 1 --2- 172.21.15.133:0/2563977854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f855c1025c0 0x7f855c071f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:45.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.502+0000 7f8563464700 1 --2- 172.21.15.133:0/2563977854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f855c102f50 0x7f855c0724e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:45.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.502+0000 7f8563464700 1 -- 172.21.15.133:0/2563977854 >> 172.21.15.133:0/2563977854 conn(0x7f855c0fe100 msgr2=0x7f855c0ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:45.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.502+0000 7f8563464700 1 -- 172.21.15.133:0/2563977854 shutdown_connections 2023-11-27T15:21:45.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.502+0000 7f8563464700 1 -- 172.21.15.133:0/2563977854 wait complete. 2023-11-27T15:21:45.560 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:45.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:45 smithi177 ceph-mon[131504]: pgmap v311: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:21:45.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.592+0000 7fc53fe34700 1 -- 172.21.15.133:0/24340307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc538100ee0 msgr2=0x7fc538104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:45.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.592+0000 7fc53fe34700 1 --2- 172.21.15.133:0/24340307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc538100ee0 0x7fc538104eb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc5280099c0 tx=0x7fc528009cd0 comp rx=0 tx=0).stop 2023-11-27T15:21:45.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.592+0000 7fc53fe34700 1 -- 172.21.15.133:0/24340307 shutdown_connections 2023-11-27T15:21:45.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.592+0000 7fc53fe34700 1 --2- 172.21.15.133:0/24340307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc5381055e0 0x7fc53810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:45.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.592+0000 7fc53fe34700 1 --2- 172.21.15.133:0/24340307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc538100540 0x7fc538100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:45.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.592+0000 7fc53fe34700 1 --2- 172.21.15.133:0/24340307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc538100ee0 0x7fc538104eb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:45.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.592+0000 7fc53fe34700 1 -- 172.21.15.133:0/24340307 >> 172.21.15.133:0/24340307 conn(0x7fc5380fbdc0 msgr2=0x7fc5380fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:45.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.593+0000 7fc53fe34700 1 -- 172.21.15.133:0/24340307 shutdown_connections 2023-11-27T15:21:45.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.593+0000 7fc53fe34700 1 -- 172.21.15.133:0/24340307 wait complete. 2023-11-27T15:21:45.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.594+0000 7fc53fe34700 1 Processor -- start 2023-11-27T15:21:45.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.594+0000 7fc53fe34700 1 -- start start 2023-11-27T15:21:45.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.594+0000 7fc53fe34700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc538100540 0x7fc5381985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:45.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.594+0000 7fc53fe34700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc538100ee0 0x7fc538198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:45.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.594+0000 7fc53fe34700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5381055e0 0x7fc53819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:45.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.594+0000 7fc53fe34700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53819d580 con 0x7fc538100540 2023-11-27T15:21:45.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.594+0000 7fc53fe34700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53819d6f0 con 0x7fc5381055e0 2023-11-27T15:21:45.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.594+0000 7fc53fe34700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53819d8b0 con 0x7fc538100ee0 2023-11-27T15:21:45.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.594+0000 7fc53dbd0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc538100540 0x7fc5381985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:45.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.594+0000 7fc53d3cf700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc538100ee0 0x7fc538198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:45.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.595+0000 7fc53dbd0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc538100540 0x7fc5381985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60244/0 (socket says 172.21.15.133:60244) 2023-11-27T15:21:45.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.595+0000 7fc53dbd0700 1 -- 172.21.15.133:0/2260483400 learned_addr learned my addr 172.21.15.133:0/2260483400 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:45.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.595+0000 7fc53d3cf700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc538100ee0 0x7fc538198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:35368/0 (socket says 172.21.15.133:35368) 2023-11-27T15:21:45.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.595+0000 7fc53e3d1700 1 --2- 172.21.15.133:0/2260483400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5381055e0 0x7fc53819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:45.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.595+0000 7fc53e3d1700 1 -- 172.21.15.133:0/2260483400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc538100ee0 msgr2=0x7fc538198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:45.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.595+0000 7fc53e3d1700 1 --2- 172.21.15.133:0/2260483400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc538100ee0 0x7fc538198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:45.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.595+0000 7fc53e3d1700 1 -- 172.21.15.133:0/2260483400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc538100540 msgr2=0x7fc5381985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:45.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.595+0000 7fc53e3d1700 1 --2- 172.21.15.133:0/2260483400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc538100540 0x7fc5381985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:45.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.595+0000 7fc53e3d1700 1 -- 172.21.15.133:0/2260483400 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5280095f0 con 0x7fc5381055e0 2023-11-27T15:21:45.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.595+0000 7fc53e3d1700 1 --2- 172.21.15.133:0/2260483400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5381055e0 0x7fc53819cf70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc53000efc0 tx=0x7fc53000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:45.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.596+0000 7fc52effd700 1 -- 172.21.15.133:0/2260483400 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc530018070 con 0x7fc5381055e0 2023-11-27T15:21:45.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.596+0000 7fc52effd700 1 -- 172.21.15.133:0/2260483400 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc53000f040 con 0x7fc5381055e0 2023-11-27T15:21:45.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.596+0000 7fc52effd700 1 -- 172.21.15.133:0/2260483400 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53000cd40 con 0x7fc5381055e0 2023-11-27T15:21:45.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.596+0000 7fc53fe34700 1 -- 172.21.15.133:0/2260483400 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc538110360 con 0x7fc5381055e0 2023-11-27T15:21:45.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.596+0000 7fc53fe34700 1 -- 172.21.15.133:0/2260483400 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5381101d0 con 0x7fc5381055e0 2023-11-27T15:21:45.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.598+0000 7fc53fe34700 1 -- 172.21.15.133:0/2260483400 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5381104f0 con 0x7fc5381055e0 2023-11-27T15:21:45.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.600+0000 7fc52effd700 1 -- 172.21.15.133:0/2260483400 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fc530003a00 con 0x7fc5381055e0 2023-11-27T15:21:45.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.601+0000 7fc52effd700 1 --2- 172.21.15.133:0/2260483400 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc5240a1260 0x7fc5240a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:45.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.602+0000 7fc52effd700 1 -- 172.21.15.133:0/2260483400 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc5300cc860 con 0x7fc5381055e0 2023-11-27T15:21:45.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.602+0000 7fc53dbd0700 1 --2- 172.21.15.133:0/2260483400 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc5240a1260 0x7fc5240a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:45.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.603+0000 7fc53dbd0700 1 --2- 172.21.15.133:0/2260483400 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc5240a1260 0x7fc5240a3710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc534009d20 tx=0x7fc534009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:45.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.605+0000 7fc52effd700 1 -- 172.21.15.133:0/2260483400 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc530095bd0 con 0x7fc5381055e0 2023-11-27T15:21:45.652 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:45 smithi174 ceph-mon[131110]: pgmap v311: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:21:45.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.848+0000 7fc53fe34700 1 -- 172.21.15.133:0/2260483400 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7fc538066ea0 con 0x7fc5381055e0 2023-11-27T15:21:45.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.849+0000 7fc52effd700 1 -- 172.21.15.133:0/2260483400 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc538066ea0 con 0x7fc5381055e0 2023-11-27T15:21:45.853 INFO:teuthology.orchestra.run.smithi133.stdout:283467841569 2023-11-27T15:21:45.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.852+0000 7fc53fe34700 1 -- 172.21.15.133:0/2260483400 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc5240a1260 msgr2=0x7fc5240a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:45.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.852+0000 7fc53fe34700 1 --2- 172.21.15.133:0/2260483400 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc5240a1260 0x7fc5240a3710 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc534009d20 tx=0x7fc534009260 comp rx=0 tx=0).stop 2023-11-27T15:21:45.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.852+0000 7fc53fe34700 1 -- 172.21.15.133:0/2260483400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5381055e0 msgr2=0x7fc53819cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:45.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.852+0000 7fc53fe34700 1 --2- 172.21.15.133:0/2260483400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5381055e0 0x7fc53819cf70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc53000efc0 tx=0x7fc53000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:21:45.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.852+0000 7fc53fe34700 1 -- 172.21.15.133:0/2260483400 shutdown_connections 2023-11-27T15:21:45.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.852+0000 7fc53fe34700 1 --2- 172.21.15.133:0/2260483400 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc5240a1260 0x7fc5240a3710 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:45.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.852+0000 7fc53fe34700 1 --2- 172.21.15.133:0/2260483400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5381055e0 0x7fc53819cf70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:45.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.852+0000 7fc53fe34700 1 --2- 172.21.15.133:0/2260483400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc538100540 0x7fc5381985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:45.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.852+0000 7fc53fe34700 1 --2- 172.21.15.133:0/2260483400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc538100ee0 0x7fc538198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:45.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.852+0000 7fc53fe34700 1 -- 172.21.15.133:0/2260483400 >> 172.21.15.133:0/2260483400 conn(0x7fc5380fbdc0 msgr2=0x7fc53810b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:45.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.852+0000 7fc53fe34700 1 -- 172.21.15.133:0/2260483400 shutdown_connections 2023-11-27T15:21:45.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:45.852+0000 7fc53fe34700 1 -- 172.21.15.133:0/2260483400 wait complete. 2023-11-27T15:21:46.518 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:46 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2563977854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-11-27T15:21:46.518 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:46 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2260483400' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-11-27T15:21:46.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:46 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2563977854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-11-27T15:21:46.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:46 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2260483400' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-11-27T15:21:46.626 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678043 got 304942678046 for osd.10 2023-11-27T15:21:46.626 DEBUG:teuthology.parallel:result is None 2023-11-27T15:21:46.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:46 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2563977854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-11-27T15:21:46.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:46 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2260483400' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-11-27T15:21:47.273 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841567 got 283467841569 for osd.9 2023-11-27T15:21:47.273 DEBUG:teuthology.parallel:result is None 2023-11-27T15:21:47.393 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:47 smithi133 ceph-mon[97860]: pgmap v312: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:21:47.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:47 smithi177 ceph-mon[131504]: pgmap v312: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:21:47.653 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:47 smithi174 ceph-mon[131110]: pgmap v312: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:21:47.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.429+0000 7fcef8fb7700 1 -- 172.21.15.133:0/1676881236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcef4100540 msgr2=0x7fcef4100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:47.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.429+0000 7fcef8fb7700 1 --2- 172.21.15.133:0/1676881236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcef4100540 0x7fcef4100910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcee40099c0 tx=0x7fcee4009cd0 comp rx=0 tx=0).stop 2023-11-27T15:21:47.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.429+0000 7fcef8fb7700 1 -- 172.21.15.133:0/1676881236 shutdown_connections 2023-11-27T15:21:47.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.429+0000 7fcef8fb7700 1 --2- 172.21.15.133:0/1676881236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcef41055e0 0x7fcef410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:47.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.429+0000 7fcef8fb7700 1 --2- 172.21.15.133:0/1676881236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcef4100540 0x7fcef4100910 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:47.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.429+0000 7fcef8fb7700 1 --2- 172.21.15.133:0/1676881236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcef4100ee0 0x7fcef4104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:47.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.429+0000 7fcef8fb7700 1 -- 172.21.15.133:0/1676881236 >> 172.21.15.133:0/1676881236 conn(0x7fcef40fbdc0 msgr2=0x7fcef40fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:47.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.429+0000 7fcef8fb7700 1 -- 172.21.15.133:0/1676881236 shutdown_connections 2023-11-27T15:21:47.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.429+0000 7fcef8fb7700 1 -- 172.21.15.133:0/1676881236 wait complete. 2023-11-27T15:21:47.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.430+0000 7fcef8fb7700 1 Processor -- start 2023-11-27T15:21:47.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.431+0000 7fcef8fb7700 1 -- start start 2023-11-27T15:21:47.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.431+0000 7fcef8fb7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcef4100540 0x7fcef4072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:47.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.431+0000 7fcef8fb7700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcef4100ee0 0x7fcef4073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:47.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.431+0000 7fcef8fb7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcef41055e0 0x7fcef40738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:47.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.431+0000 7fcef8fb7700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef41a4810 con 0x7fcef4100ee0 2023-11-27T15:21:47.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.431+0000 7fcef8fb7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef41a4980 con 0x7fcef41055e0 2023-11-27T15:21:47.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.431+0000 7fcef8fb7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef41a4b40 con 0x7fcef4100540 2023-11-27T15:21:47.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.431+0000 7fcef1d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcef4100ee0 0x7fcef4073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:47.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.432+0000 7fcef1d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcef4100ee0 0x7fcef4073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60272/0 (socket says 172.21.15.133:60272) 2023-11-27T15:21:47.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.432+0000 7fcef1d9b700 1 -- 172.21.15.133:0/2360889649 learned_addr learned my addr 172.21.15.133:0/2360889649 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:47.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.432+0000 7fcef2d9d700 1 --2- 172.21.15.133:0/2360889649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcef41055e0 0x7fcef40738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:47.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.432+0000 7fcef259c700 1 --2- 172.21.15.133:0/2360889649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcef4100540 0x7fcef4072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:47.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.432+0000 7fcef1d9b700 1 -- 172.21.15.133:0/2360889649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcef4100540 msgr2=0x7fcef4072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:47.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.432+0000 7fcef1d9b700 1 --2- 172.21.15.133:0/2360889649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcef4100540 0x7fcef4072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:47.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.432+0000 7fcef1d9b700 1 -- 172.21.15.133:0/2360889649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcef41055e0 msgr2=0x7fcef40738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:47.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.432+0000 7fcef1d9b700 1 --2- 172.21.15.133:0/2360889649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcef41055e0 0x7fcef40738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:47.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.432+0000 7fcef1d9b700 1 -- 172.21.15.133:0/2360889649 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee40095f0 con 0x7fcef4100ee0 2023-11-27T15:21:47.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.432+0000 7fcef2d9d700 1 --2- 172.21.15.133:0/2360889649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcef41055e0 0x7fcef40738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:47.789 INFO:teuthology.orchestra.run.smithi133.stdout:163208757303 2023-11-27T15:21:47.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.432+0000 7fcef1d9b700 1 --2- 172.21.15.133:0/2360889649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcef4100ee0 0x7fcef4073330 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcee800e970 tx=0x7fcee800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:47.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.433+0000 7fcee37fe700 1 -- 172.21.15.133:0/2360889649 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee8009790 con 0x7fcef4100ee0 2023-11-27T15:21:47.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.433+0000 7fcef8fb7700 1 -- 172.21.15.133:0/2360889649 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef4110320 con 0x7fcef4100ee0 2023-11-27T15:21:47.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.433+0000 7fcee37fe700 1 -- 172.21.15.133:0/2360889649 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcee800c7e0 con 0x7fcef4100ee0 2023-11-27T15:21:47.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.433+0000 7fcee37fe700 1 -- 172.21.15.133:0/2360889649 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee80054e0 con 0x7fcef4100ee0 2023-11-27T15:21:47.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.433+0000 7fcef8fb7700 1 -- 172.21.15.133:0/2360889649 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef4110190 con 0x7fcef4100ee0 2023-11-27T15:21:47.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.434+0000 7fcef8fb7700 1 -- 172.21.15.133:0/2360889649 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef41104b0 con 0x7fcef4100ee0 2023-11-27T15:21:47.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.436+0000 7fcee37fe700 1 -- 172.21.15.133:0/2360889649 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fcee800c950 con 0x7fcef4100ee0 2023-11-27T15:21:47.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.437+0000 7fcee37fe700 1 --2- 172.21.15.133:0/2360889649 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcedc0a12e0 0x7fcedc0a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:47.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.438+0000 7fcee37fe700 1 -- 172.21.15.133:0/2360889649 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcee8014070 con 0x7fcef4100ee0 2023-11-27T15:21:47.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.439+0000 7fcef259c700 1 --2- 172.21.15.133:0/2360889649 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcedc0a12e0 0x7fcedc0a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:47.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.440+0000 7fcef259c700 1 --2- 172.21.15.133:0/2360889649 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcedc0a12e0 0x7fcedc0a3790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009fd0 tx=0x7fcee4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:47.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.445+0000 7fcee37fe700 1 -- 172.21.15.133:0/2360889649 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcee80962c0 con 0x7fcef4100ee0 2023-11-27T15:21:47.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.697+0000 7fcef8fb7700 1 -- 172.21.15.133:0/2360889649 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fcef4066ea0 con 0x7fcef4100ee0 2023-11-27T15:21:47.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.698+0000 7fcee37fe700 1 -- 172.21.15.133:0/2360889649 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcee8016330 con 0x7fcef4100ee0 2023-11-27T15:21:47.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.700+0000 7fcef8fb7700 1 -- 172.21.15.133:0/2360889649 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcedc0a12e0 msgr2=0x7fcedc0a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:47.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.700+0000 7fcef8fb7700 1 --2- 172.21.15.133:0/2360889649 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcedc0a12e0 0x7fcedc0a3790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcee4009fd0 tx=0x7fcee4005fb0 comp rx=0 tx=0).stop 2023-11-27T15:21:47.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.700+0000 7fcef8fb7700 1 -- 172.21.15.133:0/2360889649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcef4100ee0 msgr2=0x7fcef4073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:47.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.700+0000 7fcef8fb7700 1 --2- 172.21.15.133:0/2360889649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcef4100ee0 0x7fcef4073330 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcee800e970 tx=0x7fcee800ec80 comp rx=0 tx=0).stop 2023-11-27T15:21:47.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.702+0000 7fcef8fb7700 1 -- 172.21.15.133:0/2360889649 shutdown_connections 2023-11-27T15:21:47.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.702+0000 7fcef8fb7700 1 --2- 172.21.15.133:0/2360889649 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcedc0a12e0 0x7fcedc0a3790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:47.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.702+0000 7fcef8fb7700 1 --2- 172.21.15.133:0/2360889649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcef41055e0 0x7fcef40738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:47.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.702+0000 7fcef8fb7700 1 --2- 172.21.15.133:0/2360889649 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcef4100540 0x7fcef4072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:47.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.702+0000 7fcef8fb7700 1 --2- 172.21.15.133:0/2360889649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcef4100ee0 0x7fcef4073330 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:47.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.702+0000 7fcef8fb7700 1 -- 172.21.15.133:0/2360889649 >> 172.21.15.133:0/2360889649 conn(0x7fcef40fbdc0 msgr2=0x7fcef40fc880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:47.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.702+0000 7fcef8fb7700 1 -- 172.21.15.133:0/2360889649 shutdown_connections 2023-11-27T15:21:47.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:47.702+0000 7fcef8fb7700 1 -- 172.21.15.133:0/2360889649 wait complete. 2023-11-27T15:21:48.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:48 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2360889649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-11-27T15:21:48.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:48 smithi133 ceph-mon[97860]: pgmap v313: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-27T15:21:48.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.799+0000 7fe92c1b9700 1 -- 172.21.15.133:0/3880679127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe924102f50 msgr2=0x7fe92410f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:48.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.799+0000 7fe92c1b9700 1 --2- 172.21.15.133:0/3880679127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe924102f50 0x7fe92410f3d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00b0a0 tx=0x7fe91c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:21:48.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.799+0000 7fe92c1b9700 1 -- 172.21.15.133:0/3880679127 shutdown_connections 2023-11-27T15:21:48.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.799+0000 7fe92c1b9700 1 --2- 172.21.15.133:0/3880679127 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe924102f50 0x7fe92410f3d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:48.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.799+0000 7fe92c1b9700 1 --2- 172.21.15.133:0/3880679127 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9241086b0 0x7fe924108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:48.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.799+0000 7fe92c1b9700 1 --2- 172.21.15.133:0/3880679127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9241025c0 0x7fe924102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:48.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.799+0000 7fe92c1b9700 1 -- 172.21.15.133:0/3880679127 >> 172.21.15.133:0/3880679127 conn(0x7fe9240fe100 msgr2=0x7fe924100510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:48.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.799+0000 7fe92c1b9700 1 -- 172.21.15.133:0/3880679127 shutdown_connections 2023-11-27T15:21:48.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.799+0000 7fe92c1b9700 1 -- 172.21.15.133:0/3880679127 wait complete. 2023-11-27T15:21:48.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.800+0000 7fe92c1b9700 1 Processor -- start 2023-11-27T15:21:48.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.800+0000 7fe92c1b9700 1 -- start start 2023-11-27T15:21:48.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.801+0000 7fe92c1b9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9241025c0 0x7fe924198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:48.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.801+0000 7fe92c1b9700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe924102f50 0x7fe924198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:48.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.801+0000 7fe92c1b9700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9241086b0 0x7fe92419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:48.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.801+0000 7fe92c1b9700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92419d620 con 0x7fe924102f50 2023-11-27T15:21:48.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.801+0000 7fe92c1b9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92419d790 con 0x7fe9241025c0 2023-11-27T15:21:48.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.801+0000 7fe92c1b9700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92419d950 con 0x7fe9241086b0 2023-11-27T15:21:48.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.801+0000 7fe929754700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe924102f50 0x7fe924198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:48.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.801+0000 7fe929754700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe924102f50 0x7fe924198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60294/0 (socket says 172.21.15.133:60294) 2023-11-27T15:21:48.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.801+0000 7fe929754700 1 -- 172.21.15.133:0/1724725754 learned_addr learned my addr 172.21.15.133:0/1724725754 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:48.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.801+0000 7fe929f55700 1 --2- 172.21.15.133:0/1724725754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9241025c0 0x7fe924198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:48.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.801+0000 7fe92a756700 1 --2- 172.21.15.133:0/1724725754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9241086b0 0x7fe92419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:48.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.802+0000 7fe929f55700 1 -- 172.21.15.133:0/1724725754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9241086b0 msgr2=0x7fe92419d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:48.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.802+0000 7fe929f55700 1 --2- 172.21.15.133:0/1724725754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9241086b0 0x7fe92419d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:48.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.802+0000 7fe929f55700 1 -- 172.21.15.133:0/1724725754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe924102f50 msgr2=0x7fe924198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:48.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.802+0000 7fe929f55700 1 --2- 172.21.15.133:0/1724725754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe924102f50 0x7fe924198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:48.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.802+0000 7fe929f55700 1 -- 172.21.15.133:0/1724725754 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe91c009d00 con 0x7fe9241025c0 2023-11-27T15:21:48.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.802+0000 7fe92a756700 1 --2- 172.21.15.133:0/1724725754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9241086b0 0x7fe92419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:48.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.802+0000 7fe929f55700 1 --2- 172.21.15.133:0/1724725754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9241025c0 0x7fe924198660 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe92000d750 tx=0x7fe92000da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:48.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.802+0000 7fe91affd700 1 -- 172.21.15.133:0/1724725754 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe920004020 con 0x7fe9241025c0 2023-11-27T15:21:48.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.802+0000 7fe92c1b9700 1 -- 172.21.15.133:0/1724725754 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9241125d0 con 0x7fe9241025c0 2023-11-27T15:21:48.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.802+0000 7fe91affd700 1 -- 172.21.15.133:0/1724725754 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe920004830 con 0x7fe9241025c0 2023-11-27T15:21:48.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.802+0000 7fe91affd700 1 -- 172.21.15.133:0/1724725754 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9200054c0 con 0x7fe9241025c0 2023-11-27T15:21:48.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.803+0000 7fe92c1b9700 1 -- 172.21.15.133:0/1724725754 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe924112440 con 0x7fe9241025c0 2023-11-27T15:21:48.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.804+0000 7fe92c1b9700 1 -- 172.21.15.133:0/1724725754 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe924112760 con 0x7fe9241025c0 2023-11-27T15:21:48.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.805+0000 7fe91affd700 1 -- 172.21.15.133:0/1724725754 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fe9200049a0 con 0x7fe9241025c0 2023-11-27T15:21:48.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.807+0000 7fe91affd700 1 --2- 172.21.15.133:0/1724725754 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe9100a11e0 0x7fe9100a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:48.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.807+0000 7fe91affd700 1 -- 172.21.15.133:0/1724725754 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe9200cc6e0 con 0x7fe9241025c0 2023-11-27T15:21:48.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.807+0000 7fe929754700 1 --2- 172.21.15.133:0/1724725754 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe9100a11e0 0x7fe9100a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:48.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.807+0000 7fe929754700 1 --2- 172.21.15.133:0/1724725754 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe9100a11e0 0x7fe9100a3690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe924199c80 tx=0x7fe914009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:48.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:48.810+0000 7fe91affd700 1 -- 172.21.15.133:0/1724725754 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe920095960 con 0x7fe9241025c0 2023-11-27T15:21:48.944 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757300 got 163208757303 for osd.5 2023-11-27T15:21:48.945 DEBUG:teuthology.parallel:result is None 2023-11-27T15:21:49.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.074+0000 7fe92c1b9700 1 -- 172.21.15.133:0/1724725754 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe9241a5740 con 0x7fe9241025c0 2023-11-27T15:21:49.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.075+0000 7fe91affd700 1 -- 172.21.15.133:0/1724725754 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe920098f80 con 0x7fe9241025c0 2023-11-27T15:21:49.079 INFO:teuthology.orchestra.run.smithi133.stdout:55834574919 2023-11-27T15:21:49.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.077+0000 7fe92c1b9700 1 -- 172.21.15.133:0/1724725754 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe9100a11e0 msgr2=0x7fe9100a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:49.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.077+0000 7fe92c1b9700 1 --2- 172.21.15.133:0/1724725754 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe9100a11e0 0x7fe9100a3690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe924199c80 tx=0x7fe914009260 comp rx=0 tx=0).stop 2023-11-27T15:21:49.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.077+0000 7fe92c1b9700 1 -- 172.21.15.133:0/1724725754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9241025c0 msgr2=0x7fe924198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:49.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.077+0000 7fe92c1b9700 1 --2- 172.21.15.133:0/1724725754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9241025c0 0x7fe924198660 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe92000d750 tx=0x7fe92000da60 comp rx=0 tx=0).stop 2023-11-27T15:21:49.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.077+0000 7fe92c1b9700 1 -- 172.21.15.133:0/1724725754 shutdown_connections 2023-11-27T15:21:49.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.077+0000 7fe92c1b9700 1 --2- 172.21.15.133:0/1724725754 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe9100a11e0 0x7fe9100a3690 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:49.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.077+0000 7fe92c1b9700 1 --2- 172.21.15.133:0/1724725754 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9241086b0 0x7fe92419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:49.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.077+0000 7fe92c1b9700 1 --2- 172.21.15.133:0/1724725754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9241025c0 0x7fe924198660 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe92000d750 tx=0x7fe92000da60 comp rx=0 tx=0).stop 2023-11-27T15:21:49.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.077+0000 7fe92c1b9700 1 --2- 172.21.15.133:0/1724725754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe924102f50 0x7fe924198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:49.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.077+0000 7fe92c1b9700 1 -- 172.21.15.133:0/1724725754 >> 172.21.15.133:0/1724725754 conn(0x7fe9240fe100 msgr2=0x7fe9240ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:49.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.077+0000 7fe92c1b9700 1 -- 172.21.15.133:0/1724725754 shutdown_connections 2023-11-27T15:21:49.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.077+0000 7fe92c1b9700 1 -- 172.21.15.133:0/1724725754 wait complete. 2023-11-27T15:21:49.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:48 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2360889649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-11-27T15:21:49.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:48 smithi177 ceph-mon[131504]: pgmap v313: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-27T15:21:49.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:48 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2360889649' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-11-27T15:21:49.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:48 smithi174 ceph-mon[131110]: pgmap v313: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 6 op/s 2023-11-27T15:21:49.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.223+0000 7f8256432700 1 -- 172.21.15.133:0/2100849644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8250100540 msgr2=0x7f8250100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:49.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.223+0000 7f8256432700 1 --2- 172.21.15.133:0/2100849644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8250100540 0x7f8250100910 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8240009920 tx=0x7f8240009c30 comp rx=0 tx=0).stop 2023-11-27T15:21:49.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.223+0000 7f8256432700 1 -- 172.21.15.133:0/2100849644 shutdown_connections 2023-11-27T15:21:49.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.223+0000 7f8256432700 1 --2- 172.21.15.133:0/2100849644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82501055e0 0x7f825010d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:49.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.223+0000 7f8256432700 1 --2- 172.21.15.133:0/2100849644 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8250100540 0x7f8250100910 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:49.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.223+0000 7f8256432700 1 --2- 172.21.15.133:0/2100849644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8250100ee0 0x7f8250104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:49.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.223+0000 7f8256432700 1 -- 172.21.15.133:0/2100849644 >> 172.21.15.133:0/2100849644 conn(0x7f82500fbdc0 msgr2=0x7f82500fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:49.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.223+0000 7f8256432700 1 -- 172.21.15.133:0/2100849644 shutdown_connections 2023-11-27T15:21:49.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.223+0000 7f8256432700 1 -- 172.21.15.133:0/2100849644 wait complete. 2023-11-27T15:21:49.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.224+0000 7f8256432700 1 Processor -- start 2023-11-27T15:21:49.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.224+0000 7f8256432700 1 -- start start 2023-11-27T15:21:49.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.225+0000 7f8256432700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8250100540 0x7f8250072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:49.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.225+0000 7f8256432700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8250100ee0 0x7f8250073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:49.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.225+0000 7f8256432700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82501055e0 0x7f82500738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:49.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.225+0000 7f8256432700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82501a49a0 con 0x7f8250100540 2023-11-27T15:21:49.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.225+0000 7f8256432700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82501a4b10 con 0x7f8250100ee0 2023-11-27T15:21:49.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.225+0000 7f8256432700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82501a4cd0 con 0x7f82501055e0 2023-11-27T15:21:49.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.225+0000 7f824ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8250100540 0x7f8250072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:49.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.225+0000 7f824ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8250100540 0x7f8250072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60312/0 (socket says 172.21.15.133:60312) 2023-11-27T15:21:49.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.225+0000 7f824ffff700 1 -- 172.21.15.133:0/1870726452 learned_addr learned my addr 172.21.15.133:0/1870726452 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:49.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.225+0000 7f82549cf700 1 --2- 172.21.15.133:0/1870726452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82501055e0 0x7f82500738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:49.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.225+0000 7f824f7fe700 1 --2- 172.21.15.133:0/1870726452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8250100ee0 0x7f8250073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:49.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.225+0000 7f824ffff700 1 -- 172.21.15.133:0/1870726452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82501055e0 msgr2=0x7f82500738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:49.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.225+0000 7f824ffff700 1 --2- 172.21.15.133:0/1870726452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82501055e0 0x7f82500738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:49.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.225+0000 7f824ffff700 1 -- 172.21.15.133:0/1870726452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8250100ee0 msgr2=0x7f8250073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:49.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.225+0000 7f824ffff700 1 --2- 172.21.15.133:0/1870726452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8250100ee0 0x7f8250073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:49.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.225+0000 7f824ffff700 1 -- 172.21.15.133:0/1870726452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8240009520 con 0x7f8250100540 2023-11-27T15:21:49.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.225+0000 7f82549cf700 1 --2- 172.21.15.133:0/1870726452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82501055e0 0x7f82500738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:21:49.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.225+0000 7f824f7fe700 1 --2- 172.21.15.133:0/1870726452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8250100ee0 0x7f8250073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:21:49.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.226+0000 7f824ffff700 1 --2- 172.21.15.133:0/1870726452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8250100540 0x7f8250072df0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8240009840 tx=0x7f824000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:49.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.226+0000 7f824d7fa700 1 -- 172.21.15.133:0/1870726452 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824001c070 con 0x7f8250100540 2023-11-27T15:21:49.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.226+0000 7f8256432700 1 -- 172.21.15.133:0/1870726452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8250110370 con 0x7f8250100540 2023-11-27T15:21:49.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.226+0000 7f824d7fa700 1 -- 172.21.15.133:0/1870726452 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8240005040 con 0x7f8250100540 2023-11-27T15:21:49.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.226+0000 7f8256432700 1 -- 172.21.15.133:0/1870726452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82501101e0 con 0x7f8250100540 2023-11-27T15:21:49.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.226+0000 7f824d7fa700 1 -- 172.21.15.133:0/1870726452 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8240003940 con 0x7f8250100540 2023-11-27T15:21:49.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.227+0000 7f8256432700 1 -- 172.21.15.133:0/1870726452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8250110500 con 0x7f8250100540 2023-11-27T15:21:49.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.229+0000 7f824d7fa700 1 -- 172.21.15.133:0/1870726452 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f8240020030 con 0x7f8250100540 2023-11-27T15:21:49.234 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.230+0000 7f824d7fa700 1 --2- 172.21.15.133:0/1870726452 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f82380a11e0 0x7f82380a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:49.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.231+0000 7f824d7fa700 1 -- 172.21.15.133:0/1870726452 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f82400cde20 con 0x7f8250100540 2023-11-27T15:21:49.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.231+0000 7f824f7fe700 1 --2- 172.21.15.133:0/1870726452 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f82380a11e0 0x7f82380a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:49.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.232+0000 7f824f7fe700 1 --2- 172.21.15.133:0/1870726452 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f82380a11e0 0x7f82380a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f82500745e0 tx=0x7f8244009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:49.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.235+0000 7f824d7fa700 1 -- 172.21.15.133:0/1870726452 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8240097130 con 0x7f8250100540 2023-11-27T15:21:49.459 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.454+0000 7f8256432700 1 -- 172.21.15.133:0/1870726452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f8250074430 con 0x7f8250100540 2023-11-27T15:21:49.459 INFO:teuthology.orchestra.run.smithi133.stdout:219043332140 2023-11-27T15:21:49.460 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.455+0000 7f824d7fa700 1 -- 172.21.15.133:0/1870726452 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f824009a750 con 0x7f8250100540 2023-11-27T15:21:49.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.457+0000 7f8256432700 1 -- 172.21.15.133:0/1870726452 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f82380a11e0 msgr2=0x7f82380a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:49.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.457+0000 7f8256432700 1 --2- 172.21.15.133:0/1870726452 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f82380a11e0 0x7f82380a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f82500745e0 tx=0x7f8244009310 comp rx=0 tx=0).stop 2023-11-27T15:21:49.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.457+0000 7f8256432700 1 -- 172.21.15.133:0/1870726452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8250100540 msgr2=0x7f8250072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:49.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.457+0000 7f8256432700 1 --2- 172.21.15.133:0/1870726452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8250100540 0x7f8250072df0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8240009840 tx=0x7f824000bf60 comp rx=0 tx=0).stop 2023-11-27T15:21:49.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.457+0000 7f8256432700 1 -- 172.21.15.133:0/1870726452 shutdown_connections 2023-11-27T15:21:49.463 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.457+0000 7f8256432700 1 --2- 172.21.15.133:0/1870726452 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f82380a11e0 0x7f82380a3690 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:49.463 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.457+0000 7f8256432700 1 --2- 172.21.15.133:0/1870726452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f82501055e0 0x7f82500738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:49.463 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.457+0000 7f8256432700 1 --2- 172.21.15.133:0/1870726452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8250100540 0x7f8250072df0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:49.463 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.457+0000 7f8256432700 1 --2- 172.21.15.133:0/1870726452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8250100ee0 0x7f8250073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:49.463 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.457+0000 7f8256432700 1 -- 172.21.15.133:0/1870726452 >> 172.21.15.133:0/1870726452 conn(0x7f82500fbdc0 msgr2=0x7f82500fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:49.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.458+0000 7f8256432700 1 -- 172.21.15.133:0/1870726452 shutdown_connections 2023-11-27T15:21:49.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.458+0000 7f8256432700 1 -- 172.21.15.133:0/1870726452 wait complete. 2023-11-27T15:21:49.706 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574917 got 55834574919 for osd.1 2023-11-27T15:21:49.707 DEBUG:teuthology.parallel:result is None 2023-11-27T15:21:49.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.929+0000 7fb8fdef8700 1 -- 172.21.15.133:0/157120950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8f8100540 msgr2=0x7fb8f8100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:49.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.929+0000 7fb8fdef8700 1 --2- 172.21.15.133:0/157120950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8f8100540 0x7fb8f8100910 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb8e80099c0 tx=0x7fb8e8009cd0 comp rx=0 tx=0).stop 2023-11-27T15:21:49.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.929+0000 7fb8fdef8700 1 -- 172.21.15.133:0/157120950 shutdown_connections 2023-11-27T15:21:49.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.929+0000 7fb8fdef8700 1 --2- 172.21.15.133:0/157120950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8f81055e0 0x7fb8f810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:49.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.929+0000 7fb8fdef8700 1 --2- 172.21.15.133:0/157120950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8f8100540 0x7fb8f8100910 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:49.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.929+0000 7fb8fdef8700 1 --2- 172.21.15.133:0/157120950 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8f8100ee0 0x7fb8f8104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:49.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.929+0000 7fb8fdef8700 1 -- 172.21.15.133:0/157120950 >> 172.21.15.133:0/157120950 conn(0x7fb8f80fbdc0 msgr2=0x7fb8f80fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:49.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.930+0000 7fb8fdef8700 1 -- 172.21.15.133:0/157120950 shutdown_connections 2023-11-27T15:21:49.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.930+0000 7fb8fdef8700 1 -- 172.21.15.133:0/157120950 wait complete. 2023-11-27T15:21:49.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.931+0000 7fb8fdef8700 1 Processor -- start 2023-11-27T15:21:49.937 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.931+0000 7fb8fdef8700 1 -- start start 2023-11-27T15:21:49.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.932+0000 7fb8fdef8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8f8100540 0x7fb8f81963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:49.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.932+0000 7fb8fdef8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8f8100ee0 0x7fb8f8196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:49.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.932+0000 7fb8fdef8700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8f81055e0 0x7fb8f819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:49.938 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.932+0000 7fb8fdef8700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f819b3b0 con 0x7fb8f81055e0 2023-11-27T15:21:49.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.932+0000 7fb8fdef8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f819b520 con 0x7fb8f8100ee0 2023-11-27T15:21:49.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.932+0000 7fb8fdef8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f819b6e0 con 0x7fb8f8100540 2023-11-27T15:21:49.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.932+0000 7fb8f7fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8f81055e0 0x7fb8f819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:49.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.932+0000 7fb8f7fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8f81055e0 0x7fb8f819ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60326/0 (socket says 172.21.15.133:60326) 2023-11-27T15:21:49.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.932+0000 7fb8f7fff700 1 -- 172.21.15.133:0/2631677902 learned_addr learned my addr 172.21.15.133:0/2631677902 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:49.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.932+0000 7fb8f77fe700 1 --2- 172.21.15.133:0/2631677902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8f8100540 0x7fb8f81963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:49.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.932+0000 7fb8f6ffd700 1 --2- 172.21.15.133:0/2631677902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8f8100ee0 0x7fb8f8196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:49.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.933+0000 7fb8f7fff700 1 -- 172.21.15.133:0/2631677902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8f8100540 msgr2=0x7fb8f81963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:49.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.933+0000 7fb8f7fff700 1 --2- 172.21.15.133:0/2631677902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8f8100540 0x7fb8f81963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:49.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.933+0000 7fb8f7fff700 1 -- 172.21.15.133:0/2631677902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8f8100ee0 msgr2=0x7fb8f8196930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:49.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.933+0000 7fb8f7fff700 1 --2- 172.21.15.133:0/2631677902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8f8100ee0 0x7fb8f8196930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:49.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.933+0000 7fb8f7fff700 1 -- 172.21.15.133:0/2631677902 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e80095f0 con 0x7fb8f81055e0 2023-11-27T15:21:49.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.933+0000 7fb8f7fff700 1 --2- 172.21.15.133:0/2631677902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8f81055e0 0x7fb8f819ada0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb8f000ef30 tx=0x7fb8f000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:49.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.933+0000 7fb8f4ff9700 1 -- 172.21.15.133:0/2631677902 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0018070 con 0x7fb8f81055e0 2023-11-27T15:21:49.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.933+0000 7fb8f4ff9700 1 -- 172.21.15.133:0/2631677902 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8f000f040 con 0x7fb8f81055e0 2023-11-27T15:21:49.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.933+0000 7fb8f4ff9700 1 -- 172.21.15.133:0/2631677902 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f000cd00 con 0x7fb8f81055e0 2023-11-27T15:21:49.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.933+0000 7fb8fdef8700 1 -- 172.21.15.133:0/2631677902 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f8110370 con 0x7fb8f81055e0 2023-11-27T15:21:49.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.933+0000 7fb8fdef8700 1 -- 172.21.15.133:0/2631677902 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f81101e0 con 0x7fb8f81055e0 2023-11-27T15:21:49.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.934+0000 7fb8fdef8700 1 -- 172.21.15.133:0/2631677902 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f8110500 con 0x7fb8f81055e0 2023-11-27T15:21:49.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.936+0000 7fb8f4ff9700 1 -- 172.21.15.133:0/2631677902 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fb8f0003a00 con 0x7fb8f81055e0 2023-11-27T15:21:49.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.938+0000 7fb8f4ff9700 1 --2- 172.21.15.133:0/2631677902 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb8e00a1260 0x7fb8e00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:49.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.938+0000 7fb8f4ff9700 1 -- 172.21.15.133:0/2631677902 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb8f00cd1d0 con 0x7fb8f81055e0 2023-11-27T15:21:49.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.938+0000 7fb8f77fe700 1 --2- 172.21.15.133:0/2631677902 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb8e00a1260 0x7fb8e00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:49.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.939+0000 7fb8f77fe700 1 --2- 172.21.15.133:0/2631677902 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb8e00a1260 0x7fb8e00a3710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8009990 tx=0x7fb8e8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:49.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:49.940+0000 7fb8f4ff9700 1 -- 172.21.15.133:0/2631677902 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb8f00964e0 con 0x7fb8f81055e0 2023-11-27T15:21:50.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:49 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1724725754' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-11-27T15:21:50.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:49 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:21:50.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:49 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:21:50.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:49 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1870726452' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-11-27T15:21:50.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:49 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1724725754' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-11-27T15:21:50.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:49 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:21:50.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:49 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:21:50.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:49 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1870726452' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-11-27T15:21:50.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:49 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1724725754' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-11-27T15:21:50.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:49 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:21:50.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:49 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:21:50.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:49 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1870726452' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-11-27T15:21:50.180 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332137 got 219043332140 for osd.7 2023-11-27T15:21:50.180 DEBUG:teuthology.parallel:result is None 2023-11-27T15:21:50.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.183+0000 7fb8fdef8700 1 -- 172.21.15.133:0/2631677902 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fb8f8197750 con 0x7fb8f81055e0 2023-11-27T15:21:50.187 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.184+0000 7fb8f4ff9700 1 -- 172.21.15.133:0/2631677902 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb8f0099b00 con 0x7fb8f81055e0 2023-11-27T15:21:50.188 INFO:teuthology.orchestra.run.smithi133.stdout:249108103208 2023-11-27T15:21:50.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.186+0000 7fb8fdef8700 1 -- 172.21.15.133:0/2631677902 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb8e00a1260 msgr2=0x7fb8e00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:50.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.186+0000 7fb8fdef8700 1 --2- 172.21.15.133:0/2631677902 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb8e00a1260 0x7fb8e00a3710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb8e8009990 tx=0x7fb8e8005fb0 comp rx=0 tx=0).stop 2023-11-27T15:21:50.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.186+0000 7fb8fdef8700 1 -- 172.21.15.133:0/2631677902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8f81055e0 msgr2=0x7fb8f819ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:50.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.186+0000 7fb8fdef8700 1 --2- 172.21.15.133:0/2631677902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8f81055e0 0x7fb8f819ada0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb8f000ef30 tx=0x7fb8f000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:21:50.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.186+0000 7fb8fdef8700 1 -- 172.21.15.133:0/2631677902 shutdown_connections 2023-11-27T15:21:50.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.186+0000 7fb8fdef8700 1 --2- 172.21.15.133:0/2631677902 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb8e00a1260 0x7fb8e00a3710 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:50.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.186+0000 7fb8fdef8700 1 --2- 172.21.15.133:0/2631677902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8f81055e0 0x7fb8f819ada0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:50.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.186+0000 7fb8fdef8700 1 --2- 172.21.15.133:0/2631677902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8f8100540 0x7fb8f81963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:50.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.186+0000 7fb8fdef8700 1 --2- 172.21.15.133:0/2631677902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb8f8100ee0 0x7fb8f8196930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:50.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.186+0000 7fb8fdef8700 1 -- 172.21.15.133:0/2631677902 >> 172.21.15.133:0/2631677902 conn(0x7fb8f80fbdc0 msgr2=0x7fb8f80fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:50.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.186+0000 7fb8fdef8700 1 -- 172.21.15.133:0/2631677902 shutdown_connections 2023-11-27T15:21:50.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.186+0000 7fb8fdef8700 1 -- 172.21.15.133:0/2631677902 wait complete. 2023-11-27T15:21:50.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.310+0000 7f29623c0700 1 -- 172.21.15.133:0/2512917846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f295c100d00 msgr2=0x7f295c104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.310+0000 7f29623c0700 1 --2- 172.21.15.133:0/2512917846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f295c100d00 0x7f295c104cd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f294c0099c0 tx=0x7f294c009cd0 comp rx=0 tx=0).stop 2023-11-27T15:21:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.311+0000 7f29623c0700 1 -- 172.21.15.133:0/2512917846 shutdown_connections 2023-11-27T15:21:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.311+0000 7f29623c0700 1 --2- 172.21.15.133:0/2512917846 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f295c105400 0x7f295c10cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.311+0000 7f29623c0700 1 --2- 172.21.15.133:0/2512917846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f295c100360 0x7f295c100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.311+0000 7f29623c0700 1 --2- 172.21.15.133:0/2512917846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f295c100d00 0x7f295c104cd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.311+0000 7f29623c0700 1 -- 172.21.15.133:0/2512917846 >> 172.21.15.133:0/2512917846 conn(0x7f295c0fbc40 msgr2=0x7f295c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.311+0000 7f29623c0700 1 -- 172.21.15.133:0/2512917846 shutdown_connections 2023-11-27T15:21:50.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.311+0000 7f29623c0700 1 -- 172.21.15.133:0/2512917846 wait complete. 2023-11-27T15:21:50.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.312+0000 7f29623c0700 1 Processor -- start 2023-11-27T15:21:50.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.312+0000 7f29623c0700 1 -- start start 2023-11-27T15:21:50.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.312+0000 7f29623c0700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f295c100360 0x7f295c072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:50.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.312+0000 7f29623c0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f295c100d00 0x7f295c073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:50.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.312+0000 7f29623c0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f295c105400 0x7f295c1a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:50.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.312+0000 7f29623c0700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c073c20 con 0x7f295c100360 2023-11-27T15:21:50.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.312+0000 7f29623c0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c1a47d0 con 0x7f295c105400 2023-11-27T15:21:50.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.313+0000 7f29623c0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c1a4940 con 0x7f295c100d00 2023-11-27T15:21:50.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.313+0000 7f29613be700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f295c100360 0x7f295c072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:50.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.313+0000 7f2960bbd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f295c100d00 0x7f295c073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:50.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.313+0000 7f2961bbf700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f295c105400 0x7f295c1a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:50.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.313+0000 7f29613be700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f295c100360 0x7f295c072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60348/0 (socket says 172.21.15.133:60348) 2023-11-27T15:21:50.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.313+0000 7f2961bbf700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f295c105400 0x7f295c1a4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:55328/0 (socket says 172.21.15.133:55328) 2023-11-27T15:21:50.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.313+0000 7f29613be700 1 -- 172.21.15.133:0/2822418081 learned_addr learned my addr 172.21.15.133:0/2822418081 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:50.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.313+0000 7f2960bbd700 1 -- 172.21.15.133:0/2822418081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f295c105400 msgr2=0x7f295c1a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:50.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.313+0000 7f2960bbd700 1 --2- 172.21.15.133:0/2822418081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f295c105400 0x7f295c1a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:50.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.313+0000 7f2960bbd700 1 -- 172.21.15.133:0/2822418081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f295c100360 msgr2=0x7f295c072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:50.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.314+0000 7f2960bbd700 1 --2- 172.21.15.133:0/2822418081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f295c100360 0x7f295c072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:50.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.314+0000 7f2960bbd700 1 -- 172.21.15.133:0/2822418081 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2958009520 con 0x7f295c100d00 2023-11-27T15:21:50.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.314+0000 7f29613be700 1 --2- 172.21.15.133:0/2822418081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f295c100360 0x7f295c072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:21:50.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.315+0000 7f2960bbd700 1 --2- 172.21.15.133:0/2822418081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f295c100d00 0x7f295c073330 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f294c00b580 tx=0x7f294c00ba40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:50.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.315+0000 7f29527fc700 1 -- 172.21.15.133:0/2822418081 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c005220 con 0x7f295c100d00 2023-11-27T15:21:50.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.315+0000 7f29527fc700 1 -- 172.21.15.133:0/2822418081 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f294c00be80 con 0x7f295c100d00 2023-11-27T15:21:50.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.315+0000 7f29623c0700 1 -- 172.21.15.133:0/2822418081 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294c0095f0 con 0x7f295c100d00 2023-11-27T15:21:50.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.315+0000 7f29527fc700 1 -- 172.21.15.133:0/2822418081 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c00f6c0 con 0x7f295c100d00 2023-11-27T15:21:50.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.316+0000 7f29623c0700 1 -- 172.21.15.133:0/2822418081 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295c110130 con 0x7f295c100d00 2023-11-27T15:21:50.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.317+0000 7f29623c0700 1 -- 172.21.15.133:0/2822418081 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f295c10ffa0 con 0x7f295c100d00 2023-11-27T15:21:50.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.319+0000 7f29527fc700 1 -- 172.21.15.133:0/2822418081 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f294c0048a0 con 0x7f295c100d00 2023-11-27T15:21:50.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.320+0000 7f29527fc700 1 --2- 172.21.15.133:0/2822418081 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f29480a1310 0x7f29480a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:50.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.320+0000 7f29527fc700 1 -- 172.21.15.133:0/2822418081 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f294c0d6da0 con 0x7f295c100d00 2023-11-27T15:21:50.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.321+0000 7f29613be700 1 --2- 172.21.15.133:0/2822418081 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f29480a1310 0x7f29480a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:50.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.322+0000 7f29613be700 1 --2- 172.21.15.133:0/2822418081 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f29480a1310 0x7f29480a37c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2958009fd0 tx=0x7f2958009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:50.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.322+0000 7f29527fc700 1 -- 172.21.15.133:0/2822418081 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f294c0a00b0 con 0x7f295c100d00 2023-11-27T15:21:50.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.569+0000 7f29623c0700 1 -- 172.21.15.133:0/2822418081 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f295c1102c0 con 0x7f295c100d00 2023-11-27T15:21:50.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.570+0000 7f29527fc700 1 -- 172.21.15.133:0/2822418081 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f294c0a36d0 con 0x7f295c100d00 2023-11-27T15:21:50.573 INFO:teuthology.orchestra.run.smithi133.stdout:107374182462 2023-11-27T15:21:50.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.572+0000 7f29623c0700 1 -- 172.21.15.133:0/2822418081 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f29480a1310 msgr2=0x7f29480a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:50.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.572+0000 7f29623c0700 1 --2- 172.21.15.133:0/2822418081 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f29480a1310 0x7f29480a37c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2958009fd0 tx=0x7f2958009260 comp rx=0 tx=0).stop 2023-11-27T15:21:50.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.572+0000 7f29623c0700 1 -- 172.21.15.133:0/2822418081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f295c100d00 msgr2=0x7f295c073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:50.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.572+0000 7f29623c0700 1 --2- 172.21.15.133:0/2822418081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f295c100d00 0x7f295c073330 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f294c00b580 tx=0x7f294c00ba40 comp rx=0 tx=0).stop 2023-11-27T15:21:50.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.572+0000 7f29623c0700 1 -- 172.21.15.133:0/2822418081 shutdown_connections 2023-11-27T15:21:50.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.572+0000 7f29623c0700 1 --2- 172.21.15.133:0/2822418081 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f29480a1310 0x7f29480a37c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:50.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.572+0000 7f29623c0700 1 --2- 172.21.15.133:0/2822418081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f295c105400 0x7f295c1a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:50.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.572+0000 7f29623c0700 1 --2- 172.21.15.133:0/2822418081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f295c100360 0x7f295c072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:50.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.572+0000 7f29623c0700 1 --2- 172.21.15.133:0/2822418081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f295c100d00 0x7f295c073330 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:50.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.572+0000 7f29623c0700 1 -- 172.21.15.133:0/2822418081 >> 172.21.15.133:0/2822418081 conn(0x7f295c0fbc40 msgr2=0x7f295c0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:50.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.573+0000 7f29623c0700 1 -- 172.21.15.133:0/2822418081 shutdown_connections 2023-11-27T15:21:50.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:50.573+0000 7f29623c0700 1 -- 172.21.15.133:0/2822418081 wait complete. 2023-11-27T15:21:50.726 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103206 got 249108103208 for osd.8 2023-11-27T15:21:50.726 DEBUG:teuthology.parallel:result is None 2023-11-27T15:21:51.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:50 smithi133 ceph-mon[97860]: pgmap v314: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:21:51.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:50 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2631677902' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-11-27T15:21:51.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:50 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2822418081' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-11-27T15:21:51.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:50 smithi177 ceph-mon[131504]: pgmap v314: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:21:51.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:50 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2631677902' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-11-27T15:21:51.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:50 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2822418081' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-11-27T15:21:51.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:50 smithi174 ceph-mon[131110]: pgmap v314: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:21:51.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:50 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2631677902' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-11-27T15:21:51.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:50 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2822418081' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-11-27T15:21:51.203 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182459 got 107374182462 for osd.3 2023-11-27T15:21:51.203 DEBUG:teuthology.parallel:result is None 2023-11-27T15:21:51.203 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-11-27T15:21:51.203 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph pg dump --format=json 2023-11-27T15:21:52.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:51 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:52.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:51 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:21:52.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:51 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:21:52.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:51 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:21:52.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:51 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:21:52.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:51 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:21:52.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:51 smithi133 ceph-mon[97860]: Service mds.all has no related daemon already placed. 2023-11-27T15:21:52.048 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:52.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:51 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:52.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:51 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:21:52.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:51 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:21:52.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:51 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:21:52.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:51 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:21:52.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:51 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:21:52.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:51 smithi177 ceph-mon[131504]: Service mds.all has no related daemon already placed. 2023-11-27T15:21:52.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:51 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:21:52.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:51 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:21:52.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:51 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:21:52.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:51 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:21:52.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:51 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:21:52.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:51 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:21:52.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:51 smithi174 ceph-mon[131110]: Service mds.all has no related daemon already placed. 2023-11-27T15:21:52.955 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:52 smithi133 ceph-mon[97860]: pgmap v315: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:21:53.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:52 smithi177 ceph-mon[131504]: pgmap v315: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:21:53.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:52 smithi174 ceph-mon[131110]: pgmap v315: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:21:53.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.591+0000 7fe527a76700 1 -- 172.21.15.133:0/3321522036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe520100540 msgr2=0x7fe520100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:53.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.591+0000 7fe527a76700 1 --2- 172.21.15.133:0/3321522036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe520100540 0x7fe520100910 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0099c0 tx=0x7fe51c009cd0 comp rx=0 tx=0).stop 2023-11-27T15:21:53.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.591+0000 7fe527a76700 1 -- 172.21.15.133:0/3321522036 shutdown_connections 2023-11-27T15:21:53.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.591+0000 7fe527a76700 1 --2- 172.21.15.133:0/3321522036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5201055e0 0x7fe52010d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:53.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.591+0000 7fe527a76700 1 --2- 172.21.15.133:0/3321522036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe520100540 0x7fe520100910 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:53.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.591+0000 7fe527a76700 1 --2- 172.21.15.133:0/3321522036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe520100ee0 0x7fe520104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:53.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.591+0000 7fe527a76700 1 -- 172.21.15.133:0/3321522036 >> 172.21.15.133:0/3321522036 conn(0x7fe5200fbdc0 msgr2=0x7fe5200fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:53.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.591+0000 7fe527a76700 1 -- 172.21.15.133:0/3321522036 shutdown_connections 2023-11-27T15:21:53.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.591+0000 7fe527a76700 1 -- 172.21.15.133:0/3321522036 wait complete. 2023-11-27T15:21:53.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.592+0000 7fe527a76700 1 Processor -- start 2023-11-27T15:21:53.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.592+0000 7fe527a76700 1 -- start start 2023-11-27T15:21:53.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.593+0000 7fe527a76700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe520100540 0x7fe520072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:53.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.593+0000 7fe527a76700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe520100ee0 0x7fe520073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:53.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.593+0000 7fe527a76700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5201055e0 0x7fe5200738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:53.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.593+0000 7fe527a76700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5201a4810 con 0x7fe520100540 2023-11-27T15:21:53.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.593+0000 7fe527a76700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5201a4980 con 0x7fe520100ee0 2023-11-27T15:21:53.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.593+0000 7fe527a76700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5201a4b40 con 0x7fe5201055e0 2023-11-27T15:21:53.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.593+0000 7fe525812700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe520100540 0x7fe520072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:53.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.593+0000 7fe525812700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe520100540 0x7fe520072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40078/0 (socket says 172.21.15.133:40078) 2023-11-27T15:21:53.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.593+0000 7fe525812700 1 -- 172.21.15.133:0/4081175645 learned_addr learned my addr 172.21.15.133:0/4081175645 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:53.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.593+0000 7fe526013700 1 --2- 172.21.15.133:0/4081175645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5201055e0 0x7fe5200738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:53.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.593+0000 7fe525011700 1 --2- 172.21.15.133:0/4081175645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe520100ee0 0x7fe520073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:53.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.593+0000 7fe525812700 1 -- 172.21.15.133:0/4081175645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5201055e0 msgr2=0x7fe5200738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:53.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.593+0000 7fe525812700 1 --2- 172.21.15.133:0/4081175645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5201055e0 0x7fe5200738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:53.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.593+0000 7fe525812700 1 -- 172.21.15.133:0/4081175645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe520100ee0 msgr2=0x7fe520073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:53.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.593+0000 7fe525812700 1 --2- 172.21.15.133:0/4081175645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe520100ee0 0x7fe520073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:53.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.593+0000 7fe525812700 1 -- 172.21.15.133:0/4081175645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe510009520 con 0x7fe520100540 2023-11-27T15:21:53.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.593+0000 7fe526013700 1 --2- 172.21.15.133:0/4081175645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5201055e0 0x7fe5200738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:21:53.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.594+0000 7fe525812700 1 --2- 172.21.15.133:0/4081175645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe520100540 0x7fe520072df0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe51c000c00 tx=0x7fe51c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:53.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.594+0000 7fe516ffd700 1 -- 172.21.15.133:0/4081175645 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c01d070 con 0x7fe520100540 2023-11-27T15:21:53.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.594+0000 7fe527a76700 1 -- 172.21.15.133:0/4081175645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51c0095f0 con 0x7fe520100540 2023-11-27T15:21:53.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.594+0000 7fe516ffd700 1 -- 172.21.15.133:0/4081175645 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe51c005510 con 0x7fe520100540 2023-11-27T15:21:53.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.594+0000 7fe516ffd700 1 -- 172.21.15.133:0/4081175645 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c004b40 con 0x7fe520100540 2023-11-27T15:21:53.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.594+0000 7fe527a76700 1 -- 172.21.15.133:0/4081175645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe520110320 con 0x7fe520100540 2023-11-27T15:21:53.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.595+0000 7fe527a76700 1 -- 172.21.15.133:0/4081175645 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe520110190 con 0x7fe520100540 2023-11-27T15:21:53.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.597+0000 7fe516ffd700 1 -- 172.21.15.133:0/4081175645 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fe51c004cc0 con 0x7fe520100540 2023-11-27T15:21:53.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.598+0000 7fe516ffd700 1 --2- 172.21.15.133:0/4081175645 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe50c0a11e0 0x7fe50c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:53.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.598+0000 7fe516ffd700 1 -- 172.21.15.133:0/4081175645 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe51c0cdba0 con 0x7fe520100540 2023-11-27T15:21:53.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.599+0000 7fe525011700 1 --2- 172.21.15.133:0/4081175645 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe50c0a11e0 0x7fe50c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:53.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.599+0000 7fe525011700 1 --2- 172.21.15.133:0/4081175645 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe50c0a11e0 0x7fe50c0a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe5200745e0 tx=0x7fe510009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:53.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.602+0000 7fe516ffd700 1 -- 172.21.15.133:0/4081175645 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe51c096eb0 con 0x7fe520100540 2023-11-27T15:21:53.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.799+0000 7fe527a76700 1 -- 172.21.15.133:0/4081175645 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5201104b0 con 0x7fe50c0a11e0 2023-11-27T15:21:53.816 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.811+0000 7fe516ffd700 1 -- 172.21.15.133:0/4081175645 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426855 (secure 0 0 0) 0x7fe5201104b0 con 0x7fe50c0a11e0 2023-11-27T15:21:53.823 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:21:53.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.816+0000 7fe527a76700 1 -- 172.21.15.133:0/4081175645 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe50c0a11e0 msgr2=0x7fe50c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:53.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.816+0000 7fe527a76700 1 --2- 172.21.15.133:0/4081175645 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe50c0a11e0 0x7fe50c0a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe5200745e0 tx=0x7fe510009260 comp rx=0 tx=0).stop 2023-11-27T15:21:53.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.816+0000 7fe527a76700 1 -- 172.21.15.133:0/4081175645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe520100540 msgr2=0x7fe520072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:53.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.816+0000 7fe527a76700 1 --2- 172.21.15.133:0/4081175645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe520100540 0x7fe520072df0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe51c000c00 tx=0x7fe51c00bfd0 comp rx=0 tx=0).stop 2023-11-27T15:21:53.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.817+0000 7fe527a76700 1 -- 172.21.15.133:0/4081175645 shutdown_connections 2023-11-27T15:21:53.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.817+0000 7fe527a76700 1 --2- 172.21.15.133:0/4081175645 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe50c0a11e0 0x7fe50c0a3690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:53.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.817+0000 7fe527a76700 1 --2- 172.21.15.133:0/4081175645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5201055e0 0x7fe5200738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:53.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.817+0000 7fe527a76700 1 --2- 172.21.15.133:0/4081175645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe520100540 0x7fe520072df0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:53.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.817+0000 7fe527a76700 1 --2- 172.21.15.133:0/4081175645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe520100ee0 0x7fe520073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:53.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.817+0000 7fe527a76700 1 -- 172.21.15.133:0/4081175645 >> 172.21.15.133:0/4081175645 conn(0x7fe5200fbdc0 msgr2=0x7fe5200fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:53.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.817+0000 7fe527a76700 1 -- 172.21.15.133:0/4081175645 shutdown_connections 2023-11-27T15:21:53.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:53.817+0000 7fe527a76700 1 -- 172.21.15.133:0/4081175645 wait complete. 2023-11-27T15:21:53.830 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2023-11-27T15:21:54.342 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2023-11-27T15:21:53.729264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654326,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":432,"num_read_kb":332,"num_write":341,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":83032,"kb_used_data":13880,"kb_used_omap":0,"kb_used_meta":69056,"kb_avail":1124858792,"statfs":{"total":1151940427776,"available":1151855403008,"internally_reserved":0,"allocated":14213120,"data_stored":7874181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":70713344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003565"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.933701+0000","last_change":"2023-11-27T15:20:33.236352+0000","last_active":"2023-11-27T15:21:02.933701+0000","last_peered":"2023-11-27T15:21:02.933701+0000","last_clean":"2023-11-27T15:21:02.933701+0000","last_became_active":"2023-11-27T15:20:33.235974+0000","last_became_peered":"2023-11-27T15:20:33.235974+0000","last_unstale":"2023-11-27T15:21:02.933701+0000","last_undegraded":"2023-11-27T15:21:02.933701+0000","last_fullsized":"2023-11-27T15:21:02.933701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:42:55.142804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786021+0000","last_change":"2023-11-27T15:20:33.234882+0000","last_active":"2023-11-27T15:21:02.786021+0000","last_peered":"2023-11-27T15:21:02.786021+0000","last_clean":"2023-11-27T15:21:02.786021+0000","last_became_active":"2023-11-27T15:20:33.231062+0000","last_became_peered":"2023-11-27T15:20:33.231062+0000","last_unstale":"2023-11-27T15:21:02.786021+0000","last_undegraded":"2023-11-27T15:21:02.786021+0000","last_fullsized":"2023-11-27T15:21:02.786021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:18:27.469196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785364+0000","last_change":"2023-11-27T15:20:33.238280+0000","last_active":"2023-11-27T15:21:02.785364+0000","last_peered":"2023-11-27T15:21:02.785364+0000","last_clean":"2023-11-27T15:21:02.785364+0000","last_became_active":"2023-11-27T15:20:33.237602+0000","last_became_peered":"2023-11-27T15:20:33.237602+0000","last_unstale":"2023-11-27T15:21:02.785364+0000","last_undegraded":"2023-11-27T15:21:02.785364+0000","last_fullsized":"2023-11-27T15:21:02.785364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:28:48.211669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784641+0000","last_change":"2023-11-27T15:20:33.217909+0000","last_active":"2023-11-27T15:21:02.784641+0000","last_peered":"2023-11-27T15:21:02.784641+0000","last_clean":"2023-11-27T15:21:02.784641+0000","last_became_active":"2023-11-27T15:20:33.217511+0000","last_became_peered":"2023-11-27T15:20:33.217511+0000","last_unstale":"2023-11-27T15:21:02.784641+0000","last_undegraded":"2023-11-27T15:21:02.784641+0000","last_fullsized":"2023-11-27T15:21:02.784641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:37:24.759082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782641+0000","last_change":"2023-11-27T15:20:33.246630+0000","last_active":"2023-11-27T15:21:02.782641+0000","last_peered":"2023-11-27T15:21:02.782641+0000","last_clean":"2023-11-27T15:21:02.782641+0000","last_became_active":"2023-11-27T15:20:33.246249+0000","last_became_peered":"2023-11-27T15:20:33.246249+0000","last_unstale":"2023-11-27T15:21:02.782641+0000","last_undegraded":"2023-11-27T15:21:02.782641+0000","last_fullsized":"2023-11-27T15:21:02.782641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:14:48.110743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932766+0000","last_change":"2023-11-27T15:20:33.236280+0000","last_active":"2023-11-27T15:21:02.932766+0000","last_peered":"2023-11-27T15:21:02.932766+0000","last_clean":"2023-11-27T15:21:02.932766+0000","last_became_active":"2023-11-27T15:20:33.235983+0000","last_became_peered":"2023-11-27T15:20:33.235983+0000","last_unstale":"2023-11-27T15:21:02.932766+0000","last_undegraded":"2023-11-27T15:21:02.932766+0000","last_fullsized":"2023-11-27T15:21:02.932766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:32:27.538426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779828+0000","last_change":"2023-11-27T15:20:33.223085+0000","last_active":"2023-11-27T15:21:02.779828+0000","last_peered":"2023-11-27T15:21:02.779828+0000","last_clean":"2023-11-27T15:21:02.779828+0000","last_became_active":"2023-11-27T15:20:33.222809+0000","last_became_peered":"2023-11-27T15:20:33.222809+0000","last_unstale":"2023-11-27T15:21:02.779828+0000","last_undegraded":"2023-11-27T15:21:02.779828+0000","last_fullsized":"2023-11-27T15:21:02.779828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:46:23.567184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783786+0000","last_change":"2023-11-27T15:20:33.226217+0000","last_active":"2023-11-27T15:21:02.783786+0000","last_peered":"2023-11-27T15:21:02.783786+0000","last_clean":"2023-11-27T15:21:02.783786+0000","last_became_active":"2023-11-27T15:20:33.225856+0000","last_became_peered":"2023-11-27T15:20:33.225856+0000","last_unstale":"2023-11-27T15:21:02.783786+0000","last_undegraded":"2023-11-27T15:21:02.783786+0000","last_fullsized":"2023-11-27T15:21:02.783786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:41:31.773939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.795641+0000","last_change":"2023-11-27T15:20:33.227554+0000","last_active":"2023-11-27T15:21:02.795641+0000","last_peered":"2023-11-27T15:21:02.795641+0000","last_clean":"2023-11-27T15:21:02.795641+0000","last_became_active":"2023-11-27T15:20:33.227222+0000","last_became_peered":"2023-11-27T15:20:33.227222+0000","last_unstale":"2023-11-27T15:21:02.795641+0000","last_undegraded":"2023-11-27T15:21:02.795641+0000","last_fullsized":"2023-11-27T15:21:02.795641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:34:15.927242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.933040+0000","last_change":"2023-11-27T15:20:33.230528+0000","last_active":"2023-11-27T15:21:02.933040+0000","last_peered":"2023-11-27T15:21:02.933040+0000","last_clean":"2023-11-27T15:21:02.933040+0000","last_became_active":"2023-11-27T15:20:33.230148+0000","last_became_peered":"2023-11-27T15:20:33.230148+0000","last_unstale":"2023-11-27T15:21:02.933040+0000","last_undegraded":"2023-11-27T15:21:02.933040+0000","last_fullsized":"2023-11-27T15:21:02.933040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:56:46.117728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780265+0000","last_change":"2023-11-27T15:20:33.228098+0000","last_active":"2023-11-27T15:21:02.780265+0000","last_peered":"2023-11-27T15:21:02.780265+0000","last_clean":"2023-11-27T15:21:02.780265+0000","last_became_active":"2023-11-27T15:20:33.227760+0000","last_became_peered":"2023-11-27T15:20:33.227760+0000","last_unstale":"2023-11-27T15:21:02.780265+0000","last_undegraded":"2023-11-27T15:21:02.780265+0000","last_fullsized":"2023-11-27T15:21:02.780265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:27:54.894445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.935458+0000","last_change":"2023-11-27T15:21:02.934583+0000","last_active":"2023-11-27T15:21:02.935458+0000","last_peered":"2023-11-27T15:21:02.935458+0000","last_clean":"2023-11-27T15:21:02.935458+0000","last_became_active":"2023-11-27T15:21:02.933631+0000","last_became_peered":"2023-11-27T15:21:02.933631+0000","last_unstale":"2023-11-27T15:21:02.935458+0000","last_undegraded":"2023-11-27T15:21:02.935458+0000","last_fullsized":"2023-11-27T15:21:02.935458+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:45:15.679810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783578+0000","last_change":"2023-11-27T15:20:33.231255+0000","last_active":"2023-11-27T15:21:02.783578+0000","last_peered":"2023-11-27T15:21:02.783578+0000","last_clean":"2023-11-27T15:21:02.783578+0000","last_became_active":"2023-11-27T15:20:33.230867+0000","last_became_peered":"2023-11-27T15:20:33.230867+0000","last_unstale":"2023-11-27T15:21:02.783578+0000","last_undegraded":"2023-11-27T15:21:02.783578+0000","last_fullsized":"2023-11-27T15:21:02.783578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:58:03.899781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785072+0000","last_change":"2023-11-27T15:20:33.230306+0000","last_active":"2023-11-27T15:21:02.785072+0000","last_peered":"2023-11-27T15:21:02.785072+0000","last_clean":"2023-11-27T15:21:02.785072+0000","last_became_active":"2023-11-27T15:20:33.229892+0000","last_became_peered":"2023-11-27T15:20:33.229892+0000","last_unstale":"2023-11-27T15:21:02.785072+0000","last_undegraded":"2023-11-27T15:21:02.785072+0000","last_fullsized":"2023-11-27T15:21:02.785072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:24:55.365713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785143+0000","last_change":"2023-11-27T15:20:33.220592+0000","last_active":"2023-11-27T15:21:02.785143+0000","last_peered":"2023-11-27T15:21:02.785143+0000","last_clean":"2023-11-27T15:21:02.785143+0000","last_became_active":"2023-11-27T15:20:33.220258+0000","last_became_peered":"2023-11-27T15:20:33.220258+0000","last_unstale":"2023-11-27T15:21:02.785143+0000","last_undegraded":"2023-11-27T15:21:02.785143+0000","last_fullsized":"2023-11-27T15:21:02.785143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:48:48.921465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.794878+0000","last_change":"2023-11-27T15:20:33.245041+0000","last_active":"2023-11-27T15:21:02.794878+0000","last_peered":"2023-11-27T15:21:02.794878+0000","last_clean":"2023-11-27T15:21:02.794878+0000","last_became_active":"2023-11-27T15:20:33.244759+0000","last_became_peered":"2023-11-27T15:20:33.244759+0000","last_unstale":"2023-11-27T15:21:02.794878+0000","last_undegraded":"2023-11-27T15:21:02.794878+0000","last_fullsized":"2023-11-27T15:21:02.794878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:31:00.164868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.934087+0000","last_change":"2023-11-27T15:20:33.243288+0000","last_active":"2023-11-27T15:21:02.934087+0000","last_peered":"2023-11-27T15:21:02.934087+0000","last_clean":"2023-11-27T15:21:02.934087+0000","last_became_active":"2023-11-27T15:20:33.242989+0000","last_became_peered":"2023-11-27T15:20:33.242989+0000","last_unstale":"2023-11-27T15:21:02.934087+0000","last_undegraded":"2023-11-27T15:21:02.934087+0000","last_fullsized":"2023-11-27T15:21:02.934087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:25:53.302075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779978+0000","last_change":"2023-11-27T15:20:33.229613+0000","last_active":"2023-11-27T15:21:02.779978+0000","last_peered":"2023-11-27T15:21:02.779978+0000","last_clean":"2023-11-27T15:21:02.779978+0000","last_became_active":"2023-11-27T15:20:33.227747+0000","last_became_peered":"2023-11-27T15:20:33.227747+0000","last_unstale":"2023-11-27T15:21:02.779978+0000","last_undegraded":"2023-11-27T15:21:02.779978+0000","last_fullsized":"2023-11-27T15:21:02.779978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:18:24.422450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780242+0000","last_change":"2023-11-27T15:20:33.227004+0000","last_active":"2023-11-27T15:21:02.780242+0000","last_peered":"2023-11-27T15:21:02.780242+0000","last_clean":"2023-11-27T15:21:02.780242+0000","last_became_active":"2023-11-27T15:20:33.226703+0000","last_became_peered":"2023-11-27T15:20:33.226703+0000","last_unstale":"2023-11-27T15:21:02.780242+0000","last_undegraded":"2023-11-27T15:21:02.780242+0000","last_fullsized":"2023-11-27T15:21:02.780242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:18:04.531443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779865+0000","last_change":"2023-11-27T15:20:37.682021+0000","last_active":"2023-11-27T15:21:02.779865+0000","last_peered":"2023-11-27T15:21:02.779865+0000","last_clean":"2023-11-27T15:21:02.779865+0000","last_became_active":"2023-11-27T15:20:37.681600+0000","last_became_peered":"2023-11-27T15:20:37.681600+0000","last_unstale":"2023-11-27T15:21:02.779865+0000","last_undegraded":"2023-11-27T15:21:02.779865+0000","last_fullsized":"2023-11-27T15:21:02.779865+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:27:41.540353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.801451+0000","last_change":"2023-11-27T15:20:32.657542+0000","last_active":"2023-11-27T15:21:03.801451+0000","last_peered":"2023-11-27T15:21:03.801451+0000","last_clean":"2023-11-27T15:21:03.801451+0000","last_became_active":"2023-11-27T15:20:32.657158+0000","last_became_peered":"2023-11-27T15:20:32.657158+0000","last_unstale":"2023-11-27T15:21:03.801451+0000","last_undegraded":"2023-11-27T15:21:03.801451+0000","last_fullsized":"2023-11-27T15:21:03.801451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:01:39.181861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.935289+0000","last_change":"2023-11-27T15:21:02.933684+0000","last_active":"2023-11-27T15:21:02.935289+0000","last_peered":"2023-11-27T15:21:02.935289+0000","last_clean":"2023-11-27T15:21:02.935289+0000","last_became_active":"2023-11-27T15:21:02.933215+0000","last_became_peered":"2023-11-27T15:21:02.933215+0000","last_unstale":"2023-11-27T15:21:02.935289+0000","last_undegraded":"2023-11-27T15:21:02.935289+0000","last_fullsized":"2023-11-27T15:21:02.935289+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:59:39.553465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783397+0000","last_change":"2023-11-27T15:20:37.666705+0000","last_active":"2023-11-27T15:21:02.783397+0000","last_peered":"2023-11-27T15:21:02.783397+0000","last_clean":"2023-11-27T15:21:02.783397+0000","last_became_active":"2023-11-27T15:20:37.666236+0000","last_became_peered":"2023-11-27T15:20:37.666236+0000","last_unstale":"2023-11-27T15:21:02.783397+0000","last_undegraded":"2023-11-27T15:21:02.783397+0000","last_fullsized":"2023-11-27T15:21:02.783397+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:49:31.365357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785700+0000","last_change":"2023-11-27T15:20:32.663915+0000","last_active":"2023-11-27T15:21:02.785700+0000","last_peered":"2023-11-27T15:21:02.785700+0000","last_clean":"2023-11-27T15:21:02.785700+0000","last_became_active":"2023-11-27T15:20:32.663069+0000","last_became_peered":"2023-11-27T15:20:32.663069+0000","last_unstale":"2023-11-27T15:21:02.785700+0000","last_undegraded":"2023-11-27T15:21:02.785700+0000","last_fullsized":"2023-11-27T15:21:02.785700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:15:25.522191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785737+0000","last_change":"2023-11-27T15:20:33.238150+0000","last_active":"2023-11-27T15:21:02.785737+0000","last_peered":"2023-11-27T15:21:02.785737+0000","last_clean":"2023-11-27T15:21:02.785737+0000","last_became_active":"2023-11-27T15:20:33.236972+0000","last_became_peered":"2023-11-27T15:20:33.236972+0000","last_unstale":"2023-11-27T15:21:02.785737+0000","last_undegraded":"2023-11-27T15:21:02.785737+0000","last_fullsized":"2023-11-27T15:21:02.785737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:32:03.275170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783667+0000","last_change":"2023-11-27T15:20:37.668031+0000","last_active":"2023-11-27T15:21:02.783667+0000","last_peered":"2023-11-27T15:21:02.783667+0000","last_clean":"2023-11-27T15:21:02.783667+0000","last_became_active":"2023-11-27T15:20:37.667640+0000","last_became_peered":"2023-11-27T15:20:37.667640+0000","last_unstale":"2023-11-27T15:21:02.783667+0000","last_undegraded":"2023-11-27T15:21:02.783667+0000","last_fullsized":"2023-11-27T15:21:02.783667+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:45:18.290503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.800461+0000","last_change":"2023-11-27T15:20:32.655738+0000","last_active":"2023-11-27T15:21:03.800461+0000","last_peered":"2023-11-27T15:21:03.800461+0000","last_clean":"2023-11-27T15:21:03.800461+0000","last_became_active":"2023-11-27T15:20:32.655439+0000","last_became_peered":"2023-11-27T15:20:32.655439+0000","last_unstale":"2023-11-27T15:21:03.800461+0000","last_undegraded":"2023-11-27T15:21:03.800461+0000","last_fullsized":"2023-11-27T15:21:03.800461+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:24:02.616850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.795136+0000","last_change":"2023-11-27T15:20:33.229267+0000","last_active":"2023-11-27T15:21:02.795136+0000","last_peered":"2023-11-27T15:21:02.795136+0000","last_clean":"2023-11-27T15:21:02.795136+0000","last_became_active":"2023-11-27T15:20:33.228746+0000","last_became_peered":"2023-11-27T15:20:33.228746+0000","last_unstale":"2023-11-27T15:21:02.795136+0000","last_undegraded":"2023-11-27T15:21:02.795136+0000","last_fullsized":"2023-11-27T15:21:02.795136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:55:36.391108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783571+0000","last_change":"2023-11-27T15:20:37.671893+0000","last_active":"2023-11-27T15:21:02.783571+0000","last_peered":"2023-11-27T15:21:02.783571+0000","last_clean":"2023-11-27T15:21:02.783571+0000","last_became_active":"2023-11-27T15:20:37.671033+0000","last_became_peered":"2023-11-27T15:20:37.671033+0000","last_unstale":"2023-11-27T15:21:02.783571+0000","last_undegraded":"2023-11-27T15:21:02.783571+0000","last_fullsized":"2023-11-27T15:21:02.783571+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:43:21.793592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782934+0000","last_change":"2023-11-27T15:20:32.650342+0000","last_active":"2023-11-27T15:21:02.782934+0000","last_peered":"2023-11-27T15:21:02.782934+0000","last_clean":"2023-11-27T15:21:02.782934+0000","last_became_active":"2023-11-27T15:20:32.650034+0000","last_became_peered":"2023-11-27T15:20:32.650034+0000","last_unstale":"2023-11-27T15:21:02.782934+0000","last_undegraded":"2023-11-27T15:21:02.782934+0000","last_fullsized":"2023-11-27T15:21:02.782934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:29:04.356130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932717+0000","last_change":"2023-11-27T15:20:33.226005+0000","last_active":"2023-11-27T15:21:02.932717+0000","last_peered":"2023-11-27T15:21:02.932717+0000","last_clean":"2023-11-27T15:21:02.932717+0000","last_became_active":"2023-11-27T15:20:33.225685+0000","last_became_peered":"2023-11-27T15:20:33.225685+0000","last_unstale":"2023-11-27T15:21:02.932717+0000","last_undegraded":"2023-11-27T15:21:02.932717+0000","last_fullsized":"2023-11-27T15:21:02.932717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:20:11.054422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779958+0000","last_change":"2023-11-27T15:20:37.673168+0000","last_active":"2023-11-27T15:21:02.779958+0000","last_peered":"2023-11-27T15:21:02.779958+0000","last_clean":"2023-11-27T15:21:02.779958+0000","last_became_active":"2023-11-27T15:20:37.672762+0000","last_became_peered":"2023-11-27T15:20:37.672762+0000","last_unstale":"2023-11-27T15:21:02.779958+0000","last_undegraded":"2023-11-27T15:21:02.779958+0000","last_fullsized":"2023-11-27T15:21:02.779958+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:17:17.977179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":79,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:48.876841+0000","last_change":"2023-11-27T15:20:32.656402+0000","last_active":"2023-11-27T15:21:48.876841+0000","last_peered":"2023-11-27T15:21:48.876841+0000","last_clean":"2023-11-27T15:21:48.876841+0000","last_became_active":"2023-11-27T15:20:32.656112+0000","last_became_peered":"2023-11-27T15:20:32.656112+0000","last_unstale":"2023-11-27T15:21:48.876841+0000","last_undegraded":"2023-11-27T15:21:48.876841+0000","last_fullsized":"2023-11-27T15:21:48.876841+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:10:24.829148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782376+0000","last_change":"2023-11-27T15:20:33.223073+0000","last_active":"2023-11-27T15:21:02.782376+0000","last_peered":"2023-11-27T15:21:02.782376+0000","last_clean":"2023-11-27T15:21:02.782376+0000","last_became_active":"2023-11-27T15:20:33.222655+0000","last_became_peered":"2023-11-27T15:20:33.222655+0000","last_unstale":"2023-11-27T15:21:02.782376+0000","last_undegraded":"2023-11-27T15:21:02.782376+0000","last_fullsized":"2023-11-27T15:21:02.782376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:48:43.125499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779818+0000","last_change":"2023-11-27T15:20:37.678722+0000","last_active":"2023-11-27T15:21:02.779818+0000","last_peered":"2023-11-27T15:21:02.779818+0000","last_clean":"2023-11-27T15:21:02.779818+0000","last_became_active":"2023-11-27T15:20:37.678242+0000","last_became_peered":"2023-11-27T15:20:37.678242+0000","last_unstale":"2023-11-27T15:21:02.779818+0000","last_undegraded":"2023-11-27T15:21:02.779818+0000","last_fullsized":"2023-11-27T15:21:02.779818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:36:10.661192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.800170+0000","last_change":"2023-11-27T15:20:32.646951+0000","last_active":"2023-11-27T15:21:03.800170+0000","last_peered":"2023-11-27T15:21:03.800170+0000","last_clean":"2023-11-27T15:21:03.800170+0000","last_became_active":"2023-11-27T15:20:32.646620+0000","last_became_peered":"2023-11-27T15:20:32.646620+0000","last_unstale":"2023-11-27T15:21:03.800170+0000","last_undegraded":"2023-11-27T15:21:03.800170+0000","last_fullsized":"2023-11-27T15:21:03.800170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:13:29.369691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932503+0000","last_change":"2023-11-27T15:20:33.226576+0000","last_active":"2023-11-27T15:21:02.932503+0000","last_peered":"2023-11-27T15:21:02.932503+0000","last_clean":"2023-11-27T15:21:02.932503+0000","last_became_active":"2023-11-27T15:20:33.226181+0000","last_became_peered":"2023-11-27T15:20:33.226181+0000","last_unstale":"2023-11-27T15:21:02.932503+0000","last_undegraded":"2023-11-27T15:21:02.932503+0000","last_fullsized":"2023-11-27T15:21:02.932503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:27:44.284870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932740+0000","last_change":"2023-11-27T15:20:37.681668+0000","last_active":"2023-11-27T15:21:02.932740+0000","last_peered":"2023-11-27T15:21:02.932740+0000","last_clean":"2023-11-27T15:21:02.932740+0000","last_became_active":"2023-11-27T15:20:37.681236+0000","last_became_peered":"2023-11-27T15:20:37.681236+0000","last_unstale":"2023-11-27T15:21:02.932740+0000","last_undegraded":"2023-11-27T15:21:02.932740+0000","last_fullsized":"2023-11-27T15:21:02.932740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:29:12.216944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.800787+0000","last_change":"2023-11-27T15:20:32.659421+0000","last_active":"2023-11-27T15:21:03.800787+0000","last_peered":"2023-11-27T15:21:03.800787+0000","last_clean":"2023-11-27T15:21:03.800787+0000","last_became_active":"2023-11-27T15:20:32.658882+0000","last_became_peered":"2023-11-27T15:20:32.658882+0000","last_unstale":"2023-11-27T15:21:03.800787+0000","last_undegraded":"2023-11-27T15:21:03.800787+0000","last_fullsized":"2023-11-27T15:21:03.800787+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:08:52.781635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783921+0000","last_change":"2023-11-27T15:20:33.227635+0000","last_active":"2023-11-27T15:21:02.783921+0000","last_peered":"2023-11-27T15:21:02.783921+0000","last_clean":"2023-11-27T15:21:02.783921+0000","last_became_active":"2023-11-27T15:20:33.225217+0000","last_became_peered":"2023-11-27T15:20:33.225217+0000","last_unstale":"2023-11-27T15:21:02.783921+0000","last_undegraded":"2023-11-27T15:21:02.783921+0000","last_fullsized":"2023-11-27T15:21:02.783921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:53:42.037169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785814+0000","last_change":"2023-11-27T15:20:37.679575+0000","last_active":"2023-11-27T15:21:02.785814+0000","last_peered":"2023-11-27T15:21:02.785814+0000","last_clean":"2023-11-27T15:21:02.785814+0000","last_became_active":"2023-11-27T15:20:37.679131+0000","last_became_peered":"2023-11-27T15:20:37.679131+0000","last_unstale":"2023-11-27T15:21:02.785814+0000","last_undegraded":"2023-11-27T15:21:02.785814+0000","last_fullsized":"2023-11-27T15:21:02.785814+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:43:21.129782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.931632+0000","last_change":"2023-11-27T15:20:32.657155+0000","last_active":"2023-11-27T15:21:02.931632+0000","last_peered":"2023-11-27T15:21:02.931632+0000","last_clean":"2023-11-27T15:21:02.931632+0000","last_became_active":"2023-11-27T15:20:32.656832+0000","last_became_peered":"2023-11-27T15:20:32.656832+0000","last_unstale":"2023-11-27T15:21:02.931632+0000","last_undegraded":"2023-11-27T15:21:02.931632+0000","last_fullsized":"2023-11-27T15:21:02.931632+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:03:07.478635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779387+0000","last_change":"2023-11-27T15:20:33.229215+0000","last_active":"2023-11-27T15:21:02.779387+0000","last_peered":"2023-11-27T15:21:02.779387+0000","last_clean":"2023-11-27T15:21:02.779387+0000","last_became_active":"2023-11-27T15:20:33.228616+0000","last_became_peered":"2023-11-27T15:20:33.228616+0000","last_unstale":"2023-11-27T15:21:02.779387+0000","last_undegraded":"2023-11-27T15:21:02.779387+0000","last_fullsized":"2023-11-27T15:21:02.779387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:30:10.600160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784114+0000","last_change":"2023-11-27T15:20:37.672582+0000","last_active":"2023-11-27T15:21:02.784114+0000","last_peered":"2023-11-27T15:21:02.784114+0000","last_clean":"2023-11-27T15:21:02.784114+0000","last_became_active":"2023-11-27T15:20:37.672193+0000","last_became_peered":"2023-11-27T15:20:37.672193+0000","last_unstale":"2023-11-27T15:21:02.784114+0000","last_undegraded":"2023-11-27T15:21:02.784114+0000","last_fullsized":"2023-11-27T15:21:02.784114+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:53:51.137569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":76,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:49.877415+0000","last_change":"2023-11-27T15:20:32.647894+0000","last_active":"2023-11-27T15:21:49.877415+0000","last_peered":"2023-11-27T15:21:49.877415+0000","last_clean":"2023-11-27T15:21:49.877415+0000","last_became_active":"2023-11-27T15:20:32.647597+0000","last_became_peered":"2023-11-27T15:20:32.647597+0000","last_unstale":"2023-11-27T15:21:49.877415+0000","last_undegraded":"2023-11-27T15:21:49.877415+0000","last_fullsized":"2023-11-27T15:21:49.877415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:00:17.488397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":47,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.795490+0000","last_change":"2023-11-27T15:20:33.229382+0000","last_active":"2023-11-27T15:21:02.795490+0000","last_peered":"2023-11-27T15:21:02.795490+0000","last_clean":"2023-11-27T15:21:02.795490+0000","last_became_active":"2023-11-27T15:20:33.229031+0000","last_became_peered":"2023-11-27T15:20:33.229031+0000","last_unstale":"2023-11-27T15:21:02.795490+0000","last_undegraded":"2023-11-27T15:21:02.795490+0000","last_fullsized":"2023-11-27T15:21:02.795490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:43:48.637904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779387+0000","last_change":"2023-11-27T15:20:37.679131+0000","last_active":"2023-11-27T15:21:02.779387+0000","last_peered":"2023-11-27T15:21:02.779387+0000","last_clean":"2023-11-27T15:21:02.779387+0000","last_became_active":"2023-11-27T15:20:37.678290+0000","last_became_peered":"2023-11-27T15:20:37.678290+0000","last_unstale":"2023-11-27T15:21:02.779387+0000","last_undegraded":"2023-11-27T15:21:02.779387+0000","last_fullsized":"2023-11-27T15:21:02.779387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:55:28.895714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.798466+0000","last_change":"2023-11-27T15:21:02.797374+0000","last_active":"2023-11-27T15:21:02.798466+0000","last_peered":"2023-11-27T15:21:02.798466+0000","last_clean":"2023-11-27T15:21:02.798466+0000","last_became_active":"2023-11-27T15:21:02.796076+0000","last_became_peered":"2023-11-27T15:21:02.796076+0000","last_unstale":"2023-11-27T15:21:02.798466+0000","last_undegraded":"2023-11-27T15:21:02.798466+0000","last_fullsized":"2023-11-27T15:21:02.798466+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:07:58.898194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.935104+0000","last_change":"2023-11-27T15:21:02.935104+0000","last_active":"2023-11-27T15:21:02.935104+0000","last_peered":"2023-11-27T15:21:02.935104+0000","last_clean":"2023-11-27T15:21:02.935104+0000","last_became_active":"2023-11-27T15:21:02.934486+0000","last_became_peered":"2023-11-27T15:21:02.934486+0000","last_unstale":"2023-11-27T15:21:02.935104+0000","last_undegraded":"2023-11-27T15:21:02.935104+0000","last_fullsized":"2023-11-27T15:21:02.935104+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:06:11.768697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783910+0000","last_change":"2023-11-27T15:20:37.670389+0000","last_active":"2023-11-27T15:21:02.783910+0000","last_peered":"2023-11-27T15:21:02.783910+0000","last_clean":"2023-11-27T15:21:02.783910+0000","last_became_active":"2023-11-27T15:20:37.669512+0000","last_became_peered":"2023-11-27T15:20:37.669512+0000","last_unstale":"2023-11-27T15:21:02.783910+0000","last_undegraded":"2023-11-27T15:21:02.783910+0000","last_fullsized":"2023-11-27T15:21:02.783910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:47:38.447258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932618+0000","last_change":"2023-11-27T15:20:32.658287+0000","last_active":"2023-11-27T15:21:02.932618+0000","last_peered":"2023-11-27T15:21:02.932618+0000","last_clean":"2023-11-27T15:21:02.932618+0000","last_became_active":"2023-11-27T15:20:32.657925+0000","last_became_peered":"2023-11-27T15:20:32.657925+0000","last_unstale":"2023-11-27T15:21:02.932618+0000","last_undegraded":"2023-11-27T15:21:02.932618+0000","last_fullsized":"2023-11-27T15:21:02.932618+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:49:45.401491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.795421+0000","last_change":"2023-11-27T15:20:33.220240+0000","last_active":"2023-11-27T15:21:02.795421+0000","last_peered":"2023-11-27T15:21:02.795421+0000","last_clean":"2023-11-27T15:21:02.795421+0000","last_became_active":"2023-11-27T15:20:33.219962+0000","last_became_peered":"2023-11-27T15:20:33.219962+0000","last_unstale":"2023-11-27T15:21:02.795421+0000","last_undegraded":"2023-11-27T15:21:02.795421+0000","last_fullsized":"2023-11-27T15:21:02.795421+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:22:09.271036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779503+0000","last_change":"2023-11-27T15:20:37.672203+0000","last_active":"2023-11-27T15:21:02.779503+0000","last_peered":"2023-11-27T15:21:02.779503+0000","last_clean":"2023-11-27T15:21:02.779503+0000","last_became_active":"2023-11-27T15:20:37.671149+0000","last_became_peered":"2023-11-27T15:20:37.671149+0000","last_unstale":"2023-11-27T15:21:02.779503+0000","last_undegraded":"2023-11-27T15:21:02.779503+0000","last_fullsized":"2023-11-27T15:21:02.779503+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:14:24.426741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'33","reported_seq":161,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:50.878872+0000","last_change":"2023-11-27T15:20:32.646637+0000","last_active":"2023-11-27T15:21:50.878872+0000","last_peered":"2023-11-27T15:21:50.878872+0000","last_clean":"2023-11-27T15:21:50.878872+0000","last_became_active":"2023-11-27T15:20:32.646292+0000","last_became_peered":"2023-11-27T15:20:32.646292+0000","last_unstale":"2023-11-27T15:21:50.878872+0000","last_undegraded":"2023-11-27T15:21:50.878872+0000","last_fullsized":"2023-11-27T15:21:50.878872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:01:39.494636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59808,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":102,"num_read_kb":110,"num_write":34,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782815+0000","last_change":"2023-11-27T15:20:33.217078+0000","last_active":"2023-11-27T15:21:02.782815+0000","last_peered":"2023-11-27T15:21:02.782815+0000","last_clean":"2023-11-27T15:21:02.782815+0000","last_became_active":"2023-11-27T15:20:33.216787+0000","last_became_peered":"2023-11-27T15:20:33.216787+0000","last_unstale":"2023-11-27T15:21:02.782815+0000","last_undegraded":"2023-11-27T15:21:02.782815+0000","last_fullsized":"2023-11-27T15:21:02.782815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:33:46.415082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.933136+0000","last_change":"2023-11-27T15:20:37.671800+0000","last_active":"2023-11-27T15:21:02.933136+0000","last_peered":"2023-11-27T15:21:02.933136+0000","last_clean":"2023-11-27T15:21:02.933136+0000","last_became_active":"2023-11-27T15:20:37.671374+0000","last_became_peered":"2023-11-27T15:20:37.671374+0000","last_unstale":"2023-11-27T15:21:02.933136+0000","last_undegraded":"2023-11-27T15:21:02.933136+0000","last_fullsized":"2023-11-27T15:21:02.933136+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:04:07.644987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.802237+0000","last_change":"2023-11-27T15:20:32.659508+0000","last_active":"2023-11-27T15:21:03.802237+0000","last_peered":"2023-11-27T15:21:03.802237+0000","last_clean":"2023-11-27T15:21:03.802237+0000","last_became_active":"2023-11-27T15:20:32.659226+0000","last_became_peered":"2023-11-27T15:20:32.659226+0000","last_unstale":"2023-11-27T15:21:03.802237+0000","last_undegraded":"2023-11-27T15:21:03.802237+0000","last_fullsized":"2023-11-27T15:21:03.802237+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:42:20.386331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786639+0000","last_change":"2023-11-27T15:20:33.230012+0000","last_active":"2023-11-27T15:21:02.786639+0000","last_peered":"2023-11-27T15:21:02.786639+0000","last_clean":"2023-11-27T15:21:02.786639+0000","last_became_active":"2023-11-27T15:20:33.225193+0000","last_became_peered":"2023-11-27T15:20:33.225193+0000","last_unstale":"2023-11-27T15:21:02.786639+0000","last_undegraded":"2023-11-27T15:21:02.786639+0000","last_fullsized":"2023-11-27T15:21:02.786639+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:31:18.056712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932755+0000","last_change":"2023-11-27T15:20:37.672430+0000","last_active":"2023-11-27T15:21:02.932755+0000","last_peered":"2023-11-27T15:21:02.932755+0000","last_clean":"2023-11-27T15:21:02.932755+0000","last_became_active":"2023-11-27T15:20:37.671982+0000","last_became_peered":"2023-11-27T15:20:37.671982+0000","last_unstale":"2023-11-27T15:21:02.932755+0000","last_undegraded":"2023-11-27T15:21:02.932755+0000","last_fullsized":"2023-11-27T15:21:02.932755+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:53:27.660441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932051+0000","last_change":"2023-11-27T15:20:32.655015+0000","last_active":"2023-11-27T15:21:02.932051+0000","last_peered":"2023-11-27T15:21:02.932051+0000","last_clean":"2023-11-27T15:21:02.932051+0000","last_became_active":"2023-11-27T15:20:32.654662+0000","last_became_peered":"2023-11-27T15:20:32.654662+0000","last_unstale":"2023-11-27T15:21:02.932051+0000","last_undegraded":"2023-11-27T15:21:02.932051+0000","last_fullsized":"2023-11-27T15:21:02.932051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:10:34.935987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780064+0000","last_change":"2023-11-27T15:20:33.233980+0000","last_active":"2023-11-27T15:21:02.780064+0000","last_peered":"2023-11-27T15:21:02.780064+0000","last_clean":"2023-11-27T15:21:02.780064+0000","last_became_active":"2023-11-27T15:20:33.233581+0000","last_became_peered":"2023-11-27T15:20:33.233581+0000","last_unstale":"2023-11-27T15:21:02.780064+0000","last_undegraded":"2023-11-27T15:21:02.780064+0000","last_fullsized":"2023-11-27T15:21:02.780064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:21:09.061874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779860+0000","last_change":"2023-11-27T15:20:37.682359+0000","last_active":"2023-11-27T15:21:02.779860+0000","last_peered":"2023-11-27T15:21:02.779860+0000","last_clean":"2023-11-27T15:21:02.779860+0000","last_became_active":"2023-11-27T15:20:37.681888+0000","last_became_peered":"2023-11-27T15:20:37.681888+0000","last_unstale":"2023-11-27T15:21:02.779860+0000","last_undegraded":"2023-11-27T15:21:02.779860+0000","last_fullsized":"2023-11-27T15:21:02.779860+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:22:58.040147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":78,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:47.875294+0000","last_change":"2023-11-27T15:20:32.653388+0000","last_active":"2023-11-27T15:21:47.875294+0000","last_peered":"2023-11-27T15:21:47.875294+0000","last_clean":"2023-11-27T15:21:47.875294+0000","last_became_active":"2023-11-27T15:20:32.653007+0000","last_became_peered":"2023-11-27T15:20:32.653007+0000","last_unstale":"2023-11-27T15:21:47.875294+0000","last_undegraded":"2023-11-27T15:21:47.875294+0000","last_fullsized":"2023-11-27T15:21:47.875294+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:02:11.654541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779871+0000","last_change":"2023-11-27T15:20:33.216539+0000","last_active":"2023-11-27T15:21:02.779871+0000","last_peered":"2023-11-27T15:21:02.779871+0000","last_clean":"2023-11-27T15:21:02.779871+0000","last_became_active":"2023-11-27T15:20:33.216166+0000","last_became_peered":"2023-11-27T15:20:33.216166+0000","last_unstale":"2023-11-27T15:21:02.779871+0000","last_undegraded":"2023-11-27T15:21:02.779871+0000","last_fullsized":"2023-11-27T15:21:02.779871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:12:37.881354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786643+0000","last_change":"2023-11-27T15:20:37.675735+0000","last_active":"2023-11-27T15:21:02.786643+0000","last_peered":"2023-11-27T15:21:02.786643+0000","last_clean":"2023-11-27T15:21:02.786643+0000","last_became_active":"2023-11-27T15:20:37.669685+0000","last_became_peered":"2023-11-27T15:20:37.669685+0000","last_unstale":"2023-11-27T15:21:02.786643+0000","last_undegraded":"2023-11-27T15:21:02.786643+0000","last_fullsized":"2023-11-27T15:21:02.786643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:47:13.695674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.803274+0000","last_change":"2023-11-27T15:20:32.657567+0000","last_active":"2023-11-27T15:21:03.803274+0000","last_peered":"2023-11-27T15:21:03.803274+0000","last_clean":"2023-11-27T15:21:03.803274+0000","last_became_active":"2023-11-27T15:20:32.657251+0000","last_became_peered":"2023-11-27T15:20:32.657251+0000","last_unstale":"2023-11-27T15:21:03.803274+0000","last_undegraded":"2023-11-27T15:21:03.803274+0000","last_fullsized":"2023-11-27T15:21:03.803274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:23:00.166520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932029+0000","last_change":"2023-11-27T15:20:33.225019+0000","last_active":"2023-11-27T15:21:02.932029+0000","last_peered":"2023-11-27T15:21:02.932029+0000","last_clean":"2023-11-27T15:21:02.932029+0000","last_became_active":"2023-11-27T15:20:33.224622+0000","last_became_peered":"2023-11-27T15:20:33.224622+0000","last_unstale":"2023-11-27T15:21:02.932029+0000","last_undegraded":"2023-11-27T15:21:02.932029+0000","last_fullsized":"2023-11-27T15:21:02.932029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:44:13.508838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.931675+0000","last_change":"2023-11-27T15:20:37.680808+0000","last_active":"2023-11-27T15:21:02.931675+0000","last_peered":"2023-11-27T15:21:02.931675+0000","last_clean":"2023-11-27T15:21:02.931675+0000","last_became_active":"2023-11-27T15:20:37.680348+0000","last_became_peered":"2023-11-27T15:20:37.680348+0000","last_unstale":"2023-11-27T15:21:02.931675+0000","last_undegraded":"2023-11-27T15:21:02.931675+0000","last_fullsized":"2023-11-27T15:21:02.931675+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:48:00.800370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:04.804666+0000","last_change":"2023-11-27T15:20:32.651093+0000","last_active":"2023-11-27T15:21:04.804666+0000","last_peered":"2023-11-27T15:21:04.804666+0000","last_clean":"2023-11-27T15:21:04.804666+0000","last_became_active":"2023-11-27T15:20:32.650657+0000","last_became_peered":"2023-11-27T15:20:32.650657+0000","last_unstale":"2023-11-27T15:21:04.804666+0000","last_undegraded":"2023-11-27T15:21:04.804666+0000","last_fullsized":"2023-11-27T15:21:04.804666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:28:31.347970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.933913+0000","last_change":"2023-11-27T15:20:33.224806+0000","last_active":"2023-11-27T15:21:02.933913+0000","last_peered":"2023-11-27T15:21:02.933913+0000","last_clean":"2023-11-27T15:21:02.933913+0000","last_became_active":"2023-11-27T15:20:33.224362+0000","last_became_peered":"2023-11-27T15:20:33.224362+0000","last_unstale":"2023-11-27T15:21:02.933913+0000","last_undegraded":"2023-11-27T15:21:02.933913+0000","last_fullsized":"2023-11-27T15:21:02.933913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:34:11.704495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782260+0000","last_change":"2023-11-27T15:20:37.849274+0000","last_active":"2023-11-27T15:21:02.782260+0000","last_peered":"2023-11-27T15:21:02.782260+0000","last_clean":"2023-11-27T15:21:02.782260+0000","last_became_active":"2023-11-27T15:20:37.848839+0000","last_became_peered":"2023-11-27T15:20:37.848839+0000","last_unstale":"2023-11-27T15:21:02.782260+0000","last_undegraded":"2023-11-27T15:21:02.782260+0000","last_fullsized":"2023-11-27T15:21:02.782260+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:46:25.321810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779849+0000","last_change":"2023-11-27T15:20:32.660835+0000","last_active":"2023-11-27T15:21:02.779849+0000","last_peered":"2023-11-27T15:21:02.779849+0000","last_clean":"2023-11-27T15:21:02.779849+0000","last_became_active":"2023-11-27T15:20:32.660431+0000","last_became_peered":"2023-11-27T15:20:32.660431+0000","last_unstale":"2023-11-27T15:21:02.779849+0000","last_undegraded":"2023-11-27T15:21:02.779849+0000","last_fullsized":"2023-11-27T15:21:02.779849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:52:11.617105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932764+0000","last_change":"2023-11-27T15:20:33.245772+0000","last_active":"2023-11-27T15:21:02.932764+0000","last_peered":"2023-11-27T15:21:02.932764+0000","last_clean":"2023-11-27T15:21:02.932764+0000","last_became_active":"2023-11-27T15:20:33.245468+0000","last_became_peered":"2023-11-27T15:20:33.245468+0000","last_unstale":"2023-11-27T15:21:02.932764+0000","last_undegraded":"2023-11-27T15:21:02.932764+0000","last_fullsized":"2023-11-27T15:21:02.932764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:24:42.963520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784658+0000","last_change":"2023-11-27T15:20:37.678972+0000","last_active":"2023-11-27T15:21:02.784658+0000","last_peered":"2023-11-27T15:21:02.784658+0000","last_clean":"2023-11-27T15:21:02.784658+0000","last_became_active":"2023-11-27T15:20:37.678304+0000","last_became_peered":"2023-11-27T15:20:37.678304+0000","last_unstale":"2023-11-27T15:21:02.784658+0000","last_undegraded":"2023-11-27T15:21:02.784658+0000","last_fullsized":"2023-11-27T15:21:02.784658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:32:12.064148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780000+0000","last_change":"2023-11-27T15:20:32.661739+0000","last_active":"2023-11-27T15:21:02.780000+0000","last_peered":"2023-11-27T15:21:02.780000+0000","last_clean":"2023-11-27T15:21:02.780000+0000","last_became_active":"2023-11-27T15:20:32.661401+0000","last_became_peered":"2023-11-27T15:20:32.661401+0000","last_unstale":"2023-11-27T15:21:02.780000+0000","last_undegraded":"2023-11-27T15:21:02.780000+0000","last_fullsized":"2023-11-27T15:21:02.780000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:47:15.469916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932830+0000","last_change":"2023-11-27T15:20:33.235595+0000","last_active":"2023-11-27T15:21:02.932830+0000","last_peered":"2023-11-27T15:21:02.932830+0000","last_clean":"2023-11-27T15:21:02.932830+0000","last_became_active":"2023-11-27T15:20:33.235277+0000","last_became_peered":"2023-11-27T15:20:33.235277+0000","last_unstale":"2023-11-27T15:21:02.932830+0000","last_undegraded":"2023-11-27T15:21:02.932830+0000","last_fullsized":"2023-11-27T15:21:02.932830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:23:31.896222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.934194+0000","last_change":"2023-11-27T15:21:02.934194+0000","last_active":"2023-11-27T15:21:02.934194+0000","last_peered":"2023-11-27T15:21:02.934194+0000","last_clean":"2023-11-27T15:21:02.934194+0000","last_became_active":"2023-11-27T15:21:02.933555+0000","last_became_peered":"2023-11-27T15:21:02.933555+0000","last_unstale":"2023-11-27T15:21:02.934194+0000","last_undegraded":"2023-11-27T15:21:02.934194+0000","last_fullsized":"2023-11-27T15:21:02.934194+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:49:59.949602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785630+0000","last_change":"2023-11-27T15:21:02.784351+0000","last_active":"2023-11-27T15:21:02.785630+0000","last_peered":"2023-11-27T15:21:02.785630+0000","last_clean":"2023-11-27T15:21:02.785630+0000","last_became_active":"2023-11-27T15:21:02.783996+0000","last_became_peered":"2023-11-27T15:21:02.783996+0000","last_unstale":"2023-11-27T15:21:02.785630+0000","last_undegraded":"2023-11-27T15:21:02.785630+0000","last_fullsized":"2023-11-27T15:21:02.785630+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:30:13.050830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":77,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:49.877453+0000","last_change":"2023-11-27T15:20:32.659823+0000","last_active":"2023-11-27T15:21:49.877453+0000","last_peered":"2023-11-27T15:21:49.877453+0000","last_clean":"2023-11-27T15:21:49.877453+0000","last_became_active":"2023-11-27T15:20:32.659472+0000","last_became_peered":"2023-11-27T15:20:32.659472+0000","last_unstale":"2023-11-27T15:21:49.877453+0000","last_undegraded":"2023-11-27T15:21:49.877453+0000","last_fullsized":"2023-11-27T15:21:49.877453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:19:47.161135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":47,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780173+0000","last_change":"2023-11-27T15:20:37.671494+0000","last_active":"2023-11-27T15:21:02.780173+0000","last_peered":"2023-11-27T15:21:02.780173+0000","last_clean":"2023-11-27T15:21:02.780173+0000","last_became_active":"2023-11-27T15:20:37.671010+0000","last_became_peered":"2023-11-27T15:20:37.671010+0000","last_unstale":"2023-11-27T15:21:02.780173+0000","last_undegraded":"2023-11-27T15:21:02.780173+0000","last_fullsized":"2023-11-27T15:21:02.780173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:09:00.036788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784810+0000","last_change":"2023-11-27T15:20:33.241669+0000","last_active":"2023-11-27T15:21:02.784810+0000","last_peered":"2023-11-27T15:21:02.784810+0000","last_clean":"2023-11-27T15:21:02.784810+0000","last_became_active":"2023-11-27T15:20:33.241373+0000","last_became_peered":"2023-11-27T15:20:33.241373+0000","last_unstale":"2023-11-27T15:21:02.784810+0000","last_undegraded":"2023-11-27T15:21:02.784810+0000","last_fullsized":"2023-11-27T15:21:02.784810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:21:19.709967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:04.804123+0000","last_change":"2023-11-27T15:20:32.647596+0000","last_active":"2023-11-27T15:21:04.804123+0000","last_peered":"2023-11-27T15:21:04.804123+0000","last_clean":"2023-11-27T15:21:04.804123+0000","last_became_active":"2023-11-27T15:20:32.647228+0000","last_became_peered":"2023-11-27T15:20:32.647228+0000","last_unstale":"2023-11-27T15:21:04.804123+0000","last_undegraded":"2023-11-27T15:21:04.804123+0000","last_fullsized":"2023-11-27T15:21:04.804123+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:40:49.354792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779663+0000","last_change":"2023-11-27T15:19:30.114855+0000","last_active":"2023-11-27T15:21:02.779663+0000","last_peered":"2023-11-27T15:21:02.779663+0000","last_clean":"2023-11-27T15:21:02.779663+0000","last_became_active":"2023-11-27T15:19:30.114419+0000","last_became_peered":"2023-11-27T15:19:30.114419+0000","last_unstale":"2023-11-27T15:21:02.779663+0000","last_undegraded":"2023-11-27T15:21:02.779663+0000","last_fullsized":"2023-11-27T15:21:02.779663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:20.216695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:20.216695+0000","last_clean_scrub_stamp":"2023-11-27T15:16:20.216695+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:14:23.715586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932257+0000","last_change":"2023-11-27T15:20:37.850539+0000","last_active":"2023-11-27T15:21:02.932257+0000","last_peered":"2023-11-27T15:21:02.932257+0000","last_clean":"2023-11-27T15:21:02.932257+0000","last_became_active":"2023-11-27T15:20:37.850044+0000","last_became_peered":"2023-11-27T15:20:37.850044+0000","last_unstale":"2023-11-27T15:21:02.932257+0000","last_undegraded":"2023-11-27T15:21:02.932257+0000","last_fullsized":"2023-11-27T15:21:02.932257+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:29:43.906707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783098+0000","last_change":"2023-11-27T15:20:33.211319+0000","last_active":"2023-11-27T15:21:02.783098+0000","last_peered":"2023-11-27T15:21:02.783098+0000","last_clean":"2023-11-27T15:21:02.783098+0000","last_became_active":"2023-11-27T15:20:33.211004+0000","last_became_peered":"2023-11-27T15:20:33.211004+0000","last_unstale":"2023-11-27T15:21:02.783098+0000","last_undegraded":"2023-11-27T15:21:02.783098+0000","last_fullsized":"2023-11-27T15:21:02.783098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:57:04.622964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.799863+0000","last_change":"2023-11-27T15:20:32.655003+0000","last_active":"2023-11-27T15:21:03.799863+0000","last_peered":"2023-11-27T15:21:03.799863+0000","last_clean":"2023-11-27T15:21:03.799863+0000","last_became_active":"2023-11-27T15:20:32.654517+0000","last_became_peered":"2023-11-27T15:20:32.654517+0000","last_unstale":"2023-11-27T15:21:03.799863+0000","last_undegraded":"2023-11-27T15:21:03.799863+0000","last_fullsized":"2023-11-27T15:21:03.799863+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:12:26.090416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932725+0000","last_change":"2023-11-27T15:20:37.673183+0000","last_active":"2023-11-27T15:21:02.932725+0000","last_peered":"2023-11-27T15:21:02.932725+0000","last_clean":"2023-11-27T15:21:02.932725+0000","last_became_active":"2023-11-27T15:20:37.672838+0000","last_became_peered":"2023-11-27T15:20:37.672838+0000","last_unstale":"2023-11-27T15:21:02.932725+0000","last_undegraded":"2023-11-27T15:21:02.932725+0000","last_fullsized":"2023-11-27T15:21:02.932725+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:47:56.458647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782739+0000","last_change":"2023-11-27T15:20:32.659540+0000","last_active":"2023-11-27T15:21:02.782739+0000","last_peered":"2023-11-27T15:21:02.782739+0000","last_clean":"2023-11-27T15:21:02.782739+0000","last_became_active":"2023-11-27T15:20:32.659211+0000","last_became_peered":"2023-11-27T15:20:32.659211+0000","last_unstale":"2023-11-27T15:21:02.782739+0000","last_undegraded":"2023-11-27T15:21:02.782739+0000","last_fullsized":"2023-11-27T15:21:02.782739+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:02:50.140093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784853+0000","last_change":"2023-11-27T15:20:33.216405+0000","last_active":"2023-11-27T15:21:02.784853+0000","last_peered":"2023-11-27T15:21:02.784853+0000","last_clean":"2023-11-27T15:21:02.784853+0000","last_became_active":"2023-11-27T15:20:33.216072+0000","last_became_peered":"2023-11-27T15:20:33.216072+0000","last_unstale":"2023-11-27T15:21:02.784853+0000","last_undegraded":"2023-11-27T15:21:02.784853+0000","last_fullsized":"2023-11-27T15:21:02.784853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:09:28.211261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785517+0000","last_change":"2023-11-27T15:20:37.849753+0000","last_active":"2023-11-27T15:21:02.785517+0000","last_peered":"2023-11-27T15:21:02.785517+0000","last_clean":"2023-11-27T15:21:02.785517+0000","last_became_active":"2023-11-27T15:20:37.848576+0000","last_became_peered":"2023-11-27T15:20:37.848576+0000","last_unstale":"2023-11-27T15:21:02.785517+0000","last_undegraded":"2023-11-27T15:21:02.785517+0000","last_fullsized":"2023-11-27T15:21:02.785517+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:59:24.696190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'6","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:11.515280+0000","last_change":"2023-11-27T15:20:32.652739+0000","last_active":"2023-11-27T15:21:11.515280+0000","last_peered":"2023-11-27T15:21:11.515280+0000","last_clean":"2023-11-27T15:21:11.515280+0000","last_became_active":"2023-11-27T15:20:32.652314+0000","last_became_peered":"2023-11-27T15:20:32.652314+0000","last_unstale":"2023-11-27T15:21:11.515280+0000","last_undegraded":"2023-11-27T15:21:11.515280+0000","last_fullsized":"2023-11-27T15:21:11.515280+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:44:25.188063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.787449+0000","last_change":"2023-11-27T15:20:33.240552+0000","last_active":"2023-11-27T15:21:02.787449+0000","last_peered":"2023-11-27T15:21:02.787449+0000","last_clean":"2023-11-27T15:21:02.787449+0000","last_became_active":"2023-11-27T15:20:33.239885+0000","last_became_peered":"2023-11-27T15:20:33.239885+0000","last_unstale":"2023-11-27T15:21:02.787449+0000","last_undegraded":"2023-11-27T15:21:02.787449+0000","last_fullsized":"2023-11-27T15:21:02.787449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:36:51.366222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779611+0000","last_change":"2023-11-27T15:20:37.669895+0000","last_active":"2023-11-27T15:21:02.779611+0000","last_peered":"2023-11-27T15:21:02.779611+0000","last_clean":"2023-11-27T15:21:02.779611+0000","last_became_active":"2023-11-27T15:20:37.669489+0000","last_became_peered":"2023-11-27T15:20:37.669489+0000","last_unstale":"2023-11-27T15:21:02.779611+0000","last_undegraded":"2023-11-27T15:21:02.779611+0000","last_fullsized":"2023-11-27T15:21:02.779611+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:18:44.798721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784573+0000","last_change":"2023-11-27T15:20:32.656773+0000","last_active":"2023-11-27T15:21:02.784573+0000","last_peered":"2023-11-27T15:21:02.784573+0000","last_clean":"2023-11-27T15:21:02.784573+0000","last_became_active":"2023-11-27T15:20:32.656451+0000","last_became_peered":"2023-11-27T15:20:32.656451+0000","last_unstale":"2023-11-27T15:21:02.784573+0000","last_undegraded":"2023-11-27T15:21:02.784573+0000","last_fullsized":"2023-11-27T15:21:02.784573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:34:42.007222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786148+0000","last_change":"2023-11-27T15:20:33.231685+0000","last_active":"2023-11-27T15:21:02.786148+0000","last_peered":"2023-11-27T15:21:02.786148+0000","last_clean":"2023-11-27T15:21:02.786148+0000","last_became_active":"2023-11-27T15:20:33.228265+0000","last_became_peered":"2023-11-27T15:20:33.228265+0000","last_unstale":"2023-11-27T15:21:02.786148+0000","last_undegraded":"2023-11-27T15:21:02.786148+0000","last_fullsized":"2023-11-27T15:21:02.786148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:18:19.094289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.935830+0000","last_change":"2023-11-27T15:21:02.935830+0000","last_active":"2023-11-27T15:21:02.935830+0000","last_peered":"2023-11-27T15:21:02.935830+0000","last_clean":"2023-11-27T15:21:02.935830+0000","last_became_active":"2023-11-27T15:21:02.935412+0000","last_became_peered":"2023-11-27T15:21:02.935412+0000","last_unstale":"2023-11-27T15:21:02.935830+0000","last_undegraded":"2023-11-27T15:21:02.935830+0000","last_fullsized":"2023-11-27T15:21:02.935830+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:26:59.988473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.794970+0000","last_change":"2023-11-27T15:20:32.660424+0000","last_active":"2023-11-27T15:21:02.794970+0000","last_peered":"2023-11-27T15:21:02.794970+0000","last_clean":"2023-11-27T15:21:02.794970+0000","last_became_active":"2023-11-27T15:20:32.660112+0000","last_became_peered":"2023-11-27T15:20:32.660112+0000","last_unstale":"2023-11-27T15:21:02.794970+0000","last_undegraded":"2023-11-27T15:21:02.794970+0000","last_fullsized":"2023-11-27T15:21:02.794970+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:59:35.405721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782221+0000","last_change":"2023-11-27T15:20:33.227677+0000","last_active":"2023-11-27T15:21:02.782221+0000","last_peered":"2023-11-27T15:21:02.782221+0000","last_clean":"2023-11-27T15:21:02.782221+0000","last_became_active":"2023-11-27T15:20:33.227334+0000","last_became_peered":"2023-11-27T15:20:33.227334+0000","last_unstale":"2023-11-27T15:21:02.782221+0000","last_undegraded":"2023-11-27T15:21:02.782221+0000","last_fullsized":"2023-11-27T15:21:02.782221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:45:49.719897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.935341+0000","last_change":"2023-11-27T15:21:02.934516+0000","last_active":"2023-11-27T15:21:02.935341+0000","last_peered":"2023-11-27T15:21:02.935341+0000","last_clean":"2023-11-27T15:21:02.935341+0000","last_became_active":"2023-11-27T15:21:02.934012+0000","last_became_peered":"2023-11-27T15:21:02.934012+0000","last_unstale":"2023-11-27T15:21:02.935341+0000","last_undegraded":"2023-11-27T15:21:02.935341+0000","last_fullsized":"2023-11-27T15:21:02.935341+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:07:11.301253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784148+0000","last_change":"2023-11-27T15:20:32.657935+0000","last_active":"2023-11-27T15:21:02.784148+0000","last_peered":"2023-11-27T15:21:02.784148+0000","last_clean":"2023-11-27T15:21:02.784148+0000","last_became_active":"2023-11-27T15:20:32.657579+0000","last_became_peered":"2023-11-27T15:20:32.657579+0000","last_unstale":"2023-11-27T15:21:02.784148+0000","last_undegraded":"2023-11-27T15:21:02.784148+0000","last_fullsized":"2023-11-27T15:21:02.784148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:30:23.087862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932840+0000","last_change":"2023-11-27T15:20:33.225817+0000","last_active":"2023-11-27T15:21:02.932840+0000","last_peered":"2023-11-27T15:21:02.932840+0000","last_clean":"2023-11-27T15:21:02.932840+0000","last_became_active":"2023-11-27T15:20:33.225438+0000","last_became_peered":"2023-11-27T15:20:33.225438+0000","last_unstale":"2023-11-27T15:21:02.932840+0000","last_undegraded":"2023-11-27T15:21:02.932840+0000","last_fullsized":"2023-11-27T15:21:02.932840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:04:27.668653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.934889+0000","last_change":"2023-11-27T15:21:02.934304+0000","last_active":"2023-11-27T15:21:02.934889+0000","last_peered":"2023-11-27T15:21:02.934889+0000","last_clean":"2023-11-27T15:21:02.934889+0000","last_became_active":"2023-11-27T15:21:02.933709+0000","last_became_peered":"2023-11-27T15:21:02.933709+0000","last_unstale":"2023-11-27T15:21:02.934889+0000","last_undegraded":"2023-11-27T15:21:02.934889+0000","last_fullsized":"2023-11-27T15:21:02.934889+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:43:38.974368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780238+0000","last_change":"2023-11-27T15:20:32.659484+0000","last_active":"2023-11-27T15:21:02.780238+0000","last_peered":"2023-11-27T15:21:02.780238+0000","last_clean":"2023-11-27T15:21:02.780238+0000","last_became_active":"2023-11-27T15:20:32.659134+0000","last_became_peered":"2023-11-27T15:20:32.659134+0000","last_unstale":"2023-11-27T15:21:02.780238+0000","last_undegraded":"2023-11-27T15:21:02.780238+0000","last_fullsized":"2023-11-27T15:21:02.780238+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:04:03.053379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784616+0000","last_change":"2023-11-27T15:20:33.235755+0000","last_active":"2023-11-27T15:21:02.784616+0000","last_peered":"2023-11-27T15:21:02.784616+0000","last_clean":"2023-11-27T15:21:02.784616+0000","last_became_active":"2023-11-27T15:20:33.234576+0000","last_became_peered":"2023-11-27T15:20:33.234576+0000","last_unstale":"2023-11-27T15:21:02.784616+0000","last_undegraded":"2023-11-27T15:21:02.784616+0000","last_fullsized":"2023-11-27T15:21:02.784616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:25:25.181797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784302+0000","last_change":"2023-11-27T15:20:37.849121+0000","last_active":"2023-11-27T15:21:02.784302+0000","last_peered":"2023-11-27T15:21:02.784302+0000","last_clean":"2023-11-27T15:21:02.784302+0000","last_became_active":"2023-11-27T15:20:37.848683+0000","last_became_peered":"2023-11-27T15:20:37.848683+0000","last_unstale":"2023-11-27T15:21:02.784302+0000","last_undegraded":"2023-11-27T15:21:02.784302+0000","last_fullsized":"2023-11-27T15:21:02.784302+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:05:39.554898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782542+0000","last_change":"2023-11-27T15:20:32.655896+0000","last_active":"2023-11-27T15:21:02.782542+0000","last_peered":"2023-11-27T15:21:02.782542+0000","last_clean":"2023-11-27T15:21:02.782542+0000","last_became_active":"2023-11-27T15:20:32.655623+0000","last_became_peered":"2023-11-27T15:20:32.655623+0000","last_unstale":"2023-11-27T15:21:02.782542+0000","last_undegraded":"2023-11-27T15:21:02.782542+0000","last_fullsized":"2023-11-27T15:21:02.782542+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:01:57.075773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779947+0000","last_change":"2023-11-27T15:20:33.227318+0000","last_active":"2023-11-27T15:21:02.779947+0000","last_peered":"2023-11-27T15:21:02.779947+0000","last_clean":"2023-11-27T15:21:02.779947+0000","last_became_active":"2023-11-27T15:20:33.226944+0000","last_became_peered":"2023-11-27T15:20:33.226944+0000","last_unstale":"2023-11-27T15:21:02.779947+0000","last_undegraded":"2023-11-27T15:21:02.779947+0000","last_fullsized":"2023-11-27T15:21:02.779947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:47:26.971182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.794990+0000","last_change":"2023-11-27T15:20:37.669098+0000","last_active":"2023-11-27T15:21:02.794990+0000","last_peered":"2023-11-27T15:21:02.794990+0000","last_clean":"2023-11-27T15:21:02.794990+0000","last_became_active":"2023-11-27T15:20:37.668693+0000","last_became_peered":"2023-11-27T15:20:37.668693+0000","last_unstale":"2023-11-27T15:21:02.794990+0000","last_undegraded":"2023-11-27T15:21:02.794990+0000","last_fullsized":"2023-11-27T15:21:02.794990+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:38:51.699084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.802617+0000","last_change":"2023-11-27T15:20:32.660285+0000","last_active":"2023-11-27T15:21:03.802617+0000","last_peered":"2023-11-27T15:21:03.802617+0000","last_clean":"2023-11-27T15:21:03.802617+0000","last_became_active":"2023-11-27T15:20:32.659982+0000","last_became_peered":"2023-11-27T15:20:32.659982+0000","last_unstale":"2023-11-27T15:21:03.802617+0000","last_undegraded":"2023-11-27T15:21:03.802617+0000","last_fullsized":"2023-11-27T15:21:03.802617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:42:36.081550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780414+0000","last_change":"2023-11-27T15:20:33.226629+0000","last_active":"2023-11-27T15:21:02.780414+0000","last_peered":"2023-11-27T15:21:02.780414+0000","last_clean":"2023-11-27T15:21:02.780414+0000","last_became_active":"2023-11-27T15:20:33.226183+0000","last_became_peered":"2023-11-27T15:20:33.226183+0000","last_unstale":"2023-11-27T15:21:02.780414+0000","last_undegraded":"2023-11-27T15:21:02.780414+0000","last_fullsized":"2023-11-27T15:21:02.780414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:33:24.895609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785284+0000","last_change":"2023-11-27T15:20:37.680987+0000","last_active":"2023-11-27T15:21:02.785284+0000","last_peered":"2023-11-27T15:21:02.785284+0000","last_clean":"2023-11-27T15:21:02.785284+0000","last_became_active":"2023-11-27T15:20:37.680219+0000","last_became_peered":"2023-11-27T15:20:37.680219+0000","last_unstale":"2023-11-27T15:21:02.785284+0000","last_undegraded":"2023-11-27T15:21:02.785284+0000","last_fullsized":"2023-11-27T15:21:02.785284+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:27:01.990771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932894+0000","last_change":"2023-11-27T15:20:32.655035+0000","last_active":"2023-11-27T15:21:02.932894+0000","last_peered":"2023-11-27T15:21:02.932894+0000","last_clean":"2023-11-27T15:21:02.932894+0000","last_became_active":"2023-11-27T15:20:32.654687+0000","last_became_peered":"2023-11-27T15:20:32.654687+0000","last_unstale":"2023-11-27T15:21:02.932894+0000","last_undegraded":"2023-11-27T15:21:02.932894+0000","last_fullsized":"2023-11-27T15:21:02.932894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:54:01.028543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780460+0000","last_change":"2023-11-27T15:20:33.245908+0000","last_active":"2023-11-27T15:21:02.780460+0000","last_peered":"2023-11-27T15:21:02.780460+0000","last_clean":"2023-11-27T15:21:02.780460+0000","last_became_active":"2023-11-27T15:20:33.245576+0000","last_became_peered":"2023-11-27T15:20:33.245576+0000","last_unstale":"2023-11-27T15:21:02.780460+0000","last_undegraded":"2023-11-27T15:21:02.780460+0000","last_fullsized":"2023-11-27T15:21:02.780460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:41:20.860779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.931498+0000","last_change":"2023-11-27T15:20:37.674253+0000","last_active":"2023-11-27T15:21:02.931498+0000","last_peered":"2023-11-27T15:21:02.931498+0000","last_clean":"2023-11-27T15:21:02.931498+0000","last_became_active":"2023-11-27T15:20:37.673708+0000","last_became_peered":"2023-11-27T15:20:37.673708+0000","last_unstale":"2023-11-27T15:21:02.931498+0000","last_undegraded":"2023-11-27T15:21:02.931498+0000","last_fullsized":"2023-11-27T15:21:02.931498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:25:32.219577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786521+0000","last_change":"2023-11-27T15:20:32.659605+0000","last_active":"2023-11-27T15:21:02.786521+0000","last_peered":"2023-11-27T15:21:02.786521+0000","last_clean":"2023-11-27T15:21:02.786521+0000","last_became_active":"2023-11-27T15:20:32.659186+0000","last_became_peered":"2023-11-27T15:20:32.659186+0000","last_unstale":"2023-11-27T15:21:02.786521+0000","last_undegraded":"2023-11-27T15:21:02.786521+0000","last_fullsized":"2023-11-27T15:21:02.786521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:19:33.727199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780674+0000","last_change":"2023-11-27T15:20:33.227914+0000","last_active":"2023-11-27T15:21:02.780674+0000","last_peered":"2023-11-27T15:21:02.780674+0000","last_clean":"2023-11-27T15:21:02.780674+0000","last_became_active":"2023-11-27T15:20:33.227572+0000","last_became_peered":"2023-11-27T15:20:33.227572+0000","last_unstale":"2023-11-27T15:21:02.780674+0000","last_undegraded":"2023-11-27T15:21:02.780674+0000","last_fullsized":"2023-11-27T15:21:02.780674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:16:00.098620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932874+0000","last_change":"2023-11-27T15:20:33.215605+0000","last_active":"2023-11-27T15:21:02.932874+0000","last_peered":"2023-11-27T15:21:02.932874+0000","last_clean":"2023-11-27T15:21:02.932874+0000","last_became_active":"2023-11-27T15:20:33.215295+0000","last_became_peered":"2023-11-27T15:20:33.215295+0000","last_unstale":"2023-11-27T15:21:02.932874+0000","last_undegraded":"2023-11-27T15:21:02.932874+0000","last_fullsized":"2023-11-27T15:21:02.932874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:46:08.937950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780509+0000","last_change":"2023-11-27T15:20:33.214014+0000","last_active":"2023-11-27T15:21:02.780509+0000","last_peered":"2023-11-27T15:21:02.780509+0000","last_clean":"2023-11-27T15:21:02.780509+0000","last_became_active":"2023-11-27T15:20:33.213685+0000","last_became_peered":"2023-11-27T15:20:33.213685+0000","last_unstale":"2023-11-27T15:21:02.780509+0000","last_undegraded":"2023-11-27T15:21:02.780509+0000","last_fullsized":"2023-11-27T15:21:02.780509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:06:53.889016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779696+0000","last_change":"2023-11-27T15:20:33.229722+0000","last_active":"2023-11-27T15:21:02.779696+0000","last_peered":"2023-11-27T15:21:02.779696+0000","last_clean":"2023-11-27T15:21:02.779696+0000","last_became_active":"2023-11-27T15:20:33.229372+0000","last_became_peered":"2023-11-27T15:20:33.229372+0000","last_unstale":"2023-11-27T15:21:02.779696+0000","last_undegraded":"2023-11-27T15:21:02.779696+0000","last_fullsized":"2023-11-27T15:21:02.779696+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:48:26.454003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785590+0000","last_change":"2023-11-27T15:20:33.235047+0000","last_active":"2023-11-27T15:21:02.785590+0000","last_peered":"2023-11-27T15:21:02.785590+0000","last_clean":"2023-11-27T15:21:02.785590+0000","last_became_active":"2023-11-27T15:20:33.233078+0000","last_became_peered":"2023-11-27T15:20:33.233078+0000","last_unstale":"2023-11-27T15:21:02.785590+0000","last_undegraded":"2023-11-27T15:21:02.785590+0000","last_fullsized":"2023-11-27T15:21:02.785590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:56:34.043375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784969+0000","last_change":"2023-11-27T15:20:33.229118+0000","last_active":"2023-11-27T15:21:02.784969+0000","last_peered":"2023-11-27T15:21:02.784969+0000","last_clean":"2023-11-27T15:21:02.784969+0000","last_became_active":"2023-11-27T15:20:33.228748+0000","last_became_peered":"2023-11-27T15:20:33.228748+0000","last_unstale":"2023-11-27T15:21:02.784969+0000","last_undegraded":"2023-11-27T15:21:02.784969+0000","last_fullsized":"2023-11-27T15:21:02.784969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:44:41.459719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.933214+0000","last_change":"2023-11-27T15:20:33.222714+0000","last_active":"2023-11-27T15:21:02.933214+0000","last_peered":"2023-11-27T15:21:02.933214+0000","last_clean":"2023-11-27T15:21:02.933214+0000","last_became_active":"2023-11-27T15:20:33.222396+0000","last_became_peered":"2023-11-27T15:20:33.222396+0000","last_unstale":"2023-11-27T15:21:02.933214+0000","last_undegraded":"2023-11-27T15:21:02.933214+0000","last_fullsized":"2023-11-27T15:21:02.933214+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:13:29.493579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780269+0000","last_change":"2023-11-27T15:20:33.223568+0000","last_active":"2023-11-27T15:21:02.780269+0000","last_peered":"2023-11-27T15:21:02.780269+0000","last_clean":"2023-11-27T15:21:02.780269+0000","last_became_active":"2023-11-27T15:20:33.220687+0000","last_became_peered":"2023-11-27T15:20:33.220687+0000","last_unstale":"2023-11-27T15:21:02.780269+0000","last_undegraded":"2023-11-27T15:21:02.780269+0000","last_fullsized":"2023-11-27T15:21:02.780269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:56:29.431345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786992+0000","last_change":"2023-11-27T15:21:02.786992+0000","last_active":"2023-11-27T15:21:02.786992+0000","last_peered":"2023-11-27T15:21:02.786992+0000","last_clean":"2023-11-27T15:21:02.786992+0000","last_became_active":"2023-11-27T15:21:02.786644+0000","last_became_peered":"2023-11-27T15:21:02.786644+0000","last_unstale":"2023-11-27T15:21:02.786992+0000","last_undegraded":"2023-11-27T15:21:02.786992+0000","last_fullsized":"2023-11-27T15:21:02.786992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:18:27.697514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.931958+0000","last_change":"2023-11-27T15:20:33.245815+0000","last_active":"2023-11-27T15:21:02.931958+0000","last_peered":"2023-11-27T15:21:02.931958+0000","last_clean":"2023-11-27T15:21:02.931958+0000","last_became_active":"2023-11-27T15:20:33.245243+0000","last_became_peered":"2023-11-27T15:20:33.245243+0000","last_unstale":"2023-11-27T15:21:02.931958+0000","last_undegraded":"2023-11-27T15:21:02.931958+0000","last_fullsized":"2023-11-27T15:21:02.931958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:26:43.029408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784226+0000","last_change":"2023-11-27T15:20:33.245595+0000","last_active":"2023-11-27T15:21:02.784226+0000","last_peered":"2023-11-27T15:21:02.784226+0000","last_clean":"2023-11-27T15:21:02.784226+0000","last_became_active":"2023-11-27T15:20:33.245294+0000","last_became_peered":"2023-11-27T15:20:33.245294+0000","last_unstale":"2023-11-27T15:21:02.784226+0000","last_undegraded":"2023-11-27T15:21:02.784226+0000","last_fullsized":"2023-11-27T15:21:02.784226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:38:31.036319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.795538+0000","last_change":"2023-11-27T15:20:33.232976+0000","last_active":"2023-11-27T15:21:02.795538+0000","last_peered":"2023-11-27T15:21:02.795538+0000","last_clean":"2023-11-27T15:21:02.795538+0000","last_became_active":"2023-11-27T15:20:33.232648+0000","last_became_peered":"2023-11-27T15:20:33.232648+0000","last_unstale":"2023-11-27T15:21:02.795538+0000","last_undegraded":"2023-11-27T15:21:02.795538+0000","last_fullsized":"2023-11-27T15:21:02.795538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:45:02.656102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783702+0000","last_change":"2023-11-27T15:20:33.226332+0000","last_active":"2023-11-27T15:21:02.783702+0000","last_peered":"2023-11-27T15:21:02.783702+0000","last_clean":"2023-11-27T15:21:02.783702+0000","last_became_active":"2023-11-27T15:20:33.226000+0000","last_became_peered":"2023-11-27T15:20:33.226000+0000","last_unstale":"2023-11-27T15:21:02.783702+0000","last_undegraded":"2023-11-27T15:21:02.783702+0000","last_fullsized":"2023-11-27T15:21:02.783702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:24:34.018543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932933+0000","last_change":"2023-11-27T15:20:33.217927+0000","last_active":"2023-11-27T15:21:02.932933+0000","last_peered":"2023-11-27T15:21:02.932933+0000","last_clean":"2023-11-27T15:21:02.932933+0000","last_became_active":"2023-11-27T15:20:33.217003+0000","last_became_peered":"2023-11-27T15:20:33.217003+0000","last_unstale":"2023-11-27T15:21:02.932933+0000","last_undegraded":"2023-11-27T15:21:02.932933+0000","last_fullsized":"2023-11-27T15:21:02.932933+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:27:50.580759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63712,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":306,"num_read_kb":223,"num_write":158,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":191136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416412,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6176,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93738976,"statfs":{"total":95995035648,"available":95988711424,"internally_reserved":0,"allocated":1007616,"data_stored":494720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.61599999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.66400000000000003}]},{"osd":2,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":4,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.84899999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.70399999999999996}]},{"osd":6,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.75600000000000001}]},{"osd":7,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.95699999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:21:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.80200000000000005}]},{"osd":9,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.877},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.96799999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:20:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.90800000000000003}]}]},{"osd":10,"up_from":71,"seq":304942678048,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7172,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93737980,"statfs":{"total":95995035648,"available":95987691520,"internally_reserved":0,"allocated":1699840,"data_stored":1146234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.438}]},{"osd":1,"last update":"Mon Nov 27 15:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.47199999999999998,"15min":0.47799999999999998},"min":{"1min":0.309,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.60099999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.46400000000000002,"15min":0.46800000000000003},"min":{"1min":0.35899999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60999999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.40699999999999997}]},{"osd":2,"last update":"Mon Nov 27 15:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.40100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.754,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.42099999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.75800000000000001,"5min":0.87,"15min":0.87},"last":0.63300000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.63100000000000001,"15min":0.64300000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71999999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.65300000000000002,"15min":0.66600000000000004},"min":{"1min":0.44700000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76600000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.52700000000000002}]},{"osd":4,"last update":"Mon Nov 27 15:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72899999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.40799999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77900000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.495}]},{"osd":5,"last update":"Mon Nov 27 15:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.63400000000000001,"15min":0.624},"min":{"1min":0.55500000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.95599999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.497,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.875,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.754}]},{"osd":6,"last update":"Mon Nov 27 15:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.89000000000000001,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.71999999999999997,"15min":0.73099999999999998},"min":{"1min":0.55700000000000005,"5min":0.496,"15min":0.496},"max":{"1min":0.79400000000000004,"5min":1.004,"15min":1.004},"last":0.62}]},{"osd":7,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.56799999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.75700000000000001,"15min":0.76800000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.93200000000000005,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.627,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.879,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.70099999999999996}]},{"osd":9,"last update":"Mon Nov 27 15:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.78500000000000003,"15min":0.79600000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.90800000000000003,"5min":1.111,"15min":1.111},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.58699999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.91900000000000004,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.73999999999999999}]},{"osd":11,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.746,"15min":0.75},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.77000000000000002,"15min":0.77900000000000003},"min":{"1min":0.56999999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.92600000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.71299999999999997}]}]},{"osd":9,"up_from":66,"seq":283467841571,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6384,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738768,"statfs":{"total":95995035648,"available":95988498432,"internally_reserved":0,"allocated":1024000,"data_stored":494956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.51500000000000001,"15min":0.52300000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40899999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.004,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.40000000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.72799999999999998,"15min":0.73699999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64700000000000002,"15min":0.65200000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80500000000000005,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.54400000000000004}]},{"osd":2,"last update":"Mon Nov 27 15:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.78800000000000003,"15min":0.79800000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.76300000000000001,"15min":0.77900000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97999999999999998,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.65100000000000002}]},{"osd":3,"last update":"Mon Nov 27 15:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.69599999999999995,"15min":0.71199999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.97099999999999997,"5min":1.141,"15min":1.141},"last":0.435},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.77500000000000002,"15min":0.79800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.127,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.67500000000000004}]},{"osd":4,"last update":"Mon Nov 27 15:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.74199999999999999,"15min":0.75700000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81899999999999995,"5min":1.171,"15min":1.171},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.80100000000000005,"15min":0.81499999999999995},"min":{"1min":0.55800000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0740000000000001,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.50800000000000001}]},{"osd":5,"last update":"Mon Nov 27 15:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.83899999999999997,"15min":0.85299999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1020000000000001,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.873,"15min":0.89300000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.72999999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.80400000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.89200000000000002,"15min":0.91300000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0640000000000001,"5min":1.202,"15min":1.202},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.93600000000000005,"15min":0.95599999999999996},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.69499999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.94699999999999995,"15min":0.97699999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0449999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.95599999999999996,"15min":0.97599999999999998},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.212,"5min":1.536,"15min":1.536},"last":0.55800000000000005}]},{"osd":10,"last update":"Mon Nov 27 15:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.91800000000000004,"15min":0.93400000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.85999999999999999,"15min":0.872},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.92100000000000004,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.48299999999999998}]},{"osd":11,"last update":"Mon Nov 27 15:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.60699999999999998}]}]},{"osd":8,"up_from":58,"seq":249108103210,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6508,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738644,"statfs":{"total":95995035648,"available":95988371456,"internally_reserved":0,"allocated":1019904,"data_stored":494545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.44400000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.59799999999999998},"min":{"1min":0.45700000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.75,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.50600000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.59999999999999998,"15min":0.59899999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.60599999999999998,"15min":0.59699999999999998},"min":{"1min":0.42699999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.749}]},{"osd":2,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.44600000000000001,"15min":0.439},"min":{"1min":0.376,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.46600000000000003,"15min":0.44800000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.66600000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.625}]},{"osd":3,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.60599999999999998,"15min":0.60099999999999998},"min":{"1min":0.503,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.45300000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.53800000000000003}]},{"osd":4,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63,"15min":0.628},"min":{"1min":0.46700000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.84699999999999998,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86599999999999999,"5min":1.377,"15min":1.377},"last":0.59199999999999997}]},{"osd":5,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.55600000000000005},"min":{"1min":0.39100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67000000000000004}]},{"osd":6,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.73799999999999999,"15min":0.68400000000000005},"min":{"1min":0.47599999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":6.4450000000000003,"5min":6.4450000000000003,"15min":6.4450000000000003},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.72299999999999998,"15min":0.70299999999999996},"min":{"1min":0.55200000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.94699999999999995}]},{"osd":7,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.73999999999999999,"15min":0.72199999999999998},"min":{"1min":0.53200000000000003,"5min":0.442,"15min":0.442},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.434,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.99299999999999999,"5min":1.323,"15min":1.323},"last":0.77100000000000002}]},{"osd":9,"last update":"Mon Nov 27 15:21:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.56799999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.78600000000000003,"15min":0.77700000000000002},"min":{"1min":0.65300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.877}]},{"osd":10,"last update":"Mon Nov 27 15:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.72099999999999997,"15min":0.70499999999999996},"min":{"1min":0.59799999999999998,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.752,"15min":0.73299999999999998},"min":{"1min":0.628,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.90200000000000002}]},{"osd":11,"last update":"Mon Nov 27 15:21:36 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.80200000000000005}]}]},{"osd":7,"up_from":51,"seq":219043332141,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7156,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93737996,"statfs":{"total":95995035648,"available":95987707904,"internally_reserved":0,"allocated":1617920,"data_stored":1090688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.436,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84699999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.44},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77100000000000002,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.46999999999999997}]},{"osd":1,"last update":"Mon Nov 27 15:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40100000000000002,"15min":0.39100000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.54300000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.439,"5min":0.40899999999999997,"15min":0.39900000000000002},"min":{"1min":0.28399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63700000000000001,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.496}]},{"osd":2,"last update":"Mon Nov 27 15:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.433,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.79400000000000004}]},{"osd":3,"last update":"Mon Nov 27 15:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.39500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89600000000000002,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.64000000000000001,"15min":0.63600000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77600000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.69399999999999995}]},{"osd":4,"last update":"Mon Nov 27 15:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50800000000000001,"15min":0.49199999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.76800000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.51800000000000002,"15min":0.52600000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70399999999999996,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.39200000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.625,"15min":0.63},"min":{"1min":0.47099999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.75},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.623,"15min":0.61899999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.61399999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68500000000000005,"15min":0.67200000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.036,"5min":1.046,"15min":1.046},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68999999999999995,"15min":0.69399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93999999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.81799999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:21:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.66900000000000004,"15min":0.65100000000000002},"min":{"1min":0.60399999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.53100000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.873}]},{"osd":9,"last update":"Mon Nov 27 15:21:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.76500000000000001,"15min":0.76700000000000002},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.97899999999999998,"5min":0.996,"15min":0.996},"last":0.76300000000000001}]},{"osd":10,"last update":"Mon Nov 27 15:21:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.499,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.51600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.85799999999999998}]},{"osd":11,"last update":"Mon Nov 27 15:21:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.93000000000000005}]}]},{"osd":6,"up_from":46,"seq":197568495666,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6428,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738724,"statfs":{"total":95995035648,"available":95988453376,"internally_reserved":0,"allocated":1003520,"data_stored":490132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47699999999999998,"15min":0.46999999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.499},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.50700000000000001,"15min":0.48499999999999999},"min":{"1min":0.36599999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.89100000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65600000000000003,"15min":0.63600000000000001},"min":{"1min":0.45300000000000001,"5min":0.441,"15min":0.441},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60899999999999999,"15min":0.57899999999999996},"min":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.55700000000000005}]},{"osd":2,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.64800000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68200000000000005,"15min":0.67400000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.91600000000000004}]},{"osd":3,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55700000000000005,"15min":0.54400000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.61299999999999999,"15min":0.59599999999999997},"min":{"1min":0.38900000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.96399999999999997}]},{"osd":4,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69399999999999995,"15min":0.68600000000000005},"min":{"1min":0.55700000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68500000000000005,"15min":0.66900000000000004},"min":{"1min":0.48799999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.63100000000000001}]},{"osd":5,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71899999999999997,"15min":0.70799999999999996},"min":{"1min":0.505,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76300000000000001,"15min":0.753},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.94899999999999995}]},{"osd":7,"last update":"Mon Nov 27 15:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.75900000000000001,"15min":0.749},"min":{"1min":0.60099999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.048,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.73499999999999999,"15min":0.71099999999999997},"min":{"1min":0.57599999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97399999999999998,"5min":1.419,"15min":1.419},"last":0.93600000000000005}]},{"osd":8,"last update":"Mon Nov 27 15:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85799999999999998,"15min":0.86399999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.22,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.84799999999999998,"15min":0.85799999999999998},"min":{"1min":0.60699999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.77100000000000002}]},{"osd":9,"last update":"Mon Nov 27 15:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.85199999999999998,"15min":0.86699999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0309999999999999,"5min":1.448,"15min":1.448},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.91200000000000003,"15min":0.92700000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1950000000000001,"5min":1.712,"15min":1.712},"last":1.0209999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.82699999999999996,"15min":0.82299999999999995},"min":{"1min":0.63800000000000001,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0880000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.89700000000000002,"15min":0.88700000000000001},"min":{"1min":0.69799999999999995,"5min":0.626,"15min":0.626},"max":{"1min":1.454,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.98299999999999998}]},{"osd":11,"last update":"Mon Nov 27 15:21:38 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":4.9290000000000003,"5min":4.9290000000000003,"15min":4.9290000000000003},"last":1.036},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":1.0429999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7516,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737636,"statfs":{"total":95995035648,"available":95987339264,"internally_reserved":0,"allocated":1069056,"data_stored":549850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.52500000000000002,"15min":0.49099999999999999},"min":{"1min":0.40500000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.54800000000000004,"15min":0.49399999999999999},"min":{"1min":0.42599999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.92200000000000004,"5min":1.012,"15min":1.012},"last":0.59199999999999997}]},{"osd":2,"last update":"Mon Nov 27 15:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.52300000000000002,"15min":0.48699999999999999},"min":{"1min":0.42099999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.81299999999999994,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.56200000000000006,"15min":0.499},"min":{"1min":0.41199999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.73199999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.55500000000000005,"15min":0.52800000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54700000000000004,"15min":0.53400000000000003},"min":{"1min":0.39700000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84899999999999998,"5min":0.88,"15min":0.88},"last":0.438}]},{"osd":4,"last update":"Mon Nov 27 15:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.44,"15min":0.40000000000000002},"min":{"1min":0.34499999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.45600000000000002,"15min":0.41399999999999998},"min":{"1min":0.316,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79000000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.48999999999999999}]},{"osd":5,"last update":"Mon Nov 27 15:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.57499999999999996,"15min":0.54700000000000004},"min":{"1min":0.48499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.88800000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.61299999999999999,"15min":0.56899999999999995},"min":{"1min":0.46600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.76400000000000001}]},{"osd":6,"last update":"Mon Nov 27 15:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.64700000000000002,"15min":0.624},"min":{"1min":0.45900000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.64400000000000002,"15min":0.62},"min":{"1min":0.53400000000000003,"5min":0.38,"15min":0.38},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.56499999999999995}]},{"osd":7,"last update":"Mon Nov 27 15:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58499999999999996,"15min":0.56399999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.621,"15min":0.59399999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95599999999999996,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.61499999999999999}]},{"osd":8,"last update":"Mon Nov 27 15:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.66700000000000004,"15min":0.64400000000000002},"min":{"1min":0.45800000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92100000000000004,"5min":0.995,"15min":0.995},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.66300000000000003,"15min":0.64400000000000002},"min":{"1min":0.53400000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.69899999999999995}]},{"osd":9,"last update":"Mon Nov 27 15:21:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.754,"15min":0.73999999999999999},"min":{"1min":0.55700000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.76700000000000002,"15min":0.75700000000000001},"min":{"1min":0.60099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0549999999999999,"5min":1.169,"15min":1.169},"last":0.71599999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.504,"5min":0.439,"15min":0.439},"max":{"1min":0.98499999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.749},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.77500000000000002,"15min":0.76900000000000002},"min":{"1min":0.57999999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.79600000000000004}]},{"osd":11,"last update":"Mon Nov 27 15:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.84799999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7412,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737740,"statfs":{"total":95995035648,"available":95987445760,"internally_reserved":0,"allocated":1028096,"data_stored":491520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 27 15:21:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.57299999999999995,"15min":0.51100000000000001},"min":{"1min":0.42599999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.89500000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59399999999999997,"15min":0.52700000000000002},"min":{"1min":0.45900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.86699999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.71899999999999997}]},{"osd":2,"last update":"Mon Nov 27 15:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.62,"15min":0.57999999999999996},"min":{"1min":0.41299999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.88700000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61899999999999999,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83299999999999996,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.66200000000000003}]},{"osd":3,"last update":"Mon Nov 27 15:21:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.53900000000000003}]},{"osd":4,"last update":"Mon Nov 27 15:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.46400000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.876,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65400000000000003,"15min":0.60799999999999998},"min":{"1min":0.51300000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.97199999999999998,"5min":1.244,"15min":1.244},"last":0.79800000000000004}]},{"osd":5,"last update":"Mon Nov 27 15:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.46000000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84599999999999997,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.70599999999999996,"15min":0.68200000000000005},"min":{"1min":0.52300000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90200000000000002,"5min":1.379,"15min":1.379},"last":0.69999999999999996}]},{"osd":6,"last update":"Mon Nov 27 15:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65500000000000003,"15min":0.627},"min":{"1min":0.41499999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.121,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63600000000000001,"15min":0.61799999999999999},"min":{"1min":0.437,"5min":0.373,"15min":0.373},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.61899999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.77800000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.755,"15min":0.754},"min":{"1min":0.59799999999999998,"5min":0.498,"15min":0.498},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76500000000000001,"15min":0.78100000000000003},"min":{"1min":0.54400000000000004,"5min":0.504,"15min":0.504},"max":{"1min":0.94099999999999995,"5min":1.327,"15min":1.327},"last":0.73699999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:21:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.874,"15min":0.88},"min":{"1min":0.68300000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.1000000000000001,"5min":1.218,"15min":1.218},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.84899999999999998,"15min":0.85799999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95399999999999996,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.94199999999999995}]},{"osd":10,"last update":"Mon Nov 27 15:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.82299999999999995,"15min":0.83099999999999996},"min":{"1min":0.63800000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.94699999999999995,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.89000000000000001,"15min":0.89300000000000002},"min":{"1min":0.70499999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.1399999999999999,"5min":1.409,"15min":1.409},"last":0.86899999999999999}]},{"osd":11,"last update":"Mon Nov 27 15:20:50 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.82299999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7600,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":93737552,"statfs":{"total":95995035648,"available":95987253248,"internally_reserved":0,"allocated":1089536,"data_stored":550483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53000000000000003,"15min":0.50600000000000001},"min":{"1min":0.42699999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.78800000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.33100000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.83799999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.64500000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:21:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.54800000000000004,"15min":0.497},"min":{"1min":0.40799999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.86199999999999999,"5min":1.079,"15min":1.079},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.55200000000000005,"15min":0.505},"min":{"1min":0.436,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.78600000000000003}]},{"osd":3,"last update":"Mon Nov 27 15:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58199999999999996,"15min":0.56299999999999994},"min":{"1min":0.443,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.57199999999999995,"15min":0.56100000000000005},"min":{"1min":0.42599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77000000000000002,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.77200000000000002}]},{"osd":4,"last update":"Mon Nov 27 15:21:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.84099999999999997}]},{"osd":5,"last update":"Mon Nov 27 15:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.45900000000000002,"15min":0.43099999999999999},"min":{"1min":0.35399999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.48199999999999998,"15min":0.45500000000000002},"min":{"1min":0.32600000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.61399999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.62,"15min":0.60299999999999998},"min":{"1min":0.42099999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.93600000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.81699999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.376,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85599999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.61599999999999999,"15min":0.60499999999999998},"min":{"1min":0.47199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.88100000000000001,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.86799999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.67200000000000004,"15min":0.66000000000000003},"min":{"1min":0.50900000000000001,"5min":0.502,"15min":0.502},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.70199999999999996,"15min":0.69099999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.98999999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:21:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.755,"15min":0.749},"min":{"1min":0.51000000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.90600000000000003,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.878},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.70999999999999996,"15min":0.68600000000000005},"min":{"1min":0.54600000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.90400000000000003}]},{"osd":11,"last update":"Mon Nov 27 15:21:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.91900000000000004}]}]},{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6760,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738392,"statfs":{"total":95995035648,"available":95988113408,"internally_reserved":0,"allocated":1015808,"data_stored":490326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.42499999999999999,"15min":0.40000000000000002},"min":{"1min":0.33400000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72099999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.442,"5min":0.45600000000000002,"15min":0.41699999999999998},"min":{"1min":0.29999999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.58399999999999996,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.41499999999999998}]},{"osd":1,"last update":"Mon Nov 27 15:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61099999999999999,"15min":0.56499999999999995},"min":{"1min":0.42999999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.872,"5min":1.056,"15min":1.056},"last":0.621},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63500000000000001,"15min":0.57099999999999995},"min":{"1min":0.40699999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90300000000000002,"5min":5.6529999999999996,"15min":5.6529999999999996},"last":0.57499999999999996}]},{"osd":2,"last update":"Mon Nov 27 15:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64000000000000001,"15min":0.59399999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85099999999999998,"5min":1.208,"15min":1.208},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.63200000000000001,"15min":0.58799999999999997},"min":{"1min":0.49099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78900000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.56100000000000005}]},{"osd":4,"last update":"Mon Nov 27 15:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.64400000000000002,"15min":0.627},"min":{"1min":0.45100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86599999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.71099999999999997,"15min":0.67400000000000004},"min":{"1min":0.57699999999999996,"5min":0.502,"15min":0.502},"max":{"1min":0.93100000000000005,"5min":1.343,"15min":1.343},"last":0.70799999999999996}]},{"osd":5,"last update":"Mon Nov 27 15:21:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68500000000000005,"15min":0.65500000000000003},"min":{"1min":0.49299999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0629999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67700000000000005,"15min":0.65600000000000003},"min":{"1min":0.55000000000000004,"5min":0.437,"15min":0.437},"max":{"1min":0.79100000000000004,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.68600000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.65600000000000003,"15min":0.63800000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0269999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.41399999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94399999999999995,"5min":1.018,"15min":1.018},"last":0.47599999999999998}]},{"osd":7,"last update":"Mon Nov 27 15:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.95499999999999996,"15min":0.97599999999999998},"min":{"1min":0.53500000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.93999999999999995,"5min":6.9619999999999997,"15min":6.9619999999999997},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.78000000000000003,"15min":0.77600000000000002},"min":{"1min":0.56599999999999995,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.99099999999999999,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.76400000000000001}]},{"osd":8,"last update":"Mon Nov 27 15:19:38 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":5.6100000000000003,"5min":5.6100000000000003,"15min":5.6100000000000003},"last":0.752},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.77500000000000002}]},{"osd":9,"last update":"Mon Nov 27 15:21:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.84299999999999997,"15min":0.85499999999999998},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1020000000000001,"5min":1.161,"15min":1.161},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.84199999999999997,"15min":0.85199999999999998},"min":{"1min":0.496,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95699999999999996,"5min":1.496,"15min":1.496},"last":0.79700000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:21:27 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.77700000000000002,"15min":0.78200000000000003},"min":{"1min":0.53400000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.93200000000000005,"5min":1.157,"15min":1.157},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.82899999999999996,"15min":0.83499999999999996},"min":{"1min":0.61899999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.98399999999999999,"5min":1.129,"15min":1.129},"last":0.78800000000000003}]},{"osd":11,"last update":"Mon Nov 27 15:21:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.82199999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920828,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6632,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738520,"statfs":{"total":95995035648,"available":95988244480,"internally_reserved":0,"allocated":1015808,"data_stored":495124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.54400000000000004,"15min":0.53700000000000003},"min":{"1min":0.38300000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53500000000000003,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77900000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.54900000000000004}]},{"osd":1,"last update":"Mon Nov 27 15:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.39000000000000001,"15min":0.35399999999999998},"min":{"1min":0.28599999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.495,"5min":0.40300000000000002,"15min":0.36699999999999999},"min":{"1min":0.32200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.55700000000000005}]},{"osd":2,"last update":"Mon Nov 27 15:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.57699999999999996,"15min":0.55700000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.59099999999999997,"15min":0.54900000000000004},"min":{"1min":0.432,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.68500000000000005}]},{"osd":3,"last update":"Mon Nov 27 15:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57699999999999996,"15min":0.55100000000000005},"min":{"1min":0.41799999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59199999999999997,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.169,"5min":1.722,"15min":1.722},"last":0.59299999999999997}]},{"osd":5,"last update":"Mon Nov 27 15:21:34 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.60799999999999998},"min":{"1min":0.42699999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76200000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.87},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.629,"15min":0.60599999999999998},"min":{"1min":0.53800000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84999999999999998,"5min":1.28,"15min":1.28},"last":0.92600000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.45800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.88},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.66900000000000004,"15min":0.65700000000000003},"min":{"1min":0.53700000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.74199999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58299999999999996,"15min":0.56599999999999995},"min":{"1min":0.40500000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.014,"5min":1.054,"15min":1.054},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61599999999999999,"15min":0.59699999999999998},"min":{"1min":0.443,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.98099999999999998}]},{"osd":8,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71399999999999997,"15min":0.70499999999999996},"min":{"1min":0.50600000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.99099999999999999,"5min":1.522,"15min":1.522},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92700000000000005,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.002}]},{"osd":9,"last update":"Mon Nov 27 15:21:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.755,"15min":0.753},"min":{"1min":0.55100000000000005,"5min":0.44,"15min":0.44},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68200000000000005,"15min":0.68100000000000005},"min":{"1min":0.52500000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.77200000000000002}]},{"osd":10,"last update":"Mon Nov 27 15:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.56299999999999994,"5min":0.503,"15min":0.503},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.64100000000000001,"15min":0.64500000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.85099999999999998}]},{"osd":11,"last update":"Mon Nov 27 15:21:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.89500000000000002}]}]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7288,"kb_used_data":1584,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93737864,"statfs":{"total":95995035648,"available":95987572736,"internally_reserved":0,"allocated":1622016,"data_stored":1085603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.38500000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67000000000000004,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52300000000000002,"15min":0.51900000000000002},"min":{"1min":0.374,"5min":0.371,"15min":0.371},"max":{"1min":0.67600000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.52900000000000003}]},{"osd":1,"last update":"Mon Nov 27 15:21:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56000000000000005,"15min":0.55000000000000004},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77100000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56499999999999995,"15min":0.53900000000000003},"min":{"1min":0.38400000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.439}]},{"osd":2,"last update":"Mon Nov 27 15:21:35 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.32600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.504,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.42299999999999999,"15min":0.42399999999999999},"min":{"1min":0.33300000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.54700000000000004,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.376}]},{"osd":3,"last update":"Mon Nov 27 15:21:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55400000000000005,"15min":0.56100000000000005},"min":{"1min":0.44600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74099999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.44500000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.64500000000000002,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.46800000000000003}]},{"osd":4,"last update":"Mon Nov 27 15:21:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60999999999999999,"15min":0.59399999999999997},"min":{"1min":0.46999999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79200000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.39100000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80400000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.45500000000000002}]},{"osd":6,"last update":"Mon Nov 27 15:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.625,"15min":0.63500000000000001},"min":{"1min":0.41899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.44400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.93500000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.65900000000000003}]},{"osd":7,"last update":"Mon Nov 27 15:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63700000000000001,"15min":0.64600000000000002},"min":{"1min":0.433,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85199999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.503},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63700000000000001,"15min":0.63800000000000001},"min":{"1min":0.48599999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83599999999999997,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.57499999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60499999999999998,"15min":0.61899999999999999},"min":{"1min":0.44400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71699999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.62},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.443,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77500000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.56299999999999994}]},{"osd":9,"last update":"Mon Nov 27 15:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.73099999999999998,"15min":0.747},"min":{"1min":0.497,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.753,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.73099999999999998,"15min":0.73799999999999999},"min":{"1min":0.54000000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.90400000000000003,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.72399999999999998}]},{"osd":10,"last update":"Mon Nov 27 15:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68000000000000005,"15min":0.67700000000000005},"min":{"1min":0.56000000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69199999999999995,"15min":0.68600000000000005},"min":{"1min":0.59199999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.86699999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.70099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-27T15:21:54.351 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph pg dump --format=json 2023-11-27T15:21:55.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:54 smithi177 ceph-mon[131504]: pgmap v316: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:21:55.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:54 smithi177 ceph-mon[131504]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:21:55.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:54 smithi174 ceph-mon[131110]: pgmap v316: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:21:55.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:54 smithi174 ceph-mon[131110]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:21:55.199 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:55.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:54 smithi133 ceph-mon[97860]: pgmap v316: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:21:55.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:54 smithi133 ceph-mon[97860]: from='client.14973 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:21:56.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.691+0000 7f755a40d700 1 -- 172.21.15.133:0/2466021617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7554100ee0 msgr2=0x7f7554104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:56.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.691+0000 7f755a40d700 1 --2- 172.21.15.133:0/2466021617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7554100ee0 0x7f7554104eb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7548009990 tx=0x7f7548009ca0 comp rx=0 tx=0).stop 2023-11-27T15:21:56.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.692+0000 7f755a40d700 1 -- 172.21.15.133:0/2466021617 shutdown_connections 2023-11-27T15:21:56.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.692+0000 7f755a40d700 1 --2- 172.21.15.133:0/2466021617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75541055e0 0x7f755410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:56.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.692+0000 7f755a40d700 1 --2- 172.21.15.133:0/2466021617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7554100540 0x7f7554100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:56.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.692+0000 7f755a40d700 1 --2- 172.21.15.133:0/2466021617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7554100ee0 0x7f7554104eb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:56.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.692+0000 7f755a40d700 1 -- 172.21.15.133:0/2466021617 >> 172.21.15.133:0/2466021617 conn(0x7f75540fbdc0 msgr2=0x7f75540fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:56.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.692+0000 7f755a40d700 1 -- 172.21.15.133:0/2466021617 shutdown_connections 2023-11-27T15:21:56.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.692+0000 7f755a40d700 1 -- 172.21.15.133:0/2466021617 wait complete. 2023-11-27T15:21:56.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.693+0000 7f755a40d700 1 Processor -- start 2023-11-27T15:21:56.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.693+0000 7f755a40d700 1 -- start start 2023-11-27T15:21:56.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.693+0000 7f755a40d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7554100540 0x7f7554072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:56.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.693+0000 7f755a40d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7554100ee0 0x7f7554073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:56.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.693+0000 7f755a40d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75541055e0 0x7f7554073870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:56.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.693+0000 7f755a40d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75541a4830 con 0x7f7554100540 2023-11-27T15:21:56.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.693+0000 7f755a40d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75541a49a0 con 0x7f75541055e0 2023-11-27T15:21:56.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.693+0000 7f755a40d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75541a4b60 con 0x7f7554100ee0 2023-11-27T15:21:56.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.693+0000 7f7553fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7554100540 0x7f7554072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:56.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.694+0000 7f7553fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7554100540 0x7f7554072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40088/0 (socket says 172.21.15.133:40088) 2023-11-27T15:21:56.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.694+0000 7f7553fff700 1 -- 172.21.15.133:0/1974761322 learned_addr learned my addr 172.21.15.133:0/1974761322 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:56.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.694+0000 7f75537fe700 1 --2- 172.21.15.133:0/1974761322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7554100ee0 0x7f7554073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:56.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.694+0000 7f75589aa700 1 --2- 172.21.15.133:0/1974761322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75541055e0 0x7f7554073870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:56.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.694+0000 7f7553fff700 1 -- 172.21.15.133:0/1974761322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7554100ee0 msgr2=0x7f7554073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:56.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.694+0000 7f7553fff700 1 --2- 172.21.15.133:0/1974761322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7554100ee0 0x7f7554073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:56.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.694+0000 7f7553fff700 1 -- 172.21.15.133:0/1974761322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75541055e0 msgr2=0x7f7554073870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:56.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.694+0000 7f7553fff700 1 --2- 172.21.15.133:0/1974761322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75541055e0 0x7f7554073870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:56.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.694+0000 7f7553fff700 1 -- 172.21.15.133:0/1974761322 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754c009c40 con 0x7f7554100540 2023-11-27T15:21:56.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.694+0000 7f7553fff700 1 --2- 172.21.15.133:0/1974761322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7554100540 0x7f7554072df0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f754400e9a0 tx=0x7f754400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:56.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.694+0000 7f75517fa700 1 -- 172.21.15.133:0/1974761322 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7544009790 con 0x7f7554100540 2023-11-27T15:21:56.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.695+0000 7f75517fa700 1 -- 172.21.15.133:0/1974761322 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f754400ce80 con 0x7f7554100540 2023-11-27T15:21:56.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.695+0000 7f755a40d700 1 -- 172.21.15.133:0/1974761322 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75480095f0 con 0x7f7554100540 2023-11-27T15:21:56.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.695+0000 7f75517fa700 1 -- 172.21.15.133:0/1974761322 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7544018680 con 0x7f7554100540 2023-11-27T15:21:56.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.695+0000 7f755a40d700 1 -- 172.21.15.133:0/1974761322 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7554110310 con 0x7f7554100540 2023-11-27T15:21:56.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.696+0000 7f755a40d700 1 -- 172.21.15.133:0/1974761322 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7554110180 con 0x7f7554100540 2023-11-27T15:21:56.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.697+0000 7f75517fa700 1 -- 172.21.15.133:0/1974761322 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f7544010430 con 0x7f7554100540 2023-11-27T15:21:56.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.699+0000 7f75517fa700 1 --2- 172.21.15.133:0/1974761322 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f753c0a1260 0x7f753c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:56.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.699+0000 7f75517fa700 1 -- 172.21.15.133:0/1974761322 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7544014070 con 0x7f7554100540 2023-11-27T15:21:56.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.699+0000 7f75537fe700 1 --2- 172.21.15.133:0/1974761322 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f753c0a1260 0x7f753c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:56.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.700+0000 7f75537fe700 1 --2- 172.21.15.133:0/1974761322 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f753c0a1260 0x7f753c0a3710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f75540745e0 tx=0x7f7548005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:56.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.702+0000 7f75517fa700 1 -- 172.21.15.133:0/1974761322 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7544051730 con 0x7f7554100540 2023-11-27T15:21:56.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.895+0000 7f755a40d700 1 -- 172.21.15.133:0/1974761322 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f75541104a0 con 0x7f753c0a1260 2023-11-27T15:21:56.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.908+0000 7f75517fa700 1 -- 172.21.15.133:0/1974761322 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426639 (secure 0 0 0) 0x7f75541104a0 con 0x7f753c0a1260 2023-11-27T15:21:56.918 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:21:56.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.913+0000 7f755a40d700 1 -- 172.21.15.133:0/1974761322 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f753c0a1260 msgr2=0x7f753c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:56.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.913+0000 7f755a40d700 1 --2- 172.21.15.133:0/1974761322 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f753c0a1260 0x7f753c0a3710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f75540745e0 tx=0x7f7548005fd0 comp rx=0 tx=0).stop 2023-11-27T15:21:56.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.913+0000 7f755a40d700 1 -- 172.21.15.133:0/1974761322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7554100540 msgr2=0x7f7554072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:56.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.913+0000 7f755a40d700 1 --2- 172.21.15.133:0/1974761322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7554100540 0x7f7554072df0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f754400e9a0 tx=0x7f754400ecb0 comp rx=0 tx=0).stop 2023-11-27T15:21:56.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.913+0000 7f755a40d700 1 -- 172.21.15.133:0/1974761322 shutdown_connections 2023-11-27T15:21:56.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.913+0000 7f755a40d700 1 --2- 172.21.15.133:0/1974761322 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f753c0a1260 0x7f753c0a3710 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:56.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.913+0000 7f755a40d700 1 --2- 172.21.15.133:0/1974761322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75541055e0 0x7f7554073870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:56.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.913+0000 7f755a40d700 1 --2- 172.21.15.133:0/1974761322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7554100540 0x7f7554072df0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:56.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.913+0000 7f755a40d700 1 --2- 172.21.15.133:0/1974761322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7554100ee0 0x7f7554073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:56.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.913+0000 7f755a40d700 1 -- 172.21.15.133:0/1974761322 >> 172.21.15.133:0/1974761322 conn(0x7f75540fbdc0 msgr2=0x7f75540fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:56.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.913+0000 7f755a40d700 1 -- 172.21.15.133:0/1974761322 shutdown_connections 2023-11-27T15:21:56.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:56.913+0000 7f755a40d700 1 -- 172.21.15.133:0/1974761322 wait complete. 2023-11-27T15:21:56.925 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2023-11-27T15:21:57.060 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:56 smithi133 ceph-mon[97860]: pgmap v317: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:21:57.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:56 smithi177 ceph-mon[131504]: pgmap v317: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:21:57.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:56 smithi174 ceph-mon[131110]: pgmap v317: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:21:57.293 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2023-11-27T15:21:55.729697+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654326,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":442,"num_read_kb":332,"num_write":341,"num_write_kb":3052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":277,"ondisk_log_size":277,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":83032,"kb_used_data":13880,"kb_used_omap":0,"kb_used_meta":69056,"kb_avail":1124858792,"statfs":{"total":1151940427776,"available":1151855403008,"internally_reserved":0,"allocated":14213120,"data_stored":7874181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":70713344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003428"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.933701+0000","last_change":"2023-11-27T15:20:33.236352+0000","last_active":"2023-11-27T15:21:02.933701+0000","last_peered":"2023-11-27T15:21:02.933701+0000","last_clean":"2023-11-27T15:21:02.933701+0000","last_became_active":"2023-11-27T15:20:33.235974+0000","last_became_peered":"2023-11-27T15:20:33.235974+0000","last_unstale":"2023-11-27T15:21:02.933701+0000","last_undegraded":"2023-11-27T15:21:02.933701+0000","last_fullsized":"2023-11-27T15:21:02.933701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:42:55.142804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786021+0000","last_change":"2023-11-27T15:20:33.234882+0000","last_active":"2023-11-27T15:21:02.786021+0000","last_peered":"2023-11-27T15:21:02.786021+0000","last_clean":"2023-11-27T15:21:02.786021+0000","last_became_active":"2023-11-27T15:20:33.231062+0000","last_became_peered":"2023-11-27T15:20:33.231062+0000","last_unstale":"2023-11-27T15:21:02.786021+0000","last_undegraded":"2023-11-27T15:21:02.786021+0000","last_fullsized":"2023-11-27T15:21:02.786021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:18:27.469196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785364+0000","last_change":"2023-11-27T15:20:33.238280+0000","last_active":"2023-11-27T15:21:02.785364+0000","last_peered":"2023-11-27T15:21:02.785364+0000","last_clean":"2023-11-27T15:21:02.785364+0000","last_became_active":"2023-11-27T15:20:33.237602+0000","last_became_peered":"2023-11-27T15:20:33.237602+0000","last_unstale":"2023-11-27T15:21:02.785364+0000","last_undegraded":"2023-11-27T15:21:02.785364+0000","last_fullsized":"2023-11-27T15:21:02.785364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:28:48.211669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784641+0000","last_change":"2023-11-27T15:20:33.217909+0000","last_active":"2023-11-27T15:21:02.784641+0000","last_peered":"2023-11-27T15:21:02.784641+0000","last_clean":"2023-11-27T15:21:02.784641+0000","last_became_active":"2023-11-27T15:20:33.217511+0000","last_became_peered":"2023-11-27T15:20:33.217511+0000","last_unstale":"2023-11-27T15:21:02.784641+0000","last_undegraded":"2023-11-27T15:21:02.784641+0000","last_fullsized":"2023-11-27T15:21:02.784641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:37:24.759082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782641+0000","last_change":"2023-11-27T15:20:33.246630+0000","last_active":"2023-11-27T15:21:02.782641+0000","last_peered":"2023-11-27T15:21:02.782641+0000","last_clean":"2023-11-27T15:21:02.782641+0000","last_became_active":"2023-11-27T15:20:33.246249+0000","last_became_peered":"2023-11-27T15:20:33.246249+0000","last_unstale":"2023-11-27T15:21:02.782641+0000","last_undegraded":"2023-11-27T15:21:02.782641+0000","last_fullsized":"2023-11-27T15:21:02.782641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:14:48.110743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932766+0000","last_change":"2023-11-27T15:20:33.236280+0000","last_active":"2023-11-27T15:21:02.932766+0000","last_peered":"2023-11-27T15:21:02.932766+0000","last_clean":"2023-11-27T15:21:02.932766+0000","last_became_active":"2023-11-27T15:20:33.235983+0000","last_became_peered":"2023-11-27T15:20:33.235983+0000","last_unstale":"2023-11-27T15:21:02.932766+0000","last_undegraded":"2023-11-27T15:21:02.932766+0000","last_fullsized":"2023-11-27T15:21:02.932766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:32:27.538426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779828+0000","last_change":"2023-11-27T15:20:33.223085+0000","last_active":"2023-11-27T15:21:02.779828+0000","last_peered":"2023-11-27T15:21:02.779828+0000","last_clean":"2023-11-27T15:21:02.779828+0000","last_became_active":"2023-11-27T15:20:33.222809+0000","last_became_peered":"2023-11-27T15:20:33.222809+0000","last_unstale":"2023-11-27T15:21:02.779828+0000","last_undegraded":"2023-11-27T15:21:02.779828+0000","last_fullsized":"2023-11-27T15:21:02.779828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:46:23.567184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783786+0000","last_change":"2023-11-27T15:20:33.226217+0000","last_active":"2023-11-27T15:21:02.783786+0000","last_peered":"2023-11-27T15:21:02.783786+0000","last_clean":"2023-11-27T15:21:02.783786+0000","last_became_active":"2023-11-27T15:20:33.225856+0000","last_became_peered":"2023-11-27T15:20:33.225856+0000","last_unstale":"2023-11-27T15:21:02.783786+0000","last_undegraded":"2023-11-27T15:21:02.783786+0000","last_fullsized":"2023-11-27T15:21:02.783786+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:41:31.773939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.795641+0000","last_change":"2023-11-27T15:20:33.227554+0000","last_active":"2023-11-27T15:21:02.795641+0000","last_peered":"2023-11-27T15:21:02.795641+0000","last_clean":"2023-11-27T15:21:02.795641+0000","last_became_active":"2023-11-27T15:20:33.227222+0000","last_became_peered":"2023-11-27T15:20:33.227222+0000","last_unstale":"2023-11-27T15:21:02.795641+0000","last_undegraded":"2023-11-27T15:21:02.795641+0000","last_fullsized":"2023-11-27T15:21:02.795641+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:34:15.927242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.933040+0000","last_change":"2023-11-27T15:20:33.230528+0000","last_active":"2023-11-27T15:21:02.933040+0000","last_peered":"2023-11-27T15:21:02.933040+0000","last_clean":"2023-11-27T15:21:02.933040+0000","last_became_active":"2023-11-27T15:20:33.230148+0000","last_became_peered":"2023-11-27T15:20:33.230148+0000","last_unstale":"2023-11-27T15:21:02.933040+0000","last_undegraded":"2023-11-27T15:21:02.933040+0000","last_fullsized":"2023-11-27T15:21:02.933040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:56:46.117728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780265+0000","last_change":"2023-11-27T15:20:33.228098+0000","last_active":"2023-11-27T15:21:02.780265+0000","last_peered":"2023-11-27T15:21:02.780265+0000","last_clean":"2023-11-27T15:21:02.780265+0000","last_became_active":"2023-11-27T15:20:33.227760+0000","last_became_peered":"2023-11-27T15:20:33.227760+0000","last_unstale":"2023-11-27T15:21:02.780265+0000","last_undegraded":"2023-11-27T15:21:02.780265+0000","last_fullsized":"2023-11-27T15:21:02.780265+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:27:54.894445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.935458+0000","last_change":"2023-11-27T15:21:02.934583+0000","last_active":"2023-11-27T15:21:02.935458+0000","last_peered":"2023-11-27T15:21:02.935458+0000","last_clean":"2023-11-27T15:21:02.935458+0000","last_became_active":"2023-11-27T15:21:02.933631+0000","last_became_peered":"2023-11-27T15:21:02.933631+0000","last_unstale":"2023-11-27T15:21:02.935458+0000","last_undegraded":"2023-11-27T15:21:02.935458+0000","last_fullsized":"2023-11-27T15:21:02.935458+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:45:15.679810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783578+0000","last_change":"2023-11-27T15:20:33.231255+0000","last_active":"2023-11-27T15:21:02.783578+0000","last_peered":"2023-11-27T15:21:02.783578+0000","last_clean":"2023-11-27T15:21:02.783578+0000","last_became_active":"2023-11-27T15:20:33.230867+0000","last_became_peered":"2023-11-27T15:20:33.230867+0000","last_unstale":"2023-11-27T15:21:02.783578+0000","last_undegraded":"2023-11-27T15:21:02.783578+0000","last_fullsized":"2023-11-27T15:21:02.783578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:58:03.899781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785072+0000","last_change":"2023-11-27T15:20:33.230306+0000","last_active":"2023-11-27T15:21:02.785072+0000","last_peered":"2023-11-27T15:21:02.785072+0000","last_clean":"2023-11-27T15:21:02.785072+0000","last_became_active":"2023-11-27T15:20:33.229892+0000","last_became_peered":"2023-11-27T15:20:33.229892+0000","last_unstale":"2023-11-27T15:21:02.785072+0000","last_undegraded":"2023-11-27T15:21:02.785072+0000","last_fullsized":"2023-11-27T15:21:02.785072+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:24:55.365713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785143+0000","last_change":"2023-11-27T15:20:33.220592+0000","last_active":"2023-11-27T15:21:02.785143+0000","last_peered":"2023-11-27T15:21:02.785143+0000","last_clean":"2023-11-27T15:21:02.785143+0000","last_became_active":"2023-11-27T15:20:33.220258+0000","last_became_peered":"2023-11-27T15:20:33.220258+0000","last_unstale":"2023-11-27T15:21:02.785143+0000","last_undegraded":"2023-11-27T15:21:02.785143+0000","last_fullsized":"2023-11-27T15:21:02.785143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:48:48.921465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.794878+0000","last_change":"2023-11-27T15:20:33.245041+0000","last_active":"2023-11-27T15:21:02.794878+0000","last_peered":"2023-11-27T15:21:02.794878+0000","last_clean":"2023-11-27T15:21:02.794878+0000","last_became_active":"2023-11-27T15:20:33.244759+0000","last_became_peered":"2023-11-27T15:20:33.244759+0000","last_unstale":"2023-11-27T15:21:02.794878+0000","last_undegraded":"2023-11-27T15:21:02.794878+0000","last_fullsized":"2023-11-27T15:21:02.794878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:31:00.164868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.934087+0000","last_change":"2023-11-27T15:20:33.243288+0000","last_active":"2023-11-27T15:21:02.934087+0000","last_peered":"2023-11-27T15:21:02.934087+0000","last_clean":"2023-11-27T15:21:02.934087+0000","last_became_active":"2023-11-27T15:20:33.242989+0000","last_became_peered":"2023-11-27T15:20:33.242989+0000","last_unstale":"2023-11-27T15:21:02.934087+0000","last_undegraded":"2023-11-27T15:21:02.934087+0000","last_fullsized":"2023-11-27T15:21:02.934087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:25:53.302075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779978+0000","last_change":"2023-11-27T15:20:33.229613+0000","last_active":"2023-11-27T15:21:02.779978+0000","last_peered":"2023-11-27T15:21:02.779978+0000","last_clean":"2023-11-27T15:21:02.779978+0000","last_became_active":"2023-11-27T15:20:33.227747+0000","last_became_peered":"2023-11-27T15:20:33.227747+0000","last_unstale":"2023-11-27T15:21:02.779978+0000","last_undegraded":"2023-11-27T15:21:02.779978+0000","last_fullsized":"2023-11-27T15:21:02.779978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:18:24.422450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780242+0000","last_change":"2023-11-27T15:20:33.227004+0000","last_active":"2023-11-27T15:21:02.780242+0000","last_peered":"2023-11-27T15:21:02.780242+0000","last_clean":"2023-11-27T15:21:02.780242+0000","last_became_active":"2023-11-27T15:20:33.226703+0000","last_became_peered":"2023-11-27T15:20:33.226703+0000","last_unstale":"2023-11-27T15:21:02.780242+0000","last_undegraded":"2023-11-27T15:21:02.780242+0000","last_fullsized":"2023-11-27T15:21:02.780242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:18:04.531443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779865+0000","last_change":"2023-11-27T15:20:37.682021+0000","last_active":"2023-11-27T15:21:02.779865+0000","last_peered":"2023-11-27T15:21:02.779865+0000","last_clean":"2023-11-27T15:21:02.779865+0000","last_became_active":"2023-11-27T15:20:37.681600+0000","last_became_peered":"2023-11-27T15:20:37.681600+0000","last_unstale":"2023-11-27T15:21:02.779865+0000","last_undegraded":"2023-11-27T15:21:02.779865+0000","last_fullsized":"2023-11-27T15:21:02.779865+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:27:41.540353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.801451+0000","last_change":"2023-11-27T15:20:32.657542+0000","last_active":"2023-11-27T15:21:03.801451+0000","last_peered":"2023-11-27T15:21:03.801451+0000","last_clean":"2023-11-27T15:21:03.801451+0000","last_became_active":"2023-11-27T15:20:32.657158+0000","last_became_peered":"2023-11-27T15:20:32.657158+0000","last_unstale":"2023-11-27T15:21:03.801451+0000","last_undegraded":"2023-11-27T15:21:03.801451+0000","last_fullsized":"2023-11-27T15:21:03.801451+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:01:39.181861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.935289+0000","last_change":"2023-11-27T15:21:02.933684+0000","last_active":"2023-11-27T15:21:02.935289+0000","last_peered":"2023-11-27T15:21:02.935289+0000","last_clean":"2023-11-27T15:21:02.935289+0000","last_became_active":"2023-11-27T15:21:02.933215+0000","last_became_peered":"2023-11-27T15:21:02.933215+0000","last_unstale":"2023-11-27T15:21:02.935289+0000","last_undegraded":"2023-11-27T15:21:02.935289+0000","last_fullsized":"2023-11-27T15:21:02.935289+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:59:39.553465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783397+0000","last_change":"2023-11-27T15:20:37.666705+0000","last_active":"2023-11-27T15:21:02.783397+0000","last_peered":"2023-11-27T15:21:02.783397+0000","last_clean":"2023-11-27T15:21:02.783397+0000","last_became_active":"2023-11-27T15:20:37.666236+0000","last_became_peered":"2023-11-27T15:20:37.666236+0000","last_unstale":"2023-11-27T15:21:02.783397+0000","last_undegraded":"2023-11-27T15:21:02.783397+0000","last_fullsized":"2023-11-27T15:21:02.783397+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:49:31.365357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785700+0000","last_change":"2023-11-27T15:20:32.663915+0000","last_active":"2023-11-27T15:21:02.785700+0000","last_peered":"2023-11-27T15:21:02.785700+0000","last_clean":"2023-11-27T15:21:02.785700+0000","last_became_active":"2023-11-27T15:20:32.663069+0000","last_became_peered":"2023-11-27T15:20:32.663069+0000","last_unstale":"2023-11-27T15:21:02.785700+0000","last_undegraded":"2023-11-27T15:21:02.785700+0000","last_fullsized":"2023-11-27T15:21:02.785700+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:15:25.522191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785737+0000","last_change":"2023-11-27T15:20:33.238150+0000","last_active":"2023-11-27T15:21:02.785737+0000","last_peered":"2023-11-27T15:21:02.785737+0000","last_clean":"2023-11-27T15:21:02.785737+0000","last_became_active":"2023-11-27T15:20:33.236972+0000","last_became_peered":"2023-11-27T15:20:33.236972+0000","last_unstale":"2023-11-27T15:21:02.785737+0000","last_undegraded":"2023-11-27T15:21:02.785737+0000","last_fullsized":"2023-11-27T15:21:02.785737+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:32:03.275170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783667+0000","last_change":"2023-11-27T15:20:37.668031+0000","last_active":"2023-11-27T15:21:02.783667+0000","last_peered":"2023-11-27T15:21:02.783667+0000","last_clean":"2023-11-27T15:21:02.783667+0000","last_became_active":"2023-11-27T15:20:37.667640+0000","last_became_peered":"2023-11-27T15:20:37.667640+0000","last_unstale":"2023-11-27T15:21:02.783667+0000","last_undegraded":"2023-11-27T15:21:02.783667+0000","last_fullsized":"2023-11-27T15:21:02.783667+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:45:18.290503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.800461+0000","last_change":"2023-11-27T15:20:32.655738+0000","last_active":"2023-11-27T15:21:03.800461+0000","last_peered":"2023-11-27T15:21:03.800461+0000","last_clean":"2023-11-27T15:21:03.800461+0000","last_became_active":"2023-11-27T15:20:32.655439+0000","last_became_peered":"2023-11-27T15:20:32.655439+0000","last_unstale":"2023-11-27T15:21:03.800461+0000","last_undegraded":"2023-11-27T15:21:03.800461+0000","last_fullsized":"2023-11-27T15:21:03.800461+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:24:02.616850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.795136+0000","last_change":"2023-11-27T15:20:33.229267+0000","last_active":"2023-11-27T15:21:02.795136+0000","last_peered":"2023-11-27T15:21:02.795136+0000","last_clean":"2023-11-27T15:21:02.795136+0000","last_became_active":"2023-11-27T15:20:33.228746+0000","last_became_peered":"2023-11-27T15:20:33.228746+0000","last_unstale":"2023-11-27T15:21:02.795136+0000","last_undegraded":"2023-11-27T15:21:02.795136+0000","last_fullsized":"2023-11-27T15:21:02.795136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:55:36.391108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783571+0000","last_change":"2023-11-27T15:20:37.671893+0000","last_active":"2023-11-27T15:21:02.783571+0000","last_peered":"2023-11-27T15:21:02.783571+0000","last_clean":"2023-11-27T15:21:02.783571+0000","last_became_active":"2023-11-27T15:20:37.671033+0000","last_became_peered":"2023-11-27T15:20:37.671033+0000","last_unstale":"2023-11-27T15:21:02.783571+0000","last_undegraded":"2023-11-27T15:21:02.783571+0000","last_fullsized":"2023-11-27T15:21:02.783571+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:43:21.793592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782934+0000","last_change":"2023-11-27T15:20:32.650342+0000","last_active":"2023-11-27T15:21:02.782934+0000","last_peered":"2023-11-27T15:21:02.782934+0000","last_clean":"2023-11-27T15:21:02.782934+0000","last_became_active":"2023-11-27T15:20:32.650034+0000","last_became_peered":"2023-11-27T15:20:32.650034+0000","last_unstale":"2023-11-27T15:21:02.782934+0000","last_undegraded":"2023-11-27T15:21:02.782934+0000","last_fullsized":"2023-11-27T15:21:02.782934+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:29:04.356130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932717+0000","last_change":"2023-11-27T15:20:33.226005+0000","last_active":"2023-11-27T15:21:02.932717+0000","last_peered":"2023-11-27T15:21:02.932717+0000","last_clean":"2023-11-27T15:21:02.932717+0000","last_became_active":"2023-11-27T15:20:33.225685+0000","last_became_peered":"2023-11-27T15:20:33.225685+0000","last_unstale":"2023-11-27T15:21:02.932717+0000","last_undegraded":"2023-11-27T15:21:02.932717+0000","last_fullsized":"2023-11-27T15:21:02.932717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:20:11.054422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779958+0000","last_change":"2023-11-27T15:20:37.673168+0000","last_active":"2023-11-27T15:21:02.779958+0000","last_peered":"2023-11-27T15:21:02.779958+0000","last_clean":"2023-11-27T15:21:02.779958+0000","last_became_active":"2023-11-27T15:20:37.672762+0000","last_became_peered":"2023-11-27T15:20:37.672762+0000","last_unstale":"2023-11-27T15:21:02.779958+0000","last_undegraded":"2023-11-27T15:21:02.779958+0000","last_fullsized":"2023-11-27T15:21:02.779958+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:17:17.977179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'5","reported_seq":84,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:53.884938+0000","last_change":"2023-11-27T15:20:32.656402+0000","last_active":"2023-11-27T15:21:53.884938+0000","last_peered":"2023-11-27T15:21:53.884938+0000","last_clean":"2023-11-27T15:21:53.884938+0000","last_became_active":"2023-11-27T15:20:32.656112+0000","last_became_peered":"2023-11-27T15:20:32.656112+0000","last_unstale":"2023-11-27T15:21:53.884938+0000","last_undegraded":"2023-11-27T15:21:53.884938+0000","last_fullsized":"2023-11-27T15:21:53.884938+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:10:24.829148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782376+0000","last_change":"2023-11-27T15:20:33.223073+0000","last_active":"2023-11-27T15:21:02.782376+0000","last_peered":"2023-11-27T15:21:02.782376+0000","last_clean":"2023-11-27T15:21:02.782376+0000","last_became_active":"2023-11-27T15:20:33.222655+0000","last_became_peered":"2023-11-27T15:20:33.222655+0000","last_unstale":"2023-11-27T15:21:02.782376+0000","last_undegraded":"2023-11-27T15:21:02.782376+0000","last_fullsized":"2023-11-27T15:21:02.782376+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:48:43.125499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779818+0000","last_change":"2023-11-27T15:20:37.678722+0000","last_active":"2023-11-27T15:21:02.779818+0000","last_peered":"2023-11-27T15:21:02.779818+0000","last_clean":"2023-11-27T15:21:02.779818+0000","last_became_active":"2023-11-27T15:20:37.678242+0000","last_became_peered":"2023-11-27T15:20:37.678242+0000","last_unstale":"2023-11-27T15:21:02.779818+0000","last_undegraded":"2023-11-27T15:21:02.779818+0000","last_fullsized":"2023-11-27T15:21:02.779818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:36:10.661192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.800170+0000","last_change":"2023-11-27T15:20:32.646951+0000","last_active":"2023-11-27T15:21:03.800170+0000","last_peered":"2023-11-27T15:21:03.800170+0000","last_clean":"2023-11-27T15:21:03.800170+0000","last_became_active":"2023-11-27T15:20:32.646620+0000","last_became_peered":"2023-11-27T15:20:32.646620+0000","last_unstale":"2023-11-27T15:21:03.800170+0000","last_undegraded":"2023-11-27T15:21:03.800170+0000","last_fullsized":"2023-11-27T15:21:03.800170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:13:29.369691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932503+0000","last_change":"2023-11-27T15:20:33.226576+0000","last_active":"2023-11-27T15:21:02.932503+0000","last_peered":"2023-11-27T15:21:02.932503+0000","last_clean":"2023-11-27T15:21:02.932503+0000","last_became_active":"2023-11-27T15:20:33.226181+0000","last_became_peered":"2023-11-27T15:20:33.226181+0000","last_unstale":"2023-11-27T15:21:02.932503+0000","last_undegraded":"2023-11-27T15:21:02.932503+0000","last_fullsized":"2023-11-27T15:21:02.932503+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:27:44.284870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932740+0000","last_change":"2023-11-27T15:20:37.681668+0000","last_active":"2023-11-27T15:21:02.932740+0000","last_peered":"2023-11-27T15:21:02.932740+0000","last_clean":"2023-11-27T15:21:02.932740+0000","last_became_active":"2023-11-27T15:20:37.681236+0000","last_became_peered":"2023-11-27T15:20:37.681236+0000","last_unstale":"2023-11-27T15:21:02.932740+0000","last_undegraded":"2023-11-27T15:21:02.932740+0000","last_fullsized":"2023-11-27T15:21:02.932740+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:29:12.216944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.800787+0000","last_change":"2023-11-27T15:20:32.659421+0000","last_active":"2023-11-27T15:21:03.800787+0000","last_peered":"2023-11-27T15:21:03.800787+0000","last_clean":"2023-11-27T15:21:03.800787+0000","last_became_active":"2023-11-27T15:20:32.658882+0000","last_became_peered":"2023-11-27T15:20:32.658882+0000","last_unstale":"2023-11-27T15:21:03.800787+0000","last_undegraded":"2023-11-27T15:21:03.800787+0000","last_fullsized":"2023-11-27T15:21:03.800787+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:08:52.781635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783921+0000","last_change":"2023-11-27T15:20:33.227635+0000","last_active":"2023-11-27T15:21:02.783921+0000","last_peered":"2023-11-27T15:21:02.783921+0000","last_clean":"2023-11-27T15:21:02.783921+0000","last_became_active":"2023-11-27T15:20:33.225217+0000","last_became_peered":"2023-11-27T15:20:33.225217+0000","last_unstale":"2023-11-27T15:21:02.783921+0000","last_undegraded":"2023-11-27T15:21:02.783921+0000","last_fullsized":"2023-11-27T15:21:02.783921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:53:42.037169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785814+0000","last_change":"2023-11-27T15:20:37.679575+0000","last_active":"2023-11-27T15:21:02.785814+0000","last_peered":"2023-11-27T15:21:02.785814+0000","last_clean":"2023-11-27T15:21:02.785814+0000","last_became_active":"2023-11-27T15:20:37.679131+0000","last_became_peered":"2023-11-27T15:20:37.679131+0000","last_unstale":"2023-11-27T15:21:02.785814+0000","last_undegraded":"2023-11-27T15:21:02.785814+0000","last_fullsized":"2023-11-27T15:21:02.785814+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:43:21.129782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.931632+0000","last_change":"2023-11-27T15:20:32.657155+0000","last_active":"2023-11-27T15:21:02.931632+0000","last_peered":"2023-11-27T15:21:02.931632+0000","last_clean":"2023-11-27T15:21:02.931632+0000","last_became_active":"2023-11-27T15:20:32.656832+0000","last_became_peered":"2023-11-27T15:20:32.656832+0000","last_unstale":"2023-11-27T15:21:02.931632+0000","last_undegraded":"2023-11-27T15:21:02.931632+0000","last_fullsized":"2023-11-27T15:21:02.931632+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:03:07.478635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779387+0000","last_change":"2023-11-27T15:20:33.229215+0000","last_active":"2023-11-27T15:21:02.779387+0000","last_peered":"2023-11-27T15:21:02.779387+0000","last_clean":"2023-11-27T15:21:02.779387+0000","last_became_active":"2023-11-27T15:20:33.228616+0000","last_became_peered":"2023-11-27T15:20:33.228616+0000","last_unstale":"2023-11-27T15:21:02.779387+0000","last_undegraded":"2023-11-27T15:21:02.779387+0000","last_fullsized":"2023-11-27T15:21:02.779387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:30:10.600160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784114+0000","last_change":"2023-11-27T15:20:37.672582+0000","last_active":"2023-11-27T15:21:02.784114+0000","last_peered":"2023-11-27T15:21:02.784114+0000","last_clean":"2023-11-27T15:21:02.784114+0000","last_became_active":"2023-11-27T15:20:37.672193+0000","last_became_peered":"2023-11-27T15:20:37.672193+0000","last_unstale":"2023-11-27T15:21:02.784114+0000","last_undegraded":"2023-11-27T15:21:02.784114+0000","last_fullsized":"2023-11-27T15:21:02.784114+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:53:51.137569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":76,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:49.877415+0000","last_change":"2023-11-27T15:20:32.647894+0000","last_active":"2023-11-27T15:21:49.877415+0000","last_peered":"2023-11-27T15:21:49.877415+0000","last_clean":"2023-11-27T15:21:49.877415+0000","last_became_active":"2023-11-27T15:20:32.647597+0000","last_became_peered":"2023-11-27T15:20:32.647597+0000","last_unstale":"2023-11-27T15:21:49.877415+0000","last_undegraded":"2023-11-27T15:21:49.877415+0000","last_fullsized":"2023-11-27T15:21:49.877415+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:00:17.488397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":47,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.795490+0000","last_change":"2023-11-27T15:20:33.229382+0000","last_active":"2023-11-27T15:21:02.795490+0000","last_peered":"2023-11-27T15:21:02.795490+0000","last_clean":"2023-11-27T15:21:02.795490+0000","last_became_active":"2023-11-27T15:20:33.229031+0000","last_became_peered":"2023-11-27T15:20:33.229031+0000","last_unstale":"2023-11-27T15:21:02.795490+0000","last_undegraded":"2023-11-27T15:21:02.795490+0000","last_fullsized":"2023-11-27T15:21:02.795490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:43:48.637904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779387+0000","last_change":"2023-11-27T15:20:37.679131+0000","last_active":"2023-11-27T15:21:02.779387+0000","last_peered":"2023-11-27T15:21:02.779387+0000","last_clean":"2023-11-27T15:21:02.779387+0000","last_became_active":"2023-11-27T15:20:37.678290+0000","last_became_peered":"2023-11-27T15:20:37.678290+0000","last_unstale":"2023-11-27T15:21:02.779387+0000","last_undegraded":"2023-11-27T15:21:02.779387+0000","last_fullsized":"2023-11-27T15:21:02.779387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:55:28.895714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.798466+0000","last_change":"2023-11-27T15:21:02.797374+0000","last_active":"2023-11-27T15:21:02.798466+0000","last_peered":"2023-11-27T15:21:02.798466+0000","last_clean":"2023-11-27T15:21:02.798466+0000","last_became_active":"2023-11-27T15:21:02.796076+0000","last_became_peered":"2023-11-27T15:21:02.796076+0000","last_unstale":"2023-11-27T15:21:02.798466+0000","last_undegraded":"2023-11-27T15:21:02.798466+0000","last_fullsized":"2023-11-27T15:21:02.798466+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:07:58.898194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.935104+0000","last_change":"2023-11-27T15:21:02.935104+0000","last_active":"2023-11-27T15:21:02.935104+0000","last_peered":"2023-11-27T15:21:02.935104+0000","last_clean":"2023-11-27T15:21:02.935104+0000","last_became_active":"2023-11-27T15:21:02.934486+0000","last_became_peered":"2023-11-27T15:21:02.934486+0000","last_unstale":"2023-11-27T15:21:02.935104+0000","last_undegraded":"2023-11-27T15:21:02.935104+0000","last_fullsized":"2023-11-27T15:21:02.935104+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:06:11.768697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783910+0000","last_change":"2023-11-27T15:20:37.670389+0000","last_active":"2023-11-27T15:21:02.783910+0000","last_peered":"2023-11-27T15:21:02.783910+0000","last_clean":"2023-11-27T15:21:02.783910+0000","last_became_active":"2023-11-27T15:20:37.669512+0000","last_became_peered":"2023-11-27T15:20:37.669512+0000","last_unstale":"2023-11-27T15:21:02.783910+0000","last_undegraded":"2023-11-27T15:21:02.783910+0000","last_fullsized":"2023-11-27T15:21:02.783910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:47:38.447258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932618+0000","last_change":"2023-11-27T15:20:32.658287+0000","last_active":"2023-11-27T15:21:02.932618+0000","last_peered":"2023-11-27T15:21:02.932618+0000","last_clean":"2023-11-27T15:21:02.932618+0000","last_became_active":"2023-11-27T15:20:32.657925+0000","last_became_peered":"2023-11-27T15:20:32.657925+0000","last_unstale":"2023-11-27T15:21:02.932618+0000","last_undegraded":"2023-11-27T15:21:02.932618+0000","last_fullsized":"2023-11-27T15:21:02.932618+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:49:45.401491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.795421+0000","last_change":"2023-11-27T15:20:33.220240+0000","last_active":"2023-11-27T15:21:02.795421+0000","last_peered":"2023-11-27T15:21:02.795421+0000","last_clean":"2023-11-27T15:21:02.795421+0000","last_became_active":"2023-11-27T15:20:33.219962+0000","last_became_peered":"2023-11-27T15:20:33.219962+0000","last_unstale":"2023-11-27T15:21:02.795421+0000","last_undegraded":"2023-11-27T15:21:02.795421+0000","last_fullsized":"2023-11-27T15:21:02.795421+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:22:09.271036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779503+0000","last_change":"2023-11-27T15:20:37.672203+0000","last_active":"2023-11-27T15:21:02.779503+0000","last_peered":"2023-11-27T15:21:02.779503+0000","last_clean":"2023-11-27T15:21:02.779503+0000","last_became_active":"2023-11-27T15:20:37.671149+0000","last_became_peered":"2023-11-27T15:20:37.671149+0000","last_unstale":"2023-11-27T15:21:02.779503+0000","last_undegraded":"2023-11-27T15:21:02.779503+0000","last_fullsized":"2023-11-27T15:21:02.779503+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:14:24.426741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"90'33","reported_seq":161,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:50.878872+0000","last_change":"2023-11-27T15:20:32.646637+0000","last_active":"2023-11-27T15:21:50.878872+0000","last_peered":"2023-11-27T15:21:50.878872+0000","last_clean":"2023-11-27T15:21:50.878872+0000","last_became_active":"2023-11-27T15:20:32.646292+0000","last_became_peered":"2023-11-27T15:20:32.646292+0000","last_unstale":"2023-11-27T15:21:50.878872+0000","last_undegraded":"2023-11-27T15:21:50.878872+0000","last_fullsized":"2023-11-27T15:21:50.878872+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:01:39.494636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59808,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":102,"num_read_kb":110,"num_write":34,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782815+0000","last_change":"2023-11-27T15:20:33.217078+0000","last_active":"2023-11-27T15:21:02.782815+0000","last_peered":"2023-11-27T15:21:02.782815+0000","last_clean":"2023-11-27T15:21:02.782815+0000","last_became_active":"2023-11-27T15:20:33.216787+0000","last_became_peered":"2023-11-27T15:20:33.216787+0000","last_unstale":"2023-11-27T15:21:02.782815+0000","last_undegraded":"2023-11-27T15:21:02.782815+0000","last_fullsized":"2023-11-27T15:21:02.782815+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:33:46.415082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.933136+0000","last_change":"2023-11-27T15:20:37.671800+0000","last_active":"2023-11-27T15:21:02.933136+0000","last_peered":"2023-11-27T15:21:02.933136+0000","last_clean":"2023-11-27T15:21:02.933136+0000","last_became_active":"2023-11-27T15:20:37.671374+0000","last_became_peered":"2023-11-27T15:20:37.671374+0000","last_unstale":"2023-11-27T15:21:02.933136+0000","last_undegraded":"2023-11-27T15:21:02.933136+0000","last_fullsized":"2023-11-27T15:21:02.933136+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:04:07.644987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.802237+0000","last_change":"2023-11-27T15:20:32.659508+0000","last_active":"2023-11-27T15:21:03.802237+0000","last_peered":"2023-11-27T15:21:03.802237+0000","last_clean":"2023-11-27T15:21:03.802237+0000","last_became_active":"2023-11-27T15:20:32.659226+0000","last_became_peered":"2023-11-27T15:20:32.659226+0000","last_unstale":"2023-11-27T15:21:03.802237+0000","last_undegraded":"2023-11-27T15:21:03.802237+0000","last_fullsized":"2023-11-27T15:21:03.802237+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:42:20.386331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786639+0000","last_change":"2023-11-27T15:20:33.230012+0000","last_active":"2023-11-27T15:21:02.786639+0000","last_peered":"2023-11-27T15:21:02.786639+0000","last_clean":"2023-11-27T15:21:02.786639+0000","last_became_active":"2023-11-27T15:20:33.225193+0000","last_became_peered":"2023-11-27T15:20:33.225193+0000","last_unstale":"2023-11-27T15:21:02.786639+0000","last_undegraded":"2023-11-27T15:21:02.786639+0000","last_fullsized":"2023-11-27T15:21:02.786639+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:31:18.056712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932755+0000","last_change":"2023-11-27T15:20:37.672430+0000","last_active":"2023-11-27T15:21:02.932755+0000","last_peered":"2023-11-27T15:21:02.932755+0000","last_clean":"2023-11-27T15:21:02.932755+0000","last_became_active":"2023-11-27T15:20:37.671982+0000","last_became_peered":"2023-11-27T15:20:37.671982+0000","last_unstale":"2023-11-27T15:21:02.932755+0000","last_undegraded":"2023-11-27T15:21:02.932755+0000","last_fullsized":"2023-11-27T15:21:02.932755+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:53:27.660441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932051+0000","last_change":"2023-11-27T15:20:32.655015+0000","last_active":"2023-11-27T15:21:02.932051+0000","last_peered":"2023-11-27T15:21:02.932051+0000","last_clean":"2023-11-27T15:21:02.932051+0000","last_became_active":"2023-11-27T15:20:32.654662+0000","last_became_peered":"2023-11-27T15:20:32.654662+0000","last_unstale":"2023-11-27T15:21:02.932051+0000","last_undegraded":"2023-11-27T15:21:02.932051+0000","last_fullsized":"2023-11-27T15:21:02.932051+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:10:34.935987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780064+0000","last_change":"2023-11-27T15:20:33.233980+0000","last_active":"2023-11-27T15:21:02.780064+0000","last_peered":"2023-11-27T15:21:02.780064+0000","last_clean":"2023-11-27T15:21:02.780064+0000","last_became_active":"2023-11-27T15:20:33.233581+0000","last_became_peered":"2023-11-27T15:20:33.233581+0000","last_unstale":"2023-11-27T15:21:02.780064+0000","last_undegraded":"2023-11-27T15:21:02.780064+0000","last_fullsized":"2023-11-27T15:21:02.780064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:21:09.061874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779860+0000","last_change":"2023-11-27T15:20:37.682359+0000","last_active":"2023-11-27T15:21:02.779860+0000","last_peered":"2023-11-27T15:21:02.779860+0000","last_clean":"2023-11-27T15:21:02.779860+0000","last_became_active":"2023-11-27T15:20:37.681888+0000","last_became_peered":"2023-11-27T15:20:37.681888+0000","last_unstale":"2023-11-27T15:21:02.779860+0000","last_undegraded":"2023-11-27T15:21:02.779860+0000","last_fullsized":"2023-11-27T15:21:02.779860+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:22:58.040147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":83,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:52.882672+0000","last_change":"2023-11-27T15:20:32.653388+0000","last_active":"2023-11-27T15:21:52.882672+0000","last_peered":"2023-11-27T15:21:52.882672+0000","last_clean":"2023-11-27T15:21:52.882672+0000","last_became_active":"2023-11-27T15:20:32.653007+0000","last_became_peered":"2023-11-27T15:20:32.653007+0000","last_unstale":"2023-11-27T15:21:52.882672+0000","last_undegraded":"2023-11-27T15:21:52.882672+0000","last_fullsized":"2023-11-27T15:21:52.882672+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:02:11.654541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779871+0000","last_change":"2023-11-27T15:20:33.216539+0000","last_active":"2023-11-27T15:21:02.779871+0000","last_peered":"2023-11-27T15:21:02.779871+0000","last_clean":"2023-11-27T15:21:02.779871+0000","last_became_active":"2023-11-27T15:20:33.216166+0000","last_became_peered":"2023-11-27T15:20:33.216166+0000","last_unstale":"2023-11-27T15:21:02.779871+0000","last_undegraded":"2023-11-27T15:21:02.779871+0000","last_fullsized":"2023-11-27T15:21:02.779871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:12:37.881354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786643+0000","last_change":"2023-11-27T15:20:37.675735+0000","last_active":"2023-11-27T15:21:02.786643+0000","last_peered":"2023-11-27T15:21:02.786643+0000","last_clean":"2023-11-27T15:21:02.786643+0000","last_became_active":"2023-11-27T15:20:37.669685+0000","last_became_peered":"2023-11-27T15:20:37.669685+0000","last_unstale":"2023-11-27T15:21:02.786643+0000","last_undegraded":"2023-11-27T15:21:02.786643+0000","last_fullsized":"2023-11-27T15:21:02.786643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:47:13.695674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.803274+0000","last_change":"2023-11-27T15:20:32.657567+0000","last_active":"2023-11-27T15:21:03.803274+0000","last_peered":"2023-11-27T15:21:03.803274+0000","last_clean":"2023-11-27T15:21:03.803274+0000","last_became_active":"2023-11-27T15:20:32.657251+0000","last_became_peered":"2023-11-27T15:20:32.657251+0000","last_unstale":"2023-11-27T15:21:03.803274+0000","last_undegraded":"2023-11-27T15:21:03.803274+0000","last_fullsized":"2023-11-27T15:21:03.803274+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:23:00.166520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932029+0000","last_change":"2023-11-27T15:20:33.225019+0000","last_active":"2023-11-27T15:21:02.932029+0000","last_peered":"2023-11-27T15:21:02.932029+0000","last_clean":"2023-11-27T15:21:02.932029+0000","last_became_active":"2023-11-27T15:20:33.224622+0000","last_became_peered":"2023-11-27T15:20:33.224622+0000","last_unstale":"2023-11-27T15:21:02.932029+0000","last_undegraded":"2023-11-27T15:21:02.932029+0000","last_fullsized":"2023-11-27T15:21:02.932029+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:44:13.508838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.931675+0000","last_change":"2023-11-27T15:20:37.680808+0000","last_active":"2023-11-27T15:21:02.931675+0000","last_peered":"2023-11-27T15:21:02.931675+0000","last_clean":"2023-11-27T15:21:02.931675+0000","last_became_active":"2023-11-27T15:20:37.680348+0000","last_became_peered":"2023-11-27T15:20:37.680348+0000","last_unstale":"2023-11-27T15:21:02.931675+0000","last_undegraded":"2023-11-27T15:21:02.931675+0000","last_fullsized":"2023-11-27T15:21:02.931675+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:48:00.800370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:04.804666+0000","last_change":"2023-11-27T15:20:32.651093+0000","last_active":"2023-11-27T15:21:04.804666+0000","last_peered":"2023-11-27T15:21:04.804666+0000","last_clean":"2023-11-27T15:21:04.804666+0000","last_became_active":"2023-11-27T15:20:32.650657+0000","last_became_peered":"2023-11-27T15:20:32.650657+0000","last_unstale":"2023-11-27T15:21:04.804666+0000","last_undegraded":"2023-11-27T15:21:04.804666+0000","last_fullsized":"2023-11-27T15:21:04.804666+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:28:31.347970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.933913+0000","last_change":"2023-11-27T15:20:33.224806+0000","last_active":"2023-11-27T15:21:02.933913+0000","last_peered":"2023-11-27T15:21:02.933913+0000","last_clean":"2023-11-27T15:21:02.933913+0000","last_became_active":"2023-11-27T15:20:33.224362+0000","last_became_peered":"2023-11-27T15:20:33.224362+0000","last_unstale":"2023-11-27T15:21:02.933913+0000","last_undegraded":"2023-11-27T15:21:02.933913+0000","last_fullsized":"2023-11-27T15:21:02.933913+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:34:11.704495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782260+0000","last_change":"2023-11-27T15:20:37.849274+0000","last_active":"2023-11-27T15:21:02.782260+0000","last_peered":"2023-11-27T15:21:02.782260+0000","last_clean":"2023-11-27T15:21:02.782260+0000","last_became_active":"2023-11-27T15:20:37.848839+0000","last_became_peered":"2023-11-27T15:20:37.848839+0000","last_unstale":"2023-11-27T15:21:02.782260+0000","last_undegraded":"2023-11-27T15:21:02.782260+0000","last_fullsized":"2023-11-27T15:21:02.782260+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:46:25.321810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779849+0000","last_change":"2023-11-27T15:20:32.660835+0000","last_active":"2023-11-27T15:21:02.779849+0000","last_peered":"2023-11-27T15:21:02.779849+0000","last_clean":"2023-11-27T15:21:02.779849+0000","last_became_active":"2023-11-27T15:20:32.660431+0000","last_became_peered":"2023-11-27T15:20:32.660431+0000","last_unstale":"2023-11-27T15:21:02.779849+0000","last_undegraded":"2023-11-27T15:21:02.779849+0000","last_fullsized":"2023-11-27T15:21:02.779849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:52:11.617105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932764+0000","last_change":"2023-11-27T15:20:33.245772+0000","last_active":"2023-11-27T15:21:02.932764+0000","last_peered":"2023-11-27T15:21:02.932764+0000","last_clean":"2023-11-27T15:21:02.932764+0000","last_became_active":"2023-11-27T15:20:33.245468+0000","last_became_peered":"2023-11-27T15:20:33.245468+0000","last_unstale":"2023-11-27T15:21:02.932764+0000","last_undegraded":"2023-11-27T15:21:02.932764+0000","last_fullsized":"2023-11-27T15:21:02.932764+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:24:42.963520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784658+0000","last_change":"2023-11-27T15:20:37.678972+0000","last_active":"2023-11-27T15:21:02.784658+0000","last_peered":"2023-11-27T15:21:02.784658+0000","last_clean":"2023-11-27T15:21:02.784658+0000","last_became_active":"2023-11-27T15:20:37.678304+0000","last_became_peered":"2023-11-27T15:20:37.678304+0000","last_unstale":"2023-11-27T15:21:02.784658+0000","last_undegraded":"2023-11-27T15:21:02.784658+0000","last_fullsized":"2023-11-27T15:21:02.784658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:32:12.064148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780000+0000","last_change":"2023-11-27T15:20:32.661739+0000","last_active":"2023-11-27T15:21:02.780000+0000","last_peered":"2023-11-27T15:21:02.780000+0000","last_clean":"2023-11-27T15:21:02.780000+0000","last_became_active":"2023-11-27T15:20:32.661401+0000","last_became_peered":"2023-11-27T15:20:32.661401+0000","last_unstale":"2023-11-27T15:21:02.780000+0000","last_undegraded":"2023-11-27T15:21:02.780000+0000","last_fullsized":"2023-11-27T15:21:02.780000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:47:15.469916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932830+0000","last_change":"2023-11-27T15:20:33.235595+0000","last_active":"2023-11-27T15:21:02.932830+0000","last_peered":"2023-11-27T15:21:02.932830+0000","last_clean":"2023-11-27T15:21:02.932830+0000","last_became_active":"2023-11-27T15:20:33.235277+0000","last_became_peered":"2023-11-27T15:20:33.235277+0000","last_unstale":"2023-11-27T15:21:02.932830+0000","last_undegraded":"2023-11-27T15:21:02.932830+0000","last_fullsized":"2023-11-27T15:21:02.932830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:23:31.896222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.934194+0000","last_change":"2023-11-27T15:21:02.934194+0000","last_active":"2023-11-27T15:21:02.934194+0000","last_peered":"2023-11-27T15:21:02.934194+0000","last_clean":"2023-11-27T15:21:02.934194+0000","last_became_active":"2023-11-27T15:21:02.933555+0000","last_became_peered":"2023-11-27T15:21:02.933555+0000","last_unstale":"2023-11-27T15:21:02.934194+0000","last_undegraded":"2023-11-27T15:21:02.934194+0000","last_fullsized":"2023-11-27T15:21:02.934194+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:49:59.949602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785630+0000","last_change":"2023-11-27T15:21:02.784351+0000","last_active":"2023-11-27T15:21:02.785630+0000","last_peered":"2023-11-27T15:21:02.785630+0000","last_clean":"2023-11-27T15:21:02.785630+0000","last_became_active":"2023-11-27T15:21:02.783996+0000","last_became_peered":"2023-11-27T15:21:02.783996+0000","last_unstale":"2023-11-27T15:21:02.785630+0000","last_undegraded":"2023-11-27T15:21:02.785630+0000","last_fullsized":"2023-11-27T15:21:02.785630+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:30:13.050830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":77,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:49.877453+0000","last_change":"2023-11-27T15:20:32.659823+0000","last_active":"2023-11-27T15:21:49.877453+0000","last_peered":"2023-11-27T15:21:49.877453+0000","last_clean":"2023-11-27T15:21:49.877453+0000","last_became_active":"2023-11-27T15:20:32.659472+0000","last_became_peered":"2023-11-27T15:20:32.659472+0000","last_unstale":"2023-11-27T15:21:49.877453+0000","last_undegraded":"2023-11-27T15:21:49.877453+0000","last_fullsized":"2023-11-27T15:21:49.877453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:19:47.161135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":47,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780173+0000","last_change":"2023-11-27T15:20:37.671494+0000","last_active":"2023-11-27T15:21:02.780173+0000","last_peered":"2023-11-27T15:21:02.780173+0000","last_clean":"2023-11-27T15:21:02.780173+0000","last_became_active":"2023-11-27T15:20:37.671010+0000","last_became_peered":"2023-11-27T15:20:37.671010+0000","last_unstale":"2023-11-27T15:21:02.780173+0000","last_undegraded":"2023-11-27T15:21:02.780173+0000","last_fullsized":"2023-11-27T15:21:02.780173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:09:00.036788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784810+0000","last_change":"2023-11-27T15:20:33.241669+0000","last_active":"2023-11-27T15:21:02.784810+0000","last_peered":"2023-11-27T15:21:02.784810+0000","last_clean":"2023-11-27T15:21:02.784810+0000","last_became_active":"2023-11-27T15:20:33.241373+0000","last_became_peered":"2023-11-27T15:20:33.241373+0000","last_unstale":"2023-11-27T15:21:02.784810+0000","last_undegraded":"2023-11-27T15:21:02.784810+0000","last_fullsized":"2023-11-27T15:21:02.784810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:21:19.709967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:04.804123+0000","last_change":"2023-11-27T15:20:32.647596+0000","last_active":"2023-11-27T15:21:04.804123+0000","last_peered":"2023-11-27T15:21:04.804123+0000","last_clean":"2023-11-27T15:21:04.804123+0000","last_became_active":"2023-11-27T15:20:32.647228+0000","last_became_peered":"2023-11-27T15:20:32.647228+0000","last_unstale":"2023-11-27T15:21:04.804123+0000","last_undegraded":"2023-11-27T15:21:04.804123+0000","last_fullsized":"2023-11-27T15:21:04.804123+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:40:49.354792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779663+0000","last_change":"2023-11-27T15:19:30.114855+0000","last_active":"2023-11-27T15:21:02.779663+0000","last_peered":"2023-11-27T15:21:02.779663+0000","last_clean":"2023-11-27T15:21:02.779663+0000","last_became_active":"2023-11-27T15:19:30.114419+0000","last_became_peered":"2023-11-27T15:19:30.114419+0000","last_unstale":"2023-11-27T15:21:02.779663+0000","last_undegraded":"2023-11-27T15:21:02.779663+0000","last_fullsized":"2023-11-27T15:21:02.779663+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:20.216695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:20.216695+0000","last_clean_scrub_stamp":"2023-11-27T15:16:20.216695+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:14:23.715586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932257+0000","last_change":"2023-11-27T15:20:37.850539+0000","last_active":"2023-11-27T15:21:02.932257+0000","last_peered":"2023-11-27T15:21:02.932257+0000","last_clean":"2023-11-27T15:21:02.932257+0000","last_became_active":"2023-11-27T15:20:37.850044+0000","last_became_peered":"2023-11-27T15:20:37.850044+0000","last_unstale":"2023-11-27T15:21:02.932257+0000","last_undegraded":"2023-11-27T15:21:02.932257+0000","last_fullsized":"2023-11-27T15:21:02.932257+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:29:43.906707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783098+0000","last_change":"2023-11-27T15:20:33.211319+0000","last_active":"2023-11-27T15:21:02.783098+0000","last_peered":"2023-11-27T15:21:02.783098+0000","last_clean":"2023-11-27T15:21:02.783098+0000","last_became_active":"2023-11-27T15:20:33.211004+0000","last_became_peered":"2023-11-27T15:20:33.211004+0000","last_unstale":"2023-11-27T15:21:02.783098+0000","last_undegraded":"2023-11-27T15:21:02.783098+0000","last_fullsized":"2023-11-27T15:21:02.783098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:57:04.622964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.799863+0000","last_change":"2023-11-27T15:20:32.655003+0000","last_active":"2023-11-27T15:21:03.799863+0000","last_peered":"2023-11-27T15:21:03.799863+0000","last_clean":"2023-11-27T15:21:03.799863+0000","last_became_active":"2023-11-27T15:20:32.654517+0000","last_became_peered":"2023-11-27T15:20:32.654517+0000","last_unstale":"2023-11-27T15:21:03.799863+0000","last_undegraded":"2023-11-27T15:21:03.799863+0000","last_fullsized":"2023-11-27T15:21:03.799863+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:12:26.090416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932725+0000","last_change":"2023-11-27T15:20:37.673183+0000","last_active":"2023-11-27T15:21:02.932725+0000","last_peered":"2023-11-27T15:21:02.932725+0000","last_clean":"2023-11-27T15:21:02.932725+0000","last_became_active":"2023-11-27T15:20:37.672838+0000","last_became_peered":"2023-11-27T15:20:37.672838+0000","last_unstale":"2023-11-27T15:21:02.932725+0000","last_undegraded":"2023-11-27T15:21:02.932725+0000","last_fullsized":"2023-11-27T15:21:02.932725+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:47:56.458647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782739+0000","last_change":"2023-11-27T15:20:32.659540+0000","last_active":"2023-11-27T15:21:02.782739+0000","last_peered":"2023-11-27T15:21:02.782739+0000","last_clean":"2023-11-27T15:21:02.782739+0000","last_became_active":"2023-11-27T15:20:32.659211+0000","last_became_peered":"2023-11-27T15:20:32.659211+0000","last_unstale":"2023-11-27T15:21:02.782739+0000","last_undegraded":"2023-11-27T15:21:02.782739+0000","last_fullsized":"2023-11-27T15:21:02.782739+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:02:50.140093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784853+0000","last_change":"2023-11-27T15:20:33.216405+0000","last_active":"2023-11-27T15:21:02.784853+0000","last_peered":"2023-11-27T15:21:02.784853+0000","last_clean":"2023-11-27T15:21:02.784853+0000","last_became_active":"2023-11-27T15:20:33.216072+0000","last_became_peered":"2023-11-27T15:20:33.216072+0000","last_unstale":"2023-11-27T15:21:02.784853+0000","last_undegraded":"2023-11-27T15:21:02.784853+0000","last_fullsized":"2023-11-27T15:21:02.784853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:09:28.211261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785517+0000","last_change":"2023-11-27T15:20:37.849753+0000","last_active":"2023-11-27T15:21:02.785517+0000","last_peered":"2023-11-27T15:21:02.785517+0000","last_clean":"2023-11-27T15:21:02.785517+0000","last_became_active":"2023-11-27T15:20:37.848576+0000","last_became_peered":"2023-11-27T15:20:37.848576+0000","last_unstale":"2023-11-27T15:21:02.785517+0000","last_undegraded":"2023-11-27T15:21:02.785517+0000","last_fullsized":"2023-11-27T15:21:02.785517+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:59:24.696190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'6","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:11.515280+0000","last_change":"2023-11-27T15:20:32.652739+0000","last_active":"2023-11-27T15:21:11.515280+0000","last_peered":"2023-11-27T15:21:11.515280+0000","last_clean":"2023-11-27T15:21:11.515280+0000","last_became_active":"2023-11-27T15:20:32.652314+0000","last_became_peered":"2023-11-27T15:20:32.652314+0000","last_unstale":"2023-11-27T15:21:11.515280+0000","last_undegraded":"2023-11-27T15:21:11.515280+0000","last_fullsized":"2023-11-27T15:21:11.515280+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:44:25.188063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.787449+0000","last_change":"2023-11-27T15:20:33.240552+0000","last_active":"2023-11-27T15:21:02.787449+0000","last_peered":"2023-11-27T15:21:02.787449+0000","last_clean":"2023-11-27T15:21:02.787449+0000","last_became_active":"2023-11-27T15:20:33.239885+0000","last_became_peered":"2023-11-27T15:20:33.239885+0000","last_unstale":"2023-11-27T15:21:02.787449+0000","last_undegraded":"2023-11-27T15:21:02.787449+0000","last_fullsized":"2023-11-27T15:21:02.787449+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:36:51.366222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779611+0000","last_change":"2023-11-27T15:20:37.669895+0000","last_active":"2023-11-27T15:21:02.779611+0000","last_peered":"2023-11-27T15:21:02.779611+0000","last_clean":"2023-11-27T15:21:02.779611+0000","last_became_active":"2023-11-27T15:20:37.669489+0000","last_became_peered":"2023-11-27T15:20:37.669489+0000","last_unstale":"2023-11-27T15:21:02.779611+0000","last_undegraded":"2023-11-27T15:21:02.779611+0000","last_fullsized":"2023-11-27T15:21:02.779611+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:18:44.798721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784573+0000","last_change":"2023-11-27T15:20:32.656773+0000","last_active":"2023-11-27T15:21:02.784573+0000","last_peered":"2023-11-27T15:21:02.784573+0000","last_clean":"2023-11-27T15:21:02.784573+0000","last_became_active":"2023-11-27T15:20:32.656451+0000","last_became_peered":"2023-11-27T15:20:32.656451+0000","last_unstale":"2023-11-27T15:21:02.784573+0000","last_undegraded":"2023-11-27T15:21:02.784573+0000","last_fullsized":"2023-11-27T15:21:02.784573+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:34:42.007222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786148+0000","last_change":"2023-11-27T15:20:33.231685+0000","last_active":"2023-11-27T15:21:02.786148+0000","last_peered":"2023-11-27T15:21:02.786148+0000","last_clean":"2023-11-27T15:21:02.786148+0000","last_became_active":"2023-11-27T15:20:33.228265+0000","last_became_peered":"2023-11-27T15:20:33.228265+0000","last_unstale":"2023-11-27T15:21:02.786148+0000","last_undegraded":"2023-11-27T15:21:02.786148+0000","last_fullsized":"2023-11-27T15:21:02.786148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:18:19.094289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.935830+0000","last_change":"2023-11-27T15:21:02.935830+0000","last_active":"2023-11-27T15:21:02.935830+0000","last_peered":"2023-11-27T15:21:02.935830+0000","last_clean":"2023-11-27T15:21:02.935830+0000","last_became_active":"2023-11-27T15:21:02.935412+0000","last_became_peered":"2023-11-27T15:21:02.935412+0000","last_unstale":"2023-11-27T15:21:02.935830+0000","last_undegraded":"2023-11-27T15:21:02.935830+0000","last_fullsized":"2023-11-27T15:21:02.935830+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:26:59.988473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.794970+0000","last_change":"2023-11-27T15:20:32.660424+0000","last_active":"2023-11-27T15:21:02.794970+0000","last_peered":"2023-11-27T15:21:02.794970+0000","last_clean":"2023-11-27T15:21:02.794970+0000","last_became_active":"2023-11-27T15:20:32.660112+0000","last_became_peered":"2023-11-27T15:20:32.660112+0000","last_unstale":"2023-11-27T15:21:02.794970+0000","last_undegraded":"2023-11-27T15:21:02.794970+0000","last_fullsized":"2023-11-27T15:21:02.794970+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:59:35.405721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782221+0000","last_change":"2023-11-27T15:20:33.227677+0000","last_active":"2023-11-27T15:21:02.782221+0000","last_peered":"2023-11-27T15:21:02.782221+0000","last_clean":"2023-11-27T15:21:02.782221+0000","last_became_active":"2023-11-27T15:20:33.227334+0000","last_became_peered":"2023-11-27T15:20:33.227334+0000","last_unstale":"2023-11-27T15:21:02.782221+0000","last_undegraded":"2023-11-27T15:21:02.782221+0000","last_fullsized":"2023-11-27T15:21:02.782221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:45:49.719897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.935341+0000","last_change":"2023-11-27T15:21:02.934516+0000","last_active":"2023-11-27T15:21:02.935341+0000","last_peered":"2023-11-27T15:21:02.935341+0000","last_clean":"2023-11-27T15:21:02.935341+0000","last_became_active":"2023-11-27T15:21:02.934012+0000","last_became_peered":"2023-11-27T15:21:02.934012+0000","last_unstale":"2023-11-27T15:21:02.935341+0000","last_undegraded":"2023-11-27T15:21:02.935341+0000","last_fullsized":"2023-11-27T15:21:02.935341+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:07:11.301253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784148+0000","last_change":"2023-11-27T15:20:32.657935+0000","last_active":"2023-11-27T15:21:02.784148+0000","last_peered":"2023-11-27T15:21:02.784148+0000","last_clean":"2023-11-27T15:21:02.784148+0000","last_became_active":"2023-11-27T15:20:32.657579+0000","last_became_peered":"2023-11-27T15:20:32.657579+0000","last_unstale":"2023-11-27T15:21:02.784148+0000","last_undegraded":"2023-11-27T15:21:02.784148+0000","last_fullsized":"2023-11-27T15:21:02.784148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:30:23.087862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932840+0000","last_change":"2023-11-27T15:20:33.225817+0000","last_active":"2023-11-27T15:21:02.932840+0000","last_peered":"2023-11-27T15:21:02.932840+0000","last_clean":"2023-11-27T15:21:02.932840+0000","last_became_active":"2023-11-27T15:20:33.225438+0000","last_became_peered":"2023-11-27T15:20:33.225438+0000","last_unstale":"2023-11-27T15:21:02.932840+0000","last_undegraded":"2023-11-27T15:21:02.932840+0000","last_fullsized":"2023-11-27T15:21:02.932840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:04:27.668653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.934889+0000","last_change":"2023-11-27T15:21:02.934304+0000","last_active":"2023-11-27T15:21:02.934889+0000","last_peered":"2023-11-27T15:21:02.934889+0000","last_clean":"2023-11-27T15:21:02.934889+0000","last_became_active":"2023-11-27T15:21:02.933709+0000","last_became_peered":"2023-11-27T15:21:02.933709+0000","last_unstale":"2023-11-27T15:21:02.934889+0000","last_undegraded":"2023-11-27T15:21:02.934889+0000","last_fullsized":"2023-11-27T15:21:02.934889+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:43:38.974368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780238+0000","last_change":"2023-11-27T15:20:32.659484+0000","last_active":"2023-11-27T15:21:02.780238+0000","last_peered":"2023-11-27T15:21:02.780238+0000","last_clean":"2023-11-27T15:21:02.780238+0000","last_became_active":"2023-11-27T15:20:32.659134+0000","last_became_peered":"2023-11-27T15:20:32.659134+0000","last_unstale":"2023-11-27T15:21:02.780238+0000","last_undegraded":"2023-11-27T15:21:02.780238+0000","last_fullsized":"2023-11-27T15:21:02.780238+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:04:03.053379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784616+0000","last_change":"2023-11-27T15:20:33.235755+0000","last_active":"2023-11-27T15:21:02.784616+0000","last_peered":"2023-11-27T15:21:02.784616+0000","last_clean":"2023-11-27T15:21:02.784616+0000","last_became_active":"2023-11-27T15:20:33.234576+0000","last_became_peered":"2023-11-27T15:20:33.234576+0000","last_unstale":"2023-11-27T15:21:02.784616+0000","last_undegraded":"2023-11-27T15:21:02.784616+0000","last_fullsized":"2023-11-27T15:21:02.784616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:25:25.181797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784302+0000","last_change":"2023-11-27T15:20:37.849121+0000","last_active":"2023-11-27T15:21:02.784302+0000","last_peered":"2023-11-27T15:21:02.784302+0000","last_clean":"2023-11-27T15:21:02.784302+0000","last_became_active":"2023-11-27T15:20:37.848683+0000","last_became_peered":"2023-11-27T15:20:37.848683+0000","last_unstale":"2023-11-27T15:21:02.784302+0000","last_undegraded":"2023-11-27T15:21:02.784302+0000","last_fullsized":"2023-11-27T15:21:02.784302+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:05:39.554898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.782542+0000","last_change":"2023-11-27T15:20:32.655896+0000","last_active":"2023-11-27T15:21:02.782542+0000","last_peered":"2023-11-27T15:21:02.782542+0000","last_clean":"2023-11-27T15:21:02.782542+0000","last_became_active":"2023-11-27T15:20:32.655623+0000","last_became_peered":"2023-11-27T15:20:32.655623+0000","last_unstale":"2023-11-27T15:21:02.782542+0000","last_undegraded":"2023-11-27T15:21:02.782542+0000","last_fullsized":"2023-11-27T15:21:02.782542+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:01:57.075773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779947+0000","last_change":"2023-11-27T15:20:33.227318+0000","last_active":"2023-11-27T15:21:02.779947+0000","last_peered":"2023-11-27T15:21:02.779947+0000","last_clean":"2023-11-27T15:21:02.779947+0000","last_became_active":"2023-11-27T15:20:33.226944+0000","last_became_peered":"2023-11-27T15:20:33.226944+0000","last_unstale":"2023-11-27T15:21:02.779947+0000","last_undegraded":"2023-11-27T15:21:02.779947+0000","last_fullsized":"2023-11-27T15:21:02.779947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:47:26.971182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.794990+0000","last_change":"2023-11-27T15:20:37.669098+0000","last_active":"2023-11-27T15:21:02.794990+0000","last_peered":"2023-11-27T15:21:02.794990+0000","last_clean":"2023-11-27T15:21:02.794990+0000","last_became_active":"2023-11-27T15:20:37.668693+0000","last_became_peered":"2023-11-27T15:20:37.668693+0000","last_unstale":"2023-11-27T15:21:02.794990+0000","last_undegraded":"2023-11-27T15:21:02.794990+0000","last_fullsized":"2023-11-27T15:21:02.794990+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:38:51.699084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:03.802617+0000","last_change":"2023-11-27T15:20:32.660285+0000","last_active":"2023-11-27T15:21:03.802617+0000","last_peered":"2023-11-27T15:21:03.802617+0000","last_clean":"2023-11-27T15:21:03.802617+0000","last_became_active":"2023-11-27T15:20:32.659982+0000","last_became_peered":"2023-11-27T15:20:32.659982+0000","last_unstale":"2023-11-27T15:21:03.802617+0000","last_undegraded":"2023-11-27T15:21:03.802617+0000","last_fullsized":"2023-11-27T15:21:03.802617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:42:36.081550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780414+0000","last_change":"2023-11-27T15:20:33.226629+0000","last_active":"2023-11-27T15:21:02.780414+0000","last_peered":"2023-11-27T15:21:02.780414+0000","last_clean":"2023-11-27T15:21:02.780414+0000","last_became_active":"2023-11-27T15:20:33.226183+0000","last_became_peered":"2023-11-27T15:20:33.226183+0000","last_unstale":"2023-11-27T15:21:02.780414+0000","last_undegraded":"2023-11-27T15:21:02.780414+0000","last_fullsized":"2023-11-27T15:21:02.780414+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:33:24.895609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785284+0000","last_change":"2023-11-27T15:20:37.680987+0000","last_active":"2023-11-27T15:21:02.785284+0000","last_peered":"2023-11-27T15:21:02.785284+0000","last_clean":"2023-11-27T15:21:02.785284+0000","last_became_active":"2023-11-27T15:20:37.680219+0000","last_became_peered":"2023-11-27T15:20:37.680219+0000","last_unstale":"2023-11-27T15:21:02.785284+0000","last_undegraded":"2023-11-27T15:21:02.785284+0000","last_fullsized":"2023-11-27T15:21:02.785284+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:27:01.990771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932894+0000","last_change":"2023-11-27T15:20:32.655035+0000","last_active":"2023-11-27T15:21:02.932894+0000","last_peered":"2023-11-27T15:21:02.932894+0000","last_clean":"2023-11-27T15:21:02.932894+0000","last_became_active":"2023-11-27T15:20:32.654687+0000","last_became_peered":"2023-11-27T15:20:32.654687+0000","last_unstale":"2023-11-27T15:21:02.932894+0000","last_undegraded":"2023-11-27T15:21:02.932894+0000","last_fullsized":"2023-11-27T15:21:02.932894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:54:01.028543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780460+0000","last_change":"2023-11-27T15:20:33.245908+0000","last_active":"2023-11-27T15:21:02.780460+0000","last_peered":"2023-11-27T15:21:02.780460+0000","last_clean":"2023-11-27T15:21:02.780460+0000","last_became_active":"2023-11-27T15:20:33.245576+0000","last_became_peered":"2023-11-27T15:20:33.245576+0000","last_unstale":"2023-11-27T15:21:02.780460+0000","last_undegraded":"2023-11-27T15:21:02.780460+0000","last_fullsized":"2023-11-27T15:21:02.780460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:41:20.860779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.931498+0000","last_change":"2023-11-27T15:20:37.674253+0000","last_active":"2023-11-27T15:21:02.931498+0000","last_peered":"2023-11-27T15:21:02.931498+0000","last_clean":"2023-11-27T15:21:02.931498+0000","last_became_active":"2023-11-27T15:20:37.673708+0000","last_became_peered":"2023-11-27T15:20:37.673708+0000","last_unstale":"2023-11-27T15:21:02.931498+0000","last_undegraded":"2023-11-27T15:21:02.931498+0000","last_fullsized":"2023-11-27T15:21:02.931498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:25:32.219577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786521+0000","last_change":"2023-11-27T15:20:32.659605+0000","last_active":"2023-11-27T15:21:02.786521+0000","last_peered":"2023-11-27T15:21:02.786521+0000","last_clean":"2023-11-27T15:21:02.786521+0000","last_became_active":"2023-11-27T15:20:32.659186+0000","last_became_peered":"2023-11-27T15:20:32.659186+0000","last_unstale":"2023-11-27T15:21:02.786521+0000","last_undegraded":"2023-11-27T15:21:02.786521+0000","last_fullsized":"2023-11-27T15:21:02.786521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:19:33.727199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780674+0000","last_change":"2023-11-27T15:20:33.227914+0000","last_active":"2023-11-27T15:21:02.780674+0000","last_peered":"2023-11-27T15:21:02.780674+0000","last_clean":"2023-11-27T15:21:02.780674+0000","last_became_active":"2023-11-27T15:20:33.227572+0000","last_became_peered":"2023-11-27T15:20:33.227572+0000","last_unstale":"2023-11-27T15:21:02.780674+0000","last_undegraded":"2023-11-27T15:21:02.780674+0000","last_fullsized":"2023-11-27T15:21:02.780674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:16:00.098620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932874+0000","last_change":"2023-11-27T15:20:33.215605+0000","last_active":"2023-11-27T15:21:02.932874+0000","last_peered":"2023-11-27T15:21:02.932874+0000","last_clean":"2023-11-27T15:21:02.932874+0000","last_became_active":"2023-11-27T15:20:33.215295+0000","last_became_peered":"2023-11-27T15:20:33.215295+0000","last_unstale":"2023-11-27T15:21:02.932874+0000","last_undegraded":"2023-11-27T15:21:02.932874+0000","last_fullsized":"2023-11-27T15:21:02.932874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:46:08.937950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780509+0000","last_change":"2023-11-27T15:20:33.214014+0000","last_active":"2023-11-27T15:21:02.780509+0000","last_peered":"2023-11-27T15:21:02.780509+0000","last_clean":"2023-11-27T15:21:02.780509+0000","last_became_active":"2023-11-27T15:20:33.213685+0000","last_became_peered":"2023-11-27T15:20:33.213685+0000","last_unstale":"2023-11-27T15:21:02.780509+0000","last_undegraded":"2023-11-27T15:21:02.780509+0000","last_fullsized":"2023-11-27T15:21:02.780509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:06:53.889016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.779696+0000","last_change":"2023-11-27T15:20:33.229722+0000","last_active":"2023-11-27T15:21:02.779696+0000","last_peered":"2023-11-27T15:21:02.779696+0000","last_clean":"2023-11-27T15:21:02.779696+0000","last_became_active":"2023-11-27T15:20:33.229372+0000","last_became_peered":"2023-11-27T15:20:33.229372+0000","last_unstale":"2023-11-27T15:21:02.779696+0000","last_undegraded":"2023-11-27T15:21:02.779696+0000","last_fullsized":"2023-11-27T15:21:02.779696+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:48:26.454003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.785590+0000","last_change":"2023-11-27T15:20:33.235047+0000","last_active":"2023-11-27T15:21:02.785590+0000","last_peered":"2023-11-27T15:21:02.785590+0000","last_clean":"2023-11-27T15:21:02.785590+0000","last_became_active":"2023-11-27T15:20:33.233078+0000","last_became_peered":"2023-11-27T15:20:33.233078+0000","last_unstale":"2023-11-27T15:21:02.785590+0000","last_undegraded":"2023-11-27T15:21:02.785590+0000","last_fullsized":"2023-11-27T15:21:02.785590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:56:34.043375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784969+0000","last_change":"2023-11-27T15:20:33.229118+0000","last_active":"2023-11-27T15:21:02.784969+0000","last_peered":"2023-11-27T15:21:02.784969+0000","last_clean":"2023-11-27T15:21:02.784969+0000","last_became_active":"2023-11-27T15:20:33.228748+0000","last_became_peered":"2023-11-27T15:20:33.228748+0000","last_unstale":"2023-11-27T15:21:02.784969+0000","last_undegraded":"2023-11-27T15:21:02.784969+0000","last_fullsized":"2023-11-27T15:21:02.784969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:44:41.459719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.933214+0000","last_change":"2023-11-27T15:20:33.222714+0000","last_active":"2023-11-27T15:21:02.933214+0000","last_peered":"2023-11-27T15:21:02.933214+0000","last_clean":"2023-11-27T15:21:02.933214+0000","last_became_active":"2023-11-27T15:20:33.222396+0000","last_became_peered":"2023-11-27T15:20:33.222396+0000","last_unstale":"2023-11-27T15:21:02.933214+0000","last_undegraded":"2023-11-27T15:21:02.933214+0000","last_fullsized":"2023-11-27T15:21:02.933214+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:13:29.493579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.780269+0000","last_change":"2023-11-27T15:20:33.223568+0000","last_active":"2023-11-27T15:21:02.780269+0000","last_peered":"2023-11-27T15:21:02.780269+0000","last_clean":"2023-11-27T15:21:02.780269+0000","last_became_active":"2023-11-27T15:20:33.220687+0000","last_became_peered":"2023-11-27T15:20:33.220687+0000","last_unstale":"2023-11-27T15:21:02.780269+0000","last_undegraded":"2023-11-27T15:21:02.780269+0000","last_fullsized":"2023-11-27T15:21:02.780269+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:56:29.431345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.786992+0000","last_change":"2023-11-27T15:21:02.786992+0000","last_active":"2023-11-27T15:21:02.786992+0000","last_peered":"2023-11-27T15:21:02.786992+0000","last_clean":"2023-11-27T15:21:02.786992+0000","last_became_active":"2023-11-27T15:21:02.786644+0000","last_became_peered":"2023-11-27T15:21:02.786644+0000","last_unstale":"2023-11-27T15:21:02.786992+0000","last_undegraded":"2023-11-27T15:21:02.786992+0000","last_fullsized":"2023-11-27T15:21:02.786992+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:18:27.697514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.931958+0000","last_change":"2023-11-27T15:20:33.245815+0000","last_active":"2023-11-27T15:21:02.931958+0000","last_peered":"2023-11-27T15:21:02.931958+0000","last_clean":"2023-11-27T15:21:02.931958+0000","last_became_active":"2023-11-27T15:20:33.245243+0000","last_became_peered":"2023-11-27T15:20:33.245243+0000","last_unstale":"2023-11-27T15:21:02.931958+0000","last_undegraded":"2023-11-27T15:21:02.931958+0000","last_fullsized":"2023-11-27T15:21:02.931958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:26:43.029408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.784226+0000","last_change":"2023-11-27T15:20:33.245595+0000","last_active":"2023-11-27T15:21:02.784226+0000","last_peered":"2023-11-27T15:21:02.784226+0000","last_clean":"2023-11-27T15:21:02.784226+0000","last_became_active":"2023-11-27T15:20:33.245294+0000","last_became_peered":"2023-11-27T15:20:33.245294+0000","last_unstale":"2023-11-27T15:21:02.784226+0000","last_undegraded":"2023-11-27T15:21:02.784226+0000","last_fullsized":"2023-11-27T15:21:02.784226+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:38:31.036319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.795538+0000","last_change":"2023-11-27T15:20:33.232976+0000","last_active":"2023-11-27T15:21:02.795538+0000","last_peered":"2023-11-27T15:21:02.795538+0000","last_clean":"2023-11-27T15:21:02.795538+0000","last_became_active":"2023-11-27T15:20:33.232648+0000","last_became_peered":"2023-11-27T15:20:33.232648+0000","last_unstale":"2023-11-27T15:21:02.795538+0000","last_undegraded":"2023-11-27T15:21:02.795538+0000","last_fullsized":"2023-11-27T15:21:02.795538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:45:02.656102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.783702+0000","last_change":"2023-11-27T15:20:33.226332+0000","last_active":"2023-11-27T15:21:02.783702+0000","last_peered":"2023-11-27T15:21:02.783702+0000","last_clean":"2023-11-27T15:21:02.783702+0000","last_became_active":"2023-11-27T15:20:33.226000+0000","last_became_peered":"2023-11-27T15:20:33.226000+0000","last_unstale":"2023-11-27T15:21:02.783702+0000","last_undegraded":"2023-11-27T15:21:02.783702+0000","last_fullsized":"2023-11-27T15:21:02.783702+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:24:34.018543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-27T15:21:02.932933+0000","last_change":"2023-11-27T15:20:33.217927+0000","last_active":"2023-11-27T15:21:02.932933+0000","last_peered":"2023-11-27T15:21:02.932933+0000","last_clean":"2023-11-27T15:21:02.932933+0000","last_became_active":"2023-11-27T15:20:33.217003+0000","last_became_peered":"2023-11-27T15:20:33.217003+0000","last_unstale":"2023-11-27T15:21:02.932933+0000","last_undegraded":"2023-11-27T15:21:02.932933+0000","last_fullsized":"2023-11-27T15:21:02.932933+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:27:50.580759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63712,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":316,"num_read_kb":223,"num_write":158,"num_write_kb":112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":191136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138,"ondisk_log_size":138,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416413,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6176,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93738976,"statfs":{"total":95995035648,"available":95988711424,"internally_reserved":0,"allocated":1007616,"data_stored":494720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.46000000000000002}]},{"osd":1,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60599999999999998,"15min":0.60299999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89200000000000002,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60899999999999999,"15min":0.61299999999999999},"min":{"1min":0.35499999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76300000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.57599999999999996}]},{"osd":2,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.30299999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59999999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.46100000000000002,"15min":0.45800000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.67200000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.432}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":4,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.86499999999999999,"5min":1.177,"15min":1.177},"last":0.627},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61299999999999999,"15min":0.61599999999999999},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84899999999999998,"5min":1.141,"15min":1.141},"last":0.61199999999999999}]},{"osd":5,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52700000000000002,"15min":0.53100000000000003},"min":{"1min":0.34399999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69299999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88700000000000001,"5min":1.032,"15min":1.032},"last":0.67000000000000004}]},{"osd":6,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.47499999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.875,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85999999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56100000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.46500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97899999999999998,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70999999999999996,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95699999999999996,"5min":1.54,"15min":1.54},"last":0.77100000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:21:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.70099999999999996}]},{"osd":9,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71599999999999997,"15min":0.71799999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92500000000000004,"5min":1.286,"15min":1.286},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.76900000000000002,"15min":0.76500000000000001},"min":{"1min":0.498,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.78400000000000003}]},{"osd":10,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.53900000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96299999999999997,"5min":1.304,"15min":1.304},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73099999999999998,"15min":0.73299999999999998},"min":{"1min":0.53000000000000003,"5min":0.501,"15min":0.501},"max":{"1min":0.95199999999999996,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.755}]}]},{"osd":10,"up_from":71,"seq":304942678048,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7172,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93737980,"statfs":{"total":95995035648,"available":95987691520,"internally_reserved":0,"allocated":1699840,"data_stored":1146234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.438}]},{"osd":1,"last update":"Mon Nov 27 15:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.47199999999999998,"15min":0.47799999999999998},"min":{"1min":0.309,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.60099999999999998,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.46400000000000002,"15min":0.46800000000000003},"min":{"1min":0.35899999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60999999999999999,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.40699999999999997}]},{"osd":2,"last update":"Mon Nov 27 15:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.40100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.754,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.42099999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.75800000000000001,"5min":0.87,"15min":0.87},"last":0.63300000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.63100000000000001,"15min":0.64300000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71999999999999997,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.65300000000000002,"15min":0.66600000000000004},"min":{"1min":0.44700000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.76600000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.52700000000000002}]},{"osd":4,"last update":"Mon Nov 27 15:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72899999999999998,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56999999999999995,"15min":0.57099999999999995},"min":{"1min":0.40799999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77900000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.495}]},{"osd":5,"last update":"Mon Nov 27 15:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.63400000000000001,"15min":0.624},"min":{"1min":0.55500000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.95599999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.497,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.875,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.754}]},{"osd":6,"last update":"Mon Nov 27 15:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.68799999999999994,"15min":0.69599999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.89000000000000001,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.71999999999999997,"15min":0.73099999999999998},"min":{"1min":0.55700000000000005,"5min":0.496,"15min":0.496},"max":{"1min":0.79400000000000004,"5min":1.004,"15min":1.004},"last":0.62}]},{"osd":7,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.56799999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.75700000000000001,"15min":0.76800000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.93200000000000005,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.627,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.879,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.70099999999999996}]},{"osd":9,"last update":"Mon Nov 27 15:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.78500000000000003,"15min":0.79600000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.90800000000000003,"5min":1.111,"15min":1.111},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.58699999999999997,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.91900000000000004,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.73999999999999999}]},{"osd":11,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.746,"15min":0.75},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.77000000000000002,"15min":0.77900000000000003},"min":{"1min":0.56999999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.92600000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.71299999999999997}]}]},{"osd":9,"up_from":66,"seq":283467841572,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6384,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738768,"statfs":{"total":95995035648,"available":95988498432,"internally_reserved":0,"allocated":1024000,"data_stored":494956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.51500000000000001,"15min":0.52300000000000002},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40899999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.004,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":1.278}]},{"osd":1,"last update":"Mon Nov 27 15:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.72799999999999998,"15min":0.73699999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64700000000000002,"15min":0.65200000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.80500000000000005,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":1.645}]},{"osd":2,"last update":"Mon Nov 27 15:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.78800000000000003,"15min":0.79800000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.4019999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.76300000000000001,"15min":0.77900000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.97999999999999998,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":1.788}]},{"osd":3,"last update":"Mon Nov 27 15:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.69599999999999995,"15min":0.71199999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.97099999999999997,"5min":1.141,"15min":1.141},"last":1.2310000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.77500000000000002,"15min":0.79800000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.127,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.85599999999999998}]},{"osd":4,"last update":"Mon Nov 27 15:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.74199999999999999,"15min":0.75700000000000001},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81899999999999995,"5min":1.171,"15min":1.171},"last":1.595},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.80100000000000005,"15min":0.81499999999999995},"min":{"1min":0.55800000000000005,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0740000000000001,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":1.304}]},{"osd":5,"last update":"Mon Nov 27 15:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.83899999999999997,"15min":0.85299999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.1020000000000001,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.5680000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.873,"15min":0.89300000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":1.4470000000000001}]},{"osd":6,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.80400000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.89200000000000002,"15min":0.91300000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0640000000000001,"5min":1.202,"15min":1.202},"last":1.496},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.93600000000000005,"15min":0.95599999999999996},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":1.5089999999999999}]},{"osd":8,"last update":"Mon Nov 27 15:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.94699999999999995,"15min":0.97699999999999998},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.0449999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.8160000000000001},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.95599999999999996,"15min":0.97599999999999998},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.212,"5min":1.536,"15min":1.536},"last":1.736}]},{"osd":10,"last update":"Mon Nov 27 15:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.91800000000000004,"15min":0.93400000000000005},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":1.5469999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.85999999999999999,"15min":0.872},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.92100000000000004,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":1.7190000000000001}]},{"osd":11,"last update":"Mon Nov 27 15:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":1.7569999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":1.659}]}]},{"osd":8,"up_from":58,"seq":249108103210,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6508,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738644,"statfs":{"total":95995035648,"available":95988371456,"internally_reserved":0,"allocated":1019904,"data_stored":494545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.44400000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.59799999999999998},"min":{"1min":0.45700000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.75,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.50600000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.59999999999999998,"15min":0.59899999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.60599999999999998,"15min":0.59699999999999998},"min":{"1min":0.42699999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.749}]},{"osd":2,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.44600000000000001,"15min":0.439},"min":{"1min":0.376,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.46600000000000003,"15min":0.44800000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.66600000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.625}]},{"osd":3,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.60599999999999998,"15min":0.60099999999999998},"min":{"1min":0.503,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.45300000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.53800000000000003}]},{"osd":4,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63,"15min":0.628},"min":{"1min":0.46700000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.84699999999999998,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86599999999999999,"5min":1.377,"15min":1.377},"last":0.59199999999999997}]},{"osd":5,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.55600000000000005},"min":{"1min":0.39100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.67000000000000004}]},{"osd":6,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.73799999999999999,"15min":0.68400000000000005},"min":{"1min":0.47599999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":6.4450000000000003,"5min":6.4450000000000003,"15min":6.4450000000000003},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.72299999999999998,"15min":0.70299999999999996},"min":{"1min":0.55200000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.94699999999999995}]},{"osd":7,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.73999999999999999,"15min":0.72199999999999998},"min":{"1min":0.53200000000000003,"5min":0.442,"15min":0.442},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.434,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.99299999999999999,"5min":1.323,"15min":1.323},"last":0.77100000000000002}]},{"osd":9,"last update":"Mon Nov 27 15:21:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.56799999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.78600000000000003,"15min":0.77700000000000002},"min":{"1min":0.65300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.877}]},{"osd":10,"last update":"Mon Nov 27 15:21:28 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.72099999999999997,"15min":0.70499999999999996},"min":{"1min":0.59799999999999998,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.752,"15min":0.73299999999999998},"min":{"1min":0.628,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.90200000000000002}]},{"osd":11,"last update":"Mon Nov 27 15:21:36 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.80200000000000005}]}]},{"osd":7,"up_from":51,"seq":219043332141,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7156,"kb_used_data":1580,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93737996,"statfs":{"total":95995035648,"available":95987707904,"internally_reserved":0,"allocated":1617920,"data_stored":1090688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.436,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84699999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.44},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77100000000000002,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.46999999999999997}]},{"osd":1,"last update":"Mon Nov 27 15:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40100000000000002,"15min":0.39100000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.54300000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.439,"5min":0.40899999999999997,"15min":0.39900000000000002},"min":{"1min":0.28399999999999997,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.63700000000000001,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.496}]},{"osd":2,"last update":"Mon Nov 27 15:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.433,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57699999999999996,"15min":0.56799999999999995},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.79400000000000004}]},{"osd":3,"last update":"Mon Nov 27 15:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.39500000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.89600000000000002,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.64000000000000001,"15min":0.63600000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77600000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.69399999999999995}]},{"osd":4,"last update":"Mon Nov 27 15:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.50800000000000001,"15min":0.49199999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.76800000000000002,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.51800000000000002,"15min":0.52600000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70399999999999996,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.39200000000000002}]},{"osd":5,"last update":"Mon Nov 27 15:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.625,"15min":0.63},"min":{"1min":0.47099999999999997,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.75},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.623,"15min":0.61899999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.61399999999999999}]},{"osd":6,"last update":"Mon Nov 27 15:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.68500000000000005,"15min":0.67200000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.036,"5min":1.046,"15min":1.046},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68999999999999995,"15min":0.69399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.93999999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.81799999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:21:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.66900000000000004,"15min":0.65100000000000002},"min":{"1min":0.60399999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.53100000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.873}]},{"osd":9,"last update":"Mon Nov 27 15:21:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71799999999999997,"15min":0.72199999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.76500000000000001,"15min":0.76700000000000002},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.97899999999999998,"5min":0.996,"15min":0.996},"last":0.76300000000000001}]},{"osd":10,"last update":"Mon Nov 27 15:21:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.499,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.67000000000000004,"15min":0.66200000000000003},"min":{"1min":0.51600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.85799999999999998}]},{"osd":11,"last update":"Mon Nov 27 15:21:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.93000000000000005}]}]},{"osd":6,"up_from":46,"seq":197568495666,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6428,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738724,"statfs":{"total":95995035648,"available":95988453376,"internally_reserved":0,"allocated":1003520,"data_stored":490132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47699999999999998,"15min":0.46999999999999997},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.499},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.50700000000000001,"15min":0.48499999999999999},"min":{"1min":0.36599999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"last":0.89100000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65600000000000003,"15min":0.63600000000000001},"min":{"1min":0.45300000000000001,"5min":0.441,"15min":0.441},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.60899999999999999,"15min":0.57899999999999996},"min":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.55700000000000005}]},{"osd":2,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66600000000000004,"15min":0.64800000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68200000000000005,"15min":0.67400000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.91600000000000004}]},{"osd":3,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55700000000000005,"15min":0.54400000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.61299999999999999,"15min":0.59599999999999997},"min":{"1min":0.38900000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.96399999999999997}]},{"osd":4,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69399999999999995,"15min":0.68600000000000005},"min":{"1min":0.55700000000000005,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.68500000000000005,"15min":0.66900000000000004},"min":{"1min":0.48799999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.63100000000000001}]},{"osd":5,"last update":"Mon Nov 27 15:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71899999999999997,"15min":0.70799999999999996},"min":{"1min":0.505,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76300000000000001,"15min":0.753},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.94899999999999995}]},{"osd":7,"last update":"Mon Nov 27 15:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.75900000000000001,"15min":0.749},"min":{"1min":0.60099999999999998,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.048,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.751,"5min":0.73499999999999999,"15min":0.71099999999999997},"min":{"1min":0.57599999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97399999999999998,"5min":1.419,"15min":1.419},"last":0.93600000000000005}]},{"osd":8,"last update":"Mon Nov 27 15:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85799999999999998,"15min":0.86399999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.22,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.84799999999999998,"15min":0.85799999999999998},"min":{"1min":0.60699999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.77100000000000002}]},{"osd":9,"last update":"Mon Nov 27 15:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.85199999999999998,"15min":0.86699999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.0309999999999999,"5min":1.448,"15min":1.448},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.91200000000000003,"15min":0.92700000000000005},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.1950000000000001,"5min":1.712,"15min":1.712},"last":1.0209999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.82699999999999996,"15min":0.82299999999999995},"min":{"1min":0.63800000000000001,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0880000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.89700000000000002,"15min":0.88700000000000001},"min":{"1min":0.69799999999999995,"5min":0.626,"15min":0.626},"max":{"1min":1.454,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.98299999999999998}]},{"osd":11,"last update":"Mon Nov 27 15:21:38 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":4.9290000000000003,"5min":4.9290000000000003,"15min":4.9290000000000003},"last":1.036},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":1.0429999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574921,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7516,"kb_used_data":1044,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737636,"statfs":{"total":95995035648,"available":95987339264,"internally_reserved":0,"allocated":1069056,"data_stored":549850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.52500000000000002,"15min":0.49099999999999999},"min":{"1min":0.40500000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.54800000000000004,"15min":0.49399999999999999},"min":{"1min":0.42599999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.92200000000000004,"5min":1.012,"15min":1.012},"last":0.505}]},{"osd":2,"last update":"Mon Nov 27 15:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.52300000000000002,"15min":0.48699999999999999},"min":{"1min":0.42099999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.81299999999999994,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.56200000000000006,"15min":0.499},"min":{"1min":0.41199999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.42699999999999999}]},{"osd":3,"last update":"Mon Nov 27 15:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.55500000000000005,"15min":0.52800000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54700000000000004,"15min":0.53400000000000003},"min":{"1min":0.39700000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84899999999999998,"5min":0.88,"15min":0.88},"last":0.66300000000000003}]},{"osd":4,"last update":"Mon Nov 27 15:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.44,"15min":0.40000000000000002},"min":{"1min":0.34499999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.626},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.45600000000000002,"15min":0.41399999999999998},"min":{"1min":0.316,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.79000000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.59699999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.57499999999999996,"15min":0.54700000000000004},"min":{"1min":0.48499999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.88800000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.61299999999999999,"15min":0.56899999999999995},"min":{"1min":0.46600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.70199999999999996}]},{"osd":6,"last update":"Mon Nov 27 15:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.64700000000000002,"15min":0.624},"min":{"1min":0.45900000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.64400000000000002,"15min":0.62},"min":{"1min":0.53400000000000003,"5min":0.38,"15min":0.38},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.64800000000000002}]},{"osd":7,"last update":"Mon Nov 27 15:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58499999999999996,"15min":0.56399999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.621,"15min":0.59399999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95599999999999996,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.68200000000000005}]},{"osd":8,"last update":"Mon Nov 27 15:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.66700000000000004,"15min":0.64400000000000002},"min":{"1min":0.45800000000000002,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92100000000000004,"5min":0.995,"15min":0.995},"last":0.754},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.66300000000000003,"15min":0.64400000000000002},"min":{"1min":0.53400000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.78500000000000003}]},{"osd":9,"last update":"Mon Nov 27 15:21:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.754,"15min":0.73999999999999999},"min":{"1min":0.55700000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.76700000000000002,"15min":0.75700000000000001},"min":{"1min":0.60099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0549999999999999,"5min":1.169,"15min":1.169},"last":0.79600000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.69199999999999995,"15min":0.68700000000000006},"min":{"1min":0.504,"5min":0.439,"15min":0.439},"max":{"1min":0.98499999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.77500000000000002,"15min":0.76900000000000002},"min":{"1min":0.57999999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.80500000000000005}]},{"osd":11,"last update":"Mon Nov 27 15:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.86399999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7412,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737740,"statfs":{"total":95995035648,"available":95987445760,"internally_reserved":0,"allocated":1028096,"data_stored":491520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 27 15:21:08 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.57299999999999995,"15min":0.51100000000000001},"min":{"1min":0.42599999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.89500000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59399999999999997,"15min":0.52700000000000002},"min":{"1min":0.45900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.86699999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.71899999999999997}]},{"osd":2,"last update":"Mon Nov 27 15:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.62,"15min":0.57999999999999996},"min":{"1min":0.41299999999999998,"5min":0.37,"15min":0.37},"max":{"1min":0.88700000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61899999999999999,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83299999999999996,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.66200000000000003}]},{"osd":3,"last update":"Mon Nov 27 15:21:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.53900000000000003}]},{"osd":4,"last update":"Mon Nov 27 15:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.46400000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.876,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.65400000000000003,"15min":0.60799999999999998},"min":{"1min":0.51300000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.97199999999999998,"5min":1.244,"15min":1.244},"last":0.79800000000000004}]},{"osd":5,"last update":"Mon Nov 27 15:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64100000000000001,"15min":0.63900000000000001},"min":{"1min":0.46000000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.84599999999999997,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.70599999999999996,"15min":0.68200000000000005},"min":{"1min":0.52300000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.90200000000000002,"5min":1.379,"15min":1.379},"last":0.69999999999999996}]},{"osd":6,"last update":"Mon Nov 27 15:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65500000000000003,"15min":0.627},"min":{"1min":0.41499999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.121,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63600000000000001,"15min":0.61799999999999999},"min":{"1min":0.437,"5min":0.373,"15min":0.373},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.61899999999999999}]},{"osd":7,"last update":"Mon Nov 27 15:21:38 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.77800000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.755,"15min":0.754},"min":{"1min":0.59799999999999998,"5min":0.498,"15min":0.498},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76500000000000001,"15min":0.78100000000000003},"min":{"1min":0.54400000000000004,"5min":0.504,"15min":0.504},"max":{"1min":0.94099999999999995,"5min":1.327,"15min":1.327},"last":0.73699999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:21:08 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.874,"15min":0.88},"min":{"1min":0.68300000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.1000000000000001,"5min":1.218,"15min":1.218},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.84899999999999998,"15min":0.85799999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95399999999999996,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.94199999999999995}]},{"osd":10,"last update":"Mon Nov 27 15:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.82299999999999995,"15min":0.83099999999999996},"min":{"1min":0.63800000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.94699999999999995,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.873,"5min":0.89000000000000001,"15min":0.89300000000000002},"min":{"1min":0.70499999999999996,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.1399999999999999,"5min":1.409,"15min":1.409},"last":0.86899999999999999}]},{"osd":11,"last update":"Mon Nov 27 15:20:50 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.82299999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7600,"kb_used_data":1064,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":93737552,"statfs":{"total":95995035648,"available":95987253248,"internally_reserved":0,"allocated":1089536,"data_stored":550483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53000000000000003,"15min":0.50600000000000001},"min":{"1min":0.42699999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.78800000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.33100000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.83799999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.59099999999999997}]},{"osd":1,"last update":"Mon Nov 27 15:21:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.54800000000000004,"15min":0.497},"min":{"1min":0.40799999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.86199999999999999,"5min":1.079,"15min":1.079},"last":0.62},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.55200000000000005,"15min":0.505},"min":{"1min":0.436,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.55400000000000005}]},{"osd":3,"last update":"Mon Nov 27 15:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.60699999999999998,"15min":0.57099999999999995},"min":{"1min":0.48999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.59499999999999997,"15min":0.56899999999999995},"min":{"1min":0.44500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.71999999999999997}]},{"osd":4,"last update":"Mon Nov 27 15:21:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.47899999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.45900000000000002,"15min":0.43099999999999999},"min":{"1min":0.35399999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.48199999999999998,"15min":0.45500000000000002},"min":{"1min":0.32600000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.496}]},{"osd":6,"last update":"Mon Nov 27 15:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.62,"15min":0.60299999999999998},"min":{"1min":0.42099999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.57999999999999996}]},{"osd":7,"last update":"Mon Nov 27 15:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.65500000000000003}]},{"osd":8,"last update":"Mon Nov 27 15:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.376,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85599999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.61599999999999999,"15min":0.60499999999999998},"min":{"1min":0.47199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.88100000000000001,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.88}]},{"osd":9,"last update":"Mon Nov 27 15:21:07 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.67200000000000004,"15min":0.66000000000000003},"min":{"1min":0.50900000000000001,"5min":0.502,"15min":0.502},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.70199999999999996,"15min":0.69099999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.86399999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:21:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.755,"15min":0.749},"min":{"1min":0.51000000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.90600000000000003,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.70999999999999996,"15min":0.68600000000000005},"min":{"1min":0.54600000000000004,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.77500000000000002}]},{"osd":11,"last update":"Mon Nov 27 15:21:33 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.93200000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6760,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738392,"statfs":{"total":95995035648,"available":95988113408,"internally_reserved":0,"allocated":1015808,"data_stored":490326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.42499999999999999,"15min":0.40000000000000002},"min":{"1min":0.33400000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72099999999999997,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.442,"5min":0.45600000000000002,"15min":0.41699999999999998},"min":{"1min":0.29999999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.58399999999999996,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.41499999999999998}]},{"osd":1,"last update":"Mon Nov 27 15:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61099999999999999,"15min":0.56499999999999995},"min":{"1min":0.42999999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.872,"5min":1.056,"15min":1.056},"last":0.621},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63500000000000001,"15min":0.57099999999999995},"min":{"1min":0.40699999999999997,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.90300000000000002,"5min":5.6529999999999996,"15min":5.6529999999999996},"last":0.57499999999999996}]},{"osd":2,"last update":"Mon Nov 27 15:20:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64000000000000001,"15min":0.59399999999999997},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.85099999999999998,"5min":1.208,"15min":1.208},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.63200000000000001,"15min":0.58799999999999997},"min":{"1min":0.49099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78900000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.56100000000000005}]},{"osd":4,"last update":"Mon Nov 27 15:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.64400000000000002,"15min":0.627},"min":{"1min":0.45100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86599999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.71099999999999997,"15min":0.67400000000000004},"min":{"1min":0.57699999999999996,"5min":0.502,"15min":0.502},"max":{"1min":0.93100000000000005,"5min":1.343,"15min":1.343},"last":0.70799999999999996}]},{"osd":5,"last update":"Mon Nov 27 15:21:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68500000000000005,"15min":0.65500000000000003},"min":{"1min":0.49299999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0629999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67700000000000005,"15min":0.65600000000000003},"min":{"1min":0.55000000000000004,"5min":0.437,"15min":0.437},"max":{"1min":0.79100000000000004,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.68600000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.65600000000000003,"15min":0.63800000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0269999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60899999999999999,"15min":0.59899999999999998},"min":{"1min":0.41399999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94399999999999995,"5min":1.018,"15min":1.018},"last":0.47599999999999998}]},{"osd":7,"last update":"Mon Nov 27 15:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.95499999999999996,"15min":0.97599999999999998},"min":{"1min":0.53500000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.93999999999999995,"5min":6.9619999999999997,"15min":6.9619999999999997},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.78000000000000003,"15min":0.77600000000000002},"min":{"1min":0.56599999999999995,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.99099999999999999,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.76400000000000001}]},{"osd":8,"last update":"Mon Nov 27 15:19:38 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.046,"15min":1.046},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":5.6100000000000003,"5min":5.6100000000000003,"15min":5.6100000000000003},"last":0.752},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.77500000000000002}]},{"osd":9,"last update":"Mon Nov 27 15:21:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.84299999999999997,"15min":0.85499999999999998},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1020000000000001,"5min":1.161,"15min":1.161},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.84199999999999997,"15min":0.85199999999999998},"min":{"1min":0.496,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.95699999999999996,"5min":1.496,"15min":1.496},"last":0.79700000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:21:27 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.77700000000000002,"15min":0.78200000000000003},"min":{"1min":0.53400000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.93200000000000005,"5min":1.157,"15min":1.157},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.82899999999999996,"15min":0.83499999999999996},"min":{"1min":0.61899999999999999,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":0.98399999999999999,"5min":1.129,"15min":1.129},"last":0.78800000000000003}]},{"osd":11,"last update":"Mon Nov 27 15:21:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.82199999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6632,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738520,"statfs":{"total":95995035648,"available":95988244480,"internally_reserved":0,"allocated":1015808,"data_stored":495124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.54400000000000004,"15min":0.53700000000000003},"min":{"1min":0.38300000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53500000000000003,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77900000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.54900000000000004}]},{"osd":1,"last update":"Mon Nov 27 15:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.39000000000000001,"15min":0.35399999999999998},"min":{"1min":0.28599999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.495,"5min":0.40300000000000002,"15min":0.36699999999999999},"min":{"1min":0.32200000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.57099999999999995}]},{"osd":2,"last update":"Mon Nov 27 15:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.57699999999999996,"15min":0.55700000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.59099999999999997,"15min":0.54900000000000004},"min":{"1min":0.432,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.68999999999999995}]},{"osd":3,"last update":"Mon Nov 27 15:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57699999999999996,"15min":0.55100000000000005},"min":{"1min":0.41799999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.423,"5min":1.423,"15min":1.423},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.59199999999999997,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.169,"5min":1.722,"15min":1.722},"last":0.71599999999999997}]},{"osd":5,"last update":"Mon Nov 27 15:21:34 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.60799999999999998},"min":{"1min":0.42699999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.76200000000000001,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.629,"15min":0.60599999999999998},"min":{"1min":0.53800000000000003,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.84999999999999998,"5min":1.28,"15min":1.28},"last":0.94299999999999995}]},{"osd":6,"last update":"Mon Nov 27 15:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.45800000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.66900000000000004,"15min":0.65700000000000003},"min":{"1min":0.53700000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.752}]},{"osd":7,"last update":"Mon Nov 27 15:21:22 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.58299999999999996,"15min":0.56599999999999995},"min":{"1min":0.40500000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.014,"5min":1.054,"15min":1.054},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.61599999999999999,"15min":0.59699999999999998},"min":{"1min":0.443,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.874}]},{"osd":8,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71399999999999997,"15min":0.70499999999999996},"min":{"1min":0.50600000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.99099999999999999,"5min":1.522,"15min":1.522},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92700000000000005,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.99099999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:21:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.755,"15min":0.753},"min":{"1min":0.55100000000000005,"5min":0.44,"15min":0.44},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68200000000000005,"15min":0.68100000000000005},"min":{"1min":0.52500000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":1.266}]},{"osd":10,"last update":"Mon Nov 27 15:21:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.56299999999999994,"5min":0.503,"15min":0.503},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.64100000000000001,"15min":0.64500000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.91800000000000004}]},{"osd":11,"last update":"Mon Nov 27 15:21:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.97499999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757305,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7288,"kb_used_data":1584,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93737864,"statfs":{"total":95995035648,"available":95987572736,"internally_reserved":0,"allocated":1622016,"data_stored":1085603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.52000000000000002,"15min":0.52200000000000002},"min":{"1min":0.38500000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67000000000000004,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.52300000000000002,"15min":0.51900000000000002},"min":{"1min":0.374,"5min":0.371,"15min":0.371},"max":{"1min":0.67600000000000005,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.38300000000000001}]},{"osd":1,"last update":"Mon Nov 27 15:21:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56000000000000005,"15min":0.55000000000000004},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.77100000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.56499999999999995,"15min":0.53900000000000003},"min":{"1min":0.38400000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.49099999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:21:35 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41399999999999998,"15min":0.41599999999999998},"min":{"1min":0.32600000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.504,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.621},{"interface":"front","average":{"1min":0.438,"5min":0.42299999999999999,"15min":0.42399999999999999},"min":{"1min":0.33300000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.54700000000000004,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.56899999999999995}]},{"osd":3,"last update":"Mon Nov 27 15:21:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.55400000000000005,"15min":0.56100000000000005},"min":{"1min":0.44600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.74099999999999999,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.44500000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.64500000000000002,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.53000000000000003}]},{"osd":4,"last update":"Mon Nov 27 15:21:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.60999999999999999,"15min":0.59399999999999997},"min":{"1min":0.46999999999999997,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79200000000000004,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.39100000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80400000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.55500000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.625,"15min":0.63500000000000001},"min":{"1min":0.41899999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.44400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.93500000000000005,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.70399999999999996}]},{"osd":7,"last update":"Mon Nov 27 15:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63700000000000001,"15min":0.64600000000000002},"min":{"1min":0.433,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85199999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63700000000000001,"15min":0.63800000000000001},"min":{"1min":0.48599999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83599999999999997,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.67700000000000005}]},{"osd":8,"last update":"Mon Nov 27 15:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60499999999999998,"15min":0.61899999999999999},"min":{"1min":0.44400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71699999999999997,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59499999999999997,"15min":0.60199999999999998},"min":{"1min":0.443,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77500000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.66700000000000004}]},{"osd":9,"last update":"Mon Nov 27 15:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.73099999999999998,"15min":0.747},"min":{"1min":0.497,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.753,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.73099999999999998,"15min":0.73799999999999999},"min":{"1min":0.54000000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.90400000000000003,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.82999999999999996}]},{"osd":10,"last update":"Mon Nov 27 15:21:29 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68000000000000005,"15min":0.67700000000000005},"min":{"1min":0.56000000000000005,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69199999999999995,"15min":0.68600000000000005},"min":{"1min":0.59199999999999997,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.86699999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.91200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-27T15:21:57.302 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-11-27T15:21:57.302 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-27T15:21:57.302 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-11-27T15:21:57.303 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph health --format=json 2023-11-27T15:21:58.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:57 smithi177 ceph-mon[131504]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:21:58.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:57 smithi174 ceph-mon[131110]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:21:58.154 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:21:58.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:57 smithi133 ceph-mon[97860]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:21:59.000 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:58 smithi133 ceph-mon[97860]: pgmap v318: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-11-27T15:21:59.090 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:58 smithi177 ceph-mon[131504]: pgmap v318: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-11-27T15:21:59.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:58 smithi174 ceph-mon[131110]: pgmap v318: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-11-27T15:21:59.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.531+0000 7ffa8e1fd700 1 -- 172.21.15.133:0/2707135719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa88100540 msgr2=0x7ffa88100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:59.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.531+0000 7ffa8e1fd700 1 --2- 172.21.15.133:0/2707135719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa88100540 0x7ffa88100910 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffa780099c0 tx=0x7ffa78009cd0 comp rx=0 tx=0).stop 2023-11-27T15:21:59.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.531+0000 7ffa8e1fd700 1 -- 172.21.15.133:0/2707135719 shutdown_connections 2023-11-27T15:21:59.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.531+0000 7ffa8e1fd700 1 --2- 172.21.15.133:0/2707135719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa881055e0 0x7ffa8810d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:59.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.531+0000 7ffa8e1fd700 1 --2- 172.21.15.133:0/2707135719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa88100540 0x7ffa88100910 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:59.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.531+0000 7ffa8e1fd700 1 --2- 172.21.15.133:0/2707135719 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa88100ee0 0x7ffa88104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:59.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.531+0000 7ffa8e1fd700 1 -- 172.21.15.133:0/2707135719 >> 172.21.15.133:0/2707135719 conn(0x7ffa880fbdc0 msgr2=0x7ffa880fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:59.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.531+0000 7ffa8e1fd700 1 -- 172.21.15.133:0/2707135719 shutdown_connections 2023-11-27T15:21:59.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.531+0000 7ffa8e1fd700 1 -- 172.21.15.133:0/2707135719 wait complete. 2023-11-27T15:21:59.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.532+0000 7ffa8e1fd700 1 Processor -- start 2023-11-27T15:21:59.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.532+0000 7ffa8e1fd700 1 -- start start 2023-11-27T15:21:59.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.532+0000 7ffa8e1fd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa88100540 0x7ffa881986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:59.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.533+0000 7ffa8e1fd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa88100ee0 0x7ffa88198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:59.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.533+0000 7ffa8e1fd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa881055e0 0x7ffa8819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:59.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.533+0000 7ffa8e1fd700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8819d5d0 con 0x7ffa88100540 2023-11-27T15:21:59.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.533+0000 7ffa8e1fd700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8819d740 con 0x7ffa88100ee0 2023-11-27T15:21:59.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.533+0000 7ffa8e1fd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8819d900 con 0x7ffa881055e0 2023-11-27T15:21:59.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.533+0000 7ffa877fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa88100540 0x7ffa881986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:59.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.533+0000 7ffa877fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa88100540 0x7ffa881986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40096/0 (socket says 172.21.15.133:40096) 2023-11-27T15:21:59.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.533+0000 7ffa877fe700 1 -- 172.21.15.133:0/2652599113 learned_addr learned my addr 172.21.15.133:0/2652599113 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:21:59.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.533+0000 7ffa86ffd700 1 --2- 172.21.15.133:0/2652599113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa88100ee0 0x7ffa88198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:59.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.533+0000 7ffa87fff700 1 --2- 172.21.15.133:0/2652599113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa881055e0 0x7ffa8819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:59.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.533+0000 7ffa877fe700 1 -- 172.21.15.133:0/2652599113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa881055e0 msgr2=0x7ffa8819cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:59.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.533+0000 7ffa877fe700 1 --2- 172.21.15.133:0/2652599113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa881055e0 0x7ffa8819cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:59.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.533+0000 7ffa877fe700 1 -- 172.21.15.133:0/2652599113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa88100ee0 msgr2=0x7ffa88198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:59.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.533+0000 7ffa877fe700 1 --2- 172.21.15.133:0/2652599113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa88100ee0 0x7ffa88198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:59.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.533+0000 7ffa877fe700 1 -- 172.21.15.133:0/2652599113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa780095f0 con 0x7ffa88100540 2023-11-27T15:21:59.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.533+0000 7ffa877fe700 1 --2- 172.21.15.133:0/2652599113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa88100540 0x7ffa881986b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800b580 tx=0x7ffa7800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:59.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.534+0000 7ffa84ff9700 1 -- 172.21.15.133:0/2652599113 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7801d070 con 0x7ffa88100540 2023-11-27T15:21:59.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.534+0000 7ffa84ff9700 1 -- 172.21.15.133:0/2652599113 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa78005050 con 0x7ffa88100540 2023-11-27T15:21:59.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.534+0000 7ffa8e1fd700 1 -- 172.21.15.133:0/2652599113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa88110370 con 0x7ffa88100540 2023-11-27T15:21:59.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.534+0000 7ffa84ff9700 1 -- 172.21.15.133:0/2652599113 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa78004a40 con 0x7ffa88100540 2023-11-27T15:21:59.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.534+0000 7ffa8e1fd700 1 -- 172.21.15.133:0/2652599113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa881101e0 con 0x7ffa88100540 2023-11-27T15:21:59.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.535+0000 7ffa8e1fd700 1 -- 172.21.15.133:0/2652599113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa88110500 con 0x7ffa88100540 2023-11-27T15:21:59.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.537+0000 7ffa84ff9700 1 -- 172.21.15.133:0/2652599113 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7ffa780051c0 con 0x7ffa88100540 2023-11-27T15:21:59.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.538+0000 7ffa84ff9700 1 --2- 172.21.15.133:0/2652599113 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ffa700a1260 0x7ffa700a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:21:59.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.538+0000 7ffa84ff9700 1 -- 172.21.15.133:0/2652599113 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ffa780cdbb0 con 0x7ffa88100540 2023-11-27T15:21:59.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.538+0000 7ffa86ffd700 1 --2- 172.21.15.133:0/2652599113 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ffa700a1260 0x7ffa700a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:21:59.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.539+0000 7ffa86ffd700 1 --2- 172.21.15.133:0/2652599113 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ffa700a1260 0x7ffa700a3710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffa88199cd0 tx=0x7ffa7c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:21:59.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.542+0000 7ffa84ff9700 1 -- 172.21.15.133:0/2652599113 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ffa78096ec0 con 0x7ffa88100540 2023-11-27T15:21:59.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.778+0000 7ffa8e1fd700 1 -- 172.21.15.133:0/2652599113 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ffa88066ea0 con 0x7ffa88100540 2023-11-27T15:21:59.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.779+0000 7ffa84ff9700 1 -- 172.21.15.133:0/2652599113 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ffa7809a4e0 con 0x7ffa88100540 2023-11-27T15:21:59.783 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:21:59.783 INFO:teuthology.orchestra.run.smithi133.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-27T15:21:59.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.781+0000 7ffa8e1fd700 1 -- 172.21.15.133:0/2652599113 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ffa700a1260 msgr2=0x7ffa700a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:59.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.781+0000 7ffa8e1fd700 1 --2- 172.21.15.133:0/2652599113 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ffa700a1260 0x7ffa700a3710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffa88199cd0 tx=0x7ffa7c009260 comp rx=0 tx=0).stop 2023-11-27T15:21:59.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.781+0000 7ffa8e1fd700 1 -- 172.21.15.133:0/2652599113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa88100540 msgr2=0x7ffa881986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:21:59.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.781+0000 7ffa8e1fd700 1 --2- 172.21.15.133:0/2652599113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa88100540 0x7ffa881986b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800b580 tx=0x7ffa7800bfd0 comp rx=0 tx=0).stop 2023-11-27T15:21:59.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.781+0000 7ffa8e1fd700 1 -- 172.21.15.133:0/2652599113 shutdown_connections 2023-11-27T15:21:59.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.781+0000 7ffa8e1fd700 1 --2- 172.21.15.133:0/2652599113 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ffa700a1260 0x7ffa700a3710 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:59.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.781+0000 7ffa8e1fd700 1 --2- 172.21.15.133:0/2652599113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffa881055e0 0x7ffa8819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:59.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.781+0000 7ffa8e1fd700 1 --2- 172.21.15.133:0/2652599113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa88100540 0x7ffa881986b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:59.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.781+0000 7ffa8e1fd700 1 --2- 172.21.15.133:0/2652599113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa88100ee0 0x7ffa88198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:21:59.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.781+0000 7ffa8e1fd700 1 -- 172.21.15.133:0/2652599113 >> 172.21.15.133:0/2652599113 conn(0x7ffa880fbdc0 msgr2=0x7ffa880fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:21:59.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.781+0000 7ffa8e1fd700 1 -- 172.21.15.133:0/2652599113 shutdown_connections 2023-11-27T15:21:59.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:21:59.781+0000 7ffa8e1fd700 1 -- 172.21.15.133:0/2652599113 wait complete. 2023-11-27T15:22:00.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:21:59 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2652599113' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-11-27T15:22:00.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:21:59 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2652599113' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-11-27T15:22:00.164 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:21:59 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2652599113' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-11-27T15:22:00.248 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-11-27T15:22:00.248 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:22:00.450 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.446+0000 7f22475ff700 1 -- 172.21.15.133:0/1308573975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f224010c860 msgr2=0x7f224010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:00.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.446+0000 7f22475ff700 1 --2- 172.21.15.133:0/1308573975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f224010c860 0x7f224010ec40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f223800b0a0 tx=0x7f223800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:00.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.447+0000 7f22475ff700 1 -- 172.21.15.133:0/1308573975 shutdown_connections 2023-11-27T15:22:00.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.447+0000 7f22475ff700 1 --2- 172.21.15.133:0/1308573975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f224010c860 0x7f224010ec40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:00.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.447+0000 7f22475ff700 1 --2- 172.21.15.133:0/1308573975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2240074b30 0x7f2240072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:00.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.447+0000 7f22475ff700 1 --2- 172.21.15.133:0/1308573975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2240073550 0x7f22400739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:00.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.447+0000 7f22475ff700 1 -- 172.21.15.133:0/1308573975 >> 172.21.15.133:0/1308573975 conn(0x7f22400fb870 msgr2=0x7f22400fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:00.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.447+0000 7f22475ff700 1 -- 172.21.15.133:0/1308573975 shutdown_connections 2023-11-27T15:22:00.451 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.447+0000 7f22475ff700 1 -- 172.21.15.133:0/1308573975 wait complete. 2023-11-27T15:22:00.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.448+0000 7f22475ff700 1 Processor -- start 2023-11-27T15:22:00.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.448+0000 7f22475ff700 1 -- start start 2023-11-27T15:22:00.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.449+0000 7f22475ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2240073550 0x7f224019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:00.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.449+0000 7f22475ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2240074b30 0x7f224019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:00.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.449+0000 7f22475ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f224010c860 0x7f22401a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:00.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.449+0000 7f22475ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22401a4220 con 0x7f2240074b30 2023-11-27T15:22:00.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.449+0000 7f22475ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22401a4360 con 0x7f2240073550 2023-11-27T15:22:00.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.449+0000 7f22475ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22401a44f0 con 0x7f224010c860 2023-11-27T15:22:00.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.449+0000 7f2244b9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2240074b30 0x7f224019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:00.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.449+0000 7f2244b9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2240074b30 0x7f224019cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40120/0 (socket says 172.21.15.133:40120) 2023-11-27T15:22:00.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.449+0000 7f2244b9a700 1 -- 172.21.15.133:0/4039073851 learned_addr learned my addr 172.21.15.133:0/4039073851 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:00.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.449+0000 7f224539b700 1 --2- 172.21.15.133:0/4039073851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2240073550 0x7f224019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:00.454 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.449+0000 7f2245b9c700 1 --2- 172.21.15.133:0/4039073851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f224010c860 0x7f22401a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:00.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.449+0000 7f2244b9a700 1 -- 172.21.15.133:0/4039073851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f224010c860 msgr2=0x7f22401a3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:00.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.449+0000 7f2244b9a700 1 --2- 172.21.15.133:0/4039073851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f224010c860 0x7f22401a3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:00.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.449+0000 7f2244b9a700 1 -- 172.21.15.133:0/4039073851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2240073550 msgr2=0x7f224019c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:00.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.449+0000 7f2244b9a700 1 --2- 172.21.15.133:0/4039073851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2240073550 0x7f224019c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:00.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.449+0000 7f2244b9a700 1 -- 172.21.15.133:0/4039073851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2238009d00 con 0x7f2240074b30 2023-11-27T15:22:00.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.449+0000 7f2245b9c700 1 --2- 172.21.15.133:0/4039073851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f224010c860 0x7f22401a3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:22:00.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.450+0000 7f2244b9a700 1 --2- 172.21.15.133:0/4039073851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2240074b30 0x7f224019cbe0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f223000e970 tx=0x7f223000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:00.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.450+0000 7f22367fc700 1 -- 172.21.15.133:0/4039073851 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230009790 con 0x7f2240074b30 2023-11-27T15:22:00.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.450+0000 7f22367fc700 1 -- 172.21.15.133:0/4039073851 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f223000eec0 con 0x7f2240074b30 2023-11-27T15:22:00.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.450+0000 7f22367fc700 1 -- 172.21.15.133:0/4039073851 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2230003cf0 con 0x7f2240074b30 2023-11-27T15:22:00.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.450+0000 7f22475ff700 1 -- 172.21.15.133:0/4039073851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2240112270 con 0x7f2240074b30 2023-11-27T15:22:00.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.450+0000 7f22475ff700 1 -- 172.21.15.133:0/4039073851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22401120e0 con 0x7f2240074b30 2023-11-27T15:22:00.456 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.451+0000 7f22475ff700 1 -- 172.21.15.133:0/4039073851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22401023b0 con 0x7f2240074b30 2023-11-27T15:22:00.456 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.453+0000 7f22367fc700 1 -- 172.21.15.133:0/4039073851 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f223000ccd0 con 0x7f2240074b30 2023-11-27T15:22:00.458 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.454+0000 7f22367fc700 1 --2- 172.21.15.133:0/4039073851 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f222c0a11e0 0x7f222c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:00.458 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.454+0000 7f22367fc700 1 -- 172.21.15.133:0/4039073851 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2230014070 con 0x7f2240074b30 2023-11-27T15:22:00.458 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.454+0000 7f224539b700 1 --2- 172.21.15.133:0/4039073851 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f222c0a11e0 0x7f222c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:00.458 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.455+0000 7f224539b700 1 --2- 172.21.15.133:0/4039073851 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f222c0a11e0 0x7f222c0a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2240074f00 tx=0x7f223c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:00.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.458+0000 7f22367fc700 1 -- 172.21.15.133:0/4039073851 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f22300962e0 con 0x7f2240074b30 2023-11-27T15:22:00.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.660+0000 7f22475ff700 1 -- 172.21.15.133:0/4039073851 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f224002cfe0 con 0x7f2240074b30 2023-11-27T15:22:00.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.661+0000 7f22367fc700 1 -- 172.21.15.133:0/4039073851 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+24465 (secure 0 0 0) 0x7f2230099900 con 0x7f2240074b30 2023-11-27T15:22:00.666 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:00.666 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":90,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:21:02.774978+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:22:00.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.664+0000 7f22475ff700 1 -- 172.21.15.133:0/4039073851 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f222c0a11e0 msgr2=0x7f222c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:00.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.664+0000 7f22475ff700 1 --2- 172.21.15.133:0/4039073851 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f222c0a11e0 0x7f222c0a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2240074f00 tx=0x7f223c009260 comp rx=0 tx=0).stop 2023-11-27T15:22:00.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.664+0000 7f22475ff700 1 -- 172.21.15.133:0/4039073851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2240074b30 msgr2=0x7f224019cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:00.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.664+0000 7f22475ff700 1 --2- 172.21.15.133:0/4039073851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2240074b30 0x7f224019cbe0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f223000e970 tx=0x7f223000ec80 comp rx=0 tx=0).stop 2023-11-27T15:22:00.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.664+0000 7f22475ff700 1 -- 172.21.15.133:0/4039073851 shutdown_connections 2023-11-27T15:22:00.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.664+0000 7f22475ff700 1 --2- 172.21.15.133:0/4039073851 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f222c0a11e0 0x7f222c0a3690 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:00.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.664+0000 7f22475ff700 1 --2- 172.21.15.133:0/4039073851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f224010c860 0x7f22401a3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:00.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.664+0000 7f22475ff700 1 --2- 172.21.15.133:0/4039073851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2240073550 0x7f224019c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:00.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.664+0000 7f22475ff700 1 --2- 172.21.15.133:0/4039073851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2240074b30 0x7f224019cbe0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:00.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.664+0000 7f22475ff700 1 -- 172.21.15.133:0/4039073851 >> 172.21.15.133:0/4039073851 conn(0x7f22400fb870 msgr2=0x7f22400fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:00.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.664+0000 7f22475ff700 1 -- 172.21.15.133:0/4039073851 shutdown_connections 2023-11-27T15:22:00.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.664+0000 7f22475ff700 1 -- 172.21.15.133:0/4039073851 wait complete. 2023-11-27T15:22:00.688 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:16:20.216150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:20:31.107146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:20:32.614089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-27T15:20:36.126040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-27T15:22:00.688 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:22:00.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.888+0000 7fe604dff700 1 -- 172.21.15.133:0/2013266460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe600101e50 msgr2=0x7fe600108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:00.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.888+0000 7fe604dff700 1 --2- 172.21.15.133:0/2013266460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe600101e50 0x7fe600108710 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800b0a0 tx=0x7fe5f800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:00.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.888+0000 7fe604dff700 1 -- 172.21.15.133:0/2013266460 shutdown_connections 2023-11-27T15:22:00.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.888+0000 7fe604dff700 1 --2- 172.21.15.133:0/2013266460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe600101e50 0x7fe600108710 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:00.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.888+0000 7fe604dff700 1 --2- 172.21.15.133:0/2013266460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6001002d0 0x7fe6001006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:00.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.888+0000 7fe604dff700 1 --2- 172.21.15.133:0/2013266460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6001014c0 0x7fe600101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:00.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.888+0000 7fe604dff700 1 -- 172.21.15.133:0/2013266460 >> 172.21.15.133:0/2013266460 conn(0x7fe6000fb8b0 msgr2=0x7fe6000fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:00.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.888+0000 7fe604dff700 1 -- 172.21.15.133:0/2013266460 shutdown_connections 2023-11-27T15:22:00.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.888+0000 7fe604dff700 1 -- 172.21.15.133:0/2013266460 wait complete. 2023-11-27T15:22:00.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.889+0000 7fe604dff700 1 Processor -- start 2023-11-27T15:22:00.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.889+0000 7fe604dff700 1 -- start start 2023-11-27T15:22:00.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.889+0000 7fe604dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6001002d0 0x7fe600198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:00.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.889+0000 7fe604dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6001014c0 0x7fe6001988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:00.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.889+0000 7fe604dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe600101e50 0x7fe60019f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:00.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.889+0000 7fe604dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60019fee0 con 0x7fe6001014c0 2023-11-27T15:22:00.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.889+0000 7fe604dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6001a0020 con 0x7fe600101e50 2023-11-27T15:22:00.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.889+0000 7fe604dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6001a01b0 con 0x7fe6001002d0 2023-11-27T15:22:00.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.890+0000 7fe5fdd9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6001014c0 0x7fe6001988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:00.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.890+0000 7fe5fed9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe600101e50 0x7fe60019f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:00.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.890+0000 7fe5fdd9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6001014c0 0x7fe6001988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40144/0 (socket says 172.21.15.133:40144) 2023-11-27T15:22:00.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.890+0000 7fe5fed9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe600101e50 0x7fe60019f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:59964/0 (socket says 172.21.15.133:59964) 2023-11-27T15:22:00.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.890+0000 7fe5fdd9b700 1 -- 172.21.15.133:0/3996672007 learned_addr learned my addr 172.21.15.133:0/3996672007 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:00.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.890+0000 7fe5fe59c700 1 --2- 172.21.15.133:0/3996672007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6001002d0 0x7fe600198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:00.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.890+0000 7fe5fed9d700 1 -- 172.21.15.133:0/3996672007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6001002d0 msgr2=0x7fe600198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:00.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.890+0000 7fe5fed9d700 1 --2- 172.21.15.133:0/3996672007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6001002d0 0x7fe600198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:00.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.890+0000 7fe5fed9d700 1 -- 172.21.15.133:0/3996672007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6001014c0 msgr2=0x7fe6001988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:00.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.890+0000 7fe5fed9d700 1 --2- 172.21.15.133:0/3996672007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6001014c0 0x7fe6001988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:00.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.890+0000 7fe5fed9d700 1 -- 172.21.15.133:0/3996672007 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f8009d00 con 0x7fe600101e50 2023-11-27T15:22:00.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.890+0000 7fe5fed9d700 1 --2- 172.21.15.133:0/3996672007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe600101e50 0x7fe60019f910 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800b070 tx=0x7fe5f800bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:00.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.891+0000 7fe5ef7fe700 1 -- 172.21.15.133:0/3996672007 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f800e040 con 0x7fe600101e50 2023-11-27T15:22:00.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.891+0000 7fe604dff700 1 -- 172.21.15.133:0/3996672007 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe60010bca0 con 0x7fe600101e50 2023-11-27T15:22:00.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.891+0000 7fe5ef7fe700 1 -- 172.21.15.133:0/3996672007 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5f8003e10 con 0x7fe600101e50 2023-11-27T15:22:00.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.891+0000 7fe5ef7fe700 1 -- 172.21.15.133:0/3996672007 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f80052a0 con 0x7fe600101e50 2023-11-27T15:22:00.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.891+0000 7fe604dff700 1 -- 172.21.15.133:0/3996672007 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60010bb10 con 0x7fe600101e50 2023-11-27T15:22:00.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.892+0000 7fe604dff700 1 -- 172.21.15.133:0/3996672007 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe600067b00 con 0x7fe600101e50 2023-11-27T15:22:00.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.894+0000 7fe5ef7fe700 1 -- 172.21.15.133:0/3996672007 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fe5f801d480 con 0x7fe600101e50 2023-11-27T15:22:00.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.895+0000 7fe5ef7fe700 1 --2- 172.21.15.133:0/3996672007 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe5e80a11e0 0x7fe5e80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:00.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.895+0000 7fe5ef7fe700 1 -- 172.21.15.133:0/3996672007 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe5f80ce220 con 0x7fe600101e50 2023-11-27T15:22:00.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.895+0000 7fe5fe59c700 1 --2- 172.21.15.133:0/3996672007 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe5e80a11e0 0x7fe5e80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:00.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.896+0000 7fe5fe59c700 1 --2- 172.21.15.133:0/3996672007 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe5e80a11e0 0x7fe5e80a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe600101320 tx=0x7fe5f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:00.903 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:00.899+0000 7fe5ef7fe700 1 -- 172.21.15.133:0/3996672007 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe5f8097530 con 0x7fe600101e50 2023-11-27T15:22:01.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:00 smithi133 ceph-mon[97860]: pgmap v319: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:22:01.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:00 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/4039073851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:01.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:00 smithi177 ceph-mon[131504]: pgmap v319: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:22:01.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:00 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/4039073851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:01.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.097+0000 7fe604dff700 1 -- 172.21.15.133:0/3996672007 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6001a0cf0 con 0x7fe600101e50 2023-11-27T15:22:01.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.098+0000 7fe5ef7fe700 1 -- 172.21.15.133:0/3996672007 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5f809ab50 con 0x7fe600101e50 2023-11-27T15:22:01.102 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-11-27T15:22:01.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.100+0000 7fe604dff700 1 -- 172.21.15.133:0/3996672007 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe5e80a11e0 msgr2=0x7fe5e80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:01.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.100+0000 7fe604dff700 1 --2- 172.21.15.133:0/3996672007 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe5e80a11e0 0x7fe5e80a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe600101320 tx=0x7fe5f0009260 comp rx=0 tx=0).stop 2023-11-27T15:22:01.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.100+0000 7fe604dff700 1 -- 172.21.15.133:0/3996672007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe600101e50 msgr2=0x7fe60019f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:01.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.100+0000 7fe604dff700 1 --2- 172.21.15.133:0/3996672007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe600101e50 0x7fe60019f910 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800b070 tx=0x7fe5f800bc80 comp rx=0 tx=0).stop 2023-11-27T15:22:01.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.100+0000 7fe604dff700 1 -- 172.21.15.133:0/3996672007 shutdown_connections 2023-11-27T15:22:01.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.100+0000 7fe604dff700 1 --2- 172.21.15.133:0/3996672007 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe5e80a11e0 0x7fe5e80a3690 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.100+0000 7fe604dff700 1 --2- 172.21.15.133:0/3996672007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe600101e50 0x7fe60019f910 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.100+0000 7fe604dff700 1 --2- 172.21.15.133:0/3996672007 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6001002d0 0x7fe600198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.100+0000 7fe604dff700 1 --2- 172.21.15.133:0/3996672007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6001014c0 0x7fe6001988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.100+0000 7fe604dff700 1 -- 172.21.15.133:0/3996672007 >> 172.21.15.133:0/3996672007 conn(0x7fe6000fb8b0 msgr2=0x7fe6000fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:01.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.100+0000 7fe604dff700 1 -- 172.21.15.133:0/3996672007 shutdown_connections 2023-11-27T15:22:01.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.100+0000 7fe604dff700 1 -- 172.21.15.133:0/3996672007 wait complete. 2023-11-27T15:22:01.124 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:22:01.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:00 smithi174 ceph-mon[131110]: pgmap v319: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:22:01.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:00 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/4039073851' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:01.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.325+0000 7fd955bff700 1 -- 172.21.15.133:0/2402286276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd950074b30 msgr2=0x7fd950072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:01.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.325+0000 7fd955bff700 1 --2- 172.21.15.133:0/2402286276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd950074b30 0x7fd950072f80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd940009990 tx=0x7fd940009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.325+0000 7fd955bff700 1 -- 172.21.15.133:0/2402286276 shutdown_connections 2023-11-27T15:22:01.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.325+0000 7fd955bff700 1 --2- 172.21.15.133:0/2402286276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd950108180 0x7fd95010a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.325+0000 7fd955bff700 1 --2- 172.21.15.133:0/2402286276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd950074b30 0x7fd950072f80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.325+0000 7fd955bff700 1 --2- 172.21.15.133:0/2402286276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd950073550 0x7fd9500739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.325+0000 7fd955bff700 1 -- 172.21.15.133:0/2402286276 >> 172.21.15.133:0/2402286276 conn(0x7fd9500fb8a0 msgr2=0x7fd9500fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:01.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.325+0000 7fd955bff700 1 -- 172.21.15.133:0/2402286276 shutdown_connections 2023-11-27T15:22:01.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.325+0000 7fd955bff700 1 -- 172.21.15.133:0/2402286276 wait complete. 2023-11-27T15:22:01.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.326+0000 7fd955bff700 1 Processor -- start 2023-11-27T15:22:01.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.326+0000 7fd955bff700 1 -- start start 2023-11-27T15:22:01.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.326+0000 7fd955bff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd950073550 0x7fd9501982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:01.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.326+0000 7fd955bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd950074b30 0x7fd950198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:01.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.326+0000 7fd955bff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd950108180 0x7fd95019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:01.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.326+0000 7fd955bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95019fe40 con 0x7fd950074b30 2023-11-27T15:22:01.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.326+0000 7fd955bff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95019ff80 con 0x7fd950108180 2023-11-27T15:22:01.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.326+0000 7fd955bff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9501a0110 con 0x7fd950073550 2023-11-27T15:22:01.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.327+0000 7fd94effd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd950074b30 0x7fd950198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:01.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.327+0000 7fd94effd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd950074b30 0x7fd950198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40152/0 (socket says 172.21.15.133:40152) 2023-11-27T15:22:01.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.327+0000 7fd94effd700 1 -- 172.21.15.133:0/2727364314 learned_addr learned my addr 172.21.15.133:0/2727364314 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:01.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.327+0000 7fd94f7fe700 1 --2- 172.21.15.133:0/2727364314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd950073550 0x7fd9501982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:01.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.327+0000 7fd94ffff700 1 --2- 172.21.15.133:0/2727364314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd950108180 0x7fd95019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:01.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.327+0000 7fd94effd700 1 -- 172.21.15.133:0/2727364314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd950073550 msgr2=0x7fd9501982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:01.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.327+0000 7fd94effd700 1 --2- 172.21.15.133:0/2727364314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd950073550 0x7fd9501982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.327+0000 7fd94effd700 1 -- 172.21.15.133:0/2727364314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd950108180 msgr2=0x7fd95019f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:01.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.327+0000 7fd94effd700 1 --2- 172.21.15.133:0/2727364314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd950108180 0x7fd95019f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.327+0000 7fd94effd700 1 -- 172.21.15.133:0/2727364314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9400095f0 con 0x7fd950074b30 2023-11-27T15:22:01.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.327+0000 7fd94effd700 1 --2- 172.21.15.133:0/2727364314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd950074b30 0x7fd950198800 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd94400e970 tx=0x7fd94400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:01.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.327+0000 7fd94cff9700 1 -- 172.21.15.133:0/2727364314 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944009790 con 0x7fd950074b30 2023-11-27T15:22:01.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.327+0000 7fd94cff9700 1 -- 172.21.15.133:0/2727364314 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd94400c7e0 con 0x7fd950074b30 2023-11-27T15:22:01.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.327+0000 7fd955bff700 1 -- 172.21.15.133:0/2727364314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd95010de80 con 0x7fd950074b30 2023-11-27T15:22:01.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.327+0000 7fd94cff9700 1 -- 172.21.15.133:0/2727364314 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd944003c70 con 0x7fd950074b30 2023-11-27T15:22:01.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.328+0000 7fd955bff700 1 -- 172.21.15.133:0/2727364314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd95010dcf0 con 0x7fd950074b30 2023-11-27T15:22:01.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.328+0000 7fd955bff700 1 -- 172.21.15.133:0/2727364314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd950106830 con 0x7fd950074b30 2023-11-27T15:22:01.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.330+0000 7fd94cff9700 1 -- 172.21.15.133:0/2727364314 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fd94400cc50 con 0x7fd950074b30 2023-11-27T15:22:01.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.332+0000 7fd94cff9700 1 --2- 172.21.15.133:0/2727364314 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9380a1260 0x7fd9380a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:01.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.332+0000 7fd94cff9700 1 -- 172.21.15.133:0/2727364314 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd944014070 con 0x7fd950074b30 2023-11-27T15:22:01.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.332+0000 7fd94f7fe700 1 --2- 172.21.15.133:0/2727364314 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9380a1260 0x7fd9380a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:01.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.333+0000 7fd94f7fe700 1 --2- 172.21.15.133:0/2727364314 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9380a1260 0x7fd9380a3710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd94000b2e0 tx=0x7fd940005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:01.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.334+0000 7fd94cff9700 1 -- 172.21.15.133:0/2727364314 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd944096310 con 0x7fd950074b30 2023-11-27T15:22:01.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.532+0000 7fd955bff700 1 -- 172.21.15.133:0/2727364314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd950061700 con 0x7fd950074b30 2023-11-27T15:22:01.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.533+0000 7fd94cff9700 1 -- 172.21.15.133:0/2727364314 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd944099930 con 0x7fd950074b30 2023-11-27T15:22:01.538 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:22:01.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.535+0000 7fd955bff700 1 -- 172.21.15.133:0/2727364314 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9380a1260 msgr2=0x7fd9380a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:01.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.535+0000 7fd955bff700 1 --2- 172.21.15.133:0/2727364314 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9380a1260 0x7fd9380a3710 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd94000b2e0 tx=0x7fd940005fb0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.535+0000 7fd955bff700 1 -- 172.21.15.133:0/2727364314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd950074b30 msgr2=0x7fd950198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:01.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.535+0000 7fd955bff700 1 --2- 172.21.15.133:0/2727364314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd950074b30 0x7fd950198800 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd94400e970 tx=0x7fd94400ec80 comp rx=0 tx=0).stop 2023-11-27T15:22:01.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.535+0000 7fd955bff700 1 -- 172.21.15.133:0/2727364314 shutdown_connections 2023-11-27T15:22:01.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.535+0000 7fd955bff700 1 --2- 172.21.15.133:0/2727364314 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9380a1260 0x7fd9380a3710 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.535+0000 7fd955bff700 1 --2- 172.21.15.133:0/2727364314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd950108180 0x7fd95019f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.535+0000 7fd955bff700 1 --2- 172.21.15.133:0/2727364314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd950073550 0x7fd9501982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.535+0000 7fd955bff700 1 --2- 172.21.15.133:0/2727364314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd950074b30 0x7fd950198800 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.535+0000 7fd955bff700 1 -- 172.21.15.133:0/2727364314 >> 172.21.15.133:0/2727364314 conn(0x7fd9500fb8a0 msgr2=0x7fd9500fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:01.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.535+0000 7fd955bff700 1 -- 172.21.15.133:0/2727364314 shutdown_connections 2023-11-27T15:22:01.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.535+0000 7fd955bff700 1 -- 172.21.15.133:0/2727364314 wait complete. 2023-11-27T15:22:01.559 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:22:01.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.760+0000 7f79471ff700 1 -- 172.21.15.133:0/718283442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7940074b30 msgr2=0x7f7940072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:01.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.760+0000 7f79471ff700 1 --2- 172.21.15.133:0/718283442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7940074b30 0x7f7940072f80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7934009990 tx=0x7f7934009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.760+0000 7f79471ff700 1 -- 172.21.15.133:0/718283442 shutdown_connections 2023-11-27T15:22:01.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.760+0000 7f79471ff700 1 --2- 172.21.15.133:0/718283442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7940108190 0x7f794010a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.760+0000 7f79471ff700 1 --2- 172.21.15.133:0/718283442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7940074b30 0x7f7940072f80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.760+0000 7f79471ff700 1 --2- 172.21.15.133:0/718283442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7940073550 0x7f79400739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.760+0000 7f79471ff700 1 -- 172.21.15.133:0/718283442 >> 172.21.15.133:0/718283442 conn(0x7f79400fb8b0 msgr2=0x7f79400fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:01.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.760+0000 7f79471ff700 1 -- 172.21.15.133:0/718283442 shutdown_connections 2023-11-27T15:22:01.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.760+0000 7f79471ff700 1 -- 172.21.15.133:0/718283442 wait complete. 2023-11-27T15:22:01.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.761+0000 7f79471ff700 1 Processor -- start 2023-11-27T15:22:01.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.761+0000 7f79471ff700 1 -- start start 2023-11-27T15:22:01.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.761+0000 7f79471ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7940073550 0x7f79401982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:01.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.761+0000 7f79471ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7940074b30 0x7f7940198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:01.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.761+0000 7f79471ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7940108190 0x7f794019f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:01.770 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.762+0000 7f79471ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794019fe50 con 0x7f7940073550 2023-11-27T15:22:01.770 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.762+0000 7f79471ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794019ff90 con 0x7f7940074b30 2023-11-27T15:22:01.770 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.762+0000 7f79471ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79401a0120 con 0x7f7940108190 2023-11-27T15:22:01.770 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.762+0000 7f7944f9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7940073550 0x7f79401982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:01.770 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.762+0000 7f7944f9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7940073550 0x7f79401982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40182/0 (socket says 172.21.15.133:40182) 2023-11-27T15:22:01.770 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.762+0000 7f7944f9b700 1 -- 172.21.15.133:0/3955108375 learned_addr learned my addr 172.21.15.133:0/3955108375 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:01.770 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.762+0000 7f793ffff700 1 --2- 172.21.15.133:0/3955108375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7940074b30 0x7f7940198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:01.770 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.762+0000 7f794579c700 1 --2- 172.21.15.133:0/3955108375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7940108190 0x7f794019f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:01.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.762+0000 7f7944f9b700 1 -- 172.21.15.133:0/3955108375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7940108190 msgr2=0x7f794019f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:01.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.762+0000 7f7944f9b700 1 --2- 172.21.15.133:0/3955108375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7940108190 0x7f794019f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.762+0000 7f7944f9b700 1 -- 172.21.15.133:0/3955108375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7940074b30 msgr2=0x7f7940198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:01.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.762+0000 7f7944f9b700 1 --2- 172.21.15.133:0/3955108375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7940074b30 0x7f7940198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.762+0000 7f7944f9b700 1 -- 172.21.15.133:0/3955108375 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79340095f0 con 0x7f7940073550 2023-11-27T15:22:01.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.762+0000 7f794579c700 1 --2- 172.21.15.133:0/3955108375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7940108190 0x7f794019f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:22:01.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.762+0000 7f7944f9b700 1 --2- 172.21.15.133:0/3955108375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7940073550 0x7f79401982d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7934004710 tx=0x7f7934009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:01.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.763+0000 7f793dffb700 1 -- 172.21.15.133:0/3955108375 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793401d070 con 0x7f7940073550 2023-11-27T15:22:01.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.763+0000 7f79471ff700 1 -- 172.21.15.133:0/3955108375 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f794010de90 con 0x7f7940073550 2023-11-27T15:22:01.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.763+0000 7f793dffb700 1 -- 172.21.15.133:0/3955108375 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7934021410 con 0x7f7940073550 2023-11-27T15:22:01.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.763+0000 7f79471ff700 1 -- 172.21.15.133:0/3955108375 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f794010dd00 con 0x7f7940073550 2023-11-27T15:22:01.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.763+0000 7f793dffb700 1 -- 172.21.15.133:0/3955108375 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79340175f0 con 0x7f7940073550 2023-11-27T15:22:01.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.764+0000 7f79471ff700 1 -- 172.21.15.133:0/3955108375 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7940106840 con 0x7f7940073550 2023-11-27T15:22:01.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.766+0000 7f793dffb700 1 -- 172.21.15.133:0/3955108375 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f7934021580 con 0x7f7940073550 2023-11-27T15:22:01.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.768+0000 7f793dffb700 1 --2- 172.21.15.133:0/3955108375 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f79300a11e0 0x7f79300a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:01.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.768+0000 7f793dffb700 1 -- 172.21.15.133:0/3955108375 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f79340cdff0 con 0x7f7940073550 2023-11-27T15:22:01.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.768+0000 7f793ffff700 1 --2- 172.21.15.133:0/3955108375 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f79300a11e0 0x7f79300a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:01.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.769+0000 7f793ffff700 1 --2- 172.21.15.133:0/3955108375 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f79300a11e0 0x7f79300a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7940074850 tx=0x7f792c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:01.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.772+0000 7f793dffb700 1 -- 172.21.15.133:0/3955108375 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7934097320 con 0x7f7940073550 2023-11-27T15:22:01.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.977+0000 7f79471ff700 1 -- 172.21.15.133:0/3955108375 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7940061700 con 0x7f7940073550 2023-11-27T15:22:01.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.978+0000 7f793dffb700 1 -- 172.21.15.133:0/3955108375 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f793409a940 con 0x7f7940073550 2023-11-27T15:22:01.983 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-11-27T15:22:01.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.980+0000 7f79471ff700 1 -- 172.21.15.133:0/3955108375 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f79300a11e0 msgr2=0x7f79300a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:01.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.980+0000 7f79471ff700 1 --2- 172.21.15.133:0/3955108375 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f79300a11e0 0x7f79300a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7940074850 tx=0x7f792c009260 comp rx=0 tx=0).stop 2023-11-27T15:22:01.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.980+0000 7f79471ff700 1 -- 172.21.15.133:0/3955108375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7940073550 msgr2=0x7f79401982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:01.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.980+0000 7f79471ff700 1 --2- 172.21.15.133:0/3955108375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7940073550 0x7f79401982d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7934004710 tx=0x7f7934009580 comp rx=0 tx=0).stop 2023-11-27T15:22:01.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.980+0000 7f79471ff700 1 -- 172.21.15.133:0/3955108375 shutdown_connections 2023-11-27T15:22:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.980+0000 7f79471ff700 1 --2- 172.21.15.133:0/3955108375 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f79300a11e0 0x7f79300a3690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.980+0000 7f79471ff700 1 --2- 172.21.15.133:0/3955108375 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7940108190 0x7f794019f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.980+0000 7f79471ff700 1 --2- 172.21.15.133:0/3955108375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7940073550 0x7f79401982d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.980+0000 7f79471ff700 1 --2- 172.21.15.133:0/3955108375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7940074b30 0x7f7940198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:01.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.980+0000 7f79471ff700 1 -- 172.21.15.133:0/3955108375 >> 172.21.15.133:0/3955108375 conn(0x7f79400fb8b0 msgr2=0x7f79400fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:01.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.980+0000 7f79471ff700 1 -- 172.21.15.133:0/3955108375 shutdown_connections 2023-11-27T15:22:01.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:01.980+0000 7f79471ff700 1 -- 172.21.15.133:0/3955108375 wait complete. 2023-11-27T15:22:02.005 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-27T15:22:02.065 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:01 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3996672007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:02.065 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:01 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2727364314' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:22:02.065 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-11-27T15:22:02.066 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:01 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-11-27T15:22:02.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:01 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3996672007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:02.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:01 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2727364314' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:22:02.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-11-27T15:22:02.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:01 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-11-27T15:22:02.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:01 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3996672007' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:02.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:01 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2727364314' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:22:02.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-11-27T15:22:02.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:01 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-11-27T15:22:02.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.201+0000 7f542bdff700 1 -- 172.21.15.133:0/693907636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5424103660 msgr2=0x7f5424105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:02.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.201+0000 7f542bdff700 1 --2- 172.21.15.133:0/693907636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5424103660 0x7f5424105a40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5414009990 tx=0x7f5414009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:02.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.202+0000 7f542bdff700 1 -- 172.21.15.133:0/693907636 shutdown_connections 2023-11-27T15:22:02.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.202+0000 7f542bdff700 1 --2- 172.21.15.133:0/693907636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5424105f80 0x7f54241083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:02.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.202+0000 7f542bdff700 1 --2- 172.21.15.133:0/693907636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5424100d40 0x7f5424103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:02.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.202+0000 7f542bdff700 1 --2- 172.21.15.133:0/693907636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5424103660 0x7f5424105a40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:02.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.202+0000 7f542bdff700 1 -- 172.21.15.133:0/693907636 >> 172.21.15.133:0/693907636 conn(0x7f54240fa7c0 msgr2=0x7f54240fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:02.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.202+0000 7f542bdff700 1 -- 172.21.15.133:0/693907636 shutdown_connections 2023-11-27T15:22:02.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.202+0000 7f542bdff700 1 -- 172.21.15.133:0/693907636 wait complete. 2023-11-27T15:22:02.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.203+0000 7f542bdff700 1 Processor -- start 2023-11-27T15:22:02.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.203+0000 7f542bdff700 1 -- start start 2023-11-27T15:22:02.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.204+0000 7f542bdff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5424100d40 0x7f54241006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:02.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.204+0000 7f542bdff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5424103660 0x7f54240fecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:02.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.204+0000 7f542bdff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5424105f80 0x7f54240ff230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:02.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.204+0000 7f542bdff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54240ff9d0 con 0x7f5424105f80 2023-11-27T15:22:02.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.204+0000 7f542bdff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54240ffb40 con 0x7f5424100d40 2023-11-27T15:22:02.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.204+0000 7f542bdff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54241a0100 con 0x7f5424103660 2023-11-27T15:22:02.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.204+0000 7f542a39c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5424105f80 0x7f54240ff230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:02.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.204+0000 7f542a39c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5424105f80 0x7f54240ff230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40212/0 (socket says 172.21.15.133:40212) 2023-11-27T15:22:02.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.204+0000 7f542a39c700 1 -- 172.21.15.133:0/681440835 learned_addr learned my addr 172.21.15.133:0/681440835 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:02.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.204+0000 7f542939a700 1 --2- 172.21.15.133:0/681440835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5424103660 0x7f54240fecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:02.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.204+0000 7f5429b9b700 1 --2- 172.21.15.133:0/681440835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5424100d40 0x7f54241006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:02.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.204+0000 7f542a39c700 1 -- 172.21.15.133:0/681440835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5424103660 msgr2=0x7f54240fecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:02.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.204+0000 7f542a39c700 1 --2- 172.21.15.133:0/681440835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5424103660 0x7f54240fecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:02.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.204+0000 7f542a39c700 1 -- 172.21.15.133:0/681440835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5424100d40 msgr2=0x7f54241006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:02.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.204+0000 7f542a39c700 1 --2- 172.21.15.133:0/681440835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5424100d40 0x7f54241006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:02.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.204+0000 7f542a39c700 1 -- 172.21.15.133:0/681440835 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54140095f0 con 0x7f5424105f80 2023-11-27T15:22:02.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.205+0000 7f542a39c700 1 --2- 172.21.15.133:0/681440835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5424105f80 0x7f54240ff230 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f541c007ee0 tx=0x7f541c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:02.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.205+0000 7f541affd700 1 -- 172.21.15.133:0/681440835 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c017070 con 0x7f5424105f80 2023-11-27T15:22:02.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.205+0000 7f541affd700 1 -- 172.21.15.133:0/681440835 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f541c00f040 con 0x7f5424105f80 2023-11-27T15:22:02.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.205+0000 7f542bdff700 1 -- 172.21.15.133:0/681440835 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542410bca0 con 0x7f5424105f80 2023-11-27T15:22:02.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.205+0000 7f541affd700 1 -- 172.21.15.133:0/681440835 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c005580 con 0x7f5424105f80 2023-11-27T15:22:02.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.205+0000 7f542bdff700 1 -- 172.21.15.133:0/681440835 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542410bb10 con 0x7f5424105f80 2023-11-27T15:22:02.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.206+0000 7f542bdff700 1 -- 172.21.15.133:0/681440835 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54240fc330 con 0x7f5424105f80 2023-11-27T15:22:02.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.208+0000 7f541affd700 1 -- 172.21.15.133:0/681440835 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f541c00db20 con 0x7f5424105f80 2023-11-27T15:22:02.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.209+0000 7f541affd700 1 --2- 172.21.15.133:0/681440835 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f54100a11e0 0x7f54100a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:02.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.209+0000 7f541affd700 1 -- 172.21.15.133:0/681440835 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f541c0cc560 con 0x7f5424105f80 2023-11-27T15:22:02.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.209+0000 7f5429b9b700 1 --2- 172.21.15.133:0/681440835 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f54100a11e0 0x7f54100a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:02.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.210+0000 7f5429b9b700 1 --2- 172.21.15.133:0/681440835 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f54100a11e0 0x7f54100a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5420009c60 tx=0x7f5420009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:02.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.213+0000 7f541affd700 1 -- 172.21.15.133:0/681440835 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f541c0957a0 con 0x7f5424105f80 2023-11-27T15:22:02.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.412+0000 7f542bdff700 1 -- 172.21.15.133:0/681440835 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5424061700 con 0x7f5424105f80 2023-11-27T15:22:02.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.413+0000 7f541affd700 1 -- 172.21.15.133:0/681440835 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) v1 ==== 103+0+11 (secure 0 0 0) 0x7f541c098dc0 con 0x7f5424105f80 2023-11-27T15:22:02.418 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:22:02.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.415+0000 7f542bdff700 1 -- 172.21.15.133:0/681440835 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f54100a11e0 msgr2=0x7f54100a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:02.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.415+0000 7f542bdff700 1 --2- 172.21.15.133:0/681440835 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f54100a11e0 0x7f54100a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5420009c60 tx=0x7f5420009260 comp rx=0 tx=0).stop 2023-11-27T15:22:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.415+0000 7f542bdff700 1 -- 172.21.15.133:0/681440835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5424105f80 msgr2=0x7f54240ff230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.415+0000 7f542bdff700 1 --2- 172.21.15.133:0/681440835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5424105f80 0x7f54240ff230 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f541c007ee0 tx=0x7f541c00d3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:02.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.415+0000 7f542bdff700 1 -- 172.21.15.133:0/681440835 shutdown_connections 2023-11-27T15:22:02.422 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.415+0000 7f542bdff700 1 --2- 172.21.15.133:0/681440835 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f54100a11e0 0x7f54100a3690 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:02.422 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.415+0000 7f542bdff700 1 --2- 172.21.15.133:0/681440835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5424105f80 0x7f54240ff230 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:02.422 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.415+0000 7f542bdff700 1 --2- 172.21.15.133:0/681440835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5424100d40 0x7f54241006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:02.422 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.415+0000 7f542bdff700 1 --2- 172.21.15.133:0/681440835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5424103660 0x7f54240fecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:02.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.415+0000 7f542bdff700 1 -- 172.21.15.133:0/681440835 >> 172.21.15.133:0/681440835 conn(0x7f54240fa7c0 msgr2=0x7f54241019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:02.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.415+0000 7f542bdff700 1 -- 172.21.15.133:0/681440835 shutdown_connections 2023-11-27T15:22:02.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.415+0000 7f542bdff700 1 -- 172.21.15.133:0/681440835 wait complete. 2023-11-27T15:22:02.439 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:22:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.632+0000 7f0b541ff700 1 -- 172.21.15.133:0/2817778505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c074b30 msgr2=0x7f0b4c072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.632+0000 7f0b541ff700 1 --2- 172.21.15.133:0/2817778505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c074b30 0x7f0b4c072f80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0b480098f0 tx=0x7f0b48009c00 comp rx=0 tx=0).stop 2023-11-27T15:22:02.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.633+0000 7f0b541ff700 1 -- 172.21.15.133:0/2817778505 shutdown_connections 2023-11-27T15:22:02.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.633+0000 7f0b541ff700 1 --2- 172.21.15.133:0/2817778505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b4c10c860 0x7f0b4c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:02.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.633+0000 7f0b541ff700 1 --2- 172.21.15.133:0/2817778505 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c074b30 0x7f0b4c072f80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:02.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.633+0000 7f0b541ff700 1 --2- 172.21.15.133:0/2817778505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b4c073550 0x7f0b4c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:02.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.633+0000 7f0b541ff700 1 -- 172.21.15.133:0/2817778505 >> 172.21.15.133:0/2817778505 conn(0x7f0b4c0fb870 msgr2=0x7f0b4c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:02.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.633+0000 7f0b541ff700 1 -- 172.21.15.133:0/2817778505 shutdown_connections 2023-11-27T15:22:02.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.633+0000 7f0b541ff700 1 -- 172.21.15.133:0/2817778505 wait complete. 2023-11-27T15:22:02.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.634+0000 7f0b541ff700 1 Processor -- start 2023-11-27T15:22:02.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.634+0000 7f0b541ff700 1 -- start start 2023-11-27T15:22:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.634+0000 7f0b541ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b4c073550 0x7f0b4c19c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.634+0000 7f0b541ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c074b30 0x7f0b4c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:02.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.634+0000 7f0b541ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b4c10c860 0x7f0b4c1a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.634+0000 7f0b541ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c1a4190 con 0x7f0b4c073550 2023-11-27T15:22:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.634+0000 7f0b541ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c1a42d0 con 0x7f0b4c10c860 2023-11-27T15:22:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.634+0000 7f0b541ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c1a4460 con 0x7f0b4c074b30 2023-11-27T15:22:02.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.634+0000 7f0b51f9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b4c073550 0x7f0b4c19c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:02.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.634+0000 7f0b5279c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b4c10c860 0x7f0b4c1a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:02.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.634+0000 7f0b51f9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b4c073550 0x7f0b4c19c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40236/0 (socket says 172.21.15.133:40236) 2023-11-27T15:22:02.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.634+0000 7f0b51f9b700 1 -- 172.21.15.133:0/1789080455 learned_addr learned my addr 172.21.15.133:0/1789080455 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:02.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.634+0000 7f0b5179a700 1 --2- 172.21.15.133:0/1789080455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c074b30 0x7f0b4c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:02.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.635+0000 7f0b51f9b700 1 -- 172.21.15.133:0/1789080455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c074b30 msgr2=0x7f0b4c19cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:02.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.635+0000 7f0b51f9b700 1 --2- 172.21.15.133:0/1789080455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c074b30 0x7f0b4c19cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:02.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.635+0000 7f0b51f9b700 1 -- 172.21.15.133:0/1789080455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b4c10c860 msgr2=0x7f0b4c1a3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:02.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.635+0000 7f0b51f9b700 1 --2- 172.21.15.133:0/1789080455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b4c10c860 0x7f0b4c1a3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:02.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.635+0000 7f0b51f9b700 1 -- 172.21.15.133:0/1789080455 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c0095f0 con 0x7f0b4c073550 2023-11-27T15:22:02.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.635+0000 7f0b51f9b700 1 --2- 172.21.15.133:0/1789080455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b4c073550 0x7f0b4c19c610 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0b48009fd0 tx=0x7f0b4800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:02.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.635+0000 7f0b42ffd700 1 -- 172.21.15.133:0/1789080455 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4801c070 con 0x7f0b4c073550 2023-11-27T15:22:02.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.635+0000 7f0b541ff700 1 -- 172.21.15.133:0/1789080455 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b48009520 con 0x7f0b4c073550 2023-11-27T15:22:02.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.635+0000 7f0b42ffd700 1 -- 172.21.15.133:0/1789080455 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b4800f7d0 con 0x7f0b4c073550 2023-11-27T15:22:02.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.635+0000 7f0b541ff700 1 -- 172.21.15.133:0/1789080455 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4c112250 con 0x7f0b4c073550 2023-11-27T15:22:02.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.635+0000 7f0b42ffd700 1 -- 172.21.15.133:0/1789080455 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b480039c0 con 0x7f0b4c073550 2023-11-27T15:22:02.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.637+0000 7f0b541ff700 1 -- 172.21.15.133:0/1789080455 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b4c1023b0 con 0x7f0b4c073550 2023-11-27T15:22:02.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.638+0000 7f0b42ffd700 1 -- 172.21.15.133:0/1789080455 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f0b48020030 con 0x7f0b4c073550 2023-11-27T15:22:02.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.640+0000 7f0b42ffd700 1 --2- 172.21.15.133:0/1789080455 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0b380a11e0 0x7f0b380a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:02.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.640+0000 7f0b42ffd700 1 -- 172.21.15.133:0/1789080455 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f0b480cdf60 con 0x7f0b4c073550 2023-11-27T15:22:02.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.640+0000 7f0b5179a700 1 --2- 172.21.15.133:0/1789080455 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0b380a11e0 0x7f0b380a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:02.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.640+0000 7f0b5179a700 1 --2- 172.21.15.133:0/1789080455 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0b380a11e0 0x7f0b380a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c074850 tx=0x7f0b3c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:02.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.643+0000 7f0b42ffd700 1 -- 172.21.15.133:0/1789080455 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0b48097230 con 0x7f0b4c073550 2023-11-27T15:22:02.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.897+0000 7f0b541ff700 1 -- 172.21.15.133:0/1789080455 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b4c067a60 con 0x7f0b4c073550 2023-11-27T15:22:02.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.897+0000 7f0b42ffd700 1 -- 172.21.15.133:0/1789080455 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10829 (secure 0 0 0) 0x7f0b4809a850 con 0x7f0b4c073550 2023-11-27T15:22:02.903 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:02.903 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14493,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2381290259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2381290259},{"type":"v1","addr":"172.21.15.133:6839","nonce":2381290259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1704875895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1704875895},{"type":"v1","addr":"172.21.15.133:6841","nonce":1704875895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:20:33.201023+0000","modified":"2023-11-27T15:21:04.794526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24521,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3492215689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3492215689},{"type":"v1","addr":"172.21.15.177:6837","nonce":3492215689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6839/2781994447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2781994447},{"type":"v1","addr":"172.21.15.174:6839","nonce":2781994447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:22:02.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.900+0000 7f0b541ff700 1 -- 172.21.15.133:0/1789080455 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0b380a11e0 msgr2=0x7f0b380a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:02.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.900+0000 7f0b541ff700 1 --2- 172.21.15.133:0/1789080455 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0b380a11e0 0x7f0b380a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c074850 tx=0x7f0b3c009310 comp rx=0 tx=0).stop 2023-11-27T15:22:02.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.900+0000 7f0b541ff700 1 -- 172.21.15.133:0/1789080455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b4c073550 msgr2=0x7f0b4c19c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:02.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.900+0000 7f0b541ff700 1 --2- 172.21.15.133:0/1789080455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b4c073550 0x7f0b4c19c610 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0b48009fd0 tx=0x7f0b4800b790 comp rx=0 tx=0).stop 2023-11-27T15:22:02.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.901+0000 7f0b541ff700 1 -- 172.21.15.133:0/1789080455 shutdown_connections 2023-11-27T15:22:02.906 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.901+0000 7f0b541ff700 1 --2- 172.21.15.133:0/1789080455 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0b380a11e0 0x7f0b380a3690 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:02.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.901+0000 7f0b541ff700 1 --2- 172.21.15.133:0/1789080455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b4c10c860 0x7f0b4c1a3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:02.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.901+0000 7f0b541ff700 1 --2- 172.21.15.133:0/1789080455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b4c073550 0x7f0b4c19c610 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:02.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.901+0000 7f0b541ff700 1 --2- 172.21.15.133:0/1789080455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b4c074b30 0x7f0b4c19cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:02.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.901+0000 7f0b541ff700 1 -- 172.21.15.133:0/1789080455 >> 172.21.15.133:0/1789080455 conn(0x7f0b4c0fb870 msgr2=0x7f0b4c0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:02.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.901+0000 7f0b541ff700 1 -- 172.21.15.133:0/1789080455 shutdown_connections 2023-11-27T15:22:02.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:02.901+0000 7f0b541ff700 1 -- 172.21.15.133:0/1789080455 wait complete. 2023-11-27T15:22:02.908 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 23 2023-11-27T15:22:02.925 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:22:03.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:02 smithi177 ceph-mon[131504]: pgmap v320: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:22:03.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-11-27T15:22:03.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:02 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-11-27T15:22:03.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:02 smithi177 ceph-mon[131504]: osdmap e91: 12 total, 12 up, 12 in 2023-11-27T15:22:03.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:02 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3955108375' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:22:03.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:02 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/681440835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:22:03.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.119+0000 7f01955ff700 1 -- 172.21.15.133:0/852507507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0190100290 msgr2=0x7f01901006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:03.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.119+0000 7f01955ff700 1 --2- 172.21.15.133:0/852507507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0190100290 0x7f01901006a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f01800098f0 tx=0x7f0180009c00 comp rx=0 tx=0).stop 2023-11-27T15:22:03.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.119+0000 7f01955ff700 1 -- 172.21.15.133:0/852507507 shutdown_connections 2023-11-27T15:22:03.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.119+0000 7f01955ff700 1 --2- 172.21.15.133:0/852507507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0190101e10 0x7f01901086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.119+0000 7f01955ff700 1 --2- 172.21.15.133:0/852507507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0190100290 0x7f01901006a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.119+0000 7f01955ff700 1 --2- 172.21.15.133:0/852507507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0190101480 0x7f01901018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.119+0000 7f01955ff700 1 -- 172.21.15.133:0/852507507 >> 172.21.15.133:0/852507507 conn(0x7f01900fb870 msgr2=0x7f01900fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:03.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.119+0000 7f01955ff700 1 -- 172.21.15.133:0/852507507 shutdown_connections 2023-11-27T15:22:03.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.119+0000 7f01955ff700 1 -- 172.21.15.133:0/852507507 wait complete. 2023-11-27T15:22:03.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.120+0000 7f01955ff700 1 Processor -- start 2023-11-27T15:22:03.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.121+0000 7f01955ff700 1 -- start start 2023-11-27T15:22:03.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.121+0000 7f01955ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0190100290 0x7f0190196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:03.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.121+0000 7f01955ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0190101480 0x7f0190196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:03.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.121+0000 7f01955ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0190101e10 0x7f019019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:03.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.121+0000 7f01955ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019019dba0 con 0x7f0190100290 2023-11-27T15:22:03.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.121+0000 7f01955ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019019dce0 con 0x7f0190101e10 2023-11-27T15:22:03.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.121+0000 7f01955ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019019de70 con 0x7f0190101480 2023-11-27T15:22:03.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.121+0000 7f018effd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0190100290 0x7f0190196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:03.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.121+0000 7f018e7fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0190101480 0x7f0190196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:03.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.121+0000 7f018effd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0190100290 0x7f0190196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59540/0 (socket says 172.21.15.133:59540) 2023-11-27T15:22:03.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.121+0000 7f018e7fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0190101480 0x7f0190196560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:58402/0 (socket says 172.21.15.133:58402) 2023-11-27T15:22:03.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.121+0000 7f018effd700 1 -- 172.21.15.133:0/1503177665 learned_addr learned my addr 172.21.15.133:0/1503177665 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:03.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.121+0000 7f018f7fe700 1 --2- 172.21.15.133:0/1503177665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0190101e10 0x7f019019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:03.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.122+0000 7f018effd700 1 -- 172.21.15.133:0/1503177665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0190101480 msgr2=0x7f0190196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:03.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.122+0000 7f018effd700 1 --2- 172.21.15.133:0/1503177665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0190101480 0x7f0190196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.122+0000 7f018effd700 1 -- 172.21.15.133:0/1503177665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0190101e10 msgr2=0x7f019019d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:03.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.122+0000 7f018effd700 1 --2- 172.21.15.133:0/1503177665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0190101e10 0x7f019019d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.122+0000 7f018effd700 1 -- 172.21.15.133:0/1503177665 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0180009520 con 0x7f0190100290 2023-11-27T15:22:03.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.122+0000 7f018effd700 1 --2- 172.21.15.133:0/1503177665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0190100290 0x7f0190196020 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0180005e90 tx=0x7f018000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:03.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.122+0000 7f0177fff700 1 -- 172.21.15.133:0/1503177665 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018001c070 con 0x7f0190100290 2023-11-27T15:22:03.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.122+0000 7f0177fff700 1 -- 172.21.15.133:0/1503177665 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f018000f7f0 con 0x7f0190100290 2023-11-27T15:22:03.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.122+0000 7f01955ff700 1 -- 172.21.15.133:0/1503177665 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019010bc40 con 0x7f0190100290 2023-11-27T15:22:03.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.122+0000 7f0177fff700 1 -- 172.21.15.133:0/1503177665 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0180003990 con 0x7f0190100290 2023-11-27T15:22:03.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.122+0000 7f01955ff700 1 -- 172.21.15.133:0/1503177665 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019010bab0 con 0x7f0190100290 2023-11-27T15:22:03.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.123+0000 7f01955ff700 1 -- 172.21.15.133:0/1503177665 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f019019e410 con 0x7f0190100290 2023-11-27T15:22:03.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.125+0000 7f0177fff700 1 -- 172.21.15.133:0/1503177665 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f0180020030 con 0x7f0190100290 2023-11-27T15:22:03.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.127+0000 7f0177fff700 1 --2- 172.21.15.133:0/1503177665 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f01780a11e0 0x7f01780a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:03.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.127+0000 7f0177fff700 1 -- 172.21.15.133:0/1503177665 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f01800ce180 con 0x7f0190100290 2023-11-27T15:22:03.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.127+0000 7f018e7fc700 1 --2- 172.21.15.133:0/1503177665 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f01780a11e0 0x7f01780a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:03.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.127+0000 7f018e7fc700 1 --2- 172.21.15.133:0/1503177665 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f01780a11e0 0x7f01780a3690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0184005fd0 tx=0x7f0184009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:03.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.129+0000 7f0177fff700 1 -- 172.21.15.133:0/1503177665 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f01800974b0 con 0x7f0190100290 2023-11-27T15:22:03.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:02 smithi174 ceph-mon[131110]: pgmap v320: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:22:03.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-11-27T15:22:03.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:02 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-11-27T15:22:03.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:02 smithi174 ceph-mon[131110]: osdmap e91: 12 total, 12 up, 12 in 2023-11-27T15:22:03.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:02 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3955108375' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:22:03.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:02 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/681440835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:22:03.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:02 smithi133 ceph-mon[97860]: pgmap v320: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:22:03.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-11-27T15:22:03.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:02 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-11-27T15:22:03.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:02 smithi133 ceph-mon[97860]: osdmap e91: 12 total, 12 up, 12 in 2023-11-27T15:22:03.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:02 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3955108375' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:22:03.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:02 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/681440835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:22:03.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.333+0000 7f01955ff700 1 -- 172.21.15.133:0/1503177665 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0190107210 con 0x7f0190100290 2023-11-27T15:22:03.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.335+0000 7f0177fff700 1 -- 172.21.15.133:0/1503177665 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24616 (secure 0 0 0) 0x7f018009aad0 con 0x7f0190100290 2023-11-27T15:22:03.340 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:03.341 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":92,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:22:02.807005+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:22:03.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.337+0000 7f01955ff700 1 -- 172.21.15.133:0/1503177665 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f01780a11e0 msgr2=0x7f01780a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.337+0000 7f01955ff700 1 --2- 172.21.15.133:0/1503177665 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f01780a11e0 0x7f01780a3690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0184005fd0 tx=0x7f0184009310 comp rx=0 tx=0).stop 2023-11-27T15:22:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.337+0000 7f01955ff700 1 -- 172.21.15.133:0/1503177665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0190100290 msgr2=0x7f0190196020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.337+0000 7f01955ff700 1 --2- 172.21.15.133:0/1503177665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0190100290 0x7f0190196020 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0180005e90 tx=0x7f018000b730 comp rx=0 tx=0).stop 2023-11-27T15:22:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.337+0000 7f01955ff700 1 -- 172.21.15.133:0/1503177665 shutdown_connections 2023-11-27T15:22:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.337+0000 7f01955ff700 1 --2- 172.21.15.133:0/1503177665 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f01780a11e0 0x7f01780a3690 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.337+0000 7f01955ff700 1 --2- 172.21.15.133:0/1503177665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0190101e10 0x7f019019d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.337+0000 7f01955ff700 1 --2- 172.21.15.133:0/1503177665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0190100290 0x7f0190196020 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.337+0000 7f01955ff700 1 --2- 172.21.15.133:0/1503177665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0190101480 0x7f0190196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.337+0000 7f01955ff700 1 -- 172.21.15.133:0/1503177665 >> 172.21.15.133:0/1503177665 conn(0x7f01900fb870 msgr2=0x7f01900fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:03.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.338+0000 7f01955ff700 1 -- 172.21.15.133:0/1503177665 shutdown_connections 2023-11-27T15:22:03.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.338+0000 7f01955ff700 1 -- 172.21.15.133:0/1503177665 wait complete. 2023-11-27T15:22:03.362 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:16:20.216150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:20:31.107146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:20:32.614089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-27T15:20:36.126040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-27T15:22:03.362 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:22:03.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.559+0000 7f84357ff700 1 -- 172.21.15.133:0/1133586832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84301014b0 msgr2=0x7f8430101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:03.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.559+0000 7f84357ff700 1 --2- 172.21.15.133:0/1133586832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84301014b0 0x7f8430101900 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f84240098f0 tx=0x7f8424009c00 comp rx=0 tx=0).stop 2023-11-27T15:22:03.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.559+0000 7f84357ff700 1 -- 172.21.15.133:0/1133586832 shutdown_connections 2023-11-27T15:22:03.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.559+0000 7f84357ff700 1 --2- 172.21.15.133:0/1133586832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8430101e40 0x7f8430108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.559+0000 7f84357ff700 1 --2- 172.21.15.133:0/1133586832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84301002c0 0x7f84301006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.559+0000 7f84357ff700 1 --2- 172.21.15.133:0/1133586832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84301014b0 0x7f8430101900 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.559+0000 7f84357ff700 1 -- 172.21.15.133:0/1133586832 >> 172.21.15.133:0/1133586832 conn(0x7f84300fb8a0 msgr2=0x7f84300fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:03.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.559+0000 7f84357ff700 1 -- 172.21.15.133:0/1133586832 shutdown_connections 2023-11-27T15:22:03.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.559+0000 7f84357ff700 1 -- 172.21.15.133:0/1133586832 wait complete. 2023-11-27T15:22:03.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.560+0000 7f84357ff700 1 Processor -- start 2023-11-27T15:22:03.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.560+0000 7f84357ff700 1 -- start start 2023-11-27T15:22:03.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.561+0000 7f84357ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84301002c0 0x7f84300748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:03.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.561+0000 7f84357ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84301014b0 0x7f8430072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:03.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.561+0000 7f84357ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8430101e40 0x7f8430073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:03.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.561+0000 7f84357ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8430073bc0 con 0x7f84301014b0 2023-11-27T15:22:03.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.561+0000 7f84357ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8430073d30 con 0x7f8430101e40 2023-11-27T15:22:03.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.561+0000 7f84357ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84301a00f0 con 0x7f84301002c0 2023-11-27T15:22:03.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.561+0000 7f842e7fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84301014b0 0x7f8430072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:03.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.561+0000 7f842e7fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84301014b0 0x7f8430072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59560/0 (socket says 172.21.15.133:59560) 2023-11-27T15:22:03.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.561+0000 7f842e7fc700 1 -- 172.21.15.133:0/121435163 learned_addr learned my addr 172.21.15.133:0/121435163 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:03.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.561+0000 7f842effd700 1 --2- 172.21.15.133:0/121435163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84301002c0 0x7f84300748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:03.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.561+0000 7f842f7fe700 1 --2- 172.21.15.133:0/121435163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8430101e40 0x7f8430073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:03.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.561+0000 7f842e7fc700 1 -- 172.21.15.133:0/121435163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84301002c0 msgr2=0x7f84300748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:03.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.561+0000 7f842e7fc700 1 --2- 172.21.15.133:0/121435163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84301002c0 0x7f84300748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.561+0000 7f842e7fc700 1 -- 172.21.15.133:0/121435163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8430101e40 msgr2=0x7f8430073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:03.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.562+0000 7f842e7fc700 1 --2- 172.21.15.133:0/121435163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8430101e40 0x7f8430073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.562+0000 7f842e7fc700 1 -- 172.21.15.133:0/121435163 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8424009520 con 0x7f84301014b0 2023-11-27T15:22:03.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.562+0000 7f842effd700 1 --2- 172.21.15.133:0/121435163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84301002c0 0x7f84300748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:03.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.562+0000 7f842e7fc700 1 --2- 172.21.15.133:0/121435163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84301014b0 0x7f8430072ee0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8424005e90 tx=0x7f842400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:03.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.562+0000 7f8417fff700 1 -- 172.21.15.133:0/121435163 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842401c070 con 0x7f84301014b0 2023-11-27T15:22:03.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.562+0000 7f84357ff700 1 -- 172.21.15.133:0/121435163 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f843010bc70 con 0x7f84301014b0 2023-11-27T15:22:03.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.562+0000 7f8417fff700 1 -- 172.21.15.133:0/121435163 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f842400f7f0 con 0x7f84301014b0 2023-11-27T15:22:03.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.562+0000 7f8417fff700 1 -- 172.21.15.133:0/121435163 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8424004a40 con 0x7f84301014b0 2023-11-27T15:22:03.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.562+0000 7f84357ff700 1 -- 172.21.15.133:0/121435163 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843010bae0 con 0x7f84301014b0 2023-11-27T15:22:03.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.563+0000 7f84357ff700 1 -- 172.21.15.133:0/121435163 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84301071f0 con 0x7f84301014b0 2023-11-27T15:22:03.572 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.565+0000 7f8417fff700 1 -- 172.21.15.133:0/121435163 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f8424020070 con 0x7f84301014b0 2023-11-27T15:22:03.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.566+0000 7f8417fff700 1 --2- 172.21.15.133:0/121435163 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f84180a11e0 0x7f84180a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:03.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.567+0000 7f8417fff700 1 -- 172.21.15.133:0/121435163 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f84240cdb10 con 0x7f84301014b0 2023-11-27T15:22:03.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.567+0000 7f842effd700 1 --2- 172.21.15.133:0/121435163 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f84180a11e0 0x7f84180a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:03.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.567+0000 7f842effd700 1 --2- 172.21.15.133:0/121435163 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f84180a11e0 0x7f84180a3690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8430101310 tx=0x7f8420009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:03.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.570+0000 7f8417fff700 1 -- 172.21.15.133:0/121435163 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8424096e40 con 0x7f84301014b0 2023-11-27T15:22:03.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.768+0000 7f84357ff700 1 -- 172.21.15.133:0/121435163 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8430067b00 con 0x7f84301014b0 2023-11-27T15:22:03.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.768+0000 7f8417fff700 1 -- 172.21.15.133:0/121435163 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f842409a460 con 0x7f84301014b0 2023-11-27T15:22:03.773 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-11-27T15:22:03.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.771+0000 7f84357ff700 1 -- 172.21.15.133:0/121435163 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f84180a11e0 msgr2=0x7f84180a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:03.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.771+0000 7f84357ff700 1 --2- 172.21.15.133:0/121435163 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f84180a11e0 0x7f84180a3690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8430101310 tx=0x7f8420009310 comp rx=0 tx=0).stop 2023-11-27T15:22:03.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.771+0000 7f84357ff700 1 -- 172.21.15.133:0/121435163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84301014b0 msgr2=0x7f8430072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:03.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.771+0000 7f84357ff700 1 --2- 172.21.15.133:0/121435163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84301014b0 0x7f8430072ee0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8424005e90 tx=0x7f842400b730 comp rx=0 tx=0).stop 2023-11-27T15:22:03.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.771+0000 7f84357ff700 1 -- 172.21.15.133:0/121435163 shutdown_connections 2023-11-27T15:22:03.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.771+0000 7f84357ff700 1 --2- 172.21.15.133:0/121435163 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f84180a11e0 0x7f84180a3690 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.771+0000 7f84357ff700 1 --2- 172.21.15.133:0/121435163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8430101e40 0x7f8430073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.771+0000 7f84357ff700 1 --2- 172.21.15.133:0/121435163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84301002c0 0x7f84300748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.771+0000 7f84357ff700 1 --2- 172.21.15.133:0/121435163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84301014b0 0x7f8430072ee0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.771+0000 7f84357ff700 1 -- 172.21.15.133:0/121435163 >> 172.21.15.133:0/121435163 conn(0x7f84300fb8a0 msgr2=0x7f84300fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:03.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.771+0000 7f84357ff700 1 -- 172.21.15.133:0/121435163 shutdown_connections 2023-11-27T15:22:03.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.771+0000 7f84357ff700 1 -- 172.21.15.133:0/121435163 wait complete. 2023-11-27T15:22:03.795 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:22:03.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.992+0000 7f69adbff700 1 -- 172.21.15.133:0/118317689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69a81014b0 msgr2=0x7f69a8101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:03.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.992+0000 7f69adbff700 1 --2- 172.21.15.133:0/118317689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69a81014b0 0x7f69a8101900 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f699c009990 tx=0x7f699c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.992+0000 7f69adbff700 1 -- 172.21.15.133:0/118317689 shutdown_connections 2023-11-27T15:22:03.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.992+0000 7f69adbff700 1 --2- 172.21.15.133:0/118317689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69a8101e40 0x7f69a8108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.992+0000 7f69adbff700 1 --2- 172.21.15.133:0/118317689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69a81002c0 0x7f69a81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.992+0000 7f69adbff700 1 --2- 172.21.15.133:0/118317689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69a81014b0 0x7f69a8101900 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:03.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.992+0000 7f69adbff700 1 -- 172.21.15.133:0/118317689 >> 172.21.15.133:0/118317689 conn(0x7f69a80fb8a0 msgr2=0x7f69a80fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:03.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.992+0000 7f69adbff700 1 -- 172.21.15.133:0/118317689 shutdown_connections 2023-11-27T15:22:03.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.993+0000 7f69adbff700 1 -- 172.21.15.133:0/118317689 wait complete. 2023-11-27T15:22:04.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.993+0000 7f69adbff700 1 Processor -- start 2023-11-27T15:22:04.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.994+0000 7f69adbff700 1 -- start start 2023-11-27T15:22:04.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.994+0000 7f69adbff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69a81002c0 0x7f69a80748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:04.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.994+0000 7f69adbff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69a81014b0 0x7f69a8072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:04.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.994+0000 7f69adbff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69a8101e40 0x7f69a8073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:04.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.994+0000 7f69adbff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a8073bc0 con 0x7f69a81014b0 2023-11-27T15:22:04.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.994+0000 7f69adbff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a8073d30 con 0x7f69a8101e40 2023-11-27T15:22:04.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.994+0000 7f69adbff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69a81a00d0 con 0x7f69a81002c0 2023-11-27T15:22:04.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.994+0000 7f69a6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69a81014b0 0x7f69a8072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:04.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.994+0000 7f69a6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69a81014b0 0x7f69a8072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59580/0 (socket says 172.21.15.133:59580) 2023-11-27T15:22:04.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.994+0000 7f69a6ffd700 1 -- 172.21.15.133:0/2114379779 learned_addr learned my addr 172.21.15.133:0/2114379779 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:04.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.994+0000 7f69a7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69a8101e40 0x7f69a8073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:04.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.994+0000 7f69a77fe700 1 --2- 172.21.15.133:0/2114379779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69a81002c0 0x7f69a80748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:04.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.995+0000 7f69a6ffd700 1 -- 172.21.15.133:0/2114379779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69a81002c0 msgr2=0x7f69a80748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:04.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.995+0000 7f69a6ffd700 1 --2- 172.21.15.133:0/2114379779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69a81002c0 0x7f69a80748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.995+0000 7f69a6ffd700 1 -- 172.21.15.133:0/2114379779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69a8101e40 msgr2=0x7f69a8073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:04.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.995+0000 7f69a6ffd700 1 --2- 172.21.15.133:0/2114379779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69a8101e40 0x7f69a8073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.995+0000 7f69a6ffd700 1 -- 172.21.15.133:0/2114379779 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f699c0095f0 con 0x7f69a81014b0 2023-11-27T15:22:04.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.995+0000 7f69a7fff700 1 --2- 172.21.15.133:0/2114379779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69a8101e40 0x7f69a8073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:04.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.995+0000 7f69a6ffd700 1 --2- 172.21.15.133:0/2114379779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69a81014b0 0x7f69a8072ee0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f699c004760 tx=0x7f699c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:04.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.995+0000 7f69a4ff9700 1 -- 172.21.15.133:0/2114379779 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699c01d070 con 0x7f69a81014b0 2023-11-27T15:22:04.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.995+0000 7f69adbff700 1 -- 172.21.15.133:0/2114379779 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a810bc70 con 0x7f69a81014b0 2023-11-27T15:22:04.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.995+0000 7f69a4ff9700 1 -- 172.21.15.133:0/2114379779 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f699c004830 con 0x7f69a81014b0 2023-11-27T15:22:04.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.995+0000 7f69a4ff9700 1 -- 172.21.15.133:0/2114379779 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f699c005350 con 0x7f69a81014b0 2023-11-27T15:22:04.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.995+0000 7f69adbff700 1 -- 172.21.15.133:0/2114379779 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69a810bae0 con 0x7f69a81014b0 2023-11-27T15:22:04.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.996+0000 7f69adbff700 1 -- 172.21.15.133:0/2114379779 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69a8107850 con 0x7f69a81014b0 2023-11-27T15:22:04.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.998+0000 7f69a4ff9700 1 -- 172.21.15.133:0/2114379779 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f699c005530 con 0x7f69a81014b0 2023-11-27T15:22:04.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:03.999+0000 7f69a4ff9700 1 --2- 172.21.15.133:0/2114379779 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f69900a11e0 0x7f69900a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:04.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.000+0000 7f69a4ff9700 1 -- 172.21.15.133:0/2114379779 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f699c0cdfe0 con 0x7f69a81014b0 2023-11-27T15:22:04.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.000+0000 7f69a77fe700 1 --2- 172.21.15.133:0/2114379779 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f69900a11e0 0x7f69900a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:04.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.000+0000 7f69a77fe700 1 --2- 172.21.15.133:0/2114379779 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f69900a11e0 0x7f69900a3690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f69a8101310 tx=0x7f6998009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:04.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.003+0000 7f69a4ff9700 1 -- 172.21.15.133:0/2114379779 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f699c05a730 con 0x7f69a81014b0 2023-11-27T15:22:04.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:03 smithi177 ceph-mon[131504]: osdmap e92: 12 total, 12 up, 12 in 2023-11-27T15:22:04.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:03 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1789080455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:04.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:03 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1503177665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:04.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:03 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/121435163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:04.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:03 smithi174 ceph-mon[131110]: osdmap e92: 12 total, 12 up, 12 in 2023-11-27T15:22:04.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:03 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1789080455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:04.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:03 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1503177665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:04.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:03 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/121435163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:04.206 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.201+0000 7f69adbff700 1 -- 172.21.15.133:0/2114379779 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f69a8067b00 con 0x7f69a81014b0 2023-11-27T15:22:04.206 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.201+0000 7f69a4ff9700 1 -- 172.21.15.133:0/2114379779 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f699c0972d0 con 0x7f69a81014b0 2023-11-27T15:22:04.207 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:22:04.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.204+0000 7f69adbff700 1 -- 172.21.15.133:0/2114379779 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f69900a11e0 msgr2=0x7f69900a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:04.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.204+0000 7f69adbff700 1 --2- 172.21.15.133:0/2114379779 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f69900a11e0 0x7f69900a3690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f69a8101310 tx=0x7f6998009260 comp rx=0 tx=0).stop 2023-11-27T15:22:04.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.204+0000 7f69adbff700 1 -- 172.21.15.133:0/2114379779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69a81014b0 msgr2=0x7f69a8072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:04.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.204+0000 7f69adbff700 1 --2- 172.21.15.133:0/2114379779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69a81014b0 0x7f69a8072ee0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f699c004760 tx=0x7f699c004790 comp rx=0 tx=0).stop 2023-11-27T15:22:04.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.204+0000 7f69adbff700 1 -- 172.21.15.133:0/2114379779 shutdown_connections 2023-11-27T15:22:04.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.204+0000 7f69adbff700 1 --2- 172.21.15.133:0/2114379779 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f69900a11e0 0x7f69900a3690 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.204+0000 7f69adbff700 1 --2- 172.21.15.133:0/2114379779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69a8101e40 0x7f69a8073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.204+0000 7f69adbff700 1 --2- 172.21.15.133:0/2114379779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f69a81002c0 0x7f69a80748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.204+0000 7f69adbff700 1 --2- 172.21.15.133:0/2114379779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69a81014b0 0x7f69a8072ee0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.204+0000 7f69adbff700 1 -- 172.21.15.133:0/2114379779 >> 172.21.15.133:0/2114379779 conn(0x7f69a80fb8a0 msgr2=0x7f69a8104990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:04.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.204+0000 7f69adbff700 1 -- 172.21.15.133:0/2114379779 shutdown_connections 2023-11-27T15:22:04.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.204+0000 7f69adbff700 1 -- 172.21.15.133:0/2114379779 wait complete. 2023-11-27T15:22:04.226 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:03 smithi133 ceph-mon[97860]: osdmap e92: 12 total, 12 up, 12 in 2023-11-27T15:22:04.226 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:03 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1789080455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:04.226 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:03 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1503177665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:04.226 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:03 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/121435163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:04.228 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:22:04.430 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.424+0000 7f3bcc5ff700 1 -- 172.21.15.133:0/3117802988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bc41002c0 msgr2=0x7f3bc41006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:04.430 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.424+0000 7f3bcc5ff700 1 --2- 172.21.15.133:0/3117802988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bc41002c0 0x7f3bc41006d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0009990 tx=0x7f3bc0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.430 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.425+0000 7f3bcc5ff700 1 -- 172.21.15.133:0/3117802988 shutdown_connections 2023-11-27T15:22:04.430 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.425+0000 7f3bcc5ff700 1 --2- 172.21.15.133:0/3117802988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bc4101e40 0x7f3bc4108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.431 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.425+0000 7f3bcc5ff700 1 --2- 172.21.15.133:0/3117802988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bc41002c0 0x7f3bc41006d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.431 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.425+0000 7f3bcc5ff700 1 --2- 172.21.15.133:0/3117802988 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bc41014b0 0x7f3bc4101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.431 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.425+0000 7f3bcc5ff700 1 -- 172.21.15.133:0/3117802988 >> 172.21.15.133:0/3117802988 conn(0x7f3bc40fb8a0 msgr2=0x7f3bc40fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:04.431 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.425+0000 7f3bcc5ff700 1 -- 172.21.15.133:0/3117802988 shutdown_connections 2023-11-27T15:22:04.431 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.425+0000 7f3bcc5ff700 1 -- 172.21.15.133:0/3117802988 wait complete. 2023-11-27T15:22:04.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.426+0000 7f3bcc5ff700 1 Processor -- start 2023-11-27T15:22:04.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.426+0000 7f3bcc5ff700 1 -- start start 2023-11-27T15:22:04.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.426+0000 7f3bcc5ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bc41002c0 0x7f3bc4196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:04.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.426+0000 7f3bcc5ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bc41014b0 0x7f3bc4196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:04.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.426+0000 7f3bcc5ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bc4101e40 0x7f3bc419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:04.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.426+0000 7f3bcc5ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc419dbb0 con 0x7f3bc4101e40 2023-11-27T15:22:04.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.426+0000 7f3bcc5ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc419dcf0 con 0x7f3bc41014b0 2023-11-27T15:22:04.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.426+0000 7f3bcc5ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc419de80 con 0x7f3bc41002c0 2023-11-27T15:22:04.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.426+0000 7f3bca39b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bc41002c0 0x7f3bc4196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:04.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.427+0000 7f3bca39b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bc41002c0 0x7f3bc4196030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:58440/0 (socket says 172.21.15.133:58440) 2023-11-27T15:22:04.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.427+0000 7f3bca39b700 1 -- 172.21.15.133:0/474837528 learned_addr learned my addr 172.21.15.133:0/474837528 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:04.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.427+0000 7f3bcab9c700 1 --2- 172.21.15.133:0/474837528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bc4101e40 0x7f3bc419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:04.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.427+0000 7f3bc9b9a700 1 --2- 172.21.15.133:0/474837528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bc41014b0 0x7f3bc4196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:04.436 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.427+0000 7f3bca39b700 1 -- 172.21.15.133:0/474837528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bc41014b0 msgr2=0x7f3bc4196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:04.436 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.427+0000 7f3bca39b700 1 --2- 172.21.15.133:0/474837528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bc41014b0 0x7f3bc4196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.436 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.427+0000 7f3bca39b700 1 -- 172.21.15.133:0/474837528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bc4101e40 msgr2=0x7f3bc419d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:04.436 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.427+0000 7f3bca39b700 1 --2- 172.21.15.133:0/474837528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bc4101e40 0x7f3bc419d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.436 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.427+0000 7f3bca39b700 1 -- 172.21.15.133:0/474837528 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc009c40 con 0x7f3bc41002c0 2023-11-27T15:22:04.436 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.427+0000 7f3bc9b9a700 1 --2- 172.21.15.133:0/474837528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bc41014b0 0x7f3bc4196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:22:04.436 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.427+0000 7f3bca39b700 1 --2- 172.21.15.133:0/474837528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bc41002c0 0x7f3bc4196030 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000b620 tx=0x7f3bc0005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:04.436 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.427+0000 7f3bbb7fe700 1 -- 172.21.15.133:0/474837528 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc001d070 con 0x7f3bc41002c0 2023-11-27T15:22:04.437 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.428+0000 7f3bcc5ff700 1 -- 172.21.15.133:0/474837528 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc00095f0 con 0x7f3bc41002c0 2023-11-27T15:22:04.437 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.428+0000 7f3bcc5ff700 1 -- 172.21.15.133:0/474837528 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc410bc20 con 0x7f3bc41002c0 2023-11-27T15:22:04.437 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.428+0000 7f3bbb7fe700 1 -- 172.21.15.133:0/474837528 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bc00047e0 con 0x7f3bc41002c0 2023-11-27T15:22:04.437 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.428+0000 7f3bbb7fe700 1 -- 172.21.15.133:0/474837528 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc00054f0 con 0x7f3bc41002c0 2023-11-27T15:22:04.437 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.429+0000 7f3bcc5ff700 1 -- 172.21.15.133:0/474837528 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba8005310 con 0x7f3bc41002c0 2023-11-27T15:22:04.438 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.431+0000 7f3bbb7fe700 1 -- 172.21.15.133:0/474837528 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f3bc000fb40 con 0x7f3bc41002c0 2023-11-27T15:22:04.438 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.432+0000 7f3bbb7fe700 1 --2- 172.21.15.133:0/474837528 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3bb00a11e0 0x7f3bb00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:04.439 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.432+0000 7f3bbb7fe700 1 -- 172.21.15.133:0/474837528 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f3bc00cde60 con 0x7f3bc41002c0 2023-11-27T15:22:04.439 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.432+0000 7f3bc9b9a700 1 --2- 172.21.15.133:0/474837528 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3bb00a11e0 0x7f3bb00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:04.439 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.433+0000 7f3bc9b9a700 1 --2- 172.21.15.133:0/474837528 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3bb00a11e0 0x7f3bb00a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009cb0 tx=0x7f3bb4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:04.439 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.434+0000 7f3bbb7fe700 1 -- 172.21.15.133:0/474837528 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3bc0097130 con 0x7f3bc41002c0 2023-11-27T15:22:04.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.633+0000 7f3bcc5ff700 1 -- 172.21.15.133:0/474837528 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ba8005f60 con 0x7f3bc41002c0 2023-11-27T15:22:04.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.633+0000 7f3bbb7fe700 1 -- 172.21.15.133:0/474837528 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3bc0030020 con 0x7f3bc41002c0 2023-11-27T15:22:04.638 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-11-27T15:22:04.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.636+0000 7f3bcc5ff700 1 -- 172.21.15.133:0/474837528 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3bb00a11e0 msgr2=0x7f3bb00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:04.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.636+0000 7f3bcc5ff700 1 --2- 172.21.15.133:0/474837528 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3bb00a11e0 0x7f3bb00a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4009cb0 tx=0x7f3bb4009260 comp rx=0 tx=0).stop 2023-11-27T15:22:04.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.636+0000 7f3bcc5ff700 1 -- 172.21.15.133:0/474837528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bc41002c0 msgr2=0x7f3bc4196030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:04.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.636+0000 7f3bcc5ff700 1 --2- 172.21.15.133:0/474837528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bc41002c0 0x7f3bc4196030 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3bc000b620 tx=0x7f3bc0005ef0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.636+0000 7f3bcc5ff700 1 -- 172.21.15.133:0/474837528 shutdown_connections 2023-11-27T15:22:04.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.636+0000 7f3bcc5ff700 1 --2- 172.21.15.133:0/474837528 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3bb00a11e0 0x7f3bb00a3690 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.636+0000 7f3bcc5ff700 1 --2- 172.21.15.133:0/474837528 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3bc4101e40 0x7f3bc419d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.636+0000 7f3bcc5ff700 1 --2- 172.21.15.133:0/474837528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3bc41002c0 0x7f3bc4196030 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.636+0000 7f3bcc5ff700 1 --2- 172.21.15.133:0/474837528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bc41014b0 0x7f3bc4196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.636+0000 7f3bcc5ff700 1 -- 172.21.15.133:0/474837528 >> 172.21.15.133:0/474837528 conn(0x7f3bc40fb8a0 msgr2=0x7f3bc41049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:04.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.636+0000 7f3bcc5ff700 1 -- 172.21.15.133:0/474837528 shutdown_connections 2023-11-27T15:22:04.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.636+0000 7f3bcc5ff700 1 -- 172.21.15.133:0/474837528 wait complete. 2023-11-27T15:22:04.660 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-27T15:22:04.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.858+0000 7fcba19ff700 1 -- 172.21.15.133:0/3634294436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb9c1014b0 msgr2=0x7fcb9c101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:04.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.858+0000 7fcba19ff700 1 --2- 172.21.15.133:0/3634294436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb9c1014b0 0x7fcb9c101900 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009990 tx=0x7fcb90009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.858+0000 7fcba19ff700 1 -- 172.21.15.133:0/3634294436 shutdown_connections 2023-11-27T15:22:04.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.858+0000 7fcba19ff700 1 --2- 172.21.15.133:0/3634294436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb9c101e40 0x7fcb9c108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.858+0000 7fcba19ff700 1 --2- 172.21.15.133:0/3634294436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb9c1002c0 0x7fcb9c1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.858+0000 7fcba19ff700 1 --2- 172.21.15.133:0/3634294436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb9c1014b0 0x7fcb9c101900 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.858+0000 7fcba19ff700 1 -- 172.21.15.133:0/3634294436 >> 172.21.15.133:0/3634294436 conn(0x7fcb9c0fb8a0 msgr2=0x7fcb9c0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:04.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.858+0000 7fcba19ff700 1 -- 172.21.15.133:0/3634294436 shutdown_connections 2023-11-27T15:22:04.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.858+0000 7fcba19ff700 1 -- 172.21.15.133:0/3634294436 wait complete. 2023-11-27T15:22:04.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.859+0000 7fcba19ff700 1 Processor -- start 2023-11-27T15:22:04.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.859+0000 7fcba19ff700 1 -- start start 2023-11-27T15:22:04.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.859+0000 7fcba19ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb9c1002c0 0x7fcb9c196050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:04.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.860+0000 7fcba19ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb9c1014b0 0x7fcb9c196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:04.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.860+0000 7fcba19ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb9c101e40 0x7fcb9c19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:04.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.860+0000 7fcba19ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c19dbd0 con 0x7fcb9c1014b0 2023-11-27T15:22:04.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.860+0000 7fcba19ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c19dd10 con 0x7fcb9c1002c0 2023-11-27T15:22:04.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.860+0000 7fcba19ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c19dea0 con 0x7fcb9c101e40 2023-11-27T15:22:04.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.860+0000 7fcb9a7fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb9c1014b0 0x7fcb9c196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:04.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.860+0000 7fcb9a7fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb9c1014b0 0x7fcb9c196590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59628/0 (socket says 172.21.15.133:59628) 2023-11-27T15:22:04.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.860+0000 7fcb9affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb9c1002c0 0x7fcb9c196050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:04.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.860+0000 7fcb9a7fc700 1 -- 172.21.15.133:0/1999987465 learned_addr learned my addr 172.21.15.133:0/1999987465 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:04.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.860+0000 7fcb9b7fe700 1 --2- 172.21.15.133:0/1999987465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb9c101e40 0x7fcb9c19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:04.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.860+0000 7fcb9a7fc700 1 -- 172.21.15.133:0/1999987465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb9c101e40 msgr2=0x7fcb9c19d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:04.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.860+0000 7fcb9a7fc700 1 --2- 172.21.15.133:0/1999987465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb9c101e40 0x7fcb9c19d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.869 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.860+0000 7fcb9a7fc700 1 -- 172.21.15.133:0/1999987465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb9c1002c0 msgr2=0x7fcb9c196050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:04.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.860+0000 7fcb9a7fc700 1 --2- 172.21.15.133:0/1999987465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb9c1002c0 0x7fcb9c196050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:04.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.860+0000 7fcb9a7fc700 1 -- 172.21.15.133:0/1999987465 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb900095f0 con 0x7fcb9c1014b0 2023-11-27T15:22:04.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.860+0000 7fcb9affd700 1 --2- 172.21.15.133:0/1999987465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb9c1002c0 0x7fcb9c196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:04.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.861+0000 7fcb9a7fc700 1 --2- 172.21.15.133:0/1999987465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb9c1014b0 0x7fcb9c196590 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000b620 tx=0x7fcb90005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:04.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.861+0000 7fcba09fd700 1 -- 172.21.15.133:0/1999987465 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9001d070 con 0x7fcb9c1014b0 2023-11-27T15:22:04.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.861+0000 7fcba09fd700 1 -- 172.21.15.133:0/1999987465 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb90022470 con 0x7fcb9c1014b0 2023-11-27T15:22:04.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.861+0000 7fcba19ff700 1 -- 172.21.15.133:0/1999987465 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb9c10bc70 con 0x7fcb9c1014b0 2023-11-27T15:22:04.870 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.861+0000 7fcba09fd700 1 -- 172.21.15.133:0/1999987465 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90017650 con 0x7fcb9c1014b0 2023-11-27T15:22:04.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.861+0000 7fcba19ff700 1 -- 172.21.15.133:0/1999987465 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb9c10bae0 con 0x7fcb9c1014b0 2023-11-27T15:22:04.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.862+0000 7fcba19ff700 1 -- 172.21.15.133:0/1999987465 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb9c1071f0 con 0x7fcb9c1014b0 2023-11-27T15:22:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.864+0000 7fcba09fd700 1 -- 172.21.15.133:0/1999987465 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fcb900225e0 con 0x7fcb9c1014b0 2023-11-27T15:22:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.865+0000 7fcba09fd700 1 --2- 172.21.15.133:0/1999987465 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcb840a11e0 0x7fcb840a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.865+0000 7fcba09fd700 1 -- 172.21.15.133:0/1999987465 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fcb900ce380 con 0x7fcb9c1014b0 2023-11-27T15:22:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.865+0000 7fcb9affd700 1 --2- 172.21.15.133:0/1999987465 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcb840a11e0 0x7fcb840a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:04.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.866+0000 7fcb9affd700 1 --2- 172.21.15.133:0/1999987465 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcb840a11e0 0x7fcb840a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c101310 tx=0x7fcb8c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:04.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:04.869+0000 7fcba09fd700 1 -- 172.21.15.133:0/1999987465 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcb900976b0 con 0x7fcb9c1014b0 2023-11-27T15:22:05.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:04 smithi133 ceph-mon[97860]: pgmap v323: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-11-27T15:22:05.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:04 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2114379779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:22:05.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:04 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/474837528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:22:05.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.068+0000 7fcba19ff700 1 -- 172.21.15.133:0/1999987465 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcb9c061700 con 0x7fcb9c1014b0 2023-11-27T15:22:05.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.069+0000 7fcba09fd700 1 -- 172.21.15.133:0/1999987465 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcb9009acd0 con 0x7fcb9c1014b0 2023-11-27T15:22:05.074 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:22:05.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.071+0000 7fcba19ff700 1 -- 172.21.15.133:0/1999987465 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcb840a11e0 msgr2=0x7fcb840a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:05.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.071+0000 7fcba19ff700 1 --2- 172.21.15.133:0/1999987465 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcb840a11e0 0x7fcb840a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c101310 tx=0x7fcb8c009260 comp rx=0 tx=0).stop 2023-11-27T15:22:05.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.071+0000 7fcba19ff700 1 -- 172.21.15.133:0/1999987465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb9c1014b0 msgr2=0x7fcb9c196590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:05.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.071+0000 7fcba19ff700 1 --2- 172.21.15.133:0/1999987465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb9c1014b0 0x7fcb9c196590 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000b620 tx=0x7fcb90005740 comp rx=0 tx=0).stop 2023-11-27T15:22:05.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.071+0000 7fcba19ff700 1 -- 172.21.15.133:0/1999987465 shutdown_connections 2023-11-27T15:22:05.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.071+0000 7fcba19ff700 1 --2- 172.21.15.133:0/1999987465 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcb840a11e0 0x7fcb840a3690 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.071+0000 7fcba19ff700 1 --2- 172.21.15.133:0/1999987465 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb9c101e40 0x7fcb9c19d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.071+0000 7fcba19ff700 1 --2- 172.21.15.133:0/1999987465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb9c1002c0 0x7fcb9c196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.071+0000 7fcba19ff700 1 --2- 172.21.15.133:0/1999987465 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb9c1014b0 0x7fcb9c196590 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.071+0000 7fcba19ff700 1 -- 172.21.15.133:0/1999987465 >> 172.21.15.133:0/1999987465 conn(0x7fcb9c0fb8a0 msgr2=0x7fcb9c0fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:05.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.071+0000 7fcba19ff700 1 -- 172.21.15.133:0/1999987465 shutdown_connections 2023-11-27T15:22:05.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.072+0000 7fcba19ff700 1 -- 172.21.15.133:0/1999987465 wait complete. 2023-11-27T15:22:05.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:04 smithi177 ceph-mon[131504]: pgmap v323: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-11-27T15:22:05.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:04 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2114379779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:22:05.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:04 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/474837528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:22:05.096 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:22:05.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:04 smithi174 ceph-mon[131110]: pgmap v323: 129 pgs: 129 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-11-27T15:22:05.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:04 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2114379779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:22:05.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:04 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/474837528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:22:05.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.289+0000 7f8ca5fff700 1 -- 172.21.15.133:0/3266875464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ca0101480 msgr2=0x7f8ca01018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:05.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.289+0000 7f8ca5fff700 1 --2- 172.21.15.133:0/3266875464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ca0101480 0x7f8ca01018d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8c94009990 tx=0x7f8c94009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.289+0000 7f8ca5fff700 1 -- 172.21.15.133:0/3266875464 shutdown_connections 2023-11-27T15:22:05.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.289+0000 7f8ca5fff700 1 --2- 172.21.15.133:0/3266875464 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ca0101e10 0x7f8ca01086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.289+0000 7f8ca5fff700 1 --2- 172.21.15.133:0/3266875464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ca0100290 0x7f8ca01006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.289+0000 7f8ca5fff700 1 --2- 172.21.15.133:0/3266875464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ca0101480 0x7f8ca01018d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.289+0000 7f8ca5fff700 1 -- 172.21.15.133:0/3266875464 >> 172.21.15.133:0/3266875464 conn(0x7f8ca00fb870 msgr2=0x7f8ca00fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:05.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.289+0000 7f8ca5fff700 1 -- 172.21.15.133:0/3266875464 shutdown_connections 2023-11-27T15:22:05.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.289+0000 7f8ca5fff700 1 -- 172.21.15.133:0/3266875464 wait complete. 2023-11-27T15:22:05.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.290+0000 7f8ca5fff700 1 Processor -- start 2023-11-27T15:22:05.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.290+0000 7f8ca5fff700 1 -- start start 2023-11-27T15:22:05.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.291+0000 7f8ca5fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ca0100290 0x7f8ca0195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:05.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.291+0000 7f8ca5fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ca0101480 0x7f8ca0196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:05.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.291+0000 7f8ca5fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ca0101e10 0x7f8ca019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.291+0000 7f8ca5fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca019db50 con 0x7f8ca0101e10 2023-11-27T15:22:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.291+0000 7f8ca5fff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca019dc90 con 0x7f8ca0101480 2023-11-27T15:22:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.291+0000 7f8ca5fff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca019de20 con 0x7f8ca0100290 2023-11-27T15:22:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.291+0000 7f8c9f7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ca0100290 0x7f8ca0195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:05.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.291+0000 7f8c9ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ca0101e10 0x7f8ca019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:05.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.291+0000 7f8c9f7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ca0100290 0x7f8ca0195fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:58462/0 (socket says 172.21.15.133:58462) 2023-11-27T15:22:05.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.291+0000 7f8c9f7fe700 1 -- 172.21.15.133:0/3129063037 learned_addr learned my addr 172.21.15.133:0/3129063037 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:05.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.291+0000 7f8c9effd700 1 --2- 172.21.15.133:0/3129063037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ca0101480 0x7f8ca0196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:05.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.292+0000 7f8c9f7fe700 1 -- 172.21.15.133:0/3129063037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ca0101480 msgr2=0x7f8ca0196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:05.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.292+0000 7f8c9f7fe700 1 --2- 172.21.15.133:0/3129063037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ca0101480 0x7f8ca0196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.292+0000 7f8c9f7fe700 1 -- 172.21.15.133:0/3129063037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ca0101e10 msgr2=0x7f8ca019d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:05.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.292+0000 7f8c9f7fe700 1 --2- 172.21.15.133:0/3129063037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ca0101e10 0x7f8ca019d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.292+0000 7f8c9f7fe700 1 -- 172.21.15.133:0/3129063037 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c940095f0 con 0x7f8ca0100290 2023-11-27T15:22:05.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.292+0000 7f8c9f7fe700 1 --2- 172.21.15.133:0/3129063037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ca0100290 0x7f8ca0195fd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000d980 tx=0x7f8c9000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:05.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.292+0000 7f8c9cff9700 1 -- 172.21.15.133:0/3129063037 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c90004020 con 0x7f8ca0100290 2023-11-27T15:22:05.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.292+0000 7f8ca5fff700 1 -- 172.21.15.133:0/3129063037 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca010bc50 con 0x7f8ca0100290 2023-11-27T15:22:05.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.292+0000 7f8c9cff9700 1 -- 172.21.15.133:0/3129063037 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c9000f690 con 0x7f8ca0100290 2023-11-27T15:22:05.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.292+0000 7f8c9cff9700 1 -- 172.21.15.133:0/3129063037 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c90003a60 con 0x7f8ca0100290 2023-11-27T15:22:05.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.292+0000 7f8ca5fff700 1 -- 172.21.15.133:0/3129063037 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca010bac0 con 0x7f8ca0100290 2023-11-27T15:22:05.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.293+0000 7f8ca5fff700 1 -- 172.21.15.133:0/3129063037 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca0107210 con 0x7f8ca0100290 2023-11-27T15:22:05.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.295+0000 7f8c9cff9700 1 -- 172.21.15.133:0/3129063037 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f8c9000fb00 con 0x7f8ca0100290 2023-11-27T15:22:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.296+0000 7f8c9cff9700 1 --2- 172.21.15.133:0/3129063037 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8c880a11e0 0x7f8c880a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:05.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.297+0000 7f8c9cff9700 1 -- 172.21.15.133:0/3129063037 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f8c900cccd0 con 0x7f8ca0100290 2023-11-27T15:22:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.297+0000 7f8c9effd700 1 --2- 172.21.15.133:0/3129063037 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8c880a11e0 0x7f8c880a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.297+0000 7f8c9effd700 1 --2- 172.21.15.133:0/3129063037 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8c880a11e0 0x7f8c880a3690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400b2e0 tx=0x7f8c94005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:05.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.299+0000 7f8c9cff9700 1 -- 172.21.15.133:0/3129063037 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8c90095f10 con 0x7f8ca0100290 2023-11-27T15:22:05.548 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.543+0000 7f8ca5fff700 1 -- 172.21.15.133:0/3129063037 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ca002cfe0 con 0x7f8ca0100290 2023-11-27T15:22:05.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.544+0000 7f8c9cff9700 1 -- 172.21.15.133:0/3129063037 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10829 (secure 0 0 0) 0x7f8c90099530 con 0x7f8ca0100290 2023-11-27T15:22:05.549 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:05.550 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14493,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2381290259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2381290259},{"type":"v1","addr":"172.21.15.133:6839","nonce":2381290259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1704875895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1704875895},{"type":"v1","addr":"172.21.15.133:6841","nonce":1704875895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:20:33.201023+0000","modified":"2023-11-27T15:21:04.794526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24521,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3492215689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3492215689},{"type":"v1","addr":"172.21.15.177:6837","nonce":3492215689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6839/2781994447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2781994447},{"type":"v1","addr":"172.21.15.174:6839","nonce":2781994447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:22:05.552 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.546+0000 7f8ca5fff700 1 -- 172.21.15.133:0/3129063037 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8c880a11e0 msgr2=0x7f8c880a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:05.552 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.546+0000 7f8ca5fff700 1 --2- 172.21.15.133:0/3129063037 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8c880a11e0 0x7f8c880a3690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400b2e0 tx=0x7f8c94005fb0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.552 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.546+0000 7f8ca5fff700 1 -- 172.21.15.133:0/3129063037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ca0100290 msgr2=0x7f8ca0195fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:05.553 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.546+0000 7f8ca5fff700 1 --2- 172.21.15.133:0/3129063037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ca0100290 0x7f8ca0195fd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000d980 tx=0x7f8c9000dc90 comp rx=0 tx=0).stop 2023-11-27T15:22:05.553 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.547+0000 7f8ca5fff700 1 -- 172.21.15.133:0/3129063037 shutdown_connections 2023-11-27T15:22:05.553 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.547+0000 7f8ca5fff700 1 --2- 172.21.15.133:0/3129063037 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8c880a11e0 0x7f8c880a3690 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.553 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.547+0000 7f8ca5fff700 1 --2- 172.21.15.133:0/3129063037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8ca0101e10 0x7f8ca019d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.547+0000 7f8ca5fff700 1 --2- 172.21.15.133:0/3129063037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ca0100290 0x7f8ca0195fd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.547+0000 7f8ca5fff700 1 --2- 172.21.15.133:0/3129063037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ca0101480 0x7f8ca0196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.547+0000 7f8ca5fff700 1 -- 172.21.15.133:0/3129063037 >> 172.21.15.133:0/3129063037 conn(0x7f8ca00fb870 msgr2=0x7f8ca00fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:05.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.547+0000 7f8ca5fff700 1 -- 172.21.15.133:0/3129063037 shutdown_connections 2023-11-27T15:22:05.555 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.547+0000 7f8ca5fff700 1 -- 172.21.15.133:0/3129063037 wait complete. 2023-11-27T15:22:05.555 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 23 2023-11-27T15:22:05.571 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:22:05.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.766+0000 7f91347ff700 1 -- 172.21.15.133:0/4229444647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f912c10c860 msgr2=0x7f912c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:05.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.766+0000 7f91347ff700 1 --2- 172.21.15.133:0/4229444647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f912c10c860 0x7f912c10ec40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f912400b0a0 tx=0x7f912400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.767+0000 7f91347ff700 1 -- 172.21.15.133:0/4229444647 shutdown_connections 2023-11-27T15:22:05.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.767+0000 7f91347ff700 1 --2- 172.21.15.133:0/4229444647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f912c10c860 0x7f912c10ec40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.767+0000 7f91347ff700 1 --2- 172.21.15.133:0/4229444647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912c074b30 0x7f912c072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.767+0000 7f91347ff700 1 --2- 172.21.15.133:0/4229444647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f912c073550 0x7f912c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.767+0000 7f91347ff700 1 -- 172.21.15.133:0/4229444647 >> 172.21.15.133:0/4229444647 conn(0x7f912c0fb870 msgr2=0x7f912c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:05.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.767+0000 7f91347ff700 1 -- 172.21.15.133:0/4229444647 shutdown_connections 2023-11-27T15:22:05.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.767+0000 7f91347ff700 1 -- 172.21.15.133:0/4229444647 wait complete. 2023-11-27T15:22:05.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.768+0000 7f91347ff700 1 Processor -- start 2023-11-27T15:22:05.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.768+0000 7f91347ff700 1 -- start start 2023-11-27T15:22:05.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.768+0000 7f91347ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f912c073550 0x7f912c19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:05.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.768+0000 7f91347ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912c074b30 0x7f912c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:05.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.768+0000 7f91347ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f912c10c860 0x7f912c1a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:05.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.768+0000 7f91347ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c1a4220 con 0x7f912c10c860 2023-11-27T15:22:05.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.768+0000 7f91347ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c1a4360 con 0x7f912c073550 2023-11-27T15:22:05.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.768+0000 7f91347ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c1a44f0 con 0x7f912c074b30 2023-11-27T15:22:05.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.768+0000 7f913259b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f912c073550 0x7f912c19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:05.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.768+0000 7f913259b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f912c073550 0x7f912c19c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:45154/0 (socket says 172.21.15.133:45154) 2023-11-27T15:22:05.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.768+0000 7f913259b700 1 -- 172.21.15.133:0/1815915282 learned_addr learned my addr 172.21.15.133:0/1815915282 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:05.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.769+0000 7f9131d9a700 1 --2- 172.21.15.133:0/1815915282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912c074b30 0x7f912c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:05.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.769+0000 7f9132d9c700 1 --2- 172.21.15.133:0/1815915282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f912c10c860 0x7f912c1a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:05.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.769+0000 7f9131d9a700 1 -- 172.21.15.133:0/1815915282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f912c073550 msgr2=0x7f912c19c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:05.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.769+0000 7f9131d9a700 1 --2- 172.21.15.133:0/1815915282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f912c073550 0x7f912c19c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.769+0000 7f9131d9a700 1 -- 172.21.15.133:0/1815915282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f912c10c860 msgr2=0x7f912c1a3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:05.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.769+0000 7f9131d9a700 1 --2- 172.21.15.133:0/1815915282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f912c10c860 0x7f912c1a3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.769+0000 7f9131d9a700 1 -- 172.21.15.133:0/1815915282 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c009520 con 0x7f912c074b30 2023-11-27T15:22:05.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.769+0000 7f9132d9c700 1 --2- 172.21.15.133:0/1815915282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f912c10c860 0x7f912c1a3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:05.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.769+0000 7f9131d9a700 1 --2- 172.21.15.133:0/1815915282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912c074b30 0x7f912c19cbe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f911c00e9c0 tx=0x7f911c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:05.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.769+0000 7f91237fe700 1 -- 172.21.15.133:0/1815915282 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c00cb70 con 0x7f912c074b30 2023-11-27T15:22:05.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.769+0000 7f91347ff700 1 -- 172.21.15.133:0/1815915282 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9124009d00 con 0x7f912c074b30 2023-11-27T15:22:05.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.770+0000 7f91237fe700 1 -- 172.21.15.133:0/1815915282 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f911c00ee60 con 0x7f912c074b30 2023-11-27T15:22:05.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.770+0000 7f91237fe700 1 -- 172.21.15.133:0/1815915282 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c018680 con 0x7f912c074b30 2023-11-27T15:22:05.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.770+0000 7f91347ff700 1 -- 172.21.15.133:0/1815915282 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912c112270 con 0x7f912c074b30 2023-11-27T15:22:05.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.771+0000 7f91347ff700 1 -- 172.21.15.133:0/1815915282 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912c1023b0 con 0x7f912c074b30 2023-11-27T15:22:05.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.773+0000 7f91237fe700 1 -- 172.21.15.133:0/1815915282 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f911c010430 con 0x7f912c074b30 2023-11-27T15:22:05.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.774+0000 7f91237fe700 1 --2- 172.21.15.133:0/1815915282 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f91180a11e0 0x7f91180a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:05.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.774+0000 7f91237fe700 1 -- 172.21.15.133:0/1815915282 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f911c014070 con 0x7f912c074b30 2023-11-27T15:22:05.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.774+0000 7f913259b700 1 --2- 172.21.15.133:0/1815915282 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f91180a11e0 0x7f91180a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:05.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.775+0000 7f913259b700 1 --2- 172.21.15.133:0/1815915282 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f91180a11e0 0x7f91180a3690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9128005950 tx=0x7f9128009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:05.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.776+0000 7f91237fe700 1 -- 172.21.15.133:0/1815915282 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f911c096820 con 0x7f912c074b30 2023-11-27T15:22:05.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.982+0000 7f91347ff700 1 -- 172.21.15.133:0/1815915282 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f912c067a60 con 0x7f912c074b30 2023-11-27T15:22:05.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.985+0000 7f91237fe700 1 -- 172.21.15.133:0/1815915282 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24616 (secure 0 0 0) 0x7f911c099e40 con 0x7f912c074b30 2023-11-27T15:22:05.990 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:05.990 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":92,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:22:02.807005+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:22:05.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.987+0000 7f91347ff700 1 -- 172.21.15.133:0/1815915282 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f91180a11e0 msgr2=0x7f91180a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:05.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.987+0000 7f91347ff700 1 --2- 172.21.15.133:0/1815915282 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f91180a11e0 0x7f91180a3690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9128005950 tx=0x7f9128009260 comp rx=0 tx=0).stop 2023-11-27T15:22:05.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.987+0000 7f91347ff700 1 -- 172.21.15.133:0/1815915282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912c074b30 msgr2=0x7f912c19cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:05.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.987+0000 7f91347ff700 1 --2- 172.21.15.133:0/1815915282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912c074b30 0x7f912c19cbe0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f911c00e9c0 tx=0x7f911c00ecd0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.987+0000 7f91347ff700 1 -- 172.21.15.133:0/1815915282 shutdown_connections 2023-11-27T15:22:05.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.987+0000 7f91347ff700 1 --2- 172.21.15.133:0/1815915282 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f91180a11e0 0x7f91180a3690 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.987+0000 7f91347ff700 1 --2- 172.21.15.133:0/1815915282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f912c10c860 0x7f912c1a3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.987+0000 7f91347ff700 1 --2- 172.21.15.133:0/1815915282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f912c073550 0x7f912c19c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.987+0000 7f91347ff700 1 --2- 172.21.15.133:0/1815915282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f912c074b30 0x7f912c19cbe0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:05.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.987+0000 7f91347ff700 1 -- 172.21.15.133:0/1815915282 >> 172.21.15.133:0/1815915282 conn(0x7f912c0fb870 msgr2=0x7f912c0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:05.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.987+0000 7f91347ff700 1 -- 172.21.15.133:0/1815915282 shutdown_connections 2023-11-27T15:22:05.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:05.987+0000 7f91347ff700 1 -- 172.21.15.133:0/1815915282 wait complete. 2023-11-27T15:22:06.011 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:22:06.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:05 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1999987465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:22:06.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:05 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3129063037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:06.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:05 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1999987465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:22:06.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:05 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3129063037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:06.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.203+0000 7fd20ffff700 1 -- 172.21.15.133:0/3623597574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd208073550 msgr2=0x7fd2080739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.203+0000 7fd20ffff700 1 --2- 172.21.15.133:0/3623597574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd208073550 0x7fd2080739c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8009990 tx=0x7fd1f8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.203+0000 7fd20ffff700 1 -- 172.21.15.133:0/3623597574 shutdown_connections 2023-11-27T15:22:06.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.204+0000 7fd20ffff700 1 --2- 172.21.15.133:0/3623597574 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd20810c880 0x7fd20810ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.204+0000 7fd20ffff700 1 --2- 172.21.15.133:0/3623597574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd208074b30 0x7fd208072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.204+0000 7fd20ffff700 1 --2- 172.21.15.133:0/3623597574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd208073550 0x7fd2080739c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.204+0000 7fd20ffff700 1 -- 172.21.15.133:0/3623597574 >> 172.21.15.133:0/3623597574 conn(0x7fd2080fb890 msgr2=0x7fd2080fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:06.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.204+0000 7fd20ffff700 1 -- 172.21.15.133:0/3623597574 shutdown_connections 2023-11-27T15:22:06.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.204+0000 7fd20ffff700 1 -- 172.21.15.133:0/3623597574 wait complete. 2023-11-27T15:22:06.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.205+0000 7fd20ffff700 1 Processor -- start 2023-11-27T15:22:06.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.205+0000 7fd20ffff700 1 -- start start 2023-11-27T15:22:06.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.205+0000 7fd20ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd208073550 0x7fd20819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.205+0000 7fd20ffff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd208074b30 0x7fd20819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.205+0000 7fd20ffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd20810c880 0x7fd2081a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.205+0000 7fd20ffff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2081a41b0 con 0x7fd208073550 2023-11-27T15:22:06.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.205+0000 7fd20ffff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2081a42f0 con 0x7fd20810c880 2023-11-27T15:22:06.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.205+0000 7fd20ffff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2081a4480 con 0x7fd208074b30 2023-11-27T15:22:06.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.205+0000 7fd20dd9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd208073550 0x7fd20819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.205+0000 7fd20dd9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd208073550 0x7fd20819c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59672/0 (socket says 172.21.15.133:59672) 2023-11-27T15:22:06.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.205+0000 7fd20dd9b700 1 -- 172.21.15.133:0/2401259311 learned_addr learned my addr 172.21.15.133:0/2401259311 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:06.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.205+0000 7fd20d59a700 1 --2- 172.21.15.133:0/2401259311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd208074b30 0x7fd20819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.205+0000 7fd20e59c700 1 --2- 172.21.15.133:0/2401259311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd20810c880 0x7fd2081a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.206+0000 7fd20dd9b700 1 -- 172.21.15.133:0/2401259311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd208074b30 msgr2=0x7fd20819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.206+0000 7fd20dd9b700 1 --2- 172.21.15.133:0/2401259311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd208074b30 0x7fd20819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.206+0000 7fd20dd9b700 1 -- 172.21.15.133:0/2401259311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd20810c880 msgr2=0x7fd2081a3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.206+0000 7fd20dd9b700 1 --2- 172.21.15.133:0/2401259311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd20810c880 0x7fd2081a3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.206+0000 7fd20dd9b700 1 -- 172.21.15.133:0/2401259311 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f80095f0 con 0x7fd208073550 2023-11-27T15:22:06.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.206+0000 7fd20dd9b700 1 --2- 172.21.15.133:0/2401259311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd208073550 0x7fd20819c630 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd20400e9f0 tx=0x7fd20400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:06.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.206+0000 7fd1feffd700 1 -- 172.21.15.133:0/2401259311 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204009750 con 0x7fd208073550 2023-11-27T15:22:06.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.206+0000 7fd1feffd700 1 -- 172.21.15.133:0/2401259311 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd20400c7c0 con 0x7fd208073550 2023-11-27T15:22:06.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.206+0000 7fd20ffff700 1 -- 172.21.15.133:0/2401259311 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2081122b0 con 0x7fd208073550 2023-11-27T15:22:06.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.206+0000 7fd1feffd700 1 -- 172.21.15.133:0/2401259311 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2040054d0 con 0x7fd208073550 2023-11-27T15:22:06.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.206+0000 7fd20ffff700 1 -- 172.21.15.133:0/2401259311 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd208112120 con 0x7fd208073550 2023-11-27T15:22:06.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.207+0000 7fd20ffff700 1 -- 172.21.15.133:0/2401259311 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2081022c0 con 0x7fd208073550 2023-11-27T15:22:06.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.209+0000 7fd1feffd700 1 -- 172.21.15.133:0/2401259311 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fd20400c930 con 0x7fd208073550 2023-11-27T15:22:06.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.211+0000 7fd1feffd700 1 --2- 172.21.15.133:0/2401259311 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd1f40a1260 0x7fd1f40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.211+0000 7fd1feffd700 1 -- 172.21.15.133:0/2401259311 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd204014070 con 0x7fd208073550 2023-11-27T15:22:06.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.211+0000 7fd20d59a700 1 --2- 172.21.15.133:0/2401259311 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd1f40a1260 0x7fd1f40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.211+0000 7fd20d59a700 1 --2- 172.21.15.133:0/2401259311 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd1f40a1260 0x7fd1f40a3710 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800b2e0 tx=0x7fd1f8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:06.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.215+0000 7fd1feffd700 1 -- 172.21.15.133:0/2401259311 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd2040961f0 con 0x7fd208073550 2023-11-27T15:22:06.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:05 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1999987465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:22:06.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:05 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3129063037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:06.461 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.457+0000 7fd20ffff700 1 -- 172.21.15.133:0/2401259311 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd208061700 con 0x7fd208073550 2023-11-27T15:22:06.462 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.458+0000 7fd1feffd700 1 -- 172.21.15.133:0/2401259311 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10829 (secure 0 0 0) 0x7fd204099810 con 0x7fd208073550 2023-11-27T15:22:06.463 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:06.463 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14493,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2381290259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2381290259},{"type":"v1","addr":"172.21.15.133:6839","nonce":2381290259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1704875895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1704875895},{"type":"v1","addr":"172.21.15.133:6841","nonce":1704875895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:20:33.201023+0000","modified":"2023-11-27T15:21:04.794526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24521,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3492215689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3492215689},{"type":"v1","addr":"172.21.15.177:6837","nonce":3492215689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6839/2781994447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2781994447},{"type":"v1","addr":"172.21.15.174:6839","nonce":2781994447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:22:06.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.460+0000 7fd20ffff700 1 -- 172.21.15.133:0/2401259311 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd1f40a1260 msgr2=0x7fd1f40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.460+0000 7fd20ffff700 1 --2- 172.21.15.133:0/2401259311 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd1f40a1260 0x7fd1f40a3710 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800b2e0 tx=0x7fd1f8005fb0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.460+0000 7fd20ffff700 1 -- 172.21.15.133:0/2401259311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd208073550 msgr2=0x7fd20819c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.460+0000 7fd20ffff700 1 --2- 172.21.15.133:0/2401259311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd208073550 0x7fd20819c630 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd20400e9f0 tx=0x7fd20400ed00 comp rx=0 tx=0).stop 2023-11-27T15:22:06.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.460+0000 7fd20ffff700 1 -- 172.21.15.133:0/2401259311 shutdown_connections 2023-11-27T15:22:06.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.460+0000 7fd20ffff700 1 --2- 172.21.15.133:0/2401259311 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd1f40a1260 0x7fd1f40a3710 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.460+0000 7fd20ffff700 1 --2- 172.21.15.133:0/2401259311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd20810c880 0x7fd2081a3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.460+0000 7fd20ffff700 1 --2- 172.21.15.133:0/2401259311 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd208073550 0x7fd20819c630 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.460+0000 7fd20ffff700 1 --2- 172.21.15.133:0/2401259311 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd208074b30 0x7fd20819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.460+0000 7fd20ffff700 1 -- 172.21.15.133:0/2401259311 >> 172.21.15.133:0/2401259311 conn(0x7fd2080fb890 msgr2=0x7fd2080fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:06.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.461+0000 7fd20ffff700 1 -- 172.21.15.133:0/2401259311 shutdown_connections 2023-11-27T15:22:06.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.461+0000 7fd20ffff700 1 -- 172.21.15.133:0/2401259311 wait complete. 2023-11-27T15:22:06.467 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 23 2023-11-27T15:22:06.485 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-11-27T15:20:33.201023+0000', 'modified': '2023-11-27T15:21:04.794526+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24560, 'mds_1': 24521, 'mds_2': 24520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24491': {'gid': 24491, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6835/637171798', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 637171798}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 637171798}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24496': {'gid': 24496, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.177:6835/3620466749', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6834', 'nonce': 3620466749}, {'type': 'v1', 'addr': '172.21.15.177:6835', 'nonce': 3620466749}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24505': {'gid': 24505, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.177:6837/3492215689', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6836', 'nonce': 3492215689}, {'type': 'v1', 'addr': '172.21.15.177:6837', 'nonce': 3492215689}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24520': {'gid': 24520, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.177:6839/1463687598', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6838', 'nonce': 1463687598}, {'type': 'v1', 'addr': '172.21.15.177:6839', 'nonce': 1463687598}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24521': {'gid': 24521, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.174:6837/1955090548', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 1955090548}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 1955090548}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24560': {'gid': 24560, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.174:6839/2781994447', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 2781994447}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 2781994447}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-27T15:22:06.485 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-11-27T15:22:06.485 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-11-27T15:22:06.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.679+0000 7fb559dff700 1 -- 172.21.15.133:0/161857464 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb554101db0 msgr2=0x7fb554108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.679+0000 7fb559dff700 1 --2- 172.21.15.133:0/161857464 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb554101db0 0x7fb554108670 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009f90 tx=0x7fb54c00b330 comp rx=0 tx=0).stop 2023-11-27T15:22:06.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.679+0000 7fb559dff700 1 -- 172.21.15.133:0/161857464 shutdown_connections 2023-11-27T15:22:06.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.679+0000 7fb559dff700 1 --2- 172.21.15.133:0/161857464 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb554101db0 0x7fb554108670 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.679+0000 7fb559dff700 1 --2- 172.21.15.133:0/161857464 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb554100230 0x7fb554100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.679+0000 7fb559dff700 1 --2- 172.21.15.133:0/161857464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb554101420 0x7fb554101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.679+0000 7fb559dff700 1 -- 172.21.15.133:0/161857464 >> 172.21.15.133:0/161857464 conn(0x7fb5540fb810 msgr2=0x7fb5540fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:06.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.679+0000 7fb559dff700 1 -- 172.21.15.133:0/161857464 shutdown_connections 2023-11-27T15:22:06.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.679+0000 7fb559dff700 1 -- 172.21.15.133:0/161857464 wait complete. 2023-11-27T15:22:06.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.680+0000 7fb559dff700 1 Processor -- start 2023-11-27T15:22:06.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.680+0000 7fb559dff700 1 -- start start 2023-11-27T15:22:06.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.680+0000 7fb559dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb554100230 0x7fb554198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.680+0000 7fb559dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb554101420 0x7fb5541985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.680+0000 7fb559dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb554101db0 0x7fb55419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.681+0000 7fb559dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55419fc00 con 0x7fb554100230 2023-11-27T15:22:06.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.681+0000 7fb559dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55419fd40 con 0x7fb554101db0 2023-11-27T15:22:06.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.681+0000 7fb559dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55419fed0 con 0x7fb554101420 2023-11-27T15:22:06.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.681+0000 7fb5537fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb554100230 0x7fb554198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.681+0000 7fb5537fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb554100230 0x7fb554198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59696/0 (socket says 172.21.15.133:59696) 2023-11-27T15:22:06.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.681+0000 7fb5537fe700 1 -- 172.21.15.133:0/3654499761 learned_addr learned my addr 172.21.15.133:0/3654499761 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:06.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.681+0000 7fb553fff700 1 --2- 172.21.15.133:0/3654499761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb554101db0 0x7fb55419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.681+0000 7fb54bfff700 1 --2- 172.21.15.133:0/3654499761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb554101420 0x7fb5541985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.681+0000 7fb5537fe700 1 -- 172.21.15.133:0/3654499761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb554101420 msgr2=0x7fb5541985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.681+0000 7fb5537fe700 1 --2- 172.21.15.133:0/3654499761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb554101420 0x7fb5541985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.681+0000 7fb5537fe700 1 -- 172.21.15.133:0/3654499761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb554101db0 msgr2=0x7fb55419f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.681+0000 7fb5537fe700 1 --2- 172.21.15.133:0/3654499761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb554101db0 0x7fb55419f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.681+0000 7fb5537fe700 1 -- 172.21.15.133:0/3654499761 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5440095f0 con 0x7fb554100230 2023-11-27T15:22:06.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.681+0000 7fb553fff700 1 --2- 172.21.15.133:0/3654499761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb554101db0 0x7fb55419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:06.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.681+0000 7fb54bfff700 1 --2- 172.21.15.133:0/3654499761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb554101420 0x7fb5541985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:22:06.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.681+0000 7fb5537fe700 1 --2- 172.21.15.133:0/3654499761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb554100230 0x7fb554198080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb54400ebf0 tx=0x7fb54400ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:06.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.682+0000 7fb5517fa700 1 -- 172.21.15.133:0/3654499761 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544009a10 con 0x7fb554100230 2023-11-27T15:22:06.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.682+0000 7fb559dff700 1 -- 172.21.15.133:0/3654499761 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb54c009c40 con 0x7fb554100230 2023-11-27T15:22:06.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.682+0000 7fb5517fa700 1 -- 172.21.15.133:0/3654499761 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb54400cd40 con 0x7fb554100230 2023-11-27T15:22:06.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.682+0000 7fb5517fa700 1 -- 172.21.15.133:0/3654499761 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544004a50 con 0x7fb554100230 2023-11-27T15:22:06.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.682+0000 7fb559dff700 1 -- 172.21.15.133:0/3654499761 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55410bb40 con 0x7fb554100230 2023-11-27T15:22:06.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.685+0000 7fb5517fa700 1 -- 172.21.15.133:0/3654499761 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fb544010430 con 0x7fb554100230 2023-11-27T15:22:06.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.686+0000 7fb56d692b80 1 Processor -- start 2023-11-27T15:22:06.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.686+0000 7fb56d692b80 1 -- start start 2023-11-27T15:22:06.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.686+0000 7fb5517fa700 1 --2- 172.21.15.133:0/3654499761 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb5400a11e0 0x7fb5400a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.686+0000 7fb56d692b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d2bb0 0x55dba27d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.687+0000 7fb56d692b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d3500 0x55dba27d3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.687+0000 7fb5517fa700 1 -- 172.21.15.133:0/3654499761 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb544014070 con 0x7fb554100230 2023-11-27T15:22:06.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.687+0000 7fb56d692b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d40c0 0x55dba27d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.687+0000 7fb56d692b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba26d8010 con 0x55dba27d40c0 2023-11-27T15:22:06.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.687+0000 7fb56d692b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba272a850 con 0x55dba27d3500 2023-11-27T15:22:06.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.687+0000 7fb56d692b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba2723ce0 con 0x55dba27d2bb0 2023-11-27T15:22:06.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.687+0000 7fb553fff700 1 --2- 172.21.15.133:0/3654499761 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb5400a11e0 0x7fb5400a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.687+0000 7fb553fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d40c0 0x55dba27d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.687+0000 7fb5537fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d2bb0 0x55dba27d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.687+0000 7fb5537fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d2bb0 0x55dba27d2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:58508/0 (socket says 172.21.15.133:58508) 2023-11-27T15:22:06.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.687+0000 7fb5537fe700 1 -- 172.21.15.133:0/4133380755 learned_addr learned my addr 172.21.15.133:0/4133380755 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:06.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.687+0000 7fb54bfff700 1 --2- 172.21.15.133:0/4133380755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d3500 0x55dba27d3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.687+0000 7fb5537fe700 1 -- 172.21.15.133:0/4133380755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d3500 msgr2=0x55dba27d3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:06.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.687+0000 7fb5537fe700 1 --2- 172.21.15.133:0/4133380755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d3500 0x55dba27d3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.687+0000 7fb5537fe700 1 -- 172.21.15.133:0/4133380755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d40c0 msgr2=0x55dba27d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:06.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.687+0000 7fb5537fe700 1 --2- 172.21.15.133:0/4133380755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d40c0 0x55dba27d4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.687+0000 7fb5537fe700 1 -- 172.21.15.133:0/4133380755 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dba25a55e0 con 0x55dba27d2bb0 2023-11-27T15:22:06.694 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.687+0000 7fb54bfff700 1 --2- 172.21.15.133:0/4133380755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d3500 0x55dba27d3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:22:06.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.688+0000 7fb5537fe700 1 --2- 172.21.15.133:0/4133380755 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d2bb0 0x55dba27d2fc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb5440968d0 tx=0x7fb544099fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdb4b09ccca20cf server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:06.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.688+0000 7fb553fff700 1 --2- 172.21.15.133:0/3654499761 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb5400a11e0 0x7fb5400a3690 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0074d0 tx=0x7fb54c008c00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:06.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.688+0000 7fb549ffb700 1 -- 172.21.15.133:0/4133380755 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54401f310 con 0x55dba27d2bb0 2023-11-27T15:22:06.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.688+0000 7fb549ffb700 1 -- 172.21.15.133:0/4133380755 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5440cd070 con 0x55dba27d2bb0 2023-11-27T15:22:06.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.688+0000 7fb549ffb700 1 -- 172.21.15.133:0/4133380755 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5440cd1e0 con 0x55dba27d2bb0 2023-11-27T15:22:06.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.688+0000 7fb56d692b80 1 -- 172.21.15.133:0/4133380755 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d2bb0 msgr2=0x55dba27d2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.688+0000 7fb56d692b80 1 --2- 172.21.15.133:0/4133380755 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d2bb0 0x55dba27d2fc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb5440968d0 tx=0x7fb544099fe0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.688+0000 7fb56d692b80 1 -- 172.21.15.133:0/4133380755 shutdown_connections 2023-11-27T15:22:06.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.689+0000 7fb56d692b80 1 --2- 172.21.15.133:0/4133380755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d40c0 0x55dba27d4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.689+0000 7fb56d692b80 1 --2- 172.21.15.133:0/4133380755 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d2bb0 0x55dba27d2fc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.689+0000 7fb56d692b80 1 --2- 172.21.15.133:0/4133380755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d3500 0x55dba27d3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.689+0000 7fb56d692b80 1 -- 172.21.15.133:0/4133380755 >> 172.21.15.133:0/4133380755 conn(0x55dba27c8310 msgr2=0x55dba27cd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:06.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.689+0000 7fb56d692b80 1 -- 172.21.15.133:0/4133380755 shutdown_connections 2023-11-27T15:22:06.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.689+0000 7fb56d692b80 1 -- 172.21.15.133:0/4133380755 wait complete. 2023-11-27T15:22:06.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.689+0000 7fb56d692b80 1 -- start start 2023-11-27T15:22:06.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.689+0000 7fb56d692b80 1 Processor -- start 2023-11-27T15:22:06.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.690+0000 7fb56d692b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d2bb0 0x55dba27dc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.690+0000 7fb56d692b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d3500 0x55dba27dcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.690+0000 7fb56d692b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d40c0 0x55dba27e3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.690+0000 7fb56d692b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba26c5890 con 0x55dba27d3500 2023-11-27T15:22:06.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.690+0000 7fb56d692b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba26c5530 con 0x55dba27d40c0 2023-11-27T15:22:06.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.690+0000 7fb56d692b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba25a78d0 con 0x55dba27d2bb0 2023-11-27T15:22:06.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.690+0000 7fb56d692b80 10 client.0 _ll_register_callbacks cb 0x55dba26e3730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:22:06.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.690+0000 7fb54bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d3500 0x55dba27dcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.690+0000 7fb5537fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d2bb0 0x55dba27dc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.690+0000 7fb553fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d40c0 0x55dba27e3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.690+0000 7fb5537fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d2bb0 0x55dba27dc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:58524/0 (socket says 172.21.15.133:58524) 2023-11-27T15:22:06.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.690+0000 7fb54bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d3500 0x55dba27dcb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59718/0 (socket says 172.21.15.133:59718) 2023-11-27T15:22:06.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.690+0000 7fb5537fe700 1 -- 172.21.15.133:0/1867685280 learned_addr learned my addr 172.21.15.133:0/1867685280 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:06.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.690+0000 7fb5537fe700 1 -- 172.21.15.133:0/1867685280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d40c0 msgr2=0x55dba27e3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:06.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.690+0000 7fb5537fe700 1 --2- 172.21.15.133:0/1867685280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d40c0 0x55dba27e3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.690+0000 7fb5537fe700 1 -- 172.21.15.133:0/1867685280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d3500 msgr2=0x55dba27dcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:06.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.690+0000 7fb5537fe700 1 --2- 172.21.15.133:0/1867685280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d3500 0x55dba27dcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.690+0000 7fb5537fe700 1 -- 172.21.15.133:0/1867685280 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55410bb40 con 0x55dba27d2bb0 2023-11-27T15:22:06.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.690+0000 7fb553fff700 1 --2- 172.21.15.133:0/1867685280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d40c0 0x55dba27e3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:06.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.691+0000 7fb5537fe700 1 --2- 172.21.15.133:0/1867685280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d2bb0 0x55dba27dc650 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb5440a22f0 tx=0x7fb544010dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfb8d712017f74b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:06.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.691+0000 7fb532ffd700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:22:06.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.691+0000 7fb532ffd700 1 -- 172.21.15.133:0/1867685280 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54401f310 con 0x55dba27d2bb0 2023-11-27T15:22:06.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.691+0000 7fb532ffd700 1 -- 172.21.15.133:0/1867685280 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb54409a3d0 con 0x55dba27d2bb0 2023-11-27T15:22:06.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.691+0000 7fb56d692b80 1 -- 172.21.15.133:0/1867685280 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dba27c5ad0 con 0x55dba27d2bb0 2023-11-27T15:22:06.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.691+0000 7fb532ffd700 1 -- 172.21.15.133:0/1867685280 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5440cad20 con 0x55dba27d2bb0 2023-11-27T15:22:06.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.691+0000 7fb56d692b80 10 client.25114 fetch_fsmap learned FSMap version 23 2023-11-27T15:22:06.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.691+0000 7fb56d692b80 1 -- 172.21.15.133:0/1867685280 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55dba26ea140 con 0x55dba27d2bb0 2023-11-27T15:22:06.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.691+0000 7fb532ffd700 1 -- 172.21.15.133:0/1867685280 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dba27c5ad0 con 0x55dba27d2bb0 2023-11-27T15:22:06.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.691+0000 7fb532ffd700 1 -- 172.21.15.133:0/1867685280 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb5440993c0 con 0x55dba27d2bb0 2023-11-27T15:22:06.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.692+0000 7fb56d692b80 10 client.25114 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:22:06.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.692+0000 7fb56d692b80 10 client.25114 resolve_mds: resolved name 'i' to daemon mds.i 2023-11-27T15:22:06.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.692+0000 7fb56d692b80 20 client.25114 populate_metadata read hostname 'smithi133' 2023-11-27T15:22:06.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.692+0000 7fb56d692b80 1 --2- 172.21.15.133:0/1867685280 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55dba27e7500 0x55dba27e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.692+0000 7fb56d692b80 4 client.25114 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:22:06.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.692+0000 7fb56d692b80 1 -- 172.21.15.133:0/1867685280 --> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dba26e6050 con 0x55dba27e7500 2023-11-27T15:22:06.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.692+0000 7fb54bfff700 1 --2- 172.21.15.133:0/1867685280 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55dba27e7500 0x55dba27e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.692+0000 7fb532ffd700 10 client.25114 ms_handle_connect on v2:172.21.15.177:6838/1463687598 2023-11-27T15:22:06.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.692+0000 7fb54bfff700 1 --2- 172.21.15.133:0/1867685280 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55dba27e7500 0x55dba27e6b00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=57dfa5c0d8790d86 server_cookie=f38d12ca17689859 in_seq=0 out_seq=0 2023-11-27T15:22:06.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.693+0000 7fb532ffd700 1 -- 172.21.15.133:0/1867685280 <== mds.2 v2:172.21.15.177:6838/1463687598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dba26e6050 con 0x55dba27e7500 2023-11-27T15:22:06.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.693+0000 7fb532ffd700 10 client.25114 handle_command_reply: tid=0 2023-11-27T15:22:06.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.693+0000 7fb56d692b80 1 client.25114 shutdown 2023-11-27T15:22:06.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.694+0000 7fb56d692b80 1 -- 172.21.15.133:0/1867685280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d2bb0 msgr2=0x55dba27dc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.694+0000 7fb56d692b80 1 --2- 172.21.15.133:0/1867685280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d2bb0 0x55dba27dc650 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb5440a22f0 tx=0x7fb544010dd0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.694+0000 7fb56d692b80 1 -- 172.21.15.133:0/1867685280 shutdown_connections 2023-11-27T15:22:06.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.694+0000 7fb56d692b80 1 --2- 172.21.15.133:0/1867685280 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55dba27e7500 0x55dba27e6b00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.694+0000 7fb56d692b80 1 --2- 172.21.15.133:0/1867685280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d40c0 0x55dba27e3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.694+0000 7fb56d692b80 1 --2- 172.21.15.133:0/1867685280 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d2bb0 0x55dba27dc650 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.694+0000 7fb56d692b80 1 --2- 172.21.15.133:0/1867685280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d3500 0x55dba27dcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.694+0000 7fb532ffd700 0 client.25114 ms_handle_reset on v2:172.21.15.177:6838/1463687598 2023-11-27T15:22:06.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.694+0000 7fb56d692b80 1 -- 172.21.15.133:0/1867685280 >> 172.21.15.133:0/1867685280 conn(0x55dba27c8310 msgr2=0x55dba27ccd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:06.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.694+0000 7fb56d692b80 1 -- 172.21.15.133:0/1867685280 shutdown_connections 2023-11-27T15:22:06.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.694+0000 7fb56d692b80 1 -- 172.21.15.133:0/1867685280 wait complete. 2023-11-27T15:22:06.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.694+0000 7fb56d692b80 20 client.25114 trim_cache size 0 max 16384 2023-11-27T15:22:06.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.712+0000 7fb56d692b80 1 Processor -- start 2023-11-27T15:22:06.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.712+0000 7fb56d692b80 1 -- start start 2023-11-27T15:22:06.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.712+0000 7fb56d692b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d2bb0 0x55dba27e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.713+0000 7fb56d692b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d3500 0x55dba27e3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.713+0000 7fb56d692b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d40c0 0x55dba27d7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.713+0000 7fb56d692b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba27dc220 con 0x55dba27d2bb0 2023-11-27T15:22:06.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.713+0000 7fb56d692b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba26e6050 con 0x55dba27d40c0 2023-11-27T15:22:06.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.713+0000 7fb56d692b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba27cead0 con 0x55dba27d3500 2023-11-27T15:22:06.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.713+0000 7fb5537fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d2bb0 0x55dba27e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.713+0000 7fb5537fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d2bb0 0x55dba27e6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59720/0 (socket says 172.21.15.133:59720) 2023-11-27T15:22:06.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.713+0000 7fb5537fe700 1 -- 172.21.15.133:0/519927704 learned_addr learned my addr 172.21.15.133:0/519927704 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:06.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.713+0000 7fb553fff700 1 --2- 172.21.15.133:0/519927704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d40c0 0x55dba27d7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.713+0000 7fb54bfff700 1 --2- 172.21.15.133:0/519927704 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d3500 0x55dba27e3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.713+0000 7fb5537fe700 1 -- 172.21.15.133:0/519927704 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d3500 msgr2=0x55dba27e3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:06.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.713+0000 7fb5537fe700 1 --2- 172.21.15.133:0/519927704 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d3500 0x55dba27e3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.713+0000 7fb5537fe700 1 -- 172.21.15.133:0/519927704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d40c0 msgr2=0x55dba27d7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:06.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.713+0000 7fb5537fe700 1 --2- 172.21.15.133:0/519927704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d40c0 0x55dba27d7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.713+0000 7fb5537fe700 1 -- 172.21.15.133:0/519927704 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dba26e9e90 con 0x55dba27d2bb0 2023-11-27T15:22:06.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.713+0000 7fb553fff700 1 --2- 172.21.15.133:0/519927704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d40c0 0x55dba27d7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:06.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.714+0000 7fb5537fe700 1 --2- 172.21.15.133:0/519927704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d2bb0 0x55dba27e6b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb54409a360 tx=0x7fb54409a990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a12e5fcb5274e88 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:06.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.714+0000 7fb54a7fc700 1 -- 172.21.15.133:0/519927704 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54401f310 con 0x55dba27d2bb0 2023-11-27T15:22:06.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.714+0000 7fb54a7fc700 1 -- 172.21.15.133:0/519927704 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb54409e400 con 0x55dba27d2bb0 2023-11-27T15:22:06.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.714+0000 7fb54a7fc700 1 -- 172.21.15.133:0/519927704 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5440cadd0 con 0x55dba27d2bb0 2023-11-27T15:22:06.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.714+0000 7fb56d692b80 1 -- 172.21.15.133:0/519927704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d2bb0 msgr2=0x55dba27e6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.714+0000 7fb56d692b80 1 --2- 172.21.15.133:0/519927704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d2bb0 0x55dba27e6b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb54409a360 tx=0x7fb54409a990 comp rx=0 tx=0).stop 2023-11-27T15:22:06.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.714+0000 7fb56d692b80 1 -- 172.21.15.133:0/519927704 shutdown_connections 2023-11-27T15:22:06.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.714+0000 7fb56d692b80 1 --2- 172.21.15.133:0/519927704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d40c0 0x55dba27d7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.714+0000 7fb56d692b80 1 --2- 172.21.15.133:0/519927704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d2bb0 0x55dba27e6b00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.714+0000 7fb56d692b80 1 --2- 172.21.15.133:0/519927704 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d3500 0x55dba27e3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.714+0000 7fb56d692b80 1 -- 172.21.15.133:0/519927704 >> 172.21.15.133:0/519927704 conn(0x55dba27c8310 msgr2=0x55dba27db2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:06.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.714+0000 7fb56d692b80 1 -- 172.21.15.133:0/519927704 shutdown_connections 2023-11-27T15:22:06.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.715+0000 7fb56d692b80 1 -- 172.21.15.133:0/519927704 wait complete. 2023-11-27T15:22:06.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.715+0000 7fb56d692b80 1 -- start start 2023-11-27T15:22:06.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.715+0000 7fb56d692b80 1 Processor -- start 2023-11-27T15:22:06.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.716+0000 7fb56d692b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d2bb0 0x55dba27e3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.716+0000 7fb56d692b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d3500 0x55dba27e3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.716+0000 7fb56d692b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d40c0 0x55dba27dfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.716+0000 7fb56d692b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba27e0470 con 0x55dba27d2bb0 2023-11-27T15:22:06.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.716+0000 7fb56d692b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba27e05e0 con 0x55dba27d3500 2023-11-27T15:22:06.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.716+0000 7fb56d692b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dba27e07a0 con 0x55dba27d40c0 2023-11-27T15:22:06.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.716+0000 7fb56d692b80 10 client.0 _ll_register_callbacks cb 0x7fb5440091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:22:06.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.716+0000 7fb5537fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d2bb0 0x55dba27e3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.716+0000 7fb5537fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d2bb0 0x55dba27e3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59726/0 (socket says 172.21.15.133:59726) 2023-11-27T15:22:06.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.716+0000 7fb5537fe700 1 -- 172.21.15.133:0/963355946 learned_addr learned my addr 172.21.15.133:0/963355946 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:06.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.716+0000 7fb553fff700 1 --2- 172.21.15.133:0/963355946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d40c0 0x55dba27dfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.716+0000 7fb54bfff700 1 --2- 172.21.15.133:0/963355946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d3500 0x55dba27e3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.716+0000 7fb5537fe700 1 -- 172.21.15.133:0/963355946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d40c0 msgr2=0x55dba27dfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:06.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.716+0000 7fb5537fe700 1 --2- 172.21.15.133:0/963355946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d40c0 0x55dba27dfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.716+0000 7fb5537fe700 1 -- 172.21.15.133:0/963355946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d3500 msgr2=0x55dba27e3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:06.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.716+0000 7fb5537fe700 1 --2- 172.21.15.133:0/963355946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d3500 0x55dba27e3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.716+0000 7fb5537fe700 1 -- 172.21.15.133:0/963355946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb554108fa0 con 0x55dba27d2bb0 2023-11-27T15:22:06.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.717+0000 7fb5537fe700 1 --2- 172.21.15.133:0/963355946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d2bb0 0x55dba27e3400 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb544096e40 tx=0x7fb5440960e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4414c86e540bd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:06.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.717+0000 7fb532ffd700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:22:06.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.717+0000 7fb532ffd700 1 -- 172.21.15.133:0/963355946 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54401f310 con 0x55dba27d2bb0 2023-11-27T15:22:06.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.717+0000 7fb532ffd700 1 -- 172.21.15.133:0/963355946 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb5440a25a0 con 0x55dba27d2bb0 2023-11-27T15:22:06.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.717+0000 7fb56d692b80 1 -- 172.21.15.133:0/963355946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5440cd070 con 0x55dba27d2bb0 2023-11-27T15:22:06.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.717+0000 7fb532ffd700 1 -- 172.21.15.133:0/963355946 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb544099d30 con 0x55dba27d2bb0 2023-11-27T15:22:06.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.717+0000 7fb56d692b80 10 client.15087 fetch_fsmap learned FSMap version 23 2023-11-27T15:22:06.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.717+0000 7fb56d692b80 1 -- 172.21.15.133:0/963355946 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55dba26ea140 con 0x55dba27d2bb0 2023-11-27T15:22:06.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.717+0000 7fb532ffd700 1 -- 172.21.15.133:0/963355946 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5440cd070 con 0x55dba27d2bb0 2023-11-27T15:22:06.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.718+0000 7fb532ffd700 1 -- 172.21.15.133:0/963355946 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb54409b240 con 0x55dba27d2bb0 2023-11-27T15:22:06.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.718+0000 7fb56d692b80 10 client.15087 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:22:06.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.718+0000 7fb56d692b80 10 client.15087 resolve_mds: resolved name 'i' to daemon mds.i 2023-11-27T15:22:06.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.718+0000 7fb56d692b80 20 client.15087 populate_metadata read hostname 'smithi133' 2023-11-27T15:22:06.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.718+0000 7fb56d692b80 1 --2- 172.21.15.133:0/963355946 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55dba283f650 0x55dba283c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.718+0000 7fb56d692b80 4 client.15087 mds_command: new command op to 24520 tid=0[{"prefix": "status"}] 2023-11-27T15:22:06.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.718+0000 7fb56d692b80 1 -- 172.21.15.133:0/963355946 --> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55dba26f2610 con 0x55dba283f650 2023-11-27T15:22:06.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.718+0000 7fb54bfff700 1 --2- 172.21.15.133:0/963355946 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55dba283f650 0x55dba283c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.719+0000 7fb532ffd700 10 client.15087 ms_handle_connect on v2:172.21.15.177:6838/1463687598 2023-11-27T15:22:06.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.719+0000 7fb54bfff700 1 --2- 172.21.15.133:0/963355946 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55dba283f650 0x55dba283c0f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6aab546c01bb60e1 server_cookie=759426904c90d81f in_seq=0 out_seq=0 2023-11-27T15:22:06.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.719+0000 7fb532ffd700 1 -- 172.21.15.133:0/963355946 <== mds.2 v2:172.21.15.177:6838/1463687598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55dba26f2610 con 0x55dba283f650 2023-11-27T15:22:06.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.719+0000 7fb532ffd700 10 client.15087 handle_command_reply: tid=0 2023-11-27T15:22:06.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.719+0000 7fb56d692b80 1 client.15087 shutdown 2023-11-27T15:22:06.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.719+0000 7fb56d692b80 1 -- 172.21.15.133:0/963355946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d2bb0 msgr2=0x55dba27e3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.719+0000 7fb56d692b80 1 --2- 172.21.15.133:0/963355946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d2bb0 0x55dba27e3400 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb544096e40 tx=0x7fb5440960e0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.719+0000 7fb56d692b80 1 -- 172.21.15.133:0/963355946 shutdown_connections 2023-11-27T15:22:06.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.719+0000 7fb56d692b80 1 --2- 172.21.15.133:0/963355946 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55dba283f650 0x55dba283c0f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.719+0000 7fb56d692b80 1 --2- 172.21.15.133:0/963355946 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dba27d40c0 0x55dba27dfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.719+0000 7fb56d692b80 1 --2- 172.21.15.133:0/963355946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dba27d2bb0 0x55dba27e3400 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.719+0000 7fb532ffd700 0 client.15087 ms_handle_reset on v2:172.21.15.177:6838/1463687598 2023-11-27T15:22:06.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.719+0000 7fb56d692b80 1 --2- 172.21.15.133:0/963355946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dba27d3500 0x55dba27e3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.719+0000 7fb56d692b80 1 -- 172.21.15.133:0/963355946 >> 172.21.15.133:0/963355946 conn(0x55dba27c8310 msgr2=0x55dba27e6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:06.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.719+0000 7fb56d692b80 1 -- 172.21.15.133:0/963355946 shutdown_connections 2023-11-27T15:22:06.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.719+0000 7fb56d692b80 1 -- 172.21.15.133:0/963355946 wait complete. 2023-11-27T15:22:06.729 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.719+0000 7fb56d692b80 20 client.15087 trim_cache size 0 max 16384 2023-11-27T15:22:06.729 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:22:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "5227da4c-8d37-11ee-95a2-87774f69a715", 2023-11-27T15:22:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 2, 2023-11-27T15:22:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24520, 2023-11-27T15:22:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2023-11-27T15:22:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2023-11-27T15:22:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2023-11-27T15:22:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 62.929572835000002, 2023-11-27T15:22:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 23, 2023-11-27T15:22:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 92, 2023-11-27T15:22:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 90, 2023-11-27T15:22:06.730 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 108.62202459 2023-11-27T15:22:06.730 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:22:06.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.720+0000 7fb559dff700 1 -- 172.21.15.133:0/3654499761 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb5400a11e0 msgr2=0x7fb5400a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.720+0000 7fb559dff700 1 --2- 172.21.15.133:0/3654499761 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb5400a11e0 0x7fb5400a3690 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb54c0074d0 tx=0x7fb54c008c00 comp rx=0 tx=0).stop 2023-11-27T15:22:06.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.720+0000 7fb559dff700 1 -- 172.21.15.133:0/3654499761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb554100230 msgr2=0x7fb554198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.731 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.720+0000 7fb559dff700 1 --2- 172.21.15.133:0/3654499761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb554100230 0x7fb554198080 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb54400ebf0 tx=0x7fb54400ef00 comp rx=0 tx=0).stop 2023-11-27T15:22:06.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.720+0000 7fb559dff700 1 -- 172.21.15.133:0/3654499761 shutdown_connections 2023-11-27T15:22:06.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.720+0000 7fb559dff700 1 --2- 172.21.15.133:0/3654499761 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb5400a11e0 0x7fb5400a3690 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.720+0000 7fb559dff700 1 --2- 172.21.15.133:0/3654499761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb554101db0 0x7fb55419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.720+0000 7fb559dff700 1 --2- 172.21.15.133:0/3654499761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb554100230 0x7fb554198080 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.720+0000 7fb559dff700 1 --2- 172.21.15.133:0/3654499761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb554101420 0x7fb5541985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.720+0000 7fb559dff700 1 -- 172.21.15.133:0/3654499761 >> 172.21.15.133:0/3654499761 conn(0x7fb5540fb810 msgr2=0x7fb554106c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:06.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.720+0000 7fb559dff700 1 -- 172.21.15.133:0/3654499761 shutdown_connections 2023-11-27T15:22:06.732 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.720+0000 7fb559dff700 1 -- 172.21.15.133:0/3654499761 wait complete. 2023-11-27T15:22:06.743 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-11-27T15:22:06.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.939+0000 7f6fe15ff700 1 -- 172.21.15.133:0/902296919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fdc074b30 msgr2=0x7f6fdc072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.939+0000 7f6fe15ff700 1 --2- 172.21.15.133:0/902296919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fdc074b30 0x7f6fdc072f80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009990 tx=0x7f6fcc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.939+0000 7f6fe15ff700 1 -- 172.21.15.133:0/902296919 shutdown_connections 2023-11-27T15:22:06.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.939+0000 7f6fe15ff700 1 --2- 172.21.15.133:0/902296919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fdc1080f0 0x7f6fdc10a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.939+0000 7f6fe15ff700 1 --2- 172.21.15.133:0/902296919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fdc074b30 0x7f6fdc072f80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.939+0000 7f6fe15ff700 1 --2- 172.21.15.133:0/902296919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fdc073550 0x7f6fdc0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.939+0000 7f6fe15ff700 1 -- 172.21.15.133:0/902296919 >> 172.21.15.133:0/902296919 conn(0x7f6fdc0fb810 msgr2=0x7f6fdc0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:06.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.939+0000 7f6fe15ff700 1 -- 172.21.15.133:0/902296919 shutdown_connections 2023-11-27T15:22:06.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.939+0000 7f6fe15ff700 1 -- 172.21.15.133:0/902296919 wait complete. 2023-11-27T15:22:06.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.940+0000 7f6fe15ff700 1 Processor -- start 2023-11-27T15:22:06.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.941+0000 7f6fe15ff700 1 -- start start 2023-11-27T15:22:06.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.941+0000 7f6fe15ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fdc073550 0x7f6fdc198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.941+0000 7f6fe15ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fdc074b30 0x7f6fdc198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.941+0000 7f6fe15ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fdc1080f0 0x7f6fdc19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.941+0000 7f6fe15ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc19fdc0 con 0x7f6fdc074b30 2023-11-27T15:22:06.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.941+0000 7f6fe15ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc19ff00 con 0x7f6fdc073550 2023-11-27T15:22:06.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.941+0000 7f6fe15ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc1a0090 con 0x7f6fdc1080f0 2023-11-27T15:22:06.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.941+0000 7f6fd3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fdc074b30 0x7f6fdc198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.941+0000 7f6fd3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fdc074b30 0x7f6fdc198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59738/0 (socket says 172.21.15.133:59738) 2023-11-27T15:22:06.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.941+0000 7f6fd3fff700 1 -- 172.21.15.133:0/946987634 learned_addr learned my addr 172.21.15.133:0/946987634 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:06.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.941+0000 7f6fdb7fe700 1 --2- 172.21.15.133:0/946987634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fdc1080f0 0x7f6fdc19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.941+0000 7f6fdaffd700 1 --2- 172.21.15.133:0/946987634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fdc073550 0x7f6fdc198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.942+0000 7f6fd3fff700 1 -- 172.21.15.133:0/946987634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fdc1080f0 msgr2=0x7f6fdc19f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.942+0000 7f6fd3fff700 1 --2- 172.21.15.133:0/946987634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fdc1080f0 0x7f6fdc19f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.942+0000 7f6fd3fff700 1 -- 172.21.15.133:0/946987634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fdc073550 msgr2=0x7f6fdc198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.942+0000 7f6fd3fff700 1 --2- 172.21.15.133:0/946987634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fdc073550 0x7f6fdc198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.942+0000 7f6fd3fff700 1 -- 172.21.15.133:0/946987634 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd4009c40 con 0x7f6fdc074b30 2023-11-27T15:22:06.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.942+0000 7f6fd3fff700 1 --2- 172.21.15.133:0/946987634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fdc074b30 0x7f6fdc198780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400e970 tx=0x7f6fc400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:06.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.942+0000 7f6fd8ff9700 1 -- 172.21.15.133:0/946987634 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4009790 con 0x7f6fdc074b30 2023-11-27T15:22:06.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.942+0000 7f6fd8ff9700 1 -- 172.21.15.133:0/946987634 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fc400c7c0 con 0x7f6fdc074b30 2023-11-27T15:22:06.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.942+0000 7f6fd8ff9700 1 -- 172.21.15.133:0/946987634 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4003c20 con 0x7f6fdc074b30 2023-11-27T15:22:06.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.942+0000 7f6fe15ff700 1 -- 172.21.15.133:0/946987634 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fcc0095f0 con 0x7f6fdc074b30 2023-11-27T15:22:06.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.942+0000 7f6fe15ff700 1 -- 172.21.15.133:0/946987634 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fdc10de00 con 0x7f6fdc074b30 2023-11-27T15:22:06.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.945+0000 7f6fd8ff9700 1 -- 172.21.15.133:0/946987634 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f6fc400c930 con 0x7f6fdc074b30 2023-11-27T15:22:06.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.947+0000 7f6ff4f2db80 1 Processor -- start 2023-11-27T15:22:06.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.948+0000 7f6ff4f2db80 1 -- start start 2023-11-27T15:22:06.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.948+0000 7f6fd8ff9700 1 --2- 172.21.15.133:0/946987634 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6fc80a11e0 0x7f6fc80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.948+0000 7f6fd8ff9700 1 -- 172.21.15.133:0/946987634 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6fc4014070 con 0x7f6fdc074b30 2023-11-27T15:22:06.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.948+0000 7f6ff4f2db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b61bb0 0x55ec56b61fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.948+0000 7f6fdb7fe700 1 --2- 172.21.15.133:0/946987634 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6fc80a11e0 0x7f6fc80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.948+0000 7f6ff4f2db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b62500 0x55ec56b62ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.948+0000 7f6ff4f2db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b630c0 0x55ec56b63530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.948+0000 7f6ff4f2db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec56a67010 con 0x55ec56b62500 2023-11-27T15:22:06.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.948+0000 7f6ff4f2db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec56ab9850 con 0x55ec56b61bb0 2023-11-27T15:22:06.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.948+0000 7f6ff4f2db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec56ab2ce0 con 0x55ec56b630c0 2023-11-27T15:22:06.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.948+0000 7f6fd3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b62500 0x55ec56b62ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.948+0000 7f6fdaffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b61bb0 0x55ec56b61fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.948+0000 7f6fdb7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b630c0 0x55ec56b63530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.948+0000 7f6fdb7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b630c0 0x55ec56b63530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:58570/0 (socket says 172.21.15.133:58570) 2023-11-27T15:22:06.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.948+0000 7f6fd3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b62500 0x55ec56b62ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59750/0 (socket says 172.21.15.133:59750) 2023-11-27T15:22:06.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.948+0000 7f6fdaffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b61bb0 0x55ec56b61fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:45238/0 (socket says 172.21.15.133:45238) 2023-11-27T15:22:06.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.948+0000 7f6fdb7fe700 1 -- 172.21.15.133:0/2313952166 learned_addr learned my addr 172.21.15.133:0/2313952166 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:06.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.949+0000 7f6fdaffd700 1 -- 172.21.15.133:0/2313952166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b630c0 msgr2=0x55ec56b63530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:06.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.949+0000 7f6fdaffd700 1 --2- 172.21.15.133:0/2313952166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b630c0 0x55ec56b63530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.949+0000 7f6fdaffd700 1 -- 172.21.15.133:0/2313952166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b62500 msgr2=0x55ec56b62ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:06.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.949+0000 7f6fdaffd700 1 --2- 172.21.15.133:0/2313952166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b62500 0x55ec56b62ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.949+0000 7f6fdaffd700 1 -- 172.21.15.133:0/2313952166 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec569345e0 con 0x55ec56b61bb0 2023-11-27T15:22:06.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.949+0000 7f6fdb7fe700 1 --2- 172.21.15.133:0/946987634 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6fc80a11e0 0x7f6fc80a3690 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400fae0 tx=0x7f6fd400fea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:06.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.949+0000 7f6fd3fff700 1 --2- 172.21.15.133:0/2313952166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b62500 0x55ec56b62ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:06.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.949+0000 7f6fdaffd700 1 --2- 172.21.15.133:0/2313952166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b61bb0 0x55ec56b61fc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0098e0 tx=0x7f6fcc005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a62d9f1ce19e4b62 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:06.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.949+0000 7f6fd17fa700 1 -- 172.21.15.133:0/2313952166 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc01d070 con 0x55ec56b61bb0 2023-11-27T15:22:06.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.950+0000 7f6fd17fa700 1 -- 172.21.15.133:0/2313952166 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fcc01e040 con 0x55ec56b61bb0 2023-11-27T15:22:06.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.950+0000 7f6fd17fa700 1 -- 172.21.15.133:0/2313952166 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc004920 con 0x55ec56b61bb0 2023-11-27T15:22:06.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.950+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/2313952166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b61bb0 msgr2=0x55ec56b61fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.950+0000 7f6ff4f2db80 1 --2- 172.21.15.133:0/2313952166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b61bb0 0x55ec56b61fc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc0098e0 tx=0x7f6fcc005ea0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.950+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/2313952166 shutdown_connections 2023-11-27T15:22:06.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.950+0000 7f6ff4f2db80 1 --2- 172.21.15.133:0/2313952166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b630c0 0x55ec56b63530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.950+0000 7f6ff4f2db80 1 --2- 172.21.15.133:0/2313952166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b61bb0 0x55ec56b61fc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.950+0000 7f6ff4f2db80 1 --2- 172.21.15.133:0/2313952166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b62500 0x55ec56b62ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.950+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/2313952166 >> 172.21.15.133:0/2313952166 conn(0x55ec56b57310 msgr2=0x55ec56b5c620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:06.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.950+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/2313952166 shutdown_connections 2023-11-27T15:22:06.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.950+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/2313952166 wait complete. 2023-11-27T15:22:06.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.951+0000 7f6ff4f2db80 1 -- start start 2023-11-27T15:22:06.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.951+0000 7f6ff4f2db80 1 Processor -- start 2023-11-27T15:22:06.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.951+0000 7f6ff4f2db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b61bb0 0x55ec56b6b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.951+0000 7f6ff4f2db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b62500 0x55ec56b6ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.951+0000 7f6ff4f2db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b630c0 0x55ec56b72ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.951+0000 7f6ff4f2db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec56a54890 con 0x55ec56b630c0 2023-11-27T15:22:06.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.951+0000 7f6ff4f2db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec56a54530 con 0x55ec56b62500 2023-11-27T15:22:06.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.951+0000 7f6ff4f2db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec569368d0 con 0x55ec56b61bb0 2023-11-27T15:22:06.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.952+0000 7f6ff4f2db80 10 client.0 _ll_register_callbacks cb 0x55ec56a72730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:22:06.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.952+0000 7f6fdb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b630c0 0x55ec56b72ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.952+0000 7f6fdaffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b61bb0 0x55ec56b6b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.952+0000 7f6fd3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b62500 0x55ec56b6ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.952+0000 7f6fdaffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b61bb0 0x55ec56b6b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:58586/0 (socket says 172.21.15.133:58586) 2023-11-27T15:22:06.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.952+0000 7f6fdaffd700 1 -- 172.21.15.133:0/3484649717 learned_addr learned my addr 172.21.15.133:0/3484649717 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:06.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.952+0000 7f6fdb7fe700 1 -- 172.21.15.133:0/3484649717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b61bb0 msgr2=0x55ec56b6b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:06.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.952+0000 7f6fdb7fe700 1 --2- 172.21.15.133:0/3484649717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b61bb0 0x55ec56b6b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.952+0000 7f6fdb7fe700 1 -- 172.21.15.133:0/3484649717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b62500 msgr2=0x55ec56b6ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:06.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.952+0000 7f6fdb7fe700 1 --2- 172.21.15.133:0/3484649717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b62500 0x55ec56b6ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.952+0000 7f6fdb7fe700 1 -- 172.21.15.133:0/3484649717 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fdc10b190 con 0x55ec56b630c0 2023-11-27T15:22:06.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.952+0000 7f6fdb7fe700 1 --2- 172.21.15.133:0/3484649717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b630c0 0x55ec56b72ab0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009fd0 tx=0x7f6fd4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7622d8c5fb32b79 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:06.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.952+0000 7f6fc27fc700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:22:06.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.953+0000 7f6fc27fc700 1 -- 172.21.15.133:0/3484649717 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd400e040 con 0x55ec56b630c0 2023-11-27T15:22:06.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.953+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/3484649717 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec56b54ad0 con 0x55ec56b630c0 2023-11-27T15:22:06.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.953+0000 7f6fc27fc700 1 -- 172.21.15.133:0/3484649717 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6fd4018e10 con 0x55ec56b630c0 2023-11-27T15:22:06.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.953+0000 7f6fc27fc700 1 -- 172.21.15.133:0/3484649717 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fd4003e10 con 0x55ec56b630c0 2023-11-27T15:22:06.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.953+0000 7f6ff4f2db80 10 client.15099 fetch_fsmap learned FSMap version 23 2023-11-27T15:22:06.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.953+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/3484649717 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ec56a79140 con 0x55ec56b630c0 2023-11-27T15:22:06.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.953+0000 7f6fc27fc700 1 -- 172.21.15.133:0/3484649717 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec56b54ad0 con 0x55ec56b630c0 2023-11-27T15:22:06.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.953+0000 7f6fc27fc700 1 -- 172.21.15.133:0/3484649717 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6fd4004450 con 0x55ec56b630c0 2023-11-27T15:22:06.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.953+0000 7f6ff4f2db80 10 client.15099 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:22:06.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.953+0000 7f6ff4f2db80 10 client.15099 resolve_mds: resolved name 'h' to daemon mds.h 2023-11-27T15:22:06.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.953+0000 7f6ff4f2db80 20 client.15099 populate_metadata read hostname 'smithi133' 2023-11-27T15:22:06.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.954+0000 7f6ff4f2db80 1 --2- 172.21.15.133:0/3484649717 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x55ec56b76780 0x55ec56b75bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.954+0000 7f6ff4f2db80 4 client.15099 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:22:06.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.954+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/3484649717 --> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec56a75050 con 0x55ec56b76780 2023-11-27T15:22:06.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.954+0000 7f6fdaffd700 1 --2- 172.21.15.133:0/3484649717 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x55ec56b76780 0x55ec56b75bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.954+0000 7f6fdaffd700 1 --2- 172.21.15.133:0/3484649717 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x55ec56b76780 0x55ec56b75bb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da4bdf596d3cccc7 server_cookie=212ad7aa02befa76 in_seq=0 out_seq=0 2023-11-27T15:22:06.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.954+0000 7f6fc27fc700 10 client.15099 ms_handle_connect on v2:172.21.15.174:6836/1955090548 2023-11-27T15:22:06.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.955+0000 7f6fc27fc700 1 -- 172.21.15.133:0/3484649717 <== mds.1 v2:172.21.15.174:6836/1955090548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ec56a75050 con 0x55ec56b76780 2023-11-27T15:22:06.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.955+0000 7f6fc27fc700 10 client.15099 handle_command_reply: tid=0 2023-11-27T15:22:06.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.955+0000 7f6ff4f2db80 1 client.15099 shutdown 2023-11-27T15:22:06.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.955+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/3484649717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b630c0 msgr2=0x55ec56b72ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.955+0000 7f6ff4f2db80 1 --2- 172.21.15.133:0/3484649717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b630c0 0x55ec56b72ab0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4009fd0 tx=0x7f6fd4004b40 comp rx=0 tx=0).stop 2023-11-27T15:22:06.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.955+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/3484649717 shutdown_connections 2023-11-27T15:22:06.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.955+0000 7f6ff4f2db80 1 --2- 172.21.15.133:0/3484649717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b630c0 0x55ec56b72ab0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.955+0000 7f6ff4f2db80 1 --2- 172.21.15.133:0/3484649717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b61bb0 0x55ec56b6b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.955+0000 7f6ff4f2db80 1 --2- 172.21.15.133:0/3484649717 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x55ec56b76780 0x55ec56b75bb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.955+0000 7f6ff4f2db80 1 --2- 172.21.15.133:0/3484649717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b62500 0x55ec56b6ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.955+0000 7f6fc27fc700 0 client.15099 ms_handle_reset on v2:172.21.15.174:6836/1955090548 2023-11-27T15:22:06.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.955+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/3484649717 >> 172.21.15.133:0/3484649717 conn(0x55ec56b57310 msgr2=0x55ec56b5bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:06.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.956+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/3484649717 shutdown_connections 2023-11-27T15:22:06.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.956+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/3484649717 wait complete. 2023-11-27T15:22:06.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.956+0000 7f6ff4f2db80 20 client.15099 trim_cache size 0 max 16384 2023-11-27T15:22:06.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.974+0000 7f6ff4f2db80 1 Processor -- start 2023-11-27T15:22:06.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.974+0000 7f6ff4f2db80 1 -- start start 2023-11-27T15:22:06.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.974+0000 7f6ff4f2db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b61bb0 0x55ec56b69440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.974+0000 7f6ff4f2db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b62500 0x55ec56b69980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.974+0000 7f6ff4f2db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b630c0 0x55ec56b729d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.974+0000 7f6ff4f2db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec56b6b100 con 0x55ec56b630c0 2023-11-27T15:22:06.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.974+0000 7f6ff4f2db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec56b747a0 con 0x55ec56b61bb0 2023-11-27T15:22:06.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.974+0000 7f6ff4f2db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec56b6c720 con 0x55ec56b62500 2023-11-27T15:22:06.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.974+0000 7f6fd3fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b62500 0x55ec56b69980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.974+0000 7f6fdb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b630c0 0x55ec56b729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.974+0000 7f6fdaffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b61bb0 0x55ec56b69440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.974+0000 7f6fd3fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b62500 0x55ec56b69980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:58594/0 (socket says 172.21.15.133:58594) 2023-11-27T15:22:06.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.974+0000 7f6fd3fff700 1 -- 172.21.15.133:0/2278332407 learned_addr learned my addr 172.21.15.133:0/2278332407 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:06.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.975+0000 7f6fd3fff700 1 -- 172.21.15.133:0/2278332407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b61bb0 msgr2=0x55ec56b69440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:06.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.975+0000 7f6fd3fff700 1 --2- 172.21.15.133:0/2278332407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b61bb0 0x55ec56b69440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.975+0000 7f6fd3fff700 1 -- 172.21.15.133:0/2278332407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b630c0 msgr2=0x55ec56b729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:06.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.975+0000 7f6fd3fff700 1 --2- 172.21.15.133:0/2278332407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b630c0 0x55ec56b729d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.975+0000 7f6fd3fff700 1 -- 172.21.15.133:0/2278332407 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec56a78e90 con 0x55ec56b62500 2023-11-27T15:22:06.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.975+0000 7f6fdaffd700 1 --2- 172.21.15.133:0/2278332407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b61bb0 0x55ec56b69440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:06.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.975+0000 7f6fd3fff700 1 --2- 172.21.15.133:0/2278332407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b62500 0x55ec56b69980 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4099be0 tx=0x7f6fc409a030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90bff8e493dd3f92 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:06.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.975+0000 7f6fd1ffb700 1 -- 172.21.15.133:0/2278332407 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4003da0 con 0x55ec56b62500 2023-11-27T15:22:06.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.975+0000 7f6fd1ffb700 1 -- 172.21.15.133:0/2278332407 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fc400cc30 con 0x55ec56b62500 2023-11-27T15:22:06.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.975+0000 7f6fd1ffb700 1 -- 172.21.15.133:0/2278332407 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc409a140 con 0x55ec56b62500 2023-11-27T15:22:06.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.976+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/2278332407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b62500 msgr2=0x55ec56b69980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.976+0000 7f6ff4f2db80 1 --2- 172.21.15.133:0/2278332407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b62500 0x55ec56b69980 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4099be0 tx=0x7f6fc409a030 comp rx=0 tx=0).stop 2023-11-27T15:22:06.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.976+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/2278332407 shutdown_connections 2023-11-27T15:22:06.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.976+0000 7f6ff4f2db80 1 --2- 172.21.15.133:0/2278332407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b630c0 0x55ec56b729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.976+0000 7f6ff4f2db80 1 --2- 172.21.15.133:0/2278332407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b61bb0 0x55ec56b69440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.976+0000 7f6ff4f2db80 1 --2- 172.21.15.133:0/2278332407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b62500 0x55ec56b69980 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.976+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/2278332407 >> 172.21.15.133:0/2278332407 conn(0x55ec56b57310 msgr2=0x55ec56b6a240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:06.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.976+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/2278332407 shutdown_connections 2023-11-27T15:22:06.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.976+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/2278332407 wait complete. 2023-11-27T15:22:06.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.976+0000 7f6ff4f2db80 1 -- start start 2023-11-27T15:22:06.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.976+0000 7f6ff4f2db80 1 Processor -- start 2023-11-27T15:22:06.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.977+0000 7f6ff4f2db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b61bb0 0x55ec56b6d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.977+0000 7f6ff4f2db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b62500 0x55ec56b6dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.977+0000 7f6ff4f2db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b630c0 0x55ec56bca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.977+0000 7f6ff4f2db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec56b72c10 con 0x55ec56b61bb0 2023-11-27T15:22:06.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.977+0000 7f6ff4f2db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec56b72d80 con 0x55ec56b630c0 2023-11-27T15:22:06.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.977+0000 7f6ff4f2db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec56bcab60 con 0x55ec56b62500 2023-11-27T15:22:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.977+0000 7f6ff4f2db80 10 client.0 _ll_register_callbacks cb 0x7f6fd4009040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:22:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.977+0000 7f6fdaffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b61bb0 0x55ec56b6d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.977+0000 7f6fd3fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b62500 0x55ec56b6dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.977+0000 7f6fdaffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b61bb0 0x55ec56b6d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59784/0 (socket says 172.21.15.133:59784) 2023-11-27T15:22:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.977+0000 7f6fdaffd700 1 -- 172.21.15.133:0/363747913 learned_addr learned my addr 172.21.15.133:0/363747913 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.977+0000 7f6fdb7fe700 1 --2- 172.21.15.133:0/363747913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b630c0 0x55ec56bca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.977+0000 7f6fdaffd700 1 -- 172.21.15.133:0/363747913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b62500 msgr2=0x55ec56b6dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.977+0000 7f6fdaffd700 1 --2- 172.21.15.133:0/363747913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b62500 0x55ec56b6dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.977+0000 7f6fdaffd700 1 -- 172.21.15.133:0/363747913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b630c0 msgr2=0x55ec56bca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.977+0000 7f6fdaffd700 1 --2- 172.21.15.133:0/363747913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b630c0 0x55ec56bca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.977+0000 7f6fdaffd700 1 -- 172.21.15.133:0/363747913 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec569345e0 con 0x55ec56b61bb0 2023-11-27T15:22:06.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.978+0000 7f6fdaffd700 1 --2- 172.21.15.133:0/363747913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b61bb0 0x55ec56b6d6a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00b680 tx=0x7f6fcc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9db05ef2d5bbf81 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:06.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.978+0000 7f6fc27fc700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:22:06.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.978+0000 7f6fc27fc700 1 -- 172.21.15.133:0/363747913 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc031040 con 0x55ec56b61bb0 2023-11-27T15:22:06.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.978+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/363747913 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6fcc01e040 con 0x55ec56b61bb0 2023-11-27T15:22:06.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.978+0000 7f6fc27fc700 1 -- 172.21.15.133:0/363747913 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6fcc033860 con 0x55ec56b61bb0 2023-11-27T15:22:06.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.978+0000 7f6fc27fc700 1 -- 172.21.15.133:0/363747913 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fcc01d040 con 0x55ec56b61bb0 2023-11-27T15:22:06.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.978+0000 7f6ff4f2db80 10 client.15105 fetch_fsmap learned FSMap version 23 2023-11-27T15:22:06.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.978+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/363747913 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f6fcc0095f0 con 0x55ec56b61bb0 2023-11-27T15:22:06.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.979+0000 7f6fc27fc700 1 -- 172.21.15.133:0/363747913 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6fcc01e040 con 0x55ec56b61bb0 2023-11-27T15:22:06.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.979+0000 7f6fc27fc700 1 -- 172.21.15.133:0/363747913 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6fcc004500 con 0x55ec56b61bb0 2023-11-27T15:22:06.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.979+0000 7f6ff4f2db80 10 client.15105 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:22:06.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.979+0000 7f6ff4f2db80 10 client.15105 resolve_mds: resolved name 'h' to daemon mds.h 2023-11-27T15:22:06.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.979+0000 7f6ff4f2db80 20 client.15105 populate_metadata read hostname 'smithi133' 2023-11-27T15:22:06.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.979+0000 7f6ff4f2db80 1 --2- 172.21.15.133:0/363747913 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x55ec56bcf3a0 0x55ec56bcbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:06.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.979+0000 7f6ff4f2db80 4 client.15105 mds_command: new command op to 24521 tid=0[{"prefix": "status"}] 2023-11-27T15:22:06.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.979+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/363747913 --> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ec56a81610 con 0x55ec56bcf3a0 2023-11-27T15:22:06.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.979+0000 7f6fd3fff700 1 --2- 172.21.15.133:0/363747913 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x55ec56bcf3a0 0x55ec56bcbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:06.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.980+0000 7f6fc27fc700 10 client.15105 ms_handle_connect on v2:172.21.15.174:6836/1955090548 2023-11-27T15:22:06.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.980+0000 7f6fd3fff700 1 --2- 172.21.15.133:0/363747913 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x55ec56bcf3a0 0x55ec56bcbd50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6511c418ce7ce87f server_cookie=ca846db74e6fbc1e in_seq=0 out_seq=0 2023-11-27T15:22:06.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.980+0000 7f6fc27fc700 1 -- 172.21.15.133:0/363747913 <== mds.1 v2:172.21.15.174:6836/1955090548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55ec56a81610 con 0x55ec56bcf3a0 2023-11-27T15:22:06.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.980+0000 7f6fc27fc700 10 client.15105 handle_command_reply: tid=0 2023-11-27T15:22:06.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.980+0000 7f6ff4f2db80 1 client.15105 shutdown 2023-11-27T15:22:06.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.980+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/363747913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b61bb0 msgr2=0x55ec56b6d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.980+0000 7f6ff4f2db80 1 --2- 172.21.15.133:0/363747913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b61bb0 0x55ec56b6d6a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00b680 tx=0x7f6fcc009310 comp rx=0 tx=0).stop 2023-11-27T15:22:06.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.980+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/363747913 shutdown_connections 2023-11-27T15:22:06.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.980+0000 7f6ff4f2db80 1 --2- 172.21.15.133:0/363747913 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x55ec56bcf3a0 0x55ec56bcbd50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.980+0000 7f6ff4f2db80 1 --2- 172.21.15.133:0/363747913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec56b630c0 0x55ec56bca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.980+0000 7f6ff4f2db80 1 --2- 172.21.15.133:0/363747913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ec56b61bb0 0x55ec56b6d6a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.980+0000 7f6ff4f2db80 1 --2- 172.21.15.133:0/363747913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ec56b62500 0x55ec56b6dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.980+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/363747913 >> 172.21.15.133:0/363747913 conn(0x55ec56b57310 msgr2=0x55ec56b69440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:06.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.980+0000 7f6fc27fc700 0 client.15105 ms_handle_reset on v2:172.21.15.174:6836/1955090548 2023-11-27T15:22:06.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.980+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/363747913 shutdown_connections 2023-11-27T15:22:06.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.980+0000 7f6ff4f2db80 1 -- 172.21.15.133:0/363747913 wait complete. 2023-11-27T15:22:06.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.980+0000 7f6ff4f2db80 20 client.15105 trim_cache size 0 max 16384 2023-11-27T15:22:06.990 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:22:06.991 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "5227da4c-8d37-11ee-95a2-87774f69a715", 2023-11-27T15:22:06.991 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 1, 2023-11-27T15:22:06.991 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24521, 2023-11-27T15:22:06.991 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2023-11-27T15:22:06.991 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2023-11-27T15:22:06.991 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2023-11-27T15:22:06.991 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 64.353973009000001, 2023-11-27T15:22:06.991 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 23, 2023-11-27T15:22:06.991 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 92, 2023-11-27T15:22:06.991 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 90, 2023-11-27T15:22:06.991 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 111.07738012599999 2023-11-27T15:22:06.991 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:22:06.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.981+0000 7f6fe15ff700 1 -- 172.21.15.133:0/946987634 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6fc80a11e0 msgr2=0x7f6fc80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.981+0000 7f6fe15ff700 1 --2- 172.21.15.133:0/946987634 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6fc80a11e0 0x7f6fc80a3690 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400fae0 tx=0x7f6fd400fea0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.981+0000 7f6fe15ff700 1 -- 172.21.15.133:0/946987634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fdc074b30 msgr2=0x7f6fdc198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:06.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.981+0000 7f6fe15ff700 1 --2- 172.21.15.133:0/946987634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fdc074b30 0x7f6fdc198780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400e970 tx=0x7f6fc400ec80 comp rx=0 tx=0).stop 2023-11-27T15:22:06.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.981+0000 7f6fe15ff700 1 -- 172.21.15.133:0/946987634 shutdown_connections 2023-11-27T15:22:06.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.981+0000 7f6fe15ff700 1 --2- 172.21.15.133:0/946987634 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6fc80a11e0 0x7f6fc80a3690 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.981+0000 7f6fe15ff700 1 --2- 172.21.15.133:0/946987634 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6fdc1080f0 0x7f6fdc19f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.981+0000 7f6fe15ff700 1 --2- 172.21.15.133:0/946987634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fdc073550 0x7f6fdc198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.981+0000 7f6fe15ff700 1 --2- 172.21.15.133:0/946987634 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6fdc074b30 0x7f6fdc198780 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:06.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.981+0000 7f6fe15ff700 1 -- 172.21.15.133:0/946987634 >> 172.21.15.133:0/946987634 conn(0x7f6fdc0fb810 msgr2=0x7f6fdc0fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:06.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.981+0000 7f6fe15ff700 1 -- 172.21.15.133:0/946987634 shutdown_connections 2023-11-27T15:22:06.993 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:06.981+0000 7f6fe15ff700 1 -- 172.21.15.133:0/946987634 wait complete. 2023-11-27T15:22:07.002 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:06 smithi133 ceph-mon[97860]: pgmap v324: 129 pgs: 1 active+recovering, 128 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 5/194 objects misplaced (2.577%); 0 B/s, 0 objects/s recovering 2023-11-27T15:22:07.002 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:06 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1815915282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:07.002 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:06 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2401259311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:07.004 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-27T15:22:07.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:06 smithi177 ceph-mon[131504]: pgmap v324: 129 pgs: 1 active+recovering, 128 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 5/194 objects misplaced (2.577%); 0 B/s, 0 objects/s recovering 2023-11-27T15:22:07.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:06 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1815915282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:07.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:06 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2401259311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:07.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:06 smithi174 ceph-mon[131110]: pgmap v324: 129 pgs: 1 active+recovering, 128 active+clean; 639 KiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 5/194 objects misplaced (2.577%); 0 B/s, 0 objects/s recovering 2023-11-27T15:22:07.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:06 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1815915282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:07.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:06 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2401259311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:07.203 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.198+0000 7f539e5ff700 1 -- 172.21.15.133:0/820816378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5398101db0 msgr2=0x7f5398108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:07.203 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.198+0000 7f539e5ff700 1 --2- 172.21.15.133:0/820816378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5398101db0 0x7f5398108670 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5390009f90 tx=0x7f539000b330 comp rx=0 tx=0).stop 2023-11-27T15:22:07.203 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.198+0000 7f539e5ff700 1 -- 172.21.15.133:0/820816378 shutdown_connections 2023-11-27T15:22:07.204 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.198+0000 7f539e5ff700 1 --2- 172.21.15.133:0/820816378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5398101db0 0x7f5398108670 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.204 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.198+0000 7f539e5ff700 1 --2- 172.21.15.133:0/820816378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5398100230 0x7f5398100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.204 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.198+0000 7f539e5ff700 1 --2- 172.21.15.133:0/820816378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5398101420 0x7f5398101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.204 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.198+0000 7f539e5ff700 1 -- 172.21.15.133:0/820816378 >> 172.21.15.133:0/820816378 conn(0x7f53980fb810 msgr2=0x7f53980fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:07.204 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.199+0000 7f539e5ff700 1 -- 172.21.15.133:0/820816378 shutdown_connections 2023-11-27T15:22:07.204 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.199+0000 7f539e5ff700 1 -- 172.21.15.133:0/820816378 wait complete. 2023-11-27T15:22:07.206 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.200+0000 7f539e5ff700 1 Processor -- start 2023-11-27T15:22:07.206 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.200+0000 7f539e5ff700 1 -- start start 2023-11-27T15:22:07.206 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.200+0000 7f539e5ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5398100230 0x7f5398196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:07.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.200+0000 7f539e5ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5398101420 0x7f53981965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:07.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.200+0000 7f539e5ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5398101db0 0x7f539819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:07.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.200+0000 7f539e5ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539819dc10 con 0x7f5398100230 2023-11-27T15:22:07.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.200+0000 7f539e5ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539819dd50 con 0x7f5398101db0 2023-11-27T15:22:07.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.200+0000 7f539e5ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539819dee0 con 0x7f5398101420 2023-11-27T15:22:07.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.200+0000 7f5397fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5398100230 0x7f5398196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:07.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.200+0000 7f53977fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5398101420 0x7f53981965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:07.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.200+0000 7f539cb9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5398101db0 0x7f539819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:07.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.200+0000 7f53977fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5398101420 0x7f53981965d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:58618/0 (socket says 172.21.15.133:58618) 2023-11-27T15:22:07.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.200+0000 7f539cb9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5398101db0 0x7f539819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:45282/0 (socket says 172.21.15.133:45282) 2023-11-27T15:22:07.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.200+0000 7f53977fe700 1 -- 172.21.15.133:0/2160464893 learned_addr learned my addr 172.21.15.133:0/2160464893 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:07.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.201+0000 7f53977fe700 1 -- 172.21.15.133:0/2160464893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5398101db0 msgr2=0x7f539819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:07.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.201+0000 7f53977fe700 1 --2- 172.21.15.133:0/2160464893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5398101db0 0x7f539819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.201+0000 7f53977fe700 1 -- 172.21.15.133:0/2160464893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5398100230 msgr2=0x7f5398196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:07.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.201+0000 7f53977fe700 1 --2- 172.21.15.133:0/2160464893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5398100230 0x7f5398196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.201+0000 7f53977fe700 1 -- 172.21.15.133:0/2160464893 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538c0095f0 con 0x7f5398101420 2023-11-27T15:22:07.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.201+0000 7f53977fe700 1 --2- 172.21.15.133:0/2160464893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5398101420 0x7f53981965d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f538c00ec10 tx=0x7f538c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:07.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.201+0000 7f53957fa700 1 -- 172.21.15.133:0/2160464893 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c00cc90 con 0x7f5398101420 2023-11-27T15:22:07.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.201+0000 7f539e5ff700 1 -- 172.21.15.133:0/2160464893 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5390009c40 con 0x7f5398101420 2023-11-27T15:22:07.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.201+0000 7f53957fa700 1 -- 172.21.15.133:0/2160464893 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f538c0044e0 con 0x7f5398101420 2023-11-27T15:22:07.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.201+0000 7f53957fa700 1 -- 172.21.15.133:0/2160464893 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c010430 con 0x7f5398101420 2023-11-27T15:22:07.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.201+0000 7f539e5ff700 1 -- 172.21.15.133:0/2160464893 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539810bc00 con 0x7f5398101420 2023-11-27T15:22:07.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.204+0000 7f53957fa700 1 -- 172.21.15.133:0/2160464893 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f538c003680 con 0x7f5398101420 2023-11-27T15:22:07.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.206+0000 7f53b1fedb80 1 Processor -- start 2023-11-27T15:22:07.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.206+0000 7f53957fa700 1 --2- 172.21.15.133:0/2160464893 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f53800a1260 0x7f53800a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:07.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.206+0000 7f53b1fedb80 1 -- start start 2023-11-27T15:22:07.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.206+0000 7f53957fa700 1 -- 172.21.15.133:0/2160464893 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f538c014070 con 0x7f5398101420 2023-11-27T15:22:07.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.206+0000 7f539cb9c700 1 --2- 172.21.15.133:0/2160464893 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f53800a1260 0x7f53800a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:07.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.206+0000 7f53b1fedb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d34bb0 0x564115d34fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:07.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.206+0000 7f53b1fedb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d35500 0x564115d35ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:07.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.206+0000 7f53b1fedb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d360c0 0x564115d36530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:07.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.206+0000 7f53b1fedb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564115c3a010 con 0x564115d360c0 2023-11-27T15:22:07.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.206+0000 7f53b1fedb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x564115c8c850 con 0x564115d34bb0 2023-11-27T15:22:07.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.206+0000 7f53b1fedb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x564115c85ce0 con 0x564115d35500 2023-11-27T15:22:07.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.206+0000 7f5397fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d34bb0 0x564115d34fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:07.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.206+0000 7f5397fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d34bb0 0x564115d34fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:45292/0 (socket says 172.21.15.133:45292) 2023-11-27T15:22:07.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.206+0000 7f539cb9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d360c0 0x564115d36530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:07.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.207+0000 7f5397fff700 1 -- 172.21.15.133:0/2977027052 learned_addr learned my addr 172.21.15.133:0/2977027052 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:07.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.207+0000 7f53977fe700 1 --2- 172.21.15.133:0/2977027052 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d35500 0x564115d35ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:07.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.207+0000 7f5397fff700 1 -- 172.21.15.133:0/2977027052 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d35500 msgr2=0x564115d35ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:07.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.207+0000 7f5397fff700 1 --2- 172.21.15.133:0/2977027052 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d35500 0x564115d35ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.207+0000 7f5397fff700 1 -- 172.21.15.133:0/2977027052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d360c0 msgr2=0x564115d36530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:07.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.207+0000 7f5397fff700 1 --2- 172.21.15.133:0/2977027052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d360c0 0x564115d36530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.207+0000 7f5397fff700 1 -- 172.21.15.133:0/2977027052 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564115b075e0 con 0x564115d34bb0 2023-11-27T15:22:07.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.207+0000 7f539cb9c700 1 --2- 172.21.15.133:0/2160464893 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f53800a1260 0x7f53800a3710 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5390009790 tx=0x7f53900077c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:07.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.207+0000 7f5397fff700 1 --2- 172.21.15.133:0/2977027052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d34bb0 0x564115d34fc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5388009d30 tx=0x7f5388011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bab54a7ebda37a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:07.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.207+0000 7f537e7fc700 1 -- 172.21.15.133:0/2977027052 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5388004220 con 0x564115d34bb0 2023-11-27T15:22:07.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.207+0000 7f537e7fc700 1 -- 172.21.15.133:0/2977027052 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5388004380 con 0x564115d34bb0 2023-11-27T15:22:07.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.207+0000 7f537e7fc700 1 -- 172.21.15.133:0/2977027052 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53880198d0 con 0x564115d34bb0 2023-11-27T15:22:07.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.208+0000 7f53b1fedb80 1 -- 172.21.15.133:0/2977027052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d34bb0 msgr2=0x564115d34fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:07.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.208+0000 7f53b1fedb80 1 --2- 172.21.15.133:0/2977027052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d34bb0 0x564115d34fc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5388009d30 tx=0x7f5388011720 comp rx=0 tx=0).stop 2023-11-27T15:22:07.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.208+0000 7f53b1fedb80 1 -- 172.21.15.133:0/2977027052 shutdown_connections 2023-11-27T15:22:07.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.208+0000 7f53b1fedb80 1 --2- 172.21.15.133:0/2977027052 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d360c0 0x564115d36530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.208+0000 7f53b1fedb80 1 --2- 172.21.15.133:0/2977027052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d34bb0 0x564115d34fc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.208+0000 7f53b1fedb80 1 --2- 172.21.15.133:0/2977027052 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d35500 0x564115d35ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.208+0000 7f53b1fedb80 1 -- 172.21.15.133:0/2977027052 >> 172.21.15.133:0/2977027052 conn(0x564115d2a310 msgr2=0x564115d2f620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:07.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.208+0000 7f53b1fedb80 1 -- 172.21.15.133:0/2977027052 shutdown_connections 2023-11-27T15:22:07.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.208+0000 7f53b1fedb80 1 -- 172.21.15.133:0/2977027052 wait complete. 2023-11-27T15:22:07.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.209+0000 7f53b1fedb80 1 -- start start 2023-11-27T15:22:07.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.209+0000 7f53b1fedb80 1 Processor -- start 2023-11-27T15:22:07.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.209+0000 7f53b1fedb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d34bb0 0x564115d3e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:07.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.209+0000 7f53b1fedb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d35500 0x564115d3ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:07.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.209+0000 7f53b1fedb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d360c0 0x564115d45a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:07.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.209+0000 7f53b1fedb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564115c27890 con 0x564115d34bb0 2023-11-27T15:22:07.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.209+0000 7f53b1fedb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x564115c27530 con 0x564115d35500 2023-11-27T15:22:07.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.209+0000 7f53b1fedb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x564115b098d0 con 0x564115d360c0 2023-11-27T15:22:07.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.210+0000 7f53b1fedb80 10 client.0 _ll_register_callbacks cb 0x564115c45730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:22:07.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.210+0000 7f5397fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d34bb0 0x564115d3e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:07.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.210+0000 7f53977fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d35500 0x564115d3ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:07.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.210+0000 7f5397fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d34bb0 0x564115d3e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59814/0 (socket says 172.21.15.133:59814) 2023-11-27T15:22:07.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.210+0000 7f5397fff700 1 -- 172.21.15.133:0/3217212671 learned_addr learned my addr 172.21.15.133:0/3217212671 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:07.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.210+0000 7f539cb9c700 1 --2- 172.21.15.133:0/3217212671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d360c0 0x564115d45a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:07.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.210+0000 7f5397fff700 1 -- 172.21.15.133:0/3217212671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d360c0 msgr2=0x564115d45a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:07.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.210+0000 7f5397fff700 1 --2- 172.21.15.133:0/3217212671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d360c0 0x564115d45a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.210+0000 7f5397fff700 1 -- 172.21.15.133:0/3217212671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d35500 msgr2=0x564115d3ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:07.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.210+0000 7f5397fff700 1 --2- 172.21.15.133:0/3217212671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d35500 0x564115d3ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.210+0000 7f5397fff700 1 -- 172.21.15.133:0/3217212671 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5388009960 con 0x564115d34bb0 2023-11-27T15:22:07.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.210+0000 7f539cb9c700 1 --2- 172.21.15.133:0/3217212671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d360c0 0x564115d45a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:07.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.210+0000 7f53977fe700 1 --2- 172.21.15.133:0/3217212671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d35500 0x564115d3ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:07.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.210+0000 7f5397fff700 1 --2- 172.21.15.133:0/3217212671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d34bb0 0x564115d3e500 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f53880097c0 tx=0x7f5388005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5170e6b850b474f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:07.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.210+0000 7f53777fe700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:22:07.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.211+0000 7f53777fe700 1 -- 172.21.15.133:0/3217212671 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5388005040 con 0x564115d34bb0 2023-11-27T15:22:07.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.211+0000 7f53777fe700 1 -- 172.21.15.133:0/3217212671 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f53880051a0 con 0x564115d34bb0 2023-11-27T15:22:07.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.211+0000 7f53b1fedb80 1 -- 172.21.15.133:0/3217212671 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564115d27ad0 con 0x564115d34bb0 2023-11-27T15:22:07.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.211+0000 7f53777fe700 1 -- 172.21.15.133:0/3217212671 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5388005360 con 0x564115d34bb0 2023-11-27T15:22:07.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.211+0000 7f53b1fedb80 10 client.15117 fetch_fsmap learned FSMap version 23 2023-11-27T15:22:07.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.211+0000 7f53b1fedb80 1 -- 172.21.15.133:0/3217212671 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564115c4c140 con 0x564115d34bb0 2023-11-27T15:22:07.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.211+0000 7f53777fe700 1 -- 172.21.15.133:0/3217212671 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564115d27ad0 con 0x564115d34bb0 2023-11-27T15:22:07.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.211+0000 7f53777fe700 1 -- 172.21.15.133:0/3217212671 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5388021040 con 0x564115d34bb0 2023-11-27T15:22:07.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.211+0000 7f53b1fedb80 10 client.15117 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:22:07.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.212+0000 7f53b1fedb80 10 client.15117 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-27T15:22:07.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.212+0000 7f53b1fedb80 20 client.15117 populate_metadata read hostname 'smithi133' 2023-11-27T15:22:07.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.212+0000 7f53b1fedb80 1 --2- 172.21.15.133:0/3217212671 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564115d496c0 0x564115d48ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:07.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.212+0000 7f53b1fedb80 4 client.15117 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:22:07.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.212+0000 7f53b1fedb80 1 -- 172.21.15.133:0/3217212671 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564115c48050 con 0x564115d496c0 2023-11-27T15:22:07.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.212+0000 7f53977fe700 1 --2- 172.21.15.133:0/3217212671 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564115d496c0 0x564115d48ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:07.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.212+0000 7f53777fe700 10 client.15117 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:22:07.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.212+0000 7f53977fe700 1 --2- 172.21.15.133:0/3217212671 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564115d496c0 0x564115d48ad0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0fddce5f16f03c2 server_cookie=38730bc23046aaed in_seq=0 out_seq=0 2023-11-27T15:22:07.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.213+0000 7f53777fe700 1 -- 172.21.15.133:0/3217212671 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564115c48050 con 0x564115d496c0 2023-11-27T15:22:07.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.213+0000 7f53777fe700 10 client.15117 handle_command_reply: tid=0 2023-11-27T15:22:07.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.213+0000 7f53b1fedb80 1 client.15117 shutdown 2023-11-27T15:22:07.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.213+0000 7f53b1fedb80 1 -- 172.21.15.133:0/3217212671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d34bb0 msgr2=0x564115d3e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:07.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.213+0000 7f53b1fedb80 1 --2- 172.21.15.133:0/3217212671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d34bb0 0x564115d3e500 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f53880097c0 tx=0x7f5388005e50 comp rx=0 tx=0).stop 2023-11-27T15:22:07.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.213+0000 7f53b1fedb80 1 -- 172.21.15.133:0/3217212671 shutdown_connections 2023-11-27T15:22:07.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.213+0000 7f53b1fedb80 1 --2- 172.21.15.133:0/3217212671 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564115d496c0 0x564115d48ad0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.213+0000 7f53b1fedb80 1 --2- 172.21.15.133:0/3217212671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d360c0 0x564115d45a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.213+0000 7f53777fe700 0 client.15117 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:22:07.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.213+0000 7f53b1fedb80 1 --2- 172.21.15.133:0/3217212671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d34bb0 0x564115d3e500 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.213+0000 7f53b1fedb80 1 --2- 172.21.15.133:0/3217212671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d35500 0x564115d3ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.213+0000 7f53b1fedb80 1 -- 172.21.15.133:0/3217212671 >> 172.21.15.133:0/3217212671 conn(0x564115d2a310 msgr2=0x564115d2ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:07.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.214+0000 7f53b1fedb80 1 -- 172.21.15.133:0/3217212671 shutdown_connections 2023-11-27T15:22:07.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.214+0000 7f53b1fedb80 1 -- 172.21.15.133:0/3217212671 wait complete. 2023-11-27T15:22:07.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.214+0000 7f53b1fedb80 20 client.15117 trim_cache size 0 max 16384 2023-11-27T15:22:07.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.232+0000 7f53b1fedb80 1 Processor -- start 2023-11-27T15:22:07.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.232+0000 7f53b1fedb80 1 -- start start 2023-11-27T15:22:07.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.232+0000 7f53b1fedb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d34bb0 0x564115d52430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:07.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.232+0000 7f53b1fedb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d35500 0x564115d45a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:07.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.232+0000 7f53b1fedb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d360c0 0x564115d3c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:07.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.232+0000 7f53b1fedb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564115d369b0 con 0x564115d360c0 2023-11-27T15:22:07.237 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.232+0000 7f53b1fedb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x564115b91f00 con 0x564115d34bb0 2023-11-27T15:22:07.238 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.232+0000 7f53b1fedb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x564115c43fa0 con 0x564115d35500 2023-11-27T15:22:07.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.233+0000 7f539cb9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d360c0 0x564115d3c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:07.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.233+0000 7f53977fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d35500 0x564115d45a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:07.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.233+0000 7f5397fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d34bb0 0x564115d52430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:07.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.233+0000 7f53977fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d35500 0x564115d45a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:58648/0 (socket says 172.21.15.133:58648) 2023-11-27T15:22:07.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.233+0000 7f53977fe700 1 -- 172.21.15.133:0/1232832708 learned_addr learned my addr 172.21.15.133:0/1232832708 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:07.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.233+0000 7f539cb9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d360c0 0x564115d3c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59822/0 (socket says 172.21.15.133:59822) 2023-11-27T15:22:07.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.233+0000 7f53977fe700 1 -- 172.21.15.133:0/1232832708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d34bb0 msgr2=0x564115d52430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:07.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.233+0000 7f53977fe700 1 --2- 172.21.15.133:0/1232832708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d34bb0 0x564115d52430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.233+0000 7f53977fe700 1 -- 172.21.15.133:0/1232832708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d360c0 msgr2=0x564115d3c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:07.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.233+0000 7f53977fe700 1 --2- 172.21.15.133:0/1232832708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d360c0 0x564115d3c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.233+0000 7f53977fe700 1 -- 172.21.15.133:0/1232832708 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564115c4be90 con 0x564115d35500 2023-11-27T15:22:07.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.233+0000 7f53977fe700 1 --2- 172.21.15.133:0/1232832708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d35500 0x564115d45a40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f538c09a180 tx=0x7f538c005d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c73a4a5b6ae9adcc server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:07.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.233+0000 7f537effd700 1 -- 172.21.15.133:0/1232832708 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c0a2640 con 0x564115d35500 2023-11-27T15:22:07.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.233+0000 7f537effd700 1 -- 172.21.15.133:0/1232832708 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f538c00cdf0 con 0x564115d35500 2023-11-27T15:22:07.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.233+0000 7f537effd700 1 -- 172.21.15.133:0/1232832708 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538c0996e0 con 0x564115d35500 2023-11-27T15:22:07.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.234+0000 7f53b1fedb80 1 -- 172.21.15.133:0/1232832708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d35500 msgr2=0x564115d45a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:07.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.234+0000 7f53b1fedb80 1 --2- 172.21.15.133:0/1232832708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d35500 0x564115d45a40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f538c09a180 tx=0x7f538c005d10 comp rx=0 tx=0).stop 2023-11-27T15:22:07.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.234+0000 7f53b1fedb80 1 -- 172.21.15.133:0/1232832708 shutdown_connections 2023-11-27T15:22:07.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.234+0000 7f53b1fedb80 1 --2- 172.21.15.133:0/1232832708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d360c0 0x564115d3c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.234+0000 7f53b1fedb80 1 --2- 172.21.15.133:0/1232832708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d34bb0 0x564115d52430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.234+0000 7f53b1fedb80 1 --2- 172.21.15.133:0/1232832708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d35500 0x564115d45a40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.234+0000 7f53b1fedb80 1 -- 172.21.15.133:0/1232832708 >> 172.21.15.133:0/1232832708 conn(0x564115d2a310 msgr2=0x564115d2f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:07.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.234+0000 7f53b1fedb80 1 -- 172.21.15.133:0/1232832708 shutdown_connections 2023-11-27T15:22:07.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.234+0000 7f53b1fedb80 1 -- 172.21.15.133:0/1232832708 wait complete. 2023-11-27T15:22:07.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.235+0000 7f53b1fedb80 1 -- start start 2023-11-27T15:22:07.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.235+0000 7f53b1fedb80 1 Processor -- start 2023-11-27T15:22:07.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.235+0000 7f53b1fedb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d34bb0 0x564115d40410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:07.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.235+0000 7f53b1fedb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d35500 0x564115d45980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:07.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.235+0000 7f53b1fedb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d360c0 0x564115d40990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:07.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.235+0000 7f53b1fedb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564115d9d560 con 0x564115d34bb0 2023-11-27T15:22:07.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.235+0000 7f53b1fedb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x564115d9d6a0 con 0x564115d360c0 2023-11-27T15:22:07.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.235+0000 7f53b1fedb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x564115d9d7e0 con 0x564115d35500 2023-11-27T15:22:07.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.235+0000 7f53b1fedb80 10 client.0 _ll_register_callbacks cb 0x7f5390007480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:22:07.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.235+0000 7f5397fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d34bb0 0x564115d40410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:07.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.235+0000 7f539cb9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d360c0 0x564115d40990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:07.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.235+0000 7f53977fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d35500 0x564115d45980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:07.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.235+0000 7f539cb9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d360c0 0x564115d40990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:45322/0 (socket says 172.21.15.133:45322) 2023-11-27T15:22:07.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.235+0000 7f539cb9c700 1 -- 172.21.15.133:0/2211273954 learned_addr learned my addr 172.21.15.133:0/2211273954 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:07.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.236+0000 7f539cb9c700 1 -- 172.21.15.133:0/2211273954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d35500 msgr2=0x564115d45980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:07.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.236+0000 7f539cb9c700 1 --2- 172.21.15.133:0/2211273954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d35500 0x564115d45980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.236+0000 7f539cb9c700 1 -- 172.21.15.133:0/2211273954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d34bb0 msgr2=0x564115d40410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:07.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.236+0000 7f539cb9c700 1 --2- 172.21.15.133:0/2211273954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d34bb0 0x564115d40410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.236+0000 7f539cb9c700 1 -- 172.21.15.133:0/2211273954 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539810bc00 con 0x564115d360c0 2023-11-27T15:22:07.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.236+0000 7f5397fff700 1 --2- 172.21.15.133:0/2211273954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d34bb0 0x564115d40410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:07.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.236+0000 7f539cb9c700 1 --2- 172.21.15.133:0/2211273954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d360c0 0x564115d40990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f539000b6c0 tx=0x7f539000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9ac72558a60fec server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:07.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.236+0000 7f53777fe700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:22:07.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.236+0000 7f53777fe700 1 -- 172.21.15.133:0/2211273954 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539000e040 con 0x564115d360c0 2023-11-27T15:22:07.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.236+0000 7f53b1fedb80 1 -- 172.21.15.133:0/2211273954 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5388004380 con 0x564115d360c0 2023-11-27T15:22:07.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.236+0000 7f53777fe700 1 -- 172.21.15.133:0/2211273954 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f53900052c0 con 0x564115d360c0 2023-11-27T15:22:07.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.237+0000 7f53777fe700 1 -- 172.21.15.133:0/2211273954 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5390005480 con 0x564115d360c0 2023-11-27T15:22:07.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.237+0000 7f53b1fedb80 10 client.25169 fetch_fsmap learned FSMap version 23 2023-11-27T15:22:07.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.237+0000 7f53b1fedb80 1 -- 172.21.15.133:0/2211273954 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564115a9cc10 con 0x564115d360c0 2023-11-27T15:22:07.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.237+0000 7f53777fe700 1 -- 172.21.15.133:0/2211273954 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5388004380 con 0x564115d360c0 2023-11-27T15:22:07.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.237+0000 7f53777fe700 1 -- 172.21.15.133:0/2211273954 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5390020a80 con 0x564115d360c0 2023-11-27T15:22:07.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.237+0000 7f53b1fedb80 10 client.25169 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:22:07.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.237+0000 7f53b1fedb80 10 client.25169 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-27T15:22:07.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.237+0000 7f53b1fedb80 20 client.25169 populate_metadata read hostname 'smithi133' 2023-11-27T15:22:07.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.237+0000 7f53b1fedb80 1 --2- 172.21.15.133:0/2211273954 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564115da2550 0x564115d9eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:07.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.237+0000 7f53b1fedb80 4 client.25169 mds_command: new command op to 24560 tid=0[{"prefix": "status"}] 2023-11-27T15:22:07.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.237+0000 7f53b1fedb80 1 -- 172.21.15.133:0/2211273954 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564115c54610 con 0x564115da2550 2023-11-27T15:22:07.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.238+0000 7f5397fff700 1 --2- 172.21.15.133:0/2211273954 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564115da2550 0x564115d9eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:07.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.238+0000 7f53777fe700 10 client.25169 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:22:07.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.238+0000 7f5397fff700 1 --2- 172.21.15.133:0/2211273954 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564115da2550 0x564115d9eec0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c42b5bbee486d0b server_cookie=8d41c3f17c4b1eb in_seq=0 out_seq=0 2023-11-27T15:22:07.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.238+0000 7f53777fe700 1 -- 172.21.15.133:0/2211273954 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x564115c54610 con 0x564115da2550 2023-11-27T15:22:07.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.238+0000 7f53777fe700 10 client.25169 handle_command_reply: tid=0 2023-11-27T15:22:07.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.238+0000 7f53b1fedb80 1 client.25169 shutdown 2023-11-27T15:22:07.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.239+0000 7f53b1fedb80 1 -- 172.21.15.133:0/2211273954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d360c0 msgr2=0x564115d40990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:07.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.239+0000 7f53b1fedb80 1 --2- 172.21.15.133:0/2211273954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d360c0 0x564115d40990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f539000b6c0 tx=0x7f539000b330 comp rx=0 tx=0).stop 2023-11-27T15:22:07.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.239+0000 7f53b1fedb80 1 -- 172.21.15.133:0/2211273954 shutdown_connections 2023-11-27T15:22:07.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.239+0000 7f53b1fedb80 1 --2- 172.21.15.133:0/2211273954 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564115da2550 0x564115d9eec0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.239+0000 7f53b1fedb80 1 --2- 172.21.15.133:0/2211273954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564115d360c0 0x564115d40990 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.239+0000 7f53b1fedb80 1 --2- 172.21.15.133:0/2211273954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564115d34bb0 0x564115d40410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.239+0000 7f53777fe700 0 client.25169 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:22:07.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.239+0000 7f53b1fedb80 1 --2- 172.21.15.133:0/2211273954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564115d35500 0x564115d45980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.239+0000 7f53b1fedb80 1 -- 172.21.15.133:0/2211273954 >> 172.21.15.133:0/2211273954 conn(0x564115d2a310 msgr2=0x564115d3c400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:07.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.239+0000 7f53b1fedb80 1 -- 172.21.15.133:0/2211273954 shutdown_connections 2023-11-27T15:22:07.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.239+0000 7f53b1fedb80 1 -- 172.21.15.133:0/2211273954 wait complete. 2023-11-27T15:22:07.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.239+0000 7f53b1fedb80 20 client.25169 trim_cache size 0 max 16384 2023-11-27T15:22:07.250 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:22:07.251 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "5227da4c-8d37-11ee-95a2-87774f69a715", 2023-11-27T15:22:07.251 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2023-11-27T15:22:07.251 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24560, 2023-11-27T15:22:07.251 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2023-11-27T15:22:07.251 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2023-11-27T15:22:07.251 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2023-11-27T15:22:07.251 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 94.036299501000002, 2023-11-27T15:22:07.251 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 23, 2023-11-27T15:22:07.251 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 92, 2023-11-27T15:22:07.251 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 90, 2023-11-27T15:22:07.251 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 100.72936561 2023-11-27T15:22:07.251 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:22:07.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.240+0000 7f539e5ff700 1 -- 172.21.15.133:0/2160464893 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f53800a1260 msgr2=0x7f53800a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:07.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.240+0000 7f539e5ff700 1 --2- 172.21.15.133:0/2160464893 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f53800a1260 0x7f53800a3710 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5390009790 tx=0x7f53900077c0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.240+0000 7f539e5ff700 1 -- 172.21.15.133:0/2160464893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5398101420 msgr2=0x7f53981965d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:07.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.240+0000 7f539e5ff700 1 --2- 172.21.15.133:0/2160464893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5398101420 0x7f53981965d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f538c00ec10 tx=0x7f538c00ef20 comp rx=0 tx=0).stop 2023-11-27T15:22:07.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.240+0000 7f539e5ff700 1 -- 172.21.15.133:0/2160464893 shutdown_connections 2023-11-27T15:22:07.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.240+0000 7f539e5ff700 1 --2- 172.21.15.133:0/2160464893 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f53800a1260 0x7f53800a3710 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.240+0000 7f539e5ff700 1 --2- 172.21.15.133:0/2160464893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5398101db0 0x7f539819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.240+0000 7f539e5ff700 1 --2- 172.21.15.133:0/2160464893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5398100230 0x7f5398196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.240+0000 7f539e5ff700 1 --2- 172.21.15.133:0/2160464893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5398101420 0x7f53981965d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:07.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.240+0000 7f539e5ff700 1 -- 172.21.15.133:0/2160464893 >> 172.21.15.133:0/2160464893 conn(0x7f53980fb810 msgr2=0x7f53980fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:07.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.240+0000 7f539e5ff700 1 -- 172.21.15.133:0/2160464893 shutdown_connections 2023-11-27T15:22:07.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:07.240+0000 7f539e5ff700 1 -- 172.21.15.133:0/2160464893 wait complete. 2023-11-27T15:22:07.263 INFO:tasks.cephadm:Setup complete, yielding 2023-11-27T15:22:07.263 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-11-27T15:22:07.272 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi133.front.sepia.ceph.com 2023-11-27T15:22:07.272 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- bash -c 'ceph orch status' 2023-11-27T15:22:08.115 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:22:08.898 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:08 smithi133 ceph-mon[97860]: pgmap v325: 129 pgs: 1 active+recovering, 128 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 5/194 objects misplaced (2.577%); 0 B/s, 0 objects/s recovering 2023-11-27T15:22:09.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:08 smithi177 ceph-mon[131504]: pgmap v325: 129 pgs: 1 active+recovering, 128 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 5/194 objects misplaced (2.577%); 0 B/s, 0 objects/s recovering 2023-11-27T15:22:09.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:08 smithi174 ceph-mon[131110]: pgmap v325: 129 pgs: 1 active+recovering, 128 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s; 5/194 objects misplaced (2.577%); 0 B/s, 0 objects/s recovering 2023-11-27T15:22:09.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.532+0000 7f9d9b59e700 1 -- 172.21.15.133:0/664520059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d9c107fe0 msgr2=0x7f9d9c10a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:09.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.532+0000 7f9d9b59e700 1 --2- 172.21.15.133:0/664520059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d9c107fe0 0x7f9d9c10a420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400b0a0 tx=0x7f9d9400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:09.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.532+0000 7f9d9b59e700 1 -- 172.21.15.133:0/664520059 shutdown_connections 2023-11-27T15:22:09.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.532+0000 7f9d9b59e700 1 --2- 172.21.15.133:0/664520059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d9c107fe0 0x7f9d9c10a420 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:09.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.532+0000 7f9d9b59e700 1 --2- 172.21.15.133:0/664520059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d9c074b20 0x7f9d9c072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:09.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.532+0000 7f9d9b59e700 1 --2- 172.21.15.133:0/664520059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d9c073550 0x7f9d9c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:09.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.532+0000 7f9d9b59e700 1 -- 172.21.15.133:0/664520059 >> 172.21.15.133:0/664520059 conn(0x7f9d9c0fb720 msgr2=0x7f9d9c0fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:09.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.533+0000 7f9d9b59e700 1 -- 172.21.15.133:0/664520059 shutdown_connections 2023-11-27T15:22:09.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.533+0000 7f9d9b59e700 1 -- 172.21.15.133:0/664520059 wait complete. 2023-11-27T15:22:09.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.534+0000 7f9d9b59e700 1 Processor -- start 2023-11-27T15:22:09.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.534+0000 7f9d9b59e700 1 -- start start 2023-11-27T15:22:09.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.535+0000 7f9d9b59e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d9c073550 0x7f9d9c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:09.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.535+0000 7f9d9b59e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d9c074b20 0x7f9d9c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:09.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.535+0000 7f9d9b59e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d9c107fe0 0x7f9d9c19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:09.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.535+0000 7f9d9b59e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c19fd00 con 0x7f9d9c107fe0 2023-11-27T15:22:09.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.535+0000 7f9d9b59e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c19fe40 con 0x7f9d9c074b20 2023-11-27T15:22:09.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.535+0000 7f9d9b59e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c19ffd0 con 0x7f9d9c073550 2023-11-27T15:22:09.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.535+0000 7f9d9ad9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d9c107fe0 0x7f9d9c19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:09.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.535+0000 7f9d9ad9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d9c107fe0 0x7f9d9c19f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59856/0 (socket says 172.21.15.133:59856) 2023-11-27T15:22:09.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.535+0000 7f9d9ad9d700 1 -- 172.21.15.133:0/3285976768 learned_addr learned my addr 172.21.15.133:0/3285976768 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:09.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.535+0000 7f9d99d9b700 1 --2- 172.21.15.133:0/3285976768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d9c074b20 0x7f9d9c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:09.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.535+0000 7f9d9a59c700 1 --2- 172.21.15.133:0/3285976768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d9c073550 0x7f9d9c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:09.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.535+0000 7f9d9ad9d700 1 -- 172.21.15.133:0/3285976768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d9c073550 msgr2=0x7f9d9c198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:09.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.535+0000 7f9d9ad9d700 1 --2- 172.21.15.133:0/3285976768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d9c073550 0x7f9d9c198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:09.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.535+0000 7f9d9ad9d700 1 -- 172.21.15.133:0/3285976768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d9c074b20 msgr2=0x7f9d9c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:09.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.535+0000 7f9d9ad9d700 1 --2- 172.21.15.133:0/3285976768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d9c074b20 0x7f9d9c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:09.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.535+0000 7f9d9ad9d700 1 -- 172.21.15.133:0/3285976768 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d94009d00 con 0x7f9d9c107fe0 2023-11-27T15:22:09.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.535+0000 7f9d99d9b700 1 --2- 172.21.15.133:0/3285976768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d9c074b20 0x7f9d9c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:22:09.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.536+0000 7f9d9ad9d700 1 --2- 172.21.15.133:0/3285976768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d9c107fe0 0x7f9d9c19f730 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400b070 tx=0x7f9d9400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:09.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.536+0000 7f9d8b7fe700 1 -- 172.21.15.133:0/3285976768 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9400e040 con 0x7f9d9c107fe0 2023-11-27T15:22:09.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.536+0000 7f9d8b7fe700 1 -- 172.21.15.133:0/3285976768 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d94003e10 con 0x7f9d9c107fe0 2023-11-27T15:22:09.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.536+0000 7f9d8b7fe700 1 -- 172.21.15.133:0/3285976768 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9401b7e0 con 0x7f9d9c107fe0 2023-11-27T15:22:09.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.536+0000 7f9d9b59e700 1 -- 172.21.15.133:0/3285976768 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d9c10dca0 con 0x7f9d9c107fe0 2023-11-27T15:22:09.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.536+0000 7f9d9b59e700 1 -- 172.21.15.133:0/3285976768 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9c10db10 con 0x7f9d9c107fe0 2023-11-27T15:22:09.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.537+0000 7f9d9b59e700 1 -- 172.21.15.133:0/3285976768 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d9c106690 con 0x7f9d9c107fe0 2023-11-27T15:22:09.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.539+0000 7f9d8b7fe700 1 -- 172.21.15.133:0/3285976768 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f9d9401da30 con 0x7f9d9c107fe0 2023-11-27T15:22:09.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.540+0000 7f9d8b7fe700 1 --2- 172.21.15.133:0/3285976768 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9d840a1260 0x7f9d840a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:09.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.540+0000 7f9d8b7fe700 1 -- 172.21.15.133:0/3285976768 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f9d940ce1f0 con 0x7f9d9c107fe0 2023-11-27T15:22:09.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.541+0000 7f9d9a59c700 1 --2- 172.21.15.133:0/3285976768 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9d840a1260 0x7f9d840a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:09.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.541+0000 7f9d9a59c700 1 --2- 172.21.15.133:0/3285976768 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9d840a1260 0x7f9d840a3710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009be0 tx=0x7f9d8c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:09.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.545+0000 7f9d8b7fe700 1 -- 172.21.15.133:0/3285976768 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9d940974c0 con 0x7f9d9c107fe0 2023-11-27T15:22:09.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.755+0000 7f9d9b59e700 1 -- 172.21.15.133:0/3285976768 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d9c061ab0 con 0x7f9d840a1260 2023-11-27T15:22:09.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.756+0000 7f9d8b7fe700 1 -- 172.21.15.133:0/3285976768 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f9d9c061ab0 con 0x7f9d840a1260 2023-11-27T15:22:09.761 INFO:teuthology.orchestra.run.smithi133.stdout:Backend: cephadm 2023-11-27T15:22:09.762 INFO:teuthology.orchestra.run.smithi133.stdout:Available: Yes 2023-11-27T15:22:09.762 INFO:teuthology.orchestra.run.smithi133.stdout:Paused: No 2023-11-27T15:22:09.763 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.758+0000 7f9d9b59e700 1 -- 172.21.15.133:0/3285976768 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9d840a1260 msgr2=0x7f9d840a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:09.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.758+0000 7f9d9b59e700 1 --2- 172.21.15.133:0/3285976768 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9d840a1260 0x7f9d840a3710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009be0 tx=0x7f9d8c009260 comp rx=0 tx=0).stop 2023-11-27T15:22:09.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.758+0000 7f9d9b59e700 1 -- 172.21.15.133:0/3285976768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d9c107fe0 msgr2=0x7f9d9c19f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:09.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.758+0000 7f9d9b59e700 1 --2- 172.21.15.133:0/3285976768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d9c107fe0 0x7f9d9c19f730 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400b070 tx=0x7f9d9400bce0 comp rx=0 tx=0).stop 2023-11-27T15:22:09.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.758+0000 7f9d9b59e700 1 -- 172.21.15.133:0/3285976768 shutdown_connections 2023-11-27T15:22:09.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.758+0000 7f9d9b59e700 1 --2- 172.21.15.133:0/3285976768 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9d840a1260 0x7f9d840a3710 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:09.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.758+0000 7f9d9b59e700 1 --2- 172.21.15.133:0/3285976768 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9d9c107fe0 0x7f9d9c19f730 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:09.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.758+0000 7f9d9b59e700 1 --2- 172.21.15.133:0/3285976768 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9d9c073550 0x7f9d9c198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:09.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.758+0000 7f9d9b59e700 1 --2- 172.21.15.133:0/3285976768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d9c074b20 0x7f9d9c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:09.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.758+0000 7f9d9b59e700 1 -- 172.21.15.133:0/3285976768 >> 172.21.15.133:0/3285976768 conn(0x7f9d9c0fb720 msgr2=0x7f9d9c0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:09.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.759+0000 7f9d9b59e700 1 -- 172.21.15.133:0/3285976768 shutdown_connections 2023-11-27T15:22:09.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:09.759+0000 7f9d9b59e700 1 -- 172.21.15.133:0/3285976768 wait complete. 2023-11-27T15:22:10.315 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- bash -c 'ceph orch ps' 2023-11-27T15:22:11.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:10 smithi177 ceph-mon[131504]: pgmap v326: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 13 B/s, 1 objects/s recovering 2023-11-27T15:22:11.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:10 smithi177 ceph-mon[131504]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:22:11.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:10 smithi174 ceph-mon[131110]: pgmap v326: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 13 B/s, 1 objects/s recovering 2023-11-27T15:22:11.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:10 smithi174 ceph-mon[131110]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:22:11.163 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:22:11.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:10 smithi133 ceph-mon[97860]: pgmap v326: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 13 B/s, 1 objects/s recovering 2023-11-27T15:22:11.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:10 smithi133 ceph-mon[97860]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:22:12.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.689+0000 7fc38b244700 1 -- 172.21.15.133:0/3446671083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3841002f0 msgr2=0x7fc384100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:12.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.689+0000 7fc38b244700 1 --2- 172.21.15.133:0/3446671083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3841002f0 0x7fc384100700 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc378009990 tx=0x7fc378009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.690+0000 7fc38b244700 1 -- 172.21.15.133:0/3446671083 shutdown_connections 2023-11-27T15:22:12.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.690+0000 7fc38b244700 1 --2- 172.21.15.133:0/3446671083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc384101e80 0x7fc384108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.690+0000 7fc38b244700 1 --2- 172.21.15.133:0/3446671083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3841002f0 0x7fc384100700 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.690+0000 7fc38b244700 1 --2- 172.21.15.133:0/3446671083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3841014f0 0x7fc384101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.690+0000 7fc38b244700 1 -- 172.21.15.133:0/3446671083 >> 172.21.15.133:0/3446671083 conn(0x7fc3840fb900 msgr2=0x7fc3840fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:12.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.690+0000 7fc38b244700 1 -- 172.21.15.133:0/3446671083 shutdown_connections 2023-11-27T15:22:12.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.690+0000 7fc38b244700 1 -- 172.21.15.133:0/3446671083 wait complete. 2023-11-27T15:22:12.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.691+0000 7fc38b244700 1 Processor -- start 2023-11-27T15:22:12.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.691+0000 7fc38b244700 1 -- start start 2023-11-27T15:22:12.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.691+0000 7fc38b244700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3841002f0 0x7fc3841982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.691+0000 7fc38b244700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3841014f0 0x7fc384198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.691+0000 7fc38b244700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc384101e80 0x7fc38419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.691+0000 7fc38b244700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38419fe80 con 0x7fc3841014f0 2023-11-27T15:22:12.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.691+0000 7fc38b244700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38419ffc0 con 0x7fc384101e80 2023-11-27T15:22:12.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.691+0000 7fc38b244700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3841a0150 con 0x7fc3841002f0 2023-11-27T15:22:12.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.691+0000 7fc388fe0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3841002f0 0x7fc3841982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:12.697 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.692+0000 7fc383fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3841014f0 0x7fc384198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:12.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.692+0000 7fc388fe0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3841002f0 0x7fc3841982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:58690/0 (socket says 172.21.15.133:58690) 2023-11-27T15:22:12.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.692+0000 7fc388fe0700 1 -- 172.21.15.133:0/2121111854 learned_addr learned my addr 172.21.15.133:0/2121111854 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:12.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.692+0000 7fc3897e1700 1 --2- 172.21.15.133:0/2121111854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc384101e80 0x7fc38419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:12.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.692+0000 7fc388fe0700 1 -- 172.21.15.133:0/2121111854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc384101e80 msgr2=0x7fc38419f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:12.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.692+0000 7fc388fe0700 1 --2- 172.21.15.133:0/2121111854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc384101e80 0x7fc38419f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.692+0000 7fc388fe0700 1 -- 172.21.15.133:0/2121111854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3841014f0 msgr2=0x7fc384198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:12.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.692+0000 7fc388fe0700 1 --2- 172.21.15.133:0/2121111854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3841014f0 0x7fc384198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.692+0000 7fc388fe0700 1 -- 172.21.15.133:0/2121111854 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37c009c40 con 0x7fc3841002f0 2023-11-27T15:22:12.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.692+0000 7fc388fe0700 1 --2- 172.21.15.133:0/2121111854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3841002f0 0x7fc3841982f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc378004740 tx=0x7fc378009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:12.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.692+0000 7fc381ffb700 1 -- 172.21.15.133:0/2121111854 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37801d070 con 0x7fc3841002f0 2023-11-27T15:22:12.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.692+0000 7fc38b244700 1 -- 172.21.15.133:0/2121111854 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3780095f0 con 0x7fc3841002f0 2023-11-27T15:22:12.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.692+0000 7fc381ffb700 1 -- 172.21.15.133:0/2121111854 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc378021410 con 0x7fc3841002f0 2023-11-27T15:22:12.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.692+0000 7fc381ffb700 1 -- 172.21.15.133:0/2121111854 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc378017650 con 0x7fc3841002f0 2023-11-27T15:22:12.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.693+0000 7fc38b244700 1 -- 172.21.15.133:0/2121111854 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc38410bcd0 con 0x7fc3841002f0 2023-11-27T15:22:12.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.694+0000 7fc38b244700 1 -- 172.21.15.133:0/2121111854 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc38410be60 con 0x7fc3841002f0 2023-11-27T15:22:12.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.695+0000 7fc381ffb700 1 -- 172.21.15.133:0/2121111854 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fc378021580 con 0x7fc3841002f0 2023-11-27T15:22:12.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.697+0000 7fc381ffb700 1 --2- 172.21.15.133:0/2121111854 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc3740a11e0 0x7fc3740a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:12.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.697+0000 7fc381ffb700 1 -- 172.21.15.133:0/2121111854 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc3780ce330 con 0x7fc3841002f0 2023-11-27T15:22:12.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.697+0000 7fc383fff700 1 --2- 172.21.15.133:0/2121111854 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc3740a11e0 0x7fc3740a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:12.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.697+0000 7fc383fff700 1 --2- 172.21.15.133:0/2121111854 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc3740a11e0 0x7fc3740a3690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc370009c60 tx=0x7fc370009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:12.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.699+0000 7fc381ffb700 1 -- 172.21.15.133:0/2121111854 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc378097600 con 0x7fc3841002f0 2023-11-27T15:22:12.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.907+0000 7fc38b244700 1 -- 172.21.15.133:0/2121111854 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc384107e90 con 0x7fc3740a11e0 2023-11-27T15:22:12.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.921+0000 7fc381ffb700 1 -- 172.21.15.133:0/2121111854 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fc384107e90 con 0x7fc3740a11e0 2023-11-27T15:22:12.926 INFO:teuthology.orchestra.run.smithi133.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-11-27T15:22:12.926 INFO:teuthology.orchestra.run.smithi133.stdout:mds.a smithi133 running (108s) 23s ago 108s 13.8M - 18.0.0-3906-g1c956514 ace6904b066b dbae139ff71a 2023-11-27T15:22:12.926 INFO:teuthology.orchestra.run.smithi133.stdout:mds.b smithi174 running (106s) 34s ago 106s 29.8M - 18.0.0-3906-g1c956514 ace6904b066b 328951b76ca0 2023-11-27T15:22:12.926 INFO:teuthology.orchestra.run.smithi133.stdout:mds.c smithi177 running (119s) 34s ago 119s 21.0M - 18.0.0-3906-g1c956514 ace6904b066b b94086894dfe 2023-11-27T15:22:12.926 INFO:teuthology.orchestra.run.smithi133.stdout:mds.d smithi133 running (2m) 23s ago 2m 13.8M - 18.0.0-3906-g1c956514 ace6904b066b 5b47e112c91b 2023-11-27T15:22:12.926 INFO:teuthology.orchestra.run.smithi133.stdout:mds.e smithi174 running (2m) 34s ago 2m 20.8M - 18.0.0-3906-g1c956514 ace6904b066b 6ff342b4d31e 2023-11-27T15:22:12.926 INFO:teuthology.orchestra.run.smithi133.stdout:mds.f smithi177 running (2m) 34s ago 2m 18.9M - 18.0.0-3906-g1c956514 ace6904b066b 63f6ff1d61b8 2023-11-27T15:22:12.926 INFO:teuthology.orchestra.run.smithi133.stdout:mds.g smithi133 running (112s) 23s ago 111s 13.7M - 18.0.0-3906-g1c956514 ace6904b066b 19d3af6baf87 2023-11-27T15:22:12.926 INFO:teuthology.orchestra.run.smithi133.stdout:mds.h smithi174 running (117s) 34s ago 117s 18.7M - 18.0.0-3906-g1c956514 ace6904b066b 9c672774345d 2023-11-27T15:22:12.926 INFO:teuthology.orchestra.run.smithi133.stdout:mds.i smithi177 running (115s) 34s ago 114s 20.1M - 18.0.0-3906-g1c956514 ace6904b066b 630cb31efe89 2023-11-27T15:22:12.927 INFO:teuthology.orchestra.run.smithi133.stdout:mds.j smithi133 running (2m) 23s ago 2m 13.9M - 18.0.0-3906-g1c956514 ace6904b066b 0ef752b6cdaf 2023-11-27T15:22:12.927 INFO:teuthology.orchestra.run.smithi133.stdout:mds.k smithi174 running (2m) 34s ago 2m 14.0M - 18.0.0-3906-g1c956514 ace6904b066b cc229957c002 2023-11-27T15:22:12.927 INFO:teuthology.orchestra.run.smithi133.stdout:mds.l smithi177 running (2m) 34s ago 2m 15.1M - 18.0.0-3906-g1c956514 ace6904b066b a96fbb654cb8 2023-11-27T15:22:12.927 INFO:teuthology.orchestra.run.smithi133.stdout:mgr.x smithi133 *:9283,8765 running (9m) 23s ago 9m 521M - 18.0.0-3906-g1c956514 ace6904b066b c47377972a0e 2023-11-27T15:22:12.927 INFO:teuthology.orchestra.run.smithi133.stdout:mgr.y smithi174 *:8443,8765 running (7m) 34s ago 7m 446M - 18.0.0-3906-g1c956514 ace6904b066b cf69fe65b2dc 2023-11-27T15:22:12.927 INFO:teuthology.orchestra.run.smithi133.stdout:mgr.z smithi177 *:8443,8765 running (7m) 34s ago 7m 447M - 18.0.0-3906-g1c956514 ace6904b066b e66306280845 2023-11-27T15:22:12.927 INFO:teuthology.orchestra.run.smithi133.stdout:mon.a smithi133 running (9m) 23s ago 9m 93.0M 2048M 18.0.0-3906-g1c956514 ace6904b066b bbdb066cdec5 2023-11-27T15:22:12.927 INFO:teuthology.orchestra.run.smithi133.stdout:mon.b smithi174 running (7m) 34s ago 7m 63.9M 2048M 18.0.0-3906-g1c956514 ace6904b066b 19b907d479d9 2023-11-27T15:22:12.927 INFO:teuthology.orchestra.run.smithi133.stdout:mon.c smithi177 running (7m) 34s ago 7m 63.7M 2048M 18.0.0-3906-g1c956514 ace6904b066b c9f577aeeaa0 2023-11-27T15:22:12.927 INFO:teuthology.orchestra.run.smithi133.stdout:osd.0 smithi133 running (6m) 23s ago 6m 71.0M 4317M 18.0.0-3906-g1c956514 ace6904b066b d1cb12fb4889 2023-11-27T15:22:12.927 INFO:teuthology.orchestra.run.smithi133.stdout:osd.1 smithi174 running (6m) 34s ago 6m 76.7M 4292M 18.0.0-3906-g1c956514 ace6904b066b 0fa99dc133a5 2023-11-27T15:22:12.927 INFO:teuthology.orchestra.run.smithi133.stdout:osd.2 smithi177 running (6m) 34s ago 6m 74.6M 4292M 18.0.0-3906-g1c956514 ace6904b066b 5d497842bf6d 2023-11-27T15:22:12.927 INFO:teuthology.orchestra.run.smithi133.stdout:osd.3 smithi133 running (5m) 23s ago 5m 67.9M 4317M 18.0.0-3906-g1c956514 ace6904b066b 22aa4425388a 2023-11-27T15:22:12.927 INFO:teuthology.orchestra.run.smithi133.stdout:osd.4 smithi174 running (5m) 34s ago 5m 65.2M 4292M 18.0.0-3906-g1c956514 ace6904b066b 19eff7272499 2023-11-27T15:22:12.928 INFO:teuthology.orchestra.run.smithi133.stdout:osd.5 smithi177 running (4m) 34s ago 4m 68.7M 4292M 18.0.0-3906-g1c956514 ace6904b066b b46450f950fe 2023-11-27T15:22:12.928 INFO:teuthology.orchestra.run.smithi133.stdout:osd.6 smithi133 running (4m) 23s ago 4m 60.8M 4317M 18.0.0-3906-g1c956514 ace6904b066b dbdf6bbd3677 2023-11-27T15:22:12.928 INFO:teuthology.orchestra.run.smithi133.stdout:osd.7 smithi174 running (4m) 34s ago 4m 68.0M 4292M 18.0.0-3906-g1c956514 ace6904b066b 078d0de0abbb 2023-11-27T15:22:12.928 INFO:teuthology.orchestra.run.smithi133.stdout:osd.8 smithi177 running (3m) 34s ago 3m 61.0M 4292M 18.0.0-3906-g1c956514 ace6904b066b f78790ff089d 2023-11-27T15:22:12.928 INFO:teuthology.orchestra.run.smithi133.stdout:osd.9 smithi133 running (3m) 23s ago 3m 60.0M 4317M 18.0.0-3906-g1c956514 ace6904b066b 38d62965c32b 2023-11-27T15:22:12.928 INFO:teuthology.orchestra.run.smithi133.stdout:osd.10 smithi174 running (2m) 34s ago 2m 65.9M 4292M 18.0.0-3906-g1c956514 ace6904b066b 408f5c681646 2023-11-27T15:22:12.928 INFO:teuthology.orchestra.run.smithi133.stdout:osd.11 smithi177 running (2m) 34s ago 2m 55.8M 4292M 18.0.0-3906-g1c956514 ace6904b066b 17e2dfe4b037 2023-11-27T15:22:12.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.923+0000 7fc38b244700 1 -- 172.21.15.133:0/2121111854 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc3740a11e0 msgr2=0x7fc3740a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:12.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.923+0000 7fc38b244700 1 --2- 172.21.15.133:0/2121111854 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc3740a11e0 0x7fc3740a3690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc370009c60 tx=0x7fc370009260 comp rx=0 tx=0).stop 2023-11-27T15:22:12.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.923+0000 7fc38b244700 1 -- 172.21.15.133:0/2121111854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3841002f0 msgr2=0x7fc3841982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:12.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.923+0000 7fc38b244700 1 --2- 172.21.15.133:0/2121111854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3841002f0 0x7fc3841982f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc378004740 tx=0x7fc378009580 comp rx=0 tx=0).stop 2023-11-27T15:22:12.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.923+0000 7fc38b244700 1 -- 172.21.15.133:0/2121111854 shutdown_connections 2023-11-27T15:22:12.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.923+0000 7fc38b244700 1 --2- 172.21.15.133:0/2121111854 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc3740a11e0 0x7fc3740a3690 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.923+0000 7fc38b244700 1 --2- 172.21.15.133:0/2121111854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc384101e80 0x7fc38419f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.923+0000 7fc38b244700 1 --2- 172.21.15.133:0/2121111854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3841002f0 0x7fc3841982f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.923+0000 7fc38b244700 1 --2- 172.21.15.133:0/2121111854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc3841014f0 0x7fc384198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:12.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.923+0000 7fc38b244700 1 -- 172.21.15.133:0/2121111854 >> 172.21.15.133:0/2121111854 conn(0x7fc3840fb900 msgr2=0x7fc3840fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:12.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.923+0000 7fc38b244700 1 -- 172.21.15.133:0/2121111854 shutdown_connections 2023-11-27T15:22:12.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:12.923+0000 7fc38b244700 1 -- 172.21.15.133:0/2121111854 wait complete. 2023-11-27T15:22:13.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:12 smithi177 ceph-mon[131504]: pgmap v327: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 11 B/s, 0 objects/s recovering 2023-11-27T15:22:13.123 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:12 smithi133 ceph-mon[97860]: pgmap v327: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 11 B/s, 0 objects/s recovering 2023-11-27T15:22:13.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:12 smithi174 ceph-mon[131110]: pgmap v327: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 11 B/s, 0 objects/s recovering 2023-11-27T15:22:13.460 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- bash -c 'ceph orch ls' 2023-11-27T15:22:14.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:13 smithi177 ceph-mon[131504]: from='client.25168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:22:14.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:13 smithi174 ceph-mon[131110]: from='client.25168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:22:14.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:13 smithi133 ceph-mon[97860]: from='client.25168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:22:14.313 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:22:14.906 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:14 smithi133 ceph-mon[97860]: pgmap v328: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 10 B/s, 0 objects/s recovering 2023-11-27T15:22:15.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:14 smithi177 ceph-mon[131504]: pgmap v328: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 10 B/s, 0 objects/s recovering 2023-11-27T15:22:15.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:14 smithi174 ceph-mon[131110]: pgmap v328: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 10 B/s, 0 objects/s recovering 2023-11-27T15:22:15.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.825+0000 7f4447649700 1 -- 172.21.15.133:0/3657942530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4440100d70 msgr2=0x7f4440103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:15.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.825+0000 7f4447649700 1 --2- 172.21.15.133:0/3657942530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4440100d70 0x7f4440103150 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f443c009990 tx=0x7f443c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:15.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.825+0000 7f4447649700 1 -- 172.21.15.133:0/3657942530 shutdown_connections 2023-11-27T15:22:15.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.825+0000 7f4447649700 1 --2- 172.21.15.133:0/3657942530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4440105fb0 0x7f4440108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:15.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.825+0000 7f4447649700 1 --2- 172.21.15.133:0/3657942530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4440100d70 0x7f4440103150 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:15.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.825+0000 7f4447649700 1 --2- 172.21.15.133:0/3657942530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4440103690 0x7f4440105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:15.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.825+0000 7f4447649700 1 -- 172.21.15.133:0/3657942530 >> 172.21.15.133:0/3657942530 conn(0x7f44400fa7f0 msgr2=0x7f44400fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:15.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.825+0000 7f4447649700 1 -- 172.21.15.133:0/3657942530 shutdown_connections 2023-11-27T15:22:15.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.825+0000 7f4447649700 1 -- 172.21.15.133:0/3657942530 wait complete. 2023-11-27T15:22:15.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.826+0000 7f4447649700 1 Processor -- start 2023-11-27T15:22:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.826+0000 7f4447649700 1 -- start start 2023-11-27T15:22:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.826+0000 7f4447649700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4440100d70 0x7f44401982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.827+0000 7f4447649700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4440103690 0x7f44401987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.827+0000 7f4447649700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4440105fb0 0x7f444019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.827+0000 7f4447649700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444019fe40 con 0x7f4440103690 2023-11-27T15:22:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.827+0000 7f4447649700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444019ff80 con 0x7f4440105fb0 2023-11-27T15:22:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.827+0000 7f4447649700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44401a0110 con 0x7f4440100d70 2023-11-27T15:22:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.827+0000 7f4444be4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4440103690 0x7f44401987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.827+0000 7f4444be4700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4440103690 0x7f44401987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58478/0 (socket says 172.21.15.133:58478) 2023-11-27T15:22:15.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.827+0000 7f4444be4700 1 -- 172.21.15.133:0/4142804568 learned_addr learned my addr 172.21.15.133:0/4142804568 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.827+0000 7f4445be6700 1 --2- 172.21.15.133:0/4142804568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4440105fb0 0x7f444019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.827+0000 7f44453e5700 1 --2- 172.21.15.133:0/4142804568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4440100d70 0x7f44401982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.827+0000 7f4444be4700 1 -- 172.21.15.133:0/4142804568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4440100d70 msgr2=0x7f44401982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.827+0000 7f4444be4700 1 --2- 172.21.15.133:0/4142804568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4440100d70 0x7f44401982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.827+0000 7f4444be4700 1 -- 172.21.15.133:0/4142804568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4440105fb0 msgr2=0x7f444019f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.827+0000 7f4444be4700 1 --2- 172.21.15.133:0/4142804568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4440105fb0 0x7f444019f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.827+0000 7f4444be4700 1 -- 172.21.15.133:0/4142804568 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c0095f0 con 0x7f4440103690 2023-11-27T15:22:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.828+0000 7f4444be4700 1 --2- 172.21.15.133:0/4142804568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4440103690 0x7f44401987f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f443000e9d0 tx=0x7f443000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.828+0000 7f44367fc700 1 -- 172.21.15.133:0/4142804568 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430009790 con 0x7f4440103690 2023-11-27T15:22:15.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.828+0000 7f44367fc700 1 -- 172.21.15.133:0/4142804568 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f443000ee70 con 0x7f4440103690 2023-11-27T15:22:15.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.828+0000 7f44367fc700 1 -- 172.21.15.133:0/4142804568 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430010ea0 con 0x7f4440103690 2023-11-27T15:22:15.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.828+0000 7f4447649700 1 -- 172.21.15.133:0/4142804568 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444010bd00 con 0x7f4440103690 2023-11-27T15:22:15.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.828+0000 7f4447649700 1 -- 172.21.15.133:0/4142804568 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444010bb70 con 0x7f4440103690 2023-11-27T15:22:15.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.829+0000 7f4447649700 1 -- 172.21.15.133:0/4142804568 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4440107ba0 con 0x7f4440103690 2023-11-27T15:22:15.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.832+0000 7f44367fc700 1 -- 172.21.15.133:0/4142804568 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f443000cbd0 con 0x7f4440103690 2023-11-27T15:22:15.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.832+0000 7f44367fc700 1 --2- 172.21.15.133:0/4142804568 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f442c0a1260 0x7f442c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:15.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.832+0000 7f44367fc700 1 -- 172.21.15.133:0/4142804568 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f4430014070 con 0x7f4440103690 2023-11-27T15:22:15.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.833+0000 7f44453e5700 1 --2- 172.21.15.133:0/4142804568 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f442c0a1260 0x7f442c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:15.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.833+0000 7f44453e5700 1 --2- 172.21.15.133:0/4142804568 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f442c0a1260 0x7f442c0a3710 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f443c00b2e0 tx=0x7f443c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:15.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:15.837+0000 7f44367fc700 1 -- 172.21.15.133:0/4142804568 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4430051730 con 0x7f4440103690 2023-11-27T15:22:16.045 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:16.040+0000 7f4447649700 1 -- 172.21.15.133:0/4142804568 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4440061c10 con 0x7f442c0a1260 2023-11-27T15:22:16.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:16.046+0000 7f44367fc700 1 -- 172.21.15.133:0/4142804568 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f4440061c10 con 0x7f442c0a1260 2023-11-27T15:22:16.051 INFO:teuthology.orchestra.run.smithi133.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-11-27T15:22:16.051 INFO:teuthology.orchestra.run.smithi133.stdout:mds.all 12/12 37s ago 2m smithi133=a;smithi133=d;smithi133=g;smithi133=j;smithi174=b;smithi174=e;smithi174=h;smithi174=k;smithi177=c;smithi177=f;smithi177=i;smithi177=l;count:12 2023-11-27T15:22:16.051 INFO:teuthology.orchestra.run.smithi133.stdout:mgr 3/3 37s ago 7m smithi133=x;smithi174=y;smithi177=z;count:3 2023-11-27T15:22:16.051 INFO:teuthology.orchestra.run.smithi133.stdout:mon 3/3 37s ago 7m smithi133:172.21.15.133=a;smithi174:172.21.15.174=b;smithi177:172.21.15.177=c;count:3 2023-11-27T15:22:16.051 INFO:teuthology.orchestra.run.smithi133.stdout:osd 12 37s ago - 2023-11-27T15:22:16.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:16.048+0000 7f4447649700 1 -- 172.21.15.133:0/4142804568 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f442c0a1260 msgr2=0x7f442c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:16.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:16.048+0000 7f4447649700 1 --2- 172.21.15.133:0/4142804568 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f442c0a1260 0x7f442c0a3710 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f443c00b2e0 tx=0x7f443c005fd0 comp rx=0 tx=0).stop 2023-11-27T15:22:16.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:16.048+0000 7f4447649700 1 -- 172.21.15.133:0/4142804568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4440103690 msgr2=0x7f44401987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:16.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:16.048+0000 7f4447649700 1 --2- 172.21.15.133:0/4142804568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4440103690 0x7f44401987f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f443000e9d0 tx=0x7f443000ece0 comp rx=0 tx=0).stop 2023-11-27T15:22:16.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:16.048+0000 7f4447649700 1 -- 172.21.15.133:0/4142804568 shutdown_connections 2023-11-27T15:22:16.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:16.048+0000 7f4447649700 1 --2- 172.21.15.133:0/4142804568 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f442c0a1260 0x7f442c0a3710 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:16.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:16.048+0000 7f4447649700 1 --2- 172.21.15.133:0/4142804568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4440105fb0 0x7f444019f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:16.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:16.048+0000 7f4447649700 1 --2- 172.21.15.133:0/4142804568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4440100d70 0x7f44401982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:16.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:16.048+0000 7f4447649700 1 --2- 172.21.15.133:0/4142804568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4440103690 0x7f44401987f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:16.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:16.048+0000 7f4447649700 1 -- 172.21.15.133:0/4142804568 >> 172.21.15.133:0/4142804568 conn(0x7f44400fa7f0 msgr2=0x7f44400ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:16.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:16.049+0000 7f4447649700 1 -- 172.21.15.133:0/4142804568 shutdown_connections 2023-11-27T15:22:16.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:16.049+0000 7f4447649700 1 -- 172.21.15.133:0/4142804568 wait complete. 2023-11-27T15:22:16.513 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- bash -c 'ceph orch host ls' 2023-11-27T15:22:17.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:16 smithi177 ceph-mon[131504]: pgmap v329: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2023-11-27T15:22:17.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:16 smithi177 ceph-mon[131504]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:22:17.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:16 smithi174 ceph-mon[131110]: pgmap v329: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2023-11-27T15:22:17.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:16 smithi174 ceph-mon[131110]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:22:17.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:16 smithi133 ceph-mon[97860]: pgmap v329: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2023-11-27T15:22:17.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:16 smithi133 ceph-mon[97860]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:22:17.374 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:22:18.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.760+0000 7fd59d990700 1 -- 172.21.15.133:0/940806286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5981014f0 msgr2=0x7fd598101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:18.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.760+0000 7fd59d990700 1 --2- 172.21.15.133:0/940806286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5981014f0 0x7fd598101940 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009990 tx=0x7fd58c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:18.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.760+0000 7fd59d990700 1 -- 172.21.15.133:0/940806286 shutdown_connections 2023-11-27T15:22:18.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.760+0000 7fd59d990700 1 --2- 172.21.15.133:0/940806286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd598101e80 0x7fd598108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:18.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.760+0000 7fd59d990700 1 --2- 172.21.15.133:0/940806286 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5981002f0 0x7fd598100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:18.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.760+0000 7fd59d990700 1 --2- 172.21.15.133:0/940806286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5981014f0 0x7fd598101940 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:18.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.760+0000 7fd59d990700 1 -- 172.21.15.133:0/940806286 >> 172.21.15.133:0/940806286 conn(0x7fd5980fb900 msgr2=0x7fd5980fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:18.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.760+0000 7fd59d990700 1 -- 172.21.15.133:0/940806286 shutdown_connections 2023-11-27T15:22:18.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.760+0000 7fd59d990700 1 -- 172.21.15.133:0/940806286 wait complete. 2023-11-27T15:22:18.767 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.761+0000 7fd59d990700 1 Processor -- start 2023-11-27T15:22:18.767 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.762+0000 7fd59d990700 1 -- start start 2023-11-27T15:22:18.767 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.762+0000 7fd59d990700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5981002f0 0x7fd598198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:18.767 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.762+0000 7fd59d990700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5981014f0 0x7fd5981987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:18.767 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.762+0000 7fd59d990700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd598101e80 0x7fd59819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:18.767 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.762+0000 7fd59d990700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59819fdf0 con 0x7fd5981002f0 2023-11-27T15:22:18.767 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.762+0000 7fd59d990700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59819ff30 con 0x7fd598101e80 2023-11-27T15:22:18.767 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.762+0000 7fd59d990700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5981a00c0 con 0x7fd5981014f0 2023-11-27T15:22:18.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.762+0000 7fd5967fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5981014f0 0x7fd5981987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:18.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.762+0000 7fd5967fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5981014f0 0x7fd5981987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:51046/0 (socket says 172.21.15.133:51046) 2023-11-27T15:22:18.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.762+0000 7fd5967fc700 1 -- 172.21.15.133:0/2791920646 learned_addr learned my addr 172.21.15.133:0/2791920646 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:18.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.762+0000 7fd596ffd700 1 --2- 172.21.15.133:0/2791920646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5981002f0 0x7fd598198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:18.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.762+0000 7fd5977fe700 1 --2- 172.21.15.133:0/2791920646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd598101e80 0x7fd59819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:18.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.763+0000 7fd5967fc700 1 -- 172.21.15.133:0/2791920646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd598101e80 msgr2=0x7fd59819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:18.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.763+0000 7fd5967fc700 1 --2- 172.21.15.133:0/2791920646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd598101e80 0x7fd59819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:18.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.763+0000 7fd5967fc700 1 -- 172.21.15.133:0/2791920646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5981002f0 msgr2=0x7fd598198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:18.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.763+0000 7fd5967fc700 1 --2- 172.21.15.133:0/2791920646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5981002f0 0x7fd598198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:18.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.763+0000 7fd5967fc700 1 -- 172.21.15.133:0/2791920646 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c0095f0 con 0x7fd5981014f0 2023-11-27T15:22:18.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.763+0000 7fd5967fc700 1 --2- 172.21.15.133:0/2791920646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5981014f0 0x7fd5981987a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd58c004760 tx=0x7fd58c004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:18.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.763+0000 7fd59c98e700 1 -- 172.21.15.133:0/2791920646 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c01d070 con 0x7fd5981014f0 2023-11-27T15:22:18.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.763+0000 7fd59d990700 1 -- 172.21.15.133:0/2791920646 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59810bca0 con 0x7fd5981014f0 2023-11-27T15:22:18.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.763+0000 7fd59c98e700 1 -- 172.21.15.133:0/2791920646 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd58c004830 con 0x7fd5981014f0 2023-11-27T15:22:18.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.763+0000 7fd59c98e700 1 -- 172.21.15.133:0/2791920646 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c005430 con 0x7fd5981014f0 2023-11-27T15:22:18.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.763+0000 7fd59d990700 1 -- 172.21.15.133:0/2791920646 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd59810bb10 con 0x7fd5981014f0 2023-11-27T15:22:18.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.764+0000 7fd59d990700 1 -- 172.21.15.133:0/2791920646 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5981a06b0 con 0x7fd5981014f0 2023-11-27T15:22:18.770 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.766+0000 7fd59c98e700 1 -- 172.21.15.133:0/2791920646 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fd58c00fad0 con 0x7fd5981014f0 2023-11-27T15:22:18.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.767+0000 7fd59c98e700 1 --2- 172.21.15.133:0/2791920646 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd5800a11e0 0x7fd5800a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:18.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.768+0000 7fd59c98e700 1 -- 172.21.15.133:0/2791920646 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd58c0ce070 con 0x7fd5981014f0 2023-11-27T15:22:18.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.768+0000 7fd596ffd700 1 --2- 172.21.15.133:0/2791920646 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd5800a11e0 0x7fd5800a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:18.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.768+0000 7fd596ffd700 1 --2- 172.21.15.133:0/2791920646 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd5800a11e0 0x7fd5800a3690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd598101350 tx=0x7fd588009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:18.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.770+0000 7fd59c98e700 1 -- 172.21.15.133:0/2791920646 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd58c097340 con 0x7fd5981014f0 2023-11-27T15:22:18.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.973+0000 7fd59d990700 1 -- 172.21.15.133:0/2791920646 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5981a0d20 con 0x7fd5800a11e0 2023-11-27T15:22:18.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.974+0000 7fd59c98e700 1 -- 172.21.15.133:0/2791920646 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fd5981a0d20 con 0x7fd5800a11e0 2023-11-27T15:22:18.978 INFO:teuthology.orchestra.run.smithi133.stdout:HOST ADDR LABELS STATUS 2023-11-27T15:22:18.978 INFO:teuthology.orchestra.run.smithi133.stdout:smithi133 172.21.15.133 2023-11-27T15:22:18.979 INFO:teuthology.orchestra.run.smithi133.stdout:smithi174 172.21.15.174 2023-11-27T15:22:18.979 INFO:teuthology.orchestra.run.smithi133.stdout:smithi177 172.21.15.177 2023-11-27T15:22:18.979 INFO:teuthology.orchestra.run.smithi133.stdout:3 hosts in cluster 2023-11-27T15:22:18.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.977+0000 7fd59d990700 1 -- 172.21.15.133:0/2791920646 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd5800a11e0 msgr2=0x7fd5800a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:18.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.977+0000 7fd59d990700 1 --2- 172.21.15.133:0/2791920646 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd5800a11e0 0x7fd5800a3690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd598101350 tx=0x7fd588009260 comp rx=0 tx=0).stop 2023-11-27T15:22:18.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.977+0000 7fd59d990700 1 -- 172.21.15.133:0/2791920646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5981014f0 msgr2=0x7fd5981987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:18.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.977+0000 7fd59d990700 1 --2- 172.21.15.133:0/2791920646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5981014f0 0x7fd5981987a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd58c004760 tx=0x7fd58c004790 comp rx=0 tx=0).stop 2023-11-27T15:22:18.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.977+0000 7fd59d990700 1 -- 172.21.15.133:0/2791920646 shutdown_connections 2023-11-27T15:22:18.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.977+0000 7fd59d990700 1 --2- 172.21.15.133:0/2791920646 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd5800a11e0 0x7fd5800a3690 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:18.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.977+0000 7fd59d990700 1 --2- 172.21.15.133:0/2791920646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd598101e80 0x7fd59819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:18.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.977+0000 7fd59d990700 1 --2- 172.21.15.133:0/2791920646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5981002f0 0x7fd598198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:18.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.977+0000 7fd59d990700 1 --2- 172.21.15.133:0/2791920646 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd5981014f0 0x7fd5981987a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:18.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.977+0000 7fd59d990700 1 -- 172.21.15.133:0/2791920646 >> 172.21.15.133:0/2791920646 conn(0x7fd5980fb900 msgr2=0x7fd5980fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:18.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.977+0000 7fd59d990700 1 -- 172.21.15.133:0/2791920646 shutdown_connections 2023-11-27T15:22:18.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:18.977+0000 7fd59d990700 1 -- 172.21.15.133:0/2791920646 wait complete. 2023-11-27T15:22:19.002 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:18 smithi133 ceph-mon[97860]: pgmap v330: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2023-11-27T15:22:19.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:18 smithi177 ceph-mon[131504]: pgmap v330: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2023-11-27T15:22:19.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:18 smithi174 ceph-mon[131110]: pgmap v330: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 9 B/s, 0 objects/s recovering 2023-11-27T15:22:19.468 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- bash -c 'ceph orch device ls' 2023-11-27T15:22:20.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:19 smithi133 ceph-mon[97860]: from='client.25180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:22:20.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:19 smithi177 ceph-mon[131504]: from='client.25180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:22:20.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:19 smithi174 ceph-mon[131110]: from='client.25180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:22:20.318 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:22:20.872 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:20 smithi133 ceph-mon[97860]: pgmap v331: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2023-11-27T15:22:21.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:20 smithi177 ceph-mon[131504]: pgmap v331: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2023-11-27T15:22:21.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:20 smithi174 ceph-mon[131110]: pgmap v331: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 9 B/s, 0 objects/s recovering 2023-11-27T15:22:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.850+0000 7fb7dfc85700 1 -- 172.21.15.133:0/967177190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7d8100d70 msgr2=0x7fb7d8107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.850+0000 7fb7dfc85700 1 --2- 172.21.15.133:0/967177190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7d8100d70 0x7fb7d8107630 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000b0a0 tx=0x7fb7d000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.850+0000 7fb7dfc85700 1 -- 172.21.15.133:0/967177190 shutdown_connections 2023-11-27T15:22:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.850+0000 7fb7dfc85700 1 --2- 172.21.15.133:0/967177190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7d8100d70 0x7fb7d8107630 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.850+0000 7fb7dfc85700 1 --2- 172.21.15.133:0/967177190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d80ff1e0 0x7fb7d80ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.850+0000 7fb7dfc85700 1 --2- 172.21.15.133:0/967177190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7d81003e0 0x7fb7d8100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:21.855 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.850+0000 7fb7dfc85700 1 -- 172.21.15.133:0/967177190 >> 172.21.15.133:0/967177190 conn(0x7fb7d80fa7f0 msgr2=0x7fb7d80fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:21.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.850+0000 7fb7dfc85700 1 -- 172.21.15.133:0/967177190 shutdown_connections 2023-11-27T15:22:21.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.850+0000 7fb7dfc85700 1 -- 172.21.15.133:0/967177190 wait complete. 2023-11-27T15:22:21.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.852+0000 7fb7dfc85700 1 Processor -- start 2023-11-27T15:22:21.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.852+0000 7fb7dfc85700 1 -- start start 2023-11-27T15:22:21.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.852+0000 7fb7dfc85700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d80ff1e0 0x7fb7d8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:21.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.852+0000 7fb7dfc85700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7d81003e0 0x7fb7d81988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:21.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.852+0000 7fb7dfc85700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7d8100d70 0x7fb7d819f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:21.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.852+0000 7fb7dfc85700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d819ff10 con 0x7fb7d8100d70 2023-11-27T15:22:21.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.852+0000 7fb7dfc85700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d81a0050 con 0x7fb7d80ff1e0 2023-11-27T15:22:21.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.852+0000 7fb7dfc85700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d81a01e0 con 0x7fb7d81003e0 2023-11-27T15:22:21.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.853+0000 7fb7de222700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7d8100d70 0x7fb7d819f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:21.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.853+0000 7fb7dda21700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d80ff1e0 0x7fb7d8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:21.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.853+0000 7fb7dd220700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7d81003e0 0x7fb7d81988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:21.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.853+0000 7fb7de222700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7d8100d70 0x7fb7d819f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58496/0 (socket says 172.21.15.133:58496) 2023-11-27T15:22:21.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.853+0000 7fb7de222700 1 -- 172.21.15.133:0/3679892862 learned_addr learned my addr 172.21.15.133:0/3679892862 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:21.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.853+0000 7fb7dda21700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d80ff1e0 0x7fb7d8198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:49966/0 (socket says 172.21.15.133:49966) 2023-11-27T15:22:21.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.853+0000 7fb7de222700 1 -- 172.21.15.133:0/3679892862 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7d81003e0 msgr2=0x7fb7d81988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:21.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.853+0000 7fb7de222700 1 --2- 172.21.15.133:0/3679892862 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7d81003e0 0x7fb7d81988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:21.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.853+0000 7fb7de222700 1 -- 172.21.15.133:0/3679892862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d80ff1e0 msgr2=0x7fb7d8198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:21.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.853+0000 7fb7de222700 1 --2- 172.21.15.133:0/3679892862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d80ff1e0 0x7fb7d8198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:21.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.853+0000 7fb7de222700 1 -- 172.21.15.133:0/3679892862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d0009d00 con 0x7fb7d8100d70 2023-11-27T15:22:21.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.853+0000 7fb7dda21700 1 --2- 172.21.15.133:0/3679892862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d80ff1e0 0x7fb7d8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:21.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.853+0000 7fb7de222700 1 --2- 172.21.15.133:0/3679892862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7d8100d70 0x7fb7d819f940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000b070 tx=0x7fb7d000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:21.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.853+0000 7fb7ceffd700 1 -- 172.21.15.133:0/3679892862 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d000e040 con 0x7fb7d8100d70 2023-11-27T15:22:21.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.854+0000 7fb7dfc85700 1 -- 172.21.15.133:0/3679892862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d810ab90 con 0x7fb7d8100d70 2023-11-27T15:22:21.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.854+0000 7fb7ceffd700 1 -- 172.21.15.133:0/3679892862 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7d00094c0 con 0x7fb7d8100d70 2023-11-27T15:22:21.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.854+0000 7fb7ceffd700 1 -- 172.21.15.133:0/3679892862 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0005280 con 0x7fb7d8100d70 2023-11-27T15:22:21.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.854+0000 7fb7dfc85700 1 -- 172.21.15.133:0/3679892862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d810aa00 con 0x7fb7d8100d70 2023-11-27T15:22:21.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.855+0000 7fb7dfc85700 1 -- 172.21.15.133:0/3679892862 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d8066ea0 con 0x7fb7d8100d70 2023-11-27T15:22:21.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.857+0000 7fb7ceffd700 1 -- 172.21.15.133:0/3679892862 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fb7d001d470 con 0x7fb7d8100d70 2023-11-27T15:22:21.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.858+0000 7fb7ceffd700 1 --2- 172.21.15.133:0/3679892862 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb7c40a11e0 0x7fb7c40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:21.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.858+0000 7fb7ceffd700 1 -- 172.21.15.133:0/3679892862 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb7d00ce360 con 0x7fb7d8100d70 2023-11-27T15:22:21.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.858+0000 7fb7dda21700 1 --2- 172.21.15.133:0/3679892862 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb7c40a11e0 0x7fb7c40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:21.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.859+0000 7fb7dda21700 1 --2- 172.21.15.133:0/3679892862 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb7c40a11e0 0x7fb7c40a3690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8100240 tx=0x7fb7d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:21.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:21.862+0000 7fb7ceffd700 1 -- 172.21.15.133:0/3679892862 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb7d0097630 con 0x7fb7d8100d70 2023-11-27T15:22:22.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:22.069+0000 7fb7dfc85700 1 -- 172.21.15.133:0/3679892862 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7d8068d00 con 0x7fb7c40a11e0 2023-11-27T15:22:22.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:22.071+0000 7fb7ceffd700 1 -- 172.21.15.133:0/3679892862 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7fb7d8068d00 con 0x7fb7c40a11e0 2023-11-27T15:22:22.075 INFO:teuthology.orchestra.run.smithi133.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-11-27T15:22:22.075 INFO:teuthology.orchestra.run.smithi133.stdout:smithi133 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620401AQ400BGN 372G No 3m ago LVM detected, locked 2023-11-27T15:22:22.075 INFO:teuthology.orchestra.run.smithi133.stdout:smithi174 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62050074400BGN 372G No 2m ago LVM detected, locked 2023-11-27T15:22:22.075 INFO:teuthology.orchestra.run.smithi133.stdout:smithi177 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233007S400BGN 372G No 2m ago LVM detected, locked 2023-11-27T15:22:22.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:22.073+0000 7fb7dfc85700 1 -- 172.21.15.133:0/3679892862 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb7c40a11e0 msgr2=0x7fb7c40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:22.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:22.073+0000 7fb7dfc85700 1 --2- 172.21.15.133:0/3679892862 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb7c40a11e0 0x7fb7c40a3690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8100240 tx=0x7fb7d4009260 comp rx=0 tx=0).stop 2023-11-27T15:22:22.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:22.073+0000 7fb7dfc85700 1 -- 172.21.15.133:0/3679892862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7d8100d70 msgr2=0x7fb7d819f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:22.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:22.073+0000 7fb7dfc85700 1 --2- 172.21.15.133:0/3679892862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7d8100d70 0x7fb7d819f940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000b070 tx=0x7fb7d000bce0 comp rx=0 tx=0).stop 2023-11-27T15:22:22.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:22.073+0000 7fb7dfc85700 1 -- 172.21.15.133:0/3679892862 shutdown_connections 2023-11-27T15:22:22.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:22.073+0000 7fb7dfc85700 1 --2- 172.21.15.133:0/3679892862 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb7c40a11e0 0x7fb7c40a3690 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:22.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:22.073+0000 7fb7dfc85700 1 --2- 172.21.15.133:0/3679892862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7d8100d70 0x7fb7d819f940 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:22.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:22.073+0000 7fb7dfc85700 1 --2- 172.21.15.133:0/3679892862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7d80ff1e0 0x7fb7d8198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:22.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:22.073+0000 7fb7dfc85700 1 --2- 172.21.15.133:0/3679892862 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7d81003e0 0x7fb7d81988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:22.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:22.073+0000 7fb7dfc85700 1 -- 172.21.15.133:0/3679892862 >> 172.21.15.133:0/3679892862 conn(0x7fb7d80fa7f0 msgr2=0x7fb7d80685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:22.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:22.074+0000 7fb7dfc85700 1 -- 172.21.15.133:0/3679892862 shutdown_connections 2023-11-27T15:22:22.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:22.074+0000 7fb7dfc85700 1 -- 172.21.15.133:0/3679892862 wait complete. 2023-11-27T15:22:22.513 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-11-27T15:22:22.522 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi133.front.sepia.ceph.com 2023-11-27T15:22:22.523 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- bash -c 'ceph fs dump' 2023-11-27T15:22:23.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:22 smithi177 ceph-mon[131504]: pgmap v332: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:22:23.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:22 smithi177 ceph-mon[131504]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:22:23.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:22 smithi174 ceph-mon[131110]: pgmap v332: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:22:23.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:22 smithi174 ceph-mon[131110]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:22:23.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:22 smithi133 ceph-mon[97860]: pgmap v332: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:22:23.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:22 smithi133 ceph-mon[97860]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-11-27T15:22:23.376 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:22:24.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.874+0000 7fe8f2318700 1 -- 172.21.15.133:0/2343148855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8ec1081c0 msgr2=0x7fe8ec10a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:24.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.874+0000 7fe8f2318700 1 --2- 172.21.15.133:0/2343148855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8ec1081c0 0x7fe8ec10a600 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400b0a0 tx=0x7fe8e400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:24.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.874+0000 7fe8f2318700 1 -- 172.21.15.133:0/2343148855 shutdown_connections 2023-11-27T15:22:24.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.874+0000 7fe8f2318700 1 --2- 172.21.15.133:0/2343148855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8ec1081c0 0x7fe8ec10a600 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:24.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.874+0000 7fe8f2318700 1 --2- 172.21.15.133:0/2343148855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ec074b20 0x7fe8ec072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:24.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.874+0000 7fe8f2318700 1 --2- 172.21.15.133:0/2343148855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8ec073550 0x7fe8ec0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:24.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.874+0000 7fe8f2318700 1 -- 172.21.15.133:0/2343148855 >> 172.21.15.133:0/2343148855 conn(0x7fe8ec0fb900 msgr2=0x7fe8ec0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:24.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.874+0000 7fe8f2318700 1 -- 172.21.15.133:0/2343148855 shutdown_connections 2023-11-27T15:22:24.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.874+0000 7fe8f2318700 1 -- 172.21.15.133:0/2343148855 wait complete. 2023-11-27T15:22:24.880 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.875+0000 7fe8f2318700 1 Processor -- start 2023-11-27T15:22:24.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.875+0000 7fe8f2318700 1 -- start start 2023-11-27T15:22:24.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.875+0000 7fe8f2318700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8ec073550 0x7fe8ec198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:24.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.875+0000 7fe8f2318700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8ec074b20 0x7fe8ec198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:24.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.876+0000 7fe8f2318700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ec1081c0 0x7fe8ec19f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:24.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.876+0000 7fe8f2318700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec19fec0 con 0x7fe8ec073550 2023-11-27T15:22:24.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.876+0000 7fe8f2318700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec1a0000 con 0x7fe8ec074b20 2023-11-27T15:22:24.881 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.876+0000 7fe8f2318700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec1a0190 con 0x7fe8ec1081c0 2023-11-27T15:22:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.876+0000 7fe8ebfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8ec073550 0x7fe8ec198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:24.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.876+0000 7fe8ebfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8ec073550 0x7fe8ec198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54838/0 (socket says 172.21.15.133:54838) 2023-11-27T15:22:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.876+0000 7fe8ebfff700 1 -- 172.21.15.133:0/1276607756 learned_addr learned my addr 172.21.15.133:0/1276607756 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.876+0000 7fe8eb7fe700 1 --2- 172.21.15.133:0/1276607756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8ec074b20 0x7fe8ec198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.876+0000 7fe8f08b5700 1 --2- 172.21.15.133:0/1276607756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ec1081c0 0x7fe8ec19f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.876+0000 7fe8ebfff700 1 -- 172.21.15.133:0/1276607756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ec1081c0 msgr2=0x7fe8ec19f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.876+0000 7fe8ebfff700 1 --2- 172.21.15.133:0/1276607756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ec1081c0 0x7fe8ec19f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.876+0000 7fe8ebfff700 1 -- 172.21.15.133:0/1276607756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8ec074b20 msgr2=0x7fe8ec198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.876+0000 7fe8ebfff700 1 --2- 172.21.15.133:0/1276607756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8ec074b20 0x7fe8ec198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.876+0000 7fe8ebfff700 1 -- 172.21.15.133:0/1276607756 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e0009520 con 0x7fe8ec073550 2023-11-27T15:22:24.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.876+0000 7fe8eb7fe700 1 --2- 172.21.15.133:0/1276607756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8ec074b20 0x7fe8ec198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:24.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.877+0000 7fe8ebfff700 1 --2- 172.21.15.133:0/1276607756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8ec073550 0x7fe8ec198330 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00eb60 tx=0x7fe8dc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:24.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.877+0000 7fe8e97fa700 1 -- 172.21.15.133:0/1276607756 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc00cbd0 con 0x7fe8ec073550 2023-11-27T15:22:24.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.877+0000 7fe8e97fa700 1 -- 172.21.15.133:0/1276607756 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8dc0044e0 con 0x7fe8ec073550 2023-11-27T15:22:24.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.877+0000 7fe8f2318700 1 -- 172.21.15.133:0/1276607756 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8dc009520 con 0x7fe8ec073550 2023-11-27T15:22:24.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.877+0000 7fe8e97fa700 1 -- 172.21.15.133:0/1276607756 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc003df0 con 0x7fe8ec073550 2023-11-27T15:22:24.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.877+0000 7fe8f2318700 1 -- 172.21.15.133:0/1276607756 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e4009d00 con 0x7fe8ec073550 2023-11-27T15:22:24.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.878+0000 7fe8f2318700 1 -- 172.21.15.133:0/1276607756 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ec106870 con 0x7fe8ec073550 2023-11-27T15:22:24.885 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.880+0000 7fe8e97fa700 1 -- 172.21.15.133:0/1276607756 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fe8dc004000 con 0x7fe8ec073550 2023-11-27T15:22:24.886 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.881+0000 7fe8e97fa700 1 --2- 172.21.15.133:0/1276607756 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe8d40a1260 0x7fe8d40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:24.886 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.881+0000 7fe8e97fa700 1 -- 172.21.15.133:0/1276607756 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe8dc014070 con 0x7fe8ec073550 2023-11-27T15:22:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.881+0000 7fe8eb7fe700 1 --2- 172.21.15.133:0/1276607756 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe8d40a1260 0x7fe8d40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:24.887 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.882+0000 7fe8eb7fe700 1 --2- 172.21.15.133:0/1276607756 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe8d40a1260 0x7fe8d40a3710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009fd0 tx=0x7fe8e0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:24.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:24.885+0000 7fe8e97fa700 1 -- 172.21.15.133:0/1276607756 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe8dc0963b0 con 0x7fe8ec073550 2023-11-27T15:22:25.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:24 smithi133 ceph-mon[97860]: pgmap v333: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:22:25.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:24 smithi177 ceph-mon[131504]: pgmap v333: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:22:25.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:25.123+0000 7fe8f2318700 1 -- 172.21.15.133:0/1276607756 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe8ec02cbc0 con 0x7fe8ec073550 2023-11-27T15:22:25.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:25.124+0000 7fe8e97fa700 1 -- 172.21.15.133:0/1276607756 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2915 (secure 0 0 0) 0x7fe8dc0999d0 con 0x7fe8ec073550 2023-11-27T15:22:25.128 INFO:teuthology.orchestra.run.smithi133.stdout:e23 2023-11-27T15:22:25.128 INFO:teuthology.orchestra.run.smithi133.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-11-27T15:22:25.129 INFO:teuthology.orchestra.run.smithi133.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-27T15:22:25.129 INFO:teuthology.orchestra.run.smithi133.stdout:legacy client fscid: 1 2023-11-27T15:22:25.129 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:25.129 INFO:teuthology.orchestra.run.smithi133.stdout:Filesystem 'cephfs' (1) 2023-11-27T15:22:25.129 INFO:teuthology.orchestra.run.smithi133.stdout:fs_name cephfs 2023-11-27T15:22:25.129 INFO:teuthology.orchestra.run.smithi133.stdout:epoch 23 2023-11-27T15:22:25.129 INFO:teuthology.orchestra.run.smithi133.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-11-27T15:22:25.129 INFO:teuthology.orchestra.run.smithi133.stdout:created 2023-11-27T15:20:33.201023+0000 2023-11-27T15:22:25.130 INFO:teuthology.orchestra.run.smithi133.stdout:modified 2023-11-27T15:21:04.794526+0000 2023-11-27T15:22:25.130 INFO:teuthology.orchestra.run.smithi133.stdout:tableserver 0 2023-11-27T15:22:25.130 INFO:teuthology.orchestra.run.smithi133.stdout:root 0 2023-11-27T15:22:25.130 INFO:teuthology.orchestra.run.smithi133.stdout:session_timeout 300 2023-11-27T15:22:25.130 INFO:teuthology.orchestra.run.smithi133.stdout:session_autoclose 300 2023-11-27T15:22:25.130 INFO:teuthology.orchestra.run.smithi133.stdout:max_file_size 1099511627776 2023-11-27T15:22:25.130 INFO:teuthology.orchestra.run.smithi133.stdout:max_xattr_size 65536 2023-11-27T15:22:25.130 INFO:teuthology.orchestra.run.smithi133.stdout:required_client_features {} 2023-11-27T15:22:25.131 INFO:teuthology.orchestra.run.smithi133.stdout:last_failure 0 2023-11-27T15:22:25.131 INFO:teuthology.orchestra.run.smithi133.stdout:last_failure_osd_epoch 0 2023-11-27T15:22:25.131 INFO:teuthology.orchestra.run.smithi133.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-11-27T15:22:25.131 INFO:teuthology.orchestra.run.smithi133.stdout:max_mds 3 2023-11-27T15:22:25.131 INFO:teuthology.orchestra.run.smithi133.stdout:in 0,1,2 2023-11-27T15:22:25.131 INFO:teuthology.orchestra.run.smithi133.stdout:up {0=24560,1=24521,2=24520} 2023-11-27T15:22:25.131 INFO:teuthology.orchestra.run.smithi133.stdout:failed 2023-11-27T15:22:25.131 INFO:teuthology.orchestra.run.smithi133.stdout:damaged 2023-11-27T15:22:25.131 INFO:teuthology.orchestra.run.smithi133.stdout:stopped 2023-11-27T15:22:25.132 INFO:teuthology.orchestra.run.smithi133.stdout:data_pools [3,4] 2023-11-27T15:22:25.132 INFO:teuthology.orchestra.run.smithi133.stdout:metadata_pool 2 2023-11-27T15:22:25.132 INFO:teuthology.orchestra.run.smithi133.stdout:inline_data disabled 2023-11-27T15:22:25.132 INFO:teuthology.orchestra.run.smithi133.stdout:balancer 2023-11-27T15:22:25.132 INFO:teuthology.orchestra.run.smithi133.stdout:bal_rank_mask -1 2023-11-27T15:22:25.132 INFO:teuthology.orchestra.run.smithi133.stdout:standby_count_wanted 0 2023-11-27T15:22:25.132 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b{0:24560} state up:active seq 3 addr [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:22:25.132 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c{0:24505} state up:standby-replay seq 1 addr [v2:172.21.15.177:6836/3492215689,v1:172.21.15.177:6837/3492215689] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:22:25.132 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.h{1:24521} state up:active seq 13 addr [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:22:25.133 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.f{1:24496} state up:standby-replay seq 1 addr [v2:172.21.15.177:6834/3620466749,v1:172.21.15.177:6835/3620466749] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:22:25.133 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.i{2:24520} state up:active seq 13 addr [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:22:25.133 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.e{2:24491} state up:standby-replay seq 1 addr [v2:172.21.15.174:6834/637171798,v1:172.21.15.174:6835/637171798] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:22:25.133 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:25.133 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:25.133 INFO:teuthology.orchestra.run.smithi133.stdout:Standby daemons: 2023-11-27T15:22:25.133 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:25.133 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d{-1:14442} state up:standby seq 1 addr [v2:172.21.15.133:6836/2249018853,v1:172.21.15.133:6837/2249018853] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:22:25.133 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.g{-1:14493} state up:standby seq 1 addr [v2:172.21.15.133:6838/2381290259,v1:172.21.15.133:6839/2381290259] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:22:25.134 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a{-1:14505} state up:standby seq 1 addr [v2:172.21.15.133:6840/1704875895,v1:172.21.15.133:6841/1704875895] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:22:25.134 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.l{-1:24439} state up:standby seq 1 addr [v2:172.21.15.177:6832/1329176662,v1:172.21.15.177:6833/1329176662] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:22:25.134 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.k{-1:24455} state up:standby seq 1 addr [v2:172.21.15.174:6832/2946349291,v1:172.21.15.174:6833/2946349291] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:22:25.134 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.j{-1:24467} state up:standby seq 1 addr [v2:172.21.15.133:6834/1877742522,v1:172.21.15.133:6835/1877742522] compat {c=[1],r=[1],i=[7ff]}] 2023-11-27T15:22:25.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:25.126+0000 7fe8f2318700 1 -- 172.21.15.133:0/1276607756 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe8d40a1260 msgr2=0x7fe8d40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:25.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:25.126+0000 7fe8f2318700 1 --2- 172.21.15.133:0/1276607756 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe8d40a1260 0x7fe8d40a3710 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009fd0 tx=0x7fe8e0009260 comp rx=0 tx=0).stop 2023-11-27T15:22:25.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:25.126+0000 7fe8f2318700 1 -- 172.21.15.133:0/1276607756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8ec073550 msgr2=0x7fe8ec198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:25.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:25.126+0000 7fe8f2318700 1 --2- 172.21.15.133:0/1276607756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8ec073550 0x7fe8ec198330 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00eb60 tx=0x7fe8dc00ee70 comp rx=0 tx=0).stop 2023-11-27T15:22:25.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:25.126+0000 7fe8f2318700 1 -- 172.21.15.133:0/1276607756 shutdown_connections 2023-11-27T15:22:25.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:25.126+0000 7fe8f2318700 1 --2- 172.21.15.133:0/1276607756 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe8d40a1260 0x7fe8d40a3710 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:25.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:25.126+0000 7fe8f2318700 1 --2- 172.21.15.133:0/1276607756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8ec1081c0 0x7fe8ec19f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:25.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:25.126+0000 7fe8f2318700 1 --2- 172.21.15.133:0/1276607756 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8ec073550 0x7fe8ec198330 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:25.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:25.126+0000 7fe8f2318700 1 --2- 172.21.15.133:0/1276607756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe8ec074b20 0x7fe8ec198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:25.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:25.126+0000 7fe8f2318700 1 -- 172.21.15.133:0/1276607756 >> 172.21.15.133:0/1276607756 conn(0x7fe8ec0fb900 msgr2=0x7fe8ec0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:25.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:25.127+0000 7fe8f2318700 1 -- 172.21.15.133:0/1276607756 shutdown_connections 2023-11-27T15:22:25.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:25.127+0000 7fe8f2318700 1 -- 172.21.15.133:0/1276607756 wait complete. 2023-11-27T15:22:25.136 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 23 2023-11-27T15:22:25.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:24 smithi174 ceph-mon[131110]: pgmap v333: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:22:25.573 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- bash -c 'ceph osd dump' 2023-11-27T15:22:26.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:25 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1276607756' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-11-27T15:22:26.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:25 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1276607756' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-11-27T15:22:26.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:25 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1276607756' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-11-27T15:22:26.424 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:22:27.005 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:26 smithi133 ceph-mon[97860]: pgmap v334: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:22:27.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:26 smithi177 ceph-mon[131504]: pgmap v334: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:22:27.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:26 smithi174 ceph-mon[131110]: pgmap v334: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:22:27.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.915+0000 7f311b2ea700 1 -- 172.21.15.133:0/236509495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31141014f0 msgr2=0x7f3114101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:27.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.915+0000 7f311b2ea700 1 --2- 172.21.15.133:0/236509495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31141014f0 0x7f3114101940 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3104009990 tx=0x7f3104009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:27.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.915+0000 7f311b2ea700 1 -- 172.21.15.133:0/236509495 shutdown_connections 2023-11-27T15:22:27.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.915+0000 7f311b2ea700 1 --2- 172.21.15.133:0/236509495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3114101e80 0x7f3114108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:27.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.915+0000 7f311b2ea700 1 --2- 172.21.15.133:0/236509495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31141002f0 0x7f3114100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:27.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.915+0000 7f311b2ea700 1 --2- 172.21.15.133:0/236509495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31141014f0 0x7f3114101940 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:27.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.915+0000 7f311b2ea700 1 -- 172.21.15.133:0/236509495 >> 172.21.15.133:0/236509495 conn(0x7f31140fb900 msgr2=0x7f31140fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:27.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.915+0000 7f311b2ea700 1 -- 172.21.15.133:0/236509495 shutdown_connections 2023-11-27T15:22:27.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.915+0000 7f311b2ea700 1 -- 172.21.15.133:0/236509495 wait complete. 2023-11-27T15:22:27.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.916+0000 7f311b2ea700 1 Processor -- start 2023-11-27T15:22:27.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.916+0000 7f311b2ea700 1 -- start start 2023-11-27T15:22:27.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.916+0000 7f311b2ea700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31141002f0 0x7f3114198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:27.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.917+0000 7f311b2ea700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31141014f0 0x7f31141987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:27.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.917+0000 7f311b2ea700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3114101e80 0x7f311419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:27.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.917+0000 7f311b2ea700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311419fe00 con 0x7f3114101e80 2023-11-27T15:22:27.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.917+0000 7f311b2ea700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311419ff40 con 0x7f31141002f0 2023-11-27T15:22:27.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.917+0000 7f311b2ea700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31141a00d0 con 0x7f31141014f0 2023-11-27T15:22:27.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.917+0000 7f3119887700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3114101e80 0x7f311419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:27.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.917+0000 7f3119887700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3114101e80 0x7f311419f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54854/0 (socket says 172.21.15.133:54854) 2023-11-27T15:22:27.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.917+0000 7f3119887700 1 -- 172.21.15.133:0/797171061 learned_addr learned my addr 172.21.15.133:0/797171061 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:27.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.917+0000 7f3119086700 1 --2- 172.21.15.133:0/797171061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31141002f0 0x7f3114198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:27.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.917+0000 7f3118885700 1 --2- 172.21.15.133:0/797171061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31141014f0 0x7f31141987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:27.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.917+0000 7f3119887700 1 -- 172.21.15.133:0/797171061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31141014f0 msgr2=0x7f31141987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:27.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.917+0000 7f3119887700 1 --2- 172.21.15.133:0/797171061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31141014f0 0x7f31141987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:27.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.917+0000 7f3119887700 1 -- 172.21.15.133:0/797171061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31141002f0 msgr2=0x7f3114198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:27.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.917+0000 7f3119887700 1 --2- 172.21.15.133:0/797171061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31141002f0 0x7f3114198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:27.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.917+0000 7f3119887700 1 -- 172.21.15.133:0/797171061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31040095f0 con 0x7f3114101e80 2023-11-27T15:22:27.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.918+0000 7f3119887700 1 --2- 172.21.15.133:0/797171061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3114101e80 0x7f311419f830 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f310c00ef60 tx=0x7f310c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:27.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.918+0000 7f310a7fc700 1 -- 172.21.15.133:0/797171061 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c018070 con 0x7f3114101e80 2023-11-27T15:22:27.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.918+0000 7f310a7fc700 1 -- 172.21.15.133:0/797171061 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f310c00f040 con 0x7f3114101e80 2023-11-27T15:22:27.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.918+0000 7f311b2ea700 1 -- 172.21.15.133:0/797171061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f311410bc90 con 0x7f3114101e80 2023-11-27T15:22:27.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.918+0000 7f310a7fc700 1 -- 172.21.15.133:0/797171061 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c00ccc0 con 0x7f3114101e80 2023-11-27T15:22:27.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.918+0000 7f311b2ea700 1 -- 172.21.15.133:0/797171061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311410bb00 con 0x7f3114101e80 2023-11-27T15:22:27.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.919+0000 7f311b2ea700 1 -- 172.21.15.133:0/797171061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31141929f0 con 0x7f3114101e80 2023-11-27T15:22:27.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.921+0000 7f310a7fc700 1 -- 172.21.15.133:0/797171061 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f310c003a00 con 0x7f3114101e80 2023-11-27T15:22:27.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.922+0000 7f310a7fc700 1 --2- 172.21.15.133:0/797171061 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f31000a11e0 0x7f31000a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:27.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.922+0000 7f310a7fc700 1 -- 172.21.15.133:0/797171061 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f310c0cc8b0 con 0x7f3114101e80 2023-11-27T15:22:27.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.922+0000 7f3119086700 1 --2- 172.21.15.133:0/797171061 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f31000a11e0 0x7f31000a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:27.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.923+0000 7f3119086700 1 --2- 172.21.15.133:0/797171061 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f31000a11e0 0x7f31000a3690 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3114101350 tx=0x7f3110009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:27.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:27.926+0000 7f310a7fc700 1 -- 172.21.15.133:0/797171061 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f310c095b80 con 0x7f3114101e80 2023-11-27T15:22:28.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.124+0000 7f311b2ea700 1 -- 172.21.15.133:0/797171061 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f311402cfe0 con 0x7f3114101e80 2023-11-27T15:22:28.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.125+0000 7f310a7fc700 1 -- 172.21.15.133:0/797171061 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) v1 ==== 56+0+5672 (secure 0 0 0) 0x7f310c0991a0 con 0x7f3114101e80 2023-11-27T15:22:28.130 INFO:teuthology.orchestra.run.smithi133.stdout:epoch 92 2023-11-27T15:22:28.130 INFO:teuthology.orchestra.run.smithi133.stdout:fsid 5227da4c-8d37-11ee-95a2-87774f69a715 2023-11-27T15:22:28.130 INFO:teuthology.orchestra.run.smithi133.stdout:created 2023-11-27T15:13:01.701530+0000 2023-11-27T15:22:28.130 INFO:teuthology.orchestra.run.smithi133.stdout:modified 2023-11-27T15:22:02.807005+0000 2023-11-27T15:22:28.130 INFO:teuthology.orchestra.run.smithi133.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-11-27T15:22:28.130 INFO:teuthology.orchestra.run.smithi133.stdout:crush_version 27 2023-11-27T15:22:28.130 INFO:teuthology.orchestra.run.smithi133.stdout:full_ratio 0.95 2023-11-27T15:22:28.130 INFO:teuthology.orchestra.run.smithi133.stdout:backfillfull_ratio 0.9 2023-11-27T15:22:28.130 INFO:teuthology.orchestra.run.smithi133.stdout:nearfull_ratio 0.85 2023-11-27T15:22:28.131 INFO:teuthology.orchestra.run.smithi133.stdout:require_min_compat_client luminous 2023-11-27T15:22:28.131 INFO:teuthology.orchestra.run.smithi133.stdout:min_compat_client luminous 2023-11-27T15:22:28.131 INFO:teuthology.orchestra.run.smithi133.stdout:require_osd_release reef 2023-11-27T15:22:28.131 INFO:teuthology.orchestra.run.smithi133.stdout:stretch_mode_enabled false 2023-11-27T15:22:28.131 INFO:teuthology.orchestra.run.smithi133.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-11-27T15:22:28.131 INFO:teuthology.orchestra.run.smithi133.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-11-27T15:22:28.131 INFO:teuthology.orchestra.run.smithi133.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-11-27T15:22:28.131 INFO:teuthology.orchestra.run.smithi133.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-11-27T15:22:28.131 INFO:teuthology.orchestra.run.smithi133.stdout:max_osd 12 2023-11-27T15:22:28.132 INFO:teuthology.orchestra.run.smithi133.stdout:osd.0 up in weight 1 up_from 8 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6802/3850463495,v1:172.21.15.133:6803/3850463495] [v2:172.21.15.133:6804/3850463495,v1:172.21.15.133:6805/3850463495] exists,up 0ebdea23-8131-43b6-b3b1-06d1337bf063 2023-11-27T15:22:28.132 INFO:teuthology.orchestra.run.smithi133.stdout:osd.1 up in weight 1 up_from 13 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6800/3890663971,v1:172.21.15.174:6801/3890663971] [v2:172.21.15.174:6802/3890663971,v1:172.21.15.174:6803/3890663971] exists,up dc933082-b3e3-43d5-b110-c4d118df4418 2023-11-27T15:22:28.132 INFO:teuthology.orchestra.run.smithi133.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6800/2043769283,v1:172.21.15.177:6801/2043769283] [v2:172.21.15.177:6802/2043769283,v1:172.21.15.177:6803/2043769283] exists,up 4a8ba7ac-d927-4a9f-9079-105004d658a0 2023-11-27T15:22:28.132 INFO:teuthology.orchestra.run.smithi133.stdout:osd.3 up in weight 1 up_from 25 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6810/673797009,v1:172.21.15.133:6811/673797009] [v2:172.21.15.133:6812/673797009,v1:172.21.15.133:6813/673797009] exists,up 452fbca8-3b28-4258-a86b-1042888fe294 2023-11-27T15:22:28.132 INFO:teuthology.orchestra.run.smithi133.stdout:osd.4 up in weight 1 up_from 33 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6808/959536963,v1:172.21.15.174:6809/959536963] [v2:172.21.15.174:6810/959536963,v1:172.21.15.174:6811/959536963] exists,up dd04b180-bbcc-4611-9c42-936c87827fe7 2023-11-27T15:22:28.132 INFO:teuthology.orchestra.run.smithi133.stdout:osd.5 up in weight 1 up_from 38 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6808/3863044388,v1:172.21.15.177:6809/3863044388] [v2:172.21.15.177:6810/3863044388,v1:172.21.15.177:6811/3863044388] exists,up ec77b94b-3eb1-41a2-a319-fd446ce04816 2023-11-27T15:22:28.132 INFO:teuthology.orchestra.run.smithi133.stdout:osd.6 up in weight 1 up_from 46 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6818/3664128592,v1:172.21.15.133:6819/3664128592] [v2:172.21.15.133:6820/3664128592,v1:172.21.15.133:6821/3664128592] exists,up 2abbda56-2257-4a87-9f75-8491329c2df8 2023-11-27T15:22:28.132 INFO:teuthology.orchestra.run.smithi133.stdout:osd.7 up in weight 1 up_from 51 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6816/2275667196,v1:172.21.15.174:6817/2275667196] [v2:172.21.15.174:6818/2275667196,v1:172.21.15.174:6819/2275667196] exists,up 106a6b4b-99fc-44a8-a4a4-c635b25101d3 2023-11-27T15:22:28.132 INFO:teuthology.orchestra.run.smithi133.stdout:osd.8 up in weight 1 up_from 58 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6816/3783894564,v1:172.21.15.177:6817/3783894564] [v2:172.21.15.177:6818/3783894564,v1:172.21.15.177:6819/3783894564] exists,up d1541498-5236-447b-b194-d8da4226318e 2023-11-27T15:22:28.132 INFO:teuthology.orchestra.run.smithi133.stdout:osd.9 up in weight 1 up_from 66 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.133:6826/34943790,v1:172.21.15.133:6827/34943790] [v2:172.21.15.133:6828/34943790,v1:172.21.15.133:6829/34943790] exists,up 5e115ec1-93e3-41ea-a785-18b6c70b7b9c 2023-11-27T15:22:28.133 INFO:teuthology.orchestra.run.smithi133.stdout:osd.10 up in weight 1 up_from 71 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6824/880237758,v1:172.21.15.174:6825/880237758] [v2:172.21.15.174:6826/880237758,v1:172.21.15.174:6827/880237758] exists,up 342cb818-eaf2-4961-81aa-fd849bfd7be5 2023-11-27T15:22:28.133 INFO:teuthology.orchestra.run.smithi133.stdout:osd.11 up in weight 1 up_from 79 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.177:6824/1870203795,v1:172.21.15.177:6825/1870203795] [v2:172.21.15.177:6826/1870203795,v1:172.21.15.177:6827/1870203795] exists,up d9a72525-14d3-4ff6-8a06-abcd6af6396d 2023-11-27T15:22:28.133 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 2.a [2,11] 2023-11-27T15:22:28.133 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 2.10 [8,11] 2023-11-27T15:22:28.133 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 2.16 [2,4] 2023-11-27T15:22:28.133 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 3.0 [5,9] 2023-11-27T15:22:28.133 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 3.17 [4,6] 2023-11-27T15:22:28.133 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 3.1e [3,9] 2023-11-27T15:22:28.133 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 3.27 [4,8] 2023-11-27T15:22:28.133 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 3.34 [4,9] 2023-11-27T15:22:28.133 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 4.7 [5,8] 2023-11-27T15:22:28.134 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 4.c [9,10] 2023-11-27T15:22:28.134 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 4.e [9,11] 2023-11-27T15:22:28.134 INFO:teuthology.orchestra.run.smithi133.stdout:pg_upmap_items 4.f [9,8] 2023-11-27T15:22:28.134 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:6800/2374615220 expires 2023-11-28T15:14:01.628185+0000 2023-11-27T15:22:28.134 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/1594111147 expires 2023-11-28T15:14:01.628185+0000 2023-11-27T15:22:28.134 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:6801/3149426454 expires 2023-11-28T15:13:35.034149+0000 2023-11-27T15:22:28.134 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:6800/3149426454 expires 2023-11-28T15:13:35.034149+0000 2023-11-27T15:22:28.134 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/2898896137 expires 2023-11-28T15:14:01.628185+0000 2023-11-27T15:22:28.134 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/1816614695 expires 2023-11-28T15:13:35.034149+0000 2023-11-27T15:22:28.134 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/2624811615 expires 2023-11-28T15:13:35.034149+0000 2023-11-27T15:22:28.135 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:6801/2374615220 expires 2023-11-28T15:14:01.628185+0000 2023-11-27T15:22:28.135 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/4208867747 expires 2023-11-28T15:14:01.628185+0000 2023-11-27T15:22:28.135 INFO:teuthology.orchestra.run.smithi133.stdout:blocklist 172.21.15.133:0/4192331415 expires 2023-11-28T15:13:35.034149+0000 2023-11-27T15:22:28.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.128+0000 7f311b2ea700 1 -- 172.21.15.133:0/797171061 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f31000a11e0 msgr2=0x7f31000a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:28.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.128+0000 7f311b2ea700 1 --2- 172.21.15.133:0/797171061 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f31000a11e0 0x7f31000a3690 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3114101350 tx=0x7f3110009260 comp rx=0 tx=0).stop 2023-11-27T15:22:28.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.128+0000 7f311b2ea700 1 -- 172.21.15.133:0/797171061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3114101e80 msgr2=0x7f311419f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:28.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.128+0000 7f311b2ea700 1 --2- 172.21.15.133:0/797171061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3114101e80 0x7f311419f830 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f310c00ef60 tx=0x7f310c00ef90 comp rx=0 tx=0).stop 2023-11-27T15:22:28.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.128+0000 7f311b2ea700 1 -- 172.21.15.133:0/797171061 shutdown_connections 2023-11-27T15:22:28.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.128+0000 7f311b2ea700 1 --2- 172.21.15.133:0/797171061 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f31000a11e0 0x7f31000a3690 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:28.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.128+0000 7f311b2ea700 1 --2- 172.21.15.133:0/797171061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3114101e80 0x7f311419f830 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:28.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.128+0000 7f311b2ea700 1 --2- 172.21.15.133:0/797171061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31141002f0 0x7f3114198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:28.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.128+0000 7f311b2ea700 1 --2- 172.21.15.133:0/797171061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f31141014f0 0x7f31141987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:28.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.128+0000 7f311b2ea700 1 -- 172.21.15.133:0/797171061 >> 172.21.15.133:0/797171061 conn(0x7f31140fb900 msgr2=0x7f3114106e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:28.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.128+0000 7f311b2ea700 1 -- 172.21.15.133:0/797171061 shutdown_connections 2023-11-27T15:22:28.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.128+0000 7f311b2ea700 1 -- 172.21.15.133:0/797171061 wait complete. 2023-11-27T15:22:28.615 INFO:teuthology.run_tasks:Running task fs.ready... 2023-11-27T15:22:28.626 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:22:28.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.842+0000 7f073a7ff700 1 -- 172.21.15.133:0/891613300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0734101e30 msgr2=0x7f07341086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:28.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.842+0000 7f073a7ff700 1 --2- 172.21.15.133:0/891613300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0734101e30 0x7f07341086f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f072c009f90 tx=0x7f072c00b330 comp rx=0 tx=0).stop 2023-11-27T15:22:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.842+0000 7f073a7ff700 1 -- 172.21.15.133:0/891613300 shutdown_connections 2023-11-27T15:22:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.842+0000 7f073a7ff700 1 --2- 172.21.15.133:0/891613300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0734101e30 0x7f07341086f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.842+0000 7f073a7ff700 1 --2- 172.21.15.133:0/891613300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07341002b0 0x7f07341006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.842+0000 7f073a7ff700 1 --2- 172.21.15.133:0/891613300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07341014a0 0x7f07341018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.842+0000 7f073a7ff700 1 -- 172.21.15.133:0/891613300 >> 172.21.15.133:0/891613300 conn(0x7f07340fb890 msgr2=0x7f07340fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.842+0000 7f073a7ff700 1 -- 172.21.15.133:0/891613300 shutdown_connections 2023-11-27T15:22:28.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.842+0000 7f073a7ff700 1 -- 172.21.15.133:0/891613300 wait complete. 2023-11-27T15:22:28.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.844+0000 7f073a7ff700 1 Processor -- start 2023-11-27T15:22:28.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.844+0000 7f073a7ff700 1 -- start start 2023-11-27T15:22:28.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.844+0000 7f073a7ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07341002b0 0x7f07341982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:28.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.844+0000 7f073a7ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07341014a0 0x7f07341987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:28.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.844+0000 7f073a7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0734101e30 0x7f073419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:28.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.844+0000 7f073a7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073419fe30 con 0x7f0734101e30 2023-11-27T15:22:28.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.844+0000 7f073a7ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073419ff70 con 0x7f07341014a0 2023-11-27T15:22:28.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.844+0000 7f073a7ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07341a0100 con 0x7f07341002b0 2023-11-27T15:22:28.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.844+0000 7f0733fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07341002b0 0x7f07341982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:28.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.844+0000 7f0733fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07341002b0 0x7f07341982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:56722/0 (socket says 172.21.15.133:56722) 2023-11-27T15:22:28.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.844+0000 7f0733fff700 1 -- 172.21.15.133:0/4279378086 learned_addr learned my addr 172.21.15.133:0/4279378086 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:28.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.844+0000 7f0738d9c700 1 --2- 172.21.15.133:0/4279378086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0734101e30 0x7f073419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:28.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.844+0000 7f072bfff700 1 --2- 172.21.15.133:0/4279378086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07341014a0 0x7f07341987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:28.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.845+0000 7f0733fff700 1 -- 172.21.15.133:0/4279378086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07341014a0 msgr2=0x7f07341987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:28.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.845+0000 7f0733fff700 1 --2- 172.21.15.133:0/4279378086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07341014a0 0x7f07341987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:28.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.845+0000 7f0733fff700 1 -- 172.21.15.133:0/4279378086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0734101e30 msgr2=0x7f073419f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:28.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.845+0000 7f0733fff700 1 --2- 172.21.15.133:0/4279378086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0734101e30 0x7f073419f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:28.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.845+0000 7f0733fff700 1 -- 172.21.15.133:0/4279378086 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c009c40 con 0x7f07341002b0 2023-11-27T15:22:28.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.845+0000 7f0733fff700 1 --2- 172.21.15.133:0/4279378086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07341002b0 0x7f07341982b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f072400e9f0 tx=0x7f072400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:28.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.845+0000 7f0731ffb700 1 -- 172.21.15.133:0/4279378086 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0724004220 con 0x7f07341002b0 2023-11-27T15:22:28.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.845+0000 7f073a7ff700 1 -- 172.21.15.133:0/4279378086 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073410bc80 con 0x7f07341002b0 2023-11-27T15:22:28.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.845+0000 7f0731ffb700 1 -- 172.21.15.133:0/4279378086 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0724004380 con 0x7f07341002b0 2023-11-27T15:22:28.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.845+0000 7f0731ffb700 1 -- 172.21.15.133:0/4279378086 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0724003be0 con 0x7f07341002b0 2023-11-27T15:22:28.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.845+0000 7f073a7ff700 1 -- 172.21.15.133:0/4279378086 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073410baf0 con 0x7f07341002b0 2023-11-27T15:22:28.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.846+0000 7f073a7ff700 1 -- 172.21.15.133:0/4279378086 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07341a0900 con 0x7f07341002b0 2023-11-27T15:22:28.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.848+0000 7f0731ffb700 1 -- 172.21.15.133:0/4279378086 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f0724003d40 con 0x7f07341002b0 2023-11-27T15:22:28.853 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.850+0000 7f0731ffb700 1 --2- 172.21.15.133:0/4279378086 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f07200a11e0 0x7f07200a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:28.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.850+0000 7f0731ffb700 1 -- 172.21.15.133:0/4279378086 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f0724014070 con 0x7f07341002b0 2023-11-27T15:22:28.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.850+0000 7f072bfff700 1 --2- 172.21.15.133:0/4279378086 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f07200a11e0 0x7f07200a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:28.854 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.851+0000 7f072bfff700 1 --2- 172.21.15.133:0/4279378086 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f07200a11e0 0x7f07200a3690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f071c009c60 tx=0x7f071c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:28.856 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:28.852+0000 7f0731ffb700 1 -- 172.21.15.133:0/4279378086 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0724096850 con 0x7f07341002b0 2023-11-27T15:22:29.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:28 smithi133 ceph-mon[97860]: pgmap v335: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-11-27T15:22:29.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:28 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/797171061' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-11-27T15:22:29.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.051+0000 7f073a7ff700 1 -- 172.21.15.133:0/4279378086 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07341a02c0 con 0x7f07341002b0 2023-11-27T15:22:29.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.053+0000 7f0731ffb700 1 -- 172.21.15.133:0/4279378086 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24616 (secure 0 0 0) 0x7f0724099e70 con 0x7f07341002b0 2023-11-27T15:22:29.057 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:29.057 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":92,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:22:02.807005+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:22:29.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.055+0000 7f073a7ff700 1 -- 172.21.15.133:0/4279378086 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f07200a11e0 msgr2=0x7f07200a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:29.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.055+0000 7f073a7ff700 1 --2- 172.21.15.133:0/4279378086 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f07200a11e0 0x7f07200a3690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f071c009c60 tx=0x7f071c009260 comp rx=0 tx=0).stop 2023-11-27T15:22:29.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.055+0000 7f073a7ff700 1 -- 172.21.15.133:0/4279378086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07341002b0 msgr2=0x7f07341982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:29.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.055+0000 7f073a7ff700 1 --2- 172.21.15.133:0/4279378086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07341002b0 0x7f07341982b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f072400e9f0 tx=0x7f072400ed00 comp rx=0 tx=0).stop 2023-11-27T15:22:29.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.056+0000 7f073a7ff700 1 -- 172.21.15.133:0/4279378086 shutdown_connections 2023-11-27T15:22:29.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.056+0000 7f073a7ff700 1 --2- 172.21.15.133:0/4279378086 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f07200a11e0 0x7f07200a3690 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.056+0000 7f073a7ff700 1 --2- 172.21.15.133:0/4279378086 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0734101e30 0x7f073419f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.056+0000 7f073a7ff700 1 --2- 172.21.15.133:0/4279378086 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07341002b0 0x7f07341982b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.056+0000 7f073a7ff700 1 --2- 172.21.15.133:0/4279378086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07341014a0 0x7f07341987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.056+0000 7f073a7ff700 1 -- 172.21.15.133:0/4279378086 >> 172.21.15.133:0/4279378086 conn(0x7f07340fb890 msgr2=0x7f0734106d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:29.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.056+0000 7f073a7ff700 1 -- 172.21.15.133:0/4279378086 shutdown_connections 2023-11-27T15:22:29.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.056+0000 7f073a7ff700 1 -- 172.21.15.133:0/4279378086 wait complete. 2023-11-27T15:22:29.079 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:16:20.216150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:20:31.107146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:20:32.614089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-27T15:20:36.126040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-27T15:22:29.079 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:22:29.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:28 smithi177 ceph-mon[131504]: pgmap v335: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-11-27T15:22:29.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:28 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/797171061' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-11-27T15:22:29.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:28 smithi174 ceph-mon[131110]: pgmap v335: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-11-27T15:22:29.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:28 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/797171061' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-11-27T15:22:29.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.275+0000 7f43363ff700 1 -- 172.21.15.133:0/936725713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43301014c0 msgr2=0x7f4330101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:29.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.275+0000 7f43363ff700 1 --2- 172.21.15.133:0/936725713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43301014c0 0x7f4330101910 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4324009990 tx=0x7f4324009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.276+0000 7f43363ff700 1 -- 172.21.15.133:0/936725713 shutdown_connections 2023-11-27T15:22:29.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.276+0000 7f43363ff700 1 --2- 172.21.15.133:0/936725713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4330101e50 0x7f4330108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.276+0000 7f43363ff700 1 --2- 172.21.15.133:0/936725713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43301002d0 0x7f43301006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.276+0000 7f43363ff700 1 --2- 172.21.15.133:0/936725713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43301014c0 0x7f4330101910 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.276+0000 7f43363ff700 1 -- 172.21.15.133:0/936725713 >> 172.21.15.133:0/936725713 conn(0x7f43300fb8b0 msgr2=0x7f43300fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:29.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.276+0000 7f43363ff700 1 -- 172.21.15.133:0/936725713 shutdown_connections 2023-11-27T15:22:29.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.276+0000 7f43363ff700 1 -- 172.21.15.133:0/936725713 wait complete. 2023-11-27T15:22:29.282 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.277+0000 7f43363ff700 1 Processor -- start 2023-11-27T15:22:29.282 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.277+0000 7f43363ff700 1 -- start start 2023-11-27T15:22:29.282 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.277+0000 7f43363ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43301002d0 0x7f43301982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:29.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.277+0000 7f43363ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43301014c0 0x7f43301987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:29.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.277+0000 7f43363ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4330101e50 0x7f433019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:29.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.277+0000 7f43363ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433019fe20 con 0x7f43301014c0 2023-11-27T15:22:29.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.277+0000 7f43363ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433019ff60 con 0x7f4330101e50 2023-11-27T15:22:29.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.277+0000 7f43363ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43301a00f0 con 0x7f43301002d0 2023-11-27T15:22:29.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.277+0000 7f432f7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43301014c0 0x7f43301987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:29.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.277+0000 7f432f7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43301014c0 0x7f43301987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54890/0 (socket says 172.21.15.133:54890) 2023-11-27T15:22:29.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.277+0000 7f432f7fe700 1 -- 172.21.15.133:0/1912334128 learned_addr learned my addr 172.21.15.133:0/1912334128 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:29.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.278+0000 7f432ffff700 1 --2- 172.21.15.133:0/1912334128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43301002d0 0x7f43301982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:29.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.278+0000 7f433499c700 1 --2- 172.21.15.133:0/1912334128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4330101e50 0x7f433019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:29.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.278+0000 7f432f7fe700 1 -- 172.21.15.133:0/1912334128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43301002d0 msgr2=0x7f43301982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:29.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.278+0000 7f432f7fe700 1 --2- 172.21.15.133:0/1912334128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43301002d0 0x7f43301982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.278+0000 7f432f7fe700 1 -- 172.21.15.133:0/1912334128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4330101e50 msgr2=0x7f433019f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:29.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.278+0000 7f432f7fe700 1 --2- 172.21.15.133:0/1912334128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4330101e50 0x7f433019f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.278+0000 7f432f7fe700 1 -- 172.21.15.133:0/1912334128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43240095f0 con 0x7f43301014c0 2023-11-27T15:22:29.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.278+0000 7f432f7fe700 1 --2- 172.21.15.133:0/1912334128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43301014c0 0x7f43301987e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4324005910 tx=0x7f4324005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:29.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.278+0000 7f432d7fa700 1 -- 172.21.15.133:0/1912334128 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432401d070 con 0x7f43301014c0 2023-11-27T15:22:29.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.278+0000 7f432d7fa700 1 -- 172.21.15.133:0/1912334128 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4324003f60 con 0x7f43301014c0 2023-11-27T15:22:29.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.278+0000 7f432d7fa700 1 -- 172.21.15.133:0/1912334128 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4324022c30 con 0x7f43301014c0 2023-11-27T15:22:29.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.278+0000 7f43363ff700 1 -- 172.21.15.133:0/1912334128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f433010bc80 con 0x7f43301014c0 2023-11-27T15:22:29.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.279+0000 7f43363ff700 1 -- 172.21.15.133:0/1912334128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f433010baf0 con 0x7f43301014c0 2023-11-27T15:22:29.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.280+0000 7f43363ff700 1 -- 172.21.15.133:0/1912334128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4330067b00 con 0x7f43301014c0 2023-11-27T15:22:29.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.281+0000 7f432d7fa700 1 -- 172.21.15.133:0/1912334128 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f43240040d0 con 0x7f43301014c0 2023-11-27T15:22:29.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.283+0000 7f432d7fa700 1 --2- 172.21.15.133:0/1912334128 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f43180a11e0 0x7f43180a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:29.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.283+0000 7f432d7fa700 1 -- 172.21.15.133:0/1912334128 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f43240ce310 con 0x7f43301014c0 2023-11-27T15:22:29.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.283+0000 7f432ffff700 1 --2- 172.21.15.133:0/1912334128 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f43180a11e0 0x7f43180a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:29.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.283+0000 7f432ffff700 1 --2- 172.21.15.133:0/1912334128 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f43180a11e0 0x7f43180a3690 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4330101320 tx=0x7f4320007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:29.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.286+0000 7f432d7fa700 1 -- 172.21.15.133:0/1912334128 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f43240975e0 con 0x7f43301014c0 2023-11-27T15:22:29.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.484+0000 7f43363ff700 1 -- 172.21.15.133:0/1912334128 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f43301a0c10 con 0x7f43301014c0 2023-11-27T15:22:29.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.485+0000 7f432d7fa700 1 -- 172.21.15.133:0/1912334128 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f432409ac00 con 0x7f43301014c0 2023-11-27T15:22:29.489 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-11-27T15:22:29.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.487+0000 7f43363ff700 1 -- 172.21.15.133:0/1912334128 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f43180a11e0 msgr2=0x7f43180a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:29.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.487+0000 7f43363ff700 1 --2- 172.21.15.133:0/1912334128 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f43180a11e0 0x7f43180a3690 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4330101320 tx=0x7f4320007400 comp rx=0 tx=0).stop 2023-11-27T15:22:29.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.487+0000 7f43363ff700 1 -- 172.21.15.133:0/1912334128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43301014c0 msgr2=0x7f43301987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:29.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.487+0000 7f43363ff700 1 --2- 172.21.15.133:0/1912334128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43301014c0 0x7f43301987e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4324005910 tx=0x7f4324005740 comp rx=0 tx=0).stop 2023-11-27T15:22:29.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.488+0000 7f43363ff700 1 -- 172.21.15.133:0/1912334128 shutdown_connections 2023-11-27T15:22:29.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.488+0000 7f43363ff700 1 --2- 172.21.15.133:0/1912334128 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f43180a11e0 0x7f43180a3690 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.488+0000 7f43363ff700 1 --2- 172.21.15.133:0/1912334128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4330101e50 0x7f433019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.488+0000 7f43363ff700 1 --2- 172.21.15.133:0/1912334128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f43301002d0 0x7f43301982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.488+0000 7f43363ff700 1 --2- 172.21.15.133:0/1912334128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43301014c0 0x7f43301987e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.488+0000 7f43363ff700 1 -- 172.21.15.133:0/1912334128 >> 172.21.15.133:0/1912334128 conn(0x7f43300fb8b0 msgr2=0x7f43300fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:29.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.488+0000 7f43363ff700 1 -- 172.21.15.133:0/1912334128 shutdown_connections 2023-11-27T15:22:29.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.488+0000 7f43363ff700 1 -- 172.21.15.133:0/1912334128 wait complete. 2023-11-27T15:22:29.510 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:22:29.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.711+0000 7f616d5ff700 1 -- 172.21.15.133:0/3892888061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6168074b30 msgr2=0x7f6168072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:29.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.711+0000 7f616d5ff700 1 --2- 172.21.15.133:0/3892888061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6168074b30 0x7f6168072f80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6158009990 tx=0x7f6158009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.712+0000 7f616d5ff700 1 -- 172.21.15.133:0/3892888061 shutdown_connections 2023-11-27T15:22:29.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.712+0000 7f616d5ff700 1 --2- 172.21.15.133:0/3892888061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f616810c8a0 0x7f616810ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.712+0000 7f616d5ff700 1 --2- 172.21.15.133:0/3892888061 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6168074b30 0x7f6168072f80 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.712+0000 7f616d5ff700 1 --2- 172.21.15.133:0/3892888061 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6168073550 0x7f61680739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.712+0000 7f616d5ff700 1 -- 172.21.15.133:0/3892888061 >> 172.21.15.133:0/3892888061 conn(0x7f61680fb8b0 msgr2=0x7f61680fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:29.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.712+0000 7f616d5ff700 1 -- 172.21.15.133:0/3892888061 shutdown_connections 2023-11-27T15:22:29.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.712+0000 7f616d5ff700 1 -- 172.21.15.133:0/3892888061 wait complete. 2023-11-27T15:22:29.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.713+0000 7f616d5ff700 1 Processor -- start 2023-11-27T15:22:29.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.713+0000 7f616d5ff700 1 -- start start 2023-11-27T15:22:29.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.713+0000 7f616d5ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6168073550 0x7f616819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:29.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.713+0000 7f616d5ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6168074b30 0x7f616819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:29.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.713+0000 7f616d5ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f616810c8a0 0x7f61681a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:29.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.713+0000 7f616d5ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61681a4220 con 0x7f616810c8a0 2023-11-27T15:22:29.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.713+0000 7f616d5ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61681a4360 con 0x7f6168073550 2023-11-27T15:22:29.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.713+0000 7f616d5ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61681a44f0 con 0x7f6168074b30 2023-11-27T15:22:29.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.713+0000 7f6166ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6168073550 0x7f616819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:29.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.713+0000 7f6166ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6168073550 0x7f616819c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:57786/0 (socket says 172.21.15.133:57786) 2023-11-27T15:22:29.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.713+0000 7f6166ffd700 1 -- 172.21.15.133:0/698477418 learned_addr learned my addr 172.21.15.133:0/698477418 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:29.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.713+0000 7f61667fc700 1 --2- 172.21.15.133:0/698477418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6168074b30 0x7f616819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:29.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.713+0000 7f61677fe700 1 --2- 172.21.15.133:0/698477418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f616810c8a0 0x7f61681a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:29.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.714+0000 7f6166ffd700 1 -- 172.21.15.133:0/698477418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6168074b30 msgr2=0x7f616819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:29.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.714+0000 7f6166ffd700 1 --2- 172.21.15.133:0/698477418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6168074b30 0x7f616819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.714+0000 7f6166ffd700 1 -- 172.21.15.133:0/698477418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f616810c8a0 msgr2=0x7f61681a3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:29.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.714+0000 7f6166ffd700 1 --2- 172.21.15.133:0/698477418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f616810c8a0 0x7f61681a3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.714+0000 7f6166ffd700 1 -- 172.21.15.133:0/698477418 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61580095f0 con 0x7f6168073550 2023-11-27T15:22:29.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.714+0000 7f6166ffd700 1 --2- 172.21.15.133:0/698477418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6168073550 0x7f616819c6a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6158004710 tx=0x7f6158009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:29.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.714+0000 7f614ffff700 1 -- 172.21.15.133:0/698477418 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615801d070 con 0x7f6168073550 2023-11-27T15:22:29.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.714+0000 7f614ffff700 1 -- 172.21.15.133:0/698477418 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6158021410 con 0x7f6168073550 2023-11-27T15:22:29.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.714+0000 7f616d5ff700 1 -- 172.21.15.133:0/698477418 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6168112290 con 0x7f6168073550 2023-11-27T15:22:29.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.714+0000 7f614ffff700 1 -- 172.21.15.133:0/698477418 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6158017650 con 0x7f6168073550 2023-11-27T15:22:29.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.714+0000 7f616d5ff700 1 -- 172.21.15.133:0/698477418 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6168112100 con 0x7f6168073550 2023-11-27T15:22:29.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.715+0000 7f616d5ff700 1 -- 172.21.15.133:0/698477418 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61681023f0 con 0x7f6168073550 2023-11-27T15:22:29.721 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.717+0000 7f614ffff700 1 -- 172.21.15.133:0/698477418 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f6158021580 con 0x7f6168073550 2023-11-27T15:22:29.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.719+0000 7f614ffff700 1 --2- 172.21.15.133:0/698477418 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f61500a1260 0x7f61500a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:29.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.719+0000 7f614ffff700 1 -- 172.21.15.133:0/698477418 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f61580ce180 con 0x7f6168073550 2023-11-27T15:22:29.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.719+0000 7f61667fc700 1 --2- 172.21.15.133:0/698477418 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f61500a1260 0x7f61500a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:29.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.719+0000 7f61667fc700 1 --2- 172.21.15.133:0/698477418 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f61500a1260 0x7f61500a3710 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6168074850 tx=0x7f615c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:29.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.721+0000 7f614ffff700 1 -- 172.21.15.133:0/698477418 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6158097560 con 0x7f6168073550 2023-11-27T15:22:29.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:29 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/4279378086' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:29.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:29 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1912334128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:29.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.920+0000 7f616d5ff700 1 -- 172.21.15.133:0/698477418 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6168061700 con 0x7f6168073550 2023-11-27T15:22:29.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.921+0000 7f614ffff700 1 -- 172.21.15.133:0/698477418 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f615809ab80 con 0x7f6168073550 2023-11-27T15:22:29.925 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:22:29.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.923+0000 7f616d5ff700 1 -- 172.21.15.133:0/698477418 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f61500a1260 msgr2=0x7f61500a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:29.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.923+0000 7f616d5ff700 1 --2- 172.21.15.133:0/698477418 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f61500a1260 0x7f61500a3710 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6168074850 tx=0x7f615c009260 comp rx=0 tx=0).stop 2023-11-27T15:22:29.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.923+0000 7f616d5ff700 1 -- 172.21.15.133:0/698477418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6168073550 msgr2=0x7f616819c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:29.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.923+0000 7f616d5ff700 1 --2- 172.21.15.133:0/698477418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6168073550 0x7f616819c6a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6158004710 tx=0x7f6158009580 comp rx=0 tx=0).stop 2023-11-27T15:22:29.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.923+0000 7f616d5ff700 1 -- 172.21.15.133:0/698477418 shutdown_connections 2023-11-27T15:22:29.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.923+0000 7f616d5ff700 1 --2- 172.21.15.133:0/698477418 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f61500a1260 0x7f61500a3710 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.923+0000 7f616d5ff700 1 --2- 172.21.15.133:0/698477418 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f616810c8a0 0x7f61681a3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.924+0000 7f616d5ff700 1 --2- 172.21.15.133:0/698477418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6168073550 0x7f616819c6a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.924+0000 7f616d5ff700 1 --2- 172.21.15.133:0/698477418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6168074b30 0x7f616819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:29.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.924+0000 7f616d5ff700 1 -- 172.21.15.133:0/698477418 >> 172.21.15.133:0/698477418 conn(0x7f61680fb8b0 msgr2=0x7f61680fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:29.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.924+0000 7f616d5ff700 1 -- 172.21.15.133:0/698477418 shutdown_connections 2023-11-27T15:22:29.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:29.924+0000 7f616d5ff700 1 -- 172.21.15.133:0/698477418 wait complete. 2023-11-27T15:22:29.946 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:22:30.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:29 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/4279378086' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:30.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:29 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1912334128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:30.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.141+0000 7f0f1b7ff700 1 -- 172.21.15.133:0/4058856540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f14073550 msgr2=0x7f0f140739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:30.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.141+0000 7f0f1b7ff700 1 --2- 172.21.15.133:0/4058856540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f14073550 0x7f0f140739c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009990 tx=0x7f0f04009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:30.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.141+0000 7f0f1b7ff700 1 -- 172.21.15.133:0/4058856540 shutdown_connections 2023-11-27T15:22:30.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.141+0000 7f0f1b7ff700 1 --2- 172.21.15.133:0/4058856540 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f1410c890 0x7f0f1410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:30.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.141+0000 7f0f1b7ff700 1 --2- 172.21.15.133:0/4058856540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f14074b30 0x7f0f14072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:30.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.141+0000 7f0f1b7ff700 1 --2- 172.21.15.133:0/4058856540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f14073550 0x7f0f140739c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:30.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.141+0000 7f0f1b7ff700 1 -- 172.21.15.133:0/4058856540 >> 172.21.15.133:0/4058856540 conn(0x7f0f140fb8a0 msgr2=0x7f0f140fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:30.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.141+0000 7f0f1b7ff700 1 -- 172.21.15.133:0/4058856540 shutdown_connections 2023-11-27T15:22:30.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.141+0000 7f0f1b7ff700 1 -- 172.21.15.133:0/4058856540 wait complete. 2023-11-27T15:22:30.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.142+0000 7f0f1b7ff700 1 Processor -- start 2023-11-27T15:22:30.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.142+0000 7f0f1b7ff700 1 -- start start 2023-11-27T15:22:30.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.143+0000 7f0f1b7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f14073550 0x7f0f1419c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:30.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.143+0000 7f0f1b7ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f14074b30 0x7f0f1419cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:30.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.143+0000 7f0f1b7ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f1410c890 0x7f0f141a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:30.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.143+0000 7f0f1b7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f141a4230 con 0x7f0f14073550 2023-11-27T15:22:30.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.143+0000 7f0f1b7ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f141a4370 con 0x7f0f1410c890 2023-11-27T15:22:30.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.143+0000 7f0f1b7ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f141a4500 con 0x7f0f14074b30 2023-11-27T15:22:30.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.143+0000 7f0f1959b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f14073550 0x7f0f1419c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:30.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.143+0000 7f0f1959b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f14073550 0x7f0f1419c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54930/0 (socket says 172.21.15.133:54930) 2023-11-27T15:22:30.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.143+0000 7f0f1959b700 1 -- 172.21.15.133:0/2922468656 learned_addr learned my addr 172.21.15.133:0/2922468656 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:30.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.143+0000 7f0f18d9a700 1 --2- 172.21.15.133:0/2922468656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f14074b30 0x7f0f1419cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:30.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.143+0000 7f0f19d9c700 1 --2- 172.21.15.133:0/2922468656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f1410c890 0x7f0f141a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:30.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.143+0000 7f0f1959b700 1 -- 172.21.15.133:0/2922468656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f14074b30 msgr2=0x7f0f1419cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:30.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.143+0000 7f0f1959b700 1 --2- 172.21.15.133:0/2922468656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f14074b30 0x7f0f1419cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:30.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.143+0000 7f0f1959b700 1 -- 172.21.15.133:0/2922468656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f1410c890 msgr2=0x7f0f141a3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:30.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.143+0000 7f0f1959b700 1 --2- 172.21.15.133:0/2922468656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f1410c890 0x7f0f141a3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:30.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.143+0000 7f0f1959b700 1 -- 172.21.15.133:0/2922468656 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f040095f0 con 0x7f0f14073550 2023-11-27T15:22:30.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.143+0000 7f0f19d9c700 1 --2- 172.21.15.133:0/2922468656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f1410c890 0x7f0f141a3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:22:30.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.144+0000 7f0f1959b700 1 --2- 172.21.15.133:0/2922468656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f14073550 0x7f0f1419c6b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000e970 tx=0x7f0f1000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:30.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.144+0000 7f0f0a7fc700 1 -- 172.21.15.133:0/2922468656 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10009790 con 0x7f0f14073550 2023-11-27T15:22:30.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.144+0000 7f0f1b7ff700 1 -- 172.21.15.133:0/2922468656 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f141122a0 con 0x7f0f14073550 2023-11-27T15:22:30.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.144+0000 7f0f0a7fc700 1 -- 172.21.15.133:0/2922468656 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f1000eec0 con 0x7f0f14073550 2023-11-27T15:22:30.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.144+0000 7f0f0a7fc700 1 -- 172.21.15.133:0/2922468656 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f10005590 con 0x7f0f14073550 2023-11-27T15:22:30.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.144+0000 7f0f1b7ff700 1 -- 172.21.15.133:0/2922468656 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f14112110 con 0x7f0f14073550 2023-11-27T15:22:30.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.145+0000 7f0f1b7ff700 1 -- 172.21.15.133:0/2922468656 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f141023e0 con 0x7f0f14073550 2023-11-27T15:22:30.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.147+0000 7f0f0a7fc700 1 -- 172.21.15.133:0/2922468656 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f0f1000cbd0 con 0x7f0f14073550 2023-11-27T15:22:30.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.148+0000 7f0f0a7fc700 1 --2- 172.21.15.133:0/2922468656 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0f000a11e0 0x7f0f000a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:30.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.148+0000 7f0f0a7fc700 1 -- 172.21.15.133:0/2922468656 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f0f10014070 con 0x7f0f14073550 2023-11-27T15:22:30.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.149+0000 7f0f18d9a700 1 --2- 172.21.15.133:0/2922468656 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0f000a11e0 0x7f0f000a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:30.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.149+0000 7f0f18d9a700 1 --2- 172.21.15.133:0/2922468656 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0f000a11e0 0x7f0f000a3690 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400b2e0 tx=0x7f0f04005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:30.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:29 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/4279378086' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:30.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:29 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1912334128' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:30.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.150+0000 7f0f0a7fc700 1 -- 172.21.15.133:0/2922468656 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0f100964b0 con 0x7f0f14073550 2023-11-27T15:22:30.355 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.352+0000 7f0f1b7ff700 1 -- 172.21.15.133:0/2922468656 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f14067b00 con 0x7f0f14073550 2023-11-27T15:22:30.356 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.353+0000 7f0f0a7fc700 1 -- 172.21.15.133:0/2922468656 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0f10004ef0 con 0x7f0f14073550 2023-11-27T15:22:30.356 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-11-27T15:22:30.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.355+0000 7f0f1b7ff700 1 -- 172.21.15.133:0/2922468656 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0f000a11e0 msgr2=0x7f0f000a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:30.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.355+0000 7f0f1b7ff700 1 --2- 172.21.15.133:0/2922468656 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0f000a11e0 0x7f0f000a3690 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400b2e0 tx=0x7f0f04005fb0 comp rx=0 tx=0).stop 2023-11-27T15:22:30.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.355+0000 7f0f1b7ff700 1 -- 172.21.15.133:0/2922468656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f14073550 msgr2=0x7f0f1419c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:30.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.355+0000 7f0f1b7ff700 1 --2- 172.21.15.133:0/2922468656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f14073550 0x7f0f1419c6b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0f1000e970 tx=0x7f0f1000ec80 comp rx=0 tx=0).stop 2023-11-27T15:22:30.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.355+0000 7f0f1b7ff700 1 -- 172.21.15.133:0/2922468656 shutdown_connections 2023-11-27T15:22:30.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.355+0000 7f0f1b7ff700 1 --2- 172.21.15.133:0/2922468656 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0f000a11e0 0x7f0f000a3690 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:30.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.355+0000 7f0f1b7ff700 1 --2- 172.21.15.133:0/2922468656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f1410c890 0x7f0f141a3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:30.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.355+0000 7f0f1b7ff700 1 --2- 172.21.15.133:0/2922468656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f14073550 0x7f0f1419c6b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:30.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.355+0000 7f0f1b7ff700 1 --2- 172.21.15.133:0/2922468656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0f14074b30 0x7f0f1419cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:30.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.355+0000 7f0f1b7ff700 1 -- 172.21.15.133:0/2922468656 >> 172.21.15.133:0/2922468656 conn(0x7f0f140fb8a0 msgr2=0x7f0f140fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:30.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.355+0000 7f0f1b7ff700 1 -- 172.21.15.133:0/2922468656 shutdown_connections 2023-11-27T15:22:30.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.355+0000 7f0f1b7ff700 1 -- 172.21.15.133:0/2922468656 wait complete. 2023-11-27T15:22:30.378 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-27T15:22:30.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.573+0000 7f99805ff700 1 -- 172.21.15.133:0/2719874985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99781014b0 msgr2=0x7f9978101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:30.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.573+0000 7f99805ff700 1 --2- 172.21.15.133:0/2719874985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99781014b0 0x7f9978101900 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f99680098f0 tx=0x7f9968009c00 comp rx=0 tx=0).stop 2023-11-27T15:22:30.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.573+0000 7f99805ff700 1 -- 172.21.15.133:0/2719874985 shutdown_connections 2023-11-27T15:22:30.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.573+0000 7f99805ff700 1 --2- 172.21.15.133:0/2719874985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9978101e40 0x7f9978108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:30.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.573+0000 7f99805ff700 1 --2- 172.21.15.133:0/2719874985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99781002c0 0x7f99781006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:30.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.573+0000 7f99805ff700 1 --2- 172.21.15.133:0/2719874985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99781014b0 0x7f9978101900 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:30.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.573+0000 7f99805ff700 1 -- 172.21.15.133:0/2719874985 >> 172.21.15.133:0/2719874985 conn(0x7f99780fb8a0 msgr2=0x7f99780fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:30.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.573+0000 7f99805ff700 1 -- 172.21.15.133:0/2719874985 shutdown_connections 2023-11-27T15:22:30.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.573+0000 7f99805ff700 1 -- 172.21.15.133:0/2719874985 wait complete. 2023-11-27T15:22:30.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.574+0000 7f99805ff700 1 Processor -- start 2023-11-27T15:22:30.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.574+0000 7f99805ff700 1 -- start start 2023-11-27T15:22:30.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.575+0000 7f99805ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99781002c0 0x7f9978198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:30.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.575+0000 7f99805ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99781014b0 0x7f9978198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:30.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.575+0000 7f99805ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9978101e40 0x7f997819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:30.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.575+0000 7f99805ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997819fdd0 con 0x7f99781002c0 2023-11-27T15:22:30.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.575+0000 7f99805ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997819ff10 con 0x7f9978101e40 2023-11-27T15:22:30.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.575+0000 7f99805ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99781a00a0 con 0x7f99781014b0 2023-11-27T15:22:30.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.575+0000 7f997e39b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99781002c0 0x7f9978198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:30.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.575+0000 7f997db9a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99781014b0 0x7f9978198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:30.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.575+0000 7f997eb9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9978101e40 0x7f997819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:30.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.575+0000 7f997db9a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99781014b0 0x7f9978198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:56788/0 (socket says 172.21.15.133:56788) 2023-11-27T15:22:30.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.575+0000 7f997db9a700 1 -- 172.21.15.133:0/1750377300 learned_addr learned my addr 172.21.15.133:0/1750377300 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:30.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.575+0000 7f997e39b700 1 --2- 172.21.15.133:0/1750377300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99781002c0 0x7f9978198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54944/0 (socket says 172.21.15.133:54944) 2023-11-27T15:22:30.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.575+0000 7f997e39b700 1 -- 172.21.15.133:0/1750377300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99781014b0 msgr2=0x7f9978198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:30.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.575+0000 7f997e39b700 1 --2- 172.21.15.133:0/1750377300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99781014b0 0x7f9978198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:30.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.575+0000 7f997e39b700 1 -- 172.21.15.133:0/1750377300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9978101e40 msgr2=0x7f997819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:30.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.575+0000 7f997e39b700 1 --2- 172.21.15.133:0/1750377300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9978101e40 0x7f997819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:30.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.575+0000 7f997e39b700 1 -- 172.21.15.133:0/1750377300 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99700095f0 con 0x7f99781002c0 2023-11-27T15:22:30.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.575+0000 7f997eb9c700 1 --2- 172.21.15.133:0/1750377300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9978101e40 0x7f997819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:30.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.575+0000 7f997db9a700 1 --2- 172.21.15.133:0/1750377300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99781014b0 0x7f9978198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:30.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.576+0000 7f997e39b700 1 --2- 172.21.15.133:0/1750377300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99781002c0 0x7f9978198250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f997400eee0 tx=0x7f997400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:30.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.576+0000 7f996f7fe700 1 -- 172.21.15.133:0/1750377300 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974018070 con 0x7f99781002c0 2023-11-27T15:22:30.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.576+0000 7f996f7fe700 1 -- 172.21.15.133:0/1750377300 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f997400f040 con 0x7f99781002c0 2023-11-27T15:22:30.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.576+0000 7f99805ff700 1 -- 172.21.15.133:0/1750377300 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9968009520 con 0x7f99781002c0 2023-11-27T15:22:30.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.576+0000 7f996f7fe700 1 -- 172.21.15.133:0/1750377300 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9974004280 con 0x7f99781002c0 2023-11-27T15:22:30.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.576+0000 7f99805ff700 1 -- 172.21.15.133:0/1750377300 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997810bbd0 con 0x7f99781002c0 2023-11-27T15:22:30.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.577+0000 7f99805ff700 1 -- 172.21.15.133:0/1750377300 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9978067b00 con 0x7f99781002c0 2023-11-27T15:22:30.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.579+0000 7f996f7fe700 1 -- 172.21.15.133:0/1750377300 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f9974003a00 con 0x7f99781002c0 2023-11-27T15:22:30.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.580+0000 7f996f7fe700 1 --2- 172.21.15.133:0/1750377300 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f99640a1260 0x7f99640a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:30.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.580+0000 7f996f7fe700 1 -- 172.21.15.133:0/1750377300 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f99740cccf0 con 0x7f99781002c0 2023-11-27T15:22:30.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.581+0000 7f997db9a700 1 --2- 172.21.15.133:0/1750377300 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f99640a1260 0x7f99640a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:30.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.581+0000 7f997db9a700 1 --2- 172.21.15.133:0/1750377300 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f99640a1260 0x7f99640a3710 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f996800c010 tx=0x7f9968019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:30.587 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.584+0000 7f996f7fe700 1 -- 172.21.15.133:0/1750377300 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9974095fc0 con 0x7f99781002c0 2023-11-27T15:22:30.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.782+0000 7f99805ff700 1 -- 172.21.15.133:0/1750377300 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f99781a0ba0 con 0x7f99781002c0 2023-11-27T15:22:30.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.783+0000 7f996f7fe700 1 -- 172.21.15.133:0/1750377300 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f99740995e0 con 0x7f99781002c0 2023-11-27T15:22:30.787 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:22:30.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.785+0000 7f99805ff700 1 -- 172.21.15.133:0/1750377300 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f99640a1260 msgr2=0x7f99640a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:30.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.785+0000 7f99805ff700 1 --2- 172.21.15.133:0/1750377300 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f99640a1260 0x7f99640a3710 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f996800c010 tx=0x7f9968019040 comp rx=0 tx=0).stop 2023-11-27T15:22:30.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.785+0000 7f99805ff700 1 -- 172.21.15.133:0/1750377300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99781002c0 msgr2=0x7f9978198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:30.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.785+0000 7f99805ff700 1 --2- 172.21.15.133:0/1750377300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99781002c0 0x7f9978198250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f997400eee0 tx=0x7f997400c3a0 comp rx=0 tx=0).stop 2023-11-27T15:22:30.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.785+0000 7f99805ff700 1 -- 172.21.15.133:0/1750377300 shutdown_connections 2023-11-27T15:22:30.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.785+0000 7f99805ff700 1 --2- 172.21.15.133:0/1750377300 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f99640a1260 0x7f99640a3710 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:30.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.785+0000 7f99805ff700 1 --2- 172.21.15.133:0/1750377300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9978101e40 0x7f997819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:30.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.785+0000 7f99805ff700 1 --2- 172.21.15.133:0/1750377300 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99781002c0 0x7f9978198250 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:30.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.785+0000 7f99805ff700 1 --2- 172.21.15.133:0/1750377300 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f99781014b0 0x7f9978198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:30.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.785+0000 7f99805ff700 1 -- 172.21.15.133:0/1750377300 >> 172.21.15.133:0/1750377300 conn(0x7f99780fb8a0 msgr2=0x7f99780fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:30.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.785+0000 7f99805ff700 1 -- 172.21.15.133:0/1750377300 shutdown_connections 2023-11-27T15:22:30.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:30.785+0000 7f99805ff700 1 -- 172.21.15.133:0/1750377300 wait complete. 2023-11-27T15:22:30.808 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:22:31.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.004+0000 7fbda91ff700 1 -- 172.21.15.133:0/344020648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbda4101e30 msgr2=0x7fbda41086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:31.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.004+0000 7fbda91ff700 1 --2- 172.21.15.133:0/344020648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbda4101e30 0x7fbda41086f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00b0a0 tx=0x7fbd9c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.004+0000 7fbda91ff700 1 -- 172.21.15.133:0/344020648 shutdown_connections 2023-11-27T15:22:31.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.004+0000 7fbda91ff700 1 --2- 172.21.15.133:0/344020648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbda4101e30 0x7fbda41086f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.004+0000 7fbda91ff700 1 --2- 172.21.15.133:0/344020648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbda41002b0 0x7fbda41006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.004+0000 7fbda91ff700 1 --2- 172.21.15.133:0/344020648 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbda41014a0 0x7fbda41018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.004+0000 7fbda91ff700 1 -- 172.21.15.133:0/344020648 >> 172.21.15.133:0/344020648 conn(0x7fbda40fb890 msgr2=0x7fbda40fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:31.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.004+0000 7fbda91ff700 1 -- 172.21.15.133:0/344020648 shutdown_connections 2023-11-27T15:22:31.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.004+0000 7fbda91ff700 1 -- 172.21.15.133:0/344020648 wait complete. 2023-11-27T15:22:31.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.005+0000 7fbda91ff700 1 Processor -- start 2023-11-27T15:22:31.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.005+0000 7fbda91ff700 1 -- start start 2023-11-27T15:22:31.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.006+0000 7fbda91ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbda41002b0 0x7fbda4196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:31.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.006+0000 7fbda91ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbda41014a0 0x7fbda41965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:31.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.006+0000 7fbda91ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbda4101e30 0x7fbda419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:31.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.006+0000 7fbda91ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda419dbf0 con 0x7fbda41002b0 2023-11-27T15:22:31.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.006+0000 7fbda91ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda419dd30 con 0x7fbda4101e30 2023-11-27T15:22:31.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.006+0000 7fbda91ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda419dec0 con 0x7fbda41014a0 2023-11-27T15:22:31.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.006+0000 7fbda2d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbda41002b0 0x7fbda4196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:31.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.006+0000 7fbda2d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbda41002b0 0x7fbda4196070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54972/0 (socket says 172.21.15.133:54972) 2023-11-27T15:22:31.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.006+0000 7fbda2d9d700 1 -- 172.21.15.133:0/3642978812 learned_addr learned my addr 172.21.15.133:0/3642978812 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:31.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.006+0000 7fbd9bfff700 1 --2- 172.21.15.133:0/3642978812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbda41014a0 0x7fbda41965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:31.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.006+0000 7fbda359e700 1 --2- 172.21.15.133:0/3642978812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbda4101e30 0x7fbda419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:31.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.006+0000 7fbda2d9d700 1 -- 172.21.15.133:0/3642978812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbda41014a0 msgr2=0x7fbda41965b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:31.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.006+0000 7fbda2d9d700 1 --2- 172.21.15.133:0/3642978812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbda41014a0 0x7fbda41965b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.006+0000 7fbda2d9d700 1 -- 172.21.15.133:0/3642978812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbda4101e30 msgr2=0x7fbda419d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:31.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.006+0000 7fbda2d9d700 1 --2- 172.21.15.133:0/3642978812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbda4101e30 0x7fbda419d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.006+0000 7fbda2d9d700 1 -- 172.21.15.133:0/3642978812 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c009d00 con 0x7fbda41002b0 2023-11-27T15:22:31.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.007+0000 7fbda2d9d700 1 --2- 172.21.15.133:0/3642978812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbda41002b0 0x7fbda4196070 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400e970 tx=0x7fbd9400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:31.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.007+0000 7fbda0d99700 1 -- 172.21.15.133:0/3642978812 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd94009790 con 0x7fbda41002b0 2023-11-27T15:22:31.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.007+0000 7fbda0d99700 1 -- 172.21.15.133:0/3642978812 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd9400c7e0 con 0x7fbda41002b0 2023-11-27T15:22:31.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.007+0000 7fbda0d99700 1 -- 172.21.15.133:0/3642978812 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd94003c70 con 0x7fbda41002b0 2023-11-27T15:22:31.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.007+0000 7fbda91ff700 1 -- 172.21.15.133:0/3642978812 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda410bc40 con 0x7fbda41002b0 2023-11-27T15:22:31.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.007+0000 7fbda91ff700 1 -- 172.21.15.133:0/3642978812 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda410bab0 con 0x7fbda41002b0 2023-11-27T15:22:31.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.008+0000 7fbda91ff700 1 -- 172.21.15.133:0/3642978812 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda41077f0 con 0x7fbda41002b0 2023-11-27T15:22:31.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.010+0000 7fbda0d99700 1 -- 172.21.15.133:0/3642978812 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fbd9400cc50 con 0x7fbda41002b0 2023-11-27T15:22:31.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.011+0000 7fbda0d99700 1 --2- 172.21.15.133:0/3642978812 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbd900a11e0 0x7fbd900a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:31.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.012+0000 7fbda0d99700 1 -- 172.21.15.133:0/3642978812 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbd94014070 con 0x7fbda41002b0 2023-11-27T15:22:31.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.012+0000 7fbd9bfff700 1 --2- 172.21.15.133:0/3642978812 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbd900a11e0 0x7fbd900a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:31.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.012+0000 7fbd9bfff700 1 --2- 172.21.15.133:0/3642978812 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbd900a11e0 0x7fbd900a3690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009c60 tx=0x7fbd8c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:31.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.015+0000 7fbda0d99700 1 -- 172.21.15.133:0/3642978812 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbd94096220 con 0x7fbda41002b0 2023-11-27T15:22:31.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:30 smithi133 ceph-mon[97860]: pgmap v336: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:22:31.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:30 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/698477418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:22:31.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:30 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2922468656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:22:31.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:30 smithi177 ceph-mon[131504]: pgmap v336: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:22:31.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:30 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/698477418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:22:31.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:30 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2922468656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:22:31.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:30 smithi174 ceph-mon[131110]: pgmap v336: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-11-27T15:22:31.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:30 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/698477418' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:22:31.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:30 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2922468656' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:22:31.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.258+0000 7fbda91ff700 1 -- 172.21.15.133:0/3642978812 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbda402cfe0 con 0x7fbda41002b0 2023-11-27T15:22:31.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.259+0000 7fbda0d99700 1 -- 172.21.15.133:0/3642978812 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10829 (secure 0 0 0) 0x7fbd94003dd0 con 0x7fbda41002b0 2023-11-27T15:22:31.263 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:31.264 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14493,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2381290259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2381290259},{"type":"v1","addr":"172.21.15.133:6839","nonce":2381290259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1704875895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1704875895},{"type":"v1","addr":"172.21.15.133:6841","nonce":1704875895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:20:33.201023+0000","modified":"2023-11-27T15:21:04.794526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24521,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3492215689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3492215689},{"type":"v1","addr":"172.21.15.177:6837","nonce":3492215689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6839/2781994447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2781994447},{"type":"v1","addr":"172.21.15.174:6839","nonce":2781994447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:22:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.261+0000 7fbda91ff700 1 -- 172.21.15.133:0/3642978812 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbd900a11e0 msgr2=0x7fbd900a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.261+0000 7fbda91ff700 1 --2- 172.21.15.133:0/3642978812 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbd900a11e0 0x7fbd900a3690 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009c60 tx=0x7fbd8c009260 comp rx=0 tx=0).stop 2023-11-27T15:22:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.262+0000 7fbda91ff700 1 -- 172.21.15.133:0/3642978812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbda41002b0 msgr2=0x7fbda4196070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.262+0000 7fbda91ff700 1 --2- 172.21.15.133:0/3642978812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbda41002b0 0x7fbda4196070 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400e970 tx=0x7fbd9400ec80 comp rx=0 tx=0).stop 2023-11-27T15:22:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.262+0000 7fbda91ff700 1 -- 172.21.15.133:0/3642978812 shutdown_connections 2023-11-27T15:22:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.262+0000 7fbda91ff700 1 --2- 172.21.15.133:0/3642978812 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbd900a11e0 0x7fbd900a3690 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.262+0000 7fbda91ff700 1 --2- 172.21.15.133:0/3642978812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbda4101e30 0x7fbda419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.262+0000 7fbda91ff700 1 --2- 172.21.15.133:0/3642978812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbda41002b0 0x7fbda4196070 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.262+0000 7fbda91ff700 1 --2- 172.21.15.133:0/3642978812 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbda41014a0 0x7fbda41965b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.262+0000 7fbda91ff700 1 -- 172.21.15.133:0/3642978812 >> 172.21.15.133:0/3642978812 conn(0x7fbda40fb890 msgr2=0x7fbda4104930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.262+0000 7fbda91ff700 1 -- 172.21.15.133:0/3642978812 shutdown_connections 2023-11-27T15:22:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.262+0000 7fbda91ff700 1 -- 172.21.15.133:0/3642978812 wait complete. 2023-11-27T15:22:31.267 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 23 2023-11-27T15:22:31.285 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:22:31.483 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.479+0000 7fea951ff700 1 -- 172.21.15.133:0/2900028374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea901002b0 msgr2=0x7fea901006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:31.483 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.479+0000 7fea951ff700 1 --2- 172.21.15.133:0/2900028374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea901002b0 0x7fea901006c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fea800098f0 tx=0x7fea80009c00 comp rx=0 tx=0).stop 2023-11-27T15:22:31.483 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.479+0000 7fea951ff700 1 -- 172.21.15.133:0/2900028374 shutdown_connections 2023-11-27T15:22:31.484 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.479+0000 7fea951ff700 1 --2- 172.21.15.133:0/2900028374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea90101e30 0x7fea901086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.484 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.479+0000 7fea951ff700 1 --2- 172.21.15.133:0/2900028374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea901002b0 0x7fea901006c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.484 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.479+0000 7fea951ff700 1 --2- 172.21.15.133:0/2900028374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea901014a0 0x7fea901018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.484 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.479+0000 7fea951ff700 1 -- 172.21.15.133:0/2900028374 >> 172.21.15.133:0/2900028374 conn(0x7fea900fb890 msgr2=0x7fea900fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:31.484 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.479+0000 7fea951ff700 1 -- 172.21.15.133:0/2900028374 shutdown_connections 2023-11-27T15:22:31.484 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.479+0000 7fea951ff700 1 -- 172.21.15.133:0/2900028374 wait complete. 2023-11-27T15:22:31.486 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.480+0000 7fea951ff700 1 Processor -- start 2023-11-27T15:22:31.486 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.480+0000 7fea951ff700 1 -- start start 2023-11-27T15:22:31.487 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.481+0000 7fea951ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea901002b0 0x7fea900748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:31.487 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.481+0000 7fea951ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea901014a0 0x7fea90072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:31.487 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.481+0000 7fea951ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea90101e30 0x7fea90073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:31.487 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.481+0000 7fea951ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea90073bc0 con 0x7fea901014a0 2023-11-27T15:22:31.487 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.481+0000 7fea951ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea90073d30 con 0x7fea90101e30 2023-11-27T15:22:31.487 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.481+0000 7fea951ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea901a00b0 con 0x7fea901002b0 2023-11-27T15:22:31.487 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.481+0000 7fea8ed9d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea901002b0 0x7fea900748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:31.487 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.481+0000 7fea8ed9d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea901002b0 0x7fea900748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:56812/0 (socket says 172.21.15.133:56812) 2023-11-27T15:22:31.487 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.481+0000 7fea8ed9d700 1 -- 172.21.15.133:0/1383392548 learned_addr learned my addr 172.21.15.133:0/1383392548 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:31.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.481+0000 7fea87fff700 1 --2- 172.21.15.133:0/1383392548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea901014a0 0x7fea90072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:31.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.481+0000 7fea8f59e700 1 --2- 172.21.15.133:0/1383392548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea90101e30 0x7fea90073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:31.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.481+0000 7fea8ed9d700 1 -- 172.21.15.133:0/1383392548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea90101e30 msgr2=0x7fea90073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:31.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.481+0000 7fea8ed9d700 1 --2- 172.21.15.133:0/1383392548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea90101e30 0x7fea90073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.481+0000 7fea8ed9d700 1 -- 172.21.15.133:0/1383392548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea901014a0 msgr2=0x7fea90072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:31.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.481+0000 7fea8ed9d700 1 --2- 172.21.15.133:0/1383392548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea901014a0 0x7fea90072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.481+0000 7fea8ed9d700 1 -- 172.21.15.133:0/1383392548 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea80009520 con 0x7fea901002b0 2023-11-27T15:22:31.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.481+0000 7fea8f59e700 1 --2- 172.21.15.133:0/1383392548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea90101e30 0x7fea90073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:22:31.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.481+0000 7fea8ed9d700 1 --2- 172.21.15.133:0/1383392548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea901002b0 0x7fea900748a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fea80009fd0 tx=0x7fea8000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:31.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.482+0000 7fea8cd99700 1 -- 172.21.15.133:0/1383392548 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8001c070 con 0x7fea901002b0 2023-11-27T15:22:31.488 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.482+0000 7fea8cd99700 1 -- 172.21.15.133:0/1383392548 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea8000f7d0 con 0x7fea901002b0 2023-11-27T15:22:31.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.482+0000 7fea951ff700 1 -- 172.21.15.133:0/1383392548 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea9010bc00 con 0x7fea901002b0 2023-11-27T15:22:31.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.482+0000 7fea8cd99700 1 -- 172.21.15.133:0/1383392548 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea80003a80 con 0x7fea901002b0 2023-11-27T15:22:31.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.482+0000 7fea951ff700 1 -- 172.21.15.133:0/1383392548 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9010ba70 con 0x7fea901002b0 2023-11-27T15:22:31.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.483+0000 7fea951ff700 1 -- 172.21.15.133:0/1383392548 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea90107900 con 0x7fea901002b0 2023-11-27T15:22:31.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.485+0000 7fea8cd99700 1 -- 172.21.15.133:0/1383392548 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7fea80020030 con 0x7fea901002b0 2023-11-27T15:22:31.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.486+0000 7fea8cd99700 1 --2- 172.21.15.133:0/1383392548 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fea7c0a11e0 0x7fea7c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:31.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.486+0000 7fea8cd99700 1 -- 172.21.15.133:0/1383392548 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fea800ce2d0 con 0x7fea901002b0 2023-11-27T15:22:31.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.486+0000 7fea87fff700 1 --2- 172.21.15.133:0/1383392548 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fea7c0a11e0 0x7fea7c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:31.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.487+0000 7fea87fff700 1 --2- 172.21.15.133:0/1383392548 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fea7c0a11e0 0x7fea7c0a3690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fea78005fd0 tx=0x7fea78009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:31.492 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.488+0000 7fea8cd99700 1 -- 172.21.15.133:0/1383392548 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fea800975a0 con 0x7fea901002b0 2023-11-27T15:22:31.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.689+0000 7fea951ff700 1 -- 172.21.15.133:0/1383392548 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea90067a60 con 0x7fea901002b0 2023-11-27T15:22:31.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.692+0000 7fea8cd99700 1 -- 172.21.15.133:0/1383392548 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24616 (secure 0 0 0) 0x7fea8009abc0 con 0x7fea901002b0 2023-11-27T15:22:31.696 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:31.696 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":92,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:22:02.807005+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:22:31.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.694+0000 7fea951ff700 1 -- 172.21.15.133:0/1383392548 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fea7c0a11e0 msgr2=0x7fea7c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:31.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.694+0000 7fea951ff700 1 --2- 172.21.15.133:0/1383392548 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fea7c0a11e0 0x7fea7c0a3690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fea78005fd0 tx=0x7fea78009310 comp rx=0 tx=0).stop 2023-11-27T15:22:31.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.694+0000 7fea951ff700 1 -- 172.21.15.133:0/1383392548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea901002b0 msgr2=0x7fea900748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:31.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.694+0000 7fea951ff700 1 --2- 172.21.15.133:0/1383392548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea901002b0 0x7fea900748a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fea80009fd0 tx=0x7fea8000b840 comp rx=0 tx=0).stop 2023-11-27T15:22:31.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.694+0000 7fea951ff700 1 -- 172.21.15.133:0/1383392548 shutdown_connections 2023-11-27T15:22:31.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.694+0000 7fea951ff700 1 --2- 172.21.15.133:0/1383392548 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fea7c0a11e0 0x7fea7c0a3690 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.694+0000 7fea951ff700 1 --2- 172.21.15.133:0/1383392548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea90101e30 0x7fea90073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.694+0000 7fea951ff700 1 --2- 172.21.15.133:0/1383392548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fea901002b0 0x7fea900748a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.694+0000 7fea951ff700 1 --2- 172.21.15.133:0/1383392548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fea901014a0 0x7fea90072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.694+0000 7fea951ff700 1 -- 172.21.15.133:0/1383392548 >> 172.21.15.133:0/1383392548 conn(0x7fea900fb890 msgr2=0x7fea90104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:31.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.694+0000 7fea951ff700 1 -- 172.21.15.133:0/1383392548 shutdown_connections 2023-11-27T15:22:31.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.694+0000 7fea951ff700 1 -- 172.21.15.133:0/1383392548 wait complete. 2023-11-27T15:22:31.720 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:16:20.216150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:20:31.107146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:20:32.614089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-27T15:20:36.126040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-27T15:22:31.721 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:22:31.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.943+0000 7efdbe1ff700 1 -- 172.21.15.133:0/2453918908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdb81002c0 msgr2=0x7efdb81006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:31.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.943+0000 7efdbe1ff700 1 --2- 172.21.15.133:0/2453918908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdb81002c0 0x7efdb81006d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efda8009990 tx=0x7efda8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.943+0000 7efdbe1ff700 1 -- 172.21.15.133:0/2453918908 shutdown_connections 2023-11-27T15:22:31.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.943+0000 7efdbe1ff700 1 --2- 172.21.15.133:0/2453918908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdb8101e40 0x7efdb8108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.943+0000 7efdbe1ff700 1 --2- 172.21.15.133:0/2453918908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdb81002c0 0x7efdb81006d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.943+0000 7efdbe1ff700 1 --2- 172.21.15.133:0/2453918908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb81014b0 0x7efdb8101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.943+0000 7efdbe1ff700 1 -- 172.21.15.133:0/2453918908 >> 172.21.15.133:0/2453918908 conn(0x7efdb80fb8a0 msgr2=0x7efdb80fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:31.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.943+0000 7efdbe1ff700 1 -- 172.21.15.133:0/2453918908 shutdown_connections 2023-11-27T15:22:31.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.943+0000 7efdbe1ff700 1 -- 172.21.15.133:0/2453918908 wait complete. 2023-11-27T15:22:31.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.944+0000 7efdbe1ff700 1 Processor -- start 2023-11-27T15:22:31.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.944+0000 7efdbe1ff700 1 -- start start 2023-11-27T15:22:31.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.944+0000 7efdbe1ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdb81002c0 0x7efdb8196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:31.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.944+0000 7efdbe1ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb81014b0 0x7efdb8196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:31.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.945+0000 7efdbe1ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdb8101e40 0x7efdb819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:31.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.945+0000 7efdbe1ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb819db90 con 0x7efdb81014b0 2023-11-27T15:22:31.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.945+0000 7efdbe1ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb819dcd0 con 0x7efdb8101e40 2023-11-27T15:22:31.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.945+0000 7efdbe1ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb819de60 con 0x7efdb81002c0 2023-11-27T15:22:31.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.945+0000 7efdb6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb81014b0 0x7efdb8196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:31.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.945+0000 7efdb6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb81014b0 0x7efdb8196550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54994/0 (socket says 172.21.15.133:54994) 2023-11-27T15:22:31.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.945+0000 7efdb6ffd700 1 -- 172.21.15.133:0/1783671676 learned_addr learned my addr 172.21.15.133:0/1783671676 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:31.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.945+0000 7efdb77fe700 1 --2- 172.21.15.133:0/1783671676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdb81002c0 0x7efdb8196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:31.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.945+0000 7efdb7fff700 1 --2- 172.21.15.133:0/1783671676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdb8101e40 0x7efdb819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:31.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.945+0000 7efdb6ffd700 1 -- 172.21.15.133:0/1783671676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdb81002c0 msgr2=0x7efdb8196010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:31.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.945+0000 7efdb6ffd700 1 --2- 172.21.15.133:0/1783671676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdb81002c0 0x7efdb8196010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.945+0000 7efdb6ffd700 1 -- 172.21.15.133:0/1783671676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdb8101e40 msgr2=0x7efdb819d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:31.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.945+0000 7efdb6ffd700 1 --2- 172.21.15.133:0/1783671676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdb8101e40 0x7efdb819d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:31.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.945+0000 7efdb6ffd700 1 -- 172.21.15.133:0/1783671676 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda80095f0 con 0x7efdb81014b0 2023-11-27T15:22:31.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.945+0000 7efdb6ffd700 1 --2- 172.21.15.133:0/1783671676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb81014b0 0x7efdb8196550 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efdac00e9a0 tx=0x7efdac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:31.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.946+0000 7efdb4ff9700 1 -- 172.21.15.133:0/1783671676 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac009790 con 0x7efdb81014b0 2023-11-27T15:22:31.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.946+0000 7efdb4ff9700 1 -- 172.21.15.133:0/1783671676 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdac010430 con 0x7efdb81014b0 2023-11-27T15:22:31.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.946+0000 7efdbe1ff700 1 -- 172.21.15.133:0/1783671676 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb810bc10 con 0x7efdb81014b0 2023-11-27T15:22:31.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.946+0000 7efdb4ff9700 1 -- 172.21.15.133:0/1783671676 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac018680 con 0x7efdb81014b0 2023-11-27T15:22:31.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.946+0000 7efdbe1ff700 1 -- 172.21.15.133:0/1783671676 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb810ba80 con 0x7efdb81014b0 2023-11-27T15:22:31.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.947+0000 7efdbe1ff700 1 -- 172.21.15.133:0/1783671676 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb819e550 con 0x7efdb81014b0 2023-11-27T15:22:31.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.949+0000 7efdb4ff9700 1 -- 172.21.15.133:0/1783671676 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7efdac0105a0 con 0x7efdb81014b0 2023-11-27T15:22:31.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.951+0000 7efdb4ff9700 1 --2- 172.21.15.133:0/1783671676 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7efda00a11e0 0x7efda00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:31.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.951+0000 7efdb4ff9700 1 -- 172.21.15.133:0/1783671676 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7efdac014070 con 0x7efdb81014b0 2023-11-27T15:22:31.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.951+0000 7efdb77fe700 1 --2- 172.21.15.133:0/1783671676 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7efda00a11e0 0x7efda00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:31.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.952+0000 7efdb77fe700 1 --2- 172.21.15.133:0/1783671676 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7efda00a11e0 0x7efda00a3690 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efda800b2e0 tx=0x7efda8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:31.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:31.955+0000 7efdb4ff9700 1 -- 172.21.15.133:0/1783671676 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7efdac051730 con 0x7efdb81014b0 2023-11-27T15:22:32.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:31 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1750377300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:22:32.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:31 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3642978812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:32.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:31 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1750377300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:22:32.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:31 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3642978812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:32.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:31 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1750377300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:22:32.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:31 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3642978812' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:32.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.154+0000 7efdbe1ff700 1 -- 172.21.15.133:0/1783671676 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efdb81071f0 con 0x7efdb81014b0 2023-11-27T15:22:32.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.155+0000 7efdb4ff9700 1 -- 172.21.15.133:0/1783671676 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7efdac096250 con 0x7efdb81014b0 2023-11-27T15:22:32.159 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-11-27T15:22:32.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.157+0000 7efdbe1ff700 1 -- 172.21.15.133:0/1783671676 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7efda00a11e0 msgr2=0x7efda00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:32.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.157+0000 7efdbe1ff700 1 --2- 172.21.15.133:0/1783671676 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7efda00a11e0 0x7efda00a3690 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7efda800b2e0 tx=0x7efda8005fb0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.157+0000 7efdbe1ff700 1 -- 172.21.15.133:0/1783671676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb81014b0 msgr2=0x7efdb8196550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:32.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.157+0000 7efdbe1ff700 1 --2- 172.21.15.133:0/1783671676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb81014b0 0x7efdb8196550 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efdac00e9a0 tx=0x7efdac00ecb0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.157+0000 7efdbe1ff700 1 -- 172.21.15.133:0/1783671676 shutdown_connections 2023-11-27T15:22:32.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.157+0000 7efdbe1ff700 1 --2- 172.21.15.133:0/1783671676 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7efda00a11e0 0x7efda00a3690 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.157+0000 7efdbe1ff700 1 --2- 172.21.15.133:0/1783671676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdb8101e40 0x7efdb819d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.157+0000 7efdbe1ff700 1 --2- 172.21.15.133:0/1783671676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efdb81002c0 0x7efdb8196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.157+0000 7efdbe1ff700 1 --2- 172.21.15.133:0/1783671676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efdb81014b0 0x7efdb8196550 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.157+0000 7efdbe1ff700 1 -- 172.21.15.133:0/1783671676 >> 172.21.15.133:0/1783671676 conn(0x7efdb80fb8a0 msgr2=0x7efdb80fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:32.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.157+0000 7efdbe1ff700 1 -- 172.21.15.133:0/1783671676 shutdown_connections 2023-11-27T15:22:32.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.157+0000 7efdbe1ff700 1 -- 172.21.15.133:0/1783671676 wait complete. 2023-11-27T15:22:32.181 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:22:32.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.375+0000 7f97c3bff700 1 -- 172.21.15.133:0/2677021739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97bc1002c0 msgr2=0x7f97bc1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:32.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.375+0000 7f97c3bff700 1 --2- 172.21.15.133:0/2677021739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97bc1002c0 0x7f97bc1006d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f97b8009990 tx=0x7f97b8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.375+0000 7f97c3bff700 1 -- 172.21.15.133:0/2677021739 shutdown_connections 2023-11-27T15:22:32.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.375+0000 7f97c3bff700 1 --2- 172.21.15.133:0/2677021739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97bc101e40 0x7f97bc108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.375+0000 7f97c3bff700 1 --2- 172.21.15.133:0/2677021739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97bc1002c0 0x7f97bc1006d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.375+0000 7f97c3bff700 1 --2- 172.21.15.133:0/2677021739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97bc1014b0 0x7f97bc101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.375+0000 7f97c3bff700 1 -- 172.21.15.133:0/2677021739 >> 172.21.15.133:0/2677021739 conn(0x7f97bc0fb8a0 msgr2=0x7f97bc0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:32.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.375+0000 7f97c3bff700 1 -- 172.21.15.133:0/2677021739 shutdown_connections 2023-11-27T15:22:32.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.375+0000 7f97c3bff700 1 -- 172.21.15.133:0/2677021739 wait complete. 2023-11-27T15:22:32.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.376+0000 7f97c3bff700 1 Processor -- start 2023-11-27T15:22:32.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.377+0000 7f97c3bff700 1 -- start start 2023-11-27T15:22:32.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.377+0000 7f97c3bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97bc1002c0 0x7f97bc0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:32.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.377+0000 7f97c3bff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97bc1014b0 0x7f97bc072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:32.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.377+0000 7f97c3bff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97bc101e40 0x7f97bc073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:32.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.377+0000 7f97c3bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc073bc0 con 0x7f97bc1002c0 2023-11-27T15:22:32.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.377+0000 7f97c3bff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc073d30 con 0x7f97bc101e40 2023-11-27T15:22:32.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.377+0000 7f97c3bff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc1a00d0 con 0x7f97bc1014b0 2023-11-27T15:22:32.384 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.377+0000 7f97c199b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97bc1002c0 0x7f97bc0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:32.384 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.377+0000 7f97c199b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97bc1002c0 0x7f97bc0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:55008/0 (socket says 172.21.15.133:55008) 2023-11-27T15:22:32.384 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.377+0000 7f97c199b700 1 -- 172.21.15.133:0/2258462890 learned_addr learned my addr 172.21.15.133:0/2258462890 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:32.384 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.377+0000 7f97c119a700 1 --2- 172.21.15.133:0/2258462890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97bc1014b0 0x7f97bc072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:32.384 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.377+0000 7f97c219c700 1 --2- 172.21.15.133:0/2258462890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97bc101e40 0x7f97bc073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:32.384 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.378+0000 7f97c199b700 1 -- 172.21.15.133:0/2258462890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97bc1014b0 msgr2=0x7f97bc072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:32.384 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.378+0000 7f97c199b700 1 --2- 172.21.15.133:0/2258462890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97bc1014b0 0x7f97bc072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.384 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.378+0000 7f97c199b700 1 -- 172.21.15.133:0/2258462890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97bc101e40 msgr2=0x7f97bc073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:32.384 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.378+0000 7f97c199b700 1 --2- 172.21.15.133:0/2258462890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97bc101e40 0x7f97bc073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.378+0000 7f97c199b700 1 -- 172.21.15.133:0/2258462890 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b80095f0 con 0x7f97bc1002c0 2023-11-27T15:22:32.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.378+0000 7f97c199b700 1 --2- 172.21.15.133:0/2258462890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97bc1002c0 0x7f97bc0748a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f97b8004730 tx=0x7f97b8004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:32.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.378+0000 7f97b2ffd700 1 -- 172.21.15.133:0/2258462890 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b801d070 con 0x7f97bc1002c0 2023-11-27T15:22:32.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.378+0000 7f97b2ffd700 1 -- 172.21.15.133:0/2258462890 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97b8021410 con 0x7f97bc1002c0 2023-11-27T15:22:32.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.378+0000 7f97b2ffd700 1 -- 172.21.15.133:0/2258462890 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b8017650 con 0x7f97bc1002c0 2023-11-27T15:22:32.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.378+0000 7f97c3bff700 1 -- 172.21.15.133:0/2258462890 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97bc10bc90 con 0x7f97bc1002c0 2023-11-27T15:22:32.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.378+0000 7f97c3bff700 1 -- 172.21.15.133:0/2258462890 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97bc10bb00 con 0x7f97bc1002c0 2023-11-27T15:22:32.386 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.379+0000 7f97c3bff700 1 -- 172.21.15.133:0/2258462890 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97bc107860 con 0x7f97bc1002c0 2023-11-27T15:22:32.386 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.382+0000 7f97b2ffd700 1 -- 172.21.15.133:0/2258462890 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 17) v1 ==== 135767+0+0 (secure 0 0 0) 0x7f97b8021580 con 0x7f97bc1002c0 2023-11-27T15:22:32.387 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.383+0000 7f97b2ffd700 1 --2- 172.21.15.133:0/2258462890 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f97a80a1260 0x7f97a80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:32.387 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.383+0000 7f97b2ffd700 1 -- 172.21.15.133:0/2258462890 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f97b80ce810 con 0x7f97bc1002c0 2023-11-27T15:22:32.387 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.383+0000 7f97c119a700 1 --2- 172.21.15.133:0/2258462890 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f97a80a1260 0x7f97a80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:32.387 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.384+0000 7f97c119a700 1 --2- 172.21.15.133:0/2258462890 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f97a80a1260 0x7f97a80a3710 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009c60 tx=0x7f97ac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:32.388 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.385+0000 7f97b2ffd700 1 -- 172.21.15.133:0/2258462890 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f97b8097b40 con 0x7f97bc1002c0 2023-11-27T15:22:32.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.585+0000 7f97c3bff700 1 -- 172.21.15.133:0/2258462890 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f97bc061b00 con 0x7f97bc1002c0 2023-11-27T15:22:32.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.585+0000 7f97b2ffd700 1 -- 172.21.15.133:0/2258462890 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f97b809b160 con 0x7f97bc1002c0 2023-11-27T15:22:32.589 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:22:32.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.587+0000 7f97c3bff700 1 -- 172.21.15.133:0/2258462890 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f97a80a1260 msgr2=0x7f97a80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:32.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.587+0000 7f97c3bff700 1 --2- 172.21.15.133:0/2258462890 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f97a80a1260 0x7f97a80a3710 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009c60 tx=0x7f97ac009260 comp rx=0 tx=0).stop 2023-11-27T15:22:32.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.587+0000 7f97c3bff700 1 -- 172.21.15.133:0/2258462890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97bc1002c0 msgr2=0x7f97bc0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:32.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.587+0000 7f97c3bff700 1 --2- 172.21.15.133:0/2258462890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97bc1002c0 0x7f97bc0748a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f97b8004730 tx=0x7f97b8004760 comp rx=0 tx=0).stop 2023-11-27T15:22:32.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.587+0000 7f97c3bff700 1 -- 172.21.15.133:0/2258462890 shutdown_connections 2023-11-27T15:22:32.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.587+0000 7f97c3bff700 1 --2- 172.21.15.133:0/2258462890 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f97a80a1260 0x7f97a80a3710 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.587+0000 7f97c3bff700 1 --2- 172.21.15.133:0/2258462890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97bc101e40 0x7f97bc073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.587+0000 7f97c3bff700 1 --2- 172.21.15.133:0/2258462890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97bc1002c0 0x7f97bc0748a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.587+0000 7f97c3bff700 1 --2- 172.21.15.133:0/2258462890 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97bc1014b0 0x7f97bc072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.587+0000 7f97c3bff700 1 -- 172.21.15.133:0/2258462890 >> 172.21.15.133:0/2258462890 conn(0x7f97bc0fb8a0 msgr2=0x7f97bc1049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:32.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.588+0000 7f97c3bff700 1 -- 172.21.15.133:0/2258462890 shutdown_connections 2023-11-27T15:22:32.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.588+0000 7f97c3bff700 1 -- 172.21.15.133:0/2258462890 wait complete. 2023-11-27T15:22:32.611 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:22:32.818 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.814+0000 7f7125bff700 1 -- 172.21.15.133:0/2903688773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7120101e50 msgr2=0x7f7120108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:32.818 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.814+0000 7f7125bff700 1 --2- 172.21.15.133:0/2903688773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7120101e50 0x7f7120108710 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f711800b0a0 tx=0x7f711800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.818 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.814+0000 7f7125bff700 1 -- 172.21.15.133:0/2903688773 shutdown_connections 2023-11-27T15:22:32.818 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.814+0000 7f7125bff700 1 --2- 172.21.15.133:0/2903688773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7120101e50 0x7f7120108710 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.818 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.814+0000 7f7125bff700 1 --2- 172.21.15.133:0/2903688773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71201002d0 0x7f71201006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.819 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.814+0000 7f7125bff700 1 --2- 172.21.15.133:0/2903688773 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71201014c0 0x7f7120101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.819 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.814+0000 7f7125bff700 1 -- 172.21.15.133:0/2903688773 >> 172.21.15.133:0/2903688773 conn(0x7f71200fb8b0 msgr2=0x7f71200fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:32.819 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.814+0000 7f7125bff700 1 -- 172.21.15.133:0/2903688773 shutdown_connections 2023-11-27T15:22:32.819 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.814+0000 7f7125bff700 1 -- 172.21.15.133:0/2903688773 wait complete. 2023-11-27T15:22:32.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.815+0000 7f7125bff700 1 Processor -- start 2023-11-27T15:22:32.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.815+0000 7f7125bff700 1 -- start start 2023-11-27T15:22:32.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.815+0000 7f7125bff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71201002d0 0x7f7120198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:32.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.815+0000 7f7125bff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71201014c0 0x7f7120198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:32.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.815+0000 7f7125bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7120101e50 0x7f712019f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:32.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.816+0000 7f7125bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712019ff90 con 0x7f7120101e50 2023-11-27T15:22:32.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.816+0000 7f7125bff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71201a00d0 con 0x7f71201014c0 2023-11-27T15:22:32.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.816+0000 7f7125bff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71201a0260 con 0x7f71201002d0 2023-11-27T15:22:32.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.816+0000 7f711effd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71201014c0 0x7f7120198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:32.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.816+0000 7f711effd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71201014c0 0x7f7120198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:57906/0 (socket says 172.21.15.133:57906) 2023-11-27T15:22:32.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.816+0000 7f711effd700 1 -- 172.21.15.133:0/3934024958 learned_addr learned my addr 172.21.15.133:0/3934024958 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:32.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.816+0000 7f711f7fe700 1 --2- 172.21.15.133:0/3934024958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71201002d0 0x7f7120198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:32.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.816+0000 7f711ffff700 1 --2- 172.21.15.133:0/3934024958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7120101e50 0x7f712019f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:32.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.816+0000 7f711effd700 1 -- 172.21.15.133:0/3934024958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71201002d0 msgr2=0x7f7120198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:32.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.816+0000 7f711effd700 1 --2- 172.21.15.133:0/3934024958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71201002d0 0x7f7120198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.816+0000 7f711effd700 1 -- 172.21.15.133:0/3934024958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7120101e50 msgr2=0x7f712019f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:32.823 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.816+0000 7f711effd700 1 --2- 172.21.15.133:0/3934024958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7120101e50 0x7f712019f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:32.823 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.816+0000 7f711effd700 1 -- 172.21.15.133:0/3934024958 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7110009520 con 0x7f71201014c0 2023-11-27T15:22:32.823 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.816+0000 7f711ffff700 1 --2- 172.21.15.133:0/3934024958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7120101e50 0x7f712019f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:32.823 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.816+0000 7f711effd700 1 --2- 172.21.15.133:0/3934024958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71201014c0 0x7f7120198950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f711400e9a0 tx=0x7f711400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:32.823 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.817+0000 7f711cff9700 1 -- 172.21.15.133:0/3934024958 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7114009790 con 0x7f71201014c0 2023-11-27T15:22:32.823 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.817+0000 7f711cff9700 1 -- 172.21.15.133:0/3934024958 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7114010430 con 0x7f71201014c0 2023-11-27T15:22:32.823 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.817+0000 7f7125bff700 1 -- 172.21.15.133:0/3934024958 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7118009d00 con 0x7f71201014c0 2023-11-27T15:22:32.823 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.817+0000 7f711cff9700 1 -- 172.21.15.133:0/3934024958 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7114018680 con 0x7f71201014c0 2023-11-27T15:22:32.823 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.817+0000 7f7125bff700 1 -- 172.21.15.133:0/3934024958 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712010bca0 con 0x7f71201014c0 2023-11-27T15:22:32.824 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.818+0000 7f7125bff700 1 -- 172.21.15.133:0/3934024958 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7120067b00 con 0x7f71201014c0 2023-11-27T15:22:32.824 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.820+0000 7f711cff9700 1 -- 172.21.15.133:0/3934024958 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f71140105a0 con 0x7f71201014c0 2023-11-27T15:22:32.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.821+0000 7f711cff9700 1 --2- 172.21.15.133:0/3934024958 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f71080a1150 0x7f71080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:32.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.821+0000 7f711cff9700 1 -- 172.21.15.133:0/3934024958 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7114014070 con 0x7f71201014c0 2023-11-27T15:22:32.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.821+0000 7f711f7fe700 1 --2- 172.21.15.133:0/3934024958 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f71080a1150 0x7f71080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:32.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.822+0000 7f711f7fe700 1 --2- 172.21.15.133:0/3934024958 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f71080a1150 0x7f71080a3600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7120101320 tx=0x7f7110009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:32.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:32.825+0000 7f711cff9700 1 -- 172.21.15.133:0/3934024958 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7114051730 con 0x7f71201014c0 2023-11-27T15:22:33.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:32 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1383392548' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:33.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:32 smithi133 ceph-mon[97860]: pgmap v337: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:22:33.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:32 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1783671676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:33.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:32 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2258462890' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:22:33.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.027+0000 7f7125bff700 1 -- 172.21.15.133:0/3934024958 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f71201a0e60 con 0x7f71201014c0 2023-11-27T15:22:33.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.028+0000 7f711cff9700 1 -- 172.21.15.133:0/3934024958 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7114096190 con 0x7f71201014c0 2023-11-27T15:22:33.032 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-11-27T15:22:33.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.031+0000 7f7125bff700 1 -- 172.21.15.133:0/3934024958 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f71080a1150 msgr2=0x7f71080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:33.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.031+0000 7f7125bff700 1 --2- 172.21.15.133:0/3934024958 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f71080a1150 0x7f71080a3600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7120101320 tx=0x7f7110009260 comp rx=0 tx=0).stop 2023-11-27T15:22:33.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.031+0000 7f7125bff700 1 -- 172.21.15.133:0/3934024958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71201014c0 msgr2=0x7f7120198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:33.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.031+0000 7f7125bff700 1 --2- 172.21.15.133:0/3934024958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71201014c0 0x7f7120198950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f711400e9a0 tx=0x7f711400ecb0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.031+0000 7f7125bff700 1 -- 172.21.15.133:0/3934024958 shutdown_connections 2023-11-27T15:22:33.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.031+0000 7f7125bff700 1 --2- 172.21.15.133:0/3934024958 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f71080a1150 0x7f71080a3600 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.031+0000 7f7125bff700 1 --2- 172.21.15.133:0/3934024958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7120101e50 0x7f712019f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.031+0000 7f7125bff700 1 --2- 172.21.15.133:0/3934024958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71201002d0 0x7f7120198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.031+0000 7f7125bff700 1 --2- 172.21.15.133:0/3934024958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71201014c0 0x7f7120198950 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.031+0000 7f7125bff700 1 -- 172.21.15.133:0/3934024958 >> 172.21.15.133:0/3934024958 conn(0x7f71200fb8b0 msgr2=0x7f7120106d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:33.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.031+0000 7f7125bff700 1 -- 172.21.15.133:0/3934024958 shutdown_connections 2023-11-27T15:22:33.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.031+0000 7f7125bff700 1 -- 172.21.15.133:0/3934024958 wait complete. 2023-11-27T15:22:33.054 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-27T15:22:33.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:32 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1383392548' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:33.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:32 smithi177 ceph-mon[131504]: pgmap v337: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:22:33.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:32 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1783671676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:33.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:32 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2258462890' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:22:33.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:32 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1383392548' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:33.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:32 smithi174 ceph-mon[131110]: pgmap v337: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-11-27T15:22:33.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:32 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1783671676' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:33.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:32 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2258462890' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:22:33.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.252+0000 7f8950bff700 1 -- 172.21.15.133:0/3017209252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f894c1014b0 msgr2=0x7f894c101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:33.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.252+0000 7f8950bff700 1 --2- 172.21.15.133:0/3017209252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f894c1014b0 0x7f894c101900 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8940009990 tx=0x7f8940009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.252+0000 7f8950bff700 1 -- 172.21.15.133:0/3017209252 shutdown_connections 2023-11-27T15:22:33.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.252+0000 7f8950bff700 1 --2- 172.21.15.133:0/3017209252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f894c101e40 0x7f894c108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.252+0000 7f8950bff700 1 --2- 172.21.15.133:0/3017209252 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f894c1002c0 0x7f894c1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.252+0000 7f8950bff700 1 --2- 172.21.15.133:0/3017209252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f894c1014b0 0x7f894c101900 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.252+0000 7f8950bff700 1 -- 172.21.15.133:0/3017209252 >> 172.21.15.133:0/3017209252 conn(0x7f894c0fb8a0 msgr2=0x7f894c0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:33.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.252+0000 7f8950bff700 1 -- 172.21.15.133:0/3017209252 shutdown_connections 2023-11-27T15:22:33.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.252+0000 7f8950bff700 1 -- 172.21.15.133:0/3017209252 wait complete. 2023-11-27T15:22:33.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.253+0000 7f8950bff700 1 Processor -- start 2023-11-27T15:22:33.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.253+0000 7f8950bff700 1 -- start start 2023-11-27T15:22:33.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.254+0000 7f8950bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f894c1002c0 0x7f894c0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:33.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.254+0000 7f8950bff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f894c1014b0 0x7f894c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:33.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.254+0000 7f8950bff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f894c101e40 0x7f894c073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:33.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.254+0000 7f8950bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c073bc0 con 0x7f894c1002c0 2023-11-27T15:22:33.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.254+0000 7f8950bff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c073d30 con 0x7f894c1014b0 2023-11-27T15:22:33.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.254+0000 7f8950bff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c1a0050 con 0x7f894c101e40 2023-11-27T15:22:33.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.254+0000 7f894a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f894c1002c0 0x7f894c0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:33.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.254+0000 7f894a59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f894c1002c0 0x7f894c0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44954/0 (socket says 172.21.15.133:44954) 2023-11-27T15:22:33.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.254+0000 7f894a59c700 1 -- 172.21.15.133:0/328794454 learned_addr learned my addr 172.21.15.133:0/328794454 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:33.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.254+0000 7f8949d9b700 1 --2- 172.21.15.133:0/328794454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f894c1014b0 0x7f894c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:33.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.254+0000 7f894ad9d700 1 --2- 172.21.15.133:0/328794454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f894c101e40 0x7f894c073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:33.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.254+0000 7f894a59c700 1 -- 172.21.15.133:0/328794454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f894c101e40 msgr2=0x7f894c073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:33.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.254+0000 7f894a59c700 1 --2- 172.21.15.133:0/328794454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f894c101e40 0x7f894c073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.254+0000 7f894a59c700 1 -- 172.21.15.133:0/328794454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f894c1014b0 msgr2=0x7f894c072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:33.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.254+0000 7f894a59c700 1 --2- 172.21.15.133:0/328794454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f894c1014b0 0x7f894c072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.255+0000 7f894a59c700 1 -- 172.21.15.133:0/328794454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893c009520 con 0x7f894c1002c0 2023-11-27T15:22:33.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.255+0000 7f894a59c700 1 --2- 172.21.15.133:0/328794454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f894c1002c0 0x7f894c0748a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f893c00ea50 tx=0x7f893c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:33.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.255+0000 7f893b7fe700 1 -- 172.21.15.133:0/328794454 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c00cbd0 con 0x7f894c1002c0 2023-11-27T15:22:33.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.255+0000 7f893b7fe700 1 -- 172.21.15.133:0/328794454 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f893c00cd30 con 0x7f894c1002c0 2023-11-27T15:22:33.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.255+0000 7f893b7fe700 1 -- 172.21.15.133:0/328794454 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893c018680 con 0x7f894c1002c0 2023-11-27T15:22:33.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.255+0000 7f8950bff700 1 -- 172.21.15.133:0/328794454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89400095f0 con 0x7f894c1002c0 2023-11-27T15:22:33.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.255+0000 7f8950bff700 1 -- 172.21.15.133:0/328794454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894c10bc20 con 0x7f894c1002c0 2023-11-27T15:22:33.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.256+0000 7f8950bff700 1 -- 172.21.15.133:0/328794454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f894c067b00 con 0x7f894c1002c0 2023-11-27T15:22:33.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.258+0000 7f893b7fe700 1 -- 172.21.15.133:0/328794454 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f893c010430 con 0x7f894c1002c0 2023-11-27T15:22:33.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.260+0000 7f893b7fe700 1 --2- 172.21.15.133:0/328794454 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f89340a11d0 0x7f89340a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:33.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.260+0000 7f893b7fe700 1 -- 172.21.15.133:0/328794454 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f893c014070 con 0x7f894c1002c0 2023-11-27T15:22:33.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.261+0000 7f8949d9b700 1 --2- 172.21.15.133:0/328794454 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f89340a11d0 0x7f89340a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:33.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.261+0000 7f8949d9b700 1 --2- 172.21.15.133:0/328794454 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f89340a11d0 0x7f89340a3680 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f894000b2e0 tx=0x7f8940005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:33.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.265+0000 7f893b7fe700 1 -- 172.21.15.133:0/328794454 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f893c096470 con 0x7f894c1002c0 2023-11-27T15:22:33.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.463+0000 7f8950bff700 1 -- 172.21.15.133:0/328794454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f894c107170 con 0x7f894c1002c0 2023-11-27T15:22:33.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.464+0000 7f893b7fe700 1 -- 172.21.15.133:0/328794454 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f893c099a90 con 0x7f894c1002c0 2023-11-27T15:22:33.467 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:22:33.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.466+0000 7f8950bff700 1 -- 172.21.15.133:0/328794454 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f89340a11d0 msgr2=0x7f89340a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:33.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.466+0000 7f8950bff700 1 --2- 172.21.15.133:0/328794454 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f89340a11d0 0x7f89340a3680 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f894000b2e0 tx=0x7f8940005fb0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.466+0000 7f8950bff700 1 -- 172.21.15.133:0/328794454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f894c1002c0 msgr2=0x7f894c0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:33.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.466+0000 7f8950bff700 1 --2- 172.21.15.133:0/328794454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f894c1002c0 0x7f894c0748a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f893c00ea50 tx=0x7f893c00ed60 comp rx=0 tx=0).stop 2023-11-27T15:22:33.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.466+0000 7f8950bff700 1 -- 172.21.15.133:0/328794454 shutdown_connections 2023-11-27T15:22:33.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.466+0000 7f8950bff700 1 --2- 172.21.15.133:0/328794454 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f89340a11d0 0x7f89340a3680 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.466+0000 7f8950bff700 1 --2- 172.21.15.133:0/328794454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f894c101e40 0x7f894c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.466+0000 7f8950bff700 1 --2- 172.21.15.133:0/328794454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f894c1002c0 0x7f894c0748a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.466+0000 7f8950bff700 1 --2- 172.21.15.133:0/328794454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f894c1014b0 0x7f894c072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.466+0000 7f8950bff700 1 -- 172.21.15.133:0/328794454 >> 172.21.15.133:0/328794454 conn(0x7f894c0fb8a0 msgr2=0x7f894c0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:33.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.466+0000 7f8950bff700 1 -- 172.21.15.133:0/328794454 shutdown_connections 2023-11-27T15:22:33.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.466+0000 7f8950bff700 1 -- 172.21.15.133:0/328794454 wait complete. 2023-11-27T15:22:33.489 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:22:33.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.684+0000 7fe09bbff700 1 -- 172.21.15.133:0/3304448116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe094100290 msgr2=0x7fe0941006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:33.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.684+0000 7fe09bbff700 1 --2- 172.21.15.133:0/3304448116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe094100290 0x7fe0941006a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe090009990 tx=0x7fe090009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.685+0000 7fe09bbff700 1 -- 172.21.15.133:0/3304448116 shutdown_connections 2023-11-27T15:22:33.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.685+0000 7fe09bbff700 1 --2- 172.21.15.133:0/3304448116 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe094101e10 0x7fe0941086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.685+0000 7fe09bbff700 1 --2- 172.21.15.133:0/3304448116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe094100290 0x7fe0941006a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.685+0000 7fe09bbff700 1 --2- 172.21.15.133:0/3304448116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe094101480 0x7fe0941018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.685+0000 7fe09bbff700 1 -- 172.21.15.133:0/3304448116 >> 172.21.15.133:0/3304448116 conn(0x7fe0940fb870 msgr2=0x7fe0940fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:33.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.685+0000 7fe09bbff700 1 -- 172.21.15.133:0/3304448116 shutdown_connections 2023-11-27T15:22:33.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.685+0000 7fe09bbff700 1 -- 172.21.15.133:0/3304448116 wait complete. 2023-11-27T15:22:33.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.686+0000 7fe09bbff700 1 Processor -- start 2023-11-27T15:22:33.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.686+0000 7fe09bbff700 1 -- start start 2023-11-27T15:22:33.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.686+0000 7fe09bbff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe094100290 0x7fe0941981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:33.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.686+0000 7fe09bbff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe094101480 0x7fe094198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:33.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.686+0000 7fe09bbff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe094101e10 0x7fe09419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:33.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.686+0000 7fe09bbff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09419fd40 con 0x7fe094100290 2023-11-27T15:22:33.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.686+0000 7fe09bbff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09419fe80 con 0x7fe094101e10 2023-11-27T15:22:33.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.686+0000 7fe09bbff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0941a0010 con 0x7fe094101480 2023-11-27T15:22:33.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.686+0000 7fe09999b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe094100290 0x7fe0941981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:33.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.686+0000 7fe09919a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe094101480 0x7fe094198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:33.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.686+0000 7fe09919a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe094101480 0x7fe094198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:50174/0 (socket says 172.21.15.133:50174) 2023-11-27T15:22:33.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.686+0000 7fe09919a700 1 -- 172.21.15.133:0/215037347 learned_addr learned my addr 172.21.15.133:0/215037347 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:33.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.686+0000 7fe09a19c700 1 --2- 172.21.15.133:0/215037347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe094101e10 0x7fe09419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:33.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.687+0000 7fe09a19c700 1 -- 172.21.15.133:0/215037347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe094101480 msgr2=0x7fe094198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:33.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.687+0000 7fe09a19c700 1 --2- 172.21.15.133:0/215037347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe094101480 0x7fe094198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.687+0000 7fe09a19c700 1 -- 172.21.15.133:0/215037347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe094100290 msgr2=0x7fe0941981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:33.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.687+0000 7fe09a19c700 1 --2- 172.21.15.133:0/215037347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe094100290 0x7fe0941981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.687+0000 7fe09a19c700 1 -- 172.21.15.133:0/215037347 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0900095f0 con 0x7fe094101e10 2023-11-27T15:22:33.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.687+0000 7fe09919a700 1 --2- 172.21.15.133:0/215037347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe094101480 0x7fe094198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:33.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.687+0000 7fe09999b700 1 --2- 172.21.15.133:0/215037347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe094100290 0x7fe0941981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:33.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.687+0000 7fe09a19c700 1 --2- 172.21.15.133:0/215037347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe094101e10 0x7fe09419f770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00efc0 tx=0x7fe08c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:33.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.687+0000 7fe08affd700 1 -- 172.21.15.133:0/215037347 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c018070 con 0x7fe094101e10 2023-11-27T15:22:33.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.687+0000 7fe08affd700 1 -- 172.21.15.133:0/215037347 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe08c00f040 con 0x7fe094101e10 2023-11-27T15:22:33.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.687+0000 7fe09bbff700 1 -- 172.21.15.133:0/215037347 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe09410bc40 con 0x7fe094101e10 2023-11-27T15:22:33.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.687+0000 7fe08affd700 1 -- 172.21.15.133:0/215037347 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c00cd40 con 0x7fe094101e10 2023-11-27T15:22:33.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.687+0000 7fe09bbff700 1 -- 172.21.15.133:0/215037347 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09410bab0 con 0x7fe094101e10 2023-11-27T15:22:33.694 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.688+0000 7fe09bbff700 1 -- 172.21.15.133:0/215037347 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe094067a60 con 0x7fe094101e10 2023-11-27T15:22:33.694 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.690+0000 7fe08affd700 1 -- 172.21.15.133:0/215037347 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe08c003a00 con 0x7fe094101e10 2023-11-27T15:22:33.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.691+0000 7fe08affd700 1 --2- 172.21.15.133:0/215037347 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe0800a1150 0x7fe0800a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:33.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.691+0000 7fe08affd700 1 -- 172.21.15.133:0/215037347 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe08c0cc950 con 0x7fe094101e10 2023-11-27T15:22:33.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.692+0000 7fe09999b700 1 --2- 172.21.15.133:0/215037347 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe0800a1150 0x7fe0800a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:33.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.692+0000 7fe09999b700 1 --2- 172.21.15.133:0/215037347 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe0800a1150 0x7fe0800a3600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe09000b2e0 tx=0x7fe090005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:33.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.694+0000 7fe08affd700 1 -- 172.21.15.133:0/215037347 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe08c095d80 con 0x7fe094101e10 2023-11-27T15:22:33.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:33 smithi133 ceph-mon[97860]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-11-27T15:22:33.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:33 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3934024958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:22:33.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:33 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/328794454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:22:33.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.938+0000 7fe09bbff700 1 -- 172.21.15.133:0/215037347 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0941a0ac0 con 0x7fe094101e10 2023-11-27T15:22:33.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.939+0000 7fe08affd700 1 -- 172.21.15.133:0/215037347 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10829 (secure 0 0 0) 0x7fe08c0993a0 con 0x7fe094101e10 2023-11-27T15:22:33.943 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:33.944 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14493,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2381290259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2381290259},{"type":"v1","addr":"172.21.15.133:6839","nonce":2381290259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1704875895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1704875895},{"type":"v1","addr":"172.21.15.133:6841","nonce":1704875895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:20:33.201023+0000","modified":"2023-11-27T15:21:04.794526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24521,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3492215689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3492215689},{"type":"v1","addr":"172.21.15.177:6837","nonce":3492215689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6839/2781994447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2781994447},{"type":"v1","addr":"172.21.15.174:6839","nonce":2781994447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:22:33.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.942+0000 7fe09bbff700 1 -- 172.21.15.133:0/215037347 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe0800a1150 msgr2=0x7fe0800a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:33.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.942+0000 7fe09bbff700 1 --2- 172.21.15.133:0/215037347 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe0800a1150 0x7fe0800a3600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe09000b2e0 tx=0x7fe090005fb0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.942+0000 7fe09bbff700 1 -- 172.21.15.133:0/215037347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe094101e10 msgr2=0x7fe09419f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.942+0000 7fe09bbff700 1 --2- 172.21.15.133:0/215037347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe094101e10 0x7fe09419f770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00efc0 tx=0x7fe08c00c3a0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.942+0000 7fe09bbff700 1 -- 172.21.15.133:0/215037347 shutdown_connections 2023-11-27T15:22:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.942+0000 7fe09bbff700 1 --2- 172.21.15.133:0/215037347 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe0800a1150 0x7fe0800a3600 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.942+0000 7fe09bbff700 1 --2- 172.21.15.133:0/215037347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe094101e10 0x7fe09419f770 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.942+0000 7fe09bbff700 1 --2- 172.21.15.133:0/215037347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe094100290 0x7fe0941981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.942+0000 7fe09bbff700 1 --2- 172.21.15.133:0/215037347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe094101480 0x7fe094198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.942+0000 7fe09bbff700 1 -- 172.21.15.133:0/215037347 >> 172.21.15.133:0/215037347 conn(0x7fe0940fb870 msgr2=0x7fe0940fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.942+0000 7fe09bbff700 1 -- 172.21.15.133:0/215037347 shutdown_connections 2023-11-27T15:22:33.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:33.942+0000 7fe09bbff700 1 -- 172.21.15.133:0/215037347 wait complete. 2023-11-27T15:22:33.948 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 23 2023-11-27T15:22:33.965 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:22:34.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:33 smithi177 ceph-mon[131504]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-11-27T15:22:34.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:33 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3934024958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:22:34.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:33 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/328794454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:22:34.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:33 smithi174 ceph-mon[131110]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-11-27T15:22:34.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:33 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3934024958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:22:34.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:33 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/328794454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:22:34.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.159+0000 7fe1885ff700 1 -- 172.21.15.133:0/841661049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1801014a0 msgr2=0x7fe1801018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.159+0000 7fe1885ff700 1 --2- 172.21.15.133:0/841661049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1801014a0 0x7fe1801018f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe170009990 tx=0x7fe170009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.159+0000 7fe1885ff700 1 -- 172.21.15.133:0/841661049 shutdown_connections 2023-11-27T15:22:34.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.159+0000 7fe1885ff700 1 --2- 172.21.15.133:0/841661049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe180101e30 0x7fe1801086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.159+0000 7fe1885ff700 1 --2- 172.21.15.133:0/841661049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1801002b0 0x7fe1801006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.159+0000 7fe1885ff700 1 --2- 172.21.15.133:0/841661049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1801014a0 0x7fe1801018f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.159+0000 7fe1885ff700 1 -- 172.21.15.133:0/841661049 >> 172.21.15.133:0/841661049 conn(0x7fe1800fb890 msgr2=0x7fe1800fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:34.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.159+0000 7fe1885ff700 1 -- 172.21.15.133:0/841661049 shutdown_connections 2023-11-27T15:22:34.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.159+0000 7fe1885ff700 1 -- 172.21.15.133:0/841661049 wait complete. 2023-11-27T15:22:34.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.160+0000 7fe1885ff700 1 Processor -- start 2023-11-27T15:22:34.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.160+0000 7fe1885ff700 1 -- start start 2023-11-27T15:22:34.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.161+0000 7fe1885ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1801002b0 0x7fe1801981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.161+0000 7fe1885ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1801014a0 0x7fe1801986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.161+0000 7fe1885ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe180101e30 0x7fe18019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.161+0000 7fe1885ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18019fd30 con 0x7fe1801014a0 2023-11-27T15:22:34.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.161+0000 7fe1885ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18019fe70 con 0x7fe1801002b0 2023-11-27T15:22:34.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.161+0000 7fe1885ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1801a0000 con 0x7fe180101e30 2023-11-27T15:22:34.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.161+0000 7fe185b9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1801014a0 0x7fe1801986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.161+0000 7fe185b9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1801014a0 0x7fe1801986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45002/0 (socket says 172.21.15.133:45002) 2023-11-27T15:22:34.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.161+0000 7fe185b9a700 1 -- 172.21.15.133:0/1283001236 learned_addr learned my addr 172.21.15.133:0/1283001236 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:34.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.161+0000 7fe18639b700 1 --2- 172.21.15.133:0/1283001236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1801002b0 0x7fe1801981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.161+0000 7fe186b9c700 1 --2- 172.21.15.133:0/1283001236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe180101e30 0x7fe18019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.161+0000 7fe185b9a700 1 -- 172.21.15.133:0/1283001236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe180101e30 msgr2=0x7fe18019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.161+0000 7fe185b9a700 1 --2- 172.21.15.133:0/1283001236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe180101e30 0x7fe18019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.161+0000 7fe185b9a700 1 -- 172.21.15.133:0/1283001236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1801002b0 msgr2=0x7fe1801981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.161+0000 7fe185b9a700 1 --2- 172.21.15.133:0/1283001236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1801002b0 0x7fe1801981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.161+0000 7fe185b9a700 1 -- 172.21.15.133:0/1283001236 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1700095f0 con 0x7fe1801014a0 2023-11-27T15:22:34.167 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.162+0000 7fe185b9a700 1 --2- 172.21.15.133:0/1283001236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1801014a0 0x7fe1801986f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe17000b620 tx=0x7fe170005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:34.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.162+0000 7fe1777fe700 1 -- 172.21.15.133:0/1283001236 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17001d070 con 0x7fe1801014a0 2023-11-27T15:22:34.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.162+0000 7fe1777fe700 1 -- 172.21.15.133:0/1283001236 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe170022470 con 0x7fe1801014a0 2023-11-27T15:22:34.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.162+0000 7fe1885ff700 1 -- 172.21.15.133:0/1283001236 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18010bc60 con 0x7fe1801014a0 2023-11-27T15:22:34.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.162+0000 7fe1777fe700 1 -- 172.21.15.133:0/1283001236 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe170017650 con 0x7fe1801014a0 2023-11-27T15:22:34.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.162+0000 7fe1885ff700 1 -- 172.21.15.133:0/1283001236 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18010bad0 con 0x7fe1801014a0 2023-11-27T15:22:34.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.163+0000 7fe1885ff700 1 -- 172.21.15.133:0/1283001236 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe180067a60 con 0x7fe1801014a0 2023-11-27T15:22:34.168 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.165+0000 7fe1777fe700 1 -- 172.21.15.133:0/1283001236 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe1700225e0 con 0x7fe1801014a0 2023-11-27T15:22:34.170 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.166+0000 7fe1777fe700 1 --2- 172.21.15.133:0/1283001236 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe16c0a11d0 0x7fe16c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.170 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.166+0000 7fe1777fe700 1 -- 172.21.15.133:0/1283001236 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe1700ce1c0 con 0x7fe1801014a0 2023-11-27T15:22:34.170 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.166+0000 7fe18639b700 1 --2- 172.21.15.133:0/1283001236 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe16c0a11d0 0x7fe16c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.170 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.167+0000 7fe18639b700 1 --2- 172.21.15.133:0/1283001236 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe16c0a11d0 0x7fe16c0a3680 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe180101300 tx=0x7fe17c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:34.173 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.170+0000 7fe1777fe700 1 -- 172.21.15.133:0/1283001236 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe1700975f0 con 0x7fe1801014a0 2023-11-27T15:22:34.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.370+0000 7fe1885ff700 1 -- 172.21.15.133:0/1283001236 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1801a0ae0 con 0x7fe1801014a0 2023-11-27T15:22:34.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.372+0000 7fe1777fe700 1 -- 172.21.15.133:0/1283001236 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24616 (secure 0 0 0) 0x7fe17009ac10 con 0x7fe1801014a0 2023-11-27T15:22:34.376 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:34.376 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":92,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:22:02.807005+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:22:34.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.374+0000 7fe1885ff700 1 -- 172.21.15.133:0/1283001236 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe16c0a11d0 msgr2=0x7fe16c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.374+0000 7fe1885ff700 1 --2- 172.21.15.133:0/1283001236 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe16c0a11d0 0x7fe16c0a3680 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe180101300 tx=0x7fe17c00b410 comp rx=0 tx=0).stop 2023-11-27T15:22:34.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.374+0000 7fe1885ff700 1 -- 172.21.15.133:0/1283001236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1801014a0 msgr2=0x7fe1801986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.374+0000 7fe1885ff700 1 --2- 172.21.15.133:0/1283001236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1801014a0 0x7fe1801986f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe17000b620 tx=0x7fe170005740 comp rx=0 tx=0).stop 2023-11-27T15:22:34.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.374+0000 7fe1885ff700 1 -- 172.21.15.133:0/1283001236 shutdown_connections 2023-11-27T15:22:34.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.374+0000 7fe1885ff700 1 --2- 172.21.15.133:0/1283001236 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe16c0a11d0 0x7fe16c0a3680 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.374+0000 7fe1885ff700 1 --2- 172.21.15.133:0/1283001236 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe180101e30 0x7fe18019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.374+0000 7fe1885ff700 1 --2- 172.21.15.133:0/1283001236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1801002b0 0x7fe1801981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.374+0000 7fe1885ff700 1 --2- 172.21.15.133:0/1283001236 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1801014a0 0x7fe1801986f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.375+0000 7fe1885ff700 1 -- 172.21.15.133:0/1283001236 >> 172.21.15.133:0/1283001236 conn(0x7fe1800fb890 msgr2=0x7fe1800fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:34.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.375+0000 7fe1885ff700 1 -- 172.21.15.133:0/1283001236 shutdown_connections 2023-11-27T15:22:34.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.375+0000 7fe1885ff700 1 -- 172.21.15.133:0/1283001236 wait complete. 2023-11-27T15:22:34.398 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-11-27T15:20:33.201023+0000', 'modified': '2023-11-27T15:21:04.794526+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24560, 'mds_1': 24521, 'mds_2': 24520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24491': {'gid': 24491, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6835/637171798', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 637171798}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 637171798}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24496': {'gid': 24496, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.177:6835/3620466749', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6834', 'nonce': 3620466749}, {'type': 'v1', 'addr': '172.21.15.177:6835', 'nonce': 3620466749}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24505': {'gid': 24505, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.177:6837/3492215689', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6836', 'nonce': 3492215689}, {'type': 'v1', 'addr': '172.21.15.177:6837', 'nonce': 3492215689}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24520': {'gid': 24520, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.177:6839/1463687598', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6838', 'nonce': 1463687598}, {'type': 'v1', 'addr': '172.21.15.177:6839', 'nonce': 1463687598}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24521': {'gid': 24521, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.174:6837/1955090548', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 1955090548}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 1955090548}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24560': {'gid': 24560, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.174:6839/2781994447', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 2781994447}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 2781994447}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-27T15:22:34.398 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-11-27T15:22:34.398 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-11-27T15:22:34.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.593+0000 7f7e183ff700 1 -- 172.21.15.133:0/2018373429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e10074b30 msgr2=0x7f7e10072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.593+0000 7f7e183ff700 1 --2- 172.21.15.133:0/2018373429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e10074b30 0x7f7e10072f80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009990 tx=0x7f7e0c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.593+0000 7f7e183ff700 1 -- 172.21.15.133:0/2018373429 shutdown_connections 2023-11-27T15:22:34.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.593+0000 7f7e183ff700 1 --2- 172.21.15.133:0/2018373429 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e10108100 0x7f7e1010a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.593+0000 7f7e183ff700 1 --2- 172.21.15.133:0/2018373429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e10074b30 0x7f7e10072f80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.593+0000 7f7e183ff700 1 --2- 172.21.15.133:0/2018373429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e10073550 0x7f7e100739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.593+0000 7f7e183ff700 1 -- 172.21.15.133:0/2018373429 >> 172.21.15.133:0/2018373429 conn(0x7f7e100fb820 msgr2=0x7f7e100fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:34.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.593+0000 7f7e183ff700 1 -- 172.21.15.133:0/2018373429 shutdown_connections 2023-11-27T15:22:34.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.593+0000 7f7e183ff700 1 -- 172.21.15.133:0/2018373429 wait complete. 2023-11-27T15:22:34.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.594+0000 7f7e183ff700 1 Processor -- start 2023-11-27T15:22:34.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.595+0000 7f7e183ff700 1 -- start start 2023-11-27T15:22:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.595+0000 7f7e183ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e10073550 0x7f7e101981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.595+0000 7f7e183ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e10074b30 0x7f7e10198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.595+0000 7f7e183ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e10108100 0x7f7e1019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.595+0000 7f7e183ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1019fd70 con 0x7f7e10074b30 2023-11-27T15:22:34.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.595+0000 7f7e183ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1019feb0 con 0x7f7e10108100 2023-11-27T15:22:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.595+0000 7f7e183ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e101a0040 con 0x7f7e10073550 2023-11-27T15:22:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.595+0000 7f7e1599a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e10074b30 0x7f7e10198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.595+0000 7f7e1599a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e10074b30 0x7f7e10198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45028/0 (socket says 172.21.15.133:45028) 2023-11-27T15:22:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.595+0000 7f7e1599a700 1 -- 172.21.15.133:0/3378820283 learned_addr learned my addr 172.21.15.133:0/3378820283 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.595+0000 7f7e1619b700 1 --2- 172.21.15.133:0/3378820283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e10073550 0x7f7e101981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.595+0000 7f7e1699c700 1 --2- 172.21.15.133:0/3378820283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e10108100 0x7f7e1019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.596+0000 7f7e1599a700 1 -- 172.21.15.133:0/3378820283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e10073550 msgr2=0x7f7e101981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.596+0000 7f7e1599a700 1 --2- 172.21.15.133:0/3378820283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e10073550 0x7f7e101981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.596+0000 7f7e1599a700 1 -- 172.21.15.133:0/3378820283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e10108100 msgr2=0x7f7e1019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.596+0000 7f7e1599a700 1 --2- 172.21.15.133:0/3378820283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e10108100 0x7f7e1019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.596+0000 7f7e1599a700 1 -- 172.21.15.133:0/3378820283 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0c0095f0 con 0x7f7e10074b30 2023-11-27T15:22:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.596+0000 7f7e1599a700 1 --2- 172.21.15.133:0/3378820283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e10074b30 0x7f7e10198730 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000d980 tx=0x7f7e0000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.596+0000 7f7e077fe700 1 -- 172.21.15.133:0/3378820283 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e00004020 con 0x7f7e10074b30 2023-11-27T15:22:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.596+0000 7f7e183ff700 1 -- 172.21.15.133:0/3378820283 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e1010de10 con 0x7f7e10074b30 2023-11-27T15:22:34.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.596+0000 7f7e077fe700 1 -- 172.21.15.133:0/3378820283 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e0000f690 con 0x7f7e10074b30 2023-11-27T15:22:34.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.596+0000 7f7e077fe700 1 -- 172.21.15.133:0/3378820283 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e00003a60 con 0x7f7e10074b30 2023-11-27T15:22:34.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.596+0000 7f7e183ff700 1 -- 172.21.15.133:0/3378820283 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1010dc80 con 0x7f7e10074b30 2023-11-27T15:22:34.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.599+0000 7f7e077fe700 1 -- 172.21.15.133:0/3378820283 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7e0000fb00 con 0x7f7e10074b30 2023-11-27T15:22:34.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.600+0000 7f7e2bdf3b80 1 Processor -- start 2023-11-27T15:22:34.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.601+0000 7f7e077fe700 1 --2- 172.21.15.133:0/3378820283 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7dfc0a1150 0x7f7dfc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.601+0000 7f7e2bdf3b80 1 -- start start 2023-11-27T15:22:34.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.601+0000 7f7e077fe700 1 -- 172.21.15.133:0/3378820283 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7e000cc850 con 0x7f7e10074b30 2023-11-27T15:22:34.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.601+0000 7f7e2bdf3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c6bb0 0x55f8972c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.601+0000 7f7e2bdf3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c7500 0x55f8972c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.601+0000 7f7e2bdf3b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c80c0 0x55f8972c8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.601+0000 7f7e1699c700 1 --2- 172.21.15.133:0/3378820283 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7dfc0a1150 0x7f7dfc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.601+0000 7f7e2bdf3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8971cc010 con 0x55f8972c7500 2023-11-27T15:22:34.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.601+0000 7f7e2bdf3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89721e850 con 0x55f8972c6bb0 2023-11-27T15:22:34.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.601+0000 7f7e2bdf3b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f897217ce0 con 0x55f8972c80c0 2023-11-27T15:22:34.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.601+0000 7f7e1619b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c6bb0 0x55f8972c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.601+0000 7f7e1619b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c6bb0 0x55f8972c6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:36850/0 (socket says 172.21.15.133:36850) 2023-11-27T15:22:34.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.601+0000 7f7e1619b700 1 -- 172.21.15.133:0/2344269665 learned_addr learned my addr 172.21.15.133:0/2344269665 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:34.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.601+0000 7f7e1599a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c7500 0x55f8972c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.601+0000 7f7e1699c700 1 --2- 172.21.15.133:0/2344269665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c80c0 0x55f8972c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.602+0000 7f7e1619b700 1 -- 172.21.15.133:0/2344269665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c80c0 msgr2=0x55f8972c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:34.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.602+0000 7f7e1619b700 1 --2- 172.21.15.133:0/2344269665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c80c0 0x55f8972c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.602+0000 7f7e1619b700 1 -- 172.21.15.133:0/2344269665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c7500 msgr2=0x55f8972c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:34.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.602+0000 7f7e1619b700 1 --2- 172.21.15.133:0/2344269665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c7500 0x55f8972c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.602+0000 7f7e1619b700 1 -- 172.21.15.133:0/2344269665 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8970995e0 con 0x55f8972c6bb0 2023-11-27T15:22:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.602+0000 7f7e1699c700 1 --2- 172.21.15.133:0/3378820283 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7dfc0a1150 0x7f7dfc0a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009ec0 tx=0x7f7e08013480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.602+0000 7f7e1619b700 1 --2- 172.21.15.133:0/2344269665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c6bb0 0x55f8972c6fc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009990 tx=0x7f7e0c005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9cea96ff6751cc0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.602+0000 7f7e04bfe700 1 -- 172.21.15.133:0/2344269665 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c01d070 con 0x55f8972c6bb0 2023-11-27T15:22:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.602+0000 7f7e04bfe700 1 -- 172.21.15.133:0/2344269665 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e0c01e040 con 0x55f8972c6bb0 2023-11-27T15:22:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.602+0000 7f7e04bfe700 1 -- 172.21.15.133:0/2344269665 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c0048c0 con 0x55f8972c6bb0 2023-11-27T15:22:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.603+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/2344269665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c6bb0 msgr2=0x55f8972c6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.603+0000 7f7e2bdf3b80 1 --2- 172.21.15.133:0/2344269665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c6bb0 0x55f8972c6fc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009990 tx=0x7f7e0c005bc0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.603+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/2344269665 shutdown_connections 2023-11-27T15:22:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.603+0000 7f7e2bdf3b80 1 --2- 172.21.15.133:0/2344269665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c80c0 0x55f8972c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.603+0000 7f7e2bdf3b80 1 --2- 172.21.15.133:0/2344269665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c6bb0 0x55f8972c6fc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.603+0000 7f7e2bdf3b80 1 --2- 172.21.15.133:0/2344269665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c7500 0x55f8972c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.603+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/2344269665 >> 172.21.15.133:0/2344269665 conn(0x55f8972bc310 msgr2=0x55f8972c1620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.603+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/2344269665 shutdown_connections 2023-11-27T15:22:34.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.603+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/2344269665 wait complete. 2023-11-27T15:22:34.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.604+0000 7f7e2bdf3b80 1 -- start start 2023-11-27T15:22:34.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.604+0000 7f7e2bdf3b80 1 Processor -- start 2023-11-27T15:22:34.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.604+0000 7f7e2bdf3b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c6bb0 0x55f8972d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.604+0000 7f7e2bdf3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c7500 0x55f8972d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.604+0000 7f7e2bdf3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c80c0 0x55f8972d7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.604+0000 7f7e2bdf3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8971b9890 con 0x55f8972c80c0 2023-11-27T15:22:34.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.604+0000 7f7e2bdf3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8971b9530 con 0x55f8972c7500 2023-11-27T15:22:34.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.604+0000 7f7e2bdf3b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89709b8d0 con 0x55f8972c6bb0 2023-11-27T15:22:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.605+0000 7f7e2bdf3b80 10 client.0 _ll_register_callbacks cb 0x55f8971d7730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:22:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.605+0000 7f7e1619b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c6bb0 0x55f8972d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.605+0000 7f7e1599a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c7500 0x55f8972d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.605+0000 7f7e1699c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c80c0 0x55f8972d7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.605+0000 7f7e1619b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c6bb0 0x55f8972d04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:50222/0 (socket says 172.21.15.133:50222) 2023-11-27T15:22:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.605+0000 7f7e1599a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c7500 0x55f8972d09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:36856/0 (socket says 172.21.15.133:36856) 2023-11-27T15:22:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.605+0000 7f7e1619b700 1 -- 172.21.15.133:0/1579852 learned_addr learned my addr 172.21.15.133:0/1579852 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.605+0000 7f7e1619b700 1 -- 172.21.15.133:0/1579852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c7500 msgr2=0x55f8972d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.605+0000 7f7e1619b700 1 --2- 172.21.15.133:0/1579852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c7500 0x55f8972d09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.605+0000 7f7e1619b700 1 -- 172.21.15.133:0/1579852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c80c0 msgr2=0x55f8972d7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.605+0000 7f7e1619b700 1 --2- 172.21.15.133:0/1579852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c80c0 0x55f8972d7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.605+0000 7f7e1619b700 1 -- 172.21.15.133:0/1579852 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1010b1a0 con 0x55f8972c6bb0 2023-11-27T15:22:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.605+0000 7f7e1619b700 1 --2- 172.21.15.133:0/1579852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c6bb0 0x55f8972d04a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009350 tx=0x7f7e0c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f360886cbfc3fc0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:34.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.605+0000 7f7df9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:22:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.606+0000 7f7df9ffb700 1 -- 172.21.15.133:0/1579852 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c01d040 con 0x55f8972c6bb0 2023-11-27T15:22:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.606+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/1579852 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8972b9ad0 con 0x55f8972c6bb0 2023-11-27T15:22:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.606+0000 7f7df9ffb700 1 -- 172.21.15.133:0/1579852 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7e0c00fe10 con 0x55f8972c6bb0 2023-11-27T15:22:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.606+0000 7f7df9ffb700 1 -- 172.21.15.133:0/1579852 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e0c005050 con 0x55f8972c6bb0 2023-11-27T15:22:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.606+0000 7f7e2bdf3b80 10 client.25288 fetch_fsmap learned FSMap version 23 2023-11-27T15:22:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.606+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/1579852 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f8971de140 con 0x55f8972c6bb0 2023-11-27T15:22:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.606+0000 7f7df9ffb700 1 -- 172.21.15.133:0/1579852 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f8972b9ad0 con 0x55f8972c6bb0 2023-11-27T15:22:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.606+0000 7f7df9ffb700 1 -- 172.21.15.133:0/1579852 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7e0c022400 con 0x55f8972c6bb0 2023-11-27T15:22:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.606+0000 7f7e2bdf3b80 10 client.25288 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:22:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.606+0000 7f7e2bdf3b80 10 client.25288 resolve_mds: resolved name 'i' to daemon mds.i 2023-11-27T15:22:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.606+0000 7f7e2bdf3b80 20 client.25288 populate_metadata read hostname 'smithi133' 2023-11-27T15:22:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.606+0000 7f7e2bdf3b80 1 --2- 172.21.15.133:0/1579852 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55f8972db290 0x55f8972da950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.606+0000 7f7e2bdf3b80 4 client.25288 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:22:34.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.606+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/1579852 --> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8971da050 con 0x55f8972db290 2023-11-27T15:22:34.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.607+0000 7f7e1599a700 1 --2- 172.21.15.133:0/1579852 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55f8972db290 0x55f8972da950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.607+0000 7f7e1599a700 1 --2- 172.21.15.133:0/1579852 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55f8972db290 0x55f8972da950 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=705a49472d9a3f03 server_cookie=78bd5f774e998323 in_seq=0 out_seq=0 2023-11-27T15:22:34.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.607+0000 7f7df9ffb700 10 client.25288 ms_handle_connect on v2:172.21.15.177:6838/1463687598 2023-11-27T15:22:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.608+0000 7f7df9ffb700 1 -- 172.21.15.133:0/1579852 <== mds.2 v2:172.21.15.177:6838/1463687598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f8971da050 con 0x55f8972db290 2023-11-27T15:22:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.608+0000 7f7df9ffb700 10 client.25288 handle_command_reply: tid=0 2023-11-27T15:22:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.608+0000 7f7e2bdf3b80 1 client.25288 shutdown 2023-11-27T15:22:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.608+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/1579852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c6bb0 msgr2=0x55f8972d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.608+0000 7f7e2bdf3b80 1 --2- 172.21.15.133:0/1579852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c6bb0 0x55f8972d04a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009350 tx=0x7f7e0c00f690 comp rx=0 tx=0).stop 2023-11-27T15:22:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.608+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/1579852 shutdown_connections 2023-11-27T15:22:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.608+0000 7f7e2bdf3b80 1 --2- 172.21.15.133:0/1579852 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55f8972db290 0x55f8972da950 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.608+0000 7f7e2bdf3b80 1 --2- 172.21.15.133:0/1579852 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c80c0 0x55f8972d7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.608+0000 7f7e2bdf3b80 1 --2- 172.21.15.133:0/1579852 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c6bb0 0x55f8972d04a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.608+0000 7f7df9ffb700 0 client.25288 ms_handle_reset on v2:172.21.15.177:6838/1463687598 2023-11-27T15:22:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.608+0000 7f7e2bdf3b80 1 --2- 172.21.15.133:0/1579852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c7500 0x55f8972d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.608+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/1579852 >> 172.21.15.133:0/1579852 conn(0x55f8972bc310 msgr2=0x55f8972c0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.608+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/1579852 shutdown_connections 2023-11-27T15:22:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.609+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/1579852 wait complete. 2023-11-27T15:22:34.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.609+0000 7f7e2bdf3b80 20 client.25288 trim_cache size 0 max 16384 2023-11-27T15:22:34.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.626+0000 7f7e2bdf3b80 1 Processor -- start 2023-11-27T15:22:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.627+0000 7f7e2bdf3b80 1 -- start start 2023-11-27T15:22:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.627+0000 7f7e2bdf3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c6bb0 0x55f8972e4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.627+0000 7f7e2bdf3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c7500 0x55f8972d7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.627+0000 7f7e2bdf3b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c80c0 0x55f8972da950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.627+0000 7f7e2bdf3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8972d0070 con 0x55f8972c6bb0 2023-11-27T15:22:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.627+0000 7f7e2bdf3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8971c1730 con 0x55f8972c7500 2023-11-27T15:22:34.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.627+0000 7f7e2bdf3b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8971b3e10 con 0x55f8972c80c0 2023-11-27T15:22:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.627+0000 7f7e1619b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c6bb0 0x55f8972e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.627+0000 7f7e1619b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c6bb0 0x55f8972e4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45052/0 (socket says 172.21.15.133:45052) 2023-11-27T15:22:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.627+0000 7f7e1619b700 1 -- 172.21.15.133:0/146608335 learned_addr learned my addr 172.21.15.133:0/146608335 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.627+0000 7f7e1599a700 1 --2- 172.21.15.133:0/146608335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c7500 0x55f8972d7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.627+0000 7f7e1699c700 1 --2- 172.21.15.133:0/146608335 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c80c0 0x55f8972da950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.628+0000 7f7e1619b700 1 -- 172.21.15.133:0/146608335 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c80c0 msgr2=0x55f8972da950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.628+0000 7f7e1619b700 1 --2- 172.21.15.133:0/146608335 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c80c0 0x55f8972da950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.628+0000 7f7e1619b700 1 -- 172.21.15.133:0/146608335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c7500 msgr2=0x55f8972d7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.628+0000 7f7e1619b700 1 --2- 172.21.15.133:0/146608335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c7500 0x55f8972d7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.628+0000 7f7e1619b700 1 -- 172.21.15.133:0/146608335 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8971dde90 con 0x55f8972c6bb0 2023-11-27T15:22:34.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.628+0000 7f7e1599a700 1 --2- 172.21.15.133:0/146608335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c7500 0x55f8972d7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.628+0000 7f7e1619b700 1 --2- 172.21.15.133:0/146608335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c6bb0 0x55f8972e4000 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0095c0 tx=0x7f7e0c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9e06ff21464aa2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.628+0000 7f7e053ff700 1 -- 172.21.15.133:0/146608335 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c022600 con 0x55f8972c6bb0 2023-11-27T15:22:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.628+0000 7f7e053ff700 1 -- 172.21.15.133:0/146608335 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e0c00f700 con 0x55f8972c6bb0 2023-11-27T15:22:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.628+0000 7f7e053ff700 1 -- 172.21.15.133:0/146608335 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c00e640 con 0x55f8972c6bb0 2023-11-27T15:22:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.629+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/146608335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c6bb0 msgr2=0x55f8972e4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.629+0000 7f7e2bdf3b80 1 --2- 172.21.15.133:0/146608335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c6bb0 0x55f8972e4000 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c0095c0 tx=0x7f7e0c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.629+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/146608335 shutdown_connections 2023-11-27T15:22:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.629+0000 7f7e2bdf3b80 1 --2- 172.21.15.133:0/146608335 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c80c0 0x55f8972da950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.629+0000 7f7e2bdf3b80 1 --2- 172.21.15.133:0/146608335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c6bb0 0x55f8972e4000 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.629+0000 7f7e2bdf3b80 1 --2- 172.21.15.133:0/146608335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c7500 0x55f8972d7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.629+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/146608335 >> 172.21.15.133:0/146608335 conn(0x55f8972bc310 msgr2=0x55f8972cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.629+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/146608335 shutdown_connections 2023-11-27T15:22:34.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.629+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/146608335 wait complete. 2023-11-27T15:22:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.629+0000 7f7e2bdf3b80 1 -- start start 2023-11-27T15:22:34.635 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.629+0000 7f7e2bdf3b80 1 Processor -- start 2023-11-27T15:22:34.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.630+0000 7f7e2bdf3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c6bb0 0x55f8972d2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.630+0000 7f7e2bdf3b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c7500 0x55f8972d77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.630+0000 7f7e2bdf3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c80c0 0x55f8972d2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.630+0000 7f7e2bdf3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8972d2cd0 con 0x55f8972c80c0 2023-11-27T15:22:34.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.630+0000 7f7e2bdf3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8972d2e40 con 0x55f8972c6bb0 2023-11-27T15:22:34.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.630+0000 7f7e2bdf3b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89732f130 con 0x55f8972c7500 2023-11-27T15:22:34.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.630+0000 7f7e2bdf3b80 10 client.0 _ll_register_callbacks cb 0x7f7e00004750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:22:34.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.630+0000 7f7e1699c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c80c0 0x55f8972d2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.630+0000 7f7e1599a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c7500 0x55f8972d77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.630+0000 7f7e1619b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c6bb0 0x55f8972d2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.630+0000 7f7e1699c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c80c0 0x55f8972d2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45062/0 (socket says 172.21.15.133:45062) 2023-11-27T15:22:34.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.630+0000 7f7e1699c700 1 -- 172.21.15.133:0/3587415324 learned_addr learned my addr 172.21.15.133:0/3587415324 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:34.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.631+0000 7f7e1619b700 1 -- 172.21.15.133:0/3587415324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c7500 msgr2=0x55f8972d77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:34.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.631+0000 7f7e1619b700 1 --2- 172.21.15.133:0/3587415324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c7500 0x55f8972d77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.631+0000 7f7e1619b700 1 -- 172.21.15.133:0/3587415324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c80c0 msgr2=0x55f8972d2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:34.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.631+0000 7f7e1619b700 1 --2- 172.21.15.133:0/3587415324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c80c0 0x55f8972d2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.631+0000 7f7e1619b700 1 -- 172.21.15.133:0/3587415324 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e00084040 con 0x55f8972c6bb0 2023-11-27T15:22:34.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.631+0000 7f7e1599a700 1 --2- 172.21.15.133:0/3587415324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c7500 0x55f8972d77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:34.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.631+0000 7f7e1619b700 1 --2- 172.21.15.133:0/3587415324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c6bb0 0x55f8972d2250 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55f8972b9b20 tx=0x7f7e0c00bb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e201b6fe35269962 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:34.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.631+0000 7f7df9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:22:34.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.631+0000 7f7df9ffb700 1 -- 172.21.15.133:0/3587415324 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c01d070 con 0x55f8972c6bb0 2023-11-27T15:22:34.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.631+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/3587415324 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8971e6610 con 0x55f8972c6bb0 2023-11-27T15:22:34.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.631+0000 7f7df9ffb700 1 -- 172.21.15.133:0/3587415324 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7e0c000e40 con 0x55f8972c6bb0 2023-11-27T15:22:34.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.631+0000 7f7df9ffb700 1 -- 172.21.15.133:0/3587415324 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e0c022610 con 0x55f8972c6bb0 2023-11-27T15:22:34.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.631+0000 7f7e2bdf3b80 10 client.25307 fetch_fsmap learned FSMap version 23 2023-11-27T15:22:34.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.631+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/3587415324 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f8971de140 con 0x55f8972c6bb0 2023-11-27T15:22:34.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.632+0000 7f7df9ffb700 1 -- 172.21.15.133:0/3587415324 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f8971e6610 con 0x55f8972c6bb0 2023-11-27T15:22:34.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.632+0000 7f7df9ffb700 1 -- 172.21.15.133:0/3587415324 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7e0c004500 con 0x55f8972c6bb0 2023-11-27T15:22:34.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.632+0000 7f7e2bdf3b80 10 client.25307 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:22:34.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.632+0000 7f7e2bdf3b80 10 client.25307 resolve_mds: resolved name 'i' to daemon mds.i 2023-11-27T15:22:34.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.632+0000 7f7e2bdf3b80 20 client.25307 populate_metadata read hostname 'smithi133' 2023-11-27T15:22:34.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.632+0000 7f7e2bdf3b80 1 --2- 172.21.15.133:0/3587415324 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55f897333960 0x55f897330330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.632+0000 7f7e2bdf3b80 4 client.25307 mds_command: new command op to 24520 tid=0[{"prefix": "status"}] 2023-11-27T15:22:34.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.632+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/3587415324 --> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f897066f20 con 0x55f897333960 2023-11-27T15:22:34.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.632+0000 7f7e1599a700 1 --2- 172.21.15.133:0/3587415324 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55f897333960 0x55f897330330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.633+0000 7f7e1599a700 1 --2- 172.21.15.133:0/3587415324 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55f897333960 0x55f897330330 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c92c4ad7511d05fd server_cookie=701d2fcc57e731dc in_seq=0 out_seq=0 2023-11-27T15:22:34.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.633+0000 7f7df9ffb700 10 client.25307 ms_handle_connect on v2:172.21.15.177:6838/1463687598 2023-11-27T15:22:34.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.633+0000 7f7df9ffb700 1 -- 172.21.15.133:0/3587415324 <== mds.2 v2:172.21.15.177:6838/1463687598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55f897066f20 con 0x55f897333960 2023-11-27T15:22:34.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.633+0000 7f7df9ffb700 10 client.25307 handle_command_reply: tid=0 2023-11-27T15:22:34.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.633+0000 7f7e2bdf3b80 1 client.25307 shutdown 2023-11-27T15:22:34.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.633+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/3587415324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c6bb0 msgr2=0x55f8972d2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.633+0000 7f7e2bdf3b80 1 --2- 172.21.15.133:0/3587415324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c6bb0 0x55f8972d2250 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55f8972b9b20 tx=0x7f7e0c00bb30 comp rx=0 tx=0).stop 2023-11-27T15:22:34.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.633+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/3587415324 shutdown_connections 2023-11-27T15:22:34.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.633+0000 7f7e2bdf3b80 1 --2- 172.21.15.133:0/3587415324 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55f897333960 0x55f897330330 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.633+0000 7f7e2bdf3b80 1 --2- 172.21.15.133:0/3587415324 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8972c80c0 0x55f8972d2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.633+0000 7f7df9ffb700 0 client.25307 ms_handle_reset on v2:172.21.15.177:6838/1463687598 2023-11-27T15:22:34.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.633+0000 7f7e2bdf3b80 1 --2- 172.21.15.133:0/3587415324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8972c6bb0 0x55f8972d2250 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.633+0000 7f7e2bdf3b80 1 --2- 172.21.15.133:0/3587415324 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8972c7500 0x55f8972d77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.633+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/3587415324 >> 172.21.15.133:0/3587415324 conn(0x55f8972bc310 msgr2=0x55f8972e4000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:34.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.633+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/3587415324 shutdown_connections 2023-11-27T15:22:34.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.633+0000 7f7e2bdf3b80 1 -- 172.21.15.133:0/3587415324 wait complete. 2023-11-27T15:22:34.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.633+0000 7f7e2bdf3b80 20 client.25307 trim_cache size 0 max 16384 2023-11-27T15:22:34.642 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:22:34.642 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "5227da4c-8d37-11ee-95a2-87774f69a715", 2023-11-27T15:22:34.642 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 2, 2023-11-27T15:22:34.642 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24520, 2023-11-27T15:22:34.642 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2023-11-27T15:22:34.642 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2023-11-27T15:22:34.642 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2023-11-27T15:22:34.642 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 90.842657641000002, 2023-11-27T15:22:34.642 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 23, 2023-11-27T15:22:34.642 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 92, 2023-11-27T15:22:34.643 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 90, 2023-11-27T15:22:34.643 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 136.53510968099999 2023-11-27T15:22:34.643 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:22:34.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.634+0000 7f7e183ff700 1 -- 172.21.15.133:0/3378820283 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7dfc0a1150 msgr2=0x7f7dfc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.634+0000 7f7e183ff700 1 --2- 172.21.15.133:0/3378820283 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7dfc0a1150 0x7f7dfc0a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7e08009ec0 tx=0x7f7e08013480 comp rx=0 tx=0).stop 2023-11-27T15:22:34.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.634+0000 7f7e183ff700 1 -- 172.21.15.133:0/3378820283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e10074b30 msgr2=0x7f7e10198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.634+0000 7f7e183ff700 1 --2- 172.21.15.133:0/3378820283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e10074b30 0x7f7e10198730 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7e0000d980 tx=0x7f7e0000dc90 comp rx=0 tx=0).stop 2023-11-27T15:22:34.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.634+0000 7f7e183ff700 1 -- 172.21.15.133:0/3378820283 shutdown_connections 2023-11-27T15:22:34.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.634+0000 7f7e183ff700 1 --2- 172.21.15.133:0/3378820283 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7dfc0a1150 0x7f7dfc0a3600 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.634+0000 7f7e183ff700 1 --2- 172.21.15.133:0/3378820283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e10108100 0x7f7e1019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.634+0000 7f7e183ff700 1 --2- 172.21.15.133:0/3378820283 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7e10073550 0x7f7e101981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.634+0000 7f7e183ff700 1 --2- 172.21.15.133:0/3378820283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e10074b30 0x7f7e10198730 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.634+0000 7f7e183ff700 1 -- 172.21.15.133:0/3378820283 >> 172.21.15.133:0/3378820283 conn(0x7f7e100fb820 msgr2=0x7f7e100fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:34.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.634+0000 7f7e183ff700 1 -- 172.21.15.133:0/3378820283 shutdown_connections 2023-11-27T15:22:34.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.634+0000 7f7e183ff700 1 -- 172.21.15.133:0/3378820283 wait complete. 2023-11-27T15:22:34.656 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-11-27T15:22:34.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.853+0000 7fae6a3ff700 1 -- 172.21.15.133:0/4078940321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae64101430 msgr2=0x7fae64101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.853+0000 7fae6a3ff700 1 --2- 172.21.15.133:0/4078940321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae64101430 0x7fae64101880 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fae58009990 tx=0x7fae58009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.857 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.853+0000 7fae6a3ff700 1 -- 172.21.15.133:0/4078940321 shutdown_connections 2023-11-27T15:22:34.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.853+0000 7fae6a3ff700 1 --2- 172.21.15.133:0/4078940321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae64101dc0 0x7fae64108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.853+0000 7fae6a3ff700 1 --2- 172.21.15.133:0/4078940321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae64100240 0x7fae64100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.853+0000 7fae6a3ff700 1 --2- 172.21.15.133:0/4078940321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae64101430 0x7fae64101880 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.853+0000 7fae6a3ff700 1 -- 172.21.15.133:0/4078940321 >> 172.21.15.133:0/4078940321 conn(0x7fae640fb820 msgr2=0x7fae640fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:34.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.853+0000 7fae6a3ff700 1 -- 172.21.15.133:0/4078940321 shutdown_connections 2023-11-27T15:22:34.858 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.853+0000 7fae6a3ff700 1 -- 172.21.15.133:0/4078940321 wait complete. 2023-11-27T15:22:34.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.854+0000 7fae6a3ff700 1 Processor -- start 2023-11-27T15:22:34.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.854+0000 7fae6a3ff700 1 -- start start 2023-11-27T15:22:34.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.855+0000 7fae6a3ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae64100240 0x7fae640748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.855+0000 7fae6a3ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae64101430 0x7fae64072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.855+0000 7fae6a3ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae64101dc0 0x7fae64073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.855+0000 7fae6a3ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae64073bc0 con 0x7fae64101dc0 2023-11-27T15:22:34.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.855+0000 7fae6a3ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae64073d30 con 0x7fae64100240 2023-11-27T15:22:34.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.855+0000 7fae6a3ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6419ffb0 con 0x7fae64101430 2023-11-27T15:22:34.860 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.855+0000 7fae637fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae64101430 0x7fae64072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.855+0000 7fae6899c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae64101dc0 0x7fae64073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.855+0000 7fae637fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae64101430 0x7fae64072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:50240/0 (socket says 172.21.15.133:50240) 2023-11-27T15:22:34.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.855+0000 7fae637fe700 1 -- 172.21.15.133:0/3562473663 learned_addr learned my addr 172.21.15.133:0/3562473663 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:34.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.855+0000 7fae63fff700 1 --2- 172.21.15.133:0/3562473663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae64100240 0x7fae640748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.856+0000 7fae6899c700 1 -- 172.21.15.133:0/3562473663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae64101430 msgr2=0x7fae64072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.856+0000 7fae6899c700 1 --2- 172.21.15.133:0/3562473663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae64101430 0x7fae64072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.856+0000 7fae6899c700 1 -- 172.21.15.133:0/3562473663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae64100240 msgr2=0x7fae640748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.856+0000 7fae6899c700 1 --2- 172.21.15.133:0/3562473663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae64100240 0x7fae640748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.856+0000 7fae6899c700 1 -- 172.21.15.133:0/3562473663 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae580095f0 con 0x7fae64101dc0 2023-11-27T15:22:34.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.856+0000 7fae63fff700 1 --2- 172.21.15.133:0/3562473663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae64100240 0x7fae640748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:34.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.856+0000 7fae637fe700 1 --2- 172.21.15.133:0/3562473663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae64101430 0x7fae64072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:34.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.856+0000 7fae6899c700 1 --2- 172.21.15.133:0/3562473663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae64101dc0 0x7fae64073420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00efc0 tx=0x7fae5c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:34.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.856+0000 7fae617fa700 1 -- 172.21.15.133:0/3562473663 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c018070 con 0x7fae64101dc0 2023-11-27T15:22:34.861 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.856+0000 7fae617fa700 1 -- 172.21.15.133:0/3562473663 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae5c00f040 con 0x7fae64101dc0 2023-11-27T15:22:34.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.856+0000 7fae6a3ff700 1 -- 172.21.15.133:0/3562473663 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae6410bb90 con 0x7fae64101dc0 2023-11-27T15:22:34.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.856+0000 7fae617fa700 1 -- 172.21.15.133:0/3562473663 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5c00cd40 con 0x7fae64101dc0 2023-11-27T15:22:34.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.856+0000 7fae6a3ff700 1 -- 172.21.15.133:0/3562473663 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae6410ba00 con 0x7fae64101dc0 2023-11-27T15:22:34.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.859+0000 7fae617fa700 1 -- 172.21.15.133:0/3562473663 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fae5c003a00 con 0x7fae64101dc0 2023-11-27T15:22:34.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.861+0000 7fae617fa700 1 --2- 172.21.15.133:0/3562473663 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fae4c0a1150 0x7fae4c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.861+0000 7fae7dcdfb80 1 Processor -- start 2023-11-27T15:22:34.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.861+0000 7fae617fa700 1 -- 172.21.15.133:0/3562473663 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fae5c0cc880 con 0x7fae64101dc0 2023-11-27T15:22:34.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.861+0000 7fae7dcdfb80 1 -- start start 2023-11-27T15:22:34.865 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.861+0000 7fae63fff700 1 --2- 172.21.15.133:0/3562473663 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fae4c0a1150 0x7fae4c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.861+0000 7fae7dcdfb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560dbfc35bb0 0x560dbfc35fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.861+0000 7fae7dcdfb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc36500 0x560dbfc36ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.861+0000 7fae7dcdfb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc370c0 0x560dbfc37530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.861+0000 7fae7dcdfb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbfb3b010 con 0x560dbfc36500 2023-11-27T15:22:34.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.861+0000 7fae7dcdfb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbfb8d850 con 0x560dbfc370c0 2023-11-27T15:22:34.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.861+0000 7fae7dcdfb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbfb86ce0 con 0x560dbfc35bb0 2023-11-27T15:22:34.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.862+0000 7fae63fff700 1 --2- 172.21.15.133:0/3562473663 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fae4c0a1150 0x7fae4c0a3600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fae64101290 tx=0x7fae54009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:34.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.862+0000 7fae637fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc36500 0x560dbfc36ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.862+0000 7fae637fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc36500 0x560dbfc36ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45106/0 (socket says 172.21.15.133:45106) 2023-11-27T15:22:34.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.862+0000 7fae637fe700 1 -- 172.21.15.133:0/1984999761 learned_addr learned my addr 172.21.15.133:0/1984999761 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:34.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.862+0000 7fae63fff700 1 --2- 172.21.15.133:0/1984999761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560dbfc35bb0 0x560dbfc35fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.867 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.862+0000 7fae6899c700 1 --2- 172.21.15.133:0/1984999761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc370c0 0x560dbfc37530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.862+0000 7fae637fe700 1 -- 172.21.15.133:0/1984999761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560dbfc35bb0 msgr2=0x560dbfc35fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:34.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.862+0000 7fae637fe700 1 --2- 172.21.15.133:0/1984999761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560dbfc35bb0 0x560dbfc35fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.862+0000 7fae637fe700 1 -- 172.21.15.133:0/1984999761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc370c0 msgr2=0x560dbfc37530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:34.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.862+0000 7fae637fe700 1 --2- 172.21.15.133:0/1984999761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc370c0 0x560dbfc37530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.862+0000 7fae637fe700 1 -- 172.21.15.133:0/1984999761 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dbfa085e0 con 0x560dbfc36500 2023-11-27T15:22:34.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.862+0000 7fae637fe700 1 --2- 172.21.15.133:0/1984999761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc36500 0x560dbfc36ae0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fae58005f30 tx=0x7fae58005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23864e847f01278a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:34.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.863+0000 7fae4a7fc700 1 -- 172.21.15.133:0/1984999761 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5801d070 con 0x560dbfc36500 2023-11-27T15:22:34.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.863+0000 7fae4a7fc700 1 -- 172.21.15.133:0/1984999761 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae5801e040 con 0x560dbfc36500 2023-11-27T15:22:34.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.863+0000 7fae4a7fc700 1 -- 172.21.15.133:0/1984999761 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58004ea0 con 0x560dbfc36500 2023-11-27T15:22:34.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.863+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/1984999761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc36500 msgr2=0x560dbfc36ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.868 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.863+0000 7fae7dcdfb80 1 --2- 172.21.15.133:0/1984999761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc36500 0x560dbfc36ae0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fae58005f30 tx=0x7fae58005c00 comp rx=0 tx=0).stop 2023-11-27T15:22:34.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.863+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/1984999761 shutdown_connections 2023-11-27T15:22:34.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.863+0000 7fae7dcdfb80 1 --2- 172.21.15.133:0/1984999761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc370c0 0x560dbfc37530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.863+0000 7fae7dcdfb80 1 --2- 172.21.15.133:0/1984999761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560dbfc35bb0 0x560dbfc35fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.863+0000 7fae7dcdfb80 1 --2- 172.21.15.133:0/1984999761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc36500 0x560dbfc36ae0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.863+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/1984999761 >> 172.21.15.133:0/1984999761 conn(0x560dbfc2b310 msgr2=0x560dbfc30620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:34.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.863+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/1984999761 shutdown_connections 2023-11-27T15:22:34.871 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.864+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/1984999761 wait complete. 2023-11-27T15:22:34.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.864+0000 7fae7dcdfb80 1 -- start start 2023-11-27T15:22:34.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.864+0000 7fae7dcdfb80 1 Processor -- start 2023-11-27T15:22:34.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.865+0000 7fae7dcdfb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc35bb0 0x560dbfc3f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.865+0000 7fae7dcdfb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc36500 0x560dbfc3fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.865+0000 7fae7dcdfb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560dbfc370c0 0x560dbfc46a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.865+0000 7fae7dcdfb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbfb28890 con 0x560dbfc36500 2023-11-27T15:22:34.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.865+0000 7fae7dcdfb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbfb28530 con 0x560dbfc35bb0 2023-11-27T15:22:34.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.865+0000 7fae7dcdfb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbfa0a8d0 con 0x560dbfc370c0 2023-11-27T15:22:34.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.865+0000 7fae7dcdfb80 10 client.0 _ll_register_callbacks cb 0x560dbfb46730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:22:34.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.865+0000 7fae637fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc36500 0x560dbfc3fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.865+0000 7fae637fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc36500 0x560dbfc3fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45116/0 (socket says 172.21.15.133:45116) 2023-11-27T15:22:34.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.865+0000 7fae63fff700 1 --2- 172.21.15.133:0/1456491490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc35bb0 0x560dbfc3f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.865+0000 7fae637fe700 1 -- 172.21.15.133:0/1456491490 learned_addr learned my addr 172.21.15.133:0/1456491490 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:34.872 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.865+0000 7fae6899c700 1 --2- 172.21.15.133:0/1456491490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560dbfc370c0 0x560dbfc46a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.865+0000 7fae63fff700 1 -- 172.21.15.133:0/1456491490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560dbfc370c0 msgr2=0x560dbfc46a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:34.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.865+0000 7fae63fff700 1 --2- 172.21.15.133:0/1456491490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560dbfc370c0 0x560dbfc46a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.865+0000 7fae63fff700 1 -- 172.21.15.133:0/1456491490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc36500 msgr2=0x560dbfc3fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:34.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.865+0000 7fae63fff700 1 --2- 172.21.15.133:0/1456491490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc36500 0x560dbfc3fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.865+0000 7fae63fff700 1 -- 172.21.15.133:0/1456491490 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae64108fb0 con 0x560dbfc35bb0 2023-11-27T15:22:34.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.865+0000 7fae637fe700 1 --2- 172.21.15.133:0/1456491490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc36500 0x560dbfc3fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:34.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.866+0000 7fae63fff700 1 --2- 172.21.15.133:0/1456491490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc35bb0 0x560dbfc3f4f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fae540039b0 tx=0x7fae540047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58f85892cd1e355a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:34.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.866+0000 7fae437fe700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:22:34.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.866+0000 7fae437fe700 1 -- 172.21.15.133:0/1456491490 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5401fa80 con 0x560dbfc35bb0 2023-11-27T15:22:34.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.866+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/1456491490 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560dbfc28ad0 con 0x560dbfc35bb0 2023-11-27T15:22:34.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.866+0000 7fae437fe700 1 -- 172.21.15.133:0/1456491490 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fae5401fbe0 con 0x560dbfc35bb0 2023-11-27T15:22:34.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.866+0000 7fae437fe700 1 -- 172.21.15.133:0/1456491490 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae5401fda0 con 0x560dbfc35bb0 2023-11-27T15:22:34.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.866+0000 7fae7dcdfb80 10 client.25319 fetch_fsmap learned FSMap version 23 2023-11-27T15:22:34.873 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.866+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/1456491490 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560dbfb4d140 con 0x560dbfc35bb0 2023-11-27T15:22:34.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.867+0000 7fae437fe700 1 -- 172.21.15.133:0/1456491490 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560dbfc28ad0 con 0x560dbfc35bb0 2023-11-27T15:22:34.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.867+0000 7fae437fe700 1 -- 172.21.15.133:0/1456491490 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fae54005430 con 0x560dbfc35bb0 2023-11-27T15:22:34.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.867+0000 7fae7dcdfb80 10 client.25319 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:22:34.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.867+0000 7fae7dcdfb80 10 client.25319 resolve_mds: resolved name 'h' to daemon mds.h 2023-11-27T15:22:34.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.867+0000 7fae7dcdfb80 20 client.25319 populate_metadata read hostname 'smithi133' 2023-11-27T15:22:34.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.867+0000 7fae7dcdfb80 1 --2- 172.21.15.133:0/1456491490 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x560dbfc4a820 0x560dbfc49bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.867+0000 7fae7dcdfb80 4 client.25319 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:22:34.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.867+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/1456491490 --> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560dbfb49050 con 0x560dbfc4a820 2023-11-27T15:22:34.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.867+0000 7fae637fe700 1 --2- 172.21.15.133:0/1456491490 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x560dbfc4a820 0x560dbfc49bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.868+0000 7fae437fe700 10 client.25319 ms_handle_connect on v2:172.21.15.174:6836/1955090548 2023-11-27T15:22:34.874 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.868+0000 7fae637fe700 1 --2- 172.21.15.133:0/1456491490 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x560dbfc4a820 0x560dbfc49bd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=98b34f7865820a82 server_cookie=2acddef0b8983239 in_seq=0 out_seq=0 2023-11-27T15:22:34.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.868+0000 7fae437fe700 1 -- 172.21.15.133:0/1456491490 <== mds.1 v2:172.21.15.174:6836/1955090548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560dbfb49050 con 0x560dbfc4a820 2023-11-27T15:22:34.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.868+0000 7fae437fe700 10 client.25319 handle_command_reply: tid=0 2023-11-27T15:22:34.875 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.868+0000 7fae7dcdfb80 1 client.25319 shutdown 2023-11-27T15:22:34.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.869+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/1456491490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc35bb0 msgr2=0x560dbfc3f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.869+0000 7fae7dcdfb80 1 --2- 172.21.15.133:0/1456491490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc35bb0 0x560dbfc3f4f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fae540039b0 tx=0x7fae540047c0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.869+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/1456491490 shutdown_connections 2023-11-27T15:22:34.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.869+0000 7fae7dcdfb80 1 --2- 172.21.15.133:0/1456491490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560dbfc370c0 0x560dbfc46a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.869+0000 7fae7dcdfb80 1 --2- 172.21.15.133:0/1456491490 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x560dbfc4a820 0x560dbfc49bd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.869+0000 7fae7dcdfb80 1 --2- 172.21.15.133:0/1456491490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc35bb0 0x560dbfc3f4f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.869+0000 7fae7dcdfb80 1 --2- 172.21.15.133:0/1456491490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc36500 0x560dbfc3fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.869+0000 7fae437fe700 0 client.25319 ms_handle_reset on v2:172.21.15.174:6836/1955090548 2023-11-27T15:22:34.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.869+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/1456491490 >> 172.21.15.133:0/1456491490 conn(0x560dbfc2b310 msgr2=0x560dbfc2fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:34.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.869+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/1456491490 shutdown_connections 2023-11-27T15:22:34.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.869+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/1456491490 wait complete. 2023-11-27T15:22:34.876 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.869+0000 7fae7dcdfb80 20 client.25319 trim_cache size 0 max 16384 2023-11-27T15:22:34.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.887+0000 7fae7dcdfb80 1 Processor -- start 2023-11-27T15:22:34.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.887+0000 7fae7dcdfb80 1 -- start start 2023-11-27T15:22:34.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.888+0000 7fae7dcdfb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc35bb0 0x560dbfc535b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.888+0000 7fae7dcdfb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc36500 0x560dbfc46a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.888+0000 7fae7dcdfb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560dbfc370c0 0x560dbfc49bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.888+0000 7fae7dcdfb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbfc3f0c0 con 0x560dbfc35bb0 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.888+0000 7fae7dcdfb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbfb44fa0 con 0x560dbfc36500 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.888+0000 7fae7dcdfb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbfc31110 con 0x560dbfc370c0 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.888+0000 7fae63fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc35bb0 0x560dbfc535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.888+0000 7fae63fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc35bb0 0x560dbfc535b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45128/0 (socket says 172.21.15.133:45128) 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.888+0000 7fae63fff700 1 -- 172.21.15.133:0/4258229809 learned_addr learned my addr 172.21.15.133:0/4258229809 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.888+0000 7fae637fe700 1 --2- 172.21.15.133:0/4258229809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc36500 0x560dbfc46a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.888+0000 7fae6899c700 1 --2- 172.21.15.133:0/4258229809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560dbfc370c0 0x560dbfc49bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.888+0000 7fae63fff700 1 -- 172.21.15.133:0/4258229809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560dbfc370c0 msgr2=0x560dbfc49bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.888+0000 7fae63fff700 1 --2- 172.21.15.133:0/4258229809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560dbfc370c0 0x560dbfc49bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.888+0000 7fae63fff700 1 -- 172.21.15.133:0/4258229809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc36500 msgr2=0x560dbfc46a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.888+0000 7fae63fff700 1 --2- 172.21.15.133:0/4258229809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc36500 0x560dbfc46a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.888+0000 7fae63fff700 1 -- 172.21.15.133:0/4258229809 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dbfb4ce90 con 0x560dbfc35bb0 2023-11-27T15:22:34.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.888+0000 7fae6899c700 1 --2- 172.21.15.133:0/4258229809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560dbfc370c0 0x560dbfc49bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:22:34.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.889+0000 7fae63fff700 1 --2- 172.21.15.133:0/4258229809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc35bb0 0x560dbfc535b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fae54011c30 tx=0x7fae54004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06d2d775dd713cc server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:34.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.889+0000 7fae4affd700 1 -- 172.21.15.133:0/4258229809 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5401f040 con 0x560dbfc35bb0 2023-11-27T15:22:34.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.889+0000 7fae4affd700 1 -- 172.21.15.133:0/4258229809 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae5401fd40 con 0x560dbfc35bb0 2023-11-27T15:22:34.894 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.889+0000 7fae4affd700 1 -- 172.21.15.133:0/4258229809 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54027a90 con 0x560dbfc35bb0 2023-11-27T15:22:34.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.889+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/4258229809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc35bb0 msgr2=0x560dbfc535b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.889+0000 7fae7dcdfb80 1 --2- 172.21.15.133:0/4258229809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc35bb0 0x560dbfc535b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fae54011c30 tx=0x7fae54004130 comp rx=0 tx=0).stop 2023-11-27T15:22:34.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.889+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/4258229809 shutdown_connections 2023-11-27T15:22:34.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.889+0000 7fae7dcdfb80 1 --2- 172.21.15.133:0/4258229809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560dbfc370c0 0x560dbfc49bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.889+0000 7fae7dcdfb80 1 --2- 172.21.15.133:0/4258229809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc35bb0 0x560dbfc535b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.889+0000 7fae7dcdfb80 1 --2- 172.21.15.133:0/4258229809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc36500 0x560dbfc46a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.889+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/4258229809 >> 172.21.15.133:0/4258229809 conn(0x560dbfc2b310 msgr2=0x560dbfc305f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:34.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.889+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/4258229809 shutdown_connections 2023-11-27T15:22:34.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.889+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/4258229809 wait complete. 2023-11-27T15:22:34.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.890+0000 7fae7dcdfb80 1 -- start start 2023-11-27T15:22:34.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.890+0000 7fae7dcdfb80 1 Processor -- start 2023-11-27T15:22:34.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.890+0000 7fae7dcdfb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc35bb0 0x560dbfc411d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.890+0000 7fae7dcdfb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560dbfc36500 0x560dbfc41710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.890+0000 7fae7dcdfb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc370c0 0x560dbfc46640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.890+0000 7fae7dcdfb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbfc46b80 con 0x560dbfc370c0 2023-11-27T15:22:34.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.890+0000 7fae7dcdfb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbfc46cf0 con 0x560dbfc35bb0 2023-11-27T15:22:34.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.890+0000 7fae7dcdfb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dbfc9e6e0 con 0x560dbfc36500 2023-11-27T15:22:34.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.890+0000 7fae7dcdfb80 10 client.0 _ll_register_callbacks cb 0x7fae54004620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:22:34.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.891+0000 7fae63fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc35bb0 0x560dbfc411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.891+0000 7fae6899c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc370c0 0x560dbfc46640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.891+0000 7fae63fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc35bb0 0x560dbfc411d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:36932/0 (socket says 172.21.15.133:36932) 2023-11-27T15:22:34.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.891+0000 7fae63fff700 1 -- 172.21.15.133:0/1321927453 learned_addr learned my addr 172.21.15.133:0/1321927453 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:34.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.891+0000 7fae637fe700 1 --2- 172.21.15.133:0/1321927453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560dbfc36500 0x560dbfc41710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.891+0000 7fae63fff700 1 -- 172.21.15.133:0/1321927453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560dbfc36500 msgr2=0x560dbfc41710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:34.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.891+0000 7fae63fff700 1 --2- 172.21.15.133:0/1321927453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560dbfc36500 0x560dbfc41710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.891+0000 7fae63fff700 1 -- 172.21.15.133:0/1321927453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc370c0 msgr2=0x560dbfc46640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:34.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.891+0000 7fae63fff700 1 --2- 172.21.15.133:0/1321927453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc370c0 0x560dbfc46640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.891+0000 7fae63fff700 1 -- 172.21.15.133:0/1321927453 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560dbfb4d140 con 0x560dbfc35bb0 2023-11-27T15:22:34.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.891+0000 7fae6899c700 1 --2- 172.21.15.133:0/1321927453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc370c0 0x560dbfc46640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:34.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.891+0000 7fae63fff700 1 --2- 172.21.15.133:0/1321927453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc35bb0 0x560dbfc411d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fae54004010 tx=0x7fae5401fca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f9abda991c995af server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:34.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.891+0000 7fae437fe700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:22:34.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.891+0000 7fae437fe700 1 -- 172.21.15.133:0/1321927453 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5401f310 con 0x560dbfc35bb0 2023-11-27T15:22:34.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.892+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/1321927453 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae5801e040 con 0x560dbfc35bb0 2023-11-27T15:22:34.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.892+0000 7fae437fe700 1 -- 172.21.15.133:0/1321927453 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fae540288e0 con 0x560dbfc35bb0 2023-11-27T15:22:34.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.892+0000 7fae437fe700 1 -- 172.21.15.133:0/1321927453 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae54028ae0 con 0x560dbfc35bb0 2023-11-27T15:22:34.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.892+0000 7fae7dcdfb80 10 client.25325 fetch_fsmap learned FSMap version 23 2023-11-27T15:22:34.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.892+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/1321927453 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560dbf99dc10 con 0x560dbfc35bb0 2023-11-27T15:22:34.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.892+0000 7fae437fe700 1 -- 172.21.15.133:0/1321927453 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fae5801e040 con 0x560dbfc35bb0 2023-11-27T15:22:34.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.892+0000 7fae437fe700 1 -- 172.21.15.133:0/1321927453 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fae5401e420 con 0x560dbfc35bb0 2023-11-27T15:22:34.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.892+0000 7fae7dcdfb80 10 client.25325 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:22:34.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.892+0000 7fae7dcdfb80 10 client.25325 resolve_mds: resolved name 'h' to daemon mds.h 2023-11-27T15:22:34.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.892+0000 7fae7dcdfb80 20 client.25325 populate_metadata read hostname 'smithi133' 2023-11-27T15:22:34.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.892+0000 7fae7dcdfb80 1 --2- 172.21.15.133:0/1321927453 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x560dbfca2df0 0x560dbfc9f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:34.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.892+0000 7fae7dcdfb80 4 client.25325 mds_command: new command op to 24521 tid=0[{"prefix": "status"}] 2023-11-27T15:22:34.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.892+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/1321927453 --> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560dbfb55610 con 0x560dbfca2df0 2023-11-27T15:22:34.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.893+0000 7fae637fe700 1 --2- 172.21.15.133:0/1321927453 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x560dbfca2df0 0x560dbfc9f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:34.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.893+0000 7fae437fe700 10 client.25325 ms_handle_connect on v2:172.21.15.174:6836/1955090548 2023-11-27T15:22:34.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.893+0000 7fae637fe700 1 --2- 172.21.15.133:0/1321927453 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x560dbfca2df0 0x560dbfc9f7b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cc1d17a4638564e9 server_cookie=e94690058a1c3382 in_seq=0 out_seq=0 2023-11-27T15:22:34.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.893+0000 7fae437fe700 1 -- 172.21.15.133:0/1321927453 <== mds.1 v2:172.21.15.174:6836/1955090548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x560dbfb55610 con 0x560dbfca2df0 2023-11-27T15:22:34.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.893+0000 7fae437fe700 10 client.25325 handle_command_reply: tid=0 2023-11-27T15:22:34.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.893+0000 7fae7dcdfb80 1 client.25325 shutdown 2023-11-27T15:22:34.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.894+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/1321927453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc35bb0 msgr2=0x560dbfc411d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.894+0000 7fae7dcdfb80 1 --2- 172.21.15.133:0/1321927453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc35bb0 0x560dbfc411d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fae54004010 tx=0x7fae5401fca0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.894+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/1321927453 shutdown_connections 2023-11-27T15:22:34.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.894+0000 7fae7dcdfb80 1 --2- 172.21.15.133:0/1321927453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560dbfc370c0 0x560dbfc46640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.894+0000 7fae7dcdfb80 1 --2- 172.21.15.133:0/1321927453 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x560dbfca2df0 0x560dbfc9f7b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.894+0000 7fae7dcdfb80 1 --2- 172.21.15.133:0/1321927453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560dbfc35bb0 0x560dbfc411d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.894+0000 7fae7dcdfb80 1 --2- 172.21.15.133:0/1321927453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560dbfc36500 0x560dbfc41710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.894+0000 7fae437fe700 0 client.25325 ms_handle_reset on v2:172.21.15.174:6836/1955090548 2023-11-27T15:22:34.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.894+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/1321927453 >> 172.21.15.133:0/1321927453 conn(0x560dbfc2b310 msgr2=0x560dbfc49bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:34.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.894+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/1321927453 shutdown_connections 2023-11-27T15:22:34.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.894+0000 7fae7dcdfb80 1 -- 172.21.15.133:0/1321927453 wait complete. 2023-11-27T15:22:34.902 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.894+0000 7fae7dcdfb80 20 client.25325 trim_cache size 0 max 16384 2023-11-27T15:22:34.902 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:22:34.902 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "5227da4c-8d37-11ee-95a2-87774f69a715", 2023-11-27T15:22:34.903 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 1, 2023-11-27T15:22:34.903 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24521, 2023-11-27T15:22:34.903 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2023-11-27T15:22:34.903 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2023-11-27T15:22:34.903 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2023-11-27T15:22:34.903 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 92.266484886000001, 2023-11-27T15:22:34.903 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 23, 2023-11-27T15:22:34.903 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 92, 2023-11-27T15:22:34.903 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 90, 2023-11-27T15:22:34.903 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 138.98989220300001 2023-11-27T15:22:34.903 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:22:34.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.895+0000 7fae6a3ff700 1 -- 172.21.15.133:0/3562473663 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fae4c0a1150 msgr2=0x7fae4c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.895+0000 7fae6a3ff700 1 --2- 172.21.15.133:0/3562473663 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fae4c0a1150 0x7fae4c0a3600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fae64101290 tx=0x7fae54009260 comp rx=0 tx=0).stop 2023-11-27T15:22:34.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.895+0000 7fae6a3ff700 1 -- 172.21.15.133:0/3562473663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae64101dc0 msgr2=0x7fae64073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:34.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.895+0000 7fae6a3ff700 1 --2- 172.21.15.133:0/3562473663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae64101dc0 0x7fae64073420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00efc0 tx=0x7fae5c00c3a0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.895+0000 7fae6a3ff700 1 -- 172.21.15.133:0/3562473663 shutdown_connections 2023-11-27T15:22:34.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.895+0000 7fae6a3ff700 1 --2- 172.21.15.133:0/3562473663 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fae4c0a1150 0x7fae4c0a3600 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.895+0000 7fae6a3ff700 1 --2- 172.21.15.133:0/3562473663 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae64101dc0 0x7fae64073420 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.895+0000 7fae6a3ff700 1 --2- 172.21.15.133:0/3562473663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae64100240 0x7fae640748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.904 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.895+0000 7fae6a3ff700 1 --2- 172.21.15.133:0/3562473663 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fae64101430 0x7fae64072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:34.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.895+0000 7fae6a3ff700 1 -- 172.21.15.133:0/3562473663 >> 172.21.15.133:0/3562473663 conn(0x7fae640fb820 msgr2=0x7fae640fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:34.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.895+0000 7fae6a3ff700 1 -- 172.21.15.133:0/3562473663 shutdown_connections 2023-11-27T15:22:34.905 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:34.895+0000 7fae6a3ff700 1 -- 172.21.15.133:0/3562473663 wait complete. 2023-11-27T15:22:34.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:34 smithi133 ceph-mon[97860]: pgmap v338: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:22:34.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:34 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/215037347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:34.914 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:34 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1283001236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:34.917 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-27T15:22:35.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:34 smithi177 ceph-mon[131504]: pgmap v338: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:22:35.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:34 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/215037347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:35.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:34 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1283001236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:35.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.110+0000 7fd6607ff700 1 -- 172.21.15.133:0/2629344462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd658101420 msgr2=0x7fd658101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:35.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.110+0000 7fd6607ff700 1 --2- 172.21.15.133:0/2629344462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd658101420 0x7fd658101870 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd648009990 tx=0x7fd648009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.110+0000 7fd6607ff700 1 -- 172.21.15.133:0/2629344462 shutdown_connections 2023-11-27T15:22:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.110+0000 7fd6607ff700 1 --2- 172.21.15.133:0/2629344462 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd658101db0 0x7fd658108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.110+0000 7fd6607ff700 1 --2- 172.21.15.133:0/2629344462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd658100230 0x7fd658100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.110+0000 7fd6607ff700 1 --2- 172.21.15.133:0/2629344462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd658101420 0x7fd658101870 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.110+0000 7fd6607ff700 1 -- 172.21.15.133:0/2629344462 >> 172.21.15.133:0/2629344462 conn(0x7fd6580fb810 msgr2=0x7fd6580fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.110+0000 7fd6607ff700 1 -- 172.21.15.133:0/2629344462 shutdown_connections 2023-11-27T15:22:35.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.110+0000 7fd6607ff700 1 -- 172.21.15.133:0/2629344462 wait complete. 2023-11-27T15:22:35.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.111+0000 7fd6607ff700 1 Processor -- start 2023-11-27T15:22:35.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.111+0000 7fd6607ff700 1 -- start start 2023-11-27T15:22:35.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.112+0000 7fd6607ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd658100230 0x7fd6581981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:35.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.112+0000 7fd6607ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd658101420 0x7fd658198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:35.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.112+0000 7fd6607ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd658101db0 0x7fd65819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:35.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.112+0000 7fd6607ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65819fd60 con 0x7fd658100230 2023-11-27T15:22:35.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.112+0000 7fd6607ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65819fea0 con 0x7fd658101420 2023-11-27T15:22:35.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.112+0000 7fd6607ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6581a0030 con 0x7fd658101db0 2023-11-27T15:22:35.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.112+0000 7fd65e59b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd658100230 0x7fd6581981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:35.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.112+0000 7fd65e59b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd658100230 0x7fd6581981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45154/0 (socket says 172.21.15.133:45154) 2023-11-27T15:22:35.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.112+0000 7fd65e59b700 1 -- 172.21.15.133:0/652775837 learned_addr learned my addr 172.21.15.133:0/652775837 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:35.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.112+0000 7fd65ed9c700 1 --2- 172.21.15.133:0/652775837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd658101db0 0x7fd65819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:35.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.112+0000 7fd65dd9a700 1 --2- 172.21.15.133:0/652775837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd658101420 0x7fd658198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:35.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.112+0000 7fd65e59b700 1 -- 172.21.15.133:0/652775837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd658101db0 msgr2=0x7fd65819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:35.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.112+0000 7fd65e59b700 1 --2- 172.21.15.133:0/652775837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd658101db0 0x7fd65819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.112+0000 7fd65e59b700 1 -- 172.21.15.133:0/652775837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd658101420 msgr2=0x7fd658198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:35.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.112+0000 7fd65e59b700 1 --2- 172.21.15.133:0/652775837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd658101420 0x7fd658198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.112+0000 7fd65e59b700 1 -- 172.21.15.133:0/652775837 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6480095f0 con 0x7fd658100230 2023-11-27T15:22:35.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.112+0000 7fd65dd9a700 1 --2- 172.21.15.133:0/652775837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd658101420 0x7fd658198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:22:35.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.113+0000 7fd65e59b700 1 --2- 172.21.15.133:0/652775837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd658100230 0x7fd6581981e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd65400e970 tx=0x7fd65400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:35.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.113+0000 7fd64f7fe700 1 -- 172.21.15.133:0/652775837 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654009790 con 0x7fd658100230 2023-11-27T15:22:35.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.113+0000 7fd64f7fe700 1 -- 172.21.15.133:0/652775837 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd65400eec0 con 0x7fd658100230 2023-11-27T15:22:35.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.113+0000 7fd6607ff700 1 -- 172.21.15.133:0/652775837 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd65810bbf0 con 0x7fd658100230 2023-11-27T15:22:35.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.113+0000 7fd64f7fe700 1 -- 172.21.15.133:0/652775837 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654003cf0 con 0x7fd658100230 2023-11-27T15:22:35.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.113+0000 7fd6607ff700 1 -- 172.21.15.133:0/652775837 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65810ba60 con 0x7fd658100230 2023-11-27T15:22:35.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.116+0000 7fd64f7fe700 1 -- 172.21.15.133:0/652775837 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd65400ccd0 con 0x7fd658100230 2023-11-27T15:22:35.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.118+0000 7fd674221b80 1 Processor -- start 2023-11-27T15:22:35.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.118+0000 7fd674221b80 1 -- start start 2023-11-27T15:22:35.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.118+0000 7fd64f7fe700 1 --2- 172.21.15.133:0/652775837 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd6440a11d0 0x7fd6440a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:35.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.119+0000 7fd64f7fe700 1 -- 172.21.15.133:0/652775837 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd654014070 con 0x7fd658100230 2023-11-27T15:22:35.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.119+0000 7fd674221b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55993656fbb0 0x55993656ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:35.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.119+0000 7fd65ed9c700 1 --2- 172.21.15.133:0/652775837 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd6440a11d0 0x7fd6440a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:35.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.119+0000 7fd674221b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559936570500 0x559936570ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:35.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.119+0000 7fd674221b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5599365710c0 0x559936571530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:35.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.119+0000 7fd674221b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559936475010 con 0x5599365710c0 2023-11-27T15:22:35.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.119+0000 7fd674221b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599364c7850 con 0x559936570500 2023-11-27T15:22:35.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.119+0000 7fd674221b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599364c0ce0 con 0x55993656fbb0 2023-11-27T15:22:35.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.119+0000 7fd65e59b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55993656fbb0 0x55993656ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:35.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.119+0000 7fd65ed9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5599365710c0 0x559936571530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:35.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.119+0000 7fd65e59b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55993656fbb0 0x55993656ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:50300/0 (socket says 172.21.15.133:50300) 2023-11-27T15:22:35.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.119+0000 7fd65e59b700 1 -- 172.21.15.133:0/1766677445 learned_addr learned my addr 172.21.15.133:0/1766677445 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:35.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.119+0000 7fd65dd9a700 1 --2- 172.21.15.133:0/1766677445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559936570500 0x559936570ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:35.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.119+0000 7fd65e59b700 1 -- 172.21.15.133:0/1766677445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559936570500 msgr2=0x559936570ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:35.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.119+0000 7fd65e59b700 1 --2- 172.21.15.133:0/1766677445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559936570500 0x559936570ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.119+0000 7fd65e59b700 1 -- 172.21.15.133:0/1766677445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5599365710c0 msgr2=0x559936571530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:35.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.119+0000 7fd65e59b700 1 --2- 172.21.15.133:0/1766677445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5599365710c0 0x559936571530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.119+0000 7fd65e59b700 1 -- 172.21.15.133:0/1766677445 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599363425e0 con 0x55993656fbb0 2023-11-27T15:22:35.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.119+0000 7fd65dd9a700 1 --2- 172.21.15.133:0/1766677445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559936570500 0x559936570ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:22:35.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.120+0000 7fd65ed9c700 1 --2- 172.21.15.133:0/652775837 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd6440a11d0 0x7fd6440a3680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd650009c90 tx=0x7fd650004320 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:35.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.120+0000 7fd65e59b700 1 --2- 172.21.15.133:0/1766677445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55993656fbb0 0x55993656ffc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd654096940 tx=0x7fd654099830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b8a2d658b2da5a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:35.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.120+0000 7fd64cff9700 1 -- 172.21.15.133:0/1766677445 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654099ed0 con 0x55993656fbb0 2023-11-27T15:22:35.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.120+0000 7fd64cff9700 1 -- 172.21.15.133:0/1766677445 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd65409df00 con 0x55993656fbb0 2023-11-27T15:22:35.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.120+0000 7fd64cff9700 1 -- 172.21.15.133:0/1766677445 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654099620 con 0x55993656fbb0 2023-11-27T15:22:35.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.121+0000 7fd674221b80 1 -- 172.21.15.133:0/1766677445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55993656fbb0 msgr2=0x55993656ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:35.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.121+0000 7fd674221b80 1 --2- 172.21.15.133:0/1766677445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55993656fbb0 0x55993656ffc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd654096940 tx=0x7fd654099830 comp rx=0 tx=0).stop 2023-11-27T15:22:35.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.121+0000 7fd674221b80 1 -- 172.21.15.133:0/1766677445 shutdown_connections 2023-11-27T15:22:35.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.121+0000 7fd674221b80 1 --2- 172.21.15.133:0/1766677445 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5599365710c0 0x559936571530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.121+0000 7fd674221b80 1 --2- 172.21.15.133:0/1766677445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55993656fbb0 0x55993656ffc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.121+0000 7fd674221b80 1 --2- 172.21.15.133:0/1766677445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559936570500 0x559936570ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.121+0000 7fd674221b80 1 -- 172.21.15.133:0/1766677445 >> 172.21.15.133:0/1766677445 conn(0x559936565310 msgr2=0x55993656a620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:35.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.121+0000 7fd674221b80 1 -- 172.21.15.133:0/1766677445 shutdown_connections 2023-11-27T15:22:35.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.121+0000 7fd674221b80 1 -- 172.21.15.133:0/1766677445 wait complete. 2023-11-27T15:22:35.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.121+0000 7fd674221b80 1 -- start start 2023-11-27T15:22:35.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.121+0000 7fd674221b80 1 Processor -- start 2023-11-27T15:22:35.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.122+0000 7fd674221b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55993656fbb0 0x5599365795b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:35.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.122+0000 7fd674221b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559936570500 0x559936579af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:35.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.122+0000 7fd674221b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599365710c0 0x559936580b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:35.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.122+0000 7fd674221b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559936462890 con 0x55993656fbb0 2023-11-27T15:22:35.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.122+0000 7fd674221b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x559936462530 con 0x5599365710c0 2023-11-27T15:22:35.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.122+0000 7fd674221b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599363448d0 con 0x559936570500 2023-11-27T15:22:35.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.122+0000 7fd674221b80 10 client.0 _ll_register_callbacks cb 0x559936480730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:22:35.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.122+0000 7fd65e59b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55993656fbb0 0x5599365795b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:35.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.122+0000 7fd65e59b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55993656fbb0 0x5599365795b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45166/0 (socket says 172.21.15.133:45166) 2023-11-27T15:22:35.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.122+0000 7fd65ed9c700 1 --2- 172.21.15.133:0/3195176228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599365710c0 0x559936580b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:35.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.122+0000 7fd65e59b700 1 -- 172.21.15.133:0/3195176228 learned_addr learned my addr 172.21.15.133:0/3195176228 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:35.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.122+0000 7fd65dd9a700 1 --2- 172.21.15.133:0/3195176228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559936570500 0x559936579af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:35.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.123+0000 7fd65ed9c700 1 -- 172.21.15.133:0/3195176228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559936570500 msgr2=0x559936579af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:35.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.123+0000 7fd65ed9c700 1 --2- 172.21.15.133:0/3195176228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559936570500 0x559936579af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.123+0000 7fd65ed9c700 1 -- 172.21.15.133:0/3195176228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55993656fbb0 msgr2=0x5599365795b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:35.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.123+0000 7fd65ed9c700 1 --2- 172.21.15.133:0/3195176228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55993656fbb0 0x5599365795b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.123+0000 7fd65ed9c700 1 -- 172.21.15.133:0/3195176228 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd65810ba60 con 0x5599365710c0 2023-11-27T15:22:35.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.123+0000 7fd65e59b700 1 --2- 172.21.15.133:0/3195176228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55993656fbb0 0x5599365795b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:35.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.123+0000 7fd65ed9c700 1 --2- 172.21.15.133:0/3195176228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599365710c0 0x559936580b30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd650003ad0 tx=0x7fd650004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=554f31b3af54d823 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:35.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.123+0000 7fd641ffb700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:22:35.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.123+0000 7fd641ffb700 1 -- 172.21.15.133:0/3195176228 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd650004bb0 con 0x5599365710c0 2023-11-27T15:22:35.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.123+0000 7fd641ffb700 1 -- 172.21.15.133:0/3195176228 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd650004d10 con 0x5599365710c0 2023-11-27T15:22:35.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.123+0000 7fd674221b80 1 -- 172.21.15.133:0/3195176228 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559936562ad0 con 0x5599365710c0 2023-11-27T15:22:35.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.123+0000 7fd641ffb700 1 -- 172.21.15.133:0/3195176228 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd65001a8d0 con 0x5599365710c0 2023-11-27T15:22:35.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.123+0000 7fd674221b80 10 client.25337 fetch_fsmap learned FSMap version 23 2023-11-27T15:22:35.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.124+0000 7fd674221b80 1 -- 172.21.15.133:0/3195176228 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x559936487140 con 0x5599365710c0 2023-11-27T15:22:35.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.124+0000 7fd641ffb700 1 -- 172.21.15.133:0/3195176228 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559936562ad0 con 0x5599365710c0 2023-11-27T15:22:35.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.124+0000 7fd641ffb700 1 -- 172.21.15.133:0/3195176228 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd650005460 con 0x5599365710c0 2023-11-27T15:22:35.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.124+0000 7fd674221b80 10 client.25337 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:22:35.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.124+0000 7fd674221b80 10 client.25337 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-27T15:22:35.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.124+0000 7fd674221b80 20 client.25337 populate_metadata read hostname 'smithi133' 2023-11-27T15:22:35.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.124+0000 7fd674221b80 1 --2- 172.21.15.133:0/3195176228 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x559936584850 0x559936583c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:35.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.124+0000 7fd674221b80 4 client.25337 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:22:35.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.124+0000 7fd674221b80 1 -- 172.21.15.133:0/3195176228 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559936483050 con 0x559936584850 2023-11-27T15:22:35.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.124+0000 7fd65e59b700 1 --2- 172.21.15.133:0/3195176228 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x559936584850 0x559936583c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:35.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.125+0000 7fd65e59b700 1 --2- 172.21.15.133:0/3195176228 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x559936584850 0x559936583c30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32f984eaeb256484 server_cookie=bfc7c6bc2a74b9e6 in_seq=0 out_seq=0 2023-11-27T15:22:35.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.125+0000 7fd641ffb700 10 client.25337 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:22:35.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.125+0000 7fd641ffb700 1 -- 172.21.15.133:0/3195176228 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559936483050 con 0x559936584850 2023-11-27T15:22:35.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.126+0000 7fd641ffb700 10 client.25337 handle_command_reply: tid=0 2023-11-27T15:22:35.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.126+0000 7fd674221b80 1 client.25337 shutdown 2023-11-27T15:22:35.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.126+0000 7fd674221b80 1 -- 172.21.15.133:0/3195176228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599365710c0 msgr2=0x559936580b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:35.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.126+0000 7fd674221b80 1 --2- 172.21.15.133:0/3195176228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599365710c0 0x559936580b30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd650003ad0 tx=0x7fd650004b40 comp rx=0 tx=0).stop 2023-11-27T15:22:35.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.126+0000 7fd674221b80 1 -- 172.21.15.133:0/3195176228 shutdown_connections 2023-11-27T15:22:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.126+0000 7fd674221b80 1 --2- 172.21.15.133:0/3195176228 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x559936584850 0x559936583c30 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.126+0000 7fd674221b80 1 --2- 172.21.15.133:0/3195176228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599365710c0 0x559936580b30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.126+0000 7fd641ffb700 0 client.25337 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:22:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.126+0000 7fd674221b80 1 --2- 172.21.15.133:0/3195176228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55993656fbb0 0x5599365795b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.126+0000 7fd674221b80 1 --2- 172.21.15.133:0/3195176228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559936570500 0x559936579af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.126+0000 7fd674221b80 1 -- 172.21.15.133:0/3195176228 >> 172.21.15.133:0/3195176228 conn(0x559936565310 msgr2=0x559936569d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.126+0000 7fd674221b80 1 -- 172.21.15.133:0/3195176228 shutdown_connections 2023-11-27T15:22:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.126+0000 7fd674221b80 1 -- 172.21.15.133:0/3195176228 wait complete. 2023-11-27T15:22:35.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.126+0000 7fd674221b80 20 client.25337 trim_cache size 0 max 16384 2023-11-27T15:22:35.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.144+0000 7fd674221b80 1 Processor -- start 2023-11-27T15:22:35.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.144+0000 7fd674221b80 1 -- start start 2023-11-27T15:22:35.157 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:34 smithi174 ceph-mon[131110]: pgmap v338: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-11-27T15:22:35.157 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:34 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/215037347' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:35.157 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:34 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1283001236' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:35.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.145+0000 7fd674221b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55993656fbb0 0x559936583c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:35.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.145+0000 7fd674221b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559936570500 0x559936584170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:35.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.145+0000 7fd674221b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599365710c0 0x559936580af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:35.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.145+0000 7fd674221b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55993656b540 con 0x55993656fbb0 2023-11-27T15:22:35.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.145+0000 7fd674221b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x559936483050 con 0x559936570500 2023-11-27T15:22:35.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.145+0000 7fd674221b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599363cced0 con 0x5599365710c0 2023-11-27T15:22:35.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.145+0000 7fd65e59b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55993656fbb0 0x559936583c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:35.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.145+0000 7fd65e59b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55993656fbb0 0x559936583c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45172/0 (socket says 172.21.15.133:45172) 2023-11-27T15:22:35.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.145+0000 7fd65e59b700 1 -- 172.21.15.133:0/3590158284 learned_addr learned my addr 172.21.15.133:0/3590158284 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:35.158 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.145+0000 7fd65dd9a700 1 --2- 172.21.15.133:0/3590158284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559936570500 0x559936584170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:35.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.145+0000 7fd65ed9c700 1 --2- 172.21.15.133:0/3590158284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599365710c0 0x559936580af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:35.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.145+0000 7fd65e59b700 1 -- 172.21.15.133:0/3590158284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599365710c0 msgr2=0x559936580af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:35.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.145+0000 7fd65e59b700 1 --2- 172.21.15.133:0/3590158284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599365710c0 0x559936580af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.145+0000 7fd65e59b700 1 -- 172.21.15.133:0/3590158284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559936570500 msgr2=0x559936584170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:35.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.145+0000 7fd65e59b700 1 --2- 172.21.15.133:0/3590158284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559936570500 0x559936584170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.145+0000 7fd65e59b700 1 -- 172.21.15.133:0/3590158284 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559936486e90 con 0x55993656fbb0 2023-11-27T15:22:35.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.146+0000 7fd65e59b700 1 --2- 172.21.15.133:0/3590158284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55993656fbb0 0x559936583c30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd654005950 tx=0x7fd6540ca8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af509d64edf0fd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:35.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.146+0000 7fd64d7fa700 1 -- 172.21.15.133:0/3590158284 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd654099ed0 con 0x55993656fbb0 2023-11-27T15:22:35.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.146+0000 7fd64d7fa700 1 -- 172.21.15.133:0/3590158284 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd654098270 con 0x55993656fbb0 2023-11-27T15:22:35.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.146+0000 7fd64d7fa700 1 -- 172.21.15.133:0/3590158284 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6540adea0 con 0x55993656fbb0 2023-11-27T15:22:35.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.146+0000 7fd674221b80 1 -- 172.21.15.133:0/3590158284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55993656fbb0 msgr2=0x559936583c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:35.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.147+0000 7fd674221b80 1 --2- 172.21.15.133:0/3590158284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55993656fbb0 0x559936583c30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd654005950 tx=0x7fd6540ca8e0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.159 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.147+0000 7fd674221b80 1 -- 172.21.15.133:0/3590158284 shutdown_connections 2023-11-27T15:22:35.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.147+0000 7fd674221b80 1 --2- 172.21.15.133:0/3590158284 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5599365710c0 0x559936580af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.147+0000 7fd674221b80 1 --2- 172.21.15.133:0/3590158284 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55993656fbb0 0x559936583c30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.147+0000 7fd674221b80 1 --2- 172.21.15.133:0/3590158284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559936570500 0x559936584170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.147+0000 7fd674221b80 1 -- 172.21.15.133:0/3590158284 >> 172.21.15.133:0/3590158284 conn(0x559936565310 msgr2=0x5599365774e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:35.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.147+0000 7fd674221b80 1 -- 172.21.15.133:0/3590158284 shutdown_connections 2023-11-27T15:22:35.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.147+0000 7fd674221b80 1 -- 172.21.15.133:0/3590158284 wait complete. 2023-11-27T15:22:35.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.147+0000 7fd674221b80 1 -- start start 2023-11-27T15:22:35.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.147+0000 7fd674221b80 1 Processor -- start 2023-11-27T15:22:35.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.148+0000 7fd674221b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55993656fbb0 0x55993657d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:35.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.148+0000 7fd674221b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559936570500 0x55993657dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:35.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.148+0000 7fd674221b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599365710c0 0x5599365d86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:35.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.148+0000 7fd674221b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559936580b00 con 0x559936570500 2023-11-27T15:22:35.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.148+0000 7fd674221b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x559936580c70 con 0x5599365710c0 2023-11-27T15:22:35.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.148+0000 7fd674221b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x559936580e30 con 0x55993656fbb0 2023-11-27T15:22:35.160 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.148+0000 7fd674221b80 10 client.0 _ll_register_callbacks cb 0x559936480730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:22:35.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.148+0000 7fd65dd9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559936570500 0x55993657dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:35.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.148+0000 7fd65dd9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559936570500 0x55993657dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45186/0 (socket says 172.21.15.133:45186) 2023-11-27T15:22:35.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.148+0000 7fd65dd9a700 1 -- 172.21.15.133:0/1864327262 learned_addr learned my addr 172.21.15.133:0/1864327262 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:35.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.148+0000 7fd65e59b700 1 --2- 172.21.15.133:0/1864327262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55993656fbb0 0x55993657d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:35.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.148+0000 7fd65ed9c700 1 --2- 172.21.15.133:0/1864327262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599365710c0 0x5599365d86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:35.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.148+0000 7fd65dd9a700 1 -- 172.21.15.133:0/1864327262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55993656fbb0 msgr2=0x55993657d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:35.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.148+0000 7fd65dd9a700 1 --2- 172.21.15.133:0/1864327262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55993656fbb0 0x55993657d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.148+0000 7fd65dd9a700 1 -- 172.21.15.133:0/1864327262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599365710c0 msgr2=0x5599365d86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:35.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.148+0000 7fd65dd9a700 1 --2- 172.21.15.133:0/1864327262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599365710c0 0x5599365d86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.148+0000 7fd65dd9a700 1 -- 172.21.15.133:0/1864327262 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5599363425e0 con 0x559936570500 2023-11-27T15:22:35.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.148+0000 7fd65e59b700 1 --2- 172.21.15.133:0/1864327262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55993656fbb0 0x55993657d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:35.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.149+0000 7fd65dd9a700 1 --2- 172.21.15.133:0/1864327262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559936570500 0x55993657dae0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd648009990 tx=0x7fd64800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c935e6fac11a72 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:35.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.149+0000 7fd641ffb700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:22:35.161 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.149+0000 7fd641ffb700 1 -- 172.21.15.133:0/1864327262 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64800fa50 con 0x559936570500 2023-11-27T15:22:35.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.149+0000 7fd641ffb700 1 -- 172.21.15.133:0/1864327262 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd64800fbb0 con 0x559936570500 2023-11-27T15:22:35.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.149+0000 7fd674221b80 1 -- 172.21.15.133:0/1864327262 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd65409df00 con 0x559936570500 2023-11-27T15:22:35.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.149+0000 7fd641ffb700 1 -- 172.21.15.133:0/1864327262 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd64800fdb0 con 0x559936570500 2023-11-27T15:22:35.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.149+0000 7fd674221b80 10 client.15297 fetch_fsmap learned FSMap version 23 2023-11-27T15:22:35.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.149+0000 7fd674221b80 1 -- 172.21.15.133:0/1864327262 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5599362d7c10 con 0x559936570500 2023-11-27T15:22:35.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.149+0000 7fd641ffb700 1 -- 172.21.15.133:0/1864327262 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd65409df00 con 0x559936570500 2023-11-27T15:22:35.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.150+0000 7fd641ffb700 1 -- 172.21.15.133:0/1864327262 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd648017510 con 0x559936570500 2023-11-27T15:22:35.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.150+0000 7fd674221b80 10 client.15297 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:22:35.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.150+0000 7fd674221b80 10 client.15297 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-27T15:22:35.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.150+0000 7fd674221b80 20 client.15297 populate_metadata read hostname 'smithi133' 2023-11-27T15:22:35.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.150+0000 7fd674221b80 1 --2- 172.21.15.133:0/1864327262 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5599365dd7e0 0x5599365da150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:35.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.150+0000 7fd674221b80 4 client.15297 mds_command: new command op to 24560 tid=0[{"prefix": "status"}] 2023-11-27T15:22:35.162 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.150+0000 7fd674221b80 1 -- 172.21.15.133:0/1864327262 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55993648f610 con 0x5599365dd7e0 2023-11-27T15:22:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.150+0000 7fd65e59b700 1 --2- 172.21.15.133:0/1864327262 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5599365dd7e0 0x5599365da150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.150+0000 7fd65e59b700 1 --2- 172.21.15.133:0/1864327262 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5599365dd7e0 0x5599365da150 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a506294fdb083c7 server_cookie=5e6556d2a4ba796b in_seq=0 out_seq=0 2023-11-27T15:22:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.150+0000 7fd641ffb700 10 client.15297 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:22:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.151+0000 7fd641ffb700 1 -- 172.21.15.133:0/1864327262 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55993648f610 con 0x5599365dd7e0 2023-11-27T15:22:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.151+0000 7fd641ffb700 10 client.15297 handle_command_reply: tid=0 2023-11-27T15:22:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.151+0000 7fd674221b80 1 client.15297 shutdown 2023-11-27T15:22:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.151+0000 7fd674221b80 1 -- 172.21.15.133:0/1864327262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559936570500 msgr2=0x55993657dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.151+0000 7fd674221b80 1 --2- 172.21.15.133:0/1864327262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559936570500 0x55993657dae0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd648009990 tx=0x7fd64800bf40 comp rx=0 tx=0).stop 2023-11-27T15:22:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.151+0000 7fd674221b80 1 -- 172.21.15.133:0/1864327262 shutdown_connections 2023-11-27T15:22:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.151+0000 7fd674221b80 1 --2- 172.21.15.133:0/1864327262 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5599365dd7e0 0x5599365da150 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.151+0000 7fd674221b80 1 --2- 172.21.15.133:0/1864327262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599365710c0 0x5599365d86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.151+0000 7fd641ffb700 0 client.15297 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:22:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.151+0000 7fd674221b80 1 --2- 172.21.15.133:0/1864327262 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55993656fbb0 0x55993657d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.151+0000 7fd674221b80 1 --2- 172.21.15.133:0/1864327262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559936570500 0x55993657dae0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.163 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.151+0000 7fd674221b80 1 -- 172.21.15.133:0/1864327262 >> 172.21.15.133:0/1864327262 conn(0x559936565310 msgr2=0x559936583c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:35.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.151+0000 7fd674221b80 1 -- 172.21.15.133:0/1864327262 shutdown_connections 2023-11-27T15:22:35.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.151+0000 7fd674221b80 1 -- 172.21.15.133:0/1864327262 wait complete. 2023-11-27T15:22:35.164 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.151+0000 7fd674221b80 20 client.15297 trim_cache size 0 max 16384 2023-11-27T15:22:35.164 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:22:35.164 INFO:teuthology.orchestra.run.smithi133.stdout: "cluster_fsid": "5227da4c-8d37-11ee-95a2-87774f69a715", 2023-11-27T15:22:35.164 INFO:teuthology.orchestra.run.smithi133.stdout: "whoami": 0, 2023-11-27T15:22:35.164 INFO:teuthology.orchestra.run.smithi133.stdout: "id": 24560, 2023-11-27T15:22:35.164 INFO:teuthology.orchestra.run.smithi133.stdout: "want_state": "up:active", 2023-11-27T15:22:35.164 INFO:teuthology.orchestra.run.smithi133.stdout: "state": "up:active", 2023-11-27T15:22:35.164 INFO:teuthology.orchestra.run.smithi133.stdout: "fs_name": "cephfs", 2023-11-27T15:22:35.164 INFO:teuthology.orchestra.run.smithi133.stdout: "rank_uptime": 121.947623928, 2023-11-27T15:22:35.165 INFO:teuthology.orchestra.run.smithi133.stdout: "mdsmap_epoch": 23, 2023-11-27T15:22:35.165 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch": 92, 2023-11-27T15:22:35.165 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_epoch_barrier": 90, 2023-11-27T15:22:35.165 INFO:teuthology.orchestra.run.smithi133.stdout: "uptime": 128.64068994499999 2023-11-27T15:22:35.165 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:22:35.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.152+0000 7fd6607ff700 1 -- 172.21.15.133:0/652775837 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd6440a11d0 msgr2=0x7fd6440a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:35.165 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.152+0000 7fd6607ff700 1 --2- 172.21.15.133:0/652775837 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd6440a11d0 0x7fd6440a3680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd650009c90 tx=0x7fd650004320 comp rx=0 tx=0).stop 2023-11-27T15:22:35.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.152+0000 7fd6607ff700 1 -- 172.21.15.133:0/652775837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd658100230 msgr2=0x7fd6581981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:35.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.152+0000 7fd6607ff700 1 --2- 172.21.15.133:0/652775837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd658100230 0x7fd6581981e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd65400e970 tx=0x7fd65400ec80 comp rx=0 tx=0).stop 2023-11-27T15:22:35.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.152+0000 7fd6607ff700 1 -- 172.21.15.133:0/652775837 shutdown_connections 2023-11-27T15:22:35.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.152+0000 7fd6607ff700 1 --2- 172.21.15.133:0/652775837 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd6440a11d0 0x7fd6440a3680 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.152+0000 7fd6607ff700 1 --2- 172.21.15.133:0/652775837 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd658101db0 0x7fd65819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.152+0000 7fd6607ff700 1 --2- 172.21.15.133:0/652775837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd658100230 0x7fd6581981e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.152+0000 7fd6607ff700 1 --2- 172.21.15.133:0/652775837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd658101420 0x7fd658198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:35.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.152+0000 7fd6607ff700 1 -- 172.21.15.133:0/652775837 >> 172.21.15.133:0/652775837 conn(0x7fd6580fb810 msgr2=0x7fd658106ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:35.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.152+0000 7fd6607ff700 1 -- 172.21.15.133:0/652775837 shutdown_connections 2023-11-27T15:22:35.166 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:35.152+0000 7fd6607ff700 1 -- 172.21.15.133:0/652775837 wait complete. 2023-11-27T15:22:35.173 INFO:teuthology.run_tasks:Running task kclient... 2023-11-27T15:22:35.191 INFO:tasks.kclient:Mounting kernel clients... 2023-11-27T15:22:35.191 INFO:tasks.kclient:config is {'client.0': {}} 2023-11-27T15:22:35.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:22:35.191 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2023-11-27T15:22:35.235 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-11-27T15:22:35.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:22:35.236 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-27T15:22:35.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:22:35.346 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2023-11-27T15:22:35.451 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2023-11-27T15:22:35.451 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-27T15:22:35.451 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-27T15:22:35.451 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph fs ls 2023-11-27T15:22:36.300 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:22:36.854 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:36 smithi133 ceph-mon[97860]: pgmap v339: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-11-27T15:22:37.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:36 smithi177 ceph-mon[131504]: pgmap v339: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-11-27T15:22:37.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:36 smithi174 ceph-mon[131110]: pgmap v339: 129 pgs: 129 active+clean; 639 KiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-11-27T15:22:37.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.696+0000 7f2e1d7f2700 1 -- 172.21.15.133:0/3314207841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e181055e0 msgr2=0x7f2e1810d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:37.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.696+0000 7f2e1d7f2700 1 --2- 172.21.15.133:0/3314207841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e181055e0 0x7f2e1810d160 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2e1000b0d0 tx=0x7f2e1000b3e0 comp rx=0 tx=0).stop 2023-11-27T15:22:37.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.696+0000 7f2e1d7f2700 1 -- 172.21.15.133:0/3314207841 shutdown_connections 2023-11-27T15:22:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.696+0000 7f2e1d7f2700 1 --2- 172.21.15.133:0/3314207841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e181055e0 0x7f2e1810d160 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.696+0000 7f2e1d7f2700 1 --2- 172.21.15.133:0/3314207841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e18100540 0x7f2e18100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.696+0000 7f2e1d7f2700 1 --2- 172.21.15.133:0/3314207841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e18100ee0 0x7f2e18104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.696+0000 7f2e1d7f2700 1 -- 172.21.15.133:0/3314207841 >> 172.21.15.133:0/3314207841 conn(0x7f2e180fbde0 msgr2=0x7f2e180fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.696+0000 7f2e1d7f2700 1 -- 172.21.15.133:0/3314207841 shutdown_connections 2023-11-27T15:22:37.701 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.696+0000 7f2e1d7f2700 1 -- 172.21.15.133:0/3314207841 wait complete. 2023-11-27T15:22:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.697+0000 7f2e1d7f2700 1 Processor -- start 2023-11-27T15:22:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.697+0000 7f2e1d7f2700 1 -- start start 2023-11-27T15:22:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.698+0000 7f2e1d7f2700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e18100540 0x7f2e18072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.698+0000 7f2e1d7f2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e18100ee0 0x7f2e18073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.698+0000 7f2e1d7f2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e181055e0 0x7f2e180738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.698+0000 7f2e1d7f2700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e181a4900 con 0x7f2e18100540 2023-11-27T15:22:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.698+0000 7f2e1d7f2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e181a4a70 con 0x7f2e181055e0 2023-11-27T15:22:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.698+0000 7f2e1d7f2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e181a4c30 con 0x7f2e18100ee0 2023-11-27T15:22:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.698+0000 7f2e16ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e18100540 0x7f2e18072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:37.702 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.698+0000 7f2e167fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e18100ee0 0x7f2e18073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:37.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.698+0000 7f2e16ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e18100540 0x7f2e18072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45202/0 (socket says 172.21.15.133:45202) 2023-11-27T15:22:37.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.698+0000 7f2e16ffd700 1 -- 172.21.15.133:0/2643280065 learned_addr learned my addr 172.21.15.133:0/2643280065 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:37.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.698+0000 7f2e177fe700 1 --2- 172.21.15.133:0/2643280065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e181055e0 0x7f2e180738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:37.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.698+0000 7f2e167fc700 1 -- 172.21.15.133:0/2643280065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e181055e0 msgr2=0x7f2e180738b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:37.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.698+0000 7f2e167fc700 1 --2- 172.21.15.133:0/2643280065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e181055e0 0x7f2e180738b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:37.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.698+0000 7f2e167fc700 1 -- 172.21.15.133:0/2643280065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e18100540 msgr2=0x7f2e18072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:37.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.698+0000 7f2e167fc700 1 --2- 172.21.15.133:0/2643280065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e18100540 0x7f2e18072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:37.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.698+0000 7f2e167fc700 1 -- 172.21.15.133:0/2643280065 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e10009d00 con 0x7f2e18100ee0 2023-11-27T15:22:37.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.699+0000 7f2e167fc700 1 --2- 172.21.15.133:0/2643280065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e18100ee0 0x7f2e18073330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00e9c0 tx=0x7f2e0c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:37.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.699+0000 7f2dfffff700 1 -- 172.21.15.133:0/2643280065 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c00c9c0 con 0x7f2e18100ee0 2023-11-27T15:22:37.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.699+0000 7f2e1d7f2700 1 -- 172.21.15.133:0/2643280065 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e18110370 con 0x7f2e18100ee0 2023-11-27T15:22:37.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.699+0000 7f2dfffff700 1 -- 172.21.15.133:0/2643280065 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e0c010430 con 0x7f2e18100ee0 2023-11-27T15:22:37.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.699+0000 7f2dfffff700 1 -- 172.21.15.133:0/2643280065 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c018680 con 0x7f2e18100ee0 2023-11-27T15:22:37.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.699+0000 7f2e1d7f2700 1 -- 172.21.15.133:0/2643280065 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e181101e0 con 0x7f2e18100ee0 2023-11-27T15:22:37.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.700+0000 7f2e1d7f2700 1 -- 172.21.15.133:0/2643280065 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e18110500 con 0x7f2e18100ee0 2023-11-27T15:22:37.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.702+0000 7f2dfffff700 1 -- 172.21.15.133:0/2643280065 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2e0c00cb20 con 0x7f2e18100ee0 2023-11-27T15:22:37.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.703+0000 7f2dfffff700 1 --2- 172.21.15.133:0/2643280065 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2e000a1150 0x7f2e000a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:37.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.704+0000 7f2dfffff700 1 -- 172.21.15.133:0/2643280065 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f2e0c014070 con 0x7f2e18100ee0 2023-11-27T15:22:37.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.704+0000 7f2e16ffd700 1 --2- 172.21.15.133:0/2643280065 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2e000a1150 0x7f2e000a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:37.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.704+0000 7f2e16ffd700 1 --2- 172.21.15.133:0/2643280065 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2e000a1150 0x7f2e000a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2e08009d20 tx=0x7f2e08009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:37.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.706+0000 7f2dfffff700 1 -- 172.21.15.133:0/2643280065 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2e0c0960c0 con 0x7f2e18100ee0 2023-11-27T15:22:37.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.949+0000 7f2e1d7f2700 1 -- 172.21.15.133:0/2643280065 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2e18066ea0 con 0x7f2e18100ee0 2023-11-27T15:22:37.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.950+0000 7f2dfffff700 1 -- 172.21.15.133:0/2643280065 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+88 (secure 0 0 0) 0x7f2e0c0996e0 con 0x7f2e18100ee0 2023-11-27T15:22:37.954 INFO:teuthology.orchestra.run.smithi133.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-27T15:22:37.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.952+0000 7f2e1d7f2700 1 -- 172.21.15.133:0/2643280065 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2e000a1150 msgr2=0x7f2e000a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:37.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.952+0000 7f2e1d7f2700 1 --2- 172.21.15.133:0/2643280065 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2e000a1150 0x7f2e000a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2e08009d20 tx=0x7f2e08009260 comp rx=0 tx=0).stop 2023-11-27T15:22:37.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.952+0000 7f2e1d7f2700 1 -- 172.21.15.133:0/2643280065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e18100ee0 msgr2=0x7f2e18073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:37.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.952+0000 7f2e1d7f2700 1 --2- 172.21.15.133:0/2643280065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e18100ee0 0x7f2e18073330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00e9c0 tx=0x7f2e0c00ecd0 comp rx=0 tx=0).stop 2023-11-27T15:22:37.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.952+0000 7f2e1d7f2700 1 -- 172.21.15.133:0/2643280065 shutdown_connections 2023-11-27T15:22:37.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.952+0000 7f2e1d7f2700 1 --2- 172.21.15.133:0/2643280065 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2e000a1150 0x7f2e000a3600 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:37.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.952+0000 7f2e1d7f2700 1 --2- 172.21.15.133:0/2643280065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e181055e0 0x7f2e180738b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:37.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.952+0000 7f2e1d7f2700 1 --2- 172.21.15.133:0/2643280065 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e18100540 0x7f2e18072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:37.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.952+0000 7f2e1d7f2700 1 --2- 172.21.15.133:0/2643280065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e18100ee0 0x7f2e18073330 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:37.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.952+0000 7f2e1d7f2700 1 -- 172.21.15.133:0/2643280065 >> 172.21.15.133:0/2643280065 conn(0x7f2e180fbde0 msgr2=0x7f2e180fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:37.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.952+0000 7f2e1d7f2700 1 -- 172.21.15.133:0/2643280065 shutdown_connections 2023-11-27T15:22:37.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:37.952+0000 7f2e1d7f2700 1 -- 172.21.15.133:0/2643280065 wait complete. 2023-11-27T15:22:38.383 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-27T15:22:38.383 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-27T15:22:38.383 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2023-11-27T15:22:38.383 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-27T15:22:38.383 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-27T15:22:38.384 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-27T15:22:38.384 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-27T15:22:38.384 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-27T15:22:38.384 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-27T15:22:38.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:22:38.384 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2023-11-27T15:22:38.430 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-27T15:22:38.430 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-27T15:22:38.430 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2023-11-27T15:22:38.430 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-11-27T15:22:38.431 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2023-11-27T15:22:38.431 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-11-27T15:22:38.431 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-27T15:22:38.431 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2023-11-27T15:22:38.431 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2023-11-27T15:22:38.432 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-27T15:22:38.432 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2023-11-27T15:22:38.432 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2023-11-27T15:22:38.432 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-27T15:22:38.432 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2023-11-27T15:22:38.432 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-27T15:22:38.433 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2023-11-27T15:22:38.433 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-27T15:22:38.433 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 41532sec preferred_lft 41532sec 2023-11-27T15:22:38.433 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::89cb:f695:7bf5:3338/64 scope link noprefixroute 2023-11-27T15:22:38.433 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2023-11-27T15:22:38.434 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-27T15:22:38.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:22:38.435 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-11-27T15:22:38.435 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-11-27T15:22:38.435 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add name ceph-brx type bridge 2023-11-27T15:22:38.435 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr flush dev ceph-brx 2023-11-27T15:22:38.435 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set ceph-brx up 2023-11-27T15:22:38.435 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-27T15:22:38.436 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-11-27T15:22:38.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:22:38.679 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-27T15:22:38.749 INFO:teuthology.orchestra.run.smithi133.stdout:1 2023-11-27T15:22:38.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:22:38.753 DEBUG:teuthology.orchestra.run.smithi133:> route 2023-11-27T15:22:38.799 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2023-11-27T15:22:38.799 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-27T15:22:38.799 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-27T15:22:38.800 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-27T15:22:38.800 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-27T15:22:38.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:22:38.801 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-11-27T15:22:38.801 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-11-27T15:22:38.801 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-27T15:22:38.801 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-27T15:22:38.802 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-27T15:22:38.802 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-11-27T15:22:38.903 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:38 smithi133 ceph-mon[97860]: pgmap v340: 129 pgs: 129 active+clean; 639 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2023-11-27T15:22:38.903 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:38 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2643280065' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-27T15:22:38.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:22:38.975 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2023-11-27T15:22:39.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:22:39.019 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list-id 2023-11-27T15:22:39.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:38 smithi177 ceph-mon[131504]: pgmap v340: 129 pgs: 129 active+clean; 639 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2023-11-27T15:22:39.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:38 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2643280065' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-27T15:22:39.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:22:39.103 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-11-27T15:22:39.104 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-11-27T15:22:39.104 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-27T15:22:39.104 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-27T15:22:39.104 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-11-27T15:22:39.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:38 smithi174 ceph-mon[131110]: pgmap v340: 129 pgs: 129 active+clean; 639 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2023-11-27T15:22:39.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:38 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2643280065' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-11-27T15:22:39.288 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-27T15:22:39.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:22:39.288 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-11-27T15:22:39.288 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-11-27T15:22:39.288 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-27T15:22:39.288 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-27T15:22:39.288 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-27T15:22:39.288 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-27T15:22:39.288 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-27T15:22:39.289 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-11-27T15:22:39.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:22:39.533 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec bash -c ' 2023-11-27T15:22:39.533 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2023-11-27T15:22:39.533 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2023-11-27T15:22:39.533 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set dev brx.0 master ceph-brx 2023-11-27T15:22:39.533 DEBUG:teuthology.orchestra.run.smithi133:> ') 2023-11-27T15:22:39.706 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-27T15:22:39.706 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-27T15:22:39.750 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-27T15:22:39.750 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-27T15:22:39.751 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-27T15:22:39.835 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-11-27T15:22:39.835 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-11-27T15:22:39.835 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/5deb6751-ee17-4f54-a826-7cf3bc242b61 2023-11-27T15:22:39.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:22:39.836 DEBUG:teuthology.orchestra.run.smithi133:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/5deb6751-ee17-4f54-a826-7cf3bc242b61 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2023-11-27T15:22:39.979 INFO:teuthology.orchestra.run.smithi133.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2023-11-27T15:22:39.980 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2023-11-27T15:22:40.459 INFO:teuthology.orchestra.run.smithi133.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2023-11-27T15:22:40.459 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2023-11-27T15:22:40.459 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: resolved to: "172.21.15.133:3300,172.21.15.174:3300,172.21.15.177:3300" 2023-11-27T15:22:40.459 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: trying mount with new device syntax: 0@5227da4c-8d37-11ee-95a2-87774f69a715.cephfs=/volumes/_nogroup/sv_1/5deb6751-ee17-4f54-a826-7cf3bc242b61 2023-11-27T15:22:40.459 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,name=0,key=0,mon_addr=172.21.15.133:3300/172.21.15.174:3300/172.21.15.177:3300" will pass to kernel 2023-11-27T15:22:40.465 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-27T15:22:40.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:22:40.466 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-27T15:22:40.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:22:40.536 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-27T15:22:40.622 INFO:teuthology.orchestra.run.smithi133.stdout:client25352 2023-11-27T15:22:40.623 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-11-27T15:22:40.623 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-27T15:22:40.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:22:40.668 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-27T15:22:40.794 INFO:teuthology.orchestra.run.smithi133.stdout:client25352 2023-11-27T15:22:40.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:22:40.795 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-27T15:22:40.882 INFO:teuthology.orchestra.run.smithi133.stdout:client25352 2023-11-27T15:22:40.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:22:40.882 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/5227da4c-8d37-11ee-95a2-87774f69a715.client25352/status') 2023-11-27T15:22:40.990 INFO:teuthology.orchestra.run.smithi133.stdout:instance: client.25352 (3)172.21.15.133:0/3134531989 2023-11-27T15:22:40.990 INFO:teuthology.orchestra.run.smithi133.stdout:blocklisted: false 2023-11-27T15:22:40.991 INFO:teuthology.orchestra.run.smithi133.stderr:0+1 records in 2023-11-27T15:22:40.991 INFO:teuthology.orchestra.run.smithi133.stderr:0+1 records out 2023-11-27T15:22:40.991 INFO:teuthology.orchestra.run.smithi133.stderr:72 bytes copied, 6.4983e-05 s, 1.1 MB/s 2023-11-27T15:22:40.994 INFO:teuthology.run_tasks:Running task check-counter... 2023-11-27T15:22:41.009 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-11-27T15:22:41.024 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:22:41.084 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:40 smithi133 ceph-mon[97860]: pgmap v341: 129 pgs: 129 active+clean; 639 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-27T15:22:41.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:40 smithi177 ceph-mon[131504]: pgmap v341: 129 pgs: 129 active+clean; 639 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-27T15:22:41.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:40 smithi174 ceph-mon[131110]: pgmap v341: 129 pgs: 129 active+clean; 639 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-11-27T15:22:41.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.220+0000 7f76515ff700 1 -- 172.21.15.133:0/1677787191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f764c101e30 msgr2=0x7f764c1086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:41.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.220+0000 7f76515ff700 1 --2- 172.21.15.133:0/1677787191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f764c101e30 0x7f764c1086f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f764400b0a0 tx=0x7f764400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:41.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.220+0000 7f76515ff700 1 -- 172.21.15.133:0/1677787191 shutdown_connections 2023-11-27T15:22:41.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.220+0000 7f76515ff700 1 --2- 172.21.15.133:0/1677787191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f764c101e30 0x7f764c1086f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:41.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.220+0000 7f76515ff700 1 --2- 172.21.15.133:0/1677787191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f764c1002b0 0x7f764c1006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:41.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.220+0000 7f76515ff700 1 --2- 172.21.15.133:0/1677787191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f764c1014a0 0x7f764c1018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:41.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.220+0000 7f76515ff700 1 -- 172.21.15.133:0/1677787191 >> 172.21.15.133:0/1677787191 conn(0x7f764c0fb890 msgr2=0x7f764c0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:41.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.220+0000 7f76515ff700 1 -- 172.21.15.133:0/1677787191 shutdown_connections 2023-11-27T15:22:41.225 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.220+0000 7f76515ff700 1 -- 172.21.15.133:0/1677787191 wait complete. 2023-11-27T15:22:41.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.221+0000 7f76515ff700 1 Processor -- start 2023-11-27T15:22:41.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.221+0000 7f76515ff700 1 -- start start 2023-11-27T15:22:41.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.222+0000 7f76515ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f764c1002b0 0x7f764c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:41.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.222+0000 7f76515ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f764c1014a0 0x7f764c1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:41.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.222+0000 7f76515ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f764c101e30 0x7f764c19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:41.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.222+0000 7f76515ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c19fe20 con 0x7f764c101e30 2023-11-27T15:22:41.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.222+0000 7f76515ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c19ff60 con 0x7f764c1002b0 2023-11-27T15:22:41.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.222+0000 7f76515ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c1a00f0 con 0x7f764c1014a0 2023-11-27T15:22:41.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.222+0000 7f7643fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f764c1014a0 0x7f764c1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:41.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.222+0000 7f764b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f764c101e30 0x7f764c19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:41.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.222+0000 7f764affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f764c1002b0 0x7f764c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:41.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.222+0000 7f7643fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f764c1014a0 0x7f764c1987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:50366/0 (socket says 172.21.15.133:50366) 2023-11-27T15:22:41.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.222+0000 7f7643fff700 1 -- 172.21.15.133:0/541655819 learned_addr learned my addr 172.21.15.133:0/541655819 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:41.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.222+0000 7f7643fff700 1 -- 172.21.15.133:0/541655819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f764c1002b0 msgr2=0x7f764c1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:41.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.222+0000 7f7643fff700 1 --2- 172.21.15.133:0/541655819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f764c1002b0 0x7f764c1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:41.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.222+0000 7f7643fff700 1 -- 172.21.15.133:0/541655819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f764c101e30 msgr2=0x7f764c19f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:41.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.222+0000 7f7643fff700 1 --2- 172.21.15.133:0/541655819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f764c101e30 0x7f764c19f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:41.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.222+0000 7f7643fff700 1 -- 172.21.15.133:0/541655819 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7644009d00 con 0x7f764c1014a0 2023-11-27T15:22:41.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.222+0000 7f764b7fe700 1 --2- 172.21.15.133:0/541655819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f764c101e30 0x7f764c19f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:41.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.222+0000 7f7643fff700 1 --2- 172.21.15.133:0/541655819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f764c1014a0 0x7f764c1987e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f763400e9a0 tx=0x7f763400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:41.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.223+0000 7f7648ff9700 1 -- 172.21.15.133:0/541655819 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7634009790 con 0x7f764c1014a0 2023-11-27T15:22:41.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.223+0000 7f76515ff700 1 -- 172.21.15.133:0/541655819 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764c10bc80 con 0x7f764c1014a0 2023-11-27T15:22:41.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.223+0000 7f7648ff9700 1 -- 172.21.15.133:0/541655819 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7634010430 con 0x7f764c1014a0 2023-11-27T15:22:41.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.223+0000 7f7648ff9700 1 -- 172.21.15.133:0/541655819 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7634018680 con 0x7f764c1014a0 2023-11-27T15:22:41.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.223+0000 7f76515ff700 1 -- 172.21.15.133:0/541655819 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764c10baf0 con 0x7f764c1014a0 2023-11-27T15:22:41.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.224+0000 7f76515ff700 1 -- 172.21.15.133:0/541655819 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764c1a0830 con 0x7f764c1014a0 2023-11-27T15:22:41.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.226+0000 7f7648ff9700 1 -- 172.21.15.133:0/541655819 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f76340105a0 con 0x7f764c1014a0 2023-11-27T15:22:41.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.228+0000 7f7648ff9700 1 --2- 172.21.15.133:0/541655819 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f76380a1150 0x7f76380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:41.231 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.228+0000 7f7648ff9700 1 -- 172.21.15.133:0/541655819 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7634014070 con 0x7f764c1014a0 2023-11-27T15:22:41.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.228+0000 7f764affd700 1 --2- 172.21.15.133:0/541655819 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f76380a1150 0x7f76380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:41.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.229+0000 7f764affd700 1 --2- 172.21.15.133:0/541655819 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f76380a1150 0x7f76380a3600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f764c101300 tx=0x7f763c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:41.234 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.231+0000 7f7648ff9700 1 -- 172.21.15.133:0/541655819 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7634096410 con 0x7f764c1014a0 2023-11-27T15:22:41.439 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.436+0000 7f76515ff700 1 -- 172.21.15.133:0/541655819 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f764c1a0de0 con 0x7f764c1014a0 2023-11-27T15:22:41.441 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.438+0000 7f7648ff9700 1 -- 172.21.15.133:0/541655819 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24616 (secure 0 0 0) 0x7f7634026090 con 0x7f764c1014a0 2023-11-27T15:22:41.442 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:41.442 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":92,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:22:02.807005+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:22:41.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.440+0000 7f76515ff700 1 -- 172.21.15.133:0/541655819 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f76380a1150 msgr2=0x7f76380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:41.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.440+0000 7f76515ff700 1 --2- 172.21.15.133:0/541655819 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f76380a1150 0x7f76380a3600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f764c101300 tx=0x7f763c009260 comp rx=0 tx=0).stop 2023-11-27T15:22:41.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.440+0000 7f76515ff700 1 -- 172.21.15.133:0/541655819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f764c1014a0 msgr2=0x7f764c1987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:41.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.440+0000 7f76515ff700 1 --2- 172.21.15.133:0/541655819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f764c1014a0 0x7f764c1987e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f763400e9a0 tx=0x7f763400ecb0 comp rx=0 tx=0).stop 2023-11-27T15:22:41.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.440+0000 7f76515ff700 1 -- 172.21.15.133:0/541655819 shutdown_connections 2023-11-27T15:22:41.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.440+0000 7f76515ff700 1 --2- 172.21.15.133:0/541655819 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f76380a1150 0x7f76380a3600 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:41.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.440+0000 7f76515ff700 1 --2- 172.21.15.133:0/541655819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f764c101e30 0x7f764c19f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:41.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.440+0000 7f76515ff700 1 --2- 172.21.15.133:0/541655819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f764c1002b0 0x7f764c1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:41.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.440+0000 7f76515ff700 1 --2- 172.21.15.133:0/541655819 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f764c1014a0 0x7f764c1987e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:41.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.440+0000 7f76515ff700 1 -- 172.21.15.133:0/541655819 >> 172.21.15.133:0/541655819 conn(0x7f764c0fb890 msgr2=0x7f764c106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:41.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.440+0000 7f76515ff700 1 -- 172.21.15.133:0/541655819 shutdown_connections 2023-11-27T15:22:41.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.440+0000 7f76515ff700 1 -- 172.21.15.133:0/541655819 wait complete. 2023-11-27T15:22:41.463 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:16:20.216150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:20:31.107146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:20:32.614089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-27T15:20:36.126040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-27T15:22:41.464 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:22:41.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.661+0000 7f44e79ff700 1 -- 172.21.15.133:0/3594772827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44e0074ad0 msgr2=0x7f44e0072f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:41.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.661+0000 7f44e79ff700 1 --2- 172.21.15.133:0/3594772827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44e0074ad0 0x7f44e0072f20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f44dc009990 tx=0x7f44dc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:41.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.661+0000 7f44e79ff700 1 -- 172.21.15.133:0/3594772827 shutdown_connections 2023-11-27T15:22:41.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.661+0000 7f44e79ff700 1 --2- 172.21.15.133:0/3594772827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44e0108160 0x7f44e010a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:41.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.661+0000 7f44e79ff700 1 --2- 172.21.15.133:0/3594772827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44e0074ad0 0x7f44e0072f20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:41.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.661+0000 7f44e79ff700 1 --2- 172.21.15.133:0/3594772827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44e00734f0 0x7f44e0073960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:41.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.661+0000 7f44e79ff700 1 -- 172.21.15.133:0/3594772827 >> 172.21.15.133:0/3594772827 conn(0x7f44e00fb8b0 msgr2=0x7f44e00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:41.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.661+0000 7f44e79ff700 1 -- 172.21.15.133:0/3594772827 shutdown_connections 2023-11-27T15:22:41.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.661+0000 7f44e79ff700 1 -- 172.21.15.133:0/3594772827 wait complete. 2023-11-27T15:22:41.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.662+0000 7f44e79ff700 1 Processor -- start 2023-11-27T15:22:41.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.662+0000 7f44e79ff700 1 -- start start 2023-11-27T15:22:41.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.662+0000 7f44e79ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44e00734f0 0x7f44e0198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:41.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.663+0000 7f44e79ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44e0074ad0 0x7f44e01987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:41.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.663+0000 7f44e79ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44e0108160 0x7f44e019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:41.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.663+0000 7f44e79ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e019fde0 con 0x7f44e0108160 2023-11-27T15:22:41.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.663+0000 7f44e79ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e019ff20 con 0x7f44e00734f0 2023-11-27T15:22:41.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.663+0000 7f44e79ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e01a00b0 con 0x7f44e0074ad0 2023-11-27T15:22:41.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.663+0000 7f44e579b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44e00734f0 0x7f44e0198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:41.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.663+0000 7f44e579b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44e00734f0 0x7f44e0198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:37038/0 (socket says 172.21.15.133:37038) 2023-11-27T15:22:41.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.663+0000 7f44e579b700 1 -- 172.21.15.133:0/3928189131 learned_addr learned my addr 172.21.15.133:0/3928189131 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:41.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.663+0000 7f44e4f9a700 1 --2- 172.21.15.133:0/3928189131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44e0074ad0 0x7f44e01987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:41.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.663+0000 7f44e5f9c700 1 --2- 172.21.15.133:0/3928189131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44e0108160 0x7f44e019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:41.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.663+0000 7f44e579b700 1 -- 172.21.15.133:0/3928189131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44e0074ad0 msgr2=0x7f44e01987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:41.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.663+0000 7f44e579b700 1 --2- 172.21.15.133:0/3928189131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44e0074ad0 0x7f44e01987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:41.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.663+0000 7f44e579b700 1 -- 172.21.15.133:0/3928189131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44e0108160 msgr2=0x7f44e019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:41.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.663+0000 7f44e579b700 1 --2- 172.21.15.133:0/3928189131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44e0108160 0x7f44e019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:41.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.663+0000 7f44e579b700 1 -- 172.21.15.133:0/3928189131 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44dc0095f0 con 0x7f44e00734f0 2023-11-27T15:22:41.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.663+0000 7f44e579b700 1 --2- 172.21.15.133:0/3928189131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44e00734f0 0x7f44e0198260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f44dc004740 tx=0x7f44dc009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:41.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.664+0000 7f44d67fc700 1 -- 172.21.15.133:0/3928189131 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44dc01d070 con 0x7f44e00734f0 2023-11-27T15:22:41.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.664+0000 7f44e79ff700 1 -- 172.21.15.133:0/3928189131 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e010de60 con 0x7f44e00734f0 2023-11-27T15:22:41.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.664+0000 7f44d67fc700 1 -- 172.21.15.133:0/3928189131 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44dc004830 con 0x7f44e00734f0 2023-11-27T15:22:41.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.664+0000 7f44d67fc700 1 -- 172.21.15.133:0/3928189131 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44dc0053c0 con 0x7f44e00734f0 2023-11-27T15:22:41.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.664+0000 7f44e79ff700 1 -- 172.21.15.133:0/3928189131 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e010dcd0 con 0x7f44e00734f0 2023-11-27T15:22:41.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.665+0000 7f44e79ff700 1 -- 172.21.15.133:0/3928189131 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e01068a0 con 0x7f44e00734f0 2023-11-27T15:22:41.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.667+0000 7f44d67fc700 1 -- 172.21.15.133:0/3928189131 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f44dc005560 con 0x7f44e00734f0 2023-11-27T15:22:41.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.668+0000 7f44d67fc700 1 --2- 172.21.15.133:0/3928189131 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f44cc0a1150 0x7f44cc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:41.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.668+0000 7f44d67fc700 1 -- 172.21.15.133:0/3928189131 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f44dc0ce2d0 con 0x7f44e00734f0 2023-11-27T15:22:41.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.668+0000 7f44e4f9a700 1 --2- 172.21.15.133:0/3928189131 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f44cc0a1150 0x7f44cc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:41.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.669+0000 7f44e4f9a700 1 --2- 172.21.15.133:0/3928189131 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f44cc0a1150 0x7f44cc0a3600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f44d0009c60 tx=0x7f44d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:41.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.671+0000 7f44d67fc700 1 -- 172.21.15.133:0/3928189131 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f44dc097700 con 0x7f44e00734f0 2023-11-27T15:22:41.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.875+0000 7f44e79ff700 1 -- 172.21.15.133:0/3928189131 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f44e0067b00 con 0x7f44e00734f0 2023-11-27T15:22:41.879 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.876+0000 7f44d67fc700 1 -- 172.21.15.133:0/3928189131 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f44dc09ad20 con 0x7f44e00734f0 2023-11-27T15:22:41.880 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-11-27T15:22:41.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.878+0000 7f44e79ff700 1 -- 172.21.15.133:0/3928189131 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f44cc0a1150 msgr2=0x7f44cc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:41.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.878+0000 7f44e79ff700 1 --2- 172.21.15.133:0/3928189131 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f44cc0a1150 0x7f44cc0a3600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f44d0009c60 tx=0x7f44d0009260 comp rx=0 tx=0).stop 2023-11-27T15:22:41.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.879+0000 7f44e79ff700 1 -- 172.21.15.133:0/3928189131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44e00734f0 msgr2=0x7f44e0198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:41.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.879+0000 7f44e79ff700 1 --2- 172.21.15.133:0/3928189131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44e00734f0 0x7f44e0198260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f44dc004740 tx=0x7f44dc009580 comp rx=0 tx=0).stop 2023-11-27T15:22:41.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.879+0000 7f44e79ff700 1 -- 172.21.15.133:0/3928189131 shutdown_connections 2023-11-27T15:22:41.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.879+0000 7f44e79ff700 1 --2- 172.21.15.133:0/3928189131 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f44cc0a1150 0x7f44cc0a3600 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:41.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.879+0000 7f44e79ff700 1 --2- 172.21.15.133:0/3928189131 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44e0108160 0x7f44e019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:41.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.879+0000 7f44e79ff700 1 --2- 172.21.15.133:0/3928189131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44e00734f0 0x7f44e0198260 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:41.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.879+0000 7f44e79ff700 1 --2- 172.21.15.133:0/3928189131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f44e0074ad0 0x7f44e01987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:41.883 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.879+0000 7f44e79ff700 1 -- 172.21.15.133:0/3928189131 >> 172.21.15.133:0/3928189131 conn(0x7f44e00fb8b0 msgr2=0x7f44e00fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:41.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.879+0000 7f44e79ff700 1 -- 172.21.15.133:0/3928189131 shutdown_connections 2023-11-27T15:22:41.884 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:41.879+0000 7f44e79ff700 1 -- 172.21.15.133:0/3928189131 wait complete. 2023-11-27T15:22:41.902 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:22:42.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:41 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/541655819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:42.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.101+0000 7fd7cbfff700 1 -- 172.21.15.133:0/252715395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7cc101c90 msgr2=0x7fd7cc108530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.101+0000 7fd7cbfff700 1 --2- 172.21.15.133:0/252715395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7cc101c90 0x7fd7cc108530 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400b0a0 tx=0x7fd7c400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.101+0000 7fd7cbfff700 1 -- 172.21.15.133:0/252715395 shutdown_connections 2023-11-27T15:22:42.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.101+0000 7fd7cbfff700 1 --2- 172.21.15.133:0/252715395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7cc101c90 0x7fd7cc108530 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.101+0000 7fd7cbfff700 1 --2- 172.21.15.133:0/252715395 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7cc100050 0x7fd7cc100460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.101+0000 7fd7cbfff700 1 --2- 172.21.15.133:0/252715395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7cc1012e0 0x7fd7cc101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.101+0000 7fd7cbfff700 1 -- 172.21.15.133:0/252715395 >> 172.21.15.133:0/252715395 conn(0x7fd7cc0fb650 msgr2=0x7fd7cc0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:42.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.101+0000 7fd7cbfff700 1 -- 172.21.15.133:0/252715395 shutdown_connections 2023-11-27T15:22:42.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.101+0000 7fd7cbfff700 1 -- 172.21.15.133:0/252715395 wait complete. 2023-11-27T15:22:42.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.102+0000 7fd7cbfff700 1 Processor -- start 2023-11-27T15:22:42.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.102+0000 7fd7cbfff700 1 -- start start 2023-11-27T15:22:42.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.103+0000 7fd7cbfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7cc100050 0x7fd7cc198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.103+0000 7fd7cbfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7cc1012e0 0x7fd7cc1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.103+0000 7fd7cbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7cc101c90 0x7fd7cc19f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.103+0000 7fd7cbfff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc19fbe0 con 0x7fd7cc101c90 2023-11-27T15:22:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.103+0000 7fd7cbfff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc19fd20 con 0x7fd7cc1012e0 2023-11-27T15:22:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.103+0000 7fd7cbfff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7cc19feb0 con 0x7fd7cc100050 2023-11-27T15:22:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.103+0000 7fd7ca7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7cc1012e0 0x7fd7cc1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.103+0000 7fd7cb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7cc101c90 0x7fd7cc19f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.103+0000 7fd7caffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7cc100050 0x7fd7cc198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.103+0000 7fd7ca7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7cc1012e0 0x7fd7cc1985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:37066/0 (socket says 172.21.15.133:37066) 2023-11-27T15:22:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.103+0000 7fd7caffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7cc100050 0x7fd7cc198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:50412/0 (socket says 172.21.15.133:50412) 2023-11-27T15:22:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.103+0000 7fd7ca7fc700 1 -- 172.21.15.133:0/2978417666 learned_addr learned my addr 172.21.15.133:0/2978417666 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.103+0000 7fd7cb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7cc101c90 0x7fd7cc19f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:45266/0 (socket says 172.21.15.133:45266) 2023-11-27T15:22:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.103+0000 7fd7ca7fc700 1 -- 172.21.15.133:0/2978417666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7cc100050 msgr2=0x7fd7cc198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.103+0000 7fd7ca7fc700 1 --2- 172.21.15.133:0/2978417666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7cc100050 0x7fd7cc198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.103+0000 7fd7ca7fc700 1 -- 172.21.15.133:0/2978417666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7cc101c90 msgr2=0x7fd7cc19f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.103+0000 7fd7ca7fc700 1 --2- 172.21.15.133:0/2978417666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7cc101c90 0x7fd7cc19f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.103+0000 7fd7ca7fc700 1 -- 172.21.15.133:0/2978417666 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c0009520 con 0x7fd7cc1012e0 2023-11-27T15:22:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.103+0000 7fd7cb7fe700 1 --2- 172.21.15.133:0/2978417666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7cc101c90 0x7fd7cc19f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.103+0000 7fd7caffd700 1 --2- 172.21.15.133:0/2978417666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7cc100050 0x7fd7cc198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.104+0000 7fd7ca7fc700 1 --2- 172.21.15.133:0/2978417666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7cc1012e0 0x7fd7cc1985a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000eb30 tx=0x7fd7c000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.104+0000 7fd7b3fff700 1 -- 172.21.15.133:0/2978417666 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0009880 con 0x7fd7cc1012e0 2023-11-27T15:22:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.104+0000 7fd7cbfff700 1 -- 172.21.15.133:0/2978417666 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c4009d00 con 0x7fd7cc1012e0 2023-11-27T15:22:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.104+0000 7fd7b3fff700 1 -- 172.21.15.133:0/2978417666 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7c00044e0 con 0x7fd7cc1012e0 2023-11-27T15:22:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.104+0000 7fd7b3fff700 1 -- 172.21.15.133:0/2978417666 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0003ce0 con 0x7fd7cc1012e0 2023-11-27T15:22:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.104+0000 7fd7cbfff700 1 -- 172.21.15.133:0/2978417666 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7cc10ba10 con 0x7fd7cc1012e0 2023-11-27T15:22:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.105+0000 7fd7cbfff700 1 -- 172.21.15.133:0/2978417666 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7cc067980 con 0x7fd7cc1012e0 2023-11-27T15:22:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.107+0000 7fd7b3fff700 1 -- 172.21.15.133:0/2978417666 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd7c0004020 con 0x7fd7cc1012e0 2023-11-27T15:22:42.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.108+0000 7fd7b3fff700 1 --2- 172.21.15.133:0/2978417666 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd7b40a11d0 0x7fd7b40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.108+0000 7fd7b3fff700 1 -- 172.21.15.133:0/2978417666 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd7c0014070 con 0x7fd7cc1012e0 2023-11-27T15:22:42.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.109+0000 7fd7caffd700 1 --2- 172.21.15.133:0/2978417666 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd7b40a11d0 0x7fd7b40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.109+0000 7fd7caffd700 1 --2- 172.21.15.133:0/2978417666 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd7b40a11d0 0x7fd7b40a3680 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc011490 tx=0x7fd7bc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:42.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.110+0000 7fd7b3fff700 1 -- 172.21.15.133:0/2978417666 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd7c0096a90 con 0x7fd7cc1012e0 2023-11-27T15:22:42.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:41 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/541655819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:42.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:41 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/541655819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:42.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.311+0000 7fd7cbfff700 1 -- 172.21.15.133:0/2978417666 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd7cc1a0b60 con 0x7fd7cc1012e0 2023-11-27T15:22:42.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.312+0000 7fd7b3fff700 1 -- 172.21.15.133:0/2978417666 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd7c009a0b0 con 0x7fd7cc1012e0 2023-11-27T15:22:42.316 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:22:42.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.314+0000 7fd7cbfff700 1 -- 172.21.15.133:0/2978417666 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd7b40a11d0 msgr2=0x7fd7b40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.314+0000 7fd7cbfff700 1 --2- 172.21.15.133:0/2978417666 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd7b40a11d0 0x7fd7b40a3680 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc011490 tx=0x7fd7bc009260 comp rx=0 tx=0).stop 2023-11-27T15:22:42.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.314+0000 7fd7cbfff700 1 -- 172.21.15.133:0/2978417666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7cc1012e0 msgr2=0x7fd7cc1985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.314+0000 7fd7cbfff700 1 --2- 172.21.15.133:0/2978417666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7cc1012e0 0x7fd7cc1985a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000eb30 tx=0x7fd7c000ee40 comp rx=0 tx=0).stop 2023-11-27T15:22:42.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.314+0000 7fd7cbfff700 1 -- 172.21.15.133:0/2978417666 shutdown_connections 2023-11-27T15:22:42.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.315+0000 7fd7cbfff700 1 --2- 172.21.15.133:0/2978417666 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd7b40a11d0 0x7fd7b40a3680 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.315+0000 7fd7cbfff700 1 --2- 172.21.15.133:0/2978417666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd7cc101c90 0x7fd7cc19f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.315+0000 7fd7cbfff700 1 --2- 172.21.15.133:0/2978417666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd7cc100050 0x7fd7cc198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.315+0000 7fd7cbfff700 1 --2- 172.21.15.133:0/2978417666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7cc1012e0 0x7fd7cc1985a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.315+0000 7fd7cbfff700 1 -- 172.21.15.133:0/2978417666 >> 172.21.15.133:0/2978417666 conn(0x7fd7cc0fb650 msgr2=0x7fd7cc0fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:42.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.315+0000 7fd7cbfff700 1 -- 172.21.15.133:0/2978417666 shutdown_connections 2023-11-27T15:22:42.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.315+0000 7fd7cbfff700 1 -- 172.21.15.133:0/2978417666 wait complete. 2023-11-27T15:22:42.337 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:22:42.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.534+0000 7f64cb5ff700 1 -- 172.21.15.133:0/2489890344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64c41014c0 msgr2=0x7f64c4101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.534+0000 7f64cb5ff700 1 --2- 172.21.15.133:0/2489890344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64c41014c0 0x7f64c4101910 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f64b4009990 tx=0x7f64b4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.534+0000 7f64cb5ff700 1 -- 172.21.15.133:0/2489890344 shutdown_connections 2023-11-27T15:22:42.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.534+0000 7f64cb5ff700 1 --2- 172.21.15.133:0/2489890344 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64c4101e50 0x7f64c4108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.534+0000 7f64cb5ff700 1 --2- 172.21.15.133:0/2489890344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64c41002d0 0x7f64c41006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.534+0000 7f64cb5ff700 1 --2- 172.21.15.133:0/2489890344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64c41014c0 0x7f64c4101910 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.534+0000 7f64cb5ff700 1 -- 172.21.15.133:0/2489890344 >> 172.21.15.133:0/2489890344 conn(0x7f64c40fb8b0 msgr2=0x7f64c40fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:42.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.534+0000 7f64cb5ff700 1 -- 172.21.15.133:0/2489890344 shutdown_connections 2023-11-27T15:22:42.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.534+0000 7f64cb5ff700 1 -- 172.21.15.133:0/2489890344 wait complete. 2023-11-27T15:22:42.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.535+0000 7f64cb5ff700 1 Processor -- start 2023-11-27T15:22:42.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.535+0000 7f64cb5ff700 1 -- start start 2023-11-27T15:22:42.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.535+0000 7f64cb5ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64c41002d0 0x7f64c4196080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.535+0000 7f64cb5ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64c41014c0 0x7f64c41965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.535+0000 7f64cb5ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64c4101e50 0x7f64c419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.535+0000 7f64cb5ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c419dc00 con 0x7f64c41014c0 2023-11-27T15:22:42.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.535+0000 7f64cb5ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c419dd40 con 0x7f64c41002d0 2023-11-27T15:22:42.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.535+0000 7f64cb5ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c419ded0 con 0x7f64c4101e50 2023-11-27T15:22:42.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.536+0000 7f64c939b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64c41002d0 0x7f64c4196080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.536+0000 7f64c8b9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64c41014c0 0x7f64c41965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.536+0000 7f64c939b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64c41002d0 0x7f64c4196080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:37090/0 (socket says 172.21.15.133:37090) 2023-11-27T15:22:42.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.536+0000 7f64c939b700 1 -- 172.21.15.133:0/2736629222 learned_addr learned my addr 172.21.15.133:0/2736629222 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:42.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.536+0000 7f64c9b9c700 1 --2- 172.21.15.133:0/2736629222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64c4101e50 0x7f64c419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.536+0000 7f64c8b9a700 1 -- 172.21.15.133:0/2736629222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64c4101e50 msgr2=0x7f64c419d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.536+0000 7f64c8b9a700 1 --2- 172.21.15.133:0/2736629222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64c4101e50 0x7f64c419d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.536+0000 7f64c8b9a700 1 -- 172.21.15.133:0/2736629222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64c41002d0 msgr2=0x7f64c4196080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.536+0000 7f64c8b9a700 1 --2- 172.21.15.133:0/2736629222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64c41002d0 0x7f64c4196080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.536+0000 7f64c8b9a700 1 -- 172.21.15.133:0/2736629222 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b40095f0 con 0x7f64c41014c0 2023-11-27T15:22:42.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.536+0000 7f64c939b700 1 --2- 172.21.15.133:0/2736629222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64c41002d0 0x7f64c4196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:42.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.536+0000 7f64c8b9a700 1 --2- 172.21.15.133:0/2736629222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64c41014c0 0x7f64c41965c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f64b40098e0 tx=0x7f64b4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:42.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.536+0000 7f64ba7fc700 1 -- 172.21.15.133:0/2736629222 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b401d070 con 0x7f64c41014c0 2023-11-27T15:22:42.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.537+0000 7f64ba7fc700 1 -- 172.21.15.133:0/2736629222 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64b4021410 con 0x7f64c41014c0 2023-11-27T15:22:42.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.537+0000 7f64cb5ff700 1 -- 172.21.15.133:0/2736629222 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c410bc80 con 0x7f64c41014c0 2023-11-27T15:22:42.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.537+0000 7f64ba7fc700 1 -- 172.21.15.133:0/2736629222 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b4017650 con 0x7f64c41014c0 2023-11-27T15:22:42.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.537+0000 7f64cb5ff700 1 -- 172.21.15.133:0/2736629222 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c410baf0 con 0x7f64c41014c0 2023-11-27T15:22:42.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.538+0000 7f64cb5ff700 1 -- 172.21.15.133:0/2736629222 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c419e470 con 0x7f64c41014c0 2023-11-27T15:22:42.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.540+0000 7f64ba7fc700 1 -- 172.21.15.133:0/2736629222 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f64b4021580 con 0x7f64c41014c0 2023-11-27T15:22:42.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.541+0000 7f64ba7fc700 1 --2- 172.21.15.133:0/2736629222 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f64b00a1150 0x7f64b00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.541+0000 7f64ba7fc700 1 -- 172.21.15.133:0/2736629222 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f64b40cdfe0 con 0x7f64c41014c0 2023-11-27T15:22:42.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.541+0000 7f64c939b700 1 --2- 172.21.15.133:0/2736629222 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f64b00a1150 0x7f64b00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.542+0000 7f64c939b700 1 --2- 172.21.15.133:0/2736629222 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f64b00a1150 0x7f64b00a3600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f64c0003ef0 tx=0x7f64c000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:42.549 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.545+0000 7f64ba7fc700 1 -- 172.21.15.133:0/2736629222 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f64b4097450 con 0x7f64c41014c0 2023-11-27T15:22:42.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.744+0000 7f64cb5ff700 1 -- 172.21.15.133:0/2736629222 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64c4107200 con 0x7f64c41014c0 2023-11-27T15:22:42.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.744+0000 7f64ba7fc700 1 -- 172.21.15.133:0/2736629222 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f64b4026020 con 0x7f64c41014c0 2023-11-27T15:22:42.748 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-11-27T15:22:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.747+0000 7f64cb5ff700 1 -- 172.21.15.133:0/2736629222 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f64b00a1150 msgr2=0x7f64b00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.747+0000 7f64cb5ff700 1 --2- 172.21.15.133:0/2736629222 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f64b00a1150 0x7f64b00a3600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f64c0003ef0 tx=0x7f64c000b040 comp rx=0 tx=0).stop 2023-11-27T15:22:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.747+0000 7f64cb5ff700 1 -- 172.21.15.133:0/2736629222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64c41014c0 msgr2=0x7f64c41965c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.747+0000 7f64cb5ff700 1 --2- 172.21.15.133:0/2736629222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64c41014c0 0x7f64c41965c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f64b40098e0 tx=0x7f64b4005740 comp rx=0 tx=0).stop 2023-11-27T15:22:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.747+0000 7f64cb5ff700 1 -- 172.21.15.133:0/2736629222 shutdown_connections 2023-11-27T15:22:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.747+0000 7f64cb5ff700 1 --2- 172.21.15.133:0/2736629222 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f64b00a1150 0x7f64b00a3600 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.747+0000 7f64cb5ff700 1 --2- 172.21.15.133:0/2736629222 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64c4101e50 0x7f64c419d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.747+0000 7f64cb5ff700 1 --2- 172.21.15.133:0/2736629222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64c41002d0 0x7f64c4196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.747+0000 7f64cb5ff700 1 --2- 172.21.15.133:0/2736629222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64c41014c0 0x7f64c41965c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.747+0000 7f64cb5ff700 1 -- 172.21.15.133:0/2736629222 >> 172.21.15.133:0/2736629222 conn(0x7f64c40fb8b0 msgr2=0x7f64c40fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.747+0000 7f64cb5ff700 1 -- 172.21.15.133:0/2736629222 shutdown_connections 2023-11-27T15:22:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.747+0000 7f64cb5ff700 1 -- 172.21.15.133:0/2736629222 wait complete. 2023-11-27T15:22:42.769 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-27T15:22:42.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.966+0000 7f58dd1ff700 1 -- 172.21.15.133:0/4227622326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58d81014b0 msgr2=0x7f58d8101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.966+0000 7f58dd1ff700 1 --2- 172.21.15.133:0/4227622326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58d81014b0 0x7f58d8101900 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f58c0009990 tx=0x7f58c0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.966+0000 7f58dd1ff700 1 -- 172.21.15.133:0/4227622326 shutdown_connections 2023-11-27T15:22:42.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.966+0000 7f58dd1ff700 1 --2- 172.21.15.133:0/4227622326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58d8101e40 0x7f58d8108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.966+0000 7f58dd1ff700 1 --2- 172.21.15.133:0/4227622326 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d81002c0 0x7f58d81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.966+0000 7f58dd1ff700 1 --2- 172.21.15.133:0/4227622326 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58d81014b0 0x7f58d8101900 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.966+0000 7f58dd1ff700 1 -- 172.21.15.133:0/4227622326 >> 172.21.15.133:0/4227622326 conn(0x7f58d80fb8a0 msgr2=0x7f58d80fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:42.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.966+0000 7f58dd1ff700 1 -- 172.21.15.133:0/4227622326 shutdown_connections 2023-11-27T15:22:42.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.966+0000 7f58dd1ff700 1 -- 172.21.15.133:0/4227622326 wait complete. 2023-11-27T15:22:42.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.967+0000 7f58dd1ff700 1 Processor -- start 2023-11-27T15:22:42.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.967+0000 7f58dd1ff700 1 -- start start 2023-11-27T15:22:42.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.968+0000 7f58dd1ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58d81002c0 0x7f58d8198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.968+0000 7f58dd1ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d81014b0 0x7f58d81987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.968+0000 7f58dd1ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58d8101e40 0x7f58d819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.968+0000 7f58dd1ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d819fdf0 con 0x7f58d81002c0 2023-11-27T15:22:42.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.968+0000 7f58dd1ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d819ff30 con 0x7f58d8101e40 2023-11-27T15:22:42.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.968+0000 7f58dd1ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d81a00c0 con 0x7f58d81014b0 2023-11-27T15:22:42.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.968+0000 7f58d6d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58d81002c0 0x7f58d8198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.968+0000 7f58d6d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58d81002c0 0x7f58d8198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60304/0 (socket says 172.21.15.133:60304) 2023-11-27T15:22:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.968+0000 7f58d6d9d700 1 -- 172.21.15.133:0/1255396288 learned_addr learned my addr 172.21.15.133:0/1255396288 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.968+0000 7f58d759e700 1 --2- 172.21.15.133:0/1255396288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58d8101e40 0x7f58d819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.968+0000 7f58cffff700 1 --2- 172.21.15.133:0/1255396288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d81014b0 0x7f58d81987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.968+0000 7f58d6d9d700 1 -- 172.21.15.133:0/1255396288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d81014b0 msgr2=0x7f58d81987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.968+0000 7f58d6d9d700 1 --2- 172.21.15.133:0/1255396288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d81014b0 0x7f58d81987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.968+0000 7f58d6d9d700 1 -- 172.21.15.133:0/1255396288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58d8101e40 msgr2=0x7f58d819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.968+0000 7f58d6d9d700 1 --2- 172.21.15.133:0/1255396288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58d8101e40 0x7f58d819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.968+0000 7f58d6d9d700 1 -- 172.21.15.133:0/1255396288 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c00095f0 con 0x7f58d81002c0 2023-11-27T15:22:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.969+0000 7f58d6d9d700 1 --2- 172.21.15.133:0/1255396288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58d81002c0 0x7f58d8198270 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f58c800e970 tx=0x7f58c800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.969+0000 7f58d4d99700 1 -- 172.21.15.133:0/1255396288 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8009790 con 0x7f58d81002c0 2023-11-27T15:22:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.969+0000 7f58d4d99700 1 -- 172.21.15.133:0/1255396288 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58c800c760 con 0x7f58d81002c0 2023-11-27T15:22:42.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.969+0000 7f58dd1ff700 1 -- 172.21.15.133:0/1255396288 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d810bcd0 con 0x7f58d81002c0 2023-11-27T15:22:42.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.969+0000 7f58d4d99700 1 -- 172.21.15.133:0/1255396288 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8003bb0 con 0x7f58d81002c0 2023-11-27T15:22:42.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.969+0000 7f58dd1ff700 1 -- 172.21.15.133:0/1255396288 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d810bb40 con 0x7f58d81002c0 2023-11-27T15:22:42.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.970+0000 7f58dd1ff700 1 -- 172.21.15.133:0/1255396288 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d81a0860 con 0x7f58d81002c0 2023-11-27T15:22:42.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.972+0000 7f58d4d99700 1 -- 172.21.15.133:0/1255396288 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f58c8003d50 con 0x7f58d81002c0 2023-11-27T15:22:42.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.973+0000 7f58d4d99700 1 --2- 172.21.15.133:0/1255396288 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f58c40a1220 0x7f58c40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:42.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.973+0000 7f58d4d99700 1 -- 172.21.15.133:0/1255396288 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f58c8014070 con 0x7f58d81002c0 2023-11-27T15:22:42.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.973+0000 7f58cffff700 1 --2- 172.21.15.133:0/1255396288 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f58c40a1220 0x7f58c40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:42.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.974+0000 7f58cffff700 1 --2- 172.21.15.133:0/1255396288 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f58c40a1220 0x7f58c40a36d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f58c000b2e0 tx=0x7f58c0005ad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:42.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:42.977+0000 7f58d4d99700 1 -- 172.21.15.133:0/1255396288 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f58c80960f0 con 0x7f58d81002c0 2023-11-27T15:22:43.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:42 smithi133 ceph-mon[97860]: pgmap v342: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2023-11-27T15:22:43.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:42 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3928189131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:43.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:42 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2978417666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:22:43.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:42 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2736629222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:22:43.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:42 smithi177 ceph-mon[131504]: pgmap v342: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2023-11-27T15:22:43.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:42 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3928189131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:43.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:42 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2978417666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:22:43.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:42 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2736629222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:22:43.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:42 smithi174 ceph-mon[131110]: pgmap v342: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2023-11-27T15:22:43.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:42 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3928189131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:43.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:42 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2978417666' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:22:43.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:42 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2736629222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:22:43.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.178+0000 7f58dd1ff700 1 -- 172.21.15.133:0/1255396288 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f58d81a0e10 con 0x7f58d81002c0 2023-11-27T15:22:43.181 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.178+0000 7f58d4d99700 1 -- 172.21.15.133:0/1255396288 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f58c8099710 con 0x7f58d81002c0 2023-11-27T15:22:43.182 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:22:43.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.180+0000 7f58dd1ff700 1 -- 172.21.15.133:0/1255396288 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f58c40a1220 msgr2=0x7f58c40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:43.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.180+0000 7f58dd1ff700 1 --2- 172.21.15.133:0/1255396288 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f58c40a1220 0x7f58c40a36d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f58c000b2e0 tx=0x7f58c0005ad0 comp rx=0 tx=0).stop 2023-11-27T15:22:43.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.180+0000 7f58dd1ff700 1 -- 172.21.15.133:0/1255396288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58d81002c0 msgr2=0x7f58d8198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:43.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.180+0000 7f58dd1ff700 1 --2- 172.21.15.133:0/1255396288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58d81002c0 0x7f58d8198270 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f58c800e970 tx=0x7f58c800ec80 comp rx=0 tx=0).stop 2023-11-27T15:22:43.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.180+0000 7f58dd1ff700 1 -- 172.21.15.133:0/1255396288 shutdown_connections 2023-11-27T15:22:43.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.180+0000 7f58dd1ff700 1 --2- 172.21.15.133:0/1255396288 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f58c40a1220 0x7f58c40a36d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:43.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.180+0000 7f58dd1ff700 1 --2- 172.21.15.133:0/1255396288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f58d8101e40 0x7f58d819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:43.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.180+0000 7f58dd1ff700 1 --2- 172.21.15.133:0/1255396288 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f58d81002c0 0x7f58d8198270 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:43.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.180+0000 7f58dd1ff700 1 --2- 172.21.15.133:0/1255396288 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f58d81014b0 0x7f58d81987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:43.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.180+0000 7f58dd1ff700 1 -- 172.21.15.133:0/1255396288 >> 172.21.15.133:0/1255396288 conn(0x7f58d80fb8a0 msgr2=0x7f58d8106d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:43.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.181+0000 7f58dd1ff700 1 -- 172.21.15.133:0/1255396288 shutdown_connections 2023-11-27T15:22:43.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.181+0000 7f58dd1ff700 1 -- 172.21.15.133:0/1255396288 wait complete. 2023-11-27T15:22:43.204 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:22:43.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.399+0000 7fdbc1bff700 1 -- 172.21.15.133:0/3642863596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbbc101480 msgr2=0x7fdbbc1018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:43.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.399+0000 7fdbc1bff700 1 --2- 172.21.15.133:0/3642863596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbbc101480 0x7fdbbc1018d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fdba40098f0 tx=0x7fdba4009c00 comp rx=0 tx=0).stop 2023-11-27T15:22:43.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.399+0000 7fdbc1bff700 1 -- 172.21.15.133:0/3642863596 shutdown_connections 2023-11-27T15:22:43.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.399+0000 7fdbc1bff700 1 --2- 172.21.15.133:0/3642863596 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbbc101e10 0x7fdbbc1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:43.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.399+0000 7fdbc1bff700 1 --2- 172.21.15.133:0/3642863596 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbbc100290 0x7fdbbc1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:43.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.399+0000 7fdbc1bff700 1 --2- 172.21.15.133:0/3642863596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbbc101480 0x7fdbbc1018d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:43.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.399+0000 7fdbc1bff700 1 -- 172.21.15.133:0/3642863596 >> 172.21.15.133:0/3642863596 conn(0x7fdbbc0fb870 msgr2=0x7fdbbc0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:43.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.399+0000 7fdbc1bff700 1 -- 172.21.15.133:0/3642863596 shutdown_connections 2023-11-27T15:22:43.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.399+0000 7fdbc1bff700 1 -- 172.21.15.133:0/3642863596 wait complete. 2023-11-27T15:22:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.400+0000 7fdbc1bff700 1 Processor -- start 2023-11-27T15:22:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.400+0000 7fdbc1bff700 1 -- start start 2023-11-27T15:22:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.400+0000 7fdbc1bff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbbc100290 0x7fdbbc198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.400+0000 7fdbc1bff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbbc101480 0x7fdbbc198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.400+0000 7fdbc1bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbbc101e10 0x7fdbbc19f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.401+0000 7fdbc1bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc19fdc0 con 0x7fdbbc101e10 2023-11-27T15:22:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.401+0000 7fdbc1bff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc19ff00 con 0x7fdbbc100290 2023-11-27T15:22:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.401+0000 7fdbc1bff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc1a0090 con 0x7fdbbc101480 2023-11-27T15:22:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.401+0000 7fdbbbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbbc101e10 0x7fdbbc19f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:43.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.401+0000 7fdbbbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbbc101e10 0x7fdbbc19f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60316/0 (socket says 172.21.15.133:60316) 2023-11-27T15:22:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.401+0000 7fdbbbfff700 1 -- 172.21.15.133:0/2832041177 learned_addr learned my addr 172.21.15.133:0/2832041177 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.401+0000 7fdbb3fff700 1 --2- 172.21.15.133:0/2832041177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbbc101480 0x7fdbbc198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.401+0000 7fdbbb7fe700 1 --2- 172.21.15.133:0/2832041177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbbc100290 0x7fdbbc198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.401+0000 7fdbbbfff700 1 -- 172.21.15.133:0/2832041177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbbc101480 msgr2=0x7fdbbc198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.401+0000 7fdbbbfff700 1 --2- 172.21.15.133:0/2832041177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbbc101480 0x7fdbbc198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.401+0000 7fdbbbfff700 1 -- 172.21.15.133:0/2832041177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbbc100290 msgr2=0x7fdbbc198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.401+0000 7fdbbbfff700 1 --2- 172.21.15.133:0/2832041177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbbc100290 0x7fdbbc198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.401+0000 7fdbbbfff700 1 -- 172.21.15.133:0/2832041177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb4009d00 con 0x7fdbbc101e10 2023-11-27T15:22:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.401+0000 7fdbbbfff700 1 --2- 172.21.15.133:0/2832041177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbbc101e10 0x7fdbbc19f7f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400e3c0 tx=0x7fdbb400e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.402+0000 7fdbb97fa700 1 -- 172.21.15.133:0/2832041177 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb40092d0 con 0x7fdbbc101e10 2023-11-27T15:22:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.402+0000 7fdbc1bff700 1 -- 172.21.15.133:0/2832041177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba4009520 con 0x7fdbbc101e10 2023-11-27T15:22:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.402+0000 7fdbb97fa700 1 -- 172.21.15.133:0/2832041177 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbb400f040 con 0x7fdbbc101e10 2023-11-27T15:22:43.407 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.402+0000 7fdbb97fa700 1 -- 172.21.15.133:0/2832041177 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4004dd0 con 0x7fdbbc101e10 2023-11-27T15:22:43.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.402+0000 7fdbc1bff700 1 -- 172.21.15.133:0/2832041177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbbc10bc50 con 0x7fdbbc101e10 2023-11-27T15:22:43.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.403+0000 7fdbc1bff700 1 -- 172.21.15.133:0/2832041177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbbc1928e0 con 0x7fdbbc101e10 2023-11-27T15:22:43.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.404+0000 7fdbb97fa700 1 -- 172.21.15.133:0/2832041177 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fdbb4007a60 con 0x7fdbbc101e10 2023-11-27T15:22:43.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.406+0000 7fdbb97fa700 1 --2- 172.21.15.133:0/2832041177 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fdba80a1150 0x7fdba80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:43.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.406+0000 7fdbb97fa700 1 -- 172.21.15.133:0/2832041177 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fdbb40cccb0 con 0x7fdbbc101e10 2023-11-27T15:22:43.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.406+0000 7fdbbb7fe700 1 --2- 172.21.15.133:0/2832041177 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fdba80a1150 0x7fdba80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:43.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.407+0000 7fdbbb7fe700 1 --2- 172.21.15.133:0/2832041177 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fdba80a1150 0x7fdba80a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc1012e0 tx=0x7fdbac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:43.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.409+0000 7fdbb97fa700 1 -- 172.21.15.133:0/2832041177 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdbb40960e0 con 0x7fdbbc101e10 2023-11-27T15:22:43.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.610+0000 7fdbc1bff700 1 -- 172.21.15.133:0/2832041177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbbc067a60 con 0x7fdbbc101e10 2023-11-27T15:22:43.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.612+0000 7fdbb97fa700 1 -- 172.21.15.133:0/2832041177 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24616 (secure 0 0 0) 0x7fdbb4099700 con 0x7fdbbc101e10 2023-11-27T15:22:43.616 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:43.616 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":92,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:22:02.807005+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:22:43.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.614+0000 7fdbc1bff700 1 -- 172.21.15.133:0/2832041177 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fdba80a1150 msgr2=0x7fdba80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:43.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.614+0000 7fdbc1bff700 1 --2- 172.21.15.133:0/2832041177 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fdba80a1150 0x7fdba80a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc1012e0 tx=0x7fdbac009260 comp rx=0 tx=0).stop 2023-11-27T15:22:43.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.614+0000 7fdbc1bff700 1 -- 172.21.15.133:0/2832041177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbbc101e10 msgr2=0x7fdbbc19f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:43.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.614+0000 7fdbc1bff700 1 --2- 172.21.15.133:0/2832041177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbbc101e10 0x7fdbbc19f7f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400e3c0 tx=0x7fdbb400e6d0 comp rx=0 tx=0).stop 2023-11-27T15:22:43.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.614+0000 7fdbc1bff700 1 -- 172.21.15.133:0/2832041177 shutdown_connections 2023-11-27T15:22:43.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.614+0000 7fdbc1bff700 1 --2- 172.21.15.133:0/2832041177 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fdba80a1150 0x7fdba80a3600 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:43.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.614+0000 7fdbc1bff700 1 --2- 172.21.15.133:0/2832041177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdbbc101e10 0x7fdbbc19f7f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:43.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.614+0000 7fdbc1bff700 1 --2- 172.21.15.133:0/2832041177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbbc100290 0x7fdbbc198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:43.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.614+0000 7fdbc1bff700 1 --2- 172.21.15.133:0/2832041177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdbbc101480 0x7fdbbc198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:43.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.614+0000 7fdbc1bff700 1 -- 172.21.15.133:0/2832041177 >> 172.21.15.133:0/2832041177 conn(0x7fdbbc0fb870 msgr2=0x7fdbbc106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:43.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.614+0000 7fdbc1bff700 1 -- 172.21.15.133:0/2832041177 shutdown_connections 2023-11-27T15:22:43.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.614+0000 7fdbc1bff700 1 -- 172.21.15.133:0/2832041177 wait complete. 2023-11-27T15:22:43.637 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:16:20.216150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:20:31.107146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:20:32.614089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-27T15:20:36.126040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-27T15:22:43.638 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:22:43.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.837+0000 7f2b251ff700 1 -- 172.21.15.133:0/3683271658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b201014c0 msgr2=0x7f2b20101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:43.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.837+0000 7f2b251ff700 1 --2- 172.21.15.133:0/3683271658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b201014c0 0x7f2b20101910 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2b140098f0 tx=0x7f2b14009c00 comp rx=0 tx=0).stop 2023-11-27T15:22:43.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.838+0000 7f2b251ff700 1 -- 172.21.15.133:0/3683271658 shutdown_connections 2023-11-27T15:22:43.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.838+0000 7f2b251ff700 1 --2- 172.21.15.133:0/3683271658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b20101e50 0x7f2b20108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:43.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.838+0000 7f2b251ff700 1 --2- 172.21.15.133:0/3683271658 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b201002d0 0x7f2b201006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:43.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.838+0000 7f2b251ff700 1 --2- 172.21.15.133:0/3683271658 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b201014c0 0x7f2b20101910 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:43.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.838+0000 7f2b251ff700 1 -- 172.21.15.133:0/3683271658 >> 172.21.15.133:0/3683271658 conn(0x7f2b200fb8b0 msgr2=0x7f2b200fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:43.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.838+0000 7f2b251ff700 1 -- 172.21.15.133:0/3683271658 shutdown_connections 2023-11-27T15:22:43.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.838+0000 7f2b251ff700 1 -- 172.21.15.133:0/3683271658 wait complete. 2023-11-27T15:22:43.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.839+0000 7f2b251ff700 1 Processor -- start 2023-11-27T15:22:43.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.839+0000 7f2b251ff700 1 -- start start 2023-11-27T15:22:43.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.839+0000 7f2b251ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b201002d0 0x7f2b200748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:43.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.839+0000 7f2b251ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b201014c0 0x7f2b20072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:43.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.839+0000 7f2b251ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b20101e50 0x7f2b20073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:43.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.839+0000 7f2b251ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b20073bc0 con 0x7f2b201014c0 2023-11-27T15:22:43.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.839+0000 7f2b251ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b20073d30 con 0x7f2b201002d0 2023-11-27T15:22:43.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.839+0000 7f2b251ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b201a0190 con 0x7f2b20101e50 2023-11-27T15:22:43.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.839+0000 7f2b1ed9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b201002d0 0x7f2b200748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:43.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.839+0000 7f2b1f59e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b20101e50 0x7f2b20073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:43.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.839+0000 7f2b1e59c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b201014c0 0x7f2b20072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:43.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.839+0000 7f2b1ed9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b201002d0 0x7f2b200748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:45076/0 (socket says 172.21.15.133:45076) 2023-11-27T15:22:43.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.839+0000 7f2b1f59e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b20101e50 0x7f2b20073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:46090/0 (socket says 172.21.15.133:46090) 2023-11-27T15:22:43.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.839+0000 7f2b1ed9d700 1 -- 172.21.15.133:0/2784104590 learned_addr learned my addr 172.21.15.133:0/2784104590 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:43.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.840+0000 7f2b1e59c700 1 -- 172.21.15.133:0/2784104590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b20101e50 msgr2=0x7f2b20073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:43.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.840+0000 7f2b1e59c700 1 --2- 172.21.15.133:0/2784104590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b20101e50 0x7f2b20073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:43.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.840+0000 7f2b1e59c700 1 -- 172.21.15.133:0/2784104590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b201002d0 msgr2=0x7f2b200748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:43.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.840+0000 7f2b1e59c700 1 --2- 172.21.15.133:0/2784104590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b201002d0 0x7f2b200748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:43.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.840+0000 7f2b1e59c700 1 -- 172.21.15.133:0/2784104590 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b14009520 con 0x7f2b201014c0 2023-11-27T15:22:43.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.840+0000 7f2b1ed9d700 1 --2- 172.21.15.133:0/2784104590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b201002d0 0x7f2b200748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:43.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.840+0000 7f2b1e59c700 1 --2- 172.21.15.133:0/2784104590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b201014c0 0x7f2b20072ee0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009230 tx=0x7f2b1400b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:43.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.840+0000 7f2b07fff700 1 -- 172.21.15.133:0/2784104590 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1401c070 con 0x7f2b201014c0 2023-11-27T15:22:43.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.840+0000 7f2b07fff700 1 -- 172.21.15.133:0/2784104590 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b1400bbb0 con 0x7f2b201014c0 2023-11-27T15:22:43.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.840+0000 7f2b07fff700 1 -- 172.21.15.133:0/2784104590 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14003a00 con 0x7f2b201014c0 2023-11-27T15:22:43.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.840+0000 7f2b251ff700 1 -- 172.21.15.133:0/2784104590 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2010bc80 con 0x7f2b201014c0 2023-11-27T15:22:43.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.840+0000 7f2b251ff700 1 -- 172.21.15.133:0/2784104590 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2010baf0 con 0x7f2b201014c0 2023-11-27T15:22:43.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.841+0000 7f2b251ff700 1 -- 172.21.15.133:0/2784104590 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b201a06b0 con 0x7f2b201014c0 2023-11-27T15:22:43.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.843+0000 7f2b07fff700 1 -- 172.21.15.133:0/2784104590 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2b14020030 con 0x7f2b201014c0 2023-11-27T15:22:43.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.845+0000 7f2b07fff700 1 --2- 172.21.15.133:0/2784104590 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2b080a1150 0x7f2b080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:43.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.845+0000 7f2b07fff700 1 -- 172.21.15.133:0/2784104590 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f2b140cdc00 con 0x7f2b201014c0 2023-11-27T15:22:43.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.845+0000 7f2b1ed9d700 1 --2- 172.21.15.133:0/2784104590 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2b080a1150 0x7f2b080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:43.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.846+0000 7f2b1ed9d700 1 --2- 172.21.15.133:0/2784104590 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2b080a1150 0x7f2b080a3600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009fd0 tx=0x7f2b10009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:43.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:43.848+0000 7f2b07fff700 1 -- 172.21.15.133:0/2784104590 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2b14097030 con 0x7f2b201014c0 2023-11-27T15:22:44.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:43 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1255396288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:22:44.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:43 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2832041177' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:44.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.048+0000 7f2b251ff700 1 -- 172.21.15.133:0/2784104590 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2b20107250 con 0x7f2b201014c0 2023-11-27T15:22:44.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.048+0000 7f2b07fff700 1 -- 172.21.15.133:0/2784104590 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2b1409a650 con 0x7f2b201014c0 2023-11-27T15:22:44.052 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-11-27T15:22:44.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.051+0000 7f2b251ff700 1 -- 172.21.15.133:0/2784104590 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2b080a1150 msgr2=0x7f2b080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:44.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.051+0000 7f2b251ff700 1 --2- 172.21.15.133:0/2784104590 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2b080a1150 0x7f2b080a3600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009fd0 tx=0x7f2b10009310 comp rx=0 tx=0).stop 2023-11-27T15:22:44.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.051+0000 7f2b251ff700 1 -- 172.21.15.133:0/2784104590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b201014c0 msgr2=0x7f2b20072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:44.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.051+0000 7f2b251ff700 1 --2- 172.21.15.133:0/2784104590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b201014c0 0x7f2b20072ee0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009230 tx=0x7f2b1400b500 comp rx=0 tx=0).stop 2023-11-27T15:22:44.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.051+0000 7f2b251ff700 1 -- 172.21.15.133:0/2784104590 shutdown_connections 2023-11-27T15:22:44.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.051+0000 7f2b251ff700 1 --2- 172.21.15.133:0/2784104590 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2b080a1150 0x7f2b080a3600 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.051+0000 7f2b251ff700 1 --2- 172.21.15.133:0/2784104590 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2b20101e50 0x7f2b20073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.051+0000 7f2b251ff700 1 --2- 172.21.15.133:0/2784104590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b201002d0 0x7f2b200748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.051+0000 7f2b251ff700 1 --2- 172.21.15.133:0/2784104590 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2b201014c0 0x7f2b20072ee0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.051+0000 7f2b251ff700 1 -- 172.21.15.133:0/2784104590 >> 172.21.15.133:0/2784104590 conn(0x7f2b200fb8b0 msgr2=0x7f2b200fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:44.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.051+0000 7f2b251ff700 1 -- 172.21.15.133:0/2784104590 shutdown_connections 2023-11-27T15:22:44.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.051+0000 7f2b251ff700 1 -- 172.21.15.133:0/2784104590 wait complete. 2023-11-27T15:22:44.074 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:22:44.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:43 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1255396288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:22:44.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:43 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2832041177' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:44.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:43 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1255396288' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:22:44.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:43 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2832041177' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:44.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.272+0000 7f03b13ff700 1 -- 172.21.15.133:0/3679293370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03ac101e50 msgr2=0x7f03ac108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:44.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.272+0000 7f03b13ff700 1 --2- 172.21.15.133:0/3679293370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03ac101e50 0x7f03ac108710 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f03a400b0a0 tx=0x7f03a400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.273+0000 7f03b13ff700 1 -- 172.21.15.133:0/3679293370 shutdown_connections 2023-11-27T15:22:44.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.273+0000 7f03b13ff700 1 --2- 172.21.15.133:0/3679293370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03ac101e50 0x7f03ac108710 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.273+0000 7f03b13ff700 1 --2- 172.21.15.133:0/3679293370 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03ac1002d0 0x7f03ac1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.273+0000 7f03b13ff700 1 --2- 172.21.15.133:0/3679293370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03ac1014c0 0x7f03ac101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.273+0000 7f03b13ff700 1 -- 172.21.15.133:0/3679293370 >> 172.21.15.133:0/3679293370 conn(0x7f03ac0fb8b0 msgr2=0x7f03ac0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:44.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.273+0000 7f03b13ff700 1 -- 172.21.15.133:0/3679293370 shutdown_connections 2023-11-27T15:22:44.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.273+0000 7f03b13ff700 1 -- 172.21.15.133:0/3679293370 wait complete. 2023-11-27T15:22:44.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.274+0000 7f03b13ff700 1 Processor -- start 2023-11-27T15:22:44.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.274+0000 7f03b13ff700 1 -- start start 2023-11-27T15:22:44.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.274+0000 7f03b13ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03ac1002d0 0x7f03ac0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:44.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.274+0000 7f03b13ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03ac1014c0 0x7f03ac072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:44.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.274+0000 7f03b13ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03ac101e50 0x7f03ac073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:44.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.274+0000 7f03b13ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac073bc0 con 0x7f03ac1014c0 2023-11-27T15:22:44.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.274+0000 7f03b13ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac073d30 con 0x7f03ac101e50 2023-11-27T15:22:44.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.274+0000 7f03b13ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac1a01e0 con 0x7f03ac1002d0 2023-11-27T15:22:44.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.274+0000 7f03a3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03ac1014c0 0x7f03ac072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:44.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.275+0000 7f03aaffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03ac1002d0 0x7f03ac0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:44.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.275+0000 7f03a3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03ac1014c0 0x7f03ac072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60350/0 (socket says 172.21.15.133:60350) 2023-11-27T15:22:44.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.275+0000 7f03aaffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03ac1002d0 0x7f03ac0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:46114/0 (socket says 172.21.15.133:46114) 2023-11-27T15:22:44.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.275+0000 7f03a3fff700 1 -- 172.21.15.133:0/1573775614 learned_addr learned my addr 172.21.15.133:0/1573775614 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:44.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.275+0000 7f03ab7fe700 1 --2- 172.21.15.133:0/1573775614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03ac101e50 0x7f03ac073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:44.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.275+0000 7f03aaffd700 1 -- 172.21.15.133:0/1573775614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03ac101e50 msgr2=0x7f03ac073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:44.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.275+0000 7f03aaffd700 1 --2- 172.21.15.133:0/1573775614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03ac101e50 0x7f03ac073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.275+0000 7f03aaffd700 1 -- 172.21.15.133:0/1573775614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03ac1014c0 msgr2=0x7f03ac072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:44.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.275+0000 7f03aaffd700 1 --2- 172.21.15.133:0/1573775614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03ac1014c0 0x7f03ac072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.275+0000 7f03aaffd700 1 -- 172.21.15.133:0/1573775614 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a4009d00 con 0x7f03ac1002d0 2023-11-27T15:22:44.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.275+0000 7f03a3fff700 1 --2- 172.21.15.133:0/1573775614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03ac1014c0 0x7f03ac072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:44.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.275+0000 7f03aaffd700 1 --2- 172.21.15.133:0/1573775614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03ac1002d0 0x7f03ac0748a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f039c00e9c0 tx=0x7f039c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:44.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.275+0000 7f03a8ff9700 1 -- 172.21.15.133:0/1573775614 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c004220 con 0x7f03ac1002d0 2023-11-27T15:22:44.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.276+0000 7f03b13ff700 1 -- 172.21.15.133:0/1573775614 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ac10bca0 con 0x7f03ac1002d0 2023-11-27T15:22:44.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.276+0000 7f03a8ff9700 1 -- 172.21.15.133:0/1573775614 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f039c00cc80 con 0x7f03ac1002d0 2023-11-27T15:22:44.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.276+0000 7f03a8ff9700 1 -- 172.21.15.133:0/1573775614 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c003be0 con 0x7f03ac1002d0 2023-11-27T15:22:44.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.276+0000 7f03b13ff700 1 -- 172.21.15.133:0/1573775614 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ac10bb10 con 0x7f03ac1002d0 2023-11-27T15:22:44.282 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.277+0000 7f03b13ff700 1 -- 172.21.15.133:0/1573775614 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ac1077f0 con 0x7f03ac1002d0 2023-11-27T15:22:44.282 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.279+0000 7f03a8ff9700 1 -- 172.21.15.133:0/1573775614 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f039c003d40 con 0x7f03ac1002d0 2023-11-27T15:22:44.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.280+0000 7f03a8ff9700 1 --2- 172.21.15.133:0/1573775614 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f03980a11d0 0x7f03980a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:44.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.280+0000 7f03a8ff9700 1 -- 172.21.15.133:0/1573775614 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f039c014070 con 0x7f03ac1002d0 2023-11-27T15:22:44.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.280+0000 7f03a3fff700 1 --2- 172.21.15.133:0/1573775614 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f03980a11d0 0x7f03980a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:44.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.281+0000 7f03a3fff700 1 --2- 172.21.15.133:0/1573775614 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f03980a11d0 0x7f03980a3680 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0394011550 tx=0x7f0394009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:44.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.282+0000 7f03a8ff9700 1 -- 172.21.15.133:0/1573775614 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f039c096390 con 0x7f03ac1002d0 2023-11-27T15:22:44.486 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.483+0000 7f03b13ff700 1 -- 172.21.15.133:0/1573775614 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03ac067b00 con 0x7f03ac1002d0 2023-11-27T15:22:44.487 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.484+0000 7f03a8ff9700 1 -- 172.21.15.133:0/1573775614 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f039c0999b0 con 0x7f03ac1002d0 2023-11-27T15:22:44.487 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:22:44.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.486+0000 7f03b13ff700 1 -- 172.21.15.133:0/1573775614 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f03980a11d0 msgr2=0x7f03980a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:44.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.486+0000 7f03b13ff700 1 --2- 172.21.15.133:0/1573775614 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f03980a11d0 0x7f03980a3680 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0394011550 tx=0x7f0394009260 comp rx=0 tx=0).stop 2023-11-27T15:22:44.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.486+0000 7f03b13ff700 1 -- 172.21.15.133:0/1573775614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03ac1002d0 msgr2=0x7f03ac0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:44.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.486+0000 7f03b13ff700 1 --2- 172.21.15.133:0/1573775614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03ac1002d0 0x7f03ac0748a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f039c00e9c0 tx=0x7f039c00ecd0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.486+0000 7f03b13ff700 1 -- 172.21.15.133:0/1573775614 shutdown_connections 2023-11-27T15:22:44.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.486+0000 7f03b13ff700 1 --2- 172.21.15.133:0/1573775614 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f03980a11d0 0x7f03980a3680 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.486+0000 7f03b13ff700 1 --2- 172.21.15.133:0/1573775614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03ac101e50 0x7f03ac073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.486+0000 7f03b13ff700 1 --2- 172.21.15.133:0/1573775614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03ac1002d0 0x7f03ac0748a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.486+0000 7f03b13ff700 1 --2- 172.21.15.133:0/1573775614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f03ac1014c0 0x7f03ac072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.486+0000 7f03b13ff700 1 -- 172.21.15.133:0/1573775614 >> 172.21.15.133:0/1573775614 conn(0x7f03ac0fb8b0 msgr2=0x7f03ac104930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:44.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.486+0000 7f03b13ff700 1 -- 172.21.15.133:0/1573775614 shutdown_connections 2023-11-27T15:22:44.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.486+0000 7f03b13ff700 1 -- 172.21.15.133:0/1573775614 wait complete. 2023-11-27T15:22:44.509 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:22:44.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.706+0000 7f9b019ff700 1 -- 172.21.15.133:0/3206405583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9afc108190 msgr2=0x7f9afc10a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:44.710 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.706+0000 7f9b019ff700 1 --2- 172.21.15.133:0/3206405583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9afc108190 0x7f9afc10a610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9af400b0a0 tx=0x7f9af400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.707+0000 7f9b019ff700 1 -- 172.21.15.133:0/3206405583 shutdown_connections 2023-11-27T15:22:44.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.707+0000 7f9b019ff700 1 --2- 172.21.15.133:0/3206405583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9afc108190 0x7f9afc10a610 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.707+0000 7f9b019ff700 1 --2- 172.21.15.133:0/3206405583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9afc074b30 0x7f9afc072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.707+0000 7f9b019ff700 1 --2- 172.21.15.133:0/3206405583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9afc073550 0x7f9afc0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.707+0000 7f9b019ff700 1 -- 172.21.15.133:0/3206405583 >> 172.21.15.133:0/3206405583 conn(0x7f9afc0fb8b0 msgr2=0x7f9afc0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:44.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.707+0000 7f9b019ff700 1 -- 172.21.15.133:0/3206405583 shutdown_connections 2023-11-27T15:22:44.711 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.707+0000 7f9b019ff700 1 -- 172.21.15.133:0/3206405583 wait complete. 2023-11-27T15:22:44.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.708+0000 7f9b019ff700 1 Processor -- start 2023-11-27T15:22:44.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.708+0000 7f9b019ff700 1 -- start start 2023-11-27T15:22:44.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.708+0000 7f9b019ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9afc073550 0x7f9afc1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:44.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.708+0000 7f9b019ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9afc074b30 0x7f9afc1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:44.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.708+0000 7f9b019ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9afc108190 0x7f9afc19f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:44.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.708+0000 7f9b019ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc19ff20 con 0x7f9afc073550 2023-11-27T15:22:44.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.708+0000 7f9b019ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc1a0060 con 0x7f9afc108190 2023-11-27T15:22:44.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.708+0000 7f9b019ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc1a01f0 con 0x7f9afc074b30 2023-11-27T15:22:44.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.708+0000 7f9afaffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9afc073550 0x7f9afc1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:44.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.708+0000 7f9af3fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9afc074b30 0x7f9afc1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:44.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.708+0000 7f9afaffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9afc073550 0x7f9afc1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60372/0 (socket says 172.21.15.133:60372) 2023-11-27T15:22:44.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.708+0000 7f9afaffd700 1 -- 172.21.15.133:0/2880701355 learned_addr learned my addr 172.21.15.133:0/2880701355 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:44.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.708+0000 7f9afb7fe700 1 --2- 172.21.15.133:0/2880701355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9afc108190 0x7f9afc19f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:44.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.709+0000 7f9afaffd700 1 -- 172.21.15.133:0/2880701355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9afc074b30 msgr2=0x7f9afc1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:44.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.709+0000 7f9afaffd700 1 --2- 172.21.15.133:0/2880701355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9afc074b30 0x7f9afc1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.709+0000 7f9afaffd700 1 -- 172.21.15.133:0/2880701355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9afc108190 msgr2=0x7f9afc19f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:44.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.709+0000 7f9afaffd700 1 --2- 172.21.15.133:0/2880701355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9afc108190 0x7f9afc19f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.709+0000 7f9afaffd700 1 -- 172.21.15.133:0/2880701355 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af4009d00 con 0x7f9afc073550 2023-11-27T15:22:44.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.709+0000 7f9afaffd700 1 --2- 172.21.15.133:0/2880701355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9afc073550 0x7f9afc1983a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00e970 tx=0x7f9aec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:44.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.709+0000 7f9b009fd700 1 -- 172.21.15.133:0/2880701355 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec009790 con 0x7f9afc073550 2023-11-27T15:22:44.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.709+0000 7f9b019ff700 1 -- 172.21.15.133:0/2880701355 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9afc10deb0 con 0x7f9afc073550 2023-11-27T15:22:44.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.709+0000 7f9b009fd700 1 -- 172.21.15.133:0/2880701355 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9aec00c7e0 con 0x7f9afc073550 2023-11-27T15:22:44.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.709+0000 7f9b009fd700 1 -- 172.21.15.133:0/2880701355 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec0054e0 con 0x7f9afc073550 2023-11-27T15:22:44.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.709+0000 7f9b019ff700 1 -- 172.21.15.133:0/2880701355 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc10dd20 con 0x7f9afc073550 2023-11-27T15:22:44.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.710+0000 7f9b019ff700 1 -- 172.21.15.133:0/2880701355 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9afc106840 con 0x7f9afc073550 2023-11-27T15:22:44.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.712+0000 7f9b009fd700 1 -- 172.21.15.133:0/2880701355 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f9aec00c950 con 0x7f9afc073550 2023-11-27T15:22:44.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.714+0000 7f9b009fd700 1 --2- 172.21.15.133:0/2880701355 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9ae80a1150 0x7f9ae80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:44.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.714+0000 7f9b009fd700 1 -- 172.21.15.133:0/2880701355 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f9aec014070 con 0x7f9afc073550 2023-11-27T15:22:44.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.714+0000 7f9af3fff700 1 --2- 172.21.15.133:0/2880701355 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9ae80a1150 0x7f9ae80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:44.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.715+0000 7f9af3fff700 1 --2- 172.21.15.133:0/2880701355 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9ae80a1150 0x7f9ae80a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9afc074850 tx=0x7f9ae4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:44.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.717+0000 7f9b009fd700 1 -- 172.21.15.133:0/2880701355 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9aec0d2020 con 0x7f9afc073550 2023-11-27T15:22:44.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.919+0000 7f9b019ff700 1 -- 172.21.15.133:0/2880701355 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9afc061700 con 0x7f9afc073550 2023-11-27T15:22:44.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.919+0000 7f9b009fd700 1 -- 172.21.15.133:0/2880701355 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9aec096270 con 0x7f9afc073550 2023-11-27T15:22:44.923 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-11-27T15:22:44.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.921+0000 7f9b019ff700 1 -- 172.21.15.133:0/2880701355 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9ae80a1150 msgr2=0x7f9ae80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:44.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.921+0000 7f9b019ff700 1 --2- 172.21.15.133:0/2880701355 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9ae80a1150 0x7f9ae80a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9afc074850 tx=0x7f9ae4009260 comp rx=0 tx=0).stop 2023-11-27T15:22:44.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.921+0000 7f9b019ff700 1 -- 172.21.15.133:0/2880701355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9afc073550 msgr2=0x7f9afc1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:44.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.921+0000 7f9b019ff700 1 --2- 172.21.15.133:0/2880701355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9afc073550 0x7f9afc1983a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00e970 tx=0x7f9aec00ec80 comp rx=0 tx=0).stop 2023-11-27T15:22:44.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.921+0000 7f9b019ff700 1 -- 172.21.15.133:0/2880701355 shutdown_connections 2023-11-27T15:22:44.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.921+0000 7f9b019ff700 1 --2- 172.21.15.133:0/2880701355 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9ae80a1150 0x7f9ae80a3600 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.921+0000 7f9b019ff700 1 --2- 172.21.15.133:0/2880701355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9afc108190 0x7f9afc19f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.921+0000 7f9b019ff700 1 --2- 172.21.15.133:0/2880701355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9afc073550 0x7f9afc1983a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.921+0000 7f9b019ff700 1 --2- 172.21.15.133:0/2880701355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9afc074b30 0x7f9afc1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:44.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.921+0000 7f9b019ff700 1 -- 172.21.15.133:0/2880701355 >> 172.21.15.133:0/2880701355 conn(0x7f9afc0fb8b0 msgr2=0x7f9afc0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:44.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.922+0000 7f9b019ff700 1 -- 172.21.15.133:0/2880701355 shutdown_connections 2023-11-27T15:22:44.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:44.922+0000 7f9b019ff700 1 -- 172.21.15.133:0/2880701355 wait complete. 2023-11-27T15:22:44.944 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-27T15:22:45.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:44 smithi177 ceph-mon[131504]: pgmap v343: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2023-11-27T15:22:45.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:44 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2784104590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:45.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:44 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1573775614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:22:45.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.140+0000 7f0fda9ff700 1 -- 172.21.15.133:0/668784269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fd4100d40 msgr2=0x7f0fd4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:45.144 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.140+0000 7f0fda9ff700 1 --2- 172.21.15.133:0/668784269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fd4100d40 0x7f0fd4103120 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009990 tx=0x7f0fc4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.140+0000 7f0fda9ff700 1 -- 172.21.15.133:0/668784269 shutdown_connections 2023-11-27T15:22:45.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.140+0000 7f0fda9ff700 1 --2- 172.21.15.133:0/668784269 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fd4105f80 0x7f0fd41083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.140+0000 7f0fda9ff700 1 --2- 172.21.15.133:0/668784269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fd4100d40 0x7f0fd4103120 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.140+0000 7f0fda9ff700 1 --2- 172.21.15.133:0/668784269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fd4103660 0x7f0fd4105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.140+0000 7f0fda9ff700 1 -- 172.21.15.133:0/668784269 >> 172.21.15.133:0/668784269 conn(0x7f0fd40fa7c0 msgr2=0x7f0fd40fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:45.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.140+0000 7f0fda9ff700 1 -- 172.21.15.133:0/668784269 shutdown_connections 2023-11-27T15:22:45.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.140+0000 7f0fda9ff700 1 -- 172.21.15.133:0/668784269 wait complete. 2023-11-27T15:22:45.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.141+0000 7f0fda9ff700 1 Processor -- start 2023-11-27T15:22:45.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.142+0000 7f0fda9ff700 1 -- start start 2023-11-27T15:22:45.147 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.142+0000 7f0fda9ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fd4100d40 0x7f0fd41982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:45.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.142+0000 7f0fda9ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fd4103660 0x7f0fd4198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:45.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.142+0000 7f0fda9ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fd4105f80 0x7f0fd419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:45.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.142+0000 7f0fda9ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd419fe60 con 0x7f0fd4105f80 2023-11-27T15:22:45.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.142+0000 7f0fda9ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd419ffa0 con 0x7f0fd4100d40 2023-11-27T15:22:45.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.142+0000 7f0fda9ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd41a0130 con 0x7f0fd4103660 2023-11-27T15:22:45.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.142+0000 7f0fd8f9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fd4105f80 0x7f0fd419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:45.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.142+0000 7f0fd8f9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fd4105f80 0x7f0fd419f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60384/0 (socket says 172.21.15.133:60384) 2023-11-27T15:22:45.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.142+0000 7f0fd8f9c700 1 -- 172.21.15.133:0/760880454 learned_addr learned my addr 172.21.15.133:0/760880454 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:45.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.142+0000 7f0fd3fff700 1 --2- 172.21.15.133:0/760880454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fd4100d40 0x7f0fd41982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:45.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.142+0000 7f0fd37fe700 1 --2- 172.21.15.133:0/760880454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fd4103660 0x7f0fd4198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:45.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.143+0000 7f0fd8f9c700 1 -- 172.21.15.133:0/760880454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fd4103660 msgr2=0x7f0fd4198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:45.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.143+0000 7f0fd8f9c700 1 --2- 172.21.15.133:0/760880454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fd4103660 0x7f0fd4198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.143+0000 7f0fd8f9c700 1 -- 172.21.15.133:0/760880454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fd4100d40 msgr2=0x7f0fd41982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:45.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.143+0000 7f0fd8f9c700 1 --2- 172.21.15.133:0/760880454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fd4100d40 0x7f0fd41982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.143+0000 7f0fd8f9c700 1 -- 172.21.15.133:0/760880454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc40095f0 con 0x7f0fd4105f80 2023-11-27T15:22:45.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.143+0000 7f0fd3fff700 1 --2- 172.21.15.133:0/760880454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fd4100d40 0x7f0fd41982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:45.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.143+0000 7f0fd8f9c700 1 --2- 172.21.15.133:0/760880454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fd4105f80 0x7f0fd419f890 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc007ee0 tx=0x7f0fcc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:45.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.143+0000 7f0fd17fa700 1 -- 172.21.15.133:0/760880454 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc00dd20 con 0x7f0fd4105f80 2023-11-27T15:22:45.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.143+0000 7f0fd17fa700 1 -- 172.21.15.133:0/760880454 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fcc00f040 con 0x7f0fd4105f80 2023-11-27T15:22:45.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.143+0000 7f0fda9ff700 1 -- 172.21.15.133:0/760880454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd410bca0 con 0x7f0fd4105f80 2023-11-27T15:22:45.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.143+0000 7f0fd17fa700 1 -- 172.21.15.133:0/760880454 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc0048a0 con 0x7f0fd4105f80 2023-11-27T15:22:45.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.143+0000 7f0fda9ff700 1 -- 172.21.15.133:0/760880454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd410bb10 con 0x7f0fd4105f80 2023-11-27T15:22:45.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.144+0000 7f0fda9ff700 1 -- 172.21.15.133:0/760880454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd4107af0 con 0x7f0fd4105f80 2023-11-27T15:22:45.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.146+0000 7f0fd17fa700 1 -- 172.21.15.133:0/760880454 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0fcc003a00 con 0x7f0fd4105f80 2023-11-27T15:22:45.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.148+0000 7f0fd17fa700 1 --2- 172.21.15.133:0/760880454 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0fbc0a1150 0x7f0fbc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:45.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.148+0000 7f0fd17fa700 1 -- 172.21.15.133:0/760880454 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f0fcc0cc480 con 0x7f0fd4105f80 2023-11-27T15:22:45.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.148+0000 7f0fd3fff700 1 --2- 172.21.15.133:0/760880454 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0fbc0a1150 0x7f0fbc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:45.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.149+0000 7f0fd3fff700 1 --2- 172.21.15.133:0/760880454 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0fbc0a1150 0x7f0fbc0a3600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400b5c0 tx=0x7f0fc4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:45.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:44 smithi174 ceph-mon[131110]: pgmap v343: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2023-11-27T15:22:45.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:44 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2784104590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:45.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:44 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1573775614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:22:45.155 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.152+0000 7f0fd17fa700 1 -- 172.21.15.133:0/760880454 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0fcc04e730 con 0x7f0fd4105f80 2023-11-27T15:22:45.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:44 smithi133 ceph-mon[97860]: pgmap v343: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2023-11-27T15:22:45.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:44 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2784104590' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:45.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:44 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1573775614' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:22:45.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.354+0000 7f0fda9ff700 1 -- 172.21.15.133:0/760880454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0fd4061950 con 0x7f0fd4105f80 2023-11-27T15:22:45.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.354+0000 7f0fd17fa700 1 -- 172.21.15.133:0/760880454 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0fcc095820 con 0x7f0fd4105f80 2023-11-27T15:22:45.358 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:22:45.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.356+0000 7f0fda9ff700 1 -- 172.21.15.133:0/760880454 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0fbc0a1150 msgr2=0x7f0fbc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:45.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.356+0000 7f0fda9ff700 1 --2- 172.21.15.133:0/760880454 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0fbc0a1150 0x7f0fbc0a3600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400b5c0 tx=0x7f0fc4005fb0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.356+0000 7f0fda9ff700 1 -- 172.21.15.133:0/760880454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fd4105f80 msgr2=0x7f0fd419f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:45.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.356+0000 7f0fda9ff700 1 --2- 172.21.15.133:0/760880454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fd4105f80 0x7f0fd419f890 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc007ee0 tx=0x7f0fcc00d3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.356+0000 7f0fda9ff700 1 -- 172.21.15.133:0/760880454 shutdown_connections 2023-11-27T15:22:45.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.356+0000 7f0fda9ff700 1 --2- 172.21.15.133:0/760880454 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0fbc0a1150 0x7f0fbc0a3600 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.356+0000 7f0fda9ff700 1 --2- 172.21.15.133:0/760880454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fd4105f80 0x7f0fd419f890 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.356+0000 7f0fda9ff700 1 --2- 172.21.15.133:0/760880454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0fd4100d40 0x7f0fd41982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.356+0000 7f0fda9ff700 1 --2- 172.21.15.133:0/760880454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0fd4103660 0x7f0fd4198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.356+0000 7f0fda9ff700 1 -- 172.21.15.133:0/760880454 >> 172.21.15.133:0/760880454 conn(0x7f0fd40fa7c0 msgr2=0x7f0fd40ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:45.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.357+0000 7f0fda9ff700 1 -- 172.21.15.133:0/760880454 shutdown_connections 2023-11-27T15:22:45.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.357+0000 7f0fda9ff700 1 -- 172.21.15.133:0/760880454 wait complete. 2023-11-27T15:22:45.380 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-11-27T15:22:45.380 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:22:45.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.573+0000 7f36cd3ff700 1 -- 172.21.15.133:0/2850044942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36c8101e30 msgr2=0x7f36c81086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:45.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.573+0000 7f36cd3ff700 1 --2- 172.21.15.133:0/2850044942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36c8101e30 0x7f36c81086f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f36c000b0a0 tx=0x7f36c000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.573+0000 7f36cd3ff700 1 -- 172.21.15.133:0/2850044942 shutdown_connections 2023-11-27T15:22:45.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.573+0000 7f36cd3ff700 1 --2- 172.21.15.133:0/2850044942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36c8101e30 0x7f36c81086f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.573+0000 7f36cd3ff700 1 --2- 172.21.15.133:0/2850044942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36c81002b0 0x7f36c81006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.573+0000 7f36cd3ff700 1 --2- 172.21.15.133:0/2850044942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36c81014a0 0x7f36c81018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.573+0000 7f36cd3ff700 1 -- 172.21.15.133:0/2850044942 >> 172.21.15.133:0/2850044942 conn(0x7f36c80fb890 msgr2=0x7f36c80fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:45.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.573+0000 7f36cd3ff700 1 -- 172.21.15.133:0/2850044942 shutdown_connections 2023-11-27T15:22:45.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.573+0000 7f36cd3ff700 1 -- 172.21.15.133:0/2850044942 wait complete. 2023-11-27T15:22:45.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.574+0000 7f36cd3ff700 1 Processor -- start 2023-11-27T15:22:45.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.575+0000 7f36cd3ff700 1 -- start start 2023-11-27T15:22:45.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.575+0000 7f36cd3ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36c81002b0 0x7f36c8198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:45.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.575+0000 7f36cd3ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36c81014a0 0x7f36c81987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:45.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.575+0000 7f36cd3ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36c8101e30 0x7f36c819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:45.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.575+0000 7f36cd3ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c819fe10 con 0x7f36c81014a0 2023-11-27T15:22:45.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.575+0000 7f36cd3ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c819ff50 con 0x7f36c81002b0 2023-11-27T15:22:45.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.575+0000 7f36cd3ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c81a00e0 con 0x7f36c8101e30 2023-11-27T15:22:45.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.575+0000 7f36c67fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36c81014a0 0x7f36c81987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:45.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.575+0000 7f36c6ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36c81002b0 0x7f36c8198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:45.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.575+0000 7f36c67fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36c81014a0 0x7f36c81987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60406/0 (socket says 172.21.15.133:60406) 2023-11-27T15:22:45.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.575+0000 7f36c67fc700 1 -- 172.21.15.133:0/293352405 learned_addr learned my addr 172.21.15.133:0/293352405 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:45.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.575+0000 7f36c77fe700 1 --2- 172.21.15.133:0/293352405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36c8101e30 0x7f36c819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:45.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.575+0000 7f36c67fc700 1 -- 172.21.15.133:0/293352405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36c8101e30 msgr2=0x7f36c819f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:45.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.575+0000 7f36c67fc700 1 --2- 172.21.15.133:0/293352405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36c8101e30 0x7f36c819f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.575+0000 7f36c67fc700 1 -- 172.21.15.133:0/293352405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36c81002b0 msgr2=0x7f36c8198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:45.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.575+0000 7f36c67fc700 1 --2- 172.21.15.133:0/293352405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36c81002b0 0x7f36c8198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.576+0000 7f36c67fc700 1 -- 172.21.15.133:0/293352405 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c0009d00 con 0x7f36c81014a0 2023-11-27T15:22:45.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.576+0000 7f36c6ffd700 1 --2- 172.21.15.133:0/293352405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36c81002b0 0x7f36c8198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:45.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.576+0000 7f36c67fc700 1 --2- 172.21.15.133:0/293352405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36c81014a0 0x7f36c81987d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00e9a0 tx=0x7f36bc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:45.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.576+0000 7f36affff700 1 -- 172.21.15.133:0/293352405 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc009790 con 0x7f36c81014a0 2023-11-27T15:22:45.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.576+0000 7f36cd3ff700 1 -- 172.21.15.133:0/293352405 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c810bc80 con 0x7f36c81014a0 2023-11-27T15:22:45.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.576+0000 7f36affff700 1 -- 172.21.15.133:0/293352405 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f36bc010430 con 0x7f36c81014a0 2023-11-27T15:22:45.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.576+0000 7f36affff700 1 -- 172.21.15.133:0/293352405 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc018680 con 0x7f36c81014a0 2023-11-27T15:22:45.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.576+0000 7f36cd3ff700 1 -- 172.21.15.133:0/293352405 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c810baf0 con 0x7f36c81014a0 2023-11-27T15:22:45.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.577+0000 7f36cd3ff700 1 -- 172.21.15.133:0/293352405 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c81a07d0 con 0x7f36c81014a0 2023-11-27T15:22:45.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.579+0000 7f36affff700 1 -- 172.21.15.133:0/293352405 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f36bc0105a0 con 0x7f36c81014a0 2023-11-27T15:22:45.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.580+0000 7f36affff700 1 --2- 172.21.15.133:0/293352405 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f36b00a1150 0x7f36b00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:45.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.581+0000 7f36affff700 1 -- 172.21.15.133:0/293352405 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f36bc014070 con 0x7f36c81014a0 2023-11-27T15:22:45.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.581+0000 7f36c6ffd700 1 --2- 172.21.15.133:0/293352405 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f36b00a1150 0x7f36b00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:45.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.581+0000 7f36c6ffd700 1 --2- 172.21.15.133:0/293352405 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f36b00a1150 0x7f36b00a3600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f36c8101300 tx=0x7f36b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:45.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.582+0000 7f36affff700 1 -- 172.21.15.133:0/293352405 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f36bc096410 con 0x7f36c81014a0 2023-11-27T15:22:45.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.831+0000 7f36cd3ff700 1 -- 172.21.15.133:0/293352405 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36c81a0d80 con 0x7f36c81014a0 2023-11-27T15:22:45.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.832+0000 7f36affff700 1 -- 172.21.15.133:0/293352405 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10829 (secure 0 0 0) 0x7f36bc026090 con 0x7f36c81014a0 2023-11-27T15:22:45.836 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:45.836 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14493,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2381290259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2381290259},{"type":"v1","addr":"172.21.15.133:6839","nonce":2381290259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1704875895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1704875895},{"type":"v1","addr":"172.21.15.133:6841","nonce":1704875895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:20:33.201023+0000","modified":"2023-11-27T15:21:04.794526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24521,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3492215689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3492215689},{"type":"v1","addr":"172.21.15.177:6837","nonce":3492215689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6839/2781994447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2781994447},{"type":"v1","addr":"172.21.15.174:6839","nonce":2781994447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:22:45.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.835+0000 7f36cd3ff700 1 -- 172.21.15.133:0/293352405 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f36b00a1150 msgr2=0x7f36b00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:45.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.835+0000 7f36cd3ff700 1 --2- 172.21.15.133:0/293352405 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f36b00a1150 0x7f36b00a3600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f36c8101300 tx=0x7f36b8009260 comp rx=0 tx=0).stop 2023-11-27T15:22:45.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.835+0000 7f36cd3ff700 1 -- 172.21.15.133:0/293352405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36c81014a0 msgr2=0x7f36c81987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:45.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.835+0000 7f36cd3ff700 1 --2- 172.21.15.133:0/293352405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36c81014a0 0x7f36c81987d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00e9a0 tx=0x7f36bc00ecb0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.835+0000 7f36cd3ff700 1 -- 172.21.15.133:0/293352405 shutdown_connections 2023-11-27T15:22:45.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.835+0000 7f36cd3ff700 1 --2- 172.21.15.133:0/293352405 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f36b00a1150 0x7f36b00a3600 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.835+0000 7f36cd3ff700 1 --2- 172.21.15.133:0/293352405 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f36c8101e30 0x7f36c819f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.835+0000 7f36cd3ff700 1 --2- 172.21.15.133:0/293352405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36c81002b0 0x7f36c8198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.835+0000 7f36cd3ff700 1 --2- 172.21.15.133:0/293352405 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36c81014a0 0x7f36c81987d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:45.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.835+0000 7f36cd3ff700 1 -- 172.21.15.133:0/293352405 >> 172.21.15.133:0/293352405 conn(0x7f36c80fb890 msgr2=0x7f36c80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:45.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.835+0000 7f36cd3ff700 1 -- 172.21.15.133:0/293352405 shutdown_connections 2023-11-27T15:22:45.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:45.835+0000 7f36cd3ff700 1 -- 172.21.15.133:0/293352405 wait complete. 2023-11-27T15:22:45.840 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 23 2023-11-27T15:22:45.858 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-11-27T15:22:45.858 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-11-27T15:22:45.858 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-27T15:22:46.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.052+0000 7fbddf3ff700 1 -- 172.21.15.133:0/1745349323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdd81014a0 msgr2=0x7fbdd81018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:46.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.052+0000 7fbddf3ff700 1 --2- 172.21.15.133:0/1745349323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdd81014a0 0x7fbdd81018f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009990 tx=0x7fbdc8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:46.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.052+0000 7fbddf3ff700 1 -- 172.21.15.133:0/1745349323 shutdown_connections 2023-11-27T15:22:46.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.052+0000 7fbddf3ff700 1 --2- 172.21.15.133:0/1745349323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdd8101e30 0x7fbdd81086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:46.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.052+0000 7fbddf3ff700 1 --2- 172.21.15.133:0/1745349323 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdd81002b0 0x7fbdd81006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:46.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.052+0000 7fbddf3ff700 1 --2- 172.21.15.133:0/1745349323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdd81014a0 0x7fbdd81018f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:46.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.052+0000 7fbddf3ff700 1 -- 172.21.15.133:0/1745349323 >> 172.21.15.133:0/1745349323 conn(0x7fbdd80fb890 msgr2=0x7fbdd80fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:46.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.052+0000 7fbddf3ff700 1 -- 172.21.15.133:0/1745349323 shutdown_connections 2023-11-27T15:22:46.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.053+0000 7fbddf3ff700 1 -- 172.21.15.133:0/1745349323 wait complete. 2023-11-27T15:22:46.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.053+0000 7fbddf3ff700 1 Processor -- start 2023-11-27T15:22:46.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.054+0000 7fbddf3ff700 1 -- start start 2023-11-27T15:22:46.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.054+0000 7fbddf3ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdd81002b0 0x7fbdd8195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:46.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.054+0000 7fbddf3ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdd81014a0 0x7fbdd81964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.054+0000 7fbddf3ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdd8101e30 0x7fbdd819d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.054+0000 7fbddf3ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd819db20 con 0x7fbdd8101e30 2023-11-27T15:22:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.054+0000 7fbddf3ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd819dc60 con 0x7fbdd81002b0 2023-11-27T15:22:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.054+0000 7fbddf3ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd819ddf0 con 0x7fbdd81014a0 2023-11-27T15:22:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.054+0000 7fbddd99c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdd8101e30 0x7fbdd819d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.054+0000 7fbddd99c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdd8101e30 0x7fbdd819d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60420/0 (socket says 172.21.15.133:60420) 2023-11-27T15:22:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.054+0000 7fbddd99c700 1 -- 172.21.15.133:0/2227128390 learned_addr learned my addr 172.21.15.133:0/2227128390 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.054+0000 7fbddc99a700 1 --2- 172.21.15.133:0/2227128390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdd81014a0 0x7fbdd81964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.054+0000 7fbddd19b700 1 --2- 172.21.15.133:0/2227128390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdd81002b0 0x7fbdd8195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.054+0000 7fbddd99c700 1 -- 172.21.15.133:0/2227128390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdd81014a0 msgr2=0x7fbdd81964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.054+0000 7fbddd99c700 1 --2- 172.21.15.133:0/2227128390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdd81014a0 0x7fbdd81964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:46.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.055+0000 7fbddd99c700 1 -- 172.21.15.133:0/2227128390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdd81002b0 msgr2=0x7fbdd8195fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:46.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.055+0000 7fbddd99c700 1 --2- 172.21.15.133:0/2227128390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdd81002b0 0x7fbdd8195fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:46.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.055+0000 7fbddd99c700 1 -- 172.21.15.133:0/2227128390 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc80095f0 con 0x7fbdd8101e30 2023-11-27T15:22:46.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.055+0000 7fbddd99c700 1 --2- 172.21.15.133:0/2227128390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdd8101e30 0x7fbdd819d550 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000ef30 tx=0x7fbdd000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:46.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.055+0000 7fbdce7fc700 1 -- 172.21.15.133:0/2227128390 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0018070 con 0x7fbdd8101e30 2023-11-27T15:22:46.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.055+0000 7fbdce7fc700 1 -- 172.21.15.133:0/2227128390 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdd000f040 con 0x7fbdd8101e30 2023-11-27T15:22:46.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.055+0000 7fbddf3ff700 1 -- 172.21.15.133:0/2227128390 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd810bc60 con 0x7fbdd8101e30 2023-11-27T15:22:46.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.055+0000 7fbdce7fc700 1 -- 172.21.15.133:0/2227128390 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd000cd00 con 0x7fbdd8101e30 2023-11-27T15:22:46.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.055+0000 7fbddf3ff700 1 -- 172.21.15.133:0/2227128390 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd810bad0 con 0x7fbdd8101e30 2023-11-27T15:22:46.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.056+0000 7fbddf3ff700 1 -- 172.21.15.133:0/2227128390 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd8067a60 con 0x7fbdd8101e30 2023-11-27T15:22:46.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.058+0000 7fbdce7fc700 1 -- 172.21.15.133:0/2227128390 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fbdd0003a00 con 0x7fbdd8101e30 2023-11-27T15:22:46.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.060+0000 7fbdce7fc700 1 --2- 172.21.15.133:0/2227128390 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbdc40a1150 0x7fbdc40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:46.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.060+0000 7fbdce7fc700 1 -- 172.21.15.133:0/2227128390 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbdd00cc7e0 con 0x7fbdd8101e30 2023-11-27T15:22:46.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.060+0000 7fbddd19b700 1 --2- 172.21.15.133:0/2227128390 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbdc40a1150 0x7fbdc40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:46.064 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.060+0000 7fbddd19b700 1 --2- 172.21.15.133:0/2227128390 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbdc40a1150 0x7fbdc40a3600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8101300 tx=0x7fbdd4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:46.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.063+0000 7fbdce7fc700 1 -- 172.21.15.133:0/2227128390 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbdd0095c10 con 0x7fbdd8101e30 2023-11-27T15:22:46.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:45 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2880701355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:22:46.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:45 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/760880454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:22:46.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:45 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2880701355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:22:46.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:45 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/760880454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:22:46.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.256+0000 7fbddf3ff700 1 -- 172.21.15.133:0/2227128390 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbdd8060f30 con 0x7fbdc40a1150 2023-11-27T15:22:46.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.269+0000 7fbdce7fc700 1 -- 172.21.15.133:0/2227128390 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427427 (secure 0 0 0) 0x7fbdd8060f30 con 0x7fbdc40a1150 2023-11-27T15:22:46.279 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:46.282 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:45 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2880701355' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:22:46.282 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:45 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/760880454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:22:46.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.274+0000 7fbddf3ff700 1 -- 172.21.15.133:0/2227128390 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbdc40a1150 msgr2=0x7fbdc40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:46.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.274+0000 7fbddf3ff700 1 --2- 172.21.15.133:0/2227128390 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbdc40a1150 0x7fbdc40a3600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8101300 tx=0x7fbdd4009260 comp rx=0 tx=0).stop 2023-11-27T15:22:46.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.274+0000 7fbddf3ff700 1 -- 172.21.15.133:0/2227128390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdd8101e30 msgr2=0x7fbdd819d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:46.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.274+0000 7fbddf3ff700 1 --2- 172.21.15.133:0/2227128390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdd8101e30 0x7fbdd819d550 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000ef30 tx=0x7fbdd000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:22:46.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.274+0000 7fbddf3ff700 1 -- 172.21.15.133:0/2227128390 shutdown_connections 2023-11-27T15:22:46.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.275+0000 7fbddf3ff700 1 --2- 172.21.15.133:0/2227128390 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbdc40a1150 0x7fbdc40a3600 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:46.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.275+0000 7fbddf3ff700 1 --2- 172.21.15.133:0/2227128390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbdd8101e30 0x7fbdd819d550 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:46.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.275+0000 7fbddf3ff700 1 --2- 172.21.15.133:0/2227128390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdd81002b0 0x7fbdd8195fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:46.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.275+0000 7fbddf3ff700 1 --2- 172.21.15.133:0/2227128390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbdd81014a0 0x7fbdd81964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:46.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.275+0000 7fbddf3ff700 1 -- 172.21.15.133:0/2227128390 >> 172.21.15.133:0/2227128390 conn(0x7fbdd80fb890 msgr2=0x7fbdd80fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:46.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.275+0000 7fbddf3ff700 1 -- 172.21.15.133:0/2227128390 shutdown_connections 2023-11-27T15:22:46.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.275+0000 7fbddf3ff700 1 -- 172.21.15.133:0/2227128390 wait complete. 2023-11-27T15:22:46.284 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2023-11-27T15:22:46.297 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2023-11-27T15:22:45.744559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655340,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":745,"num_read_kb":485,"num_write":353,"num_write_kb":3061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":84704,"kb_used_data":14272,"kb_used_omap":0,"kb_used_meta":70336,"kb_avail":1124857120,"statfs":{"total":1151940427776,"available":1151853690880,"internally_reserved":0,"allocated":14614528,"data_stored":8119101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72024064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1014,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":34,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003000"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.767519+0000","last_change":"2023-11-27T15:20:33.236352+0000","last_active":"2023-11-27T15:22:31.767519+0000","last_peered":"2023-11-27T15:22:31.767519+0000","last_clean":"2023-11-27T15:22:31.767519+0000","last_became_active":"2023-11-27T15:20:33.235974+0000","last_became_peered":"2023-11-27T15:20:33.235974+0000","last_unstale":"2023-11-27T15:22:31.767519+0000","last_undegraded":"2023-11-27T15:22:31.767519+0000","last_fullsized":"2023-11-27T15:22:31.767519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:32:53.999483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.764610+0000","last_change":"2023-11-27T15:20:33.234882+0000","last_active":"2023-11-27T15:22:31.764610+0000","last_peered":"2023-11-27T15:22:31.764610+0000","last_clean":"2023-11-27T15:22:31.764610+0000","last_became_active":"2023-11-27T15:20:33.231062+0000","last_became_peered":"2023-11-27T15:20:33.231062+0000","last_unstale":"2023-11-27T15:22:31.764610+0000","last_undegraded":"2023-11-27T15:22:31.764610+0000","last_fullsized":"2023-11-27T15:22:31.764610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:54:38.782631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.765932+0000","last_change":"2023-11-27T15:20:33.238280+0000","last_active":"2023-11-27T15:22:31.765932+0000","last_peered":"2023-11-27T15:22:31.765932+0000","last_clean":"2023-11-27T15:22:31.765932+0000","last_became_active":"2023-11-27T15:20:33.237602+0000","last_became_peered":"2023-11-27T15:20:33.237602+0000","last_unstale":"2023-11-27T15:22:31.765932+0000","last_undegraded":"2023-11-27T15:22:31.765932+0000","last_fullsized":"2023-11-27T15:22:31.765932+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:12:10.576246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.744091+0000","last_change":"2023-11-27T15:20:33.217909+0000","last_active":"2023-11-27T15:22:31.744091+0000","last_peered":"2023-11-27T15:22:31.744091+0000","last_clean":"2023-11-27T15:22:31.744091+0000","last_became_active":"2023-11-27T15:20:33.217511+0000","last_became_peered":"2023-11-27T15:20:33.217511+0000","last_unstale":"2023-11-27T15:22:31.744091+0000","last_undegraded":"2023-11-27T15:22:31.744091+0000","last_fullsized":"2023-11-27T15:22:31.744091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:11:52.036690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.750946+0000","last_change":"2023-11-27T15:20:33.246630+0000","last_active":"2023-11-27T15:22:31.750946+0000","last_peered":"2023-11-27T15:22:31.750946+0000","last_clean":"2023-11-27T15:22:31.750946+0000","last_became_active":"2023-11-27T15:20:33.246249+0000","last_became_peered":"2023-11-27T15:20:33.246249+0000","last_unstale":"2023-11-27T15:22:31.750946+0000","last_undegraded":"2023-11-27T15:22:31.750946+0000","last_fullsized":"2023-11-27T15:22:31.750946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:07:50.677388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.749736+0000","last_change":"2023-11-27T15:20:33.236280+0000","last_active":"2023-11-27T15:22:31.749736+0000","last_peered":"2023-11-27T15:22:31.749736+0000","last_clean":"2023-11-27T15:22:31.749736+0000","last_became_active":"2023-11-27T15:20:33.235983+0000","last_became_peered":"2023-11-27T15:20:33.235983+0000","last_unstale":"2023-11-27T15:22:31.749736+0000","last_undegraded":"2023-11-27T15:22:31.749736+0000","last_fullsized":"2023-11-27T15:22:31.749736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:38:38.466940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.749774+0000","last_change":"2023-11-27T15:20:33.223085+0000","last_active":"2023-11-27T15:22:31.749774+0000","last_peered":"2023-11-27T15:22:31.749774+0000","last_clean":"2023-11-27T15:22:31.749774+0000","last_became_active":"2023-11-27T15:20:33.222809+0000","last_became_peered":"2023-11-27T15:20:33.222809+0000","last_unstale":"2023-11-27T15:22:31.749774+0000","last_undegraded":"2023-11-27T15:22:31.749774+0000","last_fullsized":"2023-11-27T15:22:31.749774+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:23:19.671424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.760153+0000","last_change":"2023-11-27T15:20:33.226217+0000","last_active":"2023-11-27T15:22:31.760153+0000","last_peered":"2023-11-27T15:22:31.760153+0000","last_clean":"2023-11-27T15:22:31.760153+0000","last_became_active":"2023-11-27T15:20:33.225856+0000","last_became_peered":"2023-11-27T15:20:33.225856+0000","last_unstale":"2023-11-27T15:22:31.760153+0000","last_undegraded":"2023-11-27T15:22:31.760153+0000","last_fullsized":"2023-11-27T15:22:31.760153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:59:41.033434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.752180+0000","last_change":"2023-11-27T15:20:33.227554+0000","last_active":"2023-11-27T15:22:31.752180+0000","last_peered":"2023-11-27T15:22:31.752180+0000","last_clean":"2023-11-27T15:22:31.752180+0000","last_became_active":"2023-11-27T15:20:33.227222+0000","last_became_peered":"2023-11-27T15:20:33.227222+0000","last_unstale":"2023-11-27T15:22:31.752180+0000","last_undegraded":"2023-11-27T15:22:31.752180+0000","last_fullsized":"2023-11-27T15:22:31.752180+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:45:28.965465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.751136+0000","last_change":"2023-11-27T15:20:33.230528+0000","last_active":"2023-11-27T15:22:31.751136+0000","last_peered":"2023-11-27T15:22:31.751136+0000","last_clean":"2023-11-27T15:22:31.751136+0000","last_became_active":"2023-11-27T15:20:33.230148+0000","last_became_peered":"2023-11-27T15:20:33.230148+0000","last_unstale":"2023-11-27T15:22:31.751136+0000","last_undegraded":"2023-11-27T15:22:31.751136+0000","last_fullsized":"2023-11-27T15:22:31.751136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:02:27.225754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.750528+0000","last_change":"2023-11-27T15:20:33.228098+0000","last_active":"2023-11-27T15:22:31.750528+0000","last_peered":"2023-11-27T15:22:31.750528+0000","last_clean":"2023-11-27T15:22:31.750528+0000","last_became_active":"2023-11-27T15:20:33.227760+0000","last_became_peered":"2023-11-27T15:20:33.227760+0000","last_unstale":"2023-11-27T15:22:31.750528+0000","last_undegraded":"2023-11-27T15:22:31.750528+0000","last_fullsized":"2023-11-27T15:22:31.750528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:08:49.344313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.756176+0000","last_change":"2023-11-27T15:21:02.934583+0000","last_active":"2023-11-27T15:22:31.756176+0000","last_peered":"2023-11-27T15:22:31.756176+0000","last_clean":"2023-11-27T15:22:31.756176+0000","last_became_active":"2023-11-27T15:21:02.933631+0000","last_became_peered":"2023-11-27T15:21:02.933631+0000","last_unstale":"2023-11-27T15:22:31.756176+0000","last_undegraded":"2023-11-27T15:22:31.756176+0000","last_fullsized":"2023-11-27T15:22:31.756176+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:22:12.200623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.754054+0000","last_change":"2023-11-27T15:20:33.231255+0000","last_active":"2023-11-27T15:22:31.754054+0000","last_peered":"2023-11-27T15:22:31.754054+0000","last_clean":"2023-11-27T15:22:31.754054+0000","last_became_active":"2023-11-27T15:20:33.230867+0000","last_became_peered":"2023-11-27T15:20:33.230867+0000","last_unstale":"2023-11-27T15:22:31.754054+0000","last_undegraded":"2023-11-27T15:22:31.754054+0000","last_fullsized":"2023-11-27T15:22:31.754054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:33:48.764065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.767126+0000","last_change":"2023-11-27T15:20:33.230306+0000","last_active":"2023-11-27T15:22:31.767126+0000","last_peered":"2023-11-27T15:22:31.767126+0000","last_clean":"2023-11-27T15:22:31.767126+0000","last_became_active":"2023-11-27T15:20:33.229892+0000","last_became_peered":"2023-11-27T15:20:33.229892+0000","last_unstale":"2023-11-27T15:22:31.767126+0000","last_undegraded":"2023-11-27T15:22:31.767126+0000","last_fullsized":"2023-11-27T15:22:31.767126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:44:51.431718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.761884+0000","last_change":"2023-11-27T15:20:33.220592+0000","last_active":"2023-11-27T15:22:31.761884+0000","last_peered":"2023-11-27T15:22:31.761884+0000","last_clean":"2023-11-27T15:22:31.761884+0000","last_became_active":"2023-11-27T15:20:33.220258+0000","last_became_peered":"2023-11-27T15:20:33.220258+0000","last_unstale":"2023-11-27T15:22:31.761884+0000","last_undegraded":"2023-11-27T15:22:31.761884+0000","last_fullsized":"2023-11-27T15:22:31.761884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:16:39.685631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.767650+0000","last_change":"2023-11-27T15:20:33.245041+0000","last_active":"2023-11-27T15:22:31.767650+0000","last_peered":"2023-11-27T15:22:31.767650+0000","last_clean":"2023-11-27T15:22:31.767650+0000","last_became_active":"2023-11-27T15:20:33.244759+0000","last_became_peered":"2023-11-27T15:20:33.244759+0000","last_unstale":"2023-11-27T15:22:31.767650+0000","last_undegraded":"2023-11-27T15:22:31.767650+0000","last_fullsized":"2023-11-27T15:22:31.767650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:19:46.046097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.767690+0000","last_change":"2023-11-27T15:20:33.243288+0000","last_active":"2023-11-27T15:22:31.767690+0000","last_peered":"2023-11-27T15:22:31.767690+0000","last_clean":"2023-11-27T15:22:31.767690+0000","last_became_active":"2023-11-27T15:20:33.242989+0000","last_became_peered":"2023-11-27T15:20:33.242989+0000","last_unstale":"2023-11-27T15:22:31.767690+0000","last_undegraded":"2023-11-27T15:22:31.767690+0000","last_fullsized":"2023-11-27T15:22:31.767690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:25:35.538012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.760176+0000","last_change":"2023-11-27T15:20:33.229613+0000","last_active":"2023-11-27T15:22:31.760176+0000","last_peered":"2023-11-27T15:22:31.760176+0000","last_clean":"2023-11-27T15:22:31.760176+0000","last_became_active":"2023-11-27T15:20:33.227747+0000","last_became_peered":"2023-11-27T15:20:33.227747+0000","last_unstale":"2023-11-27T15:22:31.760176+0000","last_undegraded":"2023-11-27T15:22:31.760176+0000","last_fullsized":"2023-11-27T15:22:31.760176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:57:48.962905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.760827+0000","last_change":"2023-11-27T15:20:33.227004+0000","last_active":"2023-11-27T15:22:31.760827+0000","last_peered":"2023-11-27T15:22:31.760827+0000","last_clean":"2023-11-27T15:22:31.760827+0000","last_became_active":"2023-11-27T15:20:33.226703+0000","last_became_peered":"2023-11-27T15:20:33.226703+0000","last_unstale":"2023-11-27T15:22:31.760827+0000","last_undegraded":"2023-11-27T15:22:31.760827+0000","last_fullsized":"2023-11-27T15:22:31.760827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:16:57.764198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818292+0000","last_change":"2023-11-27T15:20:37.682021+0000","last_active":"2023-11-27T15:22:02.818292+0000","last_peered":"2023-11-27T15:22:02.818292+0000","last_clean":"2023-11-27T15:22:02.818292+0000","last_became_active":"2023-11-27T15:20:37.681600+0000","last_became_peered":"2023-11-27T15:20:37.681600+0000","last_unstale":"2023-11-27T15:22:02.818292+0000","last_undegraded":"2023-11-27T15:22:02.818292+0000","last_fullsized":"2023-11-27T15:22:02.818292+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:16:02.041857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811637+0000","last_change":"2023-11-27T15:20:32.657542+0000","last_active":"2023-11-27T15:22:02.811637+0000","last_peered":"2023-11-27T15:22:02.811637+0000","last_clean":"2023-11-27T15:22:02.811637+0000","last_became_active":"2023-11-27T15:20:32.657158+0000","last_became_peered":"2023-11-27T15:20:32.657158+0000","last_unstale":"2023-11-27T15:22:02.811637+0000","last_undegraded":"2023-11-27T15:22:02.811637+0000","last_fullsized":"2023-11-27T15:22:02.811637+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:27:55.061003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.747709+0000","last_change":"2023-11-27T15:21:02.933684+0000","last_active":"2023-11-27T15:22:31.747709+0000","last_peered":"2023-11-27T15:22:31.747709+0000","last_clean":"2023-11-27T15:22:31.747709+0000","last_became_active":"2023-11-27T15:21:02.933215+0000","last_became_peered":"2023-11-27T15:21:02.933215+0000","last_unstale":"2023-11-27T15:22:31.747709+0000","last_undegraded":"2023-11-27T15:22:31.747709+0000","last_fullsized":"2023-11-27T15:22:31.747709+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:38:04.533244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811191+0000","last_change":"2023-11-27T15:20:37.666705+0000","last_active":"2023-11-27T15:22:02.811191+0000","last_peered":"2023-11-27T15:22:02.811191+0000","last_clean":"2023-11-27T15:22:02.811191+0000","last_became_active":"2023-11-27T15:20:37.666236+0000","last_became_peered":"2023-11-27T15:20:37.666236+0000","last_unstale":"2023-11-27T15:22:02.811191+0000","last_undegraded":"2023-11-27T15:22:02.811191+0000","last_fullsized":"2023-11-27T15:22:02.811191+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:39:04.186691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818128+0000","last_change":"2023-11-27T15:20:32.663915+0000","last_active":"2023-11-27T15:22:02.818128+0000","last_peered":"2023-11-27T15:22:02.818128+0000","last_clean":"2023-11-27T15:22:02.818128+0000","last_became_active":"2023-11-27T15:20:32.663069+0000","last_became_peered":"2023-11-27T15:20:32.663069+0000","last_unstale":"2023-11-27T15:22:02.818128+0000","last_undegraded":"2023-11-27T15:22:02.818128+0000","last_fullsized":"2023-11-27T15:22:02.818128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:59:18.356598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.765956+0000","last_change":"2023-11-27T15:20:33.238150+0000","last_active":"2023-11-27T15:22:31.765956+0000","last_peered":"2023-11-27T15:22:31.765956+0000","last_clean":"2023-11-27T15:22:31.765956+0000","last_became_active":"2023-11-27T15:20:33.236972+0000","last_became_peered":"2023-11-27T15:20:33.236972+0000","last_unstale":"2023-11-27T15:22:31.765956+0000","last_undegraded":"2023-11-27T15:22:31.765956+0000","last_fullsized":"2023-11-27T15:22:31.765956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:18:18.352219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811484+0000","last_change":"2023-11-27T15:20:37.668031+0000","last_active":"2023-11-27T15:22:02.811484+0000","last_peered":"2023-11-27T15:22:02.811484+0000","last_clean":"2023-11-27T15:22:02.811484+0000","last_became_active":"2023-11-27T15:20:37.667640+0000","last_became_peered":"2023-11-27T15:20:37.667640+0000","last_unstale":"2023-11-27T15:22:02.811484+0000","last_undegraded":"2023-11-27T15:22:02.811484+0000","last_fullsized":"2023-11-27T15:22:02.811484+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:01:43.896196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.819361+0000","last_change":"2023-11-27T15:20:32.655738+0000","last_active":"2023-11-27T15:22:02.819361+0000","last_peered":"2023-11-27T15:22:02.819361+0000","last_clean":"2023-11-27T15:22:02.819361+0000","last_became_active":"2023-11-27T15:20:32.655439+0000","last_became_peered":"2023-11-27T15:20:32.655439+0000","last_unstale":"2023-11-27T15:22:02.819361+0000","last_undegraded":"2023-11-27T15:22:02.819361+0000","last_fullsized":"2023-11-27T15:22:02.819361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:26:55.161397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.752166+0000","last_change":"2023-11-27T15:20:33.229267+0000","last_active":"2023-11-27T15:22:31.752166+0000","last_peered":"2023-11-27T15:22:31.752166+0000","last_clean":"2023-11-27T15:22:31.752166+0000","last_became_active":"2023-11-27T15:20:33.228746+0000","last_became_peered":"2023-11-27T15:20:33.228746+0000","last_unstale":"2023-11-27T15:22:31.752166+0000","last_undegraded":"2023-11-27T15:22:31.752166+0000","last_fullsized":"2023-11-27T15:22:31.752166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:55:04.819687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811343+0000","last_change":"2023-11-27T15:20:37.671893+0000","last_active":"2023-11-27T15:22:02.811343+0000","last_peered":"2023-11-27T15:22:02.811343+0000","last_clean":"2023-11-27T15:22:02.811343+0000","last_became_active":"2023-11-27T15:20:37.671033+0000","last_became_peered":"2023-11-27T15:20:37.671033+0000","last_unstale":"2023-11-27T15:22:02.811343+0000","last_undegraded":"2023-11-27T15:22:02.811343+0000","last_fullsized":"2023-11-27T15:22:02.811343+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:23:32.141225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818527+0000","last_change":"2023-11-27T15:20:32.650342+0000","last_active":"2023-11-27T15:22:02.818527+0000","last_peered":"2023-11-27T15:22:02.818527+0000","last_clean":"2023-11-27T15:22:02.818527+0000","last_became_active":"2023-11-27T15:20:32.650034+0000","last_became_peered":"2023-11-27T15:20:32.650034+0000","last_unstale":"2023-11-27T15:22:02.818527+0000","last_undegraded":"2023-11-27T15:22:02.818527+0000","last_fullsized":"2023-11-27T15:22:02.818527+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:14:01.957530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.749849+0000","last_change":"2023-11-27T15:20:33.226005+0000","last_active":"2023-11-27T15:22:31.749849+0000","last_peered":"2023-11-27T15:22:31.749849+0000","last_clean":"2023-11-27T15:22:31.749849+0000","last_became_active":"2023-11-27T15:20:33.225685+0000","last_became_peered":"2023-11-27T15:20:33.225685+0000","last_unstale":"2023-11-27T15:22:31.749849+0000","last_undegraded":"2023-11-27T15:22:31.749849+0000","last_fullsized":"2023-11-27T15:22:31.749849+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:36:39.501462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.819431+0000","last_change":"2023-11-27T15:20:37.673168+0000","last_active":"2023-11-27T15:22:02.819431+0000","last_peered":"2023-11-27T15:22:02.819431+0000","last_clean":"2023-11-27T15:22:02.819431+0000","last_became_active":"2023-11-27T15:20:37.672762+0000","last_became_peered":"2023-11-27T15:20:37.672762+0000","last_unstale":"2023-11-27T15:22:02.819431+0000","last_undegraded":"2023-11-27T15:22:02.819431+0000","last_fullsized":"2023-11-27T15:22:02.819431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:04:26.957766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'6","reported_seq":140,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:43.972931+0000","last_change":"2023-11-27T15:20:32.656402+0000","last_active":"2023-11-27T15:22:43.972931+0000","last_peered":"2023-11-27T15:22:43.972931+0000","last_clean":"2023-11-27T15:22:43.972931+0000","last_became_active":"2023-11-27T15:20:32.656112+0000","last_became_peered":"2023-11-27T15:20:32.656112+0000","last_unstale":"2023-11-27T15:22:43.972931+0000","last_undegraded":"2023-11-27T15:22:43.972931+0000","last_fullsized":"2023-11-27T15:22:43.972931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:40:54.677726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":886,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.743115+0000","last_change":"2023-11-27T15:20:33.223073+0000","last_active":"2023-11-27T15:22:31.743115+0000","last_peered":"2023-11-27T15:22:31.743115+0000","last_clean":"2023-11-27T15:22:31.743115+0000","last_became_active":"2023-11-27T15:20:33.222655+0000","last_became_peered":"2023-11-27T15:20:33.222655+0000","last_unstale":"2023-11-27T15:22:31.743115+0000","last_undegraded":"2023-11-27T15:22:31.743115+0000","last_fullsized":"2023-11-27T15:22:31.743115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:09:23.257015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.819193+0000","last_change":"2023-11-27T15:20:37.678722+0000","last_active":"2023-11-27T15:22:02.819193+0000","last_peered":"2023-11-27T15:22:02.819193+0000","last_clean":"2023-11-27T15:22:02.819193+0000","last_became_active":"2023-11-27T15:20:37.678242+0000","last_became_peered":"2023-11-27T15:20:37.678242+0000","last_unstale":"2023-11-27T15:22:02.819193+0000","last_undegraded":"2023-11-27T15:22:02.819193+0000","last_fullsized":"2023-11-27T15:22:02.819193+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:54:25.629989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.905230+0000","last_change":"2023-11-27T15:20:32.646951+0000","last_active":"2023-11-27T15:22:02.905230+0000","last_peered":"2023-11-27T15:22:02.905230+0000","last_clean":"2023-11-27T15:22:02.905230+0000","last_became_active":"2023-11-27T15:20:32.646620+0000","last_became_peered":"2023-11-27T15:20:32.646620+0000","last_unstale":"2023-11-27T15:22:02.905230+0000","last_undegraded":"2023-11-27T15:22:02.905230+0000","last_fullsized":"2023-11-27T15:22:02.905230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:36:25.079772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.751828+0000","last_change":"2023-11-27T15:20:33.226576+0000","last_active":"2023-11-27T15:22:31.751828+0000","last_peered":"2023-11-27T15:22:31.751828+0000","last_clean":"2023-11-27T15:22:31.751828+0000","last_became_active":"2023-11-27T15:20:33.226181+0000","last_became_peered":"2023-11-27T15:20:33.226181+0000","last_unstale":"2023-11-27T15:22:31.751828+0000","last_undegraded":"2023-11-27T15:22:31.751828+0000","last_fullsized":"2023-11-27T15:22:31.751828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:07:09.638906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.815632+0000","last_change":"2023-11-27T15:20:37.681668+0000","last_active":"2023-11-27T15:22:02.815632+0000","last_peered":"2023-11-27T15:22:02.815632+0000","last_clean":"2023-11-27T15:22:02.815632+0000","last_became_active":"2023-11-27T15:20:37.681236+0000","last_became_peered":"2023-11-27T15:20:37.681236+0000","last_unstale":"2023-11-27T15:22:02.815632+0000","last_undegraded":"2023-11-27T15:22:02.815632+0000","last_fullsized":"2023-11-27T15:22:02.815632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:11:46.529862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.902849+0000","last_change":"2023-11-27T15:20:32.659421+0000","last_active":"2023-11-27T15:22:02.902849+0000","last_peered":"2023-11-27T15:22:02.902849+0000","last_clean":"2023-11-27T15:22:02.902849+0000","last_became_active":"2023-11-27T15:20:32.658882+0000","last_became_peered":"2023-11-27T15:20:32.658882+0000","last_unstale":"2023-11-27T15:22:02.902849+0000","last_undegraded":"2023-11-27T15:22:02.902849+0000","last_fullsized":"2023-11-27T15:22:02.902849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:35:52.409185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.758848+0000","last_change":"2023-11-27T15:20:33.227635+0000","last_active":"2023-11-27T15:22:31.758848+0000","last_peered":"2023-11-27T15:22:31.758848+0000","last_clean":"2023-11-27T15:22:31.758848+0000","last_became_active":"2023-11-27T15:20:33.225217+0000","last_became_peered":"2023-11-27T15:20:33.225217+0000","last_unstale":"2023-11-27T15:22:31.758848+0000","last_undegraded":"2023-11-27T15:22:31.758848+0000","last_fullsized":"2023-11-27T15:22:31.758848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:11:03.899702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.812856+0000","last_change":"2023-11-27T15:20:37.679575+0000","last_active":"2023-11-27T15:22:02.812856+0000","last_peered":"2023-11-27T15:22:02.812856+0000","last_clean":"2023-11-27T15:22:02.812856+0000","last_became_active":"2023-11-27T15:20:37.679131+0000","last_became_peered":"2023-11-27T15:20:37.679131+0000","last_unstale":"2023-11-27T15:22:02.812856+0000","last_undegraded":"2023-11-27T15:22:02.812856+0000","last_fullsized":"2023-11-27T15:22:02.812856+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:03:41.337015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.908090+0000","last_change":"2023-11-27T15:20:32.657155+0000","last_active":"2023-11-27T15:22:02.908090+0000","last_peered":"2023-11-27T15:22:02.908090+0000","last_clean":"2023-11-27T15:22:02.908090+0000","last_became_active":"2023-11-27T15:20:32.656832+0000","last_became_peered":"2023-11-27T15:20:32.656832+0000","last_unstale":"2023-11-27T15:22:02.908090+0000","last_undegraded":"2023-11-27T15:22:02.908090+0000","last_fullsized":"2023-11-27T15:22:02.908090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:14:08.024415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.761021+0000","last_change":"2023-11-27T15:20:33.229215+0000","last_active":"2023-11-27T15:22:31.761021+0000","last_peered":"2023-11-27T15:22:31.761021+0000","last_clean":"2023-11-27T15:22:31.761021+0000","last_became_active":"2023-11-27T15:20:33.228616+0000","last_became_peered":"2023-11-27T15:20:33.228616+0000","last_unstale":"2023-11-27T15:22:31.761021+0000","last_undegraded":"2023-11-27T15:22:31.761021+0000","last_fullsized":"2023-11-27T15:22:31.761021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:16:50.126720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.817360+0000","last_change":"2023-11-27T15:20:37.672582+0000","last_active":"2023-11-27T15:22:02.817360+0000","last_peered":"2023-11-27T15:22:02.817360+0000","last_clean":"2023-11-27T15:22:02.817360+0000","last_became_active":"2023-11-27T15:20:37.672193+0000","last_became_peered":"2023-11-27T15:20:37.672193+0000","last_unstale":"2023-11-27T15:22:02.817360+0000","last_undegraded":"2023-11-27T15:22:02.817360+0000","last_fullsized":"2023-11-27T15:22:02.817360+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:00:48.323529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":131,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:40.449635+0000","last_change":"2023-11-27T15:20:32.647894+0000","last_active":"2023-11-27T15:22:40.449635+0000","last_peered":"2023-11-27T15:22:40.449635+0000","last_clean":"2023-11-27T15:22:40.449635+0000","last_became_active":"2023-11-27T15:20:32.647597+0000","last_became_peered":"2023-11-27T15:20:32.647597+0000","last_unstale":"2023-11-27T15:22:40.449635+0000","last_undegraded":"2023-11-27T15:22:40.449635+0000","last_fullsized":"2023-11-27T15:22:40.449635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:12:48.894187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.753452+0000","last_change":"2023-11-27T15:20:33.229382+0000","last_active":"2023-11-27T15:22:31.753452+0000","last_peered":"2023-11-27T15:22:31.753452+0000","last_clean":"2023-11-27T15:22:31.753452+0000","last_became_active":"2023-11-27T15:20:33.229031+0000","last_became_peered":"2023-11-27T15:20:33.229031+0000","last_unstale":"2023-11-27T15:22:31.753452+0000","last_undegraded":"2023-11-27T15:22:31.753452+0000","last_fullsized":"2023-11-27T15:22:31.753452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:35:27.898226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811592+0000","last_change":"2023-11-27T15:20:37.679131+0000","last_active":"2023-11-27T15:22:02.811592+0000","last_peered":"2023-11-27T15:22:02.811592+0000","last_clean":"2023-11-27T15:22:02.811592+0000","last_became_active":"2023-11-27T15:20:37.678290+0000","last_became_peered":"2023-11-27T15:20:37.678290+0000","last_unstale":"2023-11-27T15:22:02.811592+0000","last_undegraded":"2023-11-27T15:22:02.811592+0000","last_fullsized":"2023-11-27T15:22:02.811592+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:55:02.097941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.244489+0000","last_change":"2023-11-27T15:21:02.797374+0000","last_active":"2023-11-27T15:22:03.244489+0000","last_peered":"2023-11-27T15:22:03.244489+0000","last_clean":"2023-11-27T15:22:03.244489+0000","last_became_active":"2023-11-27T15:21:02.796076+0000","last_became_peered":"2023-11-27T15:21:02.796076+0000","last_unstale":"2023-11-27T15:22:03.244489+0000","last_undegraded":"2023-11-27T15:22:03.244489+0000","last_fullsized":"2023-11-27T15:22:03.244489+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:04:13.004951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.766526+0000","last_change":"2023-11-27T15:21:02.935104+0000","last_active":"2023-11-27T15:22:31.766526+0000","last_peered":"2023-11-27T15:22:31.766526+0000","last_clean":"2023-11-27T15:22:31.766526+0000","last_became_active":"2023-11-27T15:21:02.934486+0000","last_became_peered":"2023-11-27T15:21:02.934486+0000","last_unstale":"2023-11-27T15:22:31.766526+0000","last_undegraded":"2023-11-27T15:22:31.766526+0000","last_fullsized":"2023-11-27T15:22:31.766526+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:48:49.686999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811614+0000","last_change":"2023-11-27T15:20:37.670389+0000","last_active":"2023-11-27T15:22:02.811614+0000","last_peered":"2023-11-27T15:22:02.811614+0000","last_clean":"2023-11-27T15:22:02.811614+0000","last_became_active":"2023-11-27T15:20:37.669512+0000","last_became_peered":"2023-11-27T15:20:37.669512+0000","last_unstale":"2023-11-27T15:22:02.811614+0000","last_undegraded":"2023-11-27T15:22:02.811614+0000","last_fullsized":"2023-11-27T15:22:02.811614+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:25:51.789859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.908694+0000","last_change":"2023-11-27T15:20:32.658287+0000","last_active":"2023-11-27T15:22:02.908694+0000","last_peered":"2023-11-27T15:22:02.908694+0000","last_clean":"2023-11-27T15:22:02.908694+0000","last_became_active":"2023-11-27T15:20:32.657925+0000","last_became_peered":"2023-11-27T15:20:32.657925+0000","last_unstale":"2023-11-27T15:22:02.908694+0000","last_undegraded":"2023-11-27T15:22:02.908694+0000","last_fullsized":"2023-11-27T15:22:02.908694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:47:39.305351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.754013+0000","last_change":"2023-11-27T15:20:33.220240+0000","last_active":"2023-11-27T15:22:31.754013+0000","last_peered":"2023-11-27T15:22:31.754013+0000","last_clean":"2023-11-27T15:22:31.754013+0000","last_became_active":"2023-11-27T15:20:33.219962+0000","last_became_peered":"2023-11-27T15:20:33.219962+0000","last_unstale":"2023-11-27T15:22:31.754013+0000","last_undegraded":"2023-11-27T15:22:31.754013+0000","last_fullsized":"2023-11-27T15:22:31.754013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:00:01.276822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811329+0000","last_change":"2023-11-27T15:20:37.672203+0000","last_active":"2023-11-27T15:22:02.811329+0000","last_peered":"2023-11-27T15:22:02.811329+0000","last_clean":"2023-11-27T15:22:02.811329+0000","last_became_active":"2023-11-27T15:20:37.671149+0000","last_became_peered":"2023-11-27T15:20:37.671149+0000","last_unstale":"2023-11-27T15:22:02.811329+0000","last_undegraded":"2023-11-27T15:22:02.811329+0000","last_fullsized":"2023-11-27T15:22:02.811329+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:15:42.398912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'37","reported_seq":271,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:40.967635+0000","last_change":"2023-11-27T15:20:32.646637+0000","last_active":"2023-11-27T15:22:40.967635+0000","last_peered":"2023-11-27T15:22:40.967635+0000","last_clean":"2023-11-27T15:22:40.967635+0000","last_became_active":"2023-11-27T15:20:32.646292+0000","last_became_peered":"2023-11-27T15:20:32.646292+0000","last_unstale":"2023-11-27T15:22:40.967635+0000","last_undegraded":"2023-11-27T15:22:40.967635+0000","last_fullsized":"2023-11-27T15:22:40.967635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:57:01.372262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60369,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":204,"num_read_kb":162,"num_write":38,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.737253+0000","last_change":"2023-11-27T15:20:33.217078+0000","last_active":"2023-11-27T15:22:31.737253+0000","last_peered":"2023-11-27T15:22:31.737253+0000","last_clean":"2023-11-27T15:22:31.737253+0000","last_became_active":"2023-11-27T15:20:33.216787+0000","last_became_peered":"2023-11-27T15:20:33.216787+0000","last_unstale":"2023-11-27T15:22:31.737253+0000","last_undegraded":"2023-11-27T15:22:31.737253+0000","last_fullsized":"2023-11-27T15:22:31.737253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:39:29.760219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.909130+0000","last_change":"2023-11-27T15:20:37.671800+0000","last_active":"2023-11-27T15:22:02.909130+0000","last_peered":"2023-11-27T15:22:02.909130+0000","last_clean":"2023-11-27T15:22:02.909130+0000","last_became_active":"2023-11-27T15:20:37.671374+0000","last_became_peered":"2023-11-27T15:20:37.671374+0000","last_unstale":"2023-11-27T15:22:02.909130+0000","last_undegraded":"2023-11-27T15:22:02.909130+0000","last_fullsized":"2023-11-27T15:22:02.909130+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:38:25.460193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818948+0000","last_change":"2023-11-27T15:20:32.659508+0000","last_active":"2023-11-27T15:22:02.818948+0000","last_peered":"2023-11-27T15:22:02.818948+0000","last_clean":"2023-11-27T15:22:02.818948+0000","last_became_active":"2023-11-27T15:20:32.659226+0000","last_became_peered":"2023-11-27T15:20:32.659226+0000","last_unstale":"2023-11-27T15:22:02.818948+0000","last_undegraded":"2023-11-27T15:22:02.818948+0000","last_fullsized":"2023-11-27T15:22:02.818948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:18:06.852915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.750607+0000","last_change":"2023-11-27T15:20:33.230012+0000","last_active":"2023-11-27T15:22:31.750607+0000","last_peered":"2023-11-27T15:22:31.750607+0000","last_clean":"2023-11-27T15:22:31.750607+0000","last_became_active":"2023-11-27T15:20:33.225193+0000","last_became_peered":"2023-11-27T15:20:33.225193+0000","last_unstale":"2023-11-27T15:22:31.750607+0000","last_undegraded":"2023-11-27T15:22:31.750607+0000","last_fullsized":"2023-11-27T15:22:31.750607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:42:53.662577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.815274+0000","last_change":"2023-11-27T15:20:37.672430+0000","last_active":"2023-11-27T15:22:02.815274+0000","last_peered":"2023-11-27T15:22:02.815274+0000","last_clean":"2023-11-27T15:22:02.815274+0000","last_became_active":"2023-11-27T15:20:37.671982+0000","last_became_peered":"2023-11-27T15:20:37.671982+0000","last_unstale":"2023-11-27T15:22:02.815274+0000","last_undegraded":"2023-11-27T15:22:02.815274+0000","last_fullsized":"2023-11-27T15:22:02.815274+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:41:34.081786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.908478+0000","last_change":"2023-11-27T15:20:32.655015+0000","last_active":"2023-11-27T15:22:02.908478+0000","last_peered":"2023-11-27T15:22:02.908478+0000","last_clean":"2023-11-27T15:22:02.908478+0000","last_became_active":"2023-11-27T15:20:32.654662+0000","last_became_peered":"2023-11-27T15:20:32.654662+0000","last_unstale":"2023-11-27T15:22:02.908478+0000","last_undegraded":"2023-11-27T15:22:02.908478+0000","last_fullsized":"2023-11-27T15:22:02.908478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:49:03.386586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.754433+0000","last_change":"2023-11-27T15:20:33.233980+0000","last_active":"2023-11-27T15:22:31.754433+0000","last_peered":"2023-11-27T15:22:31.754433+0000","last_clean":"2023-11-27T15:22:31.754433+0000","last_became_active":"2023-11-27T15:20:33.233581+0000","last_became_peered":"2023-11-27T15:20:33.233581+0000","last_unstale":"2023-11-27T15:22:31.754433+0000","last_undegraded":"2023-11-27T15:22:31.754433+0000","last_fullsized":"2023-11-27T15:22:31.754433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:52:04.285704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811523+0000","last_change":"2023-11-27T15:20:37.682359+0000","last_active":"2023-11-27T15:22:02.811523+0000","last_peered":"2023-11-27T15:22:02.811523+0000","last_clean":"2023-11-27T15:22:02.811523+0000","last_became_active":"2023-11-27T15:20:37.681888+0000","last_became_peered":"2023-11-27T15:20:37.681888+0000","last_unstale":"2023-11-27T15:22:02.811523+0000","last_undegraded":"2023-11-27T15:22:02.811523+0000","last_fullsized":"2023-11-27T15:22:02.811523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:04:23.066492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":139,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:42.967416+0000","last_change":"2023-11-27T15:20:32.653388+0000","last_active":"2023-11-27T15:22:42.967416+0000","last_peered":"2023-11-27T15:22:42.967416+0000","last_clean":"2023-11-27T15:22:42.967416+0000","last_became_active":"2023-11-27T15:20:32.653007+0000","last_became_peered":"2023-11-27T15:20:32.653007+0000","last_unstale":"2023-11-27T15:22:42.967416+0000","last_undegraded":"2023-11-27T15:22:42.967416+0000","last_fullsized":"2023-11-27T15:22:42.967416+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:40:28.723669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1071,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.750879+0000","last_change":"2023-11-27T15:20:33.216539+0000","last_active":"2023-11-27T15:22:31.750879+0000","last_peered":"2023-11-27T15:22:31.750879+0000","last_clean":"2023-11-27T15:22:31.750879+0000","last_became_active":"2023-11-27T15:20:33.216166+0000","last_became_peered":"2023-11-27T15:20:33.216166+0000","last_unstale":"2023-11-27T15:22:31.750879+0000","last_undegraded":"2023-11-27T15:22:31.750879+0000","last_fullsized":"2023-11-27T15:22:31.750879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:17:42.918928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.902913+0000","last_change":"2023-11-27T15:20:37.675735+0000","last_active":"2023-11-27T15:22:02.902913+0000","last_peered":"2023-11-27T15:22:02.902913+0000","last_clean":"2023-11-27T15:22:02.902913+0000","last_became_active":"2023-11-27T15:20:37.669685+0000","last_became_peered":"2023-11-27T15:20:37.669685+0000","last_unstale":"2023-11-27T15:22:02.902913+0000","last_undegraded":"2023-11-27T15:22:02.902913+0000","last_fullsized":"2023-11-27T15:22:02.902913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:55:44.635583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:05.499173+0000","last_change":"2023-11-27T15:22:05.499173+0000","last_active":"2023-11-27T15:22:05.499173+0000","last_peered":"2023-11-27T15:22:05.499173+0000","last_clean":"2023-11-27T15:22:05.499173+0000","last_became_active":"2023-11-27T15:22:02.821261+0000","last_became_peered":"2023-11-27T15:22:02.821261+0000","last_unstale":"2023-11-27T15:22:05.499173+0000","last_undegraded":"2023-11-27T15:22:05.499173+0000","last_fullsized":"2023-11-27T15:22:05.499173+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:16:43.958003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.748850+0000","last_change":"2023-11-27T15:20:33.225019+0000","last_active":"2023-11-27T15:22:31.748850+0000","last_peered":"2023-11-27T15:22:31.748850+0000","last_clean":"2023-11-27T15:22:31.748850+0000","last_became_active":"2023-11-27T15:20:33.224622+0000","last_became_peered":"2023-11-27T15:20:33.224622+0000","last_unstale":"2023-11-27T15:22:31.748850+0000","last_undegraded":"2023-11-27T15:22:31.748850+0000","last_fullsized":"2023-11-27T15:22:31.748850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:18:00.816094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.908162+0000","last_change":"2023-11-27T15:20:37.680808+0000","last_active":"2023-11-27T15:22:02.908162+0000","last_peered":"2023-11-27T15:22:02.908162+0000","last_clean":"2023-11-27T15:22:02.908162+0000","last_became_active":"2023-11-27T15:20:37.680348+0000","last_became_peered":"2023-11-27T15:20:37.680348+0000","last_unstale":"2023-11-27T15:22:02.908162+0000","last_undegraded":"2023-11-27T15:22:02.908162+0000","last_fullsized":"2023-11-27T15:22:02.908162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:37:19.386150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.819004+0000","last_change":"2023-11-27T15:20:32.651093+0000","last_active":"2023-11-27T15:22:02.819004+0000","last_peered":"2023-11-27T15:22:02.819004+0000","last_clean":"2023-11-27T15:22:02.819004+0000","last_became_active":"2023-11-27T15:20:32.650657+0000","last_became_peered":"2023-11-27T15:20:32.650657+0000","last_unstale":"2023-11-27T15:22:02.819004+0000","last_undegraded":"2023-11-27T15:22:02.819004+0000","last_fullsized":"2023-11-27T15:22:02.819004+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:56:44.895712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.749518+0000","last_change":"2023-11-27T15:20:33.224806+0000","last_active":"2023-11-27T15:22:31.749518+0000","last_peered":"2023-11-27T15:22:31.749518+0000","last_clean":"2023-11-27T15:22:31.749518+0000","last_became_active":"2023-11-27T15:20:33.224362+0000","last_became_peered":"2023-11-27T15:20:33.224362+0000","last_unstale":"2023-11-27T15:22:31.749518+0000","last_undegraded":"2023-11-27T15:22:31.749518+0000","last_fullsized":"2023-11-27T15:22:31.749518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:20:50.902402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818048+0000","last_change":"2023-11-27T15:20:37.849274+0000","last_active":"2023-11-27T15:22:02.818048+0000","last_peered":"2023-11-27T15:22:02.818048+0000","last_clean":"2023-11-27T15:22:02.818048+0000","last_became_active":"2023-11-27T15:20:37.848839+0000","last_became_peered":"2023-11-27T15:20:37.848839+0000","last_unstale":"2023-11-27T15:22:02.818048+0000","last_undegraded":"2023-11-27T15:22:02.818048+0000","last_fullsized":"2023-11-27T15:22:02.818048+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:49:32.136447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811304+0000","last_change":"2023-11-27T15:20:32.660835+0000","last_active":"2023-11-27T15:22:02.811304+0000","last_peered":"2023-11-27T15:22:02.811304+0000","last_clean":"2023-11-27T15:22:02.811304+0000","last_became_active":"2023-11-27T15:20:32.660431+0000","last_became_peered":"2023-11-27T15:20:32.660431+0000","last_unstale":"2023-11-27T15:22:02.811304+0000","last_undegraded":"2023-11-27T15:22:02.811304+0000","last_fullsized":"2023-11-27T15:22:02.811304+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:48:52.707533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.759772+0000","last_change":"2023-11-27T15:20:33.245772+0000","last_active":"2023-11-27T15:22:31.759772+0000","last_peered":"2023-11-27T15:22:31.759772+0000","last_clean":"2023-11-27T15:22:31.759772+0000","last_became_active":"2023-11-27T15:20:33.245468+0000","last_became_peered":"2023-11-27T15:20:33.245468+0000","last_unstale":"2023-11-27T15:22:31.759772+0000","last_undegraded":"2023-11-27T15:22:31.759772+0000","last_fullsized":"2023-11-27T15:22:31.759772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:58:59.950032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.902964+0000","last_change":"2023-11-27T15:20:37.678972+0000","last_active":"2023-11-27T15:22:02.902964+0000","last_peered":"2023-11-27T15:22:02.902964+0000","last_clean":"2023-11-27T15:22:02.902964+0000","last_became_active":"2023-11-27T15:20:37.678304+0000","last_became_peered":"2023-11-27T15:20:37.678304+0000","last_unstale":"2023-11-27T15:22:02.902964+0000","last_undegraded":"2023-11-27T15:22:02.902964+0000","last_fullsized":"2023-11-27T15:22:02.902964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:05:43.664151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.244402+0000","last_change":"2023-11-27T15:20:32.661739+0000","last_active":"2023-11-27T15:22:03.244402+0000","last_peered":"2023-11-27T15:22:03.244402+0000","last_clean":"2023-11-27T15:22:03.244402+0000","last_became_active":"2023-11-27T15:20:32.661401+0000","last_became_peered":"2023-11-27T15:20:32.661401+0000","last_unstale":"2023-11-27T15:22:03.244402+0000","last_undegraded":"2023-11-27T15:22:03.244402+0000","last_fullsized":"2023-11-27T15:22:03.244402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:32:44.035448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.749301+0000","last_change":"2023-11-27T15:20:33.235595+0000","last_active":"2023-11-27T15:22:31.749301+0000","last_peered":"2023-11-27T15:22:31.749301+0000","last_clean":"2023-11-27T15:22:31.749301+0000","last_became_active":"2023-11-27T15:20:33.235277+0000","last_became_peered":"2023-11-27T15:20:33.235277+0000","last_unstale":"2023-11-27T15:22:31.749301+0000","last_undegraded":"2023-11-27T15:22:31.749301+0000","last_fullsized":"2023-11-27T15:22:31.749301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:13:59.365599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.903185+0000","last_change":"2023-11-27T15:21:02.934194+0000","last_active":"2023-11-27T15:22:02.903185+0000","last_peered":"2023-11-27T15:22:02.903185+0000","last_clean":"2023-11-27T15:22:02.903185+0000","last_became_active":"2023-11-27T15:21:02.933555+0000","last_became_peered":"2023-11-27T15:21:02.933555+0000","last_unstale":"2023-11-27T15:22:02.903185+0000","last_undegraded":"2023-11-27T15:22:02.903185+0000","last_fullsized":"2023-11-27T15:22:02.903185+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:16:19.222019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.755634+0000","last_change":"2023-11-27T15:21:02.784351+0000","last_active":"2023-11-27T15:22:31.755634+0000","last_peered":"2023-11-27T15:22:31.755634+0000","last_clean":"2023-11-27T15:22:31.755634+0000","last_became_active":"2023-11-27T15:21:02.783996+0000","last_became_peered":"2023-11-27T15:21:02.783996+0000","last_unstale":"2023-11-27T15:22:31.755634+0000","last_undegraded":"2023-11-27T15:22:31.755634+0000","last_fullsized":"2023-11-27T15:22:31.755634+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:17:24.113607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":133,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:40.963521+0000","last_change":"2023-11-27T15:20:32.659823+0000","last_active":"2023-11-27T15:22:40.963521+0000","last_peered":"2023-11-27T15:22:40.963521+0000","last_clean":"2023-11-27T15:22:40.963521+0000","last_became_active":"2023-11-27T15:20:32.659472+0000","last_became_peered":"2023-11-27T15:20:32.659472+0000","last_unstale":"2023-11-27T15:22:40.963521+0000","last_undegraded":"2023-11-27T15:22:40.963521+0000","last_fullsized":"2023-11-27T15:22:40.963521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:08:32.142408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.244444+0000","last_change":"2023-11-27T15:20:37.671494+0000","last_active":"2023-11-27T15:22:03.244444+0000","last_peered":"2023-11-27T15:22:03.244444+0000","last_clean":"2023-11-27T15:22:03.244444+0000","last_became_active":"2023-11-27T15:20:37.671010+0000","last_became_peered":"2023-11-27T15:20:37.671010+0000","last_unstale":"2023-11-27T15:22:03.244444+0000","last_undegraded":"2023-11-27T15:22:03.244444+0000","last_fullsized":"2023-11-27T15:22:03.244444+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:51:51.646563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.758695+0000","last_change":"2023-11-27T15:20:33.241669+0000","last_active":"2023-11-27T15:22:31.758695+0000","last_peered":"2023-11-27T15:22:31.758695+0000","last_clean":"2023-11-27T15:22:31.758695+0000","last_became_active":"2023-11-27T15:20:33.241373+0000","last_became_peered":"2023-11-27T15:20:33.241373+0000","last_unstale":"2023-11-27T15:22:31.758695+0000","last_undegraded":"2023-11-27T15:22:31.758695+0000","last_fullsized":"2023-11-27T15:22:31.758695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:03:47.587644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811591+0000","last_change":"2023-11-27T15:20:32.647596+0000","last_active":"2023-11-27T15:22:02.811591+0000","last_peered":"2023-11-27T15:22:02.811591+0000","last_clean":"2023-11-27T15:22:02.811591+0000","last_became_active":"2023-11-27T15:20:32.647228+0000","last_became_peered":"2023-11-27T15:20:32.647228+0000","last_unstale":"2023-11-27T15:22:02.811591+0000","last_undegraded":"2023-11-27T15:22:02.811591+0000","last_fullsized":"2023-11-27T15:22:02.811591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:37:28.551859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.244098+0000","last_change":"2023-11-27T15:19:30.114855+0000","last_active":"2023-11-27T15:22:03.244098+0000","last_peered":"2023-11-27T15:22:03.244098+0000","last_clean":"2023-11-27T15:22:03.244098+0000","last_became_active":"2023-11-27T15:19:30.114419+0000","last_became_peered":"2023-11-27T15:19:30.114419+0000","last_unstale":"2023-11-27T15:22:03.244098+0000","last_undegraded":"2023-11-27T15:22:03.244098+0000","last_fullsized":"2023-11-27T15:22:03.244098+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:20.216695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:20.216695+0000","last_clean_scrub_stamp":"2023-11-27T15:16:20.216695+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:02:43.854910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.908738+0000","last_change":"2023-11-27T15:20:37.850539+0000","last_active":"2023-11-27T15:22:02.908738+0000","last_peered":"2023-11-27T15:22:02.908738+0000","last_clean":"2023-11-27T15:22:02.908738+0000","last_became_active":"2023-11-27T15:20:37.850044+0000","last_became_peered":"2023-11-27T15:20:37.850044+0000","last_unstale":"2023-11-27T15:22:02.908738+0000","last_undegraded":"2023-11-27T15:22:02.908738+0000","last_fullsized":"2023-11-27T15:22:02.908738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:54:36.190506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.741984+0000","last_change":"2023-11-27T15:20:33.211319+0000","last_active":"2023-11-27T15:22:31.741984+0000","last_peered":"2023-11-27T15:22:31.741984+0000","last_clean":"2023-11-27T15:22:31.741984+0000","last_became_active":"2023-11-27T15:20:33.211004+0000","last_became_peered":"2023-11-27T15:20:33.211004+0000","last_unstale":"2023-11-27T15:22:31.741984+0000","last_undegraded":"2023-11-27T15:22:31.741984+0000","last_fullsized":"2023-11-27T15:22:31.741984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:52:10.543307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.244696+0000","last_change":"2023-11-27T15:20:32.655003+0000","last_active":"2023-11-27T15:22:03.244696+0000","last_peered":"2023-11-27T15:22:03.244696+0000","last_clean":"2023-11-27T15:22:03.244696+0000","last_became_active":"2023-11-27T15:20:32.654517+0000","last_became_peered":"2023-11-27T15:20:32.654517+0000","last_unstale":"2023-11-27T15:22:03.244696+0000","last_undegraded":"2023-11-27T15:22:03.244696+0000","last_fullsized":"2023-11-27T15:22:03.244696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:18:43.790411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.815270+0000","last_change":"2023-11-27T15:20:37.673183+0000","last_active":"2023-11-27T15:22:02.815270+0000","last_peered":"2023-11-27T15:22:02.815270+0000","last_clean":"2023-11-27T15:22:02.815270+0000","last_became_active":"2023-11-27T15:20:37.672838+0000","last_became_peered":"2023-11-27T15:20:37.672838+0000","last_unstale":"2023-11-27T15:22:02.815270+0000","last_undegraded":"2023-11-27T15:22:02.815270+0000","last_fullsized":"2023-11-27T15:22:02.815270+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:42:26.039321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818752+0000","last_change":"2023-11-27T15:20:32.659540+0000","last_active":"2023-11-27T15:22:02.818752+0000","last_peered":"2023-11-27T15:22:02.818752+0000","last_clean":"2023-11-27T15:22:02.818752+0000","last_became_active":"2023-11-27T15:20:32.659211+0000","last_became_peered":"2023-11-27T15:20:32.659211+0000","last_unstale":"2023-11-27T15:22:02.818752+0000","last_undegraded":"2023-11-27T15:22:02.818752+0000","last_fullsized":"2023-11-27T15:22:02.818752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:32:29.624516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.757012+0000","last_change":"2023-11-27T15:20:33.216405+0000","last_active":"2023-11-27T15:22:31.757012+0000","last_peered":"2023-11-27T15:22:31.757012+0000","last_clean":"2023-11-27T15:22:31.757012+0000","last_became_active":"2023-11-27T15:20:33.216072+0000","last_became_peered":"2023-11-27T15:20:33.216072+0000","last_unstale":"2023-11-27T15:22:31.757012+0000","last_undegraded":"2023-11-27T15:22:31.757012+0000","last_fullsized":"2023-11-27T15:22:31.757012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:24:07.496562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.817387+0000","last_change":"2023-11-27T15:20:37.849753+0000","last_active":"2023-11-27T15:22:02.817387+0000","last_peered":"2023-11-27T15:22:02.817387+0000","last_clean":"2023-11-27T15:22:02.817387+0000","last_became_active":"2023-11-27T15:20:37.848576+0000","last_became_peered":"2023-11-27T15:20:37.848576+0000","last_unstale":"2023-11-27T15:22:02.817387+0000","last_undegraded":"2023-11-27T15:22:02.817387+0000","last_fullsized":"2023-11-27T15:22:02.817387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:33:46.286142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'8","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:41.517782+0000","last_change":"2023-11-27T15:20:32.652739+0000","last_active":"2023-11-27T15:22:41.517782+0000","last_peered":"2023-11-27T15:22:41.517782+0000","last_clean":"2023-11-27T15:22:41.517782+0000","last_became_active":"2023-11-27T15:20:32.652314+0000","last_became_peered":"2023-11-27T15:20:32.652314+0000","last_unstale":"2023-11-27T15:22:41.517782+0000","last_undegraded":"2023-11-27T15:22:41.517782+0000","last_fullsized":"2023-11-27T15:22:41.517782+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:00:14.008740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.758452+0000","last_change":"2023-11-27T15:20:33.240552+0000","last_active":"2023-11-27T15:22:31.758452+0000","last_peered":"2023-11-27T15:22:31.758452+0000","last_clean":"2023-11-27T15:22:31.758452+0000","last_became_active":"2023-11-27T15:20:33.239885+0000","last_became_peered":"2023-11-27T15:20:33.239885+0000","last_unstale":"2023-11-27T15:22:31.758452+0000","last_undegraded":"2023-11-27T15:22:31.758452+0000","last_fullsized":"2023-11-27T15:22:31.758452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:44:48.111242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818035+0000","last_change":"2023-11-27T15:20:37.669895+0000","last_active":"2023-11-27T15:22:02.818035+0000","last_peered":"2023-11-27T15:22:02.818035+0000","last_clean":"2023-11-27T15:22:02.818035+0000","last_became_active":"2023-11-27T15:20:37.669489+0000","last_became_peered":"2023-11-27T15:20:37.669489+0000","last_unstale":"2023-11-27T15:22:02.818035+0000","last_undegraded":"2023-11-27T15:22:02.818035+0000","last_fullsized":"2023-11-27T15:22:02.818035+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:06:13.495056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.812082+0000","last_change":"2023-11-27T15:20:32.656773+0000","last_active":"2023-11-27T15:22:02.812082+0000","last_peered":"2023-11-27T15:22:02.812082+0000","last_clean":"2023-11-27T15:22:02.812082+0000","last_became_active":"2023-11-27T15:20:32.656451+0000","last_became_peered":"2023-11-27T15:20:32.656451+0000","last_unstale":"2023-11-27T15:22:02.812082+0000","last_undegraded":"2023-11-27T15:22:02.812082+0000","last_fullsized":"2023-11-27T15:22:02.812082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:03:12.399313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.765942+0000","last_change":"2023-11-27T15:20:33.231685+0000","last_active":"2023-11-27T15:22:31.765942+0000","last_peered":"2023-11-27T15:22:31.765942+0000","last_clean":"2023-11-27T15:22:31.765942+0000","last_became_active":"2023-11-27T15:20:33.228265+0000","last_became_peered":"2023-11-27T15:20:33.228265+0000","last_unstale":"2023-11-27T15:22:31.765942+0000","last_undegraded":"2023-11-27T15:22:31.765942+0000","last_fullsized":"2023-11-27T15:22:31.765942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:53:48.855000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811795+0000","last_change":"2023-11-27T15:21:02.935830+0000","last_active":"2023-11-27T15:22:02.811795+0000","last_peered":"2023-11-27T15:22:02.811795+0000","last_clean":"2023-11-27T15:22:02.811795+0000","last_became_active":"2023-11-27T15:21:02.935412+0000","last_became_peered":"2023-11-27T15:21:02.935412+0000","last_unstale":"2023-11-27T15:22:02.811795+0000","last_undegraded":"2023-11-27T15:22:02.811795+0000","last_fullsized":"2023-11-27T15:22:02.811795+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:22:23.956927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.244630+0000","last_change":"2023-11-27T15:20:32.660424+0000","last_active":"2023-11-27T15:22:03.244630+0000","last_peered":"2023-11-27T15:22:03.244630+0000","last_clean":"2023-11-27T15:22:03.244630+0000","last_became_active":"2023-11-27T15:20:32.660112+0000","last_became_peered":"2023-11-27T15:20:32.660112+0000","last_unstale":"2023-11-27T15:22:03.244630+0000","last_undegraded":"2023-11-27T15:22:03.244630+0000","last_fullsized":"2023-11-27T15:22:03.244630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:05:10.594173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.745264+0000","last_change":"2023-11-27T15:20:33.227677+0000","last_active":"2023-11-27T15:22:31.745264+0000","last_peered":"2023-11-27T15:22:31.745264+0000","last_clean":"2023-11-27T15:22:31.745264+0000","last_became_active":"2023-11-27T15:20:33.227334+0000","last_became_peered":"2023-11-27T15:20:33.227334+0000","last_unstale":"2023-11-27T15:22:31.745264+0000","last_undegraded":"2023-11-27T15:22:31.745264+0000","last_fullsized":"2023-11-27T15:22:31.745264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:19:08.573620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818782+0000","last_change":"2023-11-27T15:21:02.934516+0000","last_active":"2023-11-27T15:22:02.818782+0000","last_peered":"2023-11-27T15:22:02.818782+0000","last_clean":"2023-11-27T15:22:02.818782+0000","last_became_active":"2023-11-27T15:21:02.934012+0000","last_became_peered":"2023-11-27T15:21:02.934012+0000","last_unstale":"2023-11-27T15:22:02.818782+0000","last_undegraded":"2023-11-27T15:22:02.818782+0000","last_fullsized":"2023-11-27T15:22:02.818782+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:54:54.500574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.817444+0000","last_change":"2023-11-27T15:20:32.657935+0000","last_active":"2023-11-27T15:22:02.817444+0000","last_peered":"2023-11-27T15:22:02.817444+0000","last_clean":"2023-11-27T15:22:02.817444+0000","last_became_active":"2023-11-27T15:20:32.657579+0000","last_became_peered":"2023-11-27T15:20:32.657579+0000","last_unstale":"2023-11-27T15:22:02.817444+0000","last_undegraded":"2023-11-27T15:22:02.817444+0000","last_fullsized":"2023-11-27T15:22:02.817444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:00:47.684043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.767085+0000","last_change":"2023-11-27T15:20:33.225817+0000","last_active":"2023-11-27T15:22:31.767085+0000","last_peered":"2023-11-27T15:22:31.767085+0000","last_clean":"2023-11-27T15:22:31.767085+0000","last_became_active":"2023-11-27T15:20:33.225438+0000","last_became_peered":"2023-11-27T15:20:33.225438+0000","last_unstale":"2023-11-27T15:22:31.767085+0000","last_undegraded":"2023-11-27T15:22:31.767085+0000","last_fullsized":"2023-11-27T15:22:31.767085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:07:45.956918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811681+0000","last_change":"2023-11-27T15:21:02.934304+0000","last_active":"2023-11-27T15:22:02.811681+0000","last_peered":"2023-11-27T15:22:02.811681+0000","last_clean":"2023-11-27T15:22:02.811681+0000","last_became_active":"2023-11-27T15:21:02.933709+0000","last_became_peered":"2023-11-27T15:21:02.933709+0000","last_unstale":"2023-11-27T15:22:02.811681+0000","last_undegraded":"2023-11-27T15:22:02.811681+0000","last_fullsized":"2023-11-27T15:22:02.811681+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:52:00.767226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.119947+0000","last_change":"2023-11-27T15:22:03.119402+0000","last_active":"2023-11-27T15:22:03.119947+0000","last_peered":"2023-11-27T15:22:03.119947+0000","last_clean":"2023-11-27T15:22:03.119947+0000","last_became_active":"2023-11-27T15:22:02.819940+0000","last_became_peered":"2023-11-27T15:22:02.819940+0000","last_unstale":"2023-11-27T15:22:03.119947+0000","last_undegraded":"2023-11-27T15:22:03.119947+0000","last_fullsized":"2023-11-27T15:22:03.119947+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:41:14.055640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.758801+0000","last_change":"2023-11-27T15:20:33.235755+0000","last_active":"2023-11-27T15:22:31.758801+0000","last_peered":"2023-11-27T15:22:31.758801+0000","last_clean":"2023-11-27T15:22:31.758801+0000","last_became_active":"2023-11-27T15:20:33.234576+0000","last_became_peered":"2023-11-27T15:20:33.234576+0000","last_unstale":"2023-11-27T15:22:31.758801+0000","last_undegraded":"2023-11-27T15:22:31.758801+0000","last_fullsized":"2023-11-27T15:22:31.758801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:15:16.743978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811797+0000","last_change":"2023-11-27T15:20:37.849121+0000","last_active":"2023-11-27T15:22:02.811797+0000","last_peered":"2023-11-27T15:22:02.811797+0000","last_clean":"2023-11-27T15:22:02.811797+0000","last_became_active":"2023-11-27T15:20:37.848683+0000","last_became_peered":"2023-11-27T15:20:37.848683+0000","last_unstale":"2023-11-27T15:22:02.811797+0000","last_undegraded":"2023-11-27T15:22:02.811797+0000","last_fullsized":"2023-11-27T15:22:02.811797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:37:37.945965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.817973+0000","last_change":"2023-11-27T15:20:32.655896+0000","last_active":"2023-11-27T15:22:02.817973+0000","last_peered":"2023-11-27T15:22:02.817973+0000","last_clean":"2023-11-27T15:22:02.817973+0000","last_became_active":"2023-11-27T15:20:32.655623+0000","last_became_peered":"2023-11-27T15:20:32.655623+0000","last_unstale":"2023-11-27T15:22:02.817973+0000","last_undegraded":"2023-11-27T15:22:02.817973+0000","last_fullsized":"2023-11-27T15:22:02.817973+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:58:52.716210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.753875+0000","last_change":"2023-11-27T15:20:33.227318+0000","last_active":"2023-11-27T15:22:31.753875+0000","last_peered":"2023-11-27T15:22:31.753875+0000","last_clean":"2023-11-27T15:22:31.753875+0000","last_became_active":"2023-11-27T15:20:33.226944+0000","last_became_peered":"2023-11-27T15:20:33.226944+0000","last_unstale":"2023-11-27T15:22:31.753875+0000","last_undegraded":"2023-11-27T15:22:31.753875+0000","last_fullsized":"2023-11-27T15:22:31.753875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:13:26.176414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.244531+0000","last_change":"2023-11-27T15:20:37.669098+0000","last_active":"2023-11-27T15:22:03.244531+0000","last_peered":"2023-11-27T15:22:03.244531+0000","last_clean":"2023-11-27T15:22:03.244531+0000","last_became_active":"2023-11-27T15:20:37.668693+0000","last_became_peered":"2023-11-27T15:20:37.668693+0000","last_unstale":"2023-11-27T15:22:03.244531+0000","last_undegraded":"2023-11-27T15:22:03.244531+0000","last_fullsized":"2023-11-27T15:22:03.244531+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:45:51.479637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.819574+0000","last_change":"2023-11-27T15:20:32.660285+0000","last_active":"2023-11-27T15:22:02.819574+0000","last_peered":"2023-11-27T15:22:02.819574+0000","last_clean":"2023-11-27T15:22:02.819574+0000","last_became_active":"2023-11-27T15:20:32.659982+0000","last_became_peered":"2023-11-27T15:20:32.659982+0000","last_unstale":"2023-11-27T15:22:02.819574+0000","last_undegraded":"2023-11-27T15:22:02.819574+0000","last_fullsized":"2023-11-27T15:22:02.819574+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:50:18.637710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.750955+0000","last_change":"2023-11-27T15:20:33.226629+0000","last_active":"2023-11-27T15:22:31.750955+0000","last_peered":"2023-11-27T15:22:31.750955+0000","last_clean":"2023-11-27T15:22:31.750955+0000","last_became_active":"2023-11-27T15:20:33.226183+0000","last_became_peered":"2023-11-27T15:20:33.226183+0000","last_unstale":"2023-11-27T15:22:31.750955+0000","last_undegraded":"2023-11-27T15:22:31.750955+0000","last_fullsized":"2023-11-27T15:22:31.750955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:17:31.612135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.812710+0000","last_change":"2023-11-27T15:20:37.680987+0000","last_active":"2023-11-27T15:22:02.812710+0000","last_peered":"2023-11-27T15:22:02.812710+0000","last_clean":"2023-11-27T15:22:02.812710+0000","last_became_active":"2023-11-27T15:20:37.680219+0000","last_became_peered":"2023-11-27T15:20:37.680219+0000","last_unstale":"2023-11-27T15:22:02.812710+0000","last_undegraded":"2023-11-27T15:22:02.812710+0000","last_fullsized":"2023-11-27T15:22:02.812710+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:55:34.725659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.908876+0000","last_change":"2023-11-27T15:20:32.655035+0000","last_active":"2023-11-27T15:22:02.908876+0000","last_peered":"2023-11-27T15:22:02.908876+0000","last_clean":"2023-11-27T15:22:02.908876+0000","last_became_active":"2023-11-27T15:20:32.654687+0000","last_became_peered":"2023-11-27T15:20:32.654687+0000","last_unstale":"2023-11-27T15:22:02.908876+0000","last_undegraded":"2023-11-27T15:22:02.908876+0000","last_fullsized":"2023-11-27T15:22:02.908876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:57:01.888463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.759364+0000","last_change":"2023-11-27T15:20:33.245908+0000","last_active":"2023-11-27T15:22:31.759364+0000","last_peered":"2023-11-27T15:22:31.759364+0000","last_clean":"2023-11-27T15:22:31.759364+0000","last_became_active":"2023-11-27T15:20:33.245576+0000","last_became_peered":"2023-11-27T15:20:33.245576+0000","last_unstale":"2023-11-27T15:22:31.759364+0000","last_undegraded":"2023-11-27T15:22:31.759364+0000","last_fullsized":"2023-11-27T15:22:31.759364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:27:55.968118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.907992+0000","last_change":"2023-11-27T15:20:37.674253+0000","last_active":"2023-11-27T15:22:02.907992+0000","last_peered":"2023-11-27T15:22:02.907992+0000","last_clean":"2023-11-27T15:22:02.907992+0000","last_became_active":"2023-11-27T15:20:37.673708+0000","last_became_peered":"2023-11-27T15:20:37.673708+0000","last_unstale":"2023-11-27T15:22:02.907992+0000","last_undegraded":"2023-11-27T15:22:02.907992+0000","last_fullsized":"2023-11-27T15:22:02.907992+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:39:38.298957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.903185+0000","last_change":"2023-11-27T15:20:32.659605+0000","last_active":"2023-11-27T15:22:02.903185+0000","last_peered":"2023-11-27T15:22:02.903185+0000","last_clean":"2023-11-27T15:22:02.903185+0000","last_became_active":"2023-11-27T15:20:32.659186+0000","last_became_peered":"2023-11-27T15:20:32.659186+0000","last_unstale":"2023-11-27T15:22:02.903185+0000","last_undegraded":"2023-11-27T15:22:02.903185+0000","last_fullsized":"2023-11-27T15:22:02.903185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:38:50.310551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.745773+0000","last_change":"2023-11-27T15:20:33.227914+0000","last_active":"2023-11-27T15:22:31.745773+0000","last_peered":"2023-11-27T15:22:31.745773+0000","last_clean":"2023-11-27T15:22:31.745773+0000","last_became_active":"2023-11-27T15:20:33.227572+0000","last_became_peered":"2023-11-27T15:20:33.227572+0000","last_unstale":"2023-11-27T15:22:31.745773+0000","last_undegraded":"2023-11-27T15:22:31.745773+0000","last_fullsized":"2023-11-27T15:22:31.745773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:57:20.426893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.747905+0000","last_change":"2023-11-27T15:20:33.215605+0000","last_active":"2023-11-27T15:22:31.747905+0000","last_peered":"2023-11-27T15:22:31.747905+0000","last_clean":"2023-11-27T15:22:31.747905+0000","last_became_active":"2023-11-27T15:20:33.215295+0000","last_became_peered":"2023-11-27T15:20:33.215295+0000","last_unstale":"2023-11-27T15:22:31.747905+0000","last_undegraded":"2023-11-27T15:22:31.747905+0000","last_fullsized":"2023-11-27T15:22:31.747905+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:16:48.125214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.748947+0000","last_change":"2023-11-27T15:20:33.214014+0000","last_active":"2023-11-27T15:22:31.748947+0000","last_peered":"2023-11-27T15:22:31.748947+0000","last_clean":"2023-11-27T15:22:31.748947+0000","last_became_active":"2023-11-27T15:20:33.213685+0000","last_became_peered":"2023-11-27T15:20:33.213685+0000","last_unstale":"2023-11-27T15:22:31.748947+0000","last_undegraded":"2023-11-27T15:22:31.748947+0000","last_fullsized":"2023-11-27T15:22:31.748947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:53:01.707335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.760580+0000","last_change":"2023-11-27T15:20:33.229722+0000","last_active":"2023-11-27T15:22:31.760580+0000","last_peered":"2023-11-27T15:22:31.760580+0000","last_clean":"2023-11-27T15:22:31.760580+0000","last_became_active":"2023-11-27T15:20:33.229372+0000","last_became_peered":"2023-11-27T15:20:33.229372+0000","last_unstale":"2023-11-27T15:22:31.760580+0000","last_undegraded":"2023-11-27T15:22:31.760580+0000","last_fullsized":"2023-11-27T15:22:31.760580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:40:05.300812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.762612+0000","last_change":"2023-11-27T15:20:33.235047+0000","last_active":"2023-11-27T15:22:31.762612+0000","last_peered":"2023-11-27T15:22:31.762612+0000","last_clean":"2023-11-27T15:22:31.762612+0000","last_became_active":"2023-11-27T15:20:33.233078+0000","last_became_peered":"2023-11-27T15:20:33.233078+0000","last_unstale":"2023-11-27T15:22:31.762612+0000","last_undegraded":"2023-11-27T15:22:31.762612+0000","last_fullsized":"2023-11-27T15:22:31.762612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:45:51.860734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.751664+0000","last_change":"2023-11-27T15:20:33.229118+0000","last_active":"2023-11-27T15:22:31.751664+0000","last_peered":"2023-11-27T15:22:31.751664+0000","last_clean":"2023-11-27T15:22:31.751664+0000","last_became_active":"2023-11-27T15:20:33.228748+0000","last_became_peered":"2023-11-27T15:20:33.228748+0000","last_unstale":"2023-11-27T15:22:31.751664+0000","last_undegraded":"2023-11-27T15:22:31.751664+0000","last_fullsized":"2023-11-27T15:22:31.751664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:19:11.419230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.746259+0000","last_change":"2023-11-27T15:20:33.222714+0000","last_active":"2023-11-27T15:22:31.746259+0000","last_peered":"2023-11-27T15:22:31.746259+0000","last_clean":"2023-11-27T15:22:31.746259+0000","last_became_active":"2023-11-27T15:20:33.222396+0000","last_became_peered":"2023-11-27T15:20:33.222396+0000","last_unstale":"2023-11-27T15:22:31.746259+0000","last_undegraded":"2023-11-27T15:22:31.746259+0000","last_fullsized":"2023-11-27T15:22:31.746259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:52:29.993847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.754172+0000","last_change":"2023-11-27T15:20:33.223568+0000","last_active":"2023-11-27T15:22:31.754172+0000","last_peered":"2023-11-27T15:22:31.754172+0000","last_clean":"2023-11-27T15:22:31.754172+0000","last_became_active":"2023-11-27T15:20:33.220687+0000","last_became_peered":"2023-11-27T15:20:33.220687+0000","last_unstale":"2023-11-27T15:22:31.754172+0000","last_undegraded":"2023-11-27T15:22:31.754172+0000","last_fullsized":"2023-11-27T15:22:31.754172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:48:28.297235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.753086+0000","last_change":"2023-11-27T15:21:02.786992+0000","last_active":"2023-11-27T15:22:31.753086+0000","last_peered":"2023-11-27T15:22:31.753086+0000","last_clean":"2023-11-27T15:22:31.753086+0000","last_became_active":"2023-11-27T15:21:02.786644+0000","last_became_peered":"2023-11-27T15:21:02.786644+0000","last_unstale":"2023-11-27T15:22:31.753086+0000","last_undegraded":"2023-11-27T15:22:31.753086+0000","last_fullsized":"2023-11-27T15:22:31.753086+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:28:04.321999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.756953+0000","last_change":"2023-11-27T15:20:33.245815+0000","last_active":"2023-11-27T15:22:31.756953+0000","last_peered":"2023-11-27T15:22:31.756953+0000","last_clean":"2023-11-27T15:22:31.756953+0000","last_became_active":"2023-11-27T15:20:33.245243+0000","last_became_peered":"2023-11-27T15:20:33.245243+0000","last_unstale":"2023-11-27T15:22:31.756953+0000","last_undegraded":"2023-11-27T15:22:31.756953+0000","last_fullsized":"2023-11-27T15:22:31.756953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:29:54.401852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.753240+0000","last_change":"2023-11-27T15:20:33.245595+0000","last_active":"2023-11-27T15:22:31.753240+0000","last_peered":"2023-11-27T15:22:31.753240+0000","last_clean":"2023-11-27T15:22:31.753240+0000","last_became_active":"2023-11-27T15:20:33.245294+0000","last_became_peered":"2023-11-27T15:20:33.245294+0000","last_unstale":"2023-11-27T15:22:31.753240+0000","last_undegraded":"2023-11-27T15:22:31.753240+0000","last_fullsized":"2023-11-27T15:22:31.753240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:10:33.319483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.761634+0000","last_change":"2023-11-27T15:20:33.232976+0000","last_active":"2023-11-27T15:22:31.761634+0000","last_peered":"2023-11-27T15:22:31.761634+0000","last_clean":"2023-11-27T15:22:31.761634+0000","last_became_active":"2023-11-27T15:20:33.232648+0000","last_became_peered":"2023-11-27T15:20:33.232648+0000","last_unstale":"2023-11-27T15:22:31.761634+0000","last_undegraded":"2023-11-27T15:22:31.761634+0000","last_fullsized":"2023-11-27T15:22:31.761634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:21:07.049919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.756893+0000","last_change":"2023-11-27T15:20:33.226332+0000","last_active":"2023-11-27T15:22:31.756893+0000","last_peered":"2023-11-27T15:22:31.756893+0000","last_clean":"2023-11-27T15:22:31.756893+0000","last_became_active":"2023-11-27T15:20:33.226000+0000","last_became_peered":"2023-11-27T15:20:33.226000+0000","last_unstale":"2023-11-27T15:22:31.756893+0000","last_undegraded":"2023-11-27T15:22:31.756893+0000","last_fullsized":"2023-11-27T15:22:31.756893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:02:35.531072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.750943+0000","last_change":"2023-11-27T15:20:33.217927+0000","last_active":"2023-11-27T15:22:31.750943+0000","last_peered":"2023-11-27T15:22:31.750943+0000","last_clean":"2023-11-27T15:22:31.750943+0000","last_became_active":"2023-11-27T15:20:33.217003+0000","last_became_peered":"2023-11-27T15:20:33.217003+0000","last_unstale":"2023-11-27T15:22:31.750943+0000","last_undegraded":"2023-11-27T15:22:31.750943+0000","last_fullsized":"2023-11-27T15:22:31.750943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:51:22.192698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64726,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":619,"num_read_kb":376,"num_write":170,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":204936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416423,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6280,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":93738872,"statfs":{"total":95995035648,"available":95988604928,"internally_reserved":0,"allocated":1048576,"data_stored":514092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.55800000000000005}]},{"osd":1,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60599999999999998,"15min":0.60299999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89200000000000002,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60899999999999999,"15min":0.61299999999999999},"min":{"1min":0.35499999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76300000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.60499999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.30299999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59999999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.46100000000000002,"15min":0.45800000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.67200000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.50700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":4,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.86499999999999999,"5min":1.177,"15min":1.177},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61299999999999999,"15min":0.61599999999999999},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84899999999999998,"5min":1.141,"15min":1.141},"last":0.69999999999999996}]},{"osd":5,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52700000000000002,"15min":0.53100000000000003},"min":{"1min":0.34399999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69299999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88700000000000001,"5min":1.032,"15min":1.032},"last":0.66100000000000003}]},{"osd":6,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.47499999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.875,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85999999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.877}]},{"osd":7,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.46500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97899999999999998,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70999999999999996,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95699999999999996,"5min":1.54,"15min":1.54},"last":0.85999999999999999}]},{"osd":8,"last update":"Mon Nov 27 15:22:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68700000000000006,"15min":0.68600000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89500000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.90900000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.81599999999999995}]},{"osd":9,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71599999999999997,"15min":0.71799999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92500000000000004,"5min":1.286,"15min":1.286},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.76900000000000002,"15min":0.76500000000000001},"min":{"1min":0.498,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.84899999999999998}]},{"osd":10,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.53900000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96299999999999997,"5min":1.304,"15min":1.304},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73099999999999998,"15min":0.73299999999999998},"min":{"1min":0.53000000000000003,"5min":0.501,"15min":0.501},"max":{"1min":0.95199999999999996,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.91400000000000003}]}]},{"osd":10,"up_from":71,"seq":304942678058,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7332,"kb_used_data":1692,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93737820,"statfs":{"total":95995035648,"available":95987527680,"internally_reserved":0,"allocated":1732608,"data_stored":1166196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.55800000000000005,"15min":0.53400000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":4.5060000000000002,"5min":4.5060000000000002,"15min":4.5060000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.56699999999999995}]},{"osd":1,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46100000000000002,"15min":0.47399999999999998},"min":{"1min":0.33400000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.63600000000000001,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46100000000000002,"15min":0.46700000000000003},"min":{"1min":0.33500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42299999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.58399999999999996},"min":{"1min":0.374,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.48599999999999999}]},{"osd":3,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.60699999999999998,"15min":0.63500000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70799999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63700000000000001,"15min":0.66100000000000003},"min":{"1min":0.45500000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.73399999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.67300000000000004}]},{"osd":4,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56599999999999995,"15min":0.57999999999999996},"min":{"1min":0.40999999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80200000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.56899999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88700000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61299999999999999}]},{"osd":5,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63900000000000001,"15min":0.626},"min":{"1min":0.45600000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92700000000000005,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81499999999999995,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.57899999999999996}]},{"osd":6,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.67200000000000004,"15min":0.68999999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86299999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70599999999999996,"15min":0.72599999999999998},"min":{"1min":0.54600000000000004,"5min":0.496,"15min":0.496},"max":{"1min":0.85199999999999998,"5min":1.004,"15min":1.004},"last":0.75900000000000001}]},{"osd":7,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.56799999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.73699999999999999,"15min":0.76100000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.876,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.72799999999999998,"15min":0.73399999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97399999999999998,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69299999999999995}]},{"osd":9,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.76800000000000002,"15min":0.79000000000000004},"min":{"1min":0.58499999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.94199999999999995,"5min":1.111,"15min":1.111},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76000000000000001,"15min":0.77200000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94899999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.71699999999999997}]},{"osd":11,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.746,"15min":0.75},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.77000000000000002,"15min":0.77900000000000003},"min":{"1min":0.56999999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.92600000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.79100000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841582,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6480,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738672,"statfs":{"total":95995035648,"available":95988400128,"internally_reserved":0,"allocated":1056768,"data_stored":514216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.503,"15min":0.51900000000000002},"min":{"1min":0.33500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.59899999999999998,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.35899999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.71099999999999997,"15min":0.73099999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.99199999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.65100000000000002},"min":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.56499999999999995}]},{"osd":2,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.77000000000000002,"15min":0.79200000000000004},"min":{"1min":0.52800000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.751},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.74199999999999999,"15min":0.77200000000000002},"min":{"1min":0.48299999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.66100000000000003}]},{"osd":3,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.66900000000000004,"15min":0.70299999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.73299999999999998,"15min":0.78400000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.127,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.49399999999999999}]},{"osd":4,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.71699999999999997,"15min":0.749},"min":{"1min":0.47199999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.77300000000000002,"15min":0.80600000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.68300000000000005}]},{"osd":5,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.81699999999999995,"15min":0.84499999999999997},"min":{"1min":0.53500000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.84699999999999998,"15min":0.88400000000000001},"min":{"1min":0.55800000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.4470000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.78600000000000003}]},{"osd":6,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.80400000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.84799999999999998,"15min":0.89800000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.89900000000000002,"15min":0.94299999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.5089999999999999,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.58199999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.90700000000000003,"15min":0.96399999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8160000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.91600000000000004,"15min":0.96299999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.77600000000000002}]},{"osd":10,"last update":"Mon Nov 27 15:22:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.88800000000000001,"15min":0.92400000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.5469999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.626},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.83799999999999997,"15min":0.86499999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.71299999999999997}]},{"osd":11,"last update":"Mon Nov 27 15:22:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.82799999999999996,"15min":0.83899999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.84099999999999997,"15min":0.85199999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.80700000000000005}]}]},{"osd":8,"up_from":58,"seq":249108103220,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6596,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738556,"statfs":{"total":95995035648,"available":95988281344,"internally_reserved":0,"allocated":1044480,"data_stored":513693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.44400000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.59799999999999998},"min":{"1min":0.45700000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.75,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54400000000000004}]},{"osd":1,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.59999999999999998,"15min":0.59899999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.60599999999999998,"15min":0.59699999999999998},"min":{"1min":0.42699999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.69299999999999995}]},{"osd":2,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.44600000000000001,"15min":0.439},"min":{"1min":0.376,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.435},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.46600000000000003,"15min":0.44800000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.66600000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.47499999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.60599999999999998,"15min":0.60099999999999998},"min":{"1min":0.503,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.45300000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.57499999999999996}]},{"osd":4,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63,"15min":0.628},"min":{"1min":0.46700000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.84699999999999998,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86599999999999999,"5min":1.377,"15min":1.377},"last":0.67500000000000004}]},{"osd":5,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.55600000000000005},"min":{"1min":0.39100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60299999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.73799999999999999,"15min":0.68400000000000005},"min":{"1min":0.47599999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":6.4450000000000003,"5min":6.4450000000000003,"15min":6.4450000000000003},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.72299999999999998,"15min":0.70299999999999996},"min":{"1min":0.55200000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.76700000000000002}]},{"osd":7,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.73999999999999999,"15min":0.72199999999999998},"min":{"1min":0.53200000000000003,"5min":0.442,"15min":0.442},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.434,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.99299999999999999,"5min":1.323,"15min":1.323},"last":0.70499999999999996}]},{"osd":9,"last update":"Mon Nov 27 15:22:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.76100000000000001,"15min":0.73999999999999999},"min":{"1min":0.505,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.754},{"interface":"front","average":{"1min":1.181,"5min":0.86699999999999999,"15min":0.80400000000000005},"min":{"1min":0.62,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":6.4619999999999997,"5min":6.4619999999999997,"15min":6.4619999999999997},"last":0.79500000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:22:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.74099999999999999,"15min":0.71099999999999997},"min":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.77600000000000002,"15min":0.74099999999999999},"min":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.84399999999999997}]},{"osd":11,"last update":"Mon Nov 27 15:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84699999999999998,"15min":0.85099999999999998},"min":{"1min":0.66500000000000004,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.65200000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.73799999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332151,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7316,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93737836,"statfs":{"total":95995035648,"available":95987544064,"internally_reserved":0,"allocated":1650688,"data_stored":1110401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.436,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84699999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.44},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77100000000000002,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.46999999999999997}]},{"osd":1,"last update":"Mon Nov 27 15:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.41199999999999998,"15min":0.39500000000000002},"min":{"1min":0.34499999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64000000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.42899999999999999,"15min":0.40500000000000003},"min":{"1min":0.35299999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.495}]},{"osd":2,"last update":"Mon Nov 27 15:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.42199999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.752,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.57199999999999995},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.72399999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.48899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93899999999999995,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.63900000000000001},"min":{"1min":0.48099999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.60999999999999999}]},{"osd":4,"last update":"Mon Nov 27 15:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52500000000000002,"15min":0.498},"min":{"1min":0.48299999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.73499999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.438},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51600000000000001,"15min":0.52600000000000002},"min":{"1min":0.39200000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67100000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.628}]},{"osd":5,"last update":"Mon Nov 27 15:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.47499999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.79500000000000004,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.63600000000000001,"15min":0.623},"min":{"1min":0.54800000000000004,"5min":0.434,"15min":0.434},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.58999999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69899999999999995,"15min":0.67700000000000005},"min":{"1min":0.58599999999999997,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.49399999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.68899999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:21:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.66900000000000004,"15min":0.65100000000000002},"min":{"1min":0.60399999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.53100000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.73999999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:22:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.73399999999999999,"15min":0.72699999999999998},"min":{"1min":0.66100000000000003,"5min":0.442,"15min":0.442},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77300000000000002,"15min":0.77000000000000002},"min":{"1min":0.67300000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.751}]},{"osd":10,"last update":"Mon Nov 27 15:22:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.58499999999999996,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92100000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.68899999999999995,"15min":0.66800000000000004},"min":{"1min":0.59199999999999997,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90900000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.76200000000000001}]},{"osd":11,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76700000000000002,"15min":0.76800000000000002},"min":{"1min":0.66700000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.90600000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79900000000000004,"15min":0.79800000000000004},"min":{"1min":0.64800000000000002,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.93300000000000005,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.80700000000000005}]}]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6588,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738564,"statfs":{"total":95995035648,"available":95988289536,"internally_reserved":0,"allocated":1036288,"data_stored":509392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.49399999999999999,"15min":0.47599999999999998},"min":{"1min":0.38900000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.89600000000000002,"5min":1.044,"15min":1.044},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.54000000000000004,"15min":0.496},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.42299999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.72899999999999998,"15min":0.66000000000000003},"min":{"1min":0.52000000000000002,"5min":0.441,"15min":0.441},"max":{"1min":4.9109999999999996,"5min":4.9109999999999996,"15min":4.9109999999999996},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.629,"15min":0.58599999999999997},"min":{"1min":0.45500000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90500000000000003,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.56699999999999995}]},{"osd":2,"last update":"Mon Nov 27 15:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.69699999999999995,"15min":0.65900000000000003},"min":{"1min":0.53700000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.70899999999999996,"15min":0.68300000000000005},"min":{"1min":0.55100000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.137,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.76700000000000002}]},{"osd":3,"last update":"Mon Nov 27 15:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.58799999999999997,"15min":0.55400000000000005},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.64100000000000001,"15min":0.60499999999999998},"min":{"1min":0.42699999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.57599999999999996}]},{"osd":4,"last update":"Mon Nov 27 15:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70199999999999996,"15min":0.68799999999999994},"min":{"1min":0.51900000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0089999999999999,"5min":1.085,"15min":1.085},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69399999999999995,"15min":0.67200000000000004},"min":{"1min":0.55500000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1519999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.63200000000000001}]},{"osd":5,"last update":"Mon Nov 27 15:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.73499999999999999,"15min":0.71299999999999997},"min":{"1min":0.51100000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.179,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77900000000000003,"15min":0.75800000000000001},"min":{"1min":0.66400000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2989999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.79100000000000004}]},{"osd":7,"last update":"Mon Nov 27 15:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.77300000000000002,"15min":0.753},"min":{"1min":0.59499999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.76200000000000001,"15min":0.71999999999999997},"min":{"1min":0.55800000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.64300000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85799999999999998,"15min":0.86399999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.22,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.84799999999999998,"15min":0.85799999999999998},"min":{"1min":0.60699999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.74399999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.85399999999999998,"15min":0.86799999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.90400000000000003,"15min":0.92400000000000004},"min":{"1min":0.59999999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.2569999999999999,"5min":1.712,"15min":1.712},"last":0.84599999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:22:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.83099999999999996,"15min":0.82499999999999996},"min":{"1min":0.56499999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.90000000000000002,"15min":0.88800000000000001},"min":{"1min":0.72599999999999998,"5min":0.626,"15min":0.626},"max":{"1min":1.3999999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.81200000000000006}]},{"osd":11,"last update":"Mon Nov 27 15:21:38 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":4.9290000000000003,"5min":4.9290000000000003,"15min":4.9290000000000003},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.86599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7680,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":6592,"kb_avail":93737472,"statfs":{"total":95995035648,"available":95987171328,"internally_reserved":0,"allocated":1105920,"data_stored":575050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6750208},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.59099999999999997,"15min":0.51300000000000001},"min":{"1min":0.36699999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":5.2999999999999998,"5min":5.2999999999999998,"15min":5.2999999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.57999999999999996,"15min":0.504},"min":{"1min":0.46300000000000002,"5min":0.36699999999999999,"15min":0.33800000000000002},"max":{"1min":1.0029999999999999,"5min":1.012,"15min":1.012},"last":0.59499999999999997}]},{"osd":2,"last update":"Mon Nov 27 15:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.55500000000000005,"15min":0.498},"min":{"1min":0.38,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.752,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.61399999999999999,"15min":0.51600000000000001},"min":{"1min":0.42699999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.88700000000000001,"5min":0.999,"15min":0.999},"last":0.746}]},{"osd":3,"last update":"Mon Nov 27 15:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57199999999999995,"15min":0.53400000000000003},"min":{"1min":0.39700000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.39500000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.57199999999999995}]},{"osd":4,"last update":"Mon Nov 27 15:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.46600000000000003,"15min":0.40799999999999997},"min":{"1min":0.34399999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.73699999999999999,"5min":1.3,"15min":1.3},"last":0.874},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.47999999999999998,"15min":0.42199999999999999},"min":{"1min":0.32800000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.752,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.72299999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.60199999999999998,"15min":0.55600000000000005},"min":{"1min":0.48899999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78800000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.65700000000000003,"15min":0.58299999999999996},"min":{"1min":0.60999999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93899999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.81000000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.72099999999999997,"15min":0.64900000000000002},"min":{"1min":0.52400000000000002,"5min":0.442,"15min":0.442},"max":{"1min":5.3239999999999998,"5min":5.3239999999999998,"15min":5.3239999999999998},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66100000000000003,"15min":0.626},"min":{"1min":0.47999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.0289999999999999,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.84699999999999998}]},{"osd":7,"last update":"Mon Nov 27 15:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.62,"15min":0.57599999999999996},"min":{"1min":0.505,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0169999999999999,"5min":1.089,"15min":1.089},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.65000000000000002,"15min":0.60399999999999998},"min":{"1min":0.53200000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92100000000000004,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.91800000000000004}]},{"osd":8,"last update":"Mon Nov 27 15:22:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.70199999999999996,"15min":0.65600000000000003},"min":{"1min":0.629,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.68899999999999995,"15min":0.65300000000000002},"min":{"1min":0.55600000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91200000000000003,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.83599999999999997}]},{"osd":9,"last update":"Mon Nov 27 15:22:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":0.81799999999999995,"15min":0.76100000000000001},"min":{"1min":0.67500000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":5.3810000000000002,"5min":5.3810000000000002,"15min":5.3810000000000002},"last":0.995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.78000000000000003,"15min":0.76200000000000001},"min":{"1min":0.51600000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.95199999999999996}]},{"osd":10,"last update":"Mon Nov 27 15:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69799999999999995,"15min":0.68899999999999995},"min":{"1min":0.502,"5min":0.439,"15min":0.439},"max":{"1min":0.93400000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.79200000000000004,"15min":0.77400000000000002},"min":{"1min":0.624,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0429999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.93799999999999994}]},{"osd":11,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.79700000000000004,"15min":0.79300000000000004},"min":{"1min":0.65300000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.97699999999999998,"5min":1.024,"15min":1.024},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.88500000000000001,"15min":0.88400000000000001},"min":{"1min":0.73399999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.04,"5min":1.155,"15min":1.155},"last":0.96899999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7508,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737644,"statfs":{"total":95995035648,"available":95987347456,"internally_reserved":0,"allocated":1060864,"data_stored":510780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 27 15:22:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60099999999999998,"15min":0.52000000000000002},"min":{"1min":0.35199999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.63400000000000001,"15min":0.54000000000000004},"min":{"1min":0.40200000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.59699999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:22:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64100000000000001,"15min":0.58699999999999997},"min":{"1min":0.38400000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.95199999999999996,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.66000000000000003,"15min":0.57699999999999996},"min":{"1min":0.53100000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.79300000000000004}]},{"osd":3,"last update":"Mon Nov 27 15:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63,"15min":0.63},"min":{"1min":0.44600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.04,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91400000000000003,"5min":1.204,"15min":1.204},"last":0.54000000000000004}]},{"osd":4,"last update":"Mon Nov 27 15:22:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63900000000000001,"15min":0.61599999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68000000000000005,"15min":0.61699999999999999},"min":{"1min":0.51800000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.996,"5min":1.244,"15min":1.244},"last":0.65700000000000003}]},{"osd":5,"last update":"Mon Nov 27 15:22:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.432,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71999999999999997,"15min":0.68600000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.034,"5min":1.379,"15min":1.379},"last":0.78500000000000003}]},{"osd":6,"last update":"Mon Nov 27 15:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.68200000000000005,"15min":0.63600000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64300000000000002,"15min":0.62},"min":{"1min":0.39600000000000002,"5min":0.373,"15min":0.373},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.67300000000000004}]},{"osd":7,"last update":"Mon Nov 27 15:22:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.76800000000000002,"15min":0.75900000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.62},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.78900000000000003,"15min":0.78700000000000003},"min":{"1min":0.64500000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0880000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.81999999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.755,"15min":0.754},"min":{"1min":0.59799999999999998,"5min":0.498,"15min":0.498},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76500000000000001,"15min":0.78100000000000003},"min":{"1min":0.54400000000000004,"5min":0.504,"15min":0.504},"max":{"1min":0.94099999999999995,"5min":1.327,"15min":1.327},"last":0.874}]},{"osd":9,"last update":"Mon Nov 27 15:22:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.86699999999999999,"15min":0.878},"min":{"1min":0.65300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.0920000000000001,"5min":1.218,"15min":1.218},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.85199999999999998,"15min":0.85899999999999999},"min":{"1min":0.63600000000000001,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.86499999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:22:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.81699999999999995,"15min":0.82899999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.056,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88800000000000001,"15min":0.89200000000000002},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1599999999999999,"5min":1.409,"15min":1.409},"last":0.88}]},{"osd":11,"last update":"Mon Nov 27 15:21:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.83199999999999996,"15min":0.82699999999999996},"min":{"1min":0.67300000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.82799999999999996,"15min":0.83499999999999996},"min":{"1min":0.59299999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.96499999999999997,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.83399999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7764,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":6656,"kb_avail":93737388,"statfs":{"total":95995035648,"available":95987085312,"internally_reserved":0,"allocated":1126400,"data_stored":575995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6815744},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54400000000000004,"15min":0.51000000000000001},"min":{"1min":0.42699999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.42399999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.72499999999999998,"5min":0.83799999999999997,"15min":0.85199999999999998},"last":0.623}]},{"osd":1,"last update":"Mon Nov 27 15:22:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.58899999999999997,"15min":0.51100000000000001},"min":{"1min":0.52300000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.84999999999999998,"5min":1.079,"15min":1.079},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.58899999999999997,"15min":0.51700000000000002},"min":{"1min":0.48699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79100000000000004,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.76000000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.60699999999999998,"15min":0.57099999999999995},"min":{"1min":0.48999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.59499999999999997,"15min":0.56899999999999995},"min":{"1min":0.44500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.84099999999999997}]},{"osd":4,"last update":"Mon Nov 27 15:22:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.52100000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66800000000000004,"15min":0.67400000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84099999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.71299999999999997}]},{"osd":5,"last update":"Mon Nov 27 15:22:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.47999999999999998,"15min":0.438},"min":{"1min":0.35899999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80400000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.50800000000000001,"15min":0.46400000000000002},"min":{"1min":0.40300000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77600000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.59299999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:22:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.64100000000000001,"15min":0.61299999999999999},"min":{"1min":0.48599999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.82899999999999996,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.64100000000000001,"15min":0.60999999999999999},"min":{"1min":0.439,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93700000000000006,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.80400000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:22:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71099999999999997,"15min":0.71899999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80700000000000005,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.55500000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.88600000000000001}]},{"osd":8,"last update":"Mon Nov 27 15:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.376,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85599999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.61599999999999999,"15min":0.60499999999999998},"min":{"1min":0.47199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.88100000000000001,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.82799999999999996}]},{"osd":9,"last update":"Mon Nov 27 15:22:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.70399999999999996,"15min":0.67000000000000004},"min":{"1min":0.60399999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.871},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.73199999999999998,"15min":0.70099999999999996},"min":{"1min":0.56200000000000006,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.91800000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:22:33 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.75600000000000001,"15min":0.75},"min":{"1min":0.57499999999999996,"5min":0.44,"15min":0.44},"max":{"1min":0.89500000000000002,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.746,"15min":0.69799999999999995},"min":{"1min":0.63,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.038,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.94299999999999995}]},{"osd":11,"last update":"Mon Nov 27 15:22:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.84599999999999997,"15min":0.85199999999999998},"min":{"1min":0.504,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97199999999999998,"5min":1.034,"15min":1.034},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.80700000000000005,"15min":0.80900000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.93200000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.90800000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6920,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":5888,"kb_avail":93738232,"statfs":{"total":95995035648,"available":95987949568,"internally_reserved":0,"allocated":1048576,"data_stored":509586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6029312},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.443,"15min":0.40600000000000003},"min":{"1min":0.34899999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.375},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47199999999999998,"15min":0.42199999999999999},"min":{"1min":0.36199999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79400000000000004,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.78200000000000003}]},{"osd":1,"last update":"Mon Nov 27 15:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63800000000000001,"15min":0.57399999999999995},"min":{"1min":0.54500000000000004,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.878,"5min":1.056,"15min":1.056},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.67700000000000005,"15min":0.58499999999999996},"min":{"1min":0.502,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.907,"5min":5.6529999999999996,"15min":5.6529999999999996},"last":0.436}]},{"osd":2,"last update":"Mon Nov 27 15:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.66500000000000004,"15min":0.60199999999999998},"min":{"1min":0.52300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81799999999999995,"5min":1.208,"15min":1.208},"last":0.745},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65200000000000002,"15min":0.59499999999999997},"min":{"1min":0.46000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88700000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.72199999999999998}]},{"osd":4,"last update":"Mon Nov 27 15:22:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.69099999999999995,"15min":0.64300000000000002},"min":{"1min":0.52100000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":2.8730000000000002,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72299999999999998,"15min":0.67800000000000005},"min":{"1min":0.55000000000000004,"5min":0.502,"15min":0.502},"max":{"1min":0.83199999999999996,"5min":1.343,"15min":1.343},"last":0.59699999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70399999999999996,"15min":0.66200000000000003},"min":{"1min":0.61699999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.90800000000000003,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.69299999999999995,"15min":0.66100000000000003},"min":{"1min":0.58299999999999996,"5min":0.437,"15min":0.437},"max":{"1min":0.89700000000000002,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.68200000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.67900000000000005,"15min":0.64600000000000002},"min":{"1min":0.54600000000000004,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.63600000000000001,"15min":0.60799999999999998},"min":{"1min":0.443,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.84499999999999997}]},{"osd":7,"last update":"Mon Nov 27 15:22:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.93600000000000005,"15min":0.96999999999999997},"min":{"1min":0.63100000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.7959999999999998,"5min":6.9619999999999997,"15min":6.9619999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.53400000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0940000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.56399999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.873},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":0.81499999999999995}]},{"osd":9,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.83899999999999997,"15min":0.85399999999999998},"min":{"1min":0.64800000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83399999999999996,"15min":0.84999999999999998},"min":{"1min":0.56799999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1910000000000001,"5min":1.496,"15min":1.496},"last":0.82699999999999996}]},{"osd":10,"last update":"Mon Nov 27 15:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.80600000000000005,"15min":0.79200000000000004},"min":{"1min":0.59599999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.839,"5min":2.839,"15min":2.839},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.83299999999999996,"15min":0.83599999999999997},"min":{"1min":0.64000000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0209999999999999,"5min":1.129,"15min":1.129},"last":0.83599999999999997}]},{"osd":11,"last update":"Mon Nov 27 15:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83099999999999996,"15min":0.81999999999999995},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.84899999999999998,"15min":0.84599999999999997},"min":{"1min":0.72899999999999998,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.79100000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6792,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738360,"statfs":{"total":95995035648,"available":95988080640,"internally_reserved":0,"allocated":1048576,"data_stored":514525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.54400000000000004,"15min":0.53700000000000003},"min":{"1min":0.38300000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53500000000000003,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77900000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.54900000000000004}]},{"osd":1,"last update":"Mon Nov 27 15:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.40200000000000002,"15min":0.35799999999999998},"min":{"1min":0.26000000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.54700000000000004,"5min":1.337,"15min":1.337},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.371},"min":{"1min":0.28199999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.57399999999999995,"5min":1.476,"15min":1.476},"last":0.36199999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57399999999999995,"15min":0.55600000000000005},"min":{"1min":0.38100000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83599999999999997,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60399999999999998,"15min":0.55400000000000005},"min":{"1min":0.39300000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.68999999999999995,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.76100000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59399999999999997,"15min":0.55700000000000005},"min":{"1min":0.42199999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0109999999999999,"5min":1.423,"15min":1.423},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.59899999999999998},"min":{"1min":0.42099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.99199999999999999,"5min":1.722,"15min":1.722},"last":0.48099999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.63400000000000001,"15min":0.61099999999999999},"min":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.87,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64100000000000001,"15min":0.60999999999999999},"min":{"1min":0.433,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.94299999999999995,"5min":1.28,"15min":1.28},"last":0.68400000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:22:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88,"5min":1.393,"15min":1.393},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67400000000000004,"15min":0.65900000000000003},"min":{"1min":0.49199999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.879,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.67200000000000004}]},{"osd":7,"last update":"Mon Nov 27 15:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58999999999999997,"15min":0.56799999999999995},"min":{"1min":0.33900000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79900000000000004,"5min":1.054,"15min":1.054},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61599999999999999,"15min":0.59699999999999998},"min":{"1min":0.36099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.98099999999999998,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.621}]},{"osd":8,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71399999999999997,"15min":0.70499999999999996},"min":{"1min":0.50600000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.99099999999999999,"5min":1.522,"15min":1.522},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92700000000000005,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.71299999999999997}]},{"osd":9,"last update":"Mon Nov 27 15:22:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.79500000000000004,"15min":0.76700000000000002},"min":{"1min":0.55900000000000005,"5min":0.44,"15min":0.44},"max":{"1min":2.8969999999999998,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.52100000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.54000000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73399999999999999,"15min":0.72399999999999998},"min":{"1min":0.53800000000000003,"5min":0.503,"15min":0.503},"max":{"1min":0.91000000000000003,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91800000000000004,"5min":1.03,"15min":1.03},"last":0.65500000000000003}]},{"osd":11,"last update":"Mon Nov 27 15:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.81100000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.96899999999999997,"5min":1.234,"15min":1.234},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72399999999999998,"15min":0.72499999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.97499999999999998,"5min":1.2,"15min":1.2},"last":0.72499999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7448,"kb_used_data":1616,"kb_used_omap":0,"kb_used_meta":5824,"kb_avail":93737704,"statfs":{"total":95995035648,"available":95987408896,"internally_reserved":0,"allocated":1654784,"data_stored":1105175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5963776},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.36199999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.46000000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57799999999999996,"15min":0.54300000000000004},"min":{"1min":0.439,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.67100000000000004}]},{"osd":2,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.35199999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.621,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.36799999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48499999999999999}]},{"osd":3,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54900000000000004,"15min":0.55900000000000005},"min":{"1min":0.39600000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79200000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.443},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57999999999999996,"15min":0.57899999999999996},"min":{"1min":0.44600000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.81699999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.502}]},{"osd":4,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.622,"15min":0.59799999999999998},"min":{"1min":0.48699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.60099999999999998,"15min":0.58499999999999996},"min":{"1min":0.45500000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.58899999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.621,"15min":0.63300000000000001},"min":{"1min":0.48099999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.89900000000000002,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.54500000000000004,"5min":0.443,"15min":0.443},"max":{"1min":0.82999999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.57799999999999996}]},{"osd":7,"last update":"Mon Nov 27 15:22:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63400000000000001,"15min":0.64500000000000002},"min":{"1min":0.503,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.76000000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64300000000000002,"15min":0.64000000000000001},"min":{"1min":0.57099999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.76300000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.70399999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:22:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.48799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.47699999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.54400000000000004}]},{"osd":9,"last update":"Mon Nov 27 15:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71699999999999997,"15min":0.74299999999999999},"min":{"1min":0.51900000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86499999999999999,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.58799999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.82999999999999996,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.71799999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:22:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.68799999999999994,"15min":0.68000000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70799999999999996,"15min":0.69099999999999995},"min":{"1min":0.54700000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.73599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-27T15:22:46.304 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-27T15:22:46.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.531+0000 7fb7d1bff700 1 -- 172.21.15.133:0/1069593032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7cc0734f0 msgr2=0x7fb7cc073960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:46.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.531+0000 7fb7d1bff700 1 --2- 172.21.15.133:0/1069593032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7cc0734f0 0x7fb7cc073960 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009990 tx=0x7fb7c0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:46.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.531+0000 7fb7d1bff700 1 -- 172.21.15.133:0/1069593032 shutdown_connections 2023-11-27T15:22:46.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.531+0000 7fb7d1bff700 1 --2- 172.21.15.133:0/1069593032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7cc108140 0x7fb7cc10a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:46.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.531+0000 7fb7d1bff700 1 --2- 172.21.15.133:0/1069593032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7cc074ad0 0x7fb7cc072f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:46.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.531+0000 7fb7d1bff700 1 --2- 172.21.15.133:0/1069593032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7cc0734f0 0x7fb7cc073960 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:46.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.531+0000 7fb7d1bff700 1 -- 172.21.15.133:0/1069593032 >> 172.21.15.133:0/1069593032 conn(0x7fb7cc0fb890 msgr2=0x7fb7cc0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:46.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.532+0000 7fb7d1bff700 1 -- 172.21.15.133:0/1069593032 shutdown_connections 2023-11-27T15:22:46.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.532+0000 7fb7d1bff700 1 -- 172.21.15.133:0/1069593032 wait complete. 2023-11-27T15:22:46.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.533+0000 7fb7d1bff700 1 Processor -- start 2023-11-27T15:22:46.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.533+0000 7fb7d1bff700 1 -- start start 2023-11-27T15:22:46.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.533+0000 7fb7d1bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7cc0734f0 0x7fb7cc1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:46.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.533+0000 7fb7d1bff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7cc074ad0 0x7fb7cc198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:46.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.533+0000 7fb7d1bff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7cc108140 0x7fb7cc19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:46.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.533+0000 7fb7d1bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc19fd40 con 0x7fb7cc0734f0 2023-11-27T15:22:46.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.533+0000 7fb7d1bff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc19fe80 con 0x7fb7cc108140 2023-11-27T15:22:46.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.533+0000 7fb7d1bff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7cc1a0010 con 0x7fb7cc074ad0 2023-11-27T15:22:46.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.533+0000 7fb7cb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7cc0734f0 0x7fb7cc1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:46.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.533+0000 7fb7caffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7cc074ad0 0x7fb7cc198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:46.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.533+0000 7fb7caffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7cc074ad0 0x7fb7cc198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:46188/0 (socket says 172.21.15.133:46188) 2023-11-27T15:22:46.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.533+0000 7fb7caffd700 1 -- 172.21.15.133:0/2636772030 learned_addr learned my addr 172.21.15.133:0/2636772030 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:46.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.533+0000 7fb7cbfff700 1 --2- 172.21.15.133:0/2636772030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7cc108140 0x7fb7cc19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:46.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.534+0000 7fb7caffd700 1 -- 172.21.15.133:0/2636772030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7cc108140 msgr2=0x7fb7cc19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:46.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.534+0000 7fb7caffd700 1 --2- 172.21.15.133:0/2636772030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7cc108140 0x7fb7cc19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:46.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.534+0000 7fb7caffd700 1 -- 172.21.15.133:0/2636772030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7cc0734f0 msgr2=0x7fb7cc1981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:46.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.534+0000 7fb7caffd700 1 --2- 172.21.15.133:0/2636772030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7cc0734f0 0x7fb7cc1981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:46.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.534+0000 7fb7caffd700 1 -- 172.21.15.133:0/2636772030 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c4009c40 con 0x7fb7cc074ad0 2023-11-27T15:22:46.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.534+0000 7fb7cbfff700 1 --2- 172.21.15.133:0/2636772030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7cc108140 0x7fb7cc19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:22:46.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.534+0000 7fb7caffd700 1 --2- 172.21.15.133:0/2636772030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7cc074ad0 0x7fb7cc198700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000b680 tx=0x7fb7c0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:46.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.534+0000 7fb7c8ff9700 1 -- 172.21.15.133:0/2636772030 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c001d070 con 0x7fb7cc074ad0 2023-11-27T15:22:46.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.534+0000 7fb7c8ff9700 1 -- 172.21.15.133:0/2636772030 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb7c0022470 con 0x7fb7cc074ad0 2023-11-27T15:22:46.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.534+0000 7fb7d1bff700 1 -- 172.21.15.133:0/2636772030 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c00095f0 con 0x7fb7cc074ad0 2023-11-27T15:22:46.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.534+0000 7fb7c8ff9700 1 -- 172.21.15.133:0/2636772030 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0017650 con 0x7fb7cc074ad0 2023-11-27T15:22:46.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.534+0000 7fb7d1bff700 1 -- 172.21.15.133:0/2636772030 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7cc10de40 con 0x7fb7cc074ad0 2023-11-27T15:22:46.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.535+0000 7fb7d1bff700 1 -- 172.21.15.133:0/2636772030 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7cc10dfd0 con 0x7fb7cc074ad0 2023-11-27T15:22:46.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.538+0000 7fb7c8ff9700 1 -- 172.21.15.133:0/2636772030 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb7c00225e0 con 0x7fb7cc074ad0 2023-11-27T15:22:46.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.539+0000 7fb7c8ff9700 1 --2- 172.21.15.133:0/2636772030 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb7b40a11d0 0x7fb7b40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:46.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.539+0000 7fb7c8ff9700 1 -- 172.21.15.133:0/2636772030 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb7c00ce7b0 con 0x7fb7cc074ad0 2023-11-27T15:22:46.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.539+0000 7fb7cb7fe700 1 --2- 172.21.15.133:0/2636772030 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb7b40a11d0 0x7fb7b40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:46.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.540+0000 7fb7cb7fe700 1 --2- 172.21.15.133:0/2636772030 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb7b40a11d0 0x7fb7b40a3680 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009c60 tx=0x7fb7bc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:46.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.541+0000 7fb7c8ff9700 1 -- 172.21.15.133:0/2636772030 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb7c0097c40 con 0x7fb7cc074ad0 2023-11-27T15:22:46.740 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.736+0000 7fb7d1bff700 1 -- 172.21.15.133:0/2636772030 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb7cc060f30 con 0x7fb7b40a11d0 2023-11-27T15:22:46.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.749+0000 7fb7c8ff9700 1 -- 172.21.15.133:0/2636772030 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427427 (secure 0 0 0) 0x7fb7cc060f30 con 0x7fb7b40a11d0 2023-11-27T15:22:46.758 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:46.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.754+0000 7fb7d1bff700 1 -- 172.21.15.133:0/2636772030 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb7b40a11d0 msgr2=0x7fb7b40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:46.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.754+0000 7fb7d1bff700 1 --2- 172.21.15.133:0/2636772030 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb7b40a11d0 0x7fb7b40a3680 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009c60 tx=0x7fb7bc009260 comp rx=0 tx=0).stop 2023-11-27T15:22:46.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.754+0000 7fb7d1bff700 1 -- 172.21.15.133:0/2636772030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7cc074ad0 msgr2=0x7fb7cc198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:46.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.754+0000 7fb7d1bff700 1 --2- 172.21.15.133:0/2636772030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7cc074ad0 0x7fb7cc198700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb7c000b680 tx=0x7fb7c0005740 comp rx=0 tx=0).stop 2023-11-27T15:22:46.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.754+0000 7fb7d1bff700 1 -- 172.21.15.133:0/2636772030 shutdown_connections 2023-11-27T15:22:46.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.754+0000 7fb7d1bff700 1 --2- 172.21.15.133:0/2636772030 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb7b40a11d0 0x7fb7b40a3680 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:46.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.754+0000 7fb7d1bff700 1 --2- 172.21.15.133:0/2636772030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7cc108140 0x7fb7cc19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:46.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.754+0000 7fb7d1bff700 1 --2- 172.21.15.133:0/2636772030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb7cc0734f0 0x7fb7cc1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:46.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.754+0000 7fb7d1bff700 1 --2- 172.21.15.133:0/2636772030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7cc074ad0 0x7fb7cc198700 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:46.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.754+0000 7fb7d1bff700 1 -- 172.21.15.133:0/2636772030 >> 172.21.15.133:0/2636772030 conn(0x7fb7cc0fb890 msgr2=0x7fb7cc0fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:46.763 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.754+0000 7fb7d1bff700 1 -- 172.21.15.133:0/2636772030 shutdown_connections 2023-11-27T15:22:46.763 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:46.754+0000 7fb7d1bff700 1 -- 172.21.15.133:0/2636772030 wait complete. 2023-11-27T15:22:46.763 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2023-11-27T15:22:46.777 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2023-11-27T15:22:45.744559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655340,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":745,"num_read_kb":485,"num_write":353,"num_write_kb":3061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":84704,"kb_used_data":14272,"kb_used_omap":0,"kb_used_meta":70336,"kb_avail":1124857120,"statfs":{"total":1151940427776,"available":1151853690880,"internally_reserved":0,"allocated":14614528,"data_stored":8119101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72024064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1014,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":34,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003000"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.767519+0000","last_change":"2023-11-27T15:20:33.236352+0000","last_active":"2023-11-27T15:22:31.767519+0000","last_peered":"2023-11-27T15:22:31.767519+0000","last_clean":"2023-11-27T15:22:31.767519+0000","last_became_active":"2023-11-27T15:20:33.235974+0000","last_became_peered":"2023-11-27T15:20:33.235974+0000","last_unstale":"2023-11-27T15:22:31.767519+0000","last_undegraded":"2023-11-27T15:22:31.767519+0000","last_fullsized":"2023-11-27T15:22:31.767519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:32:53.999483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.764610+0000","last_change":"2023-11-27T15:20:33.234882+0000","last_active":"2023-11-27T15:22:31.764610+0000","last_peered":"2023-11-27T15:22:31.764610+0000","last_clean":"2023-11-27T15:22:31.764610+0000","last_became_active":"2023-11-27T15:20:33.231062+0000","last_became_peered":"2023-11-27T15:20:33.231062+0000","last_unstale":"2023-11-27T15:22:31.764610+0000","last_undegraded":"2023-11-27T15:22:31.764610+0000","last_fullsized":"2023-11-27T15:22:31.764610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:54:38.782631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.765932+0000","last_change":"2023-11-27T15:20:33.238280+0000","last_active":"2023-11-27T15:22:31.765932+0000","last_peered":"2023-11-27T15:22:31.765932+0000","last_clean":"2023-11-27T15:22:31.765932+0000","last_became_active":"2023-11-27T15:20:33.237602+0000","last_became_peered":"2023-11-27T15:20:33.237602+0000","last_unstale":"2023-11-27T15:22:31.765932+0000","last_undegraded":"2023-11-27T15:22:31.765932+0000","last_fullsized":"2023-11-27T15:22:31.765932+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:12:10.576246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.744091+0000","last_change":"2023-11-27T15:20:33.217909+0000","last_active":"2023-11-27T15:22:31.744091+0000","last_peered":"2023-11-27T15:22:31.744091+0000","last_clean":"2023-11-27T15:22:31.744091+0000","last_became_active":"2023-11-27T15:20:33.217511+0000","last_became_peered":"2023-11-27T15:20:33.217511+0000","last_unstale":"2023-11-27T15:22:31.744091+0000","last_undegraded":"2023-11-27T15:22:31.744091+0000","last_fullsized":"2023-11-27T15:22:31.744091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:11:52.036690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.750946+0000","last_change":"2023-11-27T15:20:33.246630+0000","last_active":"2023-11-27T15:22:31.750946+0000","last_peered":"2023-11-27T15:22:31.750946+0000","last_clean":"2023-11-27T15:22:31.750946+0000","last_became_active":"2023-11-27T15:20:33.246249+0000","last_became_peered":"2023-11-27T15:20:33.246249+0000","last_unstale":"2023-11-27T15:22:31.750946+0000","last_undegraded":"2023-11-27T15:22:31.750946+0000","last_fullsized":"2023-11-27T15:22:31.750946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:07:50.677388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.749736+0000","last_change":"2023-11-27T15:20:33.236280+0000","last_active":"2023-11-27T15:22:31.749736+0000","last_peered":"2023-11-27T15:22:31.749736+0000","last_clean":"2023-11-27T15:22:31.749736+0000","last_became_active":"2023-11-27T15:20:33.235983+0000","last_became_peered":"2023-11-27T15:20:33.235983+0000","last_unstale":"2023-11-27T15:22:31.749736+0000","last_undegraded":"2023-11-27T15:22:31.749736+0000","last_fullsized":"2023-11-27T15:22:31.749736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:38:38.466940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.749774+0000","last_change":"2023-11-27T15:20:33.223085+0000","last_active":"2023-11-27T15:22:31.749774+0000","last_peered":"2023-11-27T15:22:31.749774+0000","last_clean":"2023-11-27T15:22:31.749774+0000","last_became_active":"2023-11-27T15:20:33.222809+0000","last_became_peered":"2023-11-27T15:20:33.222809+0000","last_unstale":"2023-11-27T15:22:31.749774+0000","last_undegraded":"2023-11-27T15:22:31.749774+0000","last_fullsized":"2023-11-27T15:22:31.749774+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:23:19.671424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.760153+0000","last_change":"2023-11-27T15:20:33.226217+0000","last_active":"2023-11-27T15:22:31.760153+0000","last_peered":"2023-11-27T15:22:31.760153+0000","last_clean":"2023-11-27T15:22:31.760153+0000","last_became_active":"2023-11-27T15:20:33.225856+0000","last_became_peered":"2023-11-27T15:20:33.225856+0000","last_unstale":"2023-11-27T15:22:31.760153+0000","last_undegraded":"2023-11-27T15:22:31.760153+0000","last_fullsized":"2023-11-27T15:22:31.760153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:59:41.033434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.752180+0000","last_change":"2023-11-27T15:20:33.227554+0000","last_active":"2023-11-27T15:22:31.752180+0000","last_peered":"2023-11-27T15:22:31.752180+0000","last_clean":"2023-11-27T15:22:31.752180+0000","last_became_active":"2023-11-27T15:20:33.227222+0000","last_became_peered":"2023-11-27T15:20:33.227222+0000","last_unstale":"2023-11-27T15:22:31.752180+0000","last_undegraded":"2023-11-27T15:22:31.752180+0000","last_fullsized":"2023-11-27T15:22:31.752180+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:45:28.965465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.751136+0000","last_change":"2023-11-27T15:20:33.230528+0000","last_active":"2023-11-27T15:22:31.751136+0000","last_peered":"2023-11-27T15:22:31.751136+0000","last_clean":"2023-11-27T15:22:31.751136+0000","last_became_active":"2023-11-27T15:20:33.230148+0000","last_became_peered":"2023-11-27T15:20:33.230148+0000","last_unstale":"2023-11-27T15:22:31.751136+0000","last_undegraded":"2023-11-27T15:22:31.751136+0000","last_fullsized":"2023-11-27T15:22:31.751136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:02:27.225754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.750528+0000","last_change":"2023-11-27T15:20:33.228098+0000","last_active":"2023-11-27T15:22:31.750528+0000","last_peered":"2023-11-27T15:22:31.750528+0000","last_clean":"2023-11-27T15:22:31.750528+0000","last_became_active":"2023-11-27T15:20:33.227760+0000","last_became_peered":"2023-11-27T15:20:33.227760+0000","last_unstale":"2023-11-27T15:22:31.750528+0000","last_undegraded":"2023-11-27T15:22:31.750528+0000","last_fullsized":"2023-11-27T15:22:31.750528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:08:49.344313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.756176+0000","last_change":"2023-11-27T15:21:02.934583+0000","last_active":"2023-11-27T15:22:31.756176+0000","last_peered":"2023-11-27T15:22:31.756176+0000","last_clean":"2023-11-27T15:22:31.756176+0000","last_became_active":"2023-11-27T15:21:02.933631+0000","last_became_peered":"2023-11-27T15:21:02.933631+0000","last_unstale":"2023-11-27T15:22:31.756176+0000","last_undegraded":"2023-11-27T15:22:31.756176+0000","last_fullsized":"2023-11-27T15:22:31.756176+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:22:12.200623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.754054+0000","last_change":"2023-11-27T15:20:33.231255+0000","last_active":"2023-11-27T15:22:31.754054+0000","last_peered":"2023-11-27T15:22:31.754054+0000","last_clean":"2023-11-27T15:22:31.754054+0000","last_became_active":"2023-11-27T15:20:33.230867+0000","last_became_peered":"2023-11-27T15:20:33.230867+0000","last_unstale":"2023-11-27T15:22:31.754054+0000","last_undegraded":"2023-11-27T15:22:31.754054+0000","last_fullsized":"2023-11-27T15:22:31.754054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:33:48.764065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.767126+0000","last_change":"2023-11-27T15:20:33.230306+0000","last_active":"2023-11-27T15:22:31.767126+0000","last_peered":"2023-11-27T15:22:31.767126+0000","last_clean":"2023-11-27T15:22:31.767126+0000","last_became_active":"2023-11-27T15:20:33.229892+0000","last_became_peered":"2023-11-27T15:20:33.229892+0000","last_unstale":"2023-11-27T15:22:31.767126+0000","last_undegraded":"2023-11-27T15:22:31.767126+0000","last_fullsized":"2023-11-27T15:22:31.767126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:44:51.431718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.761884+0000","last_change":"2023-11-27T15:20:33.220592+0000","last_active":"2023-11-27T15:22:31.761884+0000","last_peered":"2023-11-27T15:22:31.761884+0000","last_clean":"2023-11-27T15:22:31.761884+0000","last_became_active":"2023-11-27T15:20:33.220258+0000","last_became_peered":"2023-11-27T15:20:33.220258+0000","last_unstale":"2023-11-27T15:22:31.761884+0000","last_undegraded":"2023-11-27T15:22:31.761884+0000","last_fullsized":"2023-11-27T15:22:31.761884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:16:39.685631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.767650+0000","last_change":"2023-11-27T15:20:33.245041+0000","last_active":"2023-11-27T15:22:31.767650+0000","last_peered":"2023-11-27T15:22:31.767650+0000","last_clean":"2023-11-27T15:22:31.767650+0000","last_became_active":"2023-11-27T15:20:33.244759+0000","last_became_peered":"2023-11-27T15:20:33.244759+0000","last_unstale":"2023-11-27T15:22:31.767650+0000","last_undegraded":"2023-11-27T15:22:31.767650+0000","last_fullsized":"2023-11-27T15:22:31.767650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:19:46.046097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.767690+0000","last_change":"2023-11-27T15:20:33.243288+0000","last_active":"2023-11-27T15:22:31.767690+0000","last_peered":"2023-11-27T15:22:31.767690+0000","last_clean":"2023-11-27T15:22:31.767690+0000","last_became_active":"2023-11-27T15:20:33.242989+0000","last_became_peered":"2023-11-27T15:20:33.242989+0000","last_unstale":"2023-11-27T15:22:31.767690+0000","last_undegraded":"2023-11-27T15:22:31.767690+0000","last_fullsized":"2023-11-27T15:22:31.767690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:25:35.538012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.760176+0000","last_change":"2023-11-27T15:20:33.229613+0000","last_active":"2023-11-27T15:22:31.760176+0000","last_peered":"2023-11-27T15:22:31.760176+0000","last_clean":"2023-11-27T15:22:31.760176+0000","last_became_active":"2023-11-27T15:20:33.227747+0000","last_became_peered":"2023-11-27T15:20:33.227747+0000","last_unstale":"2023-11-27T15:22:31.760176+0000","last_undegraded":"2023-11-27T15:22:31.760176+0000","last_fullsized":"2023-11-27T15:22:31.760176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:57:48.962905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.760827+0000","last_change":"2023-11-27T15:20:33.227004+0000","last_active":"2023-11-27T15:22:31.760827+0000","last_peered":"2023-11-27T15:22:31.760827+0000","last_clean":"2023-11-27T15:22:31.760827+0000","last_became_active":"2023-11-27T15:20:33.226703+0000","last_became_peered":"2023-11-27T15:20:33.226703+0000","last_unstale":"2023-11-27T15:22:31.760827+0000","last_undegraded":"2023-11-27T15:22:31.760827+0000","last_fullsized":"2023-11-27T15:22:31.760827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:16:57.764198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818292+0000","last_change":"2023-11-27T15:20:37.682021+0000","last_active":"2023-11-27T15:22:02.818292+0000","last_peered":"2023-11-27T15:22:02.818292+0000","last_clean":"2023-11-27T15:22:02.818292+0000","last_became_active":"2023-11-27T15:20:37.681600+0000","last_became_peered":"2023-11-27T15:20:37.681600+0000","last_unstale":"2023-11-27T15:22:02.818292+0000","last_undegraded":"2023-11-27T15:22:02.818292+0000","last_fullsized":"2023-11-27T15:22:02.818292+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:16:02.041857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811637+0000","last_change":"2023-11-27T15:20:32.657542+0000","last_active":"2023-11-27T15:22:02.811637+0000","last_peered":"2023-11-27T15:22:02.811637+0000","last_clean":"2023-11-27T15:22:02.811637+0000","last_became_active":"2023-11-27T15:20:32.657158+0000","last_became_peered":"2023-11-27T15:20:32.657158+0000","last_unstale":"2023-11-27T15:22:02.811637+0000","last_undegraded":"2023-11-27T15:22:02.811637+0000","last_fullsized":"2023-11-27T15:22:02.811637+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:27:55.061003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.747709+0000","last_change":"2023-11-27T15:21:02.933684+0000","last_active":"2023-11-27T15:22:31.747709+0000","last_peered":"2023-11-27T15:22:31.747709+0000","last_clean":"2023-11-27T15:22:31.747709+0000","last_became_active":"2023-11-27T15:21:02.933215+0000","last_became_peered":"2023-11-27T15:21:02.933215+0000","last_unstale":"2023-11-27T15:22:31.747709+0000","last_undegraded":"2023-11-27T15:22:31.747709+0000","last_fullsized":"2023-11-27T15:22:31.747709+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:38:04.533244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811191+0000","last_change":"2023-11-27T15:20:37.666705+0000","last_active":"2023-11-27T15:22:02.811191+0000","last_peered":"2023-11-27T15:22:02.811191+0000","last_clean":"2023-11-27T15:22:02.811191+0000","last_became_active":"2023-11-27T15:20:37.666236+0000","last_became_peered":"2023-11-27T15:20:37.666236+0000","last_unstale":"2023-11-27T15:22:02.811191+0000","last_undegraded":"2023-11-27T15:22:02.811191+0000","last_fullsized":"2023-11-27T15:22:02.811191+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:39:04.186691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818128+0000","last_change":"2023-11-27T15:20:32.663915+0000","last_active":"2023-11-27T15:22:02.818128+0000","last_peered":"2023-11-27T15:22:02.818128+0000","last_clean":"2023-11-27T15:22:02.818128+0000","last_became_active":"2023-11-27T15:20:32.663069+0000","last_became_peered":"2023-11-27T15:20:32.663069+0000","last_unstale":"2023-11-27T15:22:02.818128+0000","last_undegraded":"2023-11-27T15:22:02.818128+0000","last_fullsized":"2023-11-27T15:22:02.818128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:59:18.356598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.765956+0000","last_change":"2023-11-27T15:20:33.238150+0000","last_active":"2023-11-27T15:22:31.765956+0000","last_peered":"2023-11-27T15:22:31.765956+0000","last_clean":"2023-11-27T15:22:31.765956+0000","last_became_active":"2023-11-27T15:20:33.236972+0000","last_became_peered":"2023-11-27T15:20:33.236972+0000","last_unstale":"2023-11-27T15:22:31.765956+0000","last_undegraded":"2023-11-27T15:22:31.765956+0000","last_fullsized":"2023-11-27T15:22:31.765956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:18:18.352219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811484+0000","last_change":"2023-11-27T15:20:37.668031+0000","last_active":"2023-11-27T15:22:02.811484+0000","last_peered":"2023-11-27T15:22:02.811484+0000","last_clean":"2023-11-27T15:22:02.811484+0000","last_became_active":"2023-11-27T15:20:37.667640+0000","last_became_peered":"2023-11-27T15:20:37.667640+0000","last_unstale":"2023-11-27T15:22:02.811484+0000","last_undegraded":"2023-11-27T15:22:02.811484+0000","last_fullsized":"2023-11-27T15:22:02.811484+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:01:43.896196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.819361+0000","last_change":"2023-11-27T15:20:32.655738+0000","last_active":"2023-11-27T15:22:02.819361+0000","last_peered":"2023-11-27T15:22:02.819361+0000","last_clean":"2023-11-27T15:22:02.819361+0000","last_became_active":"2023-11-27T15:20:32.655439+0000","last_became_peered":"2023-11-27T15:20:32.655439+0000","last_unstale":"2023-11-27T15:22:02.819361+0000","last_undegraded":"2023-11-27T15:22:02.819361+0000","last_fullsized":"2023-11-27T15:22:02.819361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:26:55.161397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.752166+0000","last_change":"2023-11-27T15:20:33.229267+0000","last_active":"2023-11-27T15:22:31.752166+0000","last_peered":"2023-11-27T15:22:31.752166+0000","last_clean":"2023-11-27T15:22:31.752166+0000","last_became_active":"2023-11-27T15:20:33.228746+0000","last_became_peered":"2023-11-27T15:20:33.228746+0000","last_unstale":"2023-11-27T15:22:31.752166+0000","last_undegraded":"2023-11-27T15:22:31.752166+0000","last_fullsized":"2023-11-27T15:22:31.752166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:55:04.819687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811343+0000","last_change":"2023-11-27T15:20:37.671893+0000","last_active":"2023-11-27T15:22:02.811343+0000","last_peered":"2023-11-27T15:22:02.811343+0000","last_clean":"2023-11-27T15:22:02.811343+0000","last_became_active":"2023-11-27T15:20:37.671033+0000","last_became_peered":"2023-11-27T15:20:37.671033+0000","last_unstale":"2023-11-27T15:22:02.811343+0000","last_undegraded":"2023-11-27T15:22:02.811343+0000","last_fullsized":"2023-11-27T15:22:02.811343+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:23:32.141225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818527+0000","last_change":"2023-11-27T15:20:32.650342+0000","last_active":"2023-11-27T15:22:02.818527+0000","last_peered":"2023-11-27T15:22:02.818527+0000","last_clean":"2023-11-27T15:22:02.818527+0000","last_became_active":"2023-11-27T15:20:32.650034+0000","last_became_peered":"2023-11-27T15:20:32.650034+0000","last_unstale":"2023-11-27T15:22:02.818527+0000","last_undegraded":"2023-11-27T15:22:02.818527+0000","last_fullsized":"2023-11-27T15:22:02.818527+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:14:01.957530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.749849+0000","last_change":"2023-11-27T15:20:33.226005+0000","last_active":"2023-11-27T15:22:31.749849+0000","last_peered":"2023-11-27T15:22:31.749849+0000","last_clean":"2023-11-27T15:22:31.749849+0000","last_became_active":"2023-11-27T15:20:33.225685+0000","last_became_peered":"2023-11-27T15:20:33.225685+0000","last_unstale":"2023-11-27T15:22:31.749849+0000","last_undegraded":"2023-11-27T15:22:31.749849+0000","last_fullsized":"2023-11-27T15:22:31.749849+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:36:39.501462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.819431+0000","last_change":"2023-11-27T15:20:37.673168+0000","last_active":"2023-11-27T15:22:02.819431+0000","last_peered":"2023-11-27T15:22:02.819431+0000","last_clean":"2023-11-27T15:22:02.819431+0000","last_became_active":"2023-11-27T15:20:37.672762+0000","last_became_peered":"2023-11-27T15:20:37.672762+0000","last_unstale":"2023-11-27T15:22:02.819431+0000","last_undegraded":"2023-11-27T15:22:02.819431+0000","last_fullsized":"2023-11-27T15:22:02.819431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:04:26.957766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'6","reported_seq":140,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:43.972931+0000","last_change":"2023-11-27T15:20:32.656402+0000","last_active":"2023-11-27T15:22:43.972931+0000","last_peered":"2023-11-27T15:22:43.972931+0000","last_clean":"2023-11-27T15:22:43.972931+0000","last_became_active":"2023-11-27T15:20:32.656112+0000","last_became_peered":"2023-11-27T15:20:32.656112+0000","last_unstale":"2023-11-27T15:22:43.972931+0000","last_undegraded":"2023-11-27T15:22:43.972931+0000","last_fullsized":"2023-11-27T15:22:43.972931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:40:54.677726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":886,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.743115+0000","last_change":"2023-11-27T15:20:33.223073+0000","last_active":"2023-11-27T15:22:31.743115+0000","last_peered":"2023-11-27T15:22:31.743115+0000","last_clean":"2023-11-27T15:22:31.743115+0000","last_became_active":"2023-11-27T15:20:33.222655+0000","last_became_peered":"2023-11-27T15:20:33.222655+0000","last_unstale":"2023-11-27T15:22:31.743115+0000","last_undegraded":"2023-11-27T15:22:31.743115+0000","last_fullsized":"2023-11-27T15:22:31.743115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:09:23.257015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.819193+0000","last_change":"2023-11-27T15:20:37.678722+0000","last_active":"2023-11-27T15:22:02.819193+0000","last_peered":"2023-11-27T15:22:02.819193+0000","last_clean":"2023-11-27T15:22:02.819193+0000","last_became_active":"2023-11-27T15:20:37.678242+0000","last_became_peered":"2023-11-27T15:20:37.678242+0000","last_unstale":"2023-11-27T15:22:02.819193+0000","last_undegraded":"2023-11-27T15:22:02.819193+0000","last_fullsized":"2023-11-27T15:22:02.819193+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:54:25.629989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.905230+0000","last_change":"2023-11-27T15:20:32.646951+0000","last_active":"2023-11-27T15:22:02.905230+0000","last_peered":"2023-11-27T15:22:02.905230+0000","last_clean":"2023-11-27T15:22:02.905230+0000","last_became_active":"2023-11-27T15:20:32.646620+0000","last_became_peered":"2023-11-27T15:20:32.646620+0000","last_unstale":"2023-11-27T15:22:02.905230+0000","last_undegraded":"2023-11-27T15:22:02.905230+0000","last_fullsized":"2023-11-27T15:22:02.905230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:36:25.079772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.751828+0000","last_change":"2023-11-27T15:20:33.226576+0000","last_active":"2023-11-27T15:22:31.751828+0000","last_peered":"2023-11-27T15:22:31.751828+0000","last_clean":"2023-11-27T15:22:31.751828+0000","last_became_active":"2023-11-27T15:20:33.226181+0000","last_became_peered":"2023-11-27T15:20:33.226181+0000","last_unstale":"2023-11-27T15:22:31.751828+0000","last_undegraded":"2023-11-27T15:22:31.751828+0000","last_fullsized":"2023-11-27T15:22:31.751828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:07:09.638906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.815632+0000","last_change":"2023-11-27T15:20:37.681668+0000","last_active":"2023-11-27T15:22:02.815632+0000","last_peered":"2023-11-27T15:22:02.815632+0000","last_clean":"2023-11-27T15:22:02.815632+0000","last_became_active":"2023-11-27T15:20:37.681236+0000","last_became_peered":"2023-11-27T15:20:37.681236+0000","last_unstale":"2023-11-27T15:22:02.815632+0000","last_undegraded":"2023-11-27T15:22:02.815632+0000","last_fullsized":"2023-11-27T15:22:02.815632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:11:46.529862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.902849+0000","last_change":"2023-11-27T15:20:32.659421+0000","last_active":"2023-11-27T15:22:02.902849+0000","last_peered":"2023-11-27T15:22:02.902849+0000","last_clean":"2023-11-27T15:22:02.902849+0000","last_became_active":"2023-11-27T15:20:32.658882+0000","last_became_peered":"2023-11-27T15:20:32.658882+0000","last_unstale":"2023-11-27T15:22:02.902849+0000","last_undegraded":"2023-11-27T15:22:02.902849+0000","last_fullsized":"2023-11-27T15:22:02.902849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:35:52.409185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.758848+0000","last_change":"2023-11-27T15:20:33.227635+0000","last_active":"2023-11-27T15:22:31.758848+0000","last_peered":"2023-11-27T15:22:31.758848+0000","last_clean":"2023-11-27T15:22:31.758848+0000","last_became_active":"2023-11-27T15:20:33.225217+0000","last_became_peered":"2023-11-27T15:20:33.225217+0000","last_unstale":"2023-11-27T15:22:31.758848+0000","last_undegraded":"2023-11-27T15:22:31.758848+0000","last_fullsized":"2023-11-27T15:22:31.758848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:11:03.899702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.812856+0000","last_change":"2023-11-27T15:20:37.679575+0000","last_active":"2023-11-27T15:22:02.812856+0000","last_peered":"2023-11-27T15:22:02.812856+0000","last_clean":"2023-11-27T15:22:02.812856+0000","last_became_active":"2023-11-27T15:20:37.679131+0000","last_became_peered":"2023-11-27T15:20:37.679131+0000","last_unstale":"2023-11-27T15:22:02.812856+0000","last_undegraded":"2023-11-27T15:22:02.812856+0000","last_fullsized":"2023-11-27T15:22:02.812856+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:03:41.337015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.908090+0000","last_change":"2023-11-27T15:20:32.657155+0000","last_active":"2023-11-27T15:22:02.908090+0000","last_peered":"2023-11-27T15:22:02.908090+0000","last_clean":"2023-11-27T15:22:02.908090+0000","last_became_active":"2023-11-27T15:20:32.656832+0000","last_became_peered":"2023-11-27T15:20:32.656832+0000","last_unstale":"2023-11-27T15:22:02.908090+0000","last_undegraded":"2023-11-27T15:22:02.908090+0000","last_fullsized":"2023-11-27T15:22:02.908090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:14:08.024415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.761021+0000","last_change":"2023-11-27T15:20:33.229215+0000","last_active":"2023-11-27T15:22:31.761021+0000","last_peered":"2023-11-27T15:22:31.761021+0000","last_clean":"2023-11-27T15:22:31.761021+0000","last_became_active":"2023-11-27T15:20:33.228616+0000","last_became_peered":"2023-11-27T15:20:33.228616+0000","last_unstale":"2023-11-27T15:22:31.761021+0000","last_undegraded":"2023-11-27T15:22:31.761021+0000","last_fullsized":"2023-11-27T15:22:31.761021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:16:50.126720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.817360+0000","last_change":"2023-11-27T15:20:37.672582+0000","last_active":"2023-11-27T15:22:02.817360+0000","last_peered":"2023-11-27T15:22:02.817360+0000","last_clean":"2023-11-27T15:22:02.817360+0000","last_became_active":"2023-11-27T15:20:37.672193+0000","last_became_peered":"2023-11-27T15:20:37.672193+0000","last_unstale":"2023-11-27T15:22:02.817360+0000","last_undegraded":"2023-11-27T15:22:02.817360+0000","last_fullsized":"2023-11-27T15:22:02.817360+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:00:48.323529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":131,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:40.449635+0000","last_change":"2023-11-27T15:20:32.647894+0000","last_active":"2023-11-27T15:22:40.449635+0000","last_peered":"2023-11-27T15:22:40.449635+0000","last_clean":"2023-11-27T15:22:40.449635+0000","last_became_active":"2023-11-27T15:20:32.647597+0000","last_became_peered":"2023-11-27T15:20:32.647597+0000","last_unstale":"2023-11-27T15:22:40.449635+0000","last_undegraded":"2023-11-27T15:22:40.449635+0000","last_fullsized":"2023-11-27T15:22:40.449635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:12:48.894187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.753452+0000","last_change":"2023-11-27T15:20:33.229382+0000","last_active":"2023-11-27T15:22:31.753452+0000","last_peered":"2023-11-27T15:22:31.753452+0000","last_clean":"2023-11-27T15:22:31.753452+0000","last_became_active":"2023-11-27T15:20:33.229031+0000","last_became_peered":"2023-11-27T15:20:33.229031+0000","last_unstale":"2023-11-27T15:22:31.753452+0000","last_undegraded":"2023-11-27T15:22:31.753452+0000","last_fullsized":"2023-11-27T15:22:31.753452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:35:27.898226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811592+0000","last_change":"2023-11-27T15:20:37.679131+0000","last_active":"2023-11-27T15:22:02.811592+0000","last_peered":"2023-11-27T15:22:02.811592+0000","last_clean":"2023-11-27T15:22:02.811592+0000","last_became_active":"2023-11-27T15:20:37.678290+0000","last_became_peered":"2023-11-27T15:20:37.678290+0000","last_unstale":"2023-11-27T15:22:02.811592+0000","last_undegraded":"2023-11-27T15:22:02.811592+0000","last_fullsized":"2023-11-27T15:22:02.811592+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:55:02.097941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.244489+0000","last_change":"2023-11-27T15:21:02.797374+0000","last_active":"2023-11-27T15:22:03.244489+0000","last_peered":"2023-11-27T15:22:03.244489+0000","last_clean":"2023-11-27T15:22:03.244489+0000","last_became_active":"2023-11-27T15:21:02.796076+0000","last_became_peered":"2023-11-27T15:21:02.796076+0000","last_unstale":"2023-11-27T15:22:03.244489+0000","last_undegraded":"2023-11-27T15:22:03.244489+0000","last_fullsized":"2023-11-27T15:22:03.244489+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:04:13.004951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.766526+0000","last_change":"2023-11-27T15:21:02.935104+0000","last_active":"2023-11-27T15:22:31.766526+0000","last_peered":"2023-11-27T15:22:31.766526+0000","last_clean":"2023-11-27T15:22:31.766526+0000","last_became_active":"2023-11-27T15:21:02.934486+0000","last_became_peered":"2023-11-27T15:21:02.934486+0000","last_unstale":"2023-11-27T15:22:31.766526+0000","last_undegraded":"2023-11-27T15:22:31.766526+0000","last_fullsized":"2023-11-27T15:22:31.766526+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:48:49.686999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811614+0000","last_change":"2023-11-27T15:20:37.670389+0000","last_active":"2023-11-27T15:22:02.811614+0000","last_peered":"2023-11-27T15:22:02.811614+0000","last_clean":"2023-11-27T15:22:02.811614+0000","last_became_active":"2023-11-27T15:20:37.669512+0000","last_became_peered":"2023-11-27T15:20:37.669512+0000","last_unstale":"2023-11-27T15:22:02.811614+0000","last_undegraded":"2023-11-27T15:22:02.811614+0000","last_fullsized":"2023-11-27T15:22:02.811614+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:25:51.789859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.908694+0000","last_change":"2023-11-27T15:20:32.658287+0000","last_active":"2023-11-27T15:22:02.908694+0000","last_peered":"2023-11-27T15:22:02.908694+0000","last_clean":"2023-11-27T15:22:02.908694+0000","last_became_active":"2023-11-27T15:20:32.657925+0000","last_became_peered":"2023-11-27T15:20:32.657925+0000","last_unstale":"2023-11-27T15:22:02.908694+0000","last_undegraded":"2023-11-27T15:22:02.908694+0000","last_fullsized":"2023-11-27T15:22:02.908694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:47:39.305351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.754013+0000","last_change":"2023-11-27T15:20:33.220240+0000","last_active":"2023-11-27T15:22:31.754013+0000","last_peered":"2023-11-27T15:22:31.754013+0000","last_clean":"2023-11-27T15:22:31.754013+0000","last_became_active":"2023-11-27T15:20:33.219962+0000","last_became_peered":"2023-11-27T15:20:33.219962+0000","last_unstale":"2023-11-27T15:22:31.754013+0000","last_undegraded":"2023-11-27T15:22:31.754013+0000","last_fullsized":"2023-11-27T15:22:31.754013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:00:01.276822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811329+0000","last_change":"2023-11-27T15:20:37.672203+0000","last_active":"2023-11-27T15:22:02.811329+0000","last_peered":"2023-11-27T15:22:02.811329+0000","last_clean":"2023-11-27T15:22:02.811329+0000","last_became_active":"2023-11-27T15:20:37.671149+0000","last_became_peered":"2023-11-27T15:20:37.671149+0000","last_unstale":"2023-11-27T15:22:02.811329+0000","last_undegraded":"2023-11-27T15:22:02.811329+0000","last_fullsized":"2023-11-27T15:22:02.811329+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:15:42.398912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'37","reported_seq":271,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:40.967635+0000","last_change":"2023-11-27T15:20:32.646637+0000","last_active":"2023-11-27T15:22:40.967635+0000","last_peered":"2023-11-27T15:22:40.967635+0000","last_clean":"2023-11-27T15:22:40.967635+0000","last_became_active":"2023-11-27T15:20:32.646292+0000","last_became_peered":"2023-11-27T15:20:32.646292+0000","last_unstale":"2023-11-27T15:22:40.967635+0000","last_undegraded":"2023-11-27T15:22:40.967635+0000","last_fullsized":"2023-11-27T15:22:40.967635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:57:01.372262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60369,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":204,"num_read_kb":162,"num_write":38,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.737253+0000","last_change":"2023-11-27T15:20:33.217078+0000","last_active":"2023-11-27T15:22:31.737253+0000","last_peered":"2023-11-27T15:22:31.737253+0000","last_clean":"2023-11-27T15:22:31.737253+0000","last_became_active":"2023-11-27T15:20:33.216787+0000","last_became_peered":"2023-11-27T15:20:33.216787+0000","last_unstale":"2023-11-27T15:22:31.737253+0000","last_undegraded":"2023-11-27T15:22:31.737253+0000","last_fullsized":"2023-11-27T15:22:31.737253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:39:29.760219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.909130+0000","last_change":"2023-11-27T15:20:37.671800+0000","last_active":"2023-11-27T15:22:02.909130+0000","last_peered":"2023-11-27T15:22:02.909130+0000","last_clean":"2023-11-27T15:22:02.909130+0000","last_became_active":"2023-11-27T15:20:37.671374+0000","last_became_peered":"2023-11-27T15:20:37.671374+0000","last_unstale":"2023-11-27T15:22:02.909130+0000","last_undegraded":"2023-11-27T15:22:02.909130+0000","last_fullsized":"2023-11-27T15:22:02.909130+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:38:25.460193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818948+0000","last_change":"2023-11-27T15:20:32.659508+0000","last_active":"2023-11-27T15:22:02.818948+0000","last_peered":"2023-11-27T15:22:02.818948+0000","last_clean":"2023-11-27T15:22:02.818948+0000","last_became_active":"2023-11-27T15:20:32.659226+0000","last_became_peered":"2023-11-27T15:20:32.659226+0000","last_unstale":"2023-11-27T15:22:02.818948+0000","last_undegraded":"2023-11-27T15:22:02.818948+0000","last_fullsized":"2023-11-27T15:22:02.818948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:18:06.852915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.750607+0000","last_change":"2023-11-27T15:20:33.230012+0000","last_active":"2023-11-27T15:22:31.750607+0000","last_peered":"2023-11-27T15:22:31.750607+0000","last_clean":"2023-11-27T15:22:31.750607+0000","last_became_active":"2023-11-27T15:20:33.225193+0000","last_became_peered":"2023-11-27T15:20:33.225193+0000","last_unstale":"2023-11-27T15:22:31.750607+0000","last_undegraded":"2023-11-27T15:22:31.750607+0000","last_fullsized":"2023-11-27T15:22:31.750607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:42:53.662577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.815274+0000","last_change":"2023-11-27T15:20:37.672430+0000","last_active":"2023-11-27T15:22:02.815274+0000","last_peered":"2023-11-27T15:22:02.815274+0000","last_clean":"2023-11-27T15:22:02.815274+0000","last_became_active":"2023-11-27T15:20:37.671982+0000","last_became_peered":"2023-11-27T15:20:37.671982+0000","last_unstale":"2023-11-27T15:22:02.815274+0000","last_undegraded":"2023-11-27T15:22:02.815274+0000","last_fullsized":"2023-11-27T15:22:02.815274+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:41:34.081786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.908478+0000","last_change":"2023-11-27T15:20:32.655015+0000","last_active":"2023-11-27T15:22:02.908478+0000","last_peered":"2023-11-27T15:22:02.908478+0000","last_clean":"2023-11-27T15:22:02.908478+0000","last_became_active":"2023-11-27T15:20:32.654662+0000","last_became_peered":"2023-11-27T15:20:32.654662+0000","last_unstale":"2023-11-27T15:22:02.908478+0000","last_undegraded":"2023-11-27T15:22:02.908478+0000","last_fullsized":"2023-11-27T15:22:02.908478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:49:03.386586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.754433+0000","last_change":"2023-11-27T15:20:33.233980+0000","last_active":"2023-11-27T15:22:31.754433+0000","last_peered":"2023-11-27T15:22:31.754433+0000","last_clean":"2023-11-27T15:22:31.754433+0000","last_became_active":"2023-11-27T15:20:33.233581+0000","last_became_peered":"2023-11-27T15:20:33.233581+0000","last_unstale":"2023-11-27T15:22:31.754433+0000","last_undegraded":"2023-11-27T15:22:31.754433+0000","last_fullsized":"2023-11-27T15:22:31.754433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:52:04.285704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811523+0000","last_change":"2023-11-27T15:20:37.682359+0000","last_active":"2023-11-27T15:22:02.811523+0000","last_peered":"2023-11-27T15:22:02.811523+0000","last_clean":"2023-11-27T15:22:02.811523+0000","last_became_active":"2023-11-27T15:20:37.681888+0000","last_became_peered":"2023-11-27T15:20:37.681888+0000","last_unstale":"2023-11-27T15:22:02.811523+0000","last_undegraded":"2023-11-27T15:22:02.811523+0000","last_fullsized":"2023-11-27T15:22:02.811523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:04:23.066492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":139,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:42.967416+0000","last_change":"2023-11-27T15:20:32.653388+0000","last_active":"2023-11-27T15:22:42.967416+0000","last_peered":"2023-11-27T15:22:42.967416+0000","last_clean":"2023-11-27T15:22:42.967416+0000","last_became_active":"2023-11-27T15:20:32.653007+0000","last_became_peered":"2023-11-27T15:20:32.653007+0000","last_unstale":"2023-11-27T15:22:42.967416+0000","last_undegraded":"2023-11-27T15:22:42.967416+0000","last_fullsized":"2023-11-27T15:22:42.967416+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:40:28.723669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1071,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.750879+0000","last_change":"2023-11-27T15:20:33.216539+0000","last_active":"2023-11-27T15:22:31.750879+0000","last_peered":"2023-11-27T15:22:31.750879+0000","last_clean":"2023-11-27T15:22:31.750879+0000","last_became_active":"2023-11-27T15:20:33.216166+0000","last_became_peered":"2023-11-27T15:20:33.216166+0000","last_unstale":"2023-11-27T15:22:31.750879+0000","last_undegraded":"2023-11-27T15:22:31.750879+0000","last_fullsized":"2023-11-27T15:22:31.750879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:17:42.918928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.902913+0000","last_change":"2023-11-27T15:20:37.675735+0000","last_active":"2023-11-27T15:22:02.902913+0000","last_peered":"2023-11-27T15:22:02.902913+0000","last_clean":"2023-11-27T15:22:02.902913+0000","last_became_active":"2023-11-27T15:20:37.669685+0000","last_became_peered":"2023-11-27T15:20:37.669685+0000","last_unstale":"2023-11-27T15:22:02.902913+0000","last_undegraded":"2023-11-27T15:22:02.902913+0000","last_fullsized":"2023-11-27T15:22:02.902913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:55:44.635583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:05.499173+0000","last_change":"2023-11-27T15:22:05.499173+0000","last_active":"2023-11-27T15:22:05.499173+0000","last_peered":"2023-11-27T15:22:05.499173+0000","last_clean":"2023-11-27T15:22:05.499173+0000","last_became_active":"2023-11-27T15:22:02.821261+0000","last_became_peered":"2023-11-27T15:22:02.821261+0000","last_unstale":"2023-11-27T15:22:05.499173+0000","last_undegraded":"2023-11-27T15:22:05.499173+0000","last_fullsized":"2023-11-27T15:22:05.499173+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:16:43.958003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.748850+0000","last_change":"2023-11-27T15:20:33.225019+0000","last_active":"2023-11-27T15:22:31.748850+0000","last_peered":"2023-11-27T15:22:31.748850+0000","last_clean":"2023-11-27T15:22:31.748850+0000","last_became_active":"2023-11-27T15:20:33.224622+0000","last_became_peered":"2023-11-27T15:20:33.224622+0000","last_unstale":"2023-11-27T15:22:31.748850+0000","last_undegraded":"2023-11-27T15:22:31.748850+0000","last_fullsized":"2023-11-27T15:22:31.748850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:18:00.816094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.908162+0000","last_change":"2023-11-27T15:20:37.680808+0000","last_active":"2023-11-27T15:22:02.908162+0000","last_peered":"2023-11-27T15:22:02.908162+0000","last_clean":"2023-11-27T15:22:02.908162+0000","last_became_active":"2023-11-27T15:20:37.680348+0000","last_became_peered":"2023-11-27T15:20:37.680348+0000","last_unstale":"2023-11-27T15:22:02.908162+0000","last_undegraded":"2023-11-27T15:22:02.908162+0000","last_fullsized":"2023-11-27T15:22:02.908162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:37:19.386150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.819004+0000","last_change":"2023-11-27T15:20:32.651093+0000","last_active":"2023-11-27T15:22:02.819004+0000","last_peered":"2023-11-27T15:22:02.819004+0000","last_clean":"2023-11-27T15:22:02.819004+0000","last_became_active":"2023-11-27T15:20:32.650657+0000","last_became_peered":"2023-11-27T15:20:32.650657+0000","last_unstale":"2023-11-27T15:22:02.819004+0000","last_undegraded":"2023-11-27T15:22:02.819004+0000","last_fullsized":"2023-11-27T15:22:02.819004+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:56:44.895712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.749518+0000","last_change":"2023-11-27T15:20:33.224806+0000","last_active":"2023-11-27T15:22:31.749518+0000","last_peered":"2023-11-27T15:22:31.749518+0000","last_clean":"2023-11-27T15:22:31.749518+0000","last_became_active":"2023-11-27T15:20:33.224362+0000","last_became_peered":"2023-11-27T15:20:33.224362+0000","last_unstale":"2023-11-27T15:22:31.749518+0000","last_undegraded":"2023-11-27T15:22:31.749518+0000","last_fullsized":"2023-11-27T15:22:31.749518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:20:50.902402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818048+0000","last_change":"2023-11-27T15:20:37.849274+0000","last_active":"2023-11-27T15:22:02.818048+0000","last_peered":"2023-11-27T15:22:02.818048+0000","last_clean":"2023-11-27T15:22:02.818048+0000","last_became_active":"2023-11-27T15:20:37.848839+0000","last_became_peered":"2023-11-27T15:20:37.848839+0000","last_unstale":"2023-11-27T15:22:02.818048+0000","last_undegraded":"2023-11-27T15:22:02.818048+0000","last_fullsized":"2023-11-27T15:22:02.818048+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:49:32.136447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811304+0000","last_change":"2023-11-27T15:20:32.660835+0000","last_active":"2023-11-27T15:22:02.811304+0000","last_peered":"2023-11-27T15:22:02.811304+0000","last_clean":"2023-11-27T15:22:02.811304+0000","last_became_active":"2023-11-27T15:20:32.660431+0000","last_became_peered":"2023-11-27T15:20:32.660431+0000","last_unstale":"2023-11-27T15:22:02.811304+0000","last_undegraded":"2023-11-27T15:22:02.811304+0000","last_fullsized":"2023-11-27T15:22:02.811304+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:48:52.707533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.759772+0000","last_change":"2023-11-27T15:20:33.245772+0000","last_active":"2023-11-27T15:22:31.759772+0000","last_peered":"2023-11-27T15:22:31.759772+0000","last_clean":"2023-11-27T15:22:31.759772+0000","last_became_active":"2023-11-27T15:20:33.245468+0000","last_became_peered":"2023-11-27T15:20:33.245468+0000","last_unstale":"2023-11-27T15:22:31.759772+0000","last_undegraded":"2023-11-27T15:22:31.759772+0000","last_fullsized":"2023-11-27T15:22:31.759772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:58:59.950032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.902964+0000","last_change":"2023-11-27T15:20:37.678972+0000","last_active":"2023-11-27T15:22:02.902964+0000","last_peered":"2023-11-27T15:22:02.902964+0000","last_clean":"2023-11-27T15:22:02.902964+0000","last_became_active":"2023-11-27T15:20:37.678304+0000","last_became_peered":"2023-11-27T15:20:37.678304+0000","last_unstale":"2023-11-27T15:22:02.902964+0000","last_undegraded":"2023-11-27T15:22:02.902964+0000","last_fullsized":"2023-11-27T15:22:02.902964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:05:43.664151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.244402+0000","last_change":"2023-11-27T15:20:32.661739+0000","last_active":"2023-11-27T15:22:03.244402+0000","last_peered":"2023-11-27T15:22:03.244402+0000","last_clean":"2023-11-27T15:22:03.244402+0000","last_became_active":"2023-11-27T15:20:32.661401+0000","last_became_peered":"2023-11-27T15:20:32.661401+0000","last_unstale":"2023-11-27T15:22:03.244402+0000","last_undegraded":"2023-11-27T15:22:03.244402+0000","last_fullsized":"2023-11-27T15:22:03.244402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:32:44.035448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.749301+0000","last_change":"2023-11-27T15:20:33.235595+0000","last_active":"2023-11-27T15:22:31.749301+0000","last_peered":"2023-11-27T15:22:31.749301+0000","last_clean":"2023-11-27T15:22:31.749301+0000","last_became_active":"2023-11-27T15:20:33.235277+0000","last_became_peered":"2023-11-27T15:20:33.235277+0000","last_unstale":"2023-11-27T15:22:31.749301+0000","last_undegraded":"2023-11-27T15:22:31.749301+0000","last_fullsized":"2023-11-27T15:22:31.749301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:13:59.365599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.903185+0000","last_change":"2023-11-27T15:21:02.934194+0000","last_active":"2023-11-27T15:22:02.903185+0000","last_peered":"2023-11-27T15:22:02.903185+0000","last_clean":"2023-11-27T15:22:02.903185+0000","last_became_active":"2023-11-27T15:21:02.933555+0000","last_became_peered":"2023-11-27T15:21:02.933555+0000","last_unstale":"2023-11-27T15:22:02.903185+0000","last_undegraded":"2023-11-27T15:22:02.903185+0000","last_fullsized":"2023-11-27T15:22:02.903185+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:16:19.222019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.755634+0000","last_change":"2023-11-27T15:21:02.784351+0000","last_active":"2023-11-27T15:22:31.755634+0000","last_peered":"2023-11-27T15:22:31.755634+0000","last_clean":"2023-11-27T15:22:31.755634+0000","last_became_active":"2023-11-27T15:21:02.783996+0000","last_became_peered":"2023-11-27T15:21:02.783996+0000","last_unstale":"2023-11-27T15:22:31.755634+0000","last_undegraded":"2023-11-27T15:22:31.755634+0000","last_fullsized":"2023-11-27T15:22:31.755634+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:17:24.113607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":133,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:40.963521+0000","last_change":"2023-11-27T15:20:32.659823+0000","last_active":"2023-11-27T15:22:40.963521+0000","last_peered":"2023-11-27T15:22:40.963521+0000","last_clean":"2023-11-27T15:22:40.963521+0000","last_became_active":"2023-11-27T15:20:32.659472+0000","last_became_peered":"2023-11-27T15:20:32.659472+0000","last_unstale":"2023-11-27T15:22:40.963521+0000","last_undegraded":"2023-11-27T15:22:40.963521+0000","last_fullsized":"2023-11-27T15:22:40.963521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:08:32.142408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.244444+0000","last_change":"2023-11-27T15:20:37.671494+0000","last_active":"2023-11-27T15:22:03.244444+0000","last_peered":"2023-11-27T15:22:03.244444+0000","last_clean":"2023-11-27T15:22:03.244444+0000","last_became_active":"2023-11-27T15:20:37.671010+0000","last_became_peered":"2023-11-27T15:20:37.671010+0000","last_unstale":"2023-11-27T15:22:03.244444+0000","last_undegraded":"2023-11-27T15:22:03.244444+0000","last_fullsized":"2023-11-27T15:22:03.244444+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:51:51.646563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.758695+0000","last_change":"2023-11-27T15:20:33.241669+0000","last_active":"2023-11-27T15:22:31.758695+0000","last_peered":"2023-11-27T15:22:31.758695+0000","last_clean":"2023-11-27T15:22:31.758695+0000","last_became_active":"2023-11-27T15:20:33.241373+0000","last_became_peered":"2023-11-27T15:20:33.241373+0000","last_unstale":"2023-11-27T15:22:31.758695+0000","last_undegraded":"2023-11-27T15:22:31.758695+0000","last_fullsized":"2023-11-27T15:22:31.758695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:03:47.587644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811591+0000","last_change":"2023-11-27T15:20:32.647596+0000","last_active":"2023-11-27T15:22:02.811591+0000","last_peered":"2023-11-27T15:22:02.811591+0000","last_clean":"2023-11-27T15:22:02.811591+0000","last_became_active":"2023-11-27T15:20:32.647228+0000","last_became_peered":"2023-11-27T15:20:32.647228+0000","last_unstale":"2023-11-27T15:22:02.811591+0000","last_undegraded":"2023-11-27T15:22:02.811591+0000","last_fullsized":"2023-11-27T15:22:02.811591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:37:28.551859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.244098+0000","last_change":"2023-11-27T15:19:30.114855+0000","last_active":"2023-11-27T15:22:03.244098+0000","last_peered":"2023-11-27T15:22:03.244098+0000","last_clean":"2023-11-27T15:22:03.244098+0000","last_became_active":"2023-11-27T15:19:30.114419+0000","last_became_peered":"2023-11-27T15:19:30.114419+0000","last_unstale":"2023-11-27T15:22:03.244098+0000","last_undegraded":"2023-11-27T15:22:03.244098+0000","last_fullsized":"2023-11-27T15:22:03.244098+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:20.216695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:20.216695+0000","last_clean_scrub_stamp":"2023-11-27T15:16:20.216695+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:02:43.854910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.908738+0000","last_change":"2023-11-27T15:20:37.850539+0000","last_active":"2023-11-27T15:22:02.908738+0000","last_peered":"2023-11-27T15:22:02.908738+0000","last_clean":"2023-11-27T15:22:02.908738+0000","last_became_active":"2023-11-27T15:20:37.850044+0000","last_became_peered":"2023-11-27T15:20:37.850044+0000","last_unstale":"2023-11-27T15:22:02.908738+0000","last_undegraded":"2023-11-27T15:22:02.908738+0000","last_fullsized":"2023-11-27T15:22:02.908738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:54:36.190506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.741984+0000","last_change":"2023-11-27T15:20:33.211319+0000","last_active":"2023-11-27T15:22:31.741984+0000","last_peered":"2023-11-27T15:22:31.741984+0000","last_clean":"2023-11-27T15:22:31.741984+0000","last_became_active":"2023-11-27T15:20:33.211004+0000","last_became_peered":"2023-11-27T15:20:33.211004+0000","last_unstale":"2023-11-27T15:22:31.741984+0000","last_undegraded":"2023-11-27T15:22:31.741984+0000","last_fullsized":"2023-11-27T15:22:31.741984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:52:10.543307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.244696+0000","last_change":"2023-11-27T15:20:32.655003+0000","last_active":"2023-11-27T15:22:03.244696+0000","last_peered":"2023-11-27T15:22:03.244696+0000","last_clean":"2023-11-27T15:22:03.244696+0000","last_became_active":"2023-11-27T15:20:32.654517+0000","last_became_peered":"2023-11-27T15:20:32.654517+0000","last_unstale":"2023-11-27T15:22:03.244696+0000","last_undegraded":"2023-11-27T15:22:03.244696+0000","last_fullsized":"2023-11-27T15:22:03.244696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:18:43.790411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.815270+0000","last_change":"2023-11-27T15:20:37.673183+0000","last_active":"2023-11-27T15:22:02.815270+0000","last_peered":"2023-11-27T15:22:02.815270+0000","last_clean":"2023-11-27T15:22:02.815270+0000","last_became_active":"2023-11-27T15:20:37.672838+0000","last_became_peered":"2023-11-27T15:20:37.672838+0000","last_unstale":"2023-11-27T15:22:02.815270+0000","last_undegraded":"2023-11-27T15:22:02.815270+0000","last_fullsized":"2023-11-27T15:22:02.815270+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:42:26.039321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818752+0000","last_change":"2023-11-27T15:20:32.659540+0000","last_active":"2023-11-27T15:22:02.818752+0000","last_peered":"2023-11-27T15:22:02.818752+0000","last_clean":"2023-11-27T15:22:02.818752+0000","last_became_active":"2023-11-27T15:20:32.659211+0000","last_became_peered":"2023-11-27T15:20:32.659211+0000","last_unstale":"2023-11-27T15:22:02.818752+0000","last_undegraded":"2023-11-27T15:22:02.818752+0000","last_fullsized":"2023-11-27T15:22:02.818752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:32:29.624516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.757012+0000","last_change":"2023-11-27T15:20:33.216405+0000","last_active":"2023-11-27T15:22:31.757012+0000","last_peered":"2023-11-27T15:22:31.757012+0000","last_clean":"2023-11-27T15:22:31.757012+0000","last_became_active":"2023-11-27T15:20:33.216072+0000","last_became_peered":"2023-11-27T15:20:33.216072+0000","last_unstale":"2023-11-27T15:22:31.757012+0000","last_undegraded":"2023-11-27T15:22:31.757012+0000","last_fullsized":"2023-11-27T15:22:31.757012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:24:07.496562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.817387+0000","last_change":"2023-11-27T15:20:37.849753+0000","last_active":"2023-11-27T15:22:02.817387+0000","last_peered":"2023-11-27T15:22:02.817387+0000","last_clean":"2023-11-27T15:22:02.817387+0000","last_became_active":"2023-11-27T15:20:37.848576+0000","last_became_peered":"2023-11-27T15:20:37.848576+0000","last_unstale":"2023-11-27T15:22:02.817387+0000","last_undegraded":"2023-11-27T15:22:02.817387+0000","last_fullsized":"2023-11-27T15:22:02.817387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:33:46.286142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'8","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:41.517782+0000","last_change":"2023-11-27T15:20:32.652739+0000","last_active":"2023-11-27T15:22:41.517782+0000","last_peered":"2023-11-27T15:22:41.517782+0000","last_clean":"2023-11-27T15:22:41.517782+0000","last_became_active":"2023-11-27T15:20:32.652314+0000","last_became_peered":"2023-11-27T15:20:32.652314+0000","last_unstale":"2023-11-27T15:22:41.517782+0000","last_undegraded":"2023-11-27T15:22:41.517782+0000","last_fullsized":"2023-11-27T15:22:41.517782+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:00:14.008740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.758452+0000","last_change":"2023-11-27T15:20:33.240552+0000","last_active":"2023-11-27T15:22:31.758452+0000","last_peered":"2023-11-27T15:22:31.758452+0000","last_clean":"2023-11-27T15:22:31.758452+0000","last_became_active":"2023-11-27T15:20:33.239885+0000","last_became_peered":"2023-11-27T15:20:33.239885+0000","last_unstale":"2023-11-27T15:22:31.758452+0000","last_undegraded":"2023-11-27T15:22:31.758452+0000","last_fullsized":"2023-11-27T15:22:31.758452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:44:48.111242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818035+0000","last_change":"2023-11-27T15:20:37.669895+0000","last_active":"2023-11-27T15:22:02.818035+0000","last_peered":"2023-11-27T15:22:02.818035+0000","last_clean":"2023-11-27T15:22:02.818035+0000","last_became_active":"2023-11-27T15:20:37.669489+0000","last_became_peered":"2023-11-27T15:20:37.669489+0000","last_unstale":"2023-11-27T15:22:02.818035+0000","last_undegraded":"2023-11-27T15:22:02.818035+0000","last_fullsized":"2023-11-27T15:22:02.818035+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:06:13.495056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.812082+0000","last_change":"2023-11-27T15:20:32.656773+0000","last_active":"2023-11-27T15:22:02.812082+0000","last_peered":"2023-11-27T15:22:02.812082+0000","last_clean":"2023-11-27T15:22:02.812082+0000","last_became_active":"2023-11-27T15:20:32.656451+0000","last_became_peered":"2023-11-27T15:20:32.656451+0000","last_unstale":"2023-11-27T15:22:02.812082+0000","last_undegraded":"2023-11-27T15:22:02.812082+0000","last_fullsized":"2023-11-27T15:22:02.812082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:03:12.399313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.765942+0000","last_change":"2023-11-27T15:20:33.231685+0000","last_active":"2023-11-27T15:22:31.765942+0000","last_peered":"2023-11-27T15:22:31.765942+0000","last_clean":"2023-11-27T15:22:31.765942+0000","last_became_active":"2023-11-27T15:20:33.228265+0000","last_became_peered":"2023-11-27T15:20:33.228265+0000","last_unstale":"2023-11-27T15:22:31.765942+0000","last_undegraded":"2023-11-27T15:22:31.765942+0000","last_fullsized":"2023-11-27T15:22:31.765942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:53:48.855000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811795+0000","last_change":"2023-11-27T15:21:02.935830+0000","last_active":"2023-11-27T15:22:02.811795+0000","last_peered":"2023-11-27T15:22:02.811795+0000","last_clean":"2023-11-27T15:22:02.811795+0000","last_became_active":"2023-11-27T15:21:02.935412+0000","last_became_peered":"2023-11-27T15:21:02.935412+0000","last_unstale":"2023-11-27T15:22:02.811795+0000","last_undegraded":"2023-11-27T15:22:02.811795+0000","last_fullsized":"2023-11-27T15:22:02.811795+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:22:23.956927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.244630+0000","last_change":"2023-11-27T15:20:32.660424+0000","last_active":"2023-11-27T15:22:03.244630+0000","last_peered":"2023-11-27T15:22:03.244630+0000","last_clean":"2023-11-27T15:22:03.244630+0000","last_became_active":"2023-11-27T15:20:32.660112+0000","last_became_peered":"2023-11-27T15:20:32.660112+0000","last_unstale":"2023-11-27T15:22:03.244630+0000","last_undegraded":"2023-11-27T15:22:03.244630+0000","last_fullsized":"2023-11-27T15:22:03.244630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:05:10.594173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.745264+0000","last_change":"2023-11-27T15:20:33.227677+0000","last_active":"2023-11-27T15:22:31.745264+0000","last_peered":"2023-11-27T15:22:31.745264+0000","last_clean":"2023-11-27T15:22:31.745264+0000","last_became_active":"2023-11-27T15:20:33.227334+0000","last_became_peered":"2023-11-27T15:20:33.227334+0000","last_unstale":"2023-11-27T15:22:31.745264+0000","last_undegraded":"2023-11-27T15:22:31.745264+0000","last_fullsized":"2023-11-27T15:22:31.745264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:19:08.573620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818782+0000","last_change":"2023-11-27T15:21:02.934516+0000","last_active":"2023-11-27T15:22:02.818782+0000","last_peered":"2023-11-27T15:22:02.818782+0000","last_clean":"2023-11-27T15:22:02.818782+0000","last_became_active":"2023-11-27T15:21:02.934012+0000","last_became_peered":"2023-11-27T15:21:02.934012+0000","last_unstale":"2023-11-27T15:22:02.818782+0000","last_undegraded":"2023-11-27T15:22:02.818782+0000","last_fullsized":"2023-11-27T15:22:02.818782+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:54:54.500574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.817444+0000","last_change":"2023-11-27T15:20:32.657935+0000","last_active":"2023-11-27T15:22:02.817444+0000","last_peered":"2023-11-27T15:22:02.817444+0000","last_clean":"2023-11-27T15:22:02.817444+0000","last_became_active":"2023-11-27T15:20:32.657579+0000","last_became_peered":"2023-11-27T15:20:32.657579+0000","last_unstale":"2023-11-27T15:22:02.817444+0000","last_undegraded":"2023-11-27T15:22:02.817444+0000","last_fullsized":"2023-11-27T15:22:02.817444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:00:47.684043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.767085+0000","last_change":"2023-11-27T15:20:33.225817+0000","last_active":"2023-11-27T15:22:31.767085+0000","last_peered":"2023-11-27T15:22:31.767085+0000","last_clean":"2023-11-27T15:22:31.767085+0000","last_became_active":"2023-11-27T15:20:33.225438+0000","last_became_peered":"2023-11-27T15:20:33.225438+0000","last_unstale":"2023-11-27T15:22:31.767085+0000","last_undegraded":"2023-11-27T15:22:31.767085+0000","last_fullsized":"2023-11-27T15:22:31.767085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:07:45.956918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811681+0000","last_change":"2023-11-27T15:21:02.934304+0000","last_active":"2023-11-27T15:22:02.811681+0000","last_peered":"2023-11-27T15:22:02.811681+0000","last_clean":"2023-11-27T15:22:02.811681+0000","last_became_active":"2023-11-27T15:21:02.933709+0000","last_became_peered":"2023-11-27T15:21:02.933709+0000","last_unstale":"2023-11-27T15:22:02.811681+0000","last_undegraded":"2023-11-27T15:22:02.811681+0000","last_fullsized":"2023-11-27T15:22:02.811681+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:52:00.767226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.119947+0000","last_change":"2023-11-27T15:22:03.119402+0000","last_active":"2023-11-27T15:22:03.119947+0000","last_peered":"2023-11-27T15:22:03.119947+0000","last_clean":"2023-11-27T15:22:03.119947+0000","last_became_active":"2023-11-27T15:22:02.819940+0000","last_became_peered":"2023-11-27T15:22:02.819940+0000","last_unstale":"2023-11-27T15:22:03.119947+0000","last_undegraded":"2023-11-27T15:22:03.119947+0000","last_fullsized":"2023-11-27T15:22:03.119947+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:41:14.055640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.758801+0000","last_change":"2023-11-27T15:20:33.235755+0000","last_active":"2023-11-27T15:22:31.758801+0000","last_peered":"2023-11-27T15:22:31.758801+0000","last_clean":"2023-11-27T15:22:31.758801+0000","last_became_active":"2023-11-27T15:20:33.234576+0000","last_became_peered":"2023-11-27T15:20:33.234576+0000","last_unstale":"2023-11-27T15:22:31.758801+0000","last_undegraded":"2023-11-27T15:22:31.758801+0000","last_fullsized":"2023-11-27T15:22:31.758801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:15:16.743978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811797+0000","last_change":"2023-11-27T15:20:37.849121+0000","last_active":"2023-11-27T15:22:02.811797+0000","last_peered":"2023-11-27T15:22:02.811797+0000","last_clean":"2023-11-27T15:22:02.811797+0000","last_became_active":"2023-11-27T15:20:37.848683+0000","last_became_peered":"2023-11-27T15:20:37.848683+0000","last_unstale":"2023-11-27T15:22:02.811797+0000","last_undegraded":"2023-11-27T15:22:02.811797+0000","last_fullsized":"2023-11-27T15:22:02.811797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:37:37.945965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.817973+0000","last_change":"2023-11-27T15:20:32.655896+0000","last_active":"2023-11-27T15:22:02.817973+0000","last_peered":"2023-11-27T15:22:02.817973+0000","last_clean":"2023-11-27T15:22:02.817973+0000","last_became_active":"2023-11-27T15:20:32.655623+0000","last_became_peered":"2023-11-27T15:20:32.655623+0000","last_unstale":"2023-11-27T15:22:02.817973+0000","last_undegraded":"2023-11-27T15:22:02.817973+0000","last_fullsized":"2023-11-27T15:22:02.817973+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:58:52.716210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.753875+0000","last_change":"2023-11-27T15:20:33.227318+0000","last_active":"2023-11-27T15:22:31.753875+0000","last_peered":"2023-11-27T15:22:31.753875+0000","last_clean":"2023-11-27T15:22:31.753875+0000","last_became_active":"2023-11-27T15:20:33.226944+0000","last_became_peered":"2023-11-27T15:20:33.226944+0000","last_unstale":"2023-11-27T15:22:31.753875+0000","last_undegraded":"2023-11-27T15:22:31.753875+0000","last_fullsized":"2023-11-27T15:22:31.753875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:13:26.176414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.244531+0000","last_change":"2023-11-27T15:20:37.669098+0000","last_active":"2023-11-27T15:22:03.244531+0000","last_peered":"2023-11-27T15:22:03.244531+0000","last_clean":"2023-11-27T15:22:03.244531+0000","last_became_active":"2023-11-27T15:20:37.668693+0000","last_became_peered":"2023-11-27T15:20:37.668693+0000","last_unstale":"2023-11-27T15:22:03.244531+0000","last_undegraded":"2023-11-27T15:22:03.244531+0000","last_fullsized":"2023-11-27T15:22:03.244531+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:45:51.479637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.819574+0000","last_change":"2023-11-27T15:20:32.660285+0000","last_active":"2023-11-27T15:22:02.819574+0000","last_peered":"2023-11-27T15:22:02.819574+0000","last_clean":"2023-11-27T15:22:02.819574+0000","last_became_active":"2023-11-27T15:20:32.659982+0000","last_became_peered":"2023-11-27T15:20:32.659982+0000","last_unstale":"2023-11-27T15:22:02.819574+0000","last_undegraded":"2023-11-27T15:22:02.819574+0000","last_fullsized":"2023-11-27T15:22:02.819574+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:50:18.637710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.750955+0000","last_change":"2023-11-27T15:20:33.226629+0000","last_active":"2023-11-27T15:22:31.750955+0000","last_peered":"2023-11-27T15:22:31.750955+0000","last_clean":"2023-11-27T15:22:31.750955+0000","last_became_active":"2023-11-27T15:20:33.226183+0000","last_became_peered":"2023-11-27T15:20:33.226183+0000","last_unstale":"2023-11-27T15:22:31.750955+0000","last_undegraded":"2023-11-27T15:22:31.750955+0000","last_fullsized":"2023-11-27T15:22:31.750955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:17:31.612135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.812710+0000","last_change":"2023-11-27T15:20:37.680987+0000","last_active":"2023-11-27T15:22:02.812710+0000","last_peered":"2023-11-27T15:22:02.812710+0000","last_clean":"2023-11-27T15:22:02.812710+0000","last_became_active":"2023-11-27T15:20:37.680219+0000","last_became_peered":"2023-11-27T15:20:37.680219+0000","last_unstale":"2023-11-27T15:22:02.812710+0000","last_undegraded":"2023-11-27T15:22:02.812710+0000","last_fullsized":"2023-11-27T15:22:02.812710+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:55:34.725659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.908876+0000","last_change":"2023-11-27T15:20:32.655035+0000","last_active":"2023-11-27T15:22:02.908876+0000","last_peered":"2023-11-27T15:22:02.908876+0000","last_clean":"2023-11-27T15:22:02.908876+0000","last_became_active":"2023-11-27T15:20:32.654687+0000","last_became_peered":"2023-11-27T15:20:32.654687+0000","last_unstale":"2023-11-27T15:22:02.908876+0000","last_undegraded":"2023-11-27T15:22:02.908876+0000","last_fullsized":"2023-11-27T15:22:02.908876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:57:01.888463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.759364+0000","last_change":"2023-11-27T15:20:33.245908+0000","last_active":"2023-11-27T15:22:31.759364+0000","last_peered":"2023-11-27T15:22:31.759364+0000","last_clean":"2023-11-27T15:22:31.759364+0000","last_became_active":"2023-11-27T15:20:33.245576+0000","last_became_peered":"2023-11-27T15:20:33.245576+0000","last_unstale":"2023-11-27T15:22:31.759364+0000","last_undegraded":"2023-11-27T15:22:31.759364+0000","last_fullsized":"2023-11-27T15:22:31.759364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:27:55.968118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.907992+0000","last_change":"2023-11-27T15:20:37.674253+0000","last_active":"2023-11-27T15:22:02.907992+0000","last_peered":"2023-11-27T15:22:02.907992+0000","last_clean":"2023-11-27T15:22:02.907992+0000","last_became_active":"2023-11-27T15:20:37.673708+0000","last_became_peered":"2023-11-27T15:20:37.673708+0000","last_unstale":"2023-11-27T15:22:02.907992+0000","last_undegraded":"2023-11-27T15:22:02.907992+0000","last_fullsized":"2023-11-27T15:22:02.907992+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:39:38.298957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.903185+0000","last_change":"2023-11-27T15:20:32.659605+0000","last_active":"2023-11-27T15:22:02.903185+0000","last_peered":"2023-11-27T15:22:02.903185+0000","last_clean":"2023-11-27T15:22:02.903185+0000","last_became_active":"2023-11-27T15:20:32.659186+0000","last_became_peered":"2023-11-27T15:20:32.659186+0000","last_unstale":"2023-11-27T15:22:02.903185+0000","last_undegraded":"2023-11-27T15:22:02.903185+0000","last_fullsized":"2023-11-27T15:22:02.903185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:38:50.310551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.745773+0000","last_change":"2023-11-27T15:20:33.227914+0000","last_active":"2023-11-27T15:22:31.745773+0000","last_peered":"2023-11-27T15:22:31.745773+0000","last_clean":"2023-11-27T15:22:31.745773+0000","last_became_active":"2023-11-27T15:20:33.227572+0000","last_became_peered":"2023-11-27T15:20:33.227572+0000","last_unstale":"2023-11-27T15:22:31.745773+0000","last_undegraded":"2023-11-27T15:22:31.745773+0000","last_fullsized":"2023-11-27T15:22:31.745773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:57:20.426893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.747905+0000","last_change":"2023-11-27T15:20:33.215605+0000","last_active":"2023-11-27T15:22:31.747905+0000","last_peered":"2023-11-27T15:22:31.747905+0000","last_clean":"2023-11-27T15:22:31.747905+0000","last_became_active":"2023-11-27T15:20:33.215295+0000","last_became_peered":"2023-11-27T15:20:33.215295+0000","last_unstale":"2023-11-27T15:22:31.747905+0000","last_undegraded":"2023-11-27T15:22:31.747905+0000","last_fullsized":"2023-11-27T15:22:31.747905+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:16:48.125214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.748947+0000","last_change":"2023-11-27T15:20:33.214014+0000","last_active":"2023-11-27T15:22:31.748947+0000","last_peered":"2023-11-27T15:22:31.748947+0000","last_clean":"2023-11-27T15:22:31.748947+0000","last_became_active":"2023-11-27T15:20:33.213685+0000","last_became_peered":"2023-11-27T15:20:33.213685+0000","last_unstale":"2023-11-27T15:22:31.748947+0000","last_undegraded":"2023-11-27T15:22:31.748947+0000","last_fullsized":"2023-11-27T15:22:31.748947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:53:01.707335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.760580+0000","last_change":"2023-11-27T15:20:33.229722+0000","last_active":"2023-11-27T15:22:31.760580+0000","last_peered":"2023-11-27T15:22:31.760580+0000","last_clean":"2023-11-27T15:22:31.760580+0000","last_became_active":"2023-11-27T15:20:33.229372+0000","last_became_peered":"2023-11-27T15:20:33.229372+0000","last_unstale":"2023-11-27T15:22:31.760580+0000","last_undegraded":"2023-11-27T15:22:31.760580+0000","last_fullsized":"2023-11-27T15:22:31.760580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:40:05.300812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.762612+0000","last_change":"2023-11-27T15:20:33.235047+0000","last_active":"2023-11-27T15:22:31.762612+0000","last_peered":"2023-11-27T15:22:31.762612+0000","last_clean":"2023-11-27T15:22:31.762612+0000","last_became_active":"2023-11-27T15:20:33.233078+0000","last_became_peered":"2023-11-27T15:20:33.233078+0000","last_unstale":"2023-11-27T15:22:31.762612+0000","last_undegraded":"2023-11-27T15:22:31.762612+0000","last_fullsized":"2023-11-27T15:22:31.762612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:45:51.860734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.751664+0000","last_change":"2023-11-27T15:20:33.229118+0000","last_active":"2023-11-27T15:22:31.751664+0000","last_peered":"2023-11-27T15:22:31.751664+0000","last_clean":"2023-11-27T15:22:31.751664+0000","last_became_active":"2023-11-27T15:20:33.228748+0000","last_became_peered":"2023-11-27T15:20:33.228748+0000","last_unstale":"2023-11-27T15:22:31.751664+0000","last_undegraded":"2023-11-27T15:22:31.751664+0000","last_fullsized":"2023-11-27T15:22:31.751664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:19:11.419230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.746259+0000","last_change":"2023-11-27T15:20:33.222714+0000","last_active":"2023-11-27T15:22:31.746259+0000","last_peered":"2023-11-27T15:22:31.746259+0000","last_clean":"2023-11-27T15:22:31.746259+0000","last_became_active":"2023-11-27T15:20:33.222396+0000","last_became_peered":"2023-11-27T15:20:33.222396+0000","last_unstale":"2023-11-27T15:22:31.746259+0000","last_undegraded":"2023-11-27T15:22:31.746259+0000","last_fullsized":"2023-11-27T15:22:31.746259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:52:29.993847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.754172+0000","last_change":"2023-11-27T15:20:33.223568+0000","last_active":"2023-11-27T15:22:31.754172+0000","last_peered":"2023-11-27T15:22:31.754172+0000","last_clean":"2023-11-27T15:22:31.754172+0000","last_became_active":"2023-11-27T15:20:33.220687+0000","last_became_peered":"2023-11-27T15:20:33.220687+0000","last_unstale":"2023-11-27T15:22:31.754172+0000","last_undegraded":"2023-11-27T15:22:31.754172+0000","last_fullsized":"2023-11-27T15:22:31.754172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:48:28.297235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.753086+0000","last_change":"2023-11-27T15:21:02.786992+0000","last_active":"2023-11-27T15:22:31.753086+0000","last_peered":"2023-11-27T15:22:31.753086+0000","last_clean":"2023-11-27T15:22:31.753086+0000","last_became_active":"2023-11-27T15:21:02.786644+0000","last_became_peered":"2023-11-27T15:21:02.786644+0000","last_unstale":"2023-11-27T15:22:31.753086+0000","last_undegraded":"2023-11-27T15:22:31.753086+0000","last_fullsized":"2023-11-27T15:22:31.753086+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:28:04.321999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.756953+0000","last_change":"2023-11-27T15:20:33.245815+0000","last_active":"2023-11-27T15:22:31.756953+0000","last_peered":"2023-11-27T15:22:31.756953+0000","last_clean":"2023-11-27T15:22:31.756953+0000","last_became_active":"2023-11-27T15:20:33.245243+0000","last_became_peered":"2023-11-27T15:20:33.245243+0000","last_unstale":"2023-11-27T15:22:31.756953+0000","last_undegraded":"2023-11-27T15:22:31.756953+0000","last_fullsized":"2023-11-27T15:22:31.756953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:29:54.401852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.753240+0000","last_change":"2023-11-27T15:20:33.245595+0000","last_active":"2023-11-27T15:22:31.753240+0000","last_peered":"2023-11-27T15:22:31.753240+0000","last_clean":"2023-11-27T15:22:31.753240+0000","last_became_active":"2023-11-27T15:20:33.245294+0000","last_became_peered":"2023-11-27T15:20:33.245294+0000","last_unstale":"2023-11-27T15:22:31.753240+0000","last_undegraded":"2023-11-27T15:22:31.753240+0000","last_fullsized":"2023-11-27T15:22:31.753240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:10:33.319483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.761634+0000","last_change":"2023-11-27T15:20:33.232976+0000","last_active":"2023-11-27T15:22:31.761634+0000","last_peered":"2023-11-27T15:22:31.761634+0000","last_clean":"2023-11-27T15:22:31.761634+0000","last_became_active":"2023-11-27T15:20:33.232648+0000","last_became_peered":"2023-11-27T15:20:33.232648+0000","last_unstale":"2023-11-27T15:22:31.761634+0000","last_undegraded":"2023-11-27T15:22:31.761634+0000","last_fullsized":"2023-11-27T15:22:31.761634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:21:07.049919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.756893+0000","last_change":"2023-11-27T15:20:33.226332+0000","last_active":"2023-11-27T15:22:31.756893+0000","last_peered":"2023-11-27T15:22:31.756893+0000","last_clean":"2023-11-27T15:22:31.756893+0000","last_became_active":"2023-11-27T15:20:33.226000+0000","last_became_peered":"2023-11-27T15:20:33.226000+0000","last_unstale":"2023-11-27T15:22:31.756893+0000","last_undegraded":"2023-11-27T15:22:31.756893+0000","last_fullsized":"2023-11-27T15:22:31.756893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:02:35.531072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.750943+0000","last_change":"2023-11-27T15:20:33.217927+0000","last_active":"2023-11-27T15:22:31.750943+0000","last_peered":"2023-11-27T15:22:31.750943+0000","last_clean":"2023-11-27T15:22:31.750943+0000","last_became_active":"2023-11-27T15:20:33.217003+0000","last_became_peered":"2023-11-27T15:20:33.217003+0000","last_unstale":"2023-11-27T15:22:31.750943+0000","last_undegraded":"2023-11-27T15:22:31.750943+0000","last_fullsized":"2023-11-27T15:22:31.750943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:51:22.192698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64726,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":619,"num_read_kb":376,"num_write":170,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":204936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416423,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6280,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":93738872,"statfs":{"total":95995035648,"available":95988604928,"internally_reserved":0,"allocated":1048576,"data_stored":514092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.55800000000000005}]},{"osd":1,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60599999999999998,"15min":0.60299999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89200000000000002,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60899999999999999,"15min":0.61299999999999999},"min":{"1min":0.35499999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76300000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.60499999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.30299999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59999999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.46100000000000002,"15min":0.45800000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.67200000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.50700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":4,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.86499999999999999,"5min":1.177,"15min":1.177},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61299999999999999,"15min":0.61599999999999999},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84899999999999998,"5min":1.141,"15min":1.141},"last":0.69999999999999996}]},{"osd":5,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52700000000000002,"15min":0.53100000000000003},"min":{"1min":0.34399999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69299999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88700000000000001,"5min":1.032,"15min":1.032},"last":0.66100000000000003}]},{"osd":6,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.47499999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.875,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85999999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.877}]},{"osd":7,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.46500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97899999999999998,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70999999999999996,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95699999999999996,"5min":1.54,"15min":1.54},"last":0.85999999999999999}]},{"osd":8,"last update":"Mon Nov 27 15:22:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68700000000000006,"15min":0.68600000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89500000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.90900000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.81599999999999995}]},{"osd":9,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71599999999999997,"15min":0.71799999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92500000000000004,"5min":1.286,"15min":1.286},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.76900000000000002,"15min":0.76500000000000001},"min":{"1min":0.498,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.84899999999999998}]},{"osd":10,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.53900000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96299999999999997,"5min":1.304,"15min":1.304},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73099999999999998,"15min":0.73299999999999998},"min":{"1min":0.53000000000000003,"5min":0.501,"15min":0.501},"max":{"1min":0.95199999999999996,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.91400000000000003}]}]},{"osd":10,"up_from":71,"seq":304942678058,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7332,"kb_used_data":1692,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93737820,"statfs":{"total":95995035648,"available":95987527680,"internally_reserved":0,"allocated":1732608,"data_stored":1166196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.55800000000000005,"15min":0.53400000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":4.5060000000000002,"5min":4.5060000000000002,"15min":4.5060000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.56699999999999995}]},{"osd":1,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46100000000000002,"15min":0.47399999999999998},"min":{"1min":0.33400000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.63600000000000001,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46100000000000002,"15min":0.46700000000000003},"min":{"1min":0.33500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42299999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.58399999999999996},"min":{"1min":0.374,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.48599999999999999}]},{"osd":3,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.60699999999999998,"15min":0.63500000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70799999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63700000000000001,"15min":0.66100000000000003},"min":{"1min":0.45500000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.73399999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.67300000000000004}]},{"osd":4,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56599999999999995,"15min":0.57999999999999996},"min":{"1min":0.40999999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80200000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.56899999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88700000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61299999999999999}]},{"osd":5,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63900000000000001,"15min":0.626},"min":{"1min":0.45600000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92700000000000005,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81499999999999995,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.57899999999999996}]},{"osd":6,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.67200000000000004,"15min":0.68999999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86299999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70599999999999996,"15min":0.72599999999999998},"min":{"1min":0.54600000000000004,"5min":0.496,"15min":0.496},"max":{"1min":0.85199999999999998,"5min":1.004,"15min":1.004},"last":0.75900000000000001}]},{"osd":7,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.56799999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.73699999999999999,"15min":0.76100000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.876,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.72799999999999998,"15min":0.73399999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97399999999999998,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69299999999999995}]},{"osd":9,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.76800000000000002,"15min":0.79000000000000004},"min":{"1min":0.58499999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.94199999999999995,"5min":1.111,"15min":1.111},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76000000000000001,"15min":0.77200000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94899999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.71699999999999997}]},{"osd":11,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.746,"15min":0.75},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.77000000000000002,"15min":0.77900000000000003},"min":{"1min":0.56999999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.92600000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.79100000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841582,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6480,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738672,"statfs":{"total":95995035648,"available":95988400128,"internally_reserved":0,"allocated":1056768,"data_stored":514216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.503,"15min":0.51900000000000002},"min":{"1min":0.33500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.59899999999999998,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.35899999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.71099999999999997,"15min":0.73099999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.99199999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.65100000000000002},"min":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.56499999999999995}]},{"osd":2,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.77000000000000002,"15min":0.79200000000000004},"min":{"1min":0.52800000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.751},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.74199999999999999,"15min":0.77200000000000002},"min":{"1min":0.48299999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.66100000000000003}]},{"osd":3,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.66900000000000004,"15min":0.70299999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.73299999999999998,"15min":0.78400000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.127,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.49399999999999999}]},{"osd":4,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.71699999999999997,"15min":0.749},"min":{"1min":0.47199999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.77300000000000002,"15min":0.80600000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.68300000000000005}]},{"osd":5,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.81699999999999995,"15min":0.84499999999999997},"min":{"1min":0.53500000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.84699999999999998,"15min":0.88400000000000001},"min":{"1min":0.55800000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.4470000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.78600000000000003}]},{"osd":6,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.80400000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.84799999999999998,"15min":0.89800000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.89900000000000002,"15min":0.94299999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.5089999999999999,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.58199999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.90700000000000003,"15min":0.96399999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8160000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.91600000000000004,"15min":0.96299999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.77600000000000002}]},{"osd":10,"last update":"Mon Nov 27 15:22:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.88800000000000001,"15min":0.92400000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.5469999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.626},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.83799999999999997,"15min":0.86499999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.71299999999999997}]},{"osd":11,"last update":"Mon Nov 27 15:22:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.82799999999999996,"15min":0.83899999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.84099999999999997,"15min":0.85199999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.80700000000000005}]}]},{"osd":8,"up_from":58,"seq":249108103220,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6596,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738556,"statfs":{"total":95995035648,"available":95988281344,"internally_reserved":0,"allocated":1044480,"data_stored":513693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.44400000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.59799999999999998},"min":{"1min":0.45700000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.75,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54400000000000004}]},{"osd":1,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.59999999999999998,"15min":0.59899999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.60599999999999998,"15min":0.59699999999999998},"min":{"1min":0.42699999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.69299999999999995}]},{"osd":2,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.44600000000000001,"15min":0.439},"min":{"1min":0.376,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.435},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.46600000000000003,"15min":0.44800000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.66600000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.47499999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.60599999999999998,"15min":0.60099999999999998},"min":{"1min":0.503,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.45300000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.57499999999999996}]},{"osd":4,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63,"15min":0.628},"min":{"1min":0.46700000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.84699999999999998,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86599999999999999,"5min":1.377,"15min":1.377},"last":0.67500000000000004}]},{"osd":5,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.55600000000000005},"min":{"1min":0.39100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60299999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.73799999999999999,"15min":0.68400000000000005},"min":{"1min":0.47599999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":6.4450000000000003,"5min":6.4450000000000003,"15min":6.4450000000000003},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.72299999999999998,"15min":0.70299999999999996},"min":{"1min":0.55200000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.76700000000000002}]},{"osd":7,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.73999999999999999,"15min":0.72199999999999998},"min":{"1min":0.53200000000000003,"5min":0.442,"15min":0.442},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.434,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.99299999999999999,"5min":1.323,"15min":1.323},"last":0.70499999999999996}]},{"osd":9,"last update":"Mon Nov 27 15:22:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.76100000000000001,"15min":0.73999999999999999},"min":{"1min":0.505,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.754},{"interface":"front","average":{"1min":1.181,"5min":0.86699999999999999,"15min":0.80400000000000005},"min":{"1min":0.62,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":6.4619999999999997,"5min":6.4619999999999997,"15min":6.4619999999999997},"last":0.79500000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:22:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.74099999999999999,"15min":0.71099999999999997},"min":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.77600000000000002,"15min":0.74099999999999999},"min":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.84399999999999997}]},{"osd":11,"last update":"Mon Nov 27 15:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84699999999999998,"15min":0.85099999999999998},"min":{"1min":0.66500000000000004,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.65200000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.73799999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332151,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7316,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93737836,"statfs":{"total":95995035648,"available":95987544064,"internally_reserved":0,"allocated":1650688,"data_stored":1110401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.436,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84699999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.44},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77100000000000002,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.46999999999999997}]},{"osd":1,"last update":"Mon Nov 27 15:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.41199999999999998,"15min":0.39500000000000002},"min":{"1min":0.34499999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64000000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.42899999999999999,"15min":0.40500000000000003},"min":{"1min":0.35299999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.495}]},{"osd":2,"last update":"Mon Nov 27 15:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.42199999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.752,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.57199999999999995},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.72399999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.48899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93899999999999995,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.63900000000000001},"min":{"1min":0.48099999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.60999999999999999}]},{"osd":4,"last update":"Mon Nov 27 15:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52500000000000002,"15min":0.498},"min":{"1min":0.48299999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.73499999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.438},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51600000000000001,"15min":0.52600000000000002},"min":{"1min":0.39200000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67100000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.628}]},{"osd":5,"last update":"Mon Nov 27 15:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.47499999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.79500000000000004,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.63600000000000001,"15min":0.623},"min":{"1min":0.54800000000000004,"5min":0.434,"15min":0.434},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.58999999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69899999999999995,"15min":0.67700000000000005},"min":{"1min":0.58599999999999997,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.49399999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.68899999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:21:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.66900000000000004,"15min":0.65100000000000002},"min":{"1min":0.60399999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.53100000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.73999999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:22:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.73399999999999999,"15min":0.72699999999999998},"min":{"1min":0.66100000000000003,"5min":0.442,"15min":0.442},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77300000000000002,"15min":0.77000000000000002},"min":{"1min":0.67300000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.751}]},{"osd":10,"last update":"Mon Nov 27 15:22:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.58499999999999996,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92100000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.68899999999999995,"15min":0.66800000000000004},"min":{"1min":0.59199999999999997,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90900000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.76200000000000001}]},{"osd":11,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76700000000000002,"15min":0.76800000000000002},"min":{"1min":0.66700000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.90600000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79900000000000004,"15min":0.79800000000000004},"min":{"1min":0.64800000000000002,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.93300000000000005,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.80700000000000005}]}]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6588,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738564,"statfs":{"total":95995035648,"available":95988289536,"internally_reserved":0,"allocated":1036288,"data_stored":509392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.49399999999999999,"15min":0.47599999999999998},"min":{"1min":0.38900000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.89600000000000002,"5min":1.044,"15min":1.044},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.54000000000000004,"15min":0.496},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.42299999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.72899999999999998,"15min":0.66000000000000003},"min":{"1min":0.52000000000000002,"5min":0.441,"15min":0.441},"max":{"1min":4.9109999999999996,"5min":4.9109999999999996,"15min":4.9109999999999996},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.629,"15min":0.58599999999999997},"min":{"1min":0.45500000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90500000000000003,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.56699999999999995}]},{"osd":2,"last update":"Mon Nov 27 15:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.69699999999999995,"15min":0.65900000000000003},"min":{"1min":0.53700000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.70899999999999996,"15min":0.68300000000000005},"min":{"1min":0.55100000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.137,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.76700000000000002}]},{"osd":3,"last update":"Mon Nov 27 15:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.58799999999999997,"15min":0.55400000000000005},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.64100000000000001,"15min":0.60499999999999998},"min":{"1min":0.42699999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.57599999999999996}]},{"osd":4,"last update":"Mon Nov 27 15:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70199999999999996,"15min":0.68799999999999994},"min":{"1min":0.51900000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0089999999999999,"5min":1.085,"15min":1.085},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69399999999999995,"15min":0.67200000000000004},"min":{"1min":0.55500000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1519999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.63200000000000001}]},{"osd":5,"last update":"Mon Nov 27 15:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.73499999999999999,"15min":0.71299999999999997},"min":{"1min":0.51100000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.179,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77900000000000003,"15min":0.75800000000000001},"min":{"1min":0.66400000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2989999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.79100000000000004}]},{"osd":7,"last update":"Mon Nov 27 15:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.77300000000000002,"15min":0.753},"min":{"1min":0.59499999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.76200000000000001,"15min":0.71999999999999997},"min":{"1min":0.55800000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.64300000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85799999999999998,"15min":0.86399999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.22,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.84799999999999998,"15min":0.85799999999999998},"min":{"1min":0.60699999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.74399999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.85399999999999998,"15min":0.86799999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.90400000000000003,"15min":0.92400000000000004},"min":{"1min":0.59999999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.2569999999999999,"5min":1.712,"15min":1.712},"last":0.84599999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:22:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.83099999999999996,"15min":0.82499999999999996},"min":{"1min":0.56499999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.90000000000000002,"15min":0.88800000000000001},"min":{"1min":0.72599999999999998,"5min":0.626,"15min":0.626},"max":{"1min":1.3999999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.81200000000000006}]},{"osd":11,"last update":"Mon Nov 27 15:21:38 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":4.9290000000000003,"5min":4.9290000000000003,"15min":4.9290000000000003},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.86599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7680,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":6592,"kb_avail":93737472,"statfs":{"total":95995035648,"available":95987171328,"internally_reserved":0,"allocated":1105920,"data_stored":575050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6750208},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.59099999999999997,"15min":0.51300000000000001},"min":{"1min":0.36699999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":5.2999999999999998,"5min":5.2999999999999998,"15min":5.2999999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.57999999999999996,"15min":0.504},"min":{"1min":0.46300000000000002,"5min":0.36699999999999999,"15min":0.33800000000000002},"max":{"1min":1.0029999999999999,"5min":1.012,"15min":1.012},"last":0.59499999999999997}]},{"osd":2,"last update":"Mon Nov 27 15:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.55500000000000005,"15min":0.498},"min":{"1min":0.38,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.752,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.61399999999999999,"15min":0.51600000000000001},"min":{"1min":0.42699999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.88700000000000001,"5min":0.999,"15min":0.999},"last":0.746}]},{"osd":3,"last update":"Mon Nov 27 15:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57199999999999995,"15min":0.53400000000000003},"min":{"1min":0.39700000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.39500000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.57199999999999995}]},{"osd":4,"last update":"Mon Nov 27 15:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.46600000000000003,"15min":0.40799999999999997},"min":{"1min":0.34399999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.73699999999999999,"5min":1.3,"15min":1.3},"last":0.874},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.47999999999999998,"15min":0.42199999999999999},"min":{"1min":0.32800000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.752,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.72299999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.60199999999999998,"15min":0.55600000000000005},"min":{"1min":0.48899999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78800000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.65700000000000003,"15min":0.58299999999999996},"min":{"1min":0.60999999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93899999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.81000000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.72099999999999997,"15min":0.64900000000000002},"min":{"1min":0.52400000000000002,"5min":0.442,"15min":0.442},"max":{"1min":5.3239999999999998,"5min":5.3239999999999998,"15min":5.3239999999999998},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66100000000000003,"15min":0.626},"min":{"1min":0.47999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.0289999999999999,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.84699999999999998}]},{"osd":7,"last update":"Mon Nov 27 15:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.62,"15min":0.57599999999999996},"min":{"1min":0.505,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0169999999999999,"5min":1.089,"15min":1.089},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.65000000000000002,"15min":0.60399999999999998},"min":{"1min":0.53200000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92100000000000004,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.91800000000000004}]},{"osd":8,"last update":"Mon Nov 27 15:22:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.70199999999999996,"15min":0.65600000000000003},"min":{"1min":0.629,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.68899999999999995,"15min":0.65300000000000002},"min":{"1min":0.55600000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91200000000000003,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.83599999999999997}]},{"osd":9,"last update":"Mon Nov 27 15:22:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":0.81799999999999995,"15min":0.76100000000000001},"min":{"1min":0.67500000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":5.3810000000000002,"5min":5.3810000000000002,"15min":5.3810000000000002},"last":0.995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.78000000000000003,"15min":0.76200000000000001},"min":{"1min":0.51600000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.95199999999999996}]},{"osd":10,"last update":"Mon Nov 27 15:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69799999999999995,"15min":0.68899999999999995},"min":{"1min":0.502,"5min":0.439,"15min":0.439},"max":{"1min":0.93400000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.79200000000000004,"15min":0.77400000000000002},"min":{"1min":0.624,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0429999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.93799999999999994}]},{"osd":11,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.79700000000000004,"15min":0.79300000000000004},"min":{"1min":0.65300000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.97699999999999998,"5min":1.024,"15min":1.024},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.88500000000000001,"15min":0.88400000000000001},"min":{"1min":0.73399999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.04,"5min":1.155,"15min":1.155},"last":0.96899999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7508,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737644,"statfs":{"total":95995035648,"available":95987347456,"internally_reserved":0,"allocated":1060864,"data_stored":510780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 27 15:22:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60099999999999998,"15min":0.52000000000000002},"min":{"1min":0.35199999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.63400000000000001,"15min":0.54000000000000004},"min":{"1min":0.40200000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.59699999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:22:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64100000000000001,"15min":0.58699999999999997},"min":{"1min":0.38400000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.95199999999999996,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.66000000000000003,"15min":0.57699999999999996},"min":{"1min":0.53100000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.79300000000000004}]},{"osd":3,"last update":"Mon Nov 27 15:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63,"15min":0.63},"min":{"1min":0.44600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.04,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91400000000000003,"5min":1.204,"15min":1.204},"last":0.54000000000000004}]},{"osd":4,"last update":"Mon Nov 27 15:22:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63900000000000001,"15min":0.61599999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68000000000000005,"15min":0.61699999999999999},"min":{"1min":0.51800000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.996,"5min":1.244,"15min":1.244},"last":0.65700000000000003}]},{"osd":5,"last update":"Mon Nov 27 15:22:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.432,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71999999999999997,"15min":0.68600000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.034,"5min":1.379,"15min":1.379},"last":0.78500000000000003}]},{"osd":6,"last update":"Mon Nov 27 15:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.68200000000000005,"15min":0.63600000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64300000000000002,"15min":0.62},"min":{"1min":0.39600000000000002,"5min":0.373,"15min":0.373},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.67300000000000004}]},{"osd":7,"last update":"Mon Nov 27 15:22:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.76800000000000002,"15min":0.75900000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.62},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.78900000000000003,"15min":0.78700000000000003},"min":{"1min":0.64500000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0880000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.81999999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.755,"15min":0.754},"min":{"1min":0.59799999999999998,"5min":0.498,"15min":0.498},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76500000000000001,"15min":0.78100000000000003},"min":{"1min":0.54400000000000004,"5min":0.504,"15min":0.504},"max":{"1min":0.94099999999999995,"5min":1.327,"15min":1.327},"last":0.874}]},{"osd":9,"last update":"Mon Nov 27 15:22:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.86699999999999999,"15min":0.878},"min":{"1min":0.65300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.0920000000000001,"5min":1.218,"15min":1.218},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.85199999999999998,"15min":0.85899999999999999},"min":{"1min":0.63600000000000001,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.86499999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:22:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.81699999999999995,"15min":0.82899999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.056,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88800000000000001,"15min":0.89200000000000002},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1599999999999999,"5min":1.409,"15min":1.409},"last":0.88}]},{"osd":11,"last update":"Mon Nov 27 15:21:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.83199999999999996,"15min":0.82699999999999996},"min":{"1min":0.67300000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.82799999999999996,"15min":0.83499999999999996},"min":{"1min":0.59299999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.96499999999999997,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.83399999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7764,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":6656,"kb_avail":93737388,"statfs":{"total":95995035648,"available":95987085312,"internally_reserved":0,"allocated":1126400,"data_stored":575995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6815744},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54400000000000004,"15min":0.51000000000000001},"min":{"1min":0.42699999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.42399999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.72499999999999998,"5min":0.83799999999999997,"15min":0.85199999999999998},"last":0.623}]},{"osd":1,"last update":"Mon Nov 27 15:22:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.58899999999999997,"15min":0.51100000000000001},"min":{"1min":0.52300000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.84999999999999998,"5min":1.079,"15min":1.079},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.58899999999999997,"15min":0.51700000000000002},"min":{"1min":0.48699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79100000000000004,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.76000000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.60699999999999998,"15min":0.57099999999999995},"min":{"1min":0.48999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.59499999999999997,"15min":0.56899999999999995},"min":{"1min":0.44500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.84099999999999997}]},{"osd":4,"last update":"Mon Nov 27 15:22:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.52100000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66800000000000004,"15min":0.67400000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84099999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.71299999999999997}]},{"osd":5,"last update":"Mon Nov 27 15:22:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.47999999999999998,"15min":0.438},"min":{"1min":0.35899999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80400000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.50800000000000001,"15min":0.46400000000000002},"min":{"1min":0.40300000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77600000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.59299999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:22:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.64100000000000001,"15min":0.61299999999999999},"min":{"1min":0.48599999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.82899999999999996,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.64100000000000001,"15min":0.60999999999999999},"min":{"1min":0.439,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93700000000000006,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.80400000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:22:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71099999999999997,"15min":0.71899999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80700000000000005,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.55500000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.88600000000000001}]},{"osd":8,"last update":"Mon Nov 27 15:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.376,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85599999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.61599999999999999,"15min":0.60499999999999998},"min":{"1min":0.47199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.88100000000000001,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.82799999999999996}]},{"osd":9,"last update":"Mon Nov 27 15:22:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.70399999999999996,"15min":0.67000000000000004},"min":{"1min":0.60399999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.871},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.73199999999999998,"15min":0.70099999999999996},"min":{"1min":0.56200000000000006,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.91800000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:22:33 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.75600000000000001,"15min":0.75},"min":{"1min":0.57499999999999996,"5min":0.44,"15min":0.44},"max":{"1min":0.89500000000000002,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.746,"15min":0.69799999999999995},"min":{"1min":0.63,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.038,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.94299999999999995}]},{"osd":11,"last update":"Mon Nov 27 15:22:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.84599999999999997,"15min":0.85199999999999998},"min":{"1min":0.504,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97199999999999998,"5min":1.034,"15min":1.034},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.80700000000000005,"15min":0.80900000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.93200000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.90800000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6920,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":5888,"kb_avail":93738232,"statfs":{"total":95995035648,"available":95987949568,"internally_reserved":0,"allocated":1048576,"data_stored":509586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6029312},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.443,"15min":0.40600000000000003},"min":{"1min":0.34899999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.375},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47199999999999998,"15min":0.42199999999999999},"min":{"1min":0.36199999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79400000000000004,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.78200000000000003}]},{"osd":1,"last update":"Mon Nov 27 15:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63800000000000001,"15min":0.57399999999999995},"min":{"1min":0.54500000000000004,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.878,"5min":1.056,"15min":1.056},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.67700000000000005,"15min":0.58499999999999996},"min":{"1min":0.502,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.907,"5min":5.6529999999999996,"15min":5.6529999999999996},"last":0.436}]},{"osd":2,"last update":"Mon Nov 27 15:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.66500000000000004,"15min":0.60199999999999998},"min":{"1min":0.52300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81799999999999995,"5min":1.208,"15min":1.208},"last":0.745},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65200000000000002,"15min":0.59499999999999997},"min":{"1min":0.46000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88700000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.72199999999999998}]},{"osd":4,"last update":"Mon Nov 27 15:22:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.69099999999999995,"15min":0.64300000000000002},"min":{"1min":0.52100000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":2.8730000000000002,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72299999999999998,"15min":0.67800000000000005},"min":{"1min":0.55000000000000004,"5min":0.502,"15min":0.502},"max":{"1min":0.83199999999999996,"5min":1.343,"15min":1.343},"last":0.59699999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70399999999999996,"15min":0.66200000000000003},"min":{"1min":0.61699999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.90800000000000003,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.69299999999999995,"15min":0.66100000000000003},"min":{"1min":0.58299999999999996,"5min":0.437,"15min":0.437},"max":{"1min":0.89700000000000002,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.68200000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.67900000000000005,"15min":0.64600000000000002},"min":{"1min":0.54600000000000004,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.63600000000000001,"15min":0.60799999999999998},"min":{"1min":0.443,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.84499999999999997}]},{"osd":7,"last update":"Mon Nov 27 15:22:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.93600000000000005,"15min":0.96999999999999997},"min":{"1min":0.63100000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.7959999999999998,"5min":6.9619999999999997,"15min":6.9619999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.53400000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0940000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.56399999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.873},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":0.81499999999999995}]},{"osd":9,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.83899999999999997,"15min":0.85399999999999998},"min":{"1min":0.64800000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83399999999999996,"15min":0.84999999999999998},"min":{"1min":0.56799999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1910000000000001,"5min":1.496,"15min":1.496},"last":0.82699999999999996}]},{"osd":10,"last update":"Mon Nov 27 15:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.80600000000000005,"15min":0.79200000000000004},"min":{"1min":0.59599999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.839,"5min":2.839,"15min":2.839},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.83299999999999996,"15min":0.83599999999999997},"min":{"1min":0.64000000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0209999999999999,"5min":1.129,"15min":1.129},"last":0.83599999999999997}]},{"osd":11,"last update":"Mon Nov 27 15:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83099999999999996,"15min":0.81999999999999995},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.84899999999999998,"15min":0.84599999999999997},"min":{"1min":0.72899999999999998,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.79100000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6792,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738360,"statfs":{"total":95995035648,"available":95988080640,"internally_reserved":0,"allocated":1048576,"data_stored":514525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.54400000000000004,"15min":0.53700000000000003},"min":{"1min":0.38300000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53500000000000003,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77900000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.54900000000000004}]},{"osd":1,"last update":"Mon Nov 27 15:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.40200000000000002,"15min":0.35799999999999998},"min":{"1min":0.26000000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.54700000000000004,"5min":1.337,"15min":1.337},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.371},"min":{"1min":0.28199999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.57399999999999995,"5min":1.476,"15min":1.476},"last":0.36199999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57399999999999995,"15min":0.55600000000000005},"min":{"1min":0.38100000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83599999999999997,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60399999999999998,"15min":0.55400000000000005},"min":{"1min":0.39300000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.68999999999999995,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.76100000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59399999999999997,"15min":0.55700000000000005},"min":{"1min":0.42199999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0109999999999999,"5min":1.423,"15min":1.423},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.59899999999999998},"min":{"1min":0.42099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.99199999999999999,"5min":1.722,"15min":1.722},"last":0.48099999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.63400000000000001,"15min":0.61099999999999999},"min":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.87,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64100000000000001,"15min":0.60999999999999999},"min":{"1min":0.433,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.94299999999999995,"5min":1.28,"15min":1.28},"last":0.68400000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:22:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88,"5min":1.393,"15min":1.393},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67400000000000004,"15min":0.65900000000000003},"min":{"1min":0.49199999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.879,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.67200000000000004}]},{"osd":7,"last update":"Mon Nov 27 15:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58999999999999997,"15min":0.56799999999999995},"min":{"1min":0.33900000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79900000000000004,"5min":1.054,"15min":1.054},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61599999999999999,"15min":0.59699999999999998},"min":{"1min":0.36099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.98099999999999998,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.621}]},{"osd":8,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71399999999999997,"15min":0.70499999999999996},"min":{"1min":0.50600000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.99099999999999999,"5min":1.522,"15min":1.522},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92700000000000005,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.71299999999999997}]},{"osd":9,"last update":"Mon Nov 27 15:22:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.79500000000000004,"15min":0.76700000000000002},"min":{"1min":0.55900000000000005,"5min":0.44,"15min":0.44},"max":{"1min":2.8969999999999998,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.52100000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.54000000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73399999999999999,"15min":0.72399999999999998},"min":{"1min":0.53800000000000003,"5min":0.503,"15min":0.503},"max":{"1min":0.91000000000000003,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91800000000000004,"5min":1.03,"15min":1.03},"last":0.65500000000000003}]},{"osd":11,"last update":"Mon Nov 27 15:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.81100000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.96899999999999997,"5min":1.234,"15min":1.234},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72399999999999998,"15min":0.72499999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.97499999999999998,"5min":1.2,"15min":1.2},"last":0.72499999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7448,"kb_used_data":1616,"kb_used_omap":0,"kb_used_meta":5824,"kb_avail":93737704,"statfs":{"total":95995035648,"available":95987408896,"internally_reserved":0,"allocated":1654784,"data_stored":1105175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5963776},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.36199999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.46000000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57799999999999996,"15min":0.54300000000000004},"min":{"1min":0.439,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.67100000000000004}]},{"osd":2,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.35199999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.621,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.36799999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48499999999999999}]},{"osd":3,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54900000000000004,"15min":0.55900000000000005},"min":{"1min":0.39600000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79200000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.443},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57999999999999996,"15min":0.57899999999999996},"min":{"1min":0.44600000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.81699999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.502}]},{"osd":4,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.622,"15min":0.59799999999999998},"min":{"1min":0.48699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.60099999999999998,"15min":0.58499999999999996},"min":{"1min":0.45500000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.58899999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.621,"15min":0.63300000000000001},"min":{"1min":0.48099999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.89900000000000002,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.54500000000000004,"5min":0.443,"15min":0.443},"max":{"1min":0.82999999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.57799999999999996}]},{"osd":7,"last update":"Mon Nov 27 15:22:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63400000000000001,"15min":0.64500000000000002},"min":{"1min":0.503,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.76000000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64300000000000002,"15min":0.64000000000000001},"min":{"1min":0.57099999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.76300000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.70399999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:22:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.48799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.47699999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.54400000000000004}]},{"osd":9,"last update":"Mon Nov 27 15:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71699999999999997,"15min":0.74299999999999999},"min":{"1min":0.51900000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86499999999999999,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.58799999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.82999999999999996,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.71799999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:22:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.68799999999999994,"15min":0.68000000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70799999999999996,"15min":0.69099999999999995},"min":{"1min":0.54700000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.73599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-27T15:22:46.784 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-11-27T15:22:46.784 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-27T15:22:47.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.014+0000 7f4f54dff700 1 -- 172.21.15.133:0/1229103489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f5010c880 msgr2=0x7f4f5010ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:47.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.014+0000 7f4f54dff700 1 --2- 172.21.15.133:0/1229103489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f5010c880 0x7f4f5010ec60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800b0a0 tx=0x7f4f4800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.014+0000 7f4f54dff700 1 -- 172.21.15.133:0/1229103489 shutdown_connections 2023-11-27T15:22:47.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.014+0000 7f4f54dff700 1 --2- 172.21.15.133:0/1229103489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f5010c880 0x7f4f5010ec60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.014+0000 7f4f54dff700 1 --2- 172.21.15.133:0/1229103489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f50074b30 0x7f4f50072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.014+0000 7f4f54dff700 1 --2- 172.21.15.133:0/1229103489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f50073550 0x7f4f500739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.014+0000 7f4f54dff700 1 -- 172.21.15.133:0/1229103489 >> 172.21.15.133:0/1229103489 conn(0x7f4f500fb890 msgr2=0x7f4f500fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:47.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.014+0000 7f4f54dff700 1 -- 172.21.15.133:0/1229103489 shutdown_connections 2023-11-27T15:22:47.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.014+0000 7f4f54dff700 1 -- 172.21.15.133:0/1229103489 wait complete. 2023-11-27T15:22:47.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.015+0000 7f4f54dff700 1 Processor -- start 2023-11-27T15:22:47.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.015+0000 7f4f54dff700 1 -- start start 2023-11-27T15:22:47.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.016+0000 7f4f54dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f50073550 0x7f4f5019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:47.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.016+0000 7f4f54dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f50074b30 0x7f4f5019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:47.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.016+0000 7f4f54dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f5010c880 0x7f4f501a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:47.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.016+0000 7f4f54dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f501a4240 con 0x7f4f5010c880 2023-11-27T15:22:47.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.016+0000 7f4f54dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f501a4380 con 0x7f4f50073550 2023-11-27T15:22:47.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.016+0000 7f4f54dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f501a4510 con 0x7f4f50074b30 2023-11-27T15:22:47.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.016+0000 7f4f4ed9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f5010c880 0x7f4f501a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:47.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.016+0000 7f4f4ed9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f5010c880 0x7f4f501a3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60454/0 (socket says 172.21.15.133:60454) 2023-11-27T15:22:47.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.016+0000 7f4f4ed9d700 1 -- 172.21.15.133:0/2707614519 learned_addr learned my addr 172.21.15.133:0/2707614519 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:47.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.016+0000 7f4f47fff700 1 --2- 172.21.15.133:0/2707614519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f50074b30 0x7f4f5019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:47.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.016+0000 7f4f4e59c700 1 --2- 172.21.15.133:0/2707614519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f50073550 0x7f4f5019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:47.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.016+0000 7f4f4ed9d700 1 -- 172.21.15.133:0/2707614519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f50074b30 msgr2=0x7f4f5019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:47.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.016+0000 7f4f4ed9d700 1 --2- 172.21.15.133:0/2707614519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f50074b30 0x7f4f5019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.016+0000 7f4f4ed9d700 1 -- 172.21.15.133:0/2707614519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f50073550 msgr2=0x7f4f5019c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:47.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.016+0000 7f4f4ed9d700 1 --2- 172.21.15.133:0/2707614519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f50073550 0x7f4f5019c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.016+0000 7f4f4ed9d700 1 -- 172.21.15.133:0/2707614519 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f48009d00 con 0x7f4f5010c880 2023-11-27T15:22:47.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.016+0000 7f4f4e59c700 1 --2- 172.21.15.133:0/2707614519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f50073550 0x7f4f5019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:22:47.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.016+0000 7f4f4ed9d700 1 --2- 172.21.15.133:0/2707614519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f5010c880 0x7f4f501a3c70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009fd0 tx=0x7f4f4800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:47.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.017+0000 7f4f477fe700 1 -- 172.21.15.133:0/2707614519 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4800e040 con 0x7f4f5010c880 2023-11-27T15:22:47.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.017+0000 7f4f477fe700 1 -- 172.21.15.133:0/2707614519 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f480094a0 con 0x7f4f5010c880 2023-11-27T15:22:47.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.017+0000 7f4f477fe700 1 -- 172.21.15.133:0/2707614519 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f4801b6a0 con 0x7f4f5010c880 2023-11-27T15:22:47.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.017+0000 7f4f54dff700 1 -- 172.21.15.133:0/2707614519 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f50112290 con 0x7f4f5010c880 2023-11-27T15:22:47.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.017+0000 7f4f54dff700 1 -- 172.21.15.133:0/2707614519 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f50112100 con 0x7f4f5010c880 2023-11-27T15:22:47.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.018+0000 7f4f54dff700 1 -- 172.21.15.133:0/2707614519 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f501023d0 con 0x7f4f5010c880 2023-11-27T15:22:47.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.020+0000 7f4f477fe700 1 -- 172.21.15.133:0/2707614519 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4f4801d470 con 0x7f4f5010c880 2023-11-27T15:22:47.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.021+0000 7f4f477fe700 1 --2- 172.21.15.133:0/2707614519 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4f3c0a1150 0x7f4f3c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:47.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.021+0000 7f4f477fe700 1 -- 172.21.15.133:0/2707614519 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f4f480ce060 con 0x7f4f5010c880 2023-11-27T15:22:47.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.021+0000 7f4f4e59c700 1 --2- 172.21.15.133:0/2707614519 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4f3c0a1150 0x7f4f3c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:47.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.022+0000 7f4f4e59c700 1 --2- 172.21.15.133:0/2707614519 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4f3c0a1150 0x7f4f3c0a3600 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4f50074f00 tx=0x7f4f40009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:47.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:46 smithi133 ceph-mon[97860]: pgmap v344: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 7 op/s 2023-11-27T15:22:47.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:46 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/293352405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:47.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.025+0000 7f4f477fe700 1 -- 172.21.15.133:0/2707614519 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4f48097490 con 0x7f4f5010c880 2023-11-27T15:22:47.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:46 smithi177 ceph-mon[131504]: pgmap v344: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 7 op/s 2023-11-27T15:22:47.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:46 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/293352405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:47.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:46 smithi174 ceph-mon[131110]: pgmap v344: 129 pgs: 129 active+clean; 640 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 7 op/s 2023-11-27T15:22:47.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:46 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/293352405' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:47.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.220+0000 7f4f54dff700 1 -- 172.21.15.133:0/2707614519 --> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4f50061a30 con 0x7f4f3c0a1150 2023-11-27T15:22:47.235 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.232+0000 7f4f477fe700 1 -- 172.21.15.133:0/2707614519 <== mgr.14150 v2:172.21.15.133:6800/4209056668 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427427 (secure 0 0 0) 0x7f4f50061a30 con 0x7f4f3c0a1150 2023-11-27T15:22:47.241 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:47.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.237+0000 7f4f54dff700 1 -- 172.21.15.133:0/2707614519 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4f3c0a1150 msgr2=0x7f4f3c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:47.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.237+0000 7f4f54dff700 1 --2- 172.21.15.133:0/2707614519 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4f3c0a1150 0x7f4f3c0a3600 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4f50074f00 tx=0x7f4f40009260 comp rx=0 tx=0).stop 2023-11-27T15:22:47.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.237+0000 7f4f54dff700 1 -- 172.21.15.133:0/2707614519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f5010c880 msgr2=0x7f4f501a3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:47.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.237+0000 7f4f54dff700 1 --2- 172.21.15.133:0/2707614519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f5010c880 0x7f4f501a3c70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009fd0 tx=0x7f4f4800bce0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.238+0000 7f4f54dff700 1 -- 172.21.15.133:0/2707614519 shutdown_connections 2023-11-27T15:22:47.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.238+0000 7f4f54dff700 1 --2- 172.21.15.133:0/2707614519 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4f3c0a1150 0x7f4f3c0a3600 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.238+0000 7f4f54dff700 1 --2- 172.21.15.133:0/2707614519 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f5010c880 0x7f4f501a3c70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.238+0000 7f4f54dff700 1 --2- 172.21.15.133:0/2707614519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f50073550 0x7f4f5019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.238+0000 7f4f54dff700 1 --2- 172.21.15.133:0/2707614519 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4f50074b30 0x7f4f5019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.238+0000 7f4f54dff700 1 -- 172.21.15.133:0/2707614519 >> 172.21.15.133:0/2707614519 conn(0x7f4f500fb890 msgr2=0x7f4f500fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:47.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.238+0000 7f4f54dff700 1 -- 172.21.15.133:0/2707614519 shutdown_connections 2023-11-27T15:22:47.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.238+0000 7f4f54dff700 1 -- 172.21.15.133:0/2707614519 wait complete. 2023-11-27T15:22:47.246 INFO:teuthology.orchestra.run.smithi133.stderr:dumped all 2023-11-27T15:22:47.261 INFO:teuthology.orchestra.run.smithi133.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2023-11-27T15:22:45.744559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655340,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":745,"num_read_kb":485,"num_write":353,"num_write_kb":3061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180848,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":787,"ondisk_log_size":787,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":84704,"kb_used_data":14272,"kb_used_omap":0,"kb_used_meta":70336,"kb_avail":1124857120,"statfs":{"total":1151940427776,"available":1151853690880,"internally_reserved":0,"allocated":14614528,"data_stored":8119101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":72024064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1014,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":34,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003000"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.767519+0000","last_change":"2023-11-27T15:20:33.236352+0000","last_active":"2023-11-27T15:22:31.767519+0000","last_peered":"2023-11-27T15:22:31.767519+0000","last_clean":"2023-11-27T15:22:31.767519+0000","last_became_active":"2023-11-27T15:20:33.235974+0000","last_became_peered":"2023-11-27T15:20:33.235974+0000","last_unstale":"2023-11-27T15:22:31.767519+0000","last_undegraded":"2023-11-27T15:22:31.767519+0000","last_fullsized":"2023-11-27T15:22:31.767519+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:32:53.999483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.764610+0000","last_change":"2023-11-27T15:20:33.234882+0000","last_active":"2023-11-27T15:22:31.764610+0000","last_peered":"2023-11-27T15:22:31.764610+0000","last_clean":"2023-11-27T15:22:31.764610+0000","last_became_active":"2023-11-27T15:20:33.231062+0000","last_became_peered":"2023-11-27T15:20:33.231062+0000","last_unstale":"2023-11-27T15:22:31.764610+0000","last_undegraded":"2023-11-27T15:22:31.764610+0000","last_fullsized":"2023-11-27T15:22:31.764610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:54:38.782631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.765932+0000","last_change":"2023-11-27T15:20:33.238280+0000","last_active":"2023-11-27T15:22:31.765932+0000","last_peered":"2023-11-27T15:22:31.765932+0000","last_clean":"2023-11-27T15:22:31.765932+0000","last_became_active":"2023-11-27T15:20:33.237602+0000","last_became_peered":"2023-11-27T15:20:33.237602+0000","last_unstale":"2023-11-27T15:22:31.765932+0000","last_undegraded":"2023-11-27T15:22:31.765932+0000","last_fullsized":"2023-11-27T15:22:31.765932+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:12:10.576246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.744091+0000","last_change":"2023-11-27T15:20:33.217909+0000","last_active":"2023-11-27T15:22:31.744091+0000","last_peered":"2023-11-27T15:22:31.744091+0000","last_clean":"2023-11-27T15:22:31.744091+0000","last_became_active":"2023-11-27T15:20:33.217511+0000","last_became_peered":"2023-11-27T15:20:33.217511+0000","last_unstale":"2023-11-27T15:22:31.744091+0000","last_undegraded":"2023-11-27T15:22:31.744091+0000","last_fullsized":"2023-11-27T15:22:31.744091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:11:52.036690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.750946+0000","last_change":"2023-11-27T15:20:33.246630+0000","last_active":"2023-11-27T15:22:31.750946+0000","last_peered":"2023-11-27T15:22:31.750946+0000","last_clean":"2023-11-27T15:22:31.750946+0000","last_became_active":"2023-11-27T15:20:33.246249+0000","last_became_peered":"2023-11-27T15:20:33.246249+0000","last_unstale":"2023-11-27T15:22:31.750946+0000","last_undegraded":"2023-11-27T15:22:31.750946+0000","last_fullsized":"2023-11-27T15:22:31.750946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:07:50.677388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.749736+0000","last_change":"2023-11-27T15:20:33.236280+0000","last_active":"2023-11-27T15:22:31.749736+0000","last_peered":"2023-11-27T15:22:31.749736+0000","last_clean":"2023-11-27T15:22:31.749736+0000","last_became_active":"2023-11-27T15:20:33.235983+0000","last_became_peered":"2023-11-27T15:20:33.235983+0000","last_unstale":"2023-11-27T15:22:31.749736+0000","last_undegraded":"2023-11-27T15:22:31.749736+0000","last_fullsized":"2023-11-27T15:22:31.749736+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:38:38.466940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.749774+0000","last_change":"2023-11-27T15:20:33.223085+0000","last_active":"2023-11-27T15:22:31.749774+0000","last_peered":"2023-11-27T15:22:31.749774+0000","last_clean":"2023-11-27T15:22:31.749774+0000","last_became_active":"2023-11-27T15:20:33.222809+0000","last_became_peered":"2023-11-27T15:20:33.222809+0000","last_unstale":"2023-11-27T15:22:31.749774+0000","last_undegraded":"2023-11-27T15:22:31.749774+0000","last_fullsized":"2023-11-27T15:22:31.749774+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:23:19.671424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.760153+0000","last_change":"2023-11-27T15:20:33.226217+0000","last_active":"2023-11-27T15:22:31.760153+0000","last_peered":"2023-11-27T15:22:31.760153+0000","last_clean":"2023-11-27T15:22:31.760153+0000","last_became_active":"2023-11-27T15:20:33.225856+0000","last_became_peered":"2023-11-27T15:20:33.225856+0000","last_unstale":"2023-11-27T15:22:31.760153+0000","last_undegraded":"2023-11-27T15:22:31.760153+0000","last_fullsized":"2023-11-27T15:22:31.760153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:59:41.033434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.752180+0000","last_change":"2023-11-27T15:20:33.227554+0000","last_active":"2023-11-27T15:22:31.752180+0000","last_peered":"2023-11-27T15:22:31.752180+0000","last_clean":"2023-11-27T15:22:31.752180+0000","last_became_active":"2023-11-27T15:20:33.227222+0000","last_became_peered":"2023-11-27T15:20:33.227222+0000","last_unstale":"2023-11-27T15:22:31.752180+0000","last_undegraded":"2023-11-27T15:22:31.752180+0000","last_fullsized":"2023-11-27T15:22:31.752180+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:45:28.965465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.751136+0000","last_change":"2023-11-27T15:20:33.230528+0000","last_active":"2023-11-27T15:22:31.751136+0000","last_peered":"2023-11-27T15:22:31.751136+0000","last_clean":"2023-11-27T15:22:31.751136+0000","last_became_active":"2023-11-27T15:20:33.230148+0000","last_became_peered":"2023-11-27T15:20:33.230148+0000","last_unstale":"2023-11-27T15:22:31.751136+0000","last_undegraded":"2023-11-27T15:22:31.751136+0000","last_fullsized":"2023-11-27T15:22:31.751136+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:02:27.225754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.750528+0000","last_change":"2023-11-27T15:20:33.228098+0000","last_active":"2023-11-27T15:22:31.750528+0000","last_peered":"2023-11-27T15:22:31.750528+0000","last_clean":"2023-11-27T15:22:31.750528+0000","last_became_active":"2023-11-27T15:20:33.227760+0000","last_became_peered":"2023-11-27T15:20:33.227760+0000","last_unstale":"2023-11-27T15:22:31.750528+0000","last_undegraded":"2023-11-27T15:22:31.750528+0000","last_fullsized":"2023-11-27T15:22:31.750528+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:08:49.344313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.756176+0000","last_change":"2023-11-27T15:21:02.934583+0000","last_active":"2023-11-27T15:22:31.756176+0000","last_peered":"2023-11-27T15:22:31.756176+0000","last_clean":"2023-11-27T15:22:31.756176+0000","last_became_active":"2023-11-27T15:21:02.933631+0000","last_became_peered":"2023-11-27T15:21:02.933631+0000","last_unstale":"2023-11-27T15:22:31.756176+0000","last_undegraded":"2023-11-27T15:22:31.756176+0000","last_fullsized":"2023-11-27T15:22:31.756176+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:22:12.200623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.754054+0000","last_change":"2023-11-27T15:20:33.231255+0000","last_active":"2023-11-27T15:22:31.754054+0000","last_peered":"2023-11-27T15:22:31.754054+0000","last_clean":"2023-11-27T15:22:31.754054+0000","last_became_active":"2023-11-27T15:20:33.230867+0000","last_became_peered":"2023-11-27T15:20:33.230867+0000","last_unstale":"2023-11-27T15:22:31.754054+0000","last_undegraded":"2023-11-27T15:22:31.754054+0000","last_fullsized":"2023-11-27T15:22:31.754054+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:33:48.764065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.767126+0000","last_change":"2023-11-27T15:20:33.230306+0000","last_active":"2023-11-27T15:22:31.767126+0000","last_peered":"2023-11-27T15:22:31.767126+0000","last_clean":"2023-11-27T15:22:31.767126+0000","last_became_active":"2023-11-27T15:20:33.229892+0000","last_became_peered":"2023-11-27T15:20:33.229892+0000","last_unstale":"2023-11-27T15:22:31.767126+0000","last_undegraded":"2023-11-27T15:22:31.767126+0000","last_fullsized":"2023-11-27T15:22:31.767126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:44:51.431718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.761884+0000","last_change":"2023-11-27T15:20:33.220592+0000","last_active":"2023-11-27T15:22:31.761884+0000","last_peered":"2023-11-27T15:22:31.761884+0000","last_clean":"2023-11-27T15:22:31.761884+0000","last_became_active":"2023-11-27T15:20:33.220258+0000","last_became_peered":"2023-11-27T15:20:33.220258+0000","last_unstale":"2023-11-27T15:22:31.761884+0000","last_undegraded":"2023-11-27T15:22:31.761884+0000","last_fullsized":"2023-11-27T15:22:31.761884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:16:39.685631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.767650+0000","last_change":"2023-11-27T15:20:33.245041+0000","last_active":"2023-11-27T15:22:31.767650+0000","last_peered":"2023-11-27T15:22:31.767650+0000","last_clean":"2023-11-27T15:22:31.767650+0000","last_became_active":"2023-11-27T15:20:33.244759+0000","last_became_peered":"2023-11-27T15:20:33.244759+0000","last_unstale":"2023-11-27T15:22:31.767650+0000","last_undegraded":"2023-11-27T15:22:31.767650+0000","last_fullsized":"2023-11-27T15:22:31.767650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:19:46.046097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.767690+0000","last_change":"2023-11-27T15:20:33.243288+0000","last_active":"2023-11-27T15:22:31.767690+0000","last_peered":"2023-11-27T15:22:31.767690+0000","last_clean":"2023-11-27T15:22:31.767690+0000","last_became_active":"2023-11-27T15:20:33.242989+0000","last_became_peered":"2023-11-27T15:20:33.242989+0000","last_unstale":"2023-11-27T15:22:31.767690+0000","last_undegraded":"2023-11-27T15:22:31.767690+0000","last_fullsized":"2023-11-27T15:22:31.767690+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:25:35.538012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.760176+0000","last_change":"2023-11-27T15:20:33.229613+0000","last_active":"2023-11-27T15:22:31.760176+0000","last_peered":"2023-11-27T15:22:31.760176+0000","last_clean":"2023-11-27T15:22:31.760176+0000","last_became_active":"2023-11-27T15:20:33.227747+0000","last_became_peered":"2023-11-27T15:20:33.227747+0000","last_unstale":"2023-11-27T15:22:31.760176+0000","last_undegraded":"2023-11-27T15:22:31.760176+0000","last_fullsized":"2023-11-27T15:22:31.760176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:57:48.962905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.760827+0000","last_change":"2023-11-27T15:20:33.227004+0000","last_active":"2023-11-27T15:22:31.760827+0000","last_peered":"2023-11-27T15:22:31.760827+0000","last_clean":"2023-11-27T15:22:31.760827+0000","last_became_active":"2023-11-27T15:20:33.226703+0000","last_became_peered":"2023-11-27T15:20:33.226703+0000","last_unstale":"2023-11-27T15:22:31.760827+0000","last_undegraded":"2023-11-27T15:22:31.760827+0000","last_fullsized":"2023-11-27T15:22:31.760827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:16:57.764198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818292+0000","last_change":"2023-11-27T15:20:37.682021+0000","last_active":"2023-11-27T15:22:02.818292+0000","last_peered":"2023-11-27T15:22:02.818292+0000","last_clean":"2023-11-27T15:22:02.818292+0000","last_became_active":"2023-11-27T15:20:37.681600+0000","last_became_peered":"2023-11-27T15:20:37.681600+0000","last_unstale":"2023-11-27T15:22:02.818292+0000","last_undegraded":"2023-11-27T15:22:02.818292+0000","last_fullsized":"2023-11-27T15:22:02.818292+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:16:02.041857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811637+0000","last_change":"2023-11-27T15:20:32.657542+0000","last_active":"2023-11-27T15:22:02.811637+0000","last_peered":"2023-11-27T15:22:02.811637+0000","last_clean":"2023-11-27T15:22:02.811637+0000","last_became_active":"2023-11-27T15:20:32.657158+0000","last_became_peered":"2023-11-27T15:20:32.657158+0000","last_unstale":"2023-11-27T15:22:02.811637+0000","last_undegraded":"2023-11-27T15:22:02.811637+0000","last_fullsized":"2023-11-27T15:22:02.811637+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:27:55.061003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.747709+0000","last_change":"2023-11-27T15:21:02.933684+0000","last_active":"2023-11-27T15:22:31.747709+0000","last_peered":"2023-11-27T15:22:31.747709+0000","last_clean":"2023-11-27T15:22:31.747709+0000","last_became_active":"2023-11-27T15:21:02.933215+0000","last_became_peered":"2023-11-27T15:21:02.933215+0000","last_unstale":"2023-11-27T15:22:31.747709+0000","last_undegraded":"2023-11-27T15:22:31.747709+0000","last_fullsized":"2023-11-27T15:22:31.747709+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:38:04.533244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811191+0000","last_change":"2023-11-27T15:20:37.666705+0000","last_active":"2023-11-27T15:22:02.811191+0000","last_peered":"2023-11-27T15:22:02.811191+0000","last_clean":"2023-11-27T15:22:02.811191+0000","last_became_active":"2023-11-27T15:20:37.666236+0000","last_became_peered":"2023-11-27T15:20:37.666236+0000","last_unstale":"2023-11-27T15:22:02.811191+0000","last_undegraded":"2023-11-27T15:22:02.811191+0000","last_fullsized":"2023-11-27T15:22:02.811191+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:39:04.186691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818128+0000","last_change":"2023-11-27T15:20:32.663915+0000","last_active":"2023-11-27T15:22:02.818128+0000","last_peered":"2023-11-27T15:22:02.818128+0000","last_clean":"2023-11-27T15:22:02.818128+0000","last_became_active":"2023-11-27T15:20:32.663069+0000","last_became_peered":"2023-11-27T15:20:32.663069+0000","last_unstale":"2023-11-27T15:22:02.818128+0000","last_undegraded":"2023-11-27T15:22:02.818128+0000","last_fullsized":"2023-11-27T15:22:02.818128+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:59:18.356598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.765956+0000","last_change":"2023-11-27T15:20:33.238150+0000","last_active":"2023-11-27T15:22:31.765956+0000","last_peered":"2023-11-27T15:22:31.765956+0000","last_clean":"2023-11-27T15:22:31.765956+0000","last_became_active":"2023-11-27T15:20:33.236972+0000","last_became_peered":"2023-11-27T15:20:33.236972+0000","last_unstale":"2023-11-27T15:22:31.765956+0000","last_undegraded":"2023-11-27T15:22:31.765956+0000","last_fullsized":"2023-11-27T15:22:31.765956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:18:18.352219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811484+0000","last_change":"2023-11-27T15:20:37.668031+0000","last_active":"2023-11-27T15:22:02.811484+0000","last_peered":"2023-11-27T15:22:02.811484+0000","last_clean":"2023-11-27T15:22:02.811484+0000","last_became_active":"2023-11-27T15:20:37.667640+0000","last_became_peered":"2023-11-27T15:20:37.667640+0000","last_unstale":"2023-11-27T15:22:02.811484+0000","last_undegraded":"2023-11-27T15:22:02.811484+0000","last_fullsized":"2023-11-27T15:22:02.811484+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:01:43.896196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.819361+0000","last_change":"2023-11-27T15:20:32.655738+0000","last_active":"2023-11-27T15:22:02.819361+0000","last_peered":"2023-11-27T15:22:02.819361+0000","last_clean":"2023-11-27T15:22:02.819361+0000","last_became_active":"2023-11-27T15:20:32.655439+0000","last_became_peered":"2023-11-27T15:20:32.655439+0000","last_unstale":"2023-11-27T15:22:02.819361+0000","last_undegraded":"2023-11-27T15:22:02.819361+0000","last_fullsized":"2023-11-27T15:22:02.819361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:26:55.161397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.752166+0000","last_change":"2023-11-27T15:20:33.229267+0000","last_active":"2023-11-27T15:22:31.752166+0000","last_peered":"2023-11-27T15:22:31.752166+0000","last_clean":"2023-11-27T15:22:31.752166+0000","last_became_active":"2023-11-27T15:20:33.228746+0000","last_became_peered":"2023-11-27T15:20:33.228746+0000","last_unstale":"2023-11-27T15:22:31.752166+0000","last_undegraded":"2023-11-27T15:22:31.752166+0000","last_fullsized":"2023-11-27T15:22:31.752166+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:55:04.819687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811343+0000","last_change":"2023-11-27T15:20:37.671893+0000","last_active":"2023-11-27T15:22:02.811343+0000","last_peered":"2023-11-27T15:22:02.811343+0000","last_clean":"2023-11-27T15:22:02.811343+0000","last_became_active":"2023-11-27T15:20:37.671033+0000","last_became_peered":"2023-11-27T15:20:37.671033+0000","last_unstale":"2023-11-27T15:22:02.811343+0000","last_undegraded":"2023-11-27T15:22:02.811343+0000","last_fullsized":"2023-11-27T15:22:02.811343+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:23:32.141225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818527+0000","last_change":"2023-11-27T15:20:32.650342+0000","last_active":"2023-11-27T15:22:02.818527+0000","last_peered":"2023-11-27T15:22:02.818527+0000","last_clean":"2023-11-27T15:22:02.818527+0000","last_became_active":"2023-11-27T15:20:32.650034+0000","last_became_peered":"2023-11-27T15:20:32.650034+0000","last_unstale":"2023-11-27T15:22:02.818527+0000","last_undegraded":"2023-11-27T15:22:02.818527+0000","last_fullsized":"2023-11-27T15:22:02.818527+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:14:01.957530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.749849+0000","last_change":"2023-11-27T15:20:33.226005+0000","last_active":"2023-11-27T15:22:31.749849+0000","last_peered":"2023-11-27T15:22:31.749849+0000","last_clean":"2023-11-27T15:22:31.749849+0000","last_became_active":"2023-11-27T15:20:33.225685+0000","last_became_peered":"2023-11-27T15:20:33.225685+0000","last_unstale":"2023-11-27T15:22:31.749849+0000","last_undegraded":"2023-11-27T15:22:31.749849+0000","last_fullsized":"2023-11-27T15:22:31.749849+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:36:39.501462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.819431+0000","last_change":"2023-11-27T15:20:37.673168+0000","last_active":"2023-11-27T15:22:02.819431+0000","last_peered":"2023-11-27T15:22:02.819431+0000","last_clean":"2023-11-27T15:22:02.819431+0000","last_became_active":"2023-11-27T15:20:37.672762+0000","last_became_peered":"2023-11-27T15:20:37.672762+0000","last_unstale":"2023-11-27T15:22:02.819431+0000","last_undegraded":"2023-11-27T15:22:02.819431+0000","last_fullsized":"2023-11-27T15:22:02.819431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:04:26.957766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'6","reported_seq":140,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:43.972931+0000","last_change":"2023-11-27T15:20:32.656402+0000","last_active":"2023-11-27T15:22:43.972931+0000","last_peered":"2023-11-27T15:22:43.972931+0000","last_clean":"2023-11-27T15:22:43.972931+0000","last_became_active":"2023-11-27T15:20:32.656112+0000","last_became_peered":"2023-11-27T15:20:32.656112+0000","last_unstale":"2023-11-27T15:22:43.972931+0000","last_undegraded":"2023-11-27T15:22:43.972931+0000","last_fullsized":"2023-11-27T15:22:43.972931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:40:54.677726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":886,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.743115+0000","last_change":"2023-11-27T15:20:33.223073+0000","last_active":"2023-11-27T15:22:31.743115+0000","last_peered":"2023-11-27T15:22:31.743115+0000","last_clean":"2023-11-27T15:22:31.743115+0000","last_became_active":"2023-11-27T15:20:33.222655+0000","last_became_peered":"2023-11-27T15:20:33.222655+0000","last_unstale":"2023-11-27T15:22:31.743115+0000","last_undegraded":"2023-11-27T15:22:31.743115+0000","last_fullsized":"2023-11-27T15:22:31.743115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:09:23.257015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.819193+0000","last_change":"2023-11-27T15:20:37.678722+0000","last_active":"2023-11-27T15:22:02.819193+0000","last_peered":"2023-11-27T15:22:02.819193+0000","last_clean":"2023-11-27T15:22:02.819193+0000","last_became_active":"2023-11-27T15:20:37.678242+0000","last_became_peered":"2023-11-27T15:20:37.678242+0000","last_unstale":"2023-11-27T15:22:02.819193+0000","last_undegraded":"2023-11-27T15:22:02.819193+0000","last_fullsized":"2023-11-27T15:22:02.819193+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:54:25.629989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.905230+0000","last_change":"2023-11-27T15:20:32.646951+0000","last_active":"2023-11-27T15:22:02.905230+0000","last_peered":"2023-11-27T15:22:02.905230+0000","last_clean":"2023-11-27T15:22:02.905230+0000","last_became_active":"2023-11-27T15:20:32.646620+0000","last_became_peered":"2023-11-27T15:20:32.646620+0000","last_unstale":"2023-11-27T15:22:02.905230+0000","last_undegraded":"2023-11-27T15:22:02.905230+0000","last_fullsized":"2023-11-27T15:22:02.905230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:36:25.079772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.751828+0000","last_change":"2023-11-27T15:20:33.226576+0000","last_active":"2023-11-27T15:22:31.751828+0000","last_peered":"2023-11-27T15:22:31.751828+0000","last_clean":"2023-11-27T15:22:31.751828+0000","last_became_active":"2023-11-27T15:20:33.226181+0000","last_became_peered":"2023-11-27T15:20:33.226181+0000","last_unstale":"2023-11-27T15:22:31.751828+0000","last_undegraded":"2023-11-27T15:22:31.751828+0000","last_fullsized":"2023-11-27T15:22:31.751828+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:07:09.638906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.815632+0000","last_change":"2023-11-27T15:20:37.681668+0000","last_active":"2023-11-27T15:22:02.815632+0000","last_peered":"2023-11-27T15:22:02.815632+0000","last_clean":"2023-11-27T15:22:02.815632+0000","last_became_active":"2023-11-27T15:20:37.681236+0000","last_became_peered":"2023-11-27T15:20:37.681236+0000","last_unstale":"2023-11-27T15:22:02.815632+0000","last_undegraded":"2023-11-27T15:22:02.815632+0000","last_fullsized":"2023-11-27T15:22:02.815632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:11:46.529862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.902849+0000","last_change":"2023-11-27T15:20:32.659421+0000","last_active":"2023-11-27T15:22:02.902849+0000","last_peered":"2023-11-27T15:22:02.902849+0000","last_clean":"2023-11-27T15:22:02.902849+0000","last_became_active":"2023-11-27T15:20:32.658882+0000","last_became_peered":"2023-11-27T15:20:32.658882+0000","last_unstale":"2023-11-27T15:22:02.902849+0000","last_undegraded":"2023-11-27T15:22:02.902849+0000","last_fullsized":"2023-11-27T15:22:02.902849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:35:52.409185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.758848+0000","last_change":"2023-11-27T15:20:33.227635+0000","last_active":"2023-11-27T15:22:31.758848+0000","last_peered":"2023-11-27T15:22:31.758848+0000","last_clean":"2023-11-27T15:22:31.758848+0000","last_became_active":"2023-11-27T15:20:33.225217+0000","last_became_peered":"2023-11-27T15:20:33.225217+0000","last_unstale":"2023-11-27T15:22:31.758848+0000","last_undegraded":"2023-11-27T15:22:31.758848+0000","last_fullsized":"2023-11-27T15:22:31.758848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:11:03.899702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.812856+0000","last_change":"2023-11-27T15:20:37.679575+0000","last_active":"2023-11-27T15:22:02.812856+0000","last_peered":"2023-11-27T15:22:02.812856+0000","last_clean":"2023-11-27T15:22:02.812856+0000","last_became_active":"2023-11-27T15:20:37.679131+0000","last_became_peered":"2023-11-27T15:20:37.679131+0000","last_unstale":"2023-11-27T15:22:02.812856+0000","last_undegraded":"2023-11-27T15:22:02.812856+0000","last_fullsized":"2023-11-27T15:22:02.812856+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:03:41.337015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.908090+0000","last_change":"2023-11-27T15:20:32.657155+0000","last_active":"2023-11-27T15:22:02.908090+0000","last_peered":"2023-11-27T15:22:02.908090+0000","last_clean":"2023-11-27T15:22:02.908090+0000","last_became_active":"2023-11-27T15:20:32.656832+0000","last_became_peered":"2023-11-27T15:20:32.656832+0000","last_unstale":"2023-11-27T15:22:02.908090+0000","last_undegraded":"2023-11-27T15:22:02.908090+0000","last_fullsized":"2023-11-27T15:22:02.908090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:14:08.024415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.761021+0000","last_change":"2023-11-27T15:20:33.229215+0000","last_active":"2023-11-27T15:22:31.761021+0000","last_peered":"2023-11-27T15:22:31.761021+0000","last_clean":"2023-11-27T15:22:31.761021+0000","last_became_active":"2023-11-27T15:20:33.228616+0000","last_became_peered":"2023-11-27T15:20:33.228616+0000","last_unstale":"2023-11-27T15:22:31.761021+0000","last_undegraded":"2023-11-27T15:22:31.761021+0000","last_fullsized":"2023-11-27T15:22:31.761021+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:16:50.126720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.817360+0000","last_change":"2023-11-27T15:20:37.672582+0000","last_active":"2023-11-27T15:22:02.817360+0000","last_peered":"2023-11-27T15:22:02.817360+0000","last_clean":"2023-11-27T15:22:02.817360+0000","last_became_active":"2023-11-27T15:20:37.672193+0000","last_became_peered":"2023-11-27T15:20:37.672193+0000","last_unstale":"2023-11-27T15:22:02.817360+0000","last_undegraded":"2023-11-27T15:22:02.817360+0000","last_fullsized":"2023-11-27T15:22:02.817360+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:00:48.323529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":131,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:40.449635+0000","last_change":"2023-11-27T15:20:32.647894+0000","last_active":"2023-11-27T15:22:40.449635+0000","last_peered":"2023-11-27T15:22:40.449635+0000","last_clean":"2023-11-27T15:22:40.449635+0000","last_became_active":"2023-11-27T15:20:32.647597+0000","last_became_peered":"2023-11-27T15:20:32.647597+0000","last_unstale":"2023-11-27T15:22:40.449635+0000","last_undegraded":"2023-11-27T15:22:40.449635+0000","last_fullsized":"2023-11-27T15:22:40.449635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:12:48.894187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.753452+0000","last_change":"2023-11-27T15:20:33.229382+0000","last_active":"2023-11-27T15:22:31.753452+0000","last_peered":"2023-11-27T15:22:31.753452+0000","last_clean":"2023-11-27T15:22:31.753452+0000","last_became_active":"2023-11-27T15:20:33.229031+0000","last_became_peered":"2023-11-27T15:20:33.229031+0000","last_unstale":"2023-11-27T15:22:31.753452+0000","last_undegraded":"2023-11-27T15:22:31.753452+0000","last_fullsized":"2023-11-27T15:22:31.753452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:35:27.898226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811592+0000","last_change":"2023-11-27T15:20:37.679131+0000","last_active":"2023-11-27T15:22:02.811592+0000","last_peered":"2023-11-27T15:22:02.811592+0000","last_clean":"2023-11-27T15:22:02.811592+0000","last_became_active":"2023-11-27T15:20:37.678290+0000","last_became_peered":"2023-11-27T15:20:37.678290+0000","last_unstale":"2023-11-27T15:22:02.811592+0000","last_undegraded":"2023-11-27T15:22:02.811592+0000","last_fullsized":"2023-11-27T15:22:02.811592+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:55:02.097941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.244489+0000","last_change":"2023-11-27T15:21:02.797374+0000","last_active":"2023-11-27T15:22:03.244489+0000","last_peered":"2023-11-27T15:22:03.244489+0000","last_clean":"2023-11-27T15:22:03.244489+0000","last_became_active":"2023-11-27T15:21:02.796076+0000","last_became_peered":"2023-11-27T15:21:02.796076+0000","last_unstale":"2023-11-27T15:22:03.244489+0000","last_undegraded":"2023-11-27T15:22:03.244489+0000","last_fullsized":"2023-11-27T15:22:03.244489+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:04:13.004951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.766526+0000","last_change":"2023-11-27T15:21:02.935104+0000","last_active":"2023-11-27T15:22:31.766526+0000","last_peered":"2023-11-27T15:22:31.766526+0000","last_clean":"2023-11-27T15:22:31.766526+0000","last_became_active":"2023-11-27T15:21:02.934486+0000","last_became_peered":"2023-11-27T15:21:02.934486+0000","last_unstale":"2023-11-27T15:22:31.766526+0000","last_undegraded":"2023-11-27T15:22:31.766526+0000","last_fullsized":"2023-11-27T15:22:31.766526+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:48:49.686999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811614+0000","last_change":"2023-11-27T15:20:37.670389+0000","last_active":"2023-11-27T15:22:02.811614+0000","last_peered":"2023-11-27T15:22:02.811614+0000","last_clean":"2023-11-27T15:22:02.811614+0000","last_became_active":"2023-11-27T15:20:37.669512+0000","last_became_peered":"2023-11-27T15:20:37.669512+0000","last_unstale":"2023-11-27T15:22:02.811614+0000","last_undegraded":"2023-11-27T15:22:02.811614+0000","last_fullsized":"2023-11-27T15:22:02.811614+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:25:51.789859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.908694+0000","last_change":"2023-11-27T15:20:32.658287+0000","last_active":"2023-11-27T15:22:02.908694+0000","last_peered":"2023-11-27T15:22:02.908694+0000","last_clean":"2023-11-27T15:22:02.908694+0000","last_became_active":"2023-11-27T15:20:32.657925+0000","last_became_peered":"2023-11-27T15:20:32.657925+0000","last_unstale":"2023-11-27T15:22:02.908694+0000","last_undegraded":"2023-11-27T15:22:02.908694+0000","last_fullsized":"2023-11-27T15:22:02.908694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:47:39.305351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.754013+0000","last_change":"2023-11-27T15:20:33.220240+0000","last_active":"2023-11-27T15:22:31.754013+0000","last_peered":"2023-11-27T15:22:31.754013+0000","last_clean":"2023-11-27T15:22:31.754013+0000","last_became_active":"2023-11-27T15:20:33.219962+0000","last_became_peered":"2023-11-27T15:20:33.219962+0000","last_unstale":"2023-11-27T15:22:31.754013+0000","last_undegraded":"2023-11-27T15:22:31.754013+0000","last_fullsized":"2023-11-27T15:22:31.754013+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:00:01.276822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811329+0000","last_change":"2023-11-27T15:20:37.672203+0000","last_active":"2023-11-27T15:22:02.811329+0000","last_peered":"2023-11-27T15:22:02.811329+0000","last_clean":"2023-11-27T15:22:02.811329+0000","last_became_active":"2023-11-27T15:20:37.671149+0000","last_became_peered":"2023-11-27T15:20:37.671149+0000","last_unstale":"2023-11-27T15:22:02.811329+0000","last_undegraded":"2023-11-27T15:22:02.811329+0000","last_fullsized":"2023-11-27T15:22:02.811329+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:15:42.398912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'37","reported_seq":271,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:40.967635+0000","last_change":"2023-11-27T15:20:32.646637+0000","last_active":"2023-11-27T15:22:40.967635+0000","last_peered":"2023-11-27T15:22:40.967635+0000","last_clean":"2023-11-27T15:22:40.967635+0000","last_became_active":"2023-11-27T15:20:32.646292+0000","last_became_peered":"2023-11-27T15:20:32.646292+0000","last_unstale":"2023-11-27T15:22:40.967635+0000","last_undegraded":"2023-11-27T15:22:40.967635+0000","last_fullsized":"2023-11-27T15:22:40.967635+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:57:01.372262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60369,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":204,"num_read_kb":162,"num_write":38,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.737253+0000","last_change":"2023-11-27T15:20:33.217078+0000","last_active":"2023-11-27T15:22:31.737253+0000","last_peered":"2023-11-27T15:22:31.737253+0000","last_clean":"2023-11-27T15:22:31.737253+0000","last_became_active":"2023-11-27T15:20:33.216787+0000","last_became_peered":"2023-11-27T15:20:33.216787+0000","last_unstale":"2023-11-27T15:22:31.737253+0000","last_undegraded":"2023-11-27T15:22:31.737253+0000","last_fullsized":"2023-11-27T15:22:31.737253+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:39:29.760219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.909130+0000","last_change":"2023-11-27T15:20:37.671800+0000","last_active":"2023-11-27T15:22:02.909130+0000","last_peered":"2023-11-27T15:22:02.909130+0000","last_clean":"2023-11-27T15:22:02.909130+0000","last_became_active":"2023-11-27T15:20:37.671374+0000","last_became_peered":"2023-11-27T15:20:37.671374+0000","last_unstale":"2023-11-27T15:22:02.909130+0000","last_undegraded":"2023-11-27T15:22:02.909130+0000","last_fullsized":"2023-11-27T15:22:02.909130+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:38:25.460193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818948+0000","last_change":"2023-11-27T15:20:32.659508+0000","last_active":"2023-11-27T15:22:02.818948+0000","last_peered":"2023-11-27T15:22:02.818948+0000","last_clean":"2023-11-27T15:22:02.818948+0000","last_became_active":"2023-11-27T15:20:32.659226+0000","last_became_peered":"2023-11-27T15:20:32.659226+0000","last_unstale":"2023-11-27T15:22:02.818948+0000","last_undegraded":"2023-11-27T15:22:02.818948+0000","last_fullsized":"2023-11-27T15:22:02.818948+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:18:06.852915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.750607+0000","last_change":"2023-11-27T15:20:33.230012+0000","last_active":"2023-11-27T15:22:31.750607+0000","last_peered":"2023-11-27T15:22:31.750607+0000","last_clean":"2023-11-27T15:22:31.750607+0000","last_became_active":"2023-11-27T15:20:33.225193+0000","last_became_peered":"2023-11-27T15:20:33.225193+0000","last_unstale":"2023-11-27T15:22:31.750607+0000","last_undegraded":"2023-11-27T15:22:31.750607+0000","last_fullsized":"2023-11-27T15:22:31.750607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:42:53.662577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.815274+0000","last_change":"2023-11-27T15:20:37.672430+0000","last_active":"2023-11-27T15:22:02.815274+0000","last_peered":"2023-11-27T15:22:02.815274+0000","last_clean":"2023-11-27T15:22:02.815274+0000","last_became_active":"2023-11-27T15:20:37.671982+0000","last_became_peered":"2023-11-27T15:20:37.671982+0000","last_unstale":"2023-11-27T15:22:02.815274+0000","last_undegraded":"2023-11-27T15:22:02.815274+0000","last_fullsized":"2023-11-27T15:22:02.815274+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:41:34.081786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.908478+0000","last_change":"2023-11-27T15:20:32.655015+0000","last_active":"2023-11-27T15:22:02.908478+0000","last_peered":"2023-11-27T15:22:02.908478+0000","last_clean":"2023-11-27T15:22:02.908478+0000","last_became_active":"2023-11-27T15:20:32.654662+0000","last_became_peered":"2023-11-27T15:20:32.654662+0000","last_unstale":"2023-11-27T15:22:02.908478+0000","last_undegraded":"2023-11-27T15:22:02.908478+0000","last_fullsized":"2023-11-27T15:22:02.908478+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:49:03.386586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.754433+0000","last_change":"2023-11-27T15:20:33.233980+0000","last_active":"2023-11-27T15:22:31.754433+0000","last_peered":"2023-11-27T15:22:31.754433+0000","last_clean":"2023-11-27T15:22:31.754433+0000","last_became_active":"2023-11-27T15:20:33.233581+0000","last_became_peered":"2023-11-27T15:20:33.233581+0000","last_unstale":"2023-11-27T15:22:31.754433+0000","last_undegraded":"2023-11-27T15:22:31.754433+0000","last_fullsized":"2023-11-27T15:22:31.754433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:52:04.285704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811523+0000","last_change":"2023-11-27T15:20:37.682359+0000","last_active":"2023-11-27T15:22:02.811523+0000","last_peered":"2023-11-27T15:22:02.811523+0000","last_clean":"2023-11-27T15:22:02.811523+0000","last_became_active":"2023-11-27T15:20:37.681888+0000","last_became_peered":"2023-11-27T15:20:37.681888+0000","last_unstale":"2023-11-27T15:22:02.811523+0000","last_undegraded":"2023-11-27T15:22:02.811523+0000","last_fullsized":"2023-11-27T15:22:02.811523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:04:23.066492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":139,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:42.967416+0000","last_change":"2023-11-27T15:20:32.653388+0000","last_active":"2023-11-27T15:22:42.967416+0000","last_peered":"2023-11-27T15:22:42.967416+0000","last_clean":"2023-11-27T15:22:42.967416+0000","last_became_active":"2023-11-27T15:20:32.653007+0000","last_became_peered":"2023-11-27T15:20:32.653007+0000","last_unstale":"2023-11-27T15:22:42.967416+0000","last_undegraded":"2023-11-27T15:22:42.967416+0000","last_fullsized":"2023-11-27T15:22:42.967416+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:40:28.723669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1071,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.750879+0000","last_change":"2023-11-27T15:20:33.216539+0000","last_active":"2023-11-27T15:22:31.750879+0000","last_peered":"2023-11-27T15:22:31.750879+0000","last_clean":"2023-11-27T15:22:31.750879+0000","last_became_active":"2023-11-27T15:20:33.216166+0000","last_became_peered":"2023-11-27T15:20:33.216166+0000","last_unstale":"2023-11-27T15:22:31.750879+0000","last_undegraded":"2023-11-27T15:22:31.750879+0000","last_fullsized":"2023-11-27T15:22:31.750879+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:17:42.918928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.902913+0000","last_change":"2023-11-27T15:20:37.675735+0000","last_active":"2023-11-27T15:22:02.902913+0000","last_peered":"2023-11-27T15:22:02.902913+0000","last_clean":"2023-11-27T15:22:02.902913+0000","last_became_active":"2023-11-27T15:20:37.669685+0000","last_became_peered":"2023-11-27T15:20:37.669685+0000","last_unstale":"2023-11-27T15:22:02.902913+0000","last_undegraded":"2023-11-27T15:22:02.902913+0000","last_fullsized":"2023-11-27T15:22:02.902913+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:55:44.635583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:05.499173+0000","last_change":"2023-11-27T15:22:05.499173+0000","last_active":"2023-11-27T15:22:05.499173+0000","last_peered":"2023-11-27T15:22:05.499173+0000","last_clean":"2023-11-27T15:22:05.499173+0000","last_became_active":"2023-11-27T15:22:02.821261+0000","last_became_peered":"2023-11-27T15:22:02.821261+0000","last_unstale":"2023-11-27T15:22:05.499173+0000","last_undegraded":"2023-11-27T15:22:05.499173+0000","last_fullsized":"2023-11-27T15:22:05.499173+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:16:43.958003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.748850+0000","last_change":"2023-11-27T15:20:33.225019+0000","last_active":"2023-11-27T15:22:31.748850+0000","last_peered":"2023-11-27T15:22:31.748850+0000","last_clean":"2023-11-27T15:22:31.748850+0000","last_became_active":"2023-11-27T15:20:33.224622+0000","last_became_peered":"2023-11-27T15:20:33.224622+0000","last_unstale":"2023-11-27T15:22:31.748850+0000","last_undegraded":"2023-11-27T15:22:31.748850+0000","last_fullsized":"2023-11-27T15:22:31.748850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:18:00.816094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.908162+0000","last_change":"2023-11-27T15:20:37.680808+0000","last_active":"2023-11-27T15:22:02.908162+0000","last_peered":"2023-11-27T15:22:02.908162+0000","last_clean":"2023-11-27T15:22:02.908162+0000","last_became_active":"2023-11-27T15:20:37.680348+0000","last_became_peered":"2023-11-27T15:20:37.680348+0000","last_unstale":"2023-11-27T15:22:02.908162+0000","last_undegraded":"2023-11-27T15:22:02.908162+0000","last_fullsized":"2023-11-27T15:22:02.908162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:37:19.386150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.819004+0000","last_change":"2023-11-27T15:20:32.651093+0000","last_active":"2023-11-27T15:22:02.819004+0000","last_peered":"2023-11-27T15:22:02.819004+0000","last_clean":"2023-11-27T15:22:02.819004+0000","last_became_active":"2023-11-27T15:20:32.650657+0000","last_became_peered":"2023-11-27T15:20:32.650657+0000","last_unstale":"2023-11-27T15:22:02.819004+0000","last_undegraded":"2023-11-27T15:22:02.819004+0000","last_fullsized":"2023-11-27T15:22:02.819004+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:56:44.895712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.749518+0000","last_change":"2023-11-27T15:20:33.224806+0000","last_active":"2023-11-27T15:22:31.749518+0000","last_peered":"2023-11-27T15:22:31.749518+0000","last_clean":"2023-11-27T15:22:31.749518+0000","last_became_active":"2023-11-27T15:20:33.224362+0000","last_became_peered":"2023-11-27T15:20:33.224362+0000","last_unstale":"2023-11-27T15:22:31.749518+0000","last_undegraded":"2023-11-27T15:22:31.749518+0000","last_fullsized":"2023-11-27T15:22:31.749518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:20:50.902402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818048+0000","last_change":"2023-11-27T15:20:37.849274+0000","last_active":"2023-11-27T15:22:02.818048+0000","last_peered":"2023-11-27T15:22:02.818048+0000","last_clean":"2023-11-27T15:22:02.818048+0000","last_became_active":"2023-11-27T15:20:37.848839+0000","last_became_peered":"2023-11-27T15:20:37.848839+0000","last_unstale":"2023-11-27T15:22:02.818048+0000","last_undegraded":"2023-11-27T15:22:02.818048+0000","last_fullsized":"2023-11-27T15:22:02.818048+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:49:32.136447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811304+0000","last_change":"2023-11-27T15:20:32.660835+0000","last_active":"2023-11-27T15:22:02.811304+0000","last_peered":"2023-11-27T15:22:02.811304+0000","last_clean":"2023-11-27T15:22:02.811304+0000","last_became_active":"2023-11-27T15:20:32.660431+0000","last_became_peered":"2023-11-27T15:20:32.660431+0000","last_unstale":"2023-11-27T15:22:02.811304+0000","last_undegraded":"2023-11-27T15:22:02.811304+0000","last_fullsized":"2023-11-27T15:22:02.811304+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:48:52.707533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'4","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.759772+0000","last_change":"2023-11-27T15:20:33.245772+0000","last_active":"2023-11-27T15:22:31.759772+0000","last_peered":"2023-11-27T15:22:31.759772+0000","last_clean":"2023-11-27T15:22:31.759772+0000","last_became_active":"2023-11-27T15:20:33.245468+0000","last_became_peered":"2023-11-27T15:20:33.245468+0000","last_unstale":"2023-11-27T15:22:31.759772+0000","last_undegraded":"2023-11-27T15:22:31.759772+0000","last_fullsized":"2023-11-27T15:22:31.759772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:58:59.950032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.902964+0000","last_change":"2023-11-27T15:20:37.678972+0000","last_active":"2023-11-27T15:22:02.902964+0000","last_peered":"2023-11-27T15:22:02.902964+0000","last_clean":"2023-11-27T15:22:02.902964+0000","last_became_active":"2023-11-27T15:20:37.678304+0000","last_became_peered":"2023-11-27T15:20:37.678304+0000","last_unstale":"2023-11-27T15:22:02.902964+0000","last_undegraded":"2023-11-27T15:22:02.902964+0000","last_fullsized":"2023-11-27T15:22:02.902964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:05:43.664151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.244402+0000","last_change":"2023-11-27T15:20:32.661739+0000","last_active":"2023-11-27T15:22:03.244402+0000","last_peered":"2023-11-27T15:22:03.244402+0000","last_clean":"2023-11-27T15:22:03.244402+0000","last_became_active":"2023-11-27T15:20:32.661401+0000","last_became_peered":"2023-11-27T15:20:32.661401+0000","last_unstale":"2023-11-27T15:22:03.244402+0000","last_undegraded":"2023-11-27T15:22:03.244402+0000","last_fullsized":"2023-11-27T15:22:03.244402+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:32:44.035448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.749301+0000","last_change":"2023-11-27T15:20:33.235595+0000","last_active":"2023-11-27T15:22:31.749301+0000","last_peered":"2023-11-27T15:22:31.749301+0000","last_clean":"2023-11-27T15:22:31.749301+0000","last_became_active":"2023-11-27T15:20:33.235277+0000","last_became_peered":"2023-11-27T15:20:33.235277+0000","last_unstale":"2023-11-27T15:22:31.749301+0000","last_undegraded":"2023-11-27T15:22:31.749301+0000","last_fullsized":"2023-11-27T15:22:31.749301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:13:59.365599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.903185+0000","last_change":"2023-11-27T15:21:02.934194+0000","last_active":"2023-11-27T15:22:02.903185+0000","last_peered":"2023-11-27T15:22:02.903185+0000","last_clean":"2023-11-27T15:22:02.903185+0000","last_became_active":"2023-11-27T15:21:02.933555+0000","last_became_peered":"2023-11-27T15:21:02.933555+0000","last_unstale":"2023-11-27T15:22:02.903185+0000","last_undegraded":"2023-11-27T15:22:02.903185+0000","last_fullsized":"2023-11-27T15:22:02.903185+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:16:19.222019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.755634+0000","last_change":"2023-11-27T15:21:02.784351+0000","last_active":"2023-11-27T15:22:31.755634+0000","last_peered":"2023-11-27T15:22:31.755634+0000","last_clean":"2023-11-27T15:22:31.755634+0000","last_became_active":"2023-11-27T15:21:02.783996+0000","last_became_peered":"2023-11-27T15:21:02.783996+0000","last_unstale":"2023-11-27T15:22:31.755634+0000","last_undegraded":"2023-11-27T15:22:31.755634+0000","last_fullsized":"2023-11-27T15:22:31.755634+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:17:24.113607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":133,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:40.963521+0000","last_change":"2023-11-27T15:20:32.659823+0000","last_active":"2023-11-27T15:22:40.963521+0000","last_peered":"2023-11-27T15:22:40.963521+0000","last_clean":"2023-11-27T15:22:40.963521+0000","last_became_active":"2023-11-27T15:20:32.659472+0000","last_became_peered":"2023-11-27T15:20:32.659472+0000","last_unstale":"2023-11-27T15:22:40.963521+0000","last_undegraded":"2023-11-27T15:22:40.963521+0000","last_fullsized":"2023-11-27T15:22:40.963521+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:08:32.142408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"90'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.244444+0000","last_change":"2023-11-27T15:20:37.671494+0000","last_active":"2023-11-27T15:22:03.244444+0000","last_peered":"2023-11-27T15:22:03.244444+0000","last_clean":"2023-11-27T15:22:03.244444+0000","last_became_active":"2023-11-27T15:20:37.671010+0000","last_became_peered":"2023-11-27T15:20:37.671010+0000","last_unstale":"2023-11-27T15:22:03.244444+0000","last_undegraded":"2023-11-27T15:22:03.244444+0000","last_fullsized":"2023-11-27T15:22:03.244444+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:51:51.646563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.758695+0000","last_change":"2023-11-27T15:20:33.241669+0000","last_active":"2023-11-27T15:22:31.758695+0000","last_peered":"2023-11-27T15:22:31.758695+0000","last_clean":"2023-11-27T15:22:31.758695+0000","last_became_active":"2023-11-27T15:20:33.241373+0000","last_became_peered":"2023-11-27T15:20:33.241373+0000","last_unstale":"2023-11-27T15:22:31.758695+0000","last_undegraded":"2023-11-27T15:22:31.758695+0000","last_fullsized":"2023-11-27T15:22:31.758695+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:03:47.587644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811591+0000","last_change":"2023-11-27T15:20:32.647596+0000","last_active":"2023-11-27T15:22:02.811591+0000","last_peered":"2023-11-27T15:22:02.811591+0000","last_clean":"2023-11-27T15:22:02.811591+0000","last_became_active":"2023-11-27T15:20:32.647228+0000","last_became_peered":"2023-11-27T15:20:32.647228+0000","last_unstale":"2023-11-27T15:22:02.811591+0000","last_undegraded":"2023-11-27T15:22:02.811591+0000","last_fullsized":"2023-11-27T15:22:02.811591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:37:28.551859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.244098+0000","last_change":"2023-11-27T15:19:30.114855+0000","last_active":"2023-11-27T15:22:03.244098+0000","last_peered":"2023-11-27T15:22:03.244098+0000","last_clean":"2023-11-27T15:22:03.244098+0000","last_became_active":"2023-11-27T15:19:30.114419+0000","last_became_peered":"2023-11-27T15:19:30.114419+0000","last_unstale":"2023-11-27T15:22:03.244098+0000","last_undegraded":"2023-11-27T15:22:03.244098+0000","last_fullsized":"2023-11-27T15:22:03.244098+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:16:20.216695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:16:20.216695+0000","last_clean_scrub_stamp":"2023-11-27T15:16:20.216695+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:02:43.854910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.908738+0000","last_change":"2023-11-27T15:20:37.850539+0000","last_active":"2023-11-27T15:22:02.908738+0000","last_peered":"2023-11-27T15:22:02.908738+0000","last_clean":"2023-11-27T15:22:02.908738+0000","last_became_active":"2023-11-27T15:20:37.850044+0000","last_became_peered":"2023-11-27T15:20:37.850044+0000","last_unstale":"2023-11-27T15:22:02.908738+0000","last_undegraded":"2023-11-27T15:22:02.908738+0000","last_fullsized":"2023-11-27T15:22:02.908738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:54:36.190506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.741984+0000","last_change":"2023-11-27T15:20:33.211319+0000","last_active":"2023-11-27T15:22:31.741984+0000","last_peered":"2023-11-27T15:22:31.741984+0000","last_clean":"2023-11-27T15:22:31.741984+0000","last_became_active":"2023-11-27T15:20:33.211004+0000","last_became_peered":"2023-11-27T15:20:33.211004+0000","last_unstale":"2023-11-27T15:22:31.741984+0000","last_undegraded":"2023-11-27T15:22:31.741984+0000","last_fullsized":"2023-11-27T15:22:31.741984+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:52:10.543307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.244696+0000","last_change":"2023-11-27T15:20:32.655003+0000","last_active":"2023-11-27T15:22:03.244696+0000","last_peered":"2023-11-27T15:22:03.244696+0000","last_clean":"2023-11-27T15:22:03.244696+0000","last_became_active":"2023-11-27T15:20:32.654517+0000","last_became_peered":"2023-11-27T15:20:32.654517+0000","last_unstale":"2023-11-27T15:22:03.244696+0000","last_undegraded":"2023-11-27T15:22:03.244696+0000","last_fullsized":"2023-11-27T15:22:03.244696+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:18:43.790411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.815270+0000","last_change":"2023-11-27T15:20:37.673183+0000","last_active":"2023-11-27T15:22:02.815270+0000","last_peered":"2023-11-27T15:22:02.815270+0000","last_clean":"2023-11-27T15:22:02.815270+0000","last_became_active":"2023-11-27T15:20:37.672838+0000","last_became_peered":"2023-11-27T15:20:37.672838+0000","last_unstale":"2023-11-27T15:22:02.815270+0000","last_undegraded":"2023-11-27T15:22:02.815270+0000","last_fullsized":"2023-11-27T15:22:02.815270+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:42:26.039321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'1","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818752+0000","last_change":"2023-11-27T15:20:32.659540+0000","last_active":"2023-11-27T15:22:02.818752+0000","last_peered":"2023-11-27T15:22:02.818752+0000","last_clean":"2023-11-27T15:22:02.818752+0000","last_became_active":"2023-11-27T15:20:32.659211+0000","last_became_peered":"2023-11-27T15:20:32.659211+0000","last_unstale":"2023-11-27T15:22:02.818752+0000","last_undegraded":"2023-11-27T15:22:02.818752+0000","last_fullsized":"2023-11-27T15:22:02.818752+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:32:29.624516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.757012+0000","last_change":"2023-11-27T15:20:33.216405+0000","last_active":"2023-11-27T15:22:31.757012+0000","last_peered":"2023-11-27T15:22:31.757012+0000","last_clean":"2023-11-27T15:22:31.757012+0000","last_became_active":"2023-11-27T15:20:33.216072+0000","last_became_peered":"2023-11-27T15:20:33.216072+0000","last_unstale":"2023-11-27T15:22:31.757012+0000","last_undegraded":"2023-11-27T15:22:31.757012+0000","last_fullsized":"2023-11-27T15:22:31.757012+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:24:07.496562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.817387+0000","last_change":"2023-11-27T15:20:37.849753+0000","last_active":"2023-11-27T15:22:02.817387+0000","last_peered":"2023-11-27T15:22:02.817387+0000","last_clean":"2023-11-27T15:22:02.817387+0000","last_became_active":"2023-11-27T15:20:37.848576+0000","last_became_peered":"2023-11-27T15:20:37.848576+0000","last_unstale":"2023-11-27T15:22:02.817387+0000","last_undegraded":"2023-11-27T15:22:02.817387+0000","last_fullsized":"2023-11-27T15:22:02.817387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:33:46.286142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'8","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:41.517782+0000","last_change":"2023-11-27T15:20:32.652739+0000","last_active":"2023-11-27T15:22:41.517782+0000","last_peered":"2023-11-27T15:22:41.517782+0000","last_clean":"2023-11-27T15:22:41.517782+0000","last_became_active":"2023-11-27T15:20:32.652314+0000","last_became_peered":"2023-11-27T15:20:32.652314+0000","last_unstale":"2023-11-27T15:22:41.517782+0000","last_undegraded":"2023-11-27T15:22:41.517782+0000","last_fullsized":"2023-11-27T15:22:41.517782+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:00:14.008740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.758452+0000","last_change":"2023-11-27T15:20:33.240552+0000","last_active":"2023-11-27T15:22:31.758452+0000","last_peered":"2023-11-27T15:22:31.758452+0000","last_clean":"2023-11-27T15:22:31.758452+0000","last_became_active":"2023-11-27T15:20:33.239885+0000","last_became_peered":"2023-11-27T15:20:33.239885+0000","last_unstale":"2023-11-27T15:22:31.758452+0000","last_undegraded":"2023-11-27T15:22:31.758452+0000","last_fullsized":"2023-11-27T15:22:31.758452+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:44:48.111242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818035+0000","last_change":"2023-11-27T15:20:37.669895+0000","last_active":"2023-11-27T15:22:02.818035+0000","last_peered":"2023-11-27T15:22:02.818035+0000","last_clean":"2023-11-27T15:22:02.818035+0000","last_became_active":"2023-11-27T15:20:37.669489+0000","last_became_peered":"2023-11-27T15:20:37.669489+0000","last_unstale":"2023-11-27T15:22:02.818035+0000","last_undegraded":"2023-11-27T15:22:02.818035+0000","last_fullsized":"2023-11-27T15:22:02.818035+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:06:13.495056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.812082+0000","last_change":"2023-11-27T15:20:32.656773+0000","last_active":"2023-11-27T15:22:02.812082+0000","last_peered":"2023-11-27T15:22:02.812082+0000","last_clean":"2023-11-27T15:22:02.812082+0000","last_became_active":"2023-11-27T15:20:32.656451+0000","last_became_peered":"2023-11-27T15:20:32.656451+0000","last_unstale":"2023-11-27T15:22:02.812082+0000","last_undegraded":"2023-11-27T15:22:02.812082+0000","last_fullsized":"2023-11-27T15:22:02.812082+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:03:12.399313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.765942+0000","last_change":"2023-11-27T15:20:33.231685+0000","last_active":"2023-11-27T15:22:31.765942+0000","last_peered":"2023-11-27T15:22:31.765942+0000","last_clean":"2023-11-27T15:22:31.765942+0000","last_became_active":"2023-11-27T15:20:33.228265+0000","last_became_peered":"2023-11-27T15:20:33.228265+0000","last_unstale":"2023-11-27T15:22:31.765942+0000","last_undegraded":"2023-11-27T15:22:31.765942+0000","last_fullsized":"2023-11-27T15:22:31.765942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:53:48.855000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811795+0000","last_change":"2023-11-27T15:21:02.935830+0000","last_active":"2023-11-27T15:22:02.811795+0000","last_peered":"2023-11-27T15:22:02.811795+0000","last_clean":"2023-11-27T15:22:02.811795+0000","last_became_active":"2023-11-27T15:21:02.935412+0000","last_became_peered":"2023-11-27T15:21:02.935412+0000","last_unstale":"2023-11-27T15:22:02.811795+0000","last_undegraded":"2023-11-27T15:22:02.811795+0000","last_fullsized":"2023-11-27T15:22:02.811795+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:22:23.956927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.244630+0000","last_change":"2023-11-27T15:20:32.660424+0000","last_active":"2023-11-27T15:22:03.244630+0000","last_peered":"2023-11-27T15:22:03.244630+0000","last_clean":"2023-11-27T15:22:03.244630+0000","last_became_active":"2023-11-27T15:20:32.660112+0000","last_became_peered":"2023-11-27T15:20:32.660112+0000","last_unstale":"2023-11-27T15:22:03.244630+0000","last_undegraded":"2023-11-27T15:22:03.244630+0000","last_fullsized":"2023-11-27T15:22:03.244630+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:05:10.594173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.745264+0000","last_change":"2023-11-27T15:20:33.227677+0000","last_active":"2023-11-27T15:22:31.745264+0000","last_peered":"2023-11-27T15:22:31.745264+0000","last_clean":"2023-11-27T15:22:31.745264+0000","last_became_active":"2023-11-27T15:20:33.227334+0000","last_became_peered":"2023-11-27T15:20:33.227334+0000","last_unstale":"2023-11-27T15:22:31.745264+0000","last_undegraded":"2023-11-27T15:22:31.745264+0000","last_fullsized":"2023-11-27T15:22:31.745264+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:19:08.573620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.818782+0000","last_change":"2023-11-27T15:21:02.934516+0000","last_active":"2023-11-27T15:22:02.818782+0000","last_peered":"2023-11-27T15:22:02.818782+0000","last_clean":"2023-11-27T15:22:02.818782+0000","last_became_active":"2023-11-27T15:21:02.934012+0000","last_became_peered":"2023-11-27T15:21:02.934012+0000","last_unstale":"2023-11-27T15:22:02.818782+0000","last_undegraded":"2023-11-27T15:22:02.818782+0000","last_fullsized":"2023-11-27T15:22:02.818782+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:54:54.500574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.817444+0000","last_change":"2023-11-27T15:20:32.657935+0000","last_active":"2023-11-27T15:22:02.817444+0000","last_peered":"2023-11-27T15:22:02.817444+0000","last_clean":"2023-11-27T15:22:02.817444+0000","last_became_active":"2023-11-27T15:20:32.657579+0000","last_became_peered":"2023-11-27T15:20:32.657579+0000","last_unstale":"2023-11-27T15:22:02.817444+0000","last_undegraded":"2023-11-27T15:22:02.817444+0000","last_fullsized":"2023-11-27T15:22:02.817444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:00:47.684043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.767085+0000","last_change":"2023-11-27T15:20:33.225817+0000","last_active":"2023-11-27T15:22:31.767085+0000","last_peered":"2023-11-27T15:22:31.767085+0000","last_clean":"2023-11-27T15:22:31.767085+0000","last_became_active":"2023-11-27T15:20:33.225438+0000","last_became_peered":"2023-11-27T15:20:33.225438+0000","last_unstale":"2023-11-27T15:22:31.767085+0000","last_undegraded":"2023-11-27T15:22:31.767085+0000","last_fullsized":"2023-11-27T15:22:31.767085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:07:45.956918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"90'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811681+0000","last_change":"2023-11-27T15:21:02.934304+0000","last_active":"2023-11-27T15:22:02.811681+0000","last_peered":"2023-11-27T15:22:02.811681+0000","last_clean":"2023-11-27T15:22:02.811681+0000","last_became_active":"2023-11-27T15:21:02.933709+0000","last_became_peered":"2023-11-27T15:21:02.933709+0000","last_unstale":"2023-11-27T15:22:02.811681+0000","last_undegraded":"2023-11-27T15:22:02.811681+0000","last_fullsized":"2023-11-27T15:22:02.811681+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:52:00.767226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.119947+0000","last_change":"2023-11-27T15:22:03.119402+0000","last_active":"2023-11-27T15:22:03.119947+0000","last_peered":"2023-11-27T15:22:03.119947+0000","last_clean":"2023-11-27T15:22:03.119947+0000","last_became_active":"2023-11-27T15:22:02.819940+0000","last_became_peered":"2023-11-27T15:22:02.819940+0000","last_unstale":"2023-11-27T15:22:03.119947+0000","last_undegraded":"2023-11-27T15:22:03.119947+0000","last_fullsized":"2023-11-27T15:22:03.119947+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:41:14.055640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.758801+0000","last_change":"2023-11-27T15:20:33.235755+0000","last_active":"2023-11-27T15:22:31.758801+0000","last_peered":"2023-11-27T15:22:31.758801+0000","last_clean":"2023-11-27T15:22:31.758801+0000","last_became_active":"2023-11-27T15:20:33.234576+0000","last_became_peered":"2023-11-27T15:20:33.234576+0000","last_unstale":"2023-11-27T15:22:31.758801+0000","last_undegraded":"2023-11-27T15:22:31.758801+0000","last_fullsized":"2023-11-27T15:22:31.758801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:15:16.743978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.811797+0000","last_change":"2023-11-27T15:20:37.849121+0000","last_active":"2023-11-27T15:22:02.811797+0000","last_peered":"2023-11-27T15:22:02.811797+0000","last_clean":"2023-11-27T15:22:02.811797+0000","last_became_active":"2023-11-27T15:20:37.848683+0000","last_became_peered":"2023-11-27T15:20:37.848683+0000","last_unstale":"2023-11-27T15:22:02.811797+0000","last_undegraded":"2023-11-27T15:22:02.811797+0000","last_fullsized":"2023-11-27T15:22:02.811797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:37:37.945965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.817973+0000","last_change":"2023-11-27T15:20:32.655896+0000","last_active":"2023-11-27T15:22:02.817973+0000","last_peered":"2023-11-27T15:22:02.817973+0000","last_clean":"2023-11-27T15:22:02.817973+0000","last_became_active":"2023-11-27T15:20:32.655623+0000","last_became_peered":"2023-11-27T15:20:32.655623+0000","last_unstale":"2023-11-27T15:22:02.817973+0000","last_undegraded":"2023-11-27T15:22:02.817973+0000","last_fullsized":"2023-11-27T15:22:02.817973+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:58:52.716210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.753875+0000","last_change":"2023-11-27T15:20:33.227318+0000","last_active":"2023-11-27T15:22:31.753875+0000","last_peered":"2023-11-27T15:22:31.753875+0000","last_clean":"2023-11-27T15:22:31.753875+0000","last_became_active":"2023-11-27T15:20:33.226944+0000","last_became_peered":"2023-11-27T15:20:33.226944+0000","last_unstale":"2023-11-27T15:22:31.753875+0000","last_undegraded":"2023-11-27T15:22:31.753875+0000","last_fullsized":"2023-11-27T15:22:31.753875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:13:26.176414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:03.244531+0000","last_change":"2023-11-27T15:20:37.669098+0000","last_active":"2023-11-27T15:22:03.244531+0000","last_peered":"2023-11-27T15:22:03.244531+0000","last_clean":"2023-11-27T15:22:03.244531+0000","last_became_active":"2023-11-27T15:20:37.668693+0000","last_became_peered":"2023-11-27T15:20:37.668693+0000","last_unstale":"2023-11-27T15:22:03.244531+0000","last_undegraded":"2023-11-27T15:22:03.244531+0000","last_fullsized":"2023-11-27T15:22:03.244531+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:45:51.479637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.819574+0000","last_change":"2023-11-27T15:20:32.660285+0000","last_active":"2023-11-27T15:22:02.819574+0000","last_peered":"2023-11-27T15:22:02.819574+0000","last_clean":"2023-11-27T15:22:02.819574+0000","last_became_active":"2023-11-27T15:20:32.659982+0000","last_became_peered":"2023-11-27T15:20:32.659982+0000","last_unstale":"2023-11-27T15:22:02.819574+0000","last_undegraded":"2023-11-27T15:22:02.819574+0000","last_fullsized":"2023-11-27T15:22:02.819574+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T19:50:18.637710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.750955+0000","last_change":"2023-11-27T15:20:33.226629+0000","last_active":"2023-11-27T15:22:31.750955+0000","last_peered":"2023-11-27T15:22:31.750955+0000","last_clean":"2023-11-27T15:22:31.750955+0000","last_became_active":"2023-11-27T15:20:33.226183+0000","last_became_peered":"2023-11-27T15:20:33.226183+0000","last_unstale":"2023-11-27T15:22:31.750955+0000","last_undegraded":"2023-11-27T15:22:31.750955+0000","last_fullsized":"2023-11-27T15:22:31.750955+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T21:17:31.612135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.812710+0000","last_change":"2023-11-27T15:20:37.680987+0000","last_active":"2023-11-27T15:22:02.812710+0000","last_peered":"2023-11-27T15:22:02.812710+0000","last_clean":"2023-11-27T15:22:02.812710+0000","last_became_active":"2023-11-27T15:20:37.680219+0000","last_became_peered":"2023-11-27T15:20:37.680219+0000","last_unstale":"2023-11-27T15:22:02.812710+0000","last_undegraded":"2023-11-27T15:22:02.812710+0000","last_fullsized":"2023-11-27T15:22:02.812710+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:55:34.725659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"83'1","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.908876+0000","last_change":"2023-11-27T15:20:32.655035+0000","last_active":"2023-11-27T15:22:02.908876+0000","last_peered":"2023-11-27T15:22:02.908876+0000","last_clean":"2023-11-27T15:22:02.908876+0000","last_became_active":"2023-11-27T15:20:32.654687+0000","last_became_peered":"2023-11-27T15:20:32.654687+0000","last_unstale":"2023-11-27T15:22:02.908876+0000","last_undegraded":"2023-11-27T15:22:02.908876+0000","last_fullsized":"2023-11-27T15:22:02.908876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T23:57:01.888463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.759364+0000","last_change":"2023-11-27T15:20:33.245908+0000","last_active":"2023-11-27T15:22:31.759364+0000","last_peered":"2023-11-27T15:22:31.759364+0000","last_clean":"2023-11-27T15:22:31.759364+0000","last_became_active":"2023-11-27T15:20:33.245576+0000","last_became_peered":"2023-11-27T15:20:33.245576+0000","last_unstale":"2023-11-27T15:22:31.759364+0000","last_undegraded":"2023-11-27T15:22:31.759364+0000","last_fullsized":"2023-11-27T15:22:31.759364+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:27:55.968118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.907992+0000","last_change":"2023-11-27T15:20:37.674253+0000","last_active":"2023-11-27T15:22:02.907992+0000","last_peered":"2023-11-27T15:22:02.907992+0000","last_clean":"2023-11-27T15:22:02.907992+0000","last_became_active":"2023-11-27T15:20:37.673708+0000","last_became_peered":"2023-11-27T15:20:37.673708+0000","last_unstale":"2023-11-27T15:22:02.907992+0000","last_undegraded":"2023-11-27T15:22:02.907992+0000","last_fullsized":"2023-11-27T15:22:02.907992+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:36.636591+0000","last_clean_scrub_stamp":"2023-11-27T15:20:36.636591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:39:38.298957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"83'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:02.903185+0000","last_change":"2023-11-27T15:20:32.659605+0000","last_active":"2023-11-27T15:22:02.903185+0000","last_peered":"2023-11-27T15:22:02.903185+0000","last_clean":"2023-11-27T15:22:02.903185+0000","last_became_active":"2023-11-27T15:20:32.659186+0000","last_became_peered":"2023-11-27T15:20:32.659186+0000","last_unstale":"2023-11-27T15:22:02.903185+0000","last_undegraded":"2023-11-27T15:22:02.903185+0000","last_fullsized":"2023-11-27T15:22:02.903185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:31.621277+0000","last_clean_scrub_stamp":"2023-11-27T15:20:31.621277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T15:38:50.310551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.745773+0000","last_change":"2023-11-27T15:20:33.227914+0000","last_active":"2023-11-27T15:22:31.745773+0000","last_peered":"2023-11-27T15:22:31.745773+0000","last_clean":"2023-11-27T15:22:31.745773+0000","last_became_active":"2023-11-27T15:20:33.227572+0000","last_became_peered":"2023-11-27T15:20:33.227572+0000","last_unstale":"2023-11-27T15:22:31.745773+0000","last_undegraded":"2023-11-27T15:22:31.745773+0000","last_fullsized":"2023-11-27T15:22:31.745773+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:57:20.426893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.747905+0000","last_change":"2023-11-27T15:20:33.215605+0000","last_active":"2023-11-27T15:22:31.747905+0000","last_peered":"2023-11-27T15:22:31.747905+0000","last_clean":"2023-11-27T15:22:31.747905+0000","last_became_active":"2023-11-27T15:20:33.215295+0000","last_became_peered":"2023-11-27T15:20:33.215295+0000","last_unstale":"2023-11-27T15:22:31.747905+0000","last_undegraded":"2023-11-27T15:22:31.747905+0000","last_fullsized":"2023-11-27T15:22:31.747905+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T22:16:48.125214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.748947+0000","last_change":"2023-11-27T15:20:33.214014+0000","last_active":"2023-11-27T15:22:31.748947+0000","last_peered":"2023-11-27T15:22:31.748947+0000","last_clean":"2023-11-27T15:22:31.748947+0000","last_became_active":"2023-11-27T15:20:33.213685+0000","last_became_peered":"2023-11-27T15:20:33.213685+0000","last_unstale":"2023-11-27T15:22:31.748947+0000","last_undegraded":"2023-11-27T15:22:31.748947+0000","last_fullsized":"2023-11-27T15:22:31.748947+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:53:01.707335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.760580+0000","last_change":"2023-11-27T15:20:33.229722+0000","last_active":"2023-11-27T15:22:31.760580+0000","last_peered":"2023-11-27T15:22:31.760580+0000","last_clean":"2023-11-27T15:22:31.760580+0000","last_became_active":"2023-11-27T15:20:33.229372+0000","last_became_peered":"2023-11-27T15:20:33.229372+0000","last_unstale":"2023-11-27T15:22:31.760580+0000","last_undegraded":"2023-11-27T15:22:31.760580+0000","last_fullsized":"2023-11-27T15:22:31.760580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:40:05.300812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.762612+0000","last_change":"2023-11-27T15:20:33.235047+0000","last_active":"2023-11-27T15:22:31.762612+0000","last_peered":"2023-11-27T15:22:31.762612+0000","last_clean":"2023-11-27T15:22:31.762612+0000","last_became_active":"2023-11-27T15:20:33.233078+0000","last_became_peered":"2023-11-27T15:20:33.233078+0000","last_unstale":"2023-11-27T15:22:31.762612+0000","last_undegraded":"2023-11-27T15:22:31.762612+0000","last_fullsized":"2023-11-27T15:22:31.762612+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:45:51.860734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.751664+0000","last_change":"2023-11-27T15:20:33.229118+0000","last_active":"2023-11-27T15:22:31.751664+0000","last_peered":"2023-11-27T15:22:31.751664+0000","last_clean":"2023-11-27T15:22:31.751664+0000","last_became_active":"2023-11-27T15:20:33.228748+0000","last_became_peered":"2023-11-27T15:20:33.228748+0000","last_unstale":"2023-11-27T15:22:31.751664+0000","last_undegraded":"2023-11-27T15:22:31.751664+0000","last_fullsized":"2023-11-27T15:22:31.751664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T20:19:11.419230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.746259+0000","last_change":"2023-11-27T15:20:33.222714+0000","last_active":"2023-11-27T15:22:31.746259+0000","last_peered":"2023-11-27T15:22:31.746259+0000","last_clean":"2023-11-27T15:22:31.746259+0000","last_became_active":"2023-11-27T15:20:33.222396+0000","last_became_peered":"2023-11-27T15:20:33.222396+0000","last_unstale":"2023-11-27T15:22:31.746259+0000","last_undegraded":"2023-11-27T15:22:31.746259+0000","last_fullsized":"2023-11-27T15:22:31.746259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T01:52:29.993847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.754172+0000","last_change":"2023-11-27T15:20:33.223568+0000","last_active":"2023-11-27T15:22:31.754172+0000","last_peered":"2023-11-27T15:22:31.754172+0000","last_clean":"2023-11-27T15:22:31.754172+0000","last_became_active":"2023-11-27T15:20:33.220687+0000","last_became_peered":"2023-11-27T15:20:33.220687+0000","last_unstale":"2023-11-27T15:22:31.754172+0000","last_undegraded":"2023-11-27T15:22:31.754172+0000","last_fullsized":"2023-11-27T15:22:31.754172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:48:28.297235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.753086+0000","last_change":"2023-11-27T15:21:02.786992+0000","last_active":"2023-11-27T15:22:31.753086+0000","last_peered":"2023-11-27T15:22:31.753086+0000","last_clean":"2023-11-27T15:22:31.753086+0000","last_became_active":"2023-11-27T15:21:02.786644+0000","last_became_peered":"2023-11-27T15:21:02.786644+0000","last_unstale":"2023-11-27T15:22:31.753086+0000","last_undegraded":"2023-11-27T15:22:31.753086+0000","last_fullsized":"2023-11-27T15:22:31.753086+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T16:28:04.321999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.756953+0000","last_change":"2023-11-27T15:20:33.245815+0000","last_active":"2023-11-27T15:22:31.756953+0000","last_peered":"2023-11-27T15:22:31.756953+0000","last_clean":"2023-11-27T15:22:31.756953+0000","last_became_active":"2023-11-27T15:20:33.245243+0000","last_became_peered":"2023-11-27T15:20:33.245243+0000","last_unstale":"2023-11-27T15:22:31.756953+0000","last_undegraded":"2023-11-27T15:22:31.756953+0000","last_fullsized":"2023-11-27T15:22:31.756953+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T02:29:54.401852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.753240+0000","last_change":"2023-11-27T15:20:33.245595+0000","last_active":"2023-11-27T15:22:31.753240+0000","last_peered":"2023-11-27T15:22:31.753240+0000","last_clean":"2023-11-27T15:22:31.753240+0000","last_became_active":"2023-11-27T15:20:33.245294+0000","last_became_peered":"2023-11-27T15:20:33.245294+0000","last_unstale":"2023-11-27T15:22:31.753240+0000","last_undegraded":"2023-11-27T15:22:31.753240+0000","last_fullsized":"2023-11-27T15:22:31.753240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T03:10:33.319483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.761634+0000","last_change":"2023-11-27T15:20:33.232976+0000","last_active":"2023-11-27T15:22:31.761634+0000","last_peered":"2023-11-27T15:22:31.761634+0000","last_clean":"2023-11-27T15:22:31.761634+0000","last_became_active":"2023-11-27T15:20:33.232648+0000","last_became_peered":"2023-11-27T15:20:33.232648+0000","last_unstale":"2023-11-27T15:22:31.761634+0000","last_undegraded":"2023-11-27T15:22:31.761634+0000","last_fullsized":"2023-11-27T15:22:31.761634+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T17:21:07.049919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.756893+0000","last_change":"2023-11-27T15:20:33.226332+0000","last_active":"2023-11-27T15:22:31.756893+0000","last_peered":"2023-11-27T15:22:31.756893+0000","last_clean":"2023-11-27T15:22:31.756893+0000","last_became_active":"2023-11-27T15:20:33.226000+0000","last_became_peered":"2023-11-27T15:20:33.226000+0000","last_unstale":"2023-11-27T15:22:31.756893+0000","last_undegraded":"2023-11-27T15:22:31.756893+0000","last_fullsized":"2023-11-27T15:22:31.756893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-28T18:02:35.531072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-27T15:22:31.750943+0000","last_change":"2023-11-27T15:20:33.217927+0000","last_active":"2023-11-27T15:22:31.750943+0000","last_peered":"2023-11-27T15:22:31.750943+0000","last_clean":"2023-11-27T15:22:31.750943+0000","last_became_active":"2023-11-27T15:20:33.217003+0000","last_became_peered":"2023-11-27T15:20:33.217003+0000","last_unstale":"2023-11-27T15:22:31.750943+0000","last_undegraded":"2023-11-27T15:22:31.750943+0000","last_fullsized":"2023-11-27T15:22:31.750943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-27T15:20:32.628101+0000","last_clean_scrub_stamp":"2023-11-27T15:20:32.628101+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T00:51:22.192698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64726,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":619,"num_read_kb":376,"num_write":170,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":204936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":79,"seq":339302416423,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6280,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":93738872,"statfs":{"total":95995035648,"available":95988604928,"internally_reserved":0,"allocated":1048576,"data_stored":514092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.39200000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.55800000000000005}]},{"osd":1,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60599999999999998,"15min":0.60299999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89200000000000002,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60899999999999999,"15min":0.61299999999999999},"min":{"1min":0.35499999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76300000000000001,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.60499999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.30299999999999999,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.59999999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.46100000000000002,"15min":0.45800000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.67200000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.50700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":4,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.86499999999999999,"5min":1.177,"15min":1.177},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61299999999999999,"15min":0.61599999999999999},"min":{"1min":0.40999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.84899999999999998,"5min":1.141,"15min":1.141},"last":0.69999999999999996}]},{"osd":5,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.52700000000000002,"15min":0.53100000000000003},"min":{"1min":0.34399999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.69299999999999995,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.88700000000000001,"5min":1.032,"15min":1.032},"last":0.66100000000000003}]},{"osd":6,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.47499999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.875,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85999999999999999,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.877}]},{"osd":7,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.46500000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97899999999999998,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70999999999999996,"15min":0.71199999999999997},"min":{"1min":0.502,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95699999999999996,"5min":1.54,"15min":1.54},"last":0.85999999999999999}]},{"osd":8,"last update":"Mon Nov 27 15:22:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68700000000000006,"15min":0.68600000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89500000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.90900000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.81599999999999995}]},{"osd":9,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71599999999999997,"15min":0.71799999999999997},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92500000000000004,"5min":1.286,"15min":1.286},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.76900000000000002,"15min":0.76500000000000001},"min":{"1min":0.498,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.84899999999999998}]},{"osd":10,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.53900000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96299999999999997,"5min":1.304,"15min":1.304},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73099999999999998,"15min":0.73299999999999998},"min":{"1min":0.53000000000000003,"5min":0.501,"15min":0.501},"max":{"1min":0.95199999999999996,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.91400000000000003}]}]},{"osd":10,"up_from":71,"seq":304942678058,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7332,"kb_used_data":1692,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93737820,"statfs":{"total":95995035648,"available":95987527680,"internally_reserved":0,"allocated":1732608,"data_stored":1166196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.55800000000000005,"15min":0.53400000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":4.5060000000000002,"5min":4.5060000000000002,"15min":4.5060000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.56699999999999995}]},{"osd":1,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46100000000000002,"15min":0.47399999999999998},"min":{"1min":0.33400000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.63600000000000001,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46100000000000002,"15min":0.46700000000000003},"min":{"1min":0.33500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42299999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.59199999999999997,"15min":0.58399999999999996},"min":{"1min":0.374,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.48599999999999999}]},{"osd":3,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.60699999999999998,"15min":0.63500000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70799999999999996,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63700000000000001,"15min":0.66100000000000003},"min":{"1min":0.45500000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.73399999999999999,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.67300000000000004}]},{"osd":4,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56599999999999995,"15min":0.57999999999999996},"min":{"1min":0.40999999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80200000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.56899999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88700000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61299999999999999}]},{"osd":5,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63900000000000001,"15min":0.626},"min":{"1min":0.45600000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92700000000000005,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.42299999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81499999999999995,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.57899999999999996}]},{"osd":6,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.67200000000000004,"15min":0.68999999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86299999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.70599999999999996,"15min":0.72599999999999998},"min":{"1min":0.54600000000000004,"5min":0.496,"15min":0.496},"max":{"1min":0.85199999999999998,"5min":1.004,"15min":1.004},"last":0.75900000000000001}]},{"osd":7,"last update":"Mon Nov 27 15:20:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.56799999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.73699999999999999,"15min":0.76100000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.876,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.72799999999999998,"15min":0.73399999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97399999999999998,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.69299999999999995}]},{"osd":9,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.76800000000000002,"15min":0.79000000000000004},"min":{"1min":0.58499999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.94199999999999995,"5min":1.111,"15min":1.111},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76000000000000001,"15min":0.77200000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94899999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.71699999999999997}]},{"osd":11,"last update":"Mon Nov 27 15:21:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.746,"15min":0.75},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.77000000000000002,"15min":0.77900000000000003},"min":{"1min":0.56999999999999995,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.92600000000000005,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.79100000000000004}]}]},{"osd":9,"up_from":66,"seq":283467841582,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6480,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738672,"statfs":{"total":95995035648,"available":95988400128,"internally_reserved":0,"allocated":1056768,"data_stored":514216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.503,"15min":0.51900000000000002},"min":{"1min":0.33500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.59899999999999998,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.56999999999999995,"15min":0.57999999999999996},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.35899999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.71099999999999997,"15min":0.73099999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.99199999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64300000000000002,"15min":0.65100000000000002},"min":{"1min":0.44800000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.56499999999999995}]},{"osd":2,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.77000000000000002,"15min":0.79200000000000004},"min":{"1min":0.52800000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.751},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.74199999999999999,"15min":0.77200000000000002},"min":{"1min":0.48299999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.788,"5min":1.788,"15min":1.788},"last":0.66100000000000003}]},{"osd":3,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.66900000000000004,"15min":0.70299999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.73299999999999998,"15min":0.78400000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.127,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.49399999999999999}]},{"osd":4,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.71699999999999997,"15min":0.749},"min":{"1min":0.47199999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.77300000000000002,"15min":0.80600000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.68300000000000005}]},{"osd":5,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.81699999999999995,"15min":0.84499999999999997},"min":{"1min":0.53500000000000003,"5min":0.5,"15min":0.5},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.84699999999999998,"15min":0.88400000000000001},"min":{"1min":0.55800000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.4470000000000001,"5min":1.5369999999999999,"15min":1.5369999999999999},"last":0.78600000000000003}]},{"osd":6,"last update":"Mon Nov 27 15:20:07 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.80400000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.84799999999999998,"15min":0.89800000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.89900000000000002,"15min":0.94299999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.5089999999999999,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.58199999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.90700000000000003,"15min":0.96399999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.8160000000000001,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.91600000000000004,"15min":0.96299999999999997},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.736,"5min":1.736,"15min":1.736},"last":0.77600000000000002}]},{"osd":10,"last update":"Mon Nov 27 15:22:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.88800000000000001,"15min":0.92400000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.5469999999999999,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.626},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.83799999999999997,"15min":0.86499999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.71299999999999997}]},{"osd":11,"last update":"Mon Nov 27 15:22:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.82799999999999996,"15min":0.83899999999999997},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.84099999999999997,"15min":0.85199999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.80700000000000005}]}]},{"osd":8,"up_from":58,"seq":249108103220,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6596,"kb_used_data":1020,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738556,"statfs":{"total":95995035648,"available":95988281344,"internally_reserved":0,"allocated":1044480,"data_stored":513693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59199999999999997,"15min":0.58999999999999997},"min":{"1min":0.44400000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.59799999999999998},"min":{"1min":0.45700000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.75,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.54400000000000004}]},{"osd":1,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.59999999999999998,"15min":0.59899999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.60599999999999998,"15min":0.59699999999999998},"min":{"1min":0.42699999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.69299999999999995}]},{"osd":2,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.44600000000000001,"15min":0.439},"min":{"1min":0.376,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.435},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.46600000000000003,"15min":0.44800000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.66600000000000004,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.47499999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.60599999999999998,"15min":0.60099999999999998},"min":{"1min":0.503,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.45300000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.57499999999999996}]},{"osd":4,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63,"15min":0.628},"min":{"1min":0.46700000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.84699999999999998,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.86599999999999999,"5min":1.377,"15min":1.377},"last":0.67500000000000004}]},{"osd":5,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.59999999999999998,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.57099999999999995,"15min":0.55600000000000005},"min":{"1min":0.39100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60299999999999998}]},{"osd":6,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.73799999999999999,"15min":0.68400000000000005},"min":{"1min":0.47599999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":6.4450000000000003,"5min":6.4450000000000003,"15min":6.4450000000000003},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.72299999999999998,"15min":0.70299999999999996},"min":{"1min":0.55200000000000005,"5min":0.436,"15min":0.436},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.76700000000000002}]},{"osd":7,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.73999999999999999,"15min":0.72199999999999998},"min":{"1min":0.53200000000000003,"5min":0.442,"15min":0.442},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.73699999999999999,"15min":0.73499999999999999},"min":{"1min":0.434,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.99299999999999999,"5min":1.323,"15min":1.323},"last":0.70499999999999996}]},{"osd":9,"last update":"Mon Nov 27 15:22:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.76100000000000001,"15min":0.73999999999999999},"min":{"1min":0.505,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.754},{"interface":"front","average":{"1min":1.181,"5min":0.86699999999999999,"15min":0.80400000000000005},"min":{"1min":0.62,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":6.4619999999999997,"5min":6.4619999999999997,"15min":6.4619999999999997},"last":0.79500000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:22:33 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.74099999999999999,"15min":0.71099999999999997},"min":{"1min":0.52500000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.77600000000000002,"15min":0.74099999999999999},"min":{"1min":0.57799999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.84399999999999997}]},{"osd":11,"last update":"Mon Nov 27 15:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84699999999999998,"15min":0.85099999999999998},"min":{"1min":0.66500000000000004,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.65200000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.73799999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332151,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7316,"kb_used_data":1612,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93737836,"statfs":{"total":95995035648,"available":95987544064,"internally_reserved":0,"allocated":1650688,"data_stored":1110401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.436,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84699999999999998,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.44},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77100000000000002,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.46999999999999997}]},{"osd":1,"last update":"Mon Nov 27 15:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.41199999999999998,"15min":0.39500000000000002},"min":{"1min":0.34499999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.64000000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.42899999999999999,"15min":0.40500000000000003},"min":{"1min":0.35299999999999998,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.495}]},{"osd":2,"last update":"Mon Nov 27 15:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.42199999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.752,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58999999999999997,"15min":0.57199999999999995},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.72399999999999998}]},{"osd":3,"last update":"Mon Nov 27 15:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.48899999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93899999999999995,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64700000000000002,"15min":0.63900000000000001},"min":{"1min":0.48099999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.60999999999999999}]},{"osd":4,"last update":"Mon Nov 27 15:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.52500000000000002,"15min":0.498},"min":{"1min":0.48299999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.73499999999999999,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.438},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51600000000000001,"15min":0.52600000000000002},"min":{"1min":0.39200000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67100000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.628}]},{"osd":5,"last update":"Mon Nov 27 15:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.47499999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.79500000000000004,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.63600000000000001,"15min":0.623},"min":{"1min":0.54800000000000004,"5min":0.434,"15min":0.434},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.58999999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69899999999999995,"15min":0.67700000000000005},"min":{"1min":0.58599999999999997,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.70599999999999996,"15min":0.69999999999999996},"min":{"1min":0.49399999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":0.68899999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:21:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.66900000000000004,"15min":0.65100000000000002},"min":{"1min":0.60399999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.53100000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.73999999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:22:13 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.73399999999999999,"15min":0.72699999999999998},"min":{"1min":0.66100000000000003,"5min":0.442,"15min":0.442},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77300000000000002,"15min":0.77000000000000002},"min":{"1min":0.67300000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.751}]},{"osd":10,"last update":"Mon Nov 27 15:22:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.58499999999999996,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92100000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.68899999999999995,"15min":0.66800000000000004},"min":{"1min":0.59199999999999997,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90900000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.76200000000000001}]},{"osd":11,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76700000000000002,"15min":0.76800000000000002},"min":{"1min":0.66700000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.90600000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79900000000000004,"15min":0.79800000000000004},"min":{"1min":0.64800000000000002,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":0.93300000000000005,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.80700000000000005}]}]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6588,"kb_used_data":1012,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738564,"statfs":{"total":95995035648,"available":95988289536,"internally_reserved":0,"allocated":1036288,"data_stored":509392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.49399999999999999,"15min":0.47599999999999998},"min":{"1min":0.38900000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.89600000000000002,"5min":1.044,"15min":1.044},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.54000000000000004,"15min":0.496},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.42299999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.72899999999999998,"15min":0.66000000000000003},"min":{"1min":0.52000000000000002,"5min":0.441,"15min":0.441},"max":{"1min":4.9109999999999996,"5min":4.9109999999999996,"15min":4.9109999999999996},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.629,"15min":0.58599999999999997},"min":{"1min":0.45500000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90500000000000003,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.56699999999999995}]},{"osd":2,"last update":"Mon Nov 27 15:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.69699999999999995,"15min":0.65900000000000003},"min":{"1min":0.53700000000000003,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.70899999999999996,"15min":0.68300000000000005},"min":{"1min":0.55100000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.137,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.76700000000000002}]},{"osd":3,"last update":"Mon Nov 27 15:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.58799999999999997,"15min":0.55400000000000005},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.64100000000000001,"15min":0.60499999999999998},"min":{"1min":0.42699999999999999,"5min":0.379,"15min":0.379},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.57599999999999996}]},{"osd":4,"last update":"Mon Nov 27 15:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.70199999999999996,"15min":0.68799999999999994},"min":{"1min":0.51900000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0089999999999999,"5min":1.085,"15min":1.085},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69399999999999995,"15min":0.67200000000000004},"min":{"1min":0.55500000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1519999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.63200000000000001}]},{"osd":5,"last update":"Mon Nov 27 15:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.73499999999999999,"15min":0.71299999999999997},"min":{"1min":0.51100000000000001,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.179,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.77900000000000003,"15min":0.75800000000000001},"min":{"1min":0.66400000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2989999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.79100000000000004}]},{"osd":7,"last update":"Mon Nov 27 15:22:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.77300000000000002,"15min":0.753},"min":{"1min":0.59499999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.76200000000000001,"15min":0.71999999999999997},"min":{"1min":0.55800000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.64300000000000002}]},{"osd":8,"last update":"Mon Nov 27 15:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.85799999999999998,"15min":0.86399999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.22,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.84799999999999998,"15min":0.85799999999999998},"min":{"1min":0.60699999999999998,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.74399999999999999}]},{"osd":9,"last update":"Mon Nov 27 15:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.85399999999999998,"15min":0.86799999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.90400000000000003,"15min":0.92400000000000004},"min":{"1min":0.59999999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.2569999999999999,"5min":1.712,"15min":1.712},"last":0.84599999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:22:29 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.83099999999999996,"15min":0.82499999999999996},"min":{"1min":0.56499999999999995,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.90000000000000002,"15min":0.88800000000000001},"min":{"1min":0.72599999999999998,"5min":0.626,"15min":0.626},"max":{"1min":1.3999999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.81200000000000006}]},{"osd":11,"last update":"Mon Nov 27 15:21:38 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":4.9290000000000003,"5min":4.9290000000000003,"15min":4.9290000000000003},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.86599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7680,"kb_used_data":1080,"kb_used_omap":0,"kb_used_meta":6592,"kb_avail":93737472,"statfs":{"total":95995035648,"available":95987171328,"internally_reserved":0,"allocated":1105920,"data_stored":575050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6750208},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.59099999999999997,"15min":0.51300000000000001},"min":{"1min":0.36699999999999999,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":5.2999999999999998,"5min":5.2999999999999998,"15min":5.2999999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.57999999999999996,"15min":0.504},"min":{"1min":0.46300000000000002,"5min":0.36699999999999999,"15min":0.33800000000000002},"max":{"1min":1.0029999999999999,"5min":1.012,"15min":1.012},"last":0.59499999999999997}]},{"osd":2,"last update":"Mon Nov 27 15:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.55500000000000005,"15min":0.498},"min":{"1min":0.38,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.752,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.61399999999999999,"15min":0.51600000000000001},"min":{"1min":0.42699999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.88700000000000001,"5min":0.999,"15min":0.999},"last":0.746}]},{"osd":3,"last update":"Mon Nov 27 15:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57199999999999995,"15min":0.53400000000000003},"min":{"1min":0.39700000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55700000000000005,"15min":0.53700000000000003},"min":{"1min":0.39500000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.57199999999999995}]},{"osd":4,"last update":"Mon Nov 27 15:22:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.46600000000000003,"15min":0.40799999999999997},"min":{"1min":0.34399999999999997,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.73699999999999999,"5min":1.3,"15min":1.3},"last":0.874},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.47999999999999998,"15min":0.42199999999999999},"min":{"1min":0.32800000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.752,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.72299999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.60199999999999998,"15min":0.55600000000000005},"min":{"1min":0.48899999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78800000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.65700000000000003,"15min":0.58299999999999996},"min":{"1min":0.60999999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.93899999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.81000000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.72099999999999997,"15min":0.64900000000000002},"min":{"1min":0.52400000000000002,"5min":0.442,"15min":0.442},"max":{"1min":5.3239999999999998,"5min":5.3239999999999998,"15min":5.3239999999999998},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66100000000000003,"15min":0.626},"min":{"1min":0.47999999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.0289999999999999,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.84699999999999998}]},{"osd":7,"last update":"Mon Nov 27 15:22:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.62,"15min":0.57599999999999996},"min":{"1min":0.505,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0169999999999999,"5min":1.089,"15min":1.089},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.65000000000000002,"15min":0.60399999999999998},"min":{"1min":0.53200000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92100000000000004,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.91800000000000004}]},{"osd":8,"last update":"Mon Nov 27 15:22:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.70199999999999996,"15min":0.65600000000000003},"min":{"1min":0.629,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.68899999999999995,"15min":0.65300000000000002},"min":{"1min":0.55600000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91200000000000003,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.83599999999999997}]},{"osd":9,"last update":"Mon Nov 27 15:22:15 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":0.81799999999999995,"15min":0.76100000000000001},"min":{"1min":0.67500000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":5.3810000000000002,"5min":5.3810000000000002,"15min":5.3810000000000002},"last":0.995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.78000000000000003,"15min":0.76200000000000001},"min":{"1min":0.51600000000000001,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.95199999999999996}]},{"osd":10,"last update":"Mon Nov 27 15:22:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69799999999999995,"15min":0.68899999999999995},"min":{"1min":0.502,"5min":0.439,"15min":0.439},"max":{"1min":0.93400000000000005,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.79200000000000004,"15min":0.77400000000000002},"min":{"1min":0.624,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0429999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.93799999999999994}]},{"osd":11,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.79700000000000004,"15min":0.79300000000000004},"min":{"1min":0.65300000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":0.97699999999999998,"5min":1.024,"15min":1.024},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.88500000000000001,"15min":0.88400000000000001},"min":{"1min":0.73399999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.04,"5min":1.155,"15min":1.155},"last":0.96899999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7508,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737644,"statfs":{"total":95995035648,"available":95987347456,"internally_reserved":0,"allocated":1060864,"data_stored":510780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Nov 27 15:22:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60099999999999998,"15min":0.52000000000000002},"min":{"1min":0.35199999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.63400000000000001,"15min":0.54000000000000004},"min":{"1min":0.40200000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.59699999999999998}]},{"osd":2,"last update":"Mon Nov 27 15:22:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64100000000000001,"15min":0.58699999999999997},"min":{"1min":0.38400000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.95199999999999996,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.66000000000000003,"15min":0.57699999999999996},"min":{"1min":0.53100000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.79300000000000004}]},{"osd":3,"last update":"Mon Nov 27 15:22:34 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63,"15min":0.63},"min":{"1min":0.44600000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.04,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.91400000000000003,"5min":1.204,"15min":1.204},"last":0.54000000000000004}]},{"osd":4,"last update":"Mon Nov 27 15:22:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63900000000000001,"15min":0.61599999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68000000000000005,"15min":0.61699999999999999},"min":{"1min":0.51800000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.996,"5min":1.244,"15min":1.244},"last":0.65700000000000003}]},{"osd":5,"last update":"Mon Nov 27 15:22:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.432,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71999999999999997,"15min":0.68600000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.034,"5min":1.379,"15min":1.379},"last":0.78500000000000003}]},{"osd":6,"last update":"Mon Nov 27 15:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.68200000000000005,"15min":0.63600000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64300000000000002,"15min":0.62},"min":{"1min":0.39600000000000002,"5min":0.373,"15min":0.373},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.67300000000000004}]},{"osd":7,"last update":"Mon Nov 27 15:22:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.76800000000000002,"15min":0.75900000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.62},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.78900000000000003,"15min":0.78700000000000003},"min":{"1min":0.64500000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0880000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":0.81999999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.755,"15min":0.754},"min":{"1min":0.59799999999999998,"5min":0.498,"15min":0.498},"max":{"1min":1.8720000000000001,"5min":1.8720000000000001,"15min":1.8720000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76500000000000001,"15min":0.78100000000000003},"min":{"1min":0.54400000000000004,"5min":0.504,"15min":0.504},"max":{"1min":0.94099999999999995,"5min":1.327,"15min":1.327},"last":0.874}]},{"osd":9,"last update":"Mon Nov 27 15:22:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.86699999999999999,"15min":0.878},"min":{"1min":0.65300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.0920000000000001,"5min":1.218,"15min":1.218},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.85199999999999998,"15min":0.85899999999999999},"min":{"1min":0.63600000000000001,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.86499999999999999}]},{"osd":10,"last update":"Mon Nov 27 15:22:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.81699999999999995,"15min":0.82899999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.056,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88800000000000001,"15min":0.89200000000000002},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1599999999999999,"5min":1.409,"15min":1.409},"last":0.88}]},{"osd":11,"last update":"Mon Nov 27 15:21:54 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.83199999999999996,"15min":0.82699999999999996},"min":{"1min":0.67300000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.82799999999999996,"15min":0.83499999999999996},"min":{"1min":0.59299999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.96499999999999997,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.83399999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7764,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":6656,"kb_avail":93737388,"statfs":{"total":95995035648,"available":95987085312,"internally_reserved":0,"allocated":1126400,"data_stored":575995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6815744},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54400000000000004,"15min":0.51000000000000001},"min":{"1min":0.42699999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.42399999999999999,"5min":0.308,"15min":0.308},"max":{"1min":0.72499999999999998,"5min":0.83799999999999997,"15min":0.85199999999999998},"last":0.623}]},{"osd":1,"last update":"Mon Nov 27 15:22:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.58899999999999997,"15min":0.51100000000000001},"min":{"1min":0.52300000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.84999999999999998,"5min":1.079,"15min":1.079},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.58899999999999997,"15min":0.51700000000000002},"min":{"1min":0.48699999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.79100000000000004,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.76000000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.60699999999999998,"15min":0.57099999999999995},"min":{"1min":0.48999999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.59499999999999997,"15min":0.56899999999999995},"min":{"1min":0.44500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.84099999999999997}]},{"osd":4,"last update":"Mon Nov 27 15:22:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.52100000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66800000000000004,"15min":0.67400000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.84099999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.71299999999999997}]},{"osd":5,"last update":"Mon Nov 27 15:22:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.47999999999999998,"15min":0.438},"min":{"1min":0.35899999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.80400000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.50800000000000001,"15min":0.46400000000000002},"min":{"1min":0.40300000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77600000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.59299999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:22:04 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.64100000000000001,"15min":0.61299999999999999},"min":{"1min":0.48599999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.82899999999999996,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.64100000000000001,"15min":0.60999999999999999},"min":{"1min":0.439,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93700000000000006,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.80400000000000005}]},{"osd":7,"last update":"Mon Nov 27 15:22:40 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71099999999999997,"15min":0.71899999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.80700000000000005,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.55500000000000005,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.88600000000000001}]},{"osd":8,"last update":"Mon Nov 27 15:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.376,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85599999999999998,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.61599999999999999,"15min":0.60499999999999998},"min":{"1min":0.47199999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.88100000000000001,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.82799999999999996}]},{"osd":9,"last update":"Mon Nov 27 15:22:11 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.70399999999999996,"15min":0.67000000000000004},"min":{"1min":0.60399999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.871},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.73199999999999998,"15min":0.70099999999999996},"min":{"1min":0.56200000000000006,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.91800000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:22:33 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.75600000000000001,"15min":0.75},"min":{"1min":0.57499999999999996,"5min":0.44,"15min":0.44},"max":{"1min":0.89500000000000002,"5min":3.2999999999999998,"15min":3.2999999999999998},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.746,"15min":0.69799999999999995},"min":{"1min":0.63,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.038,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.94299999999999995}]},{"osd":11,"last update":"Mon Nov 27 15:22:33 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.84599999999999997,"15min":0.85199999999999998},"min":{"1min":0.504,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.97199999999999998,"5min":1.034,"15min":1.034},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.80700000000000005,"15min":0.80900000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.93200000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.90800000000000003}]}]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6920,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":5888,"kb_avail":93738232,"statfs":{"total":95995035648,"available":95987949568,"internally_reserved":0,"allocated":1048576,"data_stored":509586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":6029312},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.443,"15min":0.40600000000000003},"min":{"1min":0.34899999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.375},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47199999999999998,"15min":0.42199999999999999},"min":{"1min":0.36199999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79400000000000004,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.78200000000000003}]},{"osd":1,"last update":"Mon Nov 27 15:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.63800000000000001,"15min":0.57399999999999995},"min":{"1min":0.54500000000000004,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.878,"5min":1.056,"15min":1.056},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.67700000000000005,"15min":0.58499999999999996},"min":{"1min":0.502,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.907,"5min":5.6529999999999996,"15min":5.6529999999999996},"last":0.436}]},{"osd":2,"last update":"Mon Nov 27 15:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.66500000000000004,"15min":0.60199999999999998},"min":{"1min":0.52300000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81799999999999995,"5min":1.208,"15min":1.208},"last":0.745},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65200000000000002,"15min":0.59499999999999997},"min":{"1min":0.46000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88700000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.72199999999999998}]},{"osd":4,"last update":"Mon Nov 27 15:22:18 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.69099999999999995,"15min":0.64300000000000002},"min":{"1min":0.52100000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":2.8730000000000002,"5min":2.8730000000000002,"15min":2.8730000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72299999999999998,"15min":0.67800000000000005},"min":{"1min":0.55000000000000004,"5min":0.502,"15min":0.502},"max":{"1min":0.83199999999999996,"5min":1.343,"15min":1.343},"last":0.59699999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.70399999999999996,"15min":0.66200000000000003},"min":{"1min":0.61699999999999999,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.90800000000000003,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.69299999999999995,"15min":0.66100000000000003},"min":{"1min":0.58299999999999996,"5min":0.437,"15min":0.437},"max":{"1min":0.89700000000000002,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.68200000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.67900000000000005,"15min":0.64600000000000002},"min":{"1min":0.54600000000000004,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.63600000000000001,"15min":0.60799999999999998},"min":{"1min":0.443,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":2.5920000000000001,"5min":2.5920000000000001,"15min":2.5920000000000001},"last":0.84499999999999997}]},{"osd":7,"last update":"Mon Nov 27 15:22:23 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.93600000000000005,"15min":0.96999999999999997},"min":{"1min":0.63100000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.7959999999999998,"5min":6.9619999999999997,"15min":6.9619999999999997},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.53400000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0940000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.56399999999999995}]},{"osd":8,"last update":"Mon Nov 27 15:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.873},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.9449999999999998,"5min":2.9449999999999998,"15min":2.9449999999999998},"last":0.81499999999999995}]},{"osd":9,"last update":"Mon Nov 27 15:22:09 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.83899999999999997,"15min":0.85399999999999998},"min":{"1min":0.64800000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83399999999999996,"15min":0.84999999999999998},"min":{"1min":0.56799999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.1910000000000001,"5min":1.496,"15min":1.496},"last":0.82699999999999996}]},{"osd":10,"last update":"Mon Nov 27 15:22:28 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.80600000000000005,"15min":0.79200000000000004},"min":{"1min":0.59599999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.839,"5min":2.839,"15min":2.839},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.83299999999999996,"15min":0.83599999999999997},"min":{"1min":0.64000000000000001,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.0209999999999999,"5min":1.129,"15min":1.129},"last":0.83599999999999997}]},{"osd":11,"last update":"Mon Nov 27 15:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.83099999999999996,"15min":0.81999999999999995},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.84899999999999998,"15min":0.84599999999999997},"min":{"1min":0.72899999999999998,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.79100000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6792,"kb_used_data":1024,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738360,"statfs":{"total":95995035648,"available":95988080640,"internally_reserved":0,"allocated":1048576,"data_stored":514525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:20:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.54400000000000004,"15min":0.53700000000000003},"min":{"1min":0.38300000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53500000000000003,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.77900000000000003,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.54900000000000004}]},{"osd":1,"last update":"Mon Nov 27 15:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.40200000000000002,"15min":0.35799999999999998},"min":{"1min":0.26000000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.54700000000000004,"5min":1.337,"15min":1.337},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.371},"min":{"1min":0.28199999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.57399999999999995,"5min":1.476,"15min":1.476},"last":0.36199999999999999}]},{"osd":2,"last update":"Mon Nov 27 15:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57399999999999995,"15min":0.55600000000000005},"min":{"1min":0.38100000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83599999999999997,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60399999999999998,"15min":0.55400000000000005},"min":{"1min":0.39300000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.68999999999999995,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.76100000000000001}]},{"osd":3,"last update":"Mon Nov 27 15:22:19 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59399999999999997,"15min":0.55700000000000005},"min":{"1min":0.42199999999999999,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0109999999999999,"5min":1.423,"15min":1.423},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.59899999999999998},"min":{"1min":0.42099999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.99199999999999999,"5min":1.722,"15min":1.722},"last":0.48099999999999998}]},{"osd":5,"last update":"Mon Nov 27 15:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.63400000000000001,"15min":0.61099999999999999},"min":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.87,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.64100000000000001,"15min":0.60999999999999999},"min":{"1min":0.433,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.94299999999999995,"5min":1.28,"15min":1.28},"last":0.68400000000000005}]},{"osd":6,"last update":"Mon Nov 27 15:22:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88,"5min":1.393,"15min":1.393},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67400000000000004,"15min":0.65900000000000003},"min":{"1min":0.49199999999999999,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.879,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.67200000000000004}]},{"osd":7,"last update":"Mon Nov 27 15:22:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58999999999999997,"15min":0.56799999999999995},"min":{"1min":0.33900000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79900000000000004,"5min":1.054,"15min":1.054},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61599999999999999,"15min":0.59699999999999998},"min":{"1min":0.36099999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.98099999999999998,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.621}]},{"osd":8,"last update":"Mon Nov 27 15:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71399999999999997,"15min":0.70499999999999996},"min":{"1min":0.50600000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.99099999999999999,"5min":1.522,"15min":1.522},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.92700000000000005,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.71299999999999997}]},{"osd":9,"last update":"Mon Nov 27 15:22:11 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.79500000000000004,"15min":0.76700000000000002},"min":{"1min":0.55900000000000005,"5min":0.44,"15min":0.44},"max":{"1min":2.8969999999999998,"5min":2.8969999999999998,"15min":2.8969999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.52100000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.54000000000000004}]},{"osd":10,"last update":"Mon Nov 27 15:22:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73399999999999999,"15min":0.72399999999999998},"min":{"1min":0.53800000000000003,"5min":0.503,"15min":0.503},"max":{"1min":0.91000000000000003,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91800000000000004,"5min":1.03,"15min":1.03},"last":0.65500000000000003}]},{"osd":11,"last update":"Mon Nov 27 15:22:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.81100000000000005},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.96899999999999997,"5min":1.234,"15min":1.234},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72399999999999998,"15min":0.72499999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.97499999999999998,"5min":1.2,"15min":1.2},"last":0.72499999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7448,"kb_used_data":1616,"kb_used_omap":0,"kb_used_meta":5824,"kb_avail":93737704,"statfs":{"total":95995035648,"available":95987408896,"internally_reserved":0,"allocated":1654784,"data_stored":1105175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5963776},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52500000000000002,"15min":0.52300000000000002},"min":{"1min":0.34599999999999997,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.51900000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.36199999999999999}]},{"osd":1,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.46000000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57799999999999996,"15min":0.54300000000000004},"min":{"1min":0.439,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.67100000000000004}]},{"osd":2,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.35199999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.621,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.36799999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48499999999999999}]},{"osd":3,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54900000000000004,"15min":0.55900000000000005},"min":{"1min":0.39600000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79200000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.443},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.57999999999999996,"15min":0.57899999999999996},"min":{"1min":0.44600000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.81699999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.502}]},{"osd":4,"last update":"Mon Nov 27 15:22:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.622,"15min":0.59799999999999998},"min":{"1min":0.48699999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.60099999999999998,"15min":0.58499999999999996},"min":{"1min":0.45500000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.58899999999999997}]},{"osd":6,"last update":"Mon Nov 27 15:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.621,"15min":0.63300000000000001},"min":{"1min":0.48099999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.89900000000000002,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65300000000000002,"15min":0.64500000000000002},"min":{"1min":0.54500000000000004,"5min":0.443,"15min":0.443},"max":{"1min":0.82999999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.57799999999999996}]},{"osd":7,"last update":"Mon Nov 27 15:22:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63400000000000001,"15min":0.64500000000000002},"min":{"1min":0.503,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.76000000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.63},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64300000000000002,"15min":0.64000000000000001},"min":{"1min":0.57099999999999995,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.76300000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.70399999999999996}]},{"osd":8,"last update":"Mon Nov 27 15:22:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.61499999999999999,"15min":0.622},"min":{"1min":0.48799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60099999999999998,"15min":0.60399999999999998},"min":{"1min":0.47699999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.54400000000000004}]},{"osd":9,"last update":"Mon Nov 27 15:22:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71699999999999997,"15min":0.74299999999999999},"min":{"1min":0.51900000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.86499999999999999,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.58799999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.82999999999999996,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.71799999999999997}]},{"osd":10,"last update":"Mon Nov 27 15:22:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.68799999999999994,"15min":0.68000000000000005},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.70799999999999996,"15min":0.69099999999999995},"min":{"1min":0.54700000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.73599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-27T15:22:47.268 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:22:47.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.497+0000 7ffab1dff700 1 -- 172.21.15.133:0/674320828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaac0734f0 msgr2=0x7ffaac073960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:47.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.497+0000 7ffab1dff700 1 --2- 172.21.15.133:0/674320828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaac0734f0 0x7ffaac073960 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffa940098f0 tx=0x7ffa94009c00 comp rx=0 tx=0).stop 2023-11-27T15:22:47.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.497+0000 7ffab1dff700 1 -- 172.21.15.133:0/674320828 shutdown_connections 2023-11-27T15:22:47.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.497+0000 7ffab1dff700 1 --2- 172.21.15.133:0/674320828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaac108140 0x7ffaac10a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.497+0000 7ffab1dff700 1 --2- 172.21.15.133:0/674320828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffaac074ad0 0x7ffaac072f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.497+0000 7ffab1dff700 1 --2- 172.21.15.133:0/674320828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaac0734f0 0x7ffaac073960 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.497+0000 7ffab1dff700 1 -- 172.21.15.133:0/674320828 >> 172.21.15.133:0/674320828 conn(0x7ffaac0fb890 msgr2=0x7ffaac0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:47.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.497+0000 7ffab1dff700 1 -- 172.21.15.133:0/674320828 shutdown_connections 2023-11-27T15:22:47.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.497+0000 7ffab1dff700 1 -- 172.21.15.133:0/674320828 wait complete. 2023-11-27T15:22:47.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.498+0000 7ffab1dff700 1 Processor -- start 2023-11-27T15:22:47.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.498+0000 7ffab1dff700 1 -- start start 2023-11-27T15:22:47.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.498+0000 7ffab1dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaac0734f0 0x7ffaac1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:47.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.499+0000 7ffab1dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffaac074ad0 0x7ffaac198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:47.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.499+0000 7ffab1dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaac108140 0x7ffaac19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:47.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.499+0000 7ffab1dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac19fd50 con 0x7ffaac074ad0 2023-11-27T15:22:47.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.499+0000 7ffab1dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac19fe90 con 0x7ffaac0734f0 2023-11-27T15:22:47.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.499+0000 7ffab1dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac1a0020 con 0x7ffaac108140 2023-11-27T15:22:47.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.499+0000 7ffaa3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffaac074ad0 0x7ffaac198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:47.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.499+0000 7ffaa3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffaac074ad0 0x7ffaac198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60486/0 (socket says 172.21.15.133:60486) 2023-11-27T15:22:47.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.499+0000 7ffaa3fff700 1 -- 172.21.15.133:0/1510354643 learned_addr learned my addr 172.21.15.133:0/1510354643 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:47.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.499+0000 7ffaab7fe700 1 --2- 172.21.15.133:0/1510354643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaac0734f0 0x7ffaac1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:47.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.499+0000 7ffaabfff700 1 --2- 172.21.15.133:0/1510354643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaac108140 0x7ffaac19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:47.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.499+0000 7ffaa3fff700 1 -- 172.21.15.133:0/1510354643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaac108140 msgr2=0x7ffaac19f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:47.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.499+0000 7ffaa3fff700 1 --2- 172.21.15.133:0/1510354643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaac108140 0x7ffaac19f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.499+0000 7ffaa3fff700 1 -- 172.21.15.133:0/1510354643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaac0734f0 msgr2=0x7ffaac1981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:47.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.499+0000 7ffaa3fff700 1 --2- 172.21.15.133:0/1510354643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaac0734f0 0x7ffaac1981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.499+0000 7ffaa3fff700 1 -- 172.21.15.133:0/1510354643 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa94009520 con 0x7ffaac074ad0 2023-11-27T15:22:47.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.499+0000 7ffaab7fe700 1 --2- 172.21.15.133:0/1510354643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaac0734f0 0x7ffaac1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:22:47.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.499+0000 7ffaa3fff700 1 --2- 172.21.15.133:0/1510354643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffaac074ad0 0x7ffaac198710 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009fd0 tx=0x7ffa9400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:47.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.500+0000 7ffaa97fa700 1 -- 172.21.15.133:0/1510354643 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9401c070 con 0x7ffaac074ad0 2023-11-27T15:22:47.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.500+0000 7ffab1dff700 1 -- 172.21.15.133:0/1510354643 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaac10de40 con 0x7ffaac074ad0 2023-11-27T15:22:47.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.500+0000 7ffaa97fa700 1 -- 172.21.15.133:0/1510354643 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa9400f7d0 con 0x7ffaac074ad0 2023-11-27T15:22:47.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.500+0000 7ffaa97fa700 1 -- 172.21.15.133:0/1510354643 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa94004cd0 con 0x7ffaac074ad0 2023-11-27T15:22:47.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.500+0000 7ffab1dff700 1 -- 172.21.15.133:0/1510354643 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaac10dcb0 con 0x7ffaac074ad0 2023-11-27T15:22:47.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.501+0000 7ffab1dff700 1 -- 172.21.15.133:0/1510354643 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaac106880 con 0x7ffaac074ad0 2023-11-27T15:22:47.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.503+0000 7ffaa97fa700 1 -- 172.21.15.133:0/1510354643 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ffa94020070 con 0x7ffaac074ad0 2023-11-27T15:22:47.508 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.505+0000 7ffaa97fa700 1 --2- 172.21.15.133:0/1510354643 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ffa980a1150 0x7ffa980a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:47.509 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.505+0000 7ffaa97fa700 1 -- 172.21.15.133:0/1510354643 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ffa940cd9f0 con 0x7ffaac074ad0 2023-11-27T15:22:47.509 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.505+0000 7ffaab7fe700 1 --2- 172.21.15.133:0/1510354643 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ffa980a1150 0x7ffa980a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:47.509 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.506+0000 7ffaab7fe700 1 --2- 172.21.15.133:0/1510354643 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ffa980a1150 0x7ffa980a3600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c005fd0 tx=0x7ffa9c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:47.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.509+0000 7ffaa97fa700 1 -- 172.21.15.133:0/1510354643 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ffa9405c730 con 0x7ffaac074ad0 2023-11-27T15:22:47.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.710+0000 7ffab1dff700 1 -- 172.21.15.133:0/1510354643 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffaac02cfe0 con 0x7ffaac074ad0 2023-11-27T15:22:47.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.712+0000 7ffaa97fa700 1 -- 172.21.15.133:0/1510354643 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24616 (secure 0 0 0) 0x7ffa94096e80 con 0x7ffaac074ad0 2023-11-27T15:22:47.716 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:47.716 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":92,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:22:02.807005+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:22:47.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.714+0000 7ffab1dff700 1 -- 172.21.15.133:0/1510354643 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ffa980a1150 msgr2=0x7ffa980a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:47.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.714+0000 7ffab1dff700 1 --2- 172.21.15.133:0/1510354643 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ffa980a1150 0x7ffa980a3600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c005fd0 tx=0x7ffa9c009310 comp rx=0 tx=0).stop 2023-11-27T15:22:47.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.714+0000 7ffab1dff700 1 -- 172.21.15.133:0/1510354643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffaac074ad0 msgr2=0x7ffaac198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:47.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.714+0000 7ffab1dff700 1 --2- 172.21.15.133:0/1510354643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffaac074ad0 0x7ffaac198710 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ffa94009fd0 tx=0x7ffa9400b840 comp rx=0 tx=0).stop 2023-11-27T15:22:47.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.715+0000 7ffab1dff700 1 -- 172.21.15.133:0/1510354643 shutdown_connections 2023-11-27T15:22:47.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.715+0000 7ffab1dff700 1 --2- 172.21.15.133:0/1510354643 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ffa980a1150 0x7ffa980a3600 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.715+0000 7ffab1dff700 1 --2- 172.21.15.133:0/1510354643 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffaac108140 0x7ffaac19f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.715+0000 7ffab1dff700 1 --2- 172.21.15.133:0/1510354643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaac0734f0 0x7ffaac1981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.715+0000 7ffab1dff700 1 --2- 172.21.15.133:0/1510354643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffaac074ad0 0x7ffaac198710 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.715+0000 7ffab1dff700 1 -- 172.21.15.133:0/1510354643 >> 172.21.15.133:0/1510354643 conn(0x7ffaac0fb890 msgr2=0x7ffaac0fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:47.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.715+0000 7ffab1dff700 1 -- 172.21.15.133:0/1510354643 shutdown_connections 2023-11-27T15:22:47.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.715+0000 7ffab1dff700 1 -- 172.21.15.133:0/1510354643 wait complete. 2023-11-27T15:22:47.737 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:16:20.216150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:20:31.107146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:20:32.614089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-27T15:20:36.126040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-27T15:22:47.738 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:22:47.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.937+0000 7f1e8e1ff700 1 -- 172.21.15.133:0/2274300555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e88108190 msgr2=0x7f1e8810a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:47.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.937+0000 7f1e8e1ff700 1 --2- 172.21.15.133:0/2274300555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e88108190 0x7f1e8810a610 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000b0a0 tx=0x7f1e8000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.937+0000 7f1e8e1ff700 1 -- 172.21.15.133:0/2274300555 shutdown_connections 2023-11-27T15:22:47.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.937+0000 7f1e8e1ff700 1 --2- 172.21.15.133:0/2274300555 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e88108190 0x7f1e8810a610 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.937+0000 7f1e8e1ff700 1 --2- 172.21.15.133:0/2274300555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e88074b30 0x7f1e88072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.937+0000 7f1e8e1ff700 1 --2- 172.21.15.133:0/2274300555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e88073550 0x7f1e880739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.937+0000 7f1e8e1ff700 1 -- 172.21.15.133:0/2274300555 >> 172.21.15.133:0/2274300555 conn(0x7f1e880fb8b0 msgr2=0x7f1e880fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:47.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.937+0000 7f1e8e1ff700 1 -- 172.21.15.133:0/2274300555 shutdown_connections 2023-11-27T15:22:47.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.937+0000 7f1e8e1ff700 1 -- 172.21.15.133:0/2274300555 wait complete. 2023-11-27T15:22:47.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.938+0000 7f1e8e1ff700 1 Processor -- start 2023-11-27T15:22:47.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.938+0000 7f1e8e1ff700 1 -- start start 2023-11-27T15:22:47.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.938+0000 7f1e8e1ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e88073550 0x7f1e88198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:47.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.938+0000 7f1e8e1ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e88074b30 0x7f1e881988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:47.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.938+0000 7f1e8e1ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e88108190 0x7f1e8819f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:47.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.938+0000 7f1e8e1ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8819ff00 con 0x7f1e88074b30 2023-11-27T15:22:47.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.938+0000 7f1e8e1ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e881a0040 con 0x7f1e88073550 2023-11-27T15:22:47.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.938+0000 7f1e8e1ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e881a01d0 con 0x7f1e88108190 2023-11-27T15:22:47.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.939+0000 7f1e877fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e88073550 0x7f1e88198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:47.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.939+0000 7f1e877fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e88073550 0x7f1e88198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:45208/0 (socket says 172.21.15.133:45208) 2023-11-27T15:22:47.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.939+0000 7f1e877fe700 1 -- 172.21.15.133:0/2352538182 learned_addr learned my addr 172.21.15.133:0/2352538182 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:47.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.939+0000 7f1e86ffd700 1 --2- 172.21.15.133:0/2352538182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e88074b30 0x7f1e881988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:47.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.939+0000 7f1e87fff700 1 --2- 172.21.15.133:0/2352538182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e88108190 0x7f1e8819f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:47.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.939+0000 7f1e877fe700 1 -- 172.21.15.133:0/2352538182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e88108190 msgr2=0x7f1e8819f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:47.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.939+0000 7f1e877fe700 1 --2- 172.21.15.133:0/2352538182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e88108190 0x7f1e8819f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.939+0000 7f1e877fe700 1 -- 172.21.15.133:0/2352538182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e88074b30 msgr2=0x7f1e881988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:47.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.939+0000 7f1e877fe700 1 --2- 172.21.15.133:0/2352538182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e88074b30 0x7f1e881988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:47.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.939+0000 7f1e877fe700 1 -- 172.21.15.133:0/2352538182 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e80009d00 con 0x7f1e88073550 2023-11-27T15:22:47.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.939+0000 7f1e86ffd700 1 --2- 172.21.15.133:0/2352538182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e88074b30 0x7f1e881988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:47.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.939+0000 7f1e87fff700 1 --2- 172.21.15.133:0/2352538182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e88108190 0x7f1e8819f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:47.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.939+0000 7f1e877fe700 1 --2- 172.21.15.133:0/2352538182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e88073550 0x7f1e88198380 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800e970 tx=0x7f1e7800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:47.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.940+0000 7f1e84ff9700 1 -- 172.21.15.133:0/2352538182 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e78009790 con 0x7f1e88073550 2023-11-27T15:22:47.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.940+0000 7f1e8e1ff700 1 -- 172.21.15.133:0/2352538182 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e8810deb0 con 0x7f1e88073550 2023-11-27T15:22:47.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.940+0000 7f1e84ff9700 1 -- 172.21.15.133:0/2352538182 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1e7800c7e0 con 0x7f1e88073550 2023-11-27T15:22:47.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.940+0000 7f1e84ff9700 1 -- 172.21.15.133:0/2352538182 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e780054e0 con 0x7f1e88073550 2023-11-27T15:22:47.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.940+0000 7f1e8e1ff700 1 -- 172.21.15.133:0/2352538182 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e8810dd20 con 0x7f1e88073550 2023-11-27T15:22:47.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.941+0000 7f1e8e1ff700 1 -- 172.21.15.133:0/2352538182 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e88106840 con 0x7f1e88073550 2023-11-27T15:22:47.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.942+0000 7f1e84ff9700 1 -- 172.21.15.133:0/2352538182 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1e7800c950 con 0x7f1e88073550 2023-11-27T15:22:47.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.944+0000 7f1e84ff9700 1 --2- 172.21.15.133:0/2352538182 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1e700a1150 0x7f1e700a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:47.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.944+0000 7f1e84ff9700 1 -- 172.21.15.133:0/2352538182 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1e78014070 con 0x7f1e88073550 2023-11-27T15:22:47.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.944+0000 7f1e86ffd700 1 --2- 172.21.15.133:0/2352538182 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1e700a1150 0x7f1e700a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:47.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.944+0000 7f1e86ffd700 1 --2- 172.21.15.133:0/2352538182 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1e700a1150 0x7f1e700a3600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1e88074850 tx=0x7f1e7c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:47.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:47.946+0000 7f1e84ff9700 1 -- 172.21.15.133:0/2352538182 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1e78096900 con 0x7f1e88073550 2023-11-27T15:22:48.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:47 smithi133 ceph-mon[97860]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:22:48.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:47 smithi133 ceph-mon[97860]: from='client.25414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:22:48.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:47 smithi133 ceph-mon[97860]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:22:48.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:47 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1510354643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:48.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:47 smithi177 ceph-mon[131504]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:22:48.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:47 smithi177 ceph-mon[131504]: from='client.25414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:22:48.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:47 smithi177 ceph-mon[131504]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:22:48.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:47 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1510354643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:48.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.146+0000 7f1e8e1ff700 1 -- 172.21.15.133:0/2352538182 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e88061700 con 0x7f1e88073550 2023-11-27T15:22:48.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.146+0000 7f1e84ff9700 1 -- 172.21.15.133:0/2352538182 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e78099f20 con 0x7f1e88073550 2023-11-27T15:22:48.150 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-11-27T15:22:48.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.149+0000 7f1e8e1ff700 1 -- 172.21.15.133:0/2352538182 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1e700a1150 msgr2=0x7f1e700a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:48.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.149+0000 7f1e8e1ff700 1 --2- 172.21.15.133:0/2352538182 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1e700a1150 0x7f1e700a3600 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1e88074850 tx=0x7f1e7c009260 comp rx=0 tx=0).stop 2023-11-27T15:22:48.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.149+0000 7f1e8e1ff700 1 -- 172.21.15.133:0/2352538182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e88073550 msgr2=0x7f1e88198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:48.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.149+0000 7f1e8e1ff700 1 --2- 172.21.15.133:0/2352538182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e88073550 0x7f1e88198380 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800e970 tx=0x7f1e7800ec80 comp rx=0 tx=0).stop 2023-11-27T15:22:48.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.149+0000 7f1e8e1ff700 1 -- 172.21.15.133:0/2352538182 shutdown_connections 2023-11-27T15:22:48.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.149+0000 7f1e8e1ff700 1 --2- 172.21.15.133:0/2352538182 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1e700a1150 0x7f1e700a3600 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.149+0000 7f1e8e1ff700 1 --2- 172.21.15.133:0/2352538182 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e88108190 0x7f1e8819f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.149+0000 7f1e8e1ff700 1 --2- 172.21.15.133:0/2352538182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e88073550 0x7f1e88198380 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.149+0000 7f1e8e1ff700 1 --2- 172.21.15.133:0/2352538182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e88074b30 0x7f1e881988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.149+0000 7f1e8e1ff700 1 -- 172.21.15.133:0/2352538182 >> 172.21.15.133:0/2352538182 conn(0x7f1e880fb8b0 msgr2=0x7f1e880fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:48.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.149+0000 7f1e8e1ff700 1 -- 172.21.15.133:0/2352538182 shutdown_connections 2023-11-27T15:22:48.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.149+0000 7f1e8e1ff700 1 -- 172.21.15.133:0/2352538182 wait complete. 2023-11-27T15:22:48.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:47 smithi174 ceph-mon[131110]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:22:48.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:47 smithi174 ceph-mon[131110]: from='client.25414 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:22:48.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:47 smithi174 ceph-mon[131110]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-11-27T15:22:48.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:47 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1510354643' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:48.172 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:22:48.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.368+0000 7f36729ff700 1 -- 172.21.15.133:0/3187200044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f366c1002c0 msgr2=0x7f366c1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:48.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.368+0000 7f36729ff700 1 --2- 172.21.15.133:0/3187200044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f366c1002c0 0x7f366c1006d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f365c009990 tx=0x7f365c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.368+0000 7f36729ff700 1 -- 172.21.15.133:0/3187200044 shutdown_connections 2023-11-27T15:22:48.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.368+0000 7f36729ff700 1 --2- 172.21.15.133:0/3187200044 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f366c101e40 0x7f366c108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.368+0000 7f36729ff700 1 --2- 172.21.15.133:0/3187200044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f366c1002c0 0x7f366c1006d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.372 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.368+0000 7f36729ff700 1 --2- 172.21.15.133:0/3187200044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f366c1014b0 0x7f366c101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.368+0000 7f36729ff700 1 -- 172.21.15.133:0/3187200044 >> 172.21.15.133:0/3187200044 conn(0x7f366c0fb8a0 msgr2=0x7f366c0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:48.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.368+0000 7f36729ff700 1 -- 172.21.15.133:0/3187200044 shutdown_connections 2023-11-27T15:22:48.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.368+0000 7f36729ff700 1 -- 172.21.15.133:0/3187200044 wait complete. 2023-11-27T15:22:48.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.369+0000 7f36729ff700 1 Processor -- start 2023-11-27T15:22:48.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.369+0000 7f36729ff700 1 -- start start 2023-11-27T15:22:48.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.370+0000 7f36729ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f366c1002c0 0x7f366c0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:48.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.370+0000 7f36729ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f366c1014b0 0x7f366c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:48.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.370+0000 7f36729ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f366c101e40 0x7f366c073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:48.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.370+0000 7f36729ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c073bc0 con 0x7f366c1014b0 2023-11-27T15:22:48.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.370+0000 7f36729ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c073d30 con 0x7f366c1002c0 2023-11-27T15:22:48.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.370+0000 7f36729ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366c1a00f0 con 0x7f366c101e40 2023-11-27T15:22:48.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.370+0000 7f366b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f366c1014b0 0x7f366c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:48.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.370+0000 7f366b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f366c1014b0 0x7f366c072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60532/0 (socket says 172.21.15.133:60532) 2023-11-27T15:22:48.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.370+0000 7f366b7fe700 1 -- 172.21.15.133:0/1744968436 learned_addr learned my addr 172.21.15.133:0/1744968436 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:48.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.370+0000 7f366bfff700 1 --2- 172.21.15.133:0/1744968436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f366c1002c0 0x7f366c0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:48.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.370+0000 7f3670f9c700 1 --2- 172.21.15.133:0/1744968436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f366c101e40 0x7f366c073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:48.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.370+0000 7f366b7fe700 1 -- 172.21.15.133:0/1744968436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f366c101e40 msgr2=0x7f366c073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:48.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.370+0000 7f366b7fe700 1 --2- 172.21.15.133:0/1744968436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f366c101e40 0x7f366c073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.370+0000 7f366b7fe700 1 -- 172.21.15.133:0/1744968436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f366c1002c0 msgr2=0x7f366c0748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:48.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.370+0000 7f366b7fe700 1 --2- 172.21.15.133:0/1744968436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f366c1002c0 0x7f366c0748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.370+0000 7f366b7fe700 1 -- 172.21.15.133:0/1744968436 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c0095f0 con 0x7f366c1014b0 2023-11-27T15:22:48.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.371+0000 7f366b7fe700 1 --2- 172.21.15.133:0/1744968436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f366c1014b0 0x7f366c072ee0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f366000e9a0 tx=0x7f366000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:48.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.371+0000 7f36697fa700 1 -- 172.21.15.133:0/1744968436 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660009790 con 0x7f366c1014b0 2023-11-27T15:22:48.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.371+0000 7f36697fa700 1 -- 172.21.15.133:0/1744968436 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3660010430 con 0x7f366c1014b0 2023-11-27T15:22:48.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.371+0000 7f36729ff700 1 -- 172.21.15.133:0/1744968436 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f366c10bc90 con 0x7f366c1014b0 2023-11-27T15:22:48.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.371+0000 7f36697fa700 1 -- 172.21.15.133:0/1744968436 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660018680 con 0x7f366c1014b0 2023-11-27T15:22:48.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.371+0000 7f36729ff700 1 -- 172.21.15.133:0/1744968436 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f366c10bb00 con 0x7f366c1014b0 2023-11-27T15:22:48.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.372+0000 7f36729ff700 1 -- 172.21.15.133:0/1744968436 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366c107860 con 0x7f366c1014b0 2023-11-27T15:22:48.377 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.374+0000 7f36697fa700 1 -- 172.21.15.133:0/1744968436 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f36600105a0 con 0x7f366c1014b0 2023-11-27T15:22:48.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.375+0000 7f36697fa700 1 --2- 172.21.15.133:0/1744968436 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f36540a1150 0x7f36540a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:48.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.375+0000 7f36697fa700 1 -- 172.21.15.133:0/1744968436 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f3660014070 con 0x7f366c1014b0 2023-11-27T15:22:48.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.376+0000 7f366bfff700 1 --2- 172.21.15.133:0/1744968436 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f36540a1150 0x7f36540a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:48.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.376+0000 7f366bfff700 1 --2- 172.21.15.133:0/1744968436 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f36540a1150 0x7f36540a3600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f365c00b2e0 tx=0x7f365c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:48.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.379+0000 7f36697fa700 1 -- 172.21.15.133:0/1744968436 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3660051730 con 0x7f366c1014b0 2023-11-27T15:22:48.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.577+0000 7f36729ff700 1 -- 172.21.15.133:0/1744968436 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f366c061700 con 0x7f366c1014b0 2023-11-27T15:22:48.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.577+0000 7f36697fa700 1 -- 172.21.15.133:0/1744968436 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f36600d3080 con 0x7f366c1014b0 2023-11-27T15:22:48.581 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:22:48.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.579+0000 7f36729ff700 1 -- 172.21.15.133:0/1744968436 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f36540a1150 msgr2=0x7f36540a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.579+0000 7f36729ff700 1 --2- 172.21.15.133:0/1744968436 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f36540a1150 0x7f36540a3600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f365c00b2e0 tx=0x7f365c005fb0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.579+0000 7f36729ff700 1 -- 172.21.15.133:0/1744968436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f366c1014b0 msgr2=0x7f366c072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.579+0000 7f36729ff700 1 --2- 172.21.15.133:0/1744968436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f366c1014b0 0x7f366c072ee0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f366000e9a0 tx=0x7f366000ecb0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.580+0000 7f36729ff700 1 -- 172.21.15.133:0/1744968436 shutdown_connections 2023-11-27T15:22:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.580+0000 7f36729ff700 1 --2- 172.21.15.133:0/1744968436 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f36540a1150 0x7f36540a3600 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.580+0000 7f36729ff700 1 --2- 172.21.15.133:0/1744968436 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f366c101e40 0x7f366c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.580+0000 7f36729ff700 1 --2- 172.21.15.133:0/1744968436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f366c1002c0 0x7f366c0748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.580+0000 7f36729ff700 1 --2- 172.21.15.133:0/1744968436 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f366c1014b0 0x7f366c072ee0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.580+0000 7f36729ff700 1 -- 172.21.15.133:0/1744968436 >> 172.21.15.133:0/1744968436 conn(0x7f366c0fb8a0 msgr2=0x7f366c1049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.580+0000 7f36729ff700 1 -- 172.21.15.133:0/1744968436 shutdown_connections 2023-11-27T15:22:48.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.580+0000 7f36729ff700 1 -- 172.21.15.133:0/1744968436 wait complete. 2023-11-27T15:22:48.602 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:22:48.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.804+0000 7fe7a83ff700 1 -- 172.21.15.133:0/2968067800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7a0108180 msgr2=0x7fe7a010a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:48.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.804+0000 7fe7a83ff700 1 --2- 172.21.15.133:0/2968067800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7a0108180 0x7fe7a010a600 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe79800b0a0 tx=0x7fe79800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.804+0000 7fe7a83ff700 1 -- 172.21.15.133:0/2968067800 shutdown_connections 2023-11-27T15:22:48.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.804+0000 7fe7a83ff700 1 --2- 172.21.15.133:0/2968067800 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7a0108180 0x7fe7a010a600 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.804+0000 7fe7a83ff700 1 --2- 172.21.15.133:0/2968067800 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7a0074b30 0x7fe7a0072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.804+0000 7fe7a83ff700 1 --2- 172.21.15.133:0/2968067800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7a0073550 0x7fe7a00739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.804+0000 7fe7a83ff700 1 -- 172.21.15.133:0/2968067800 >> 172.21.15.133:0/2968067800 conn(0x7fe7a00fb8a0 msgr2=0x7fe7a00fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:48.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.804+0000 7fe7a83ff700 1 -- 172.21.15.133:0/2968067800 shutdown_connections 2023-11-27T15:22:48.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.804+0000 7fe7a83ff700 1 -- 172.21.15.133:0/2968067800 wait complete. 2023-11-27T15:22:48.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.805+0000 7fe7a83ff700 1 Processor -- start 2023-11-27T15:22:48.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.805+0000 7fe7a83ff700 1 -- start start 2023-11-27T15:22:48.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.806+0000 7fe7a83ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7a0073550 0x7fe7a0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:48.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.806+0000 7fe7a83ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7a0074b30 0x7fe7a01988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:48.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.806+0000 7fe7a83ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7a0108180 0x7fe7a019f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:48.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.806+0000 7fe7a83ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a019ff10 con 0x7fe7a0073550 2023-11-27T15:22:48.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.806+0000 7fe7a83ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a01a0050 con 0x7fe7a0108180 2023-11-27T15:22:48.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.806+0000 7fe7a83ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a01a01e0 con 0x7fe7a0074b30 2023-11-27T15:22:48.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.806+0000 7fe7a619b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7a0073550 0x7fe7a0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:48.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.806+0000 7fe7a599a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7a0074b30 0x7fe7a01988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:48.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.806+0000 7fe7a599a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7a0074b30 0x7fe7a01988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:46276/0 (socket says 172.21.15.133:46276) 2023-11-27T15:22:48.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.806+0000 7fe7a599a700 1 -- 172.21.15.133:0/1997895134 learned_addr learned my addr 172.21.15.133:0/1997895134 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:48.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.806+0000 7fe7a699c700 1 --2- 172.21.15.133:0/1997895134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7a0108180 0x7fe7a019f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:48.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.806+0000 7fe7a599a700 1 -- 172.21.15.133:0/1997895134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7a0108180 msgr2=0x7fe7a019f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:48.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.806+0000 7fe7a599a700 1 --2- 172.21.15.133:0/1997895134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7a0108180 0x7fe7a019f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.806+0000 7fe7a599a700 1 -- 172.21.15.133:0/1997895134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7a0073550 msgr2=0x7fe7a0198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:48.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.806+0000 7fe7a599a700 1 --2- 172.21.15.133:0/1997895134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7a0073550 0x7fe7a0198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:48.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.806+0000 7fe7a599a700 1 -- 172.21.15.133:0/1997895134 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe798009d00 con 0x7fe7a0074b30 2023-11-27T15:22:48.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.806+0000 7fe7a619b700 1 --2- 172.21.15.133:0/1997895134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7a0073550 0x7fe7a0198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:48.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.807+0000 7fe7a599a700 1 --2- 172.21.15.133:0/1997895134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7a0074b30 0x7fe7a01988d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe79000e970 tx=0x7fe79000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:48.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.807+0000 7fe7977fe700 1 -- 172.21.15.133:0/1997895134 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790009790 con 0x7fe7a0074b30 2023-11-27T15:22:48.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.807+0000 7fe7a83ff700 1 -- 172.21.15.133:0/1997895134 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a010dea0 con 0x7fe7a0074b30 2023-11-27T15:22:48.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.807+0000 7fe7977fe700 1 -- 172.21.15.133:0/1997895134 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe79000c7e0 con 0x7fe7a0074b30 2023-11-27T15:22:48.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.807+0000 7fe7977fe700 1 -- 172.21.15.133:0/1997895134 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe790003c70 con 0x7fe7a0074b30 2023-11-27T15:22:48.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.807+0000 7fe7a83ff700 1 -- 172.21.15.133:0/1997895134 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a010dd10 con 0x7fe7a0074b30 2023-11-27T15:22:48.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.808+0000 7fe7a83ff700 1 -- 172.21.15.133:0/1997895134 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a0106f40 con 0x7fe7a0074b30 2023-11-27T15:22:48.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.810+0000 7fe7977fe700 1 -- 172.21.15.133:0/1997895134 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe79000cc50 con 0x7fe7a0074b30 2023-11-27T15:22:48.815 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.811+0000 7fe7977fe700 1 --2- 172.21.15.133:0/1997895134 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe78c0a1150 0x7fe78c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:48.815 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.811+0000 7fe7977fe700 1 -- 172.21.15.133:0/1997895134 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe790014070 con 0x7fe7a0074b30 2023-11-27T15:22:48.815 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.811+0000 7fe7a619b700 1 --2- 172.21.15.133:0/1997895134 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe78c0a1150 0x7fe78c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:48.815 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.812+0000 7fe7a619b700 1 --2- 172.21.15.133:0/1997895134 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe78c0a1150 0x7fe78c0a3600 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0074f00 tx=0x7fe79c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:48.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:48.813+0000 7fe7977fe700 1 -- 172.21.15.133:0/1997895134 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe7900967f0 con 0x7fe7a0074b30 2023-11-27T15:22:49.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.012+0000 7fe7a83ff700 1 -- 172.21.15.133:0/1997895134 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe7a0061700 con 0x7fe7a0074b30 2023-11-27T15:22:49.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.013+0000 7fe7977fe700 1 -- 172.21.15.133:0/1997895134 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe790003dd0 con 0x7fe7a0074b30 2023-11-27T15:22:49.017 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-11-27T15:22:49.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.015+0000 7fe7a83ff700 1 -- 172.21.15.133:0/1997895134 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe78c0a1150 msgr2=0x7fe78c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.015+0000 7fe7a83ff700 1 --2- 172.21.15.133:0/1997895134 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe78c0a1150 0x7fe78c0a3600 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0074f00 tx=0x7fe79c009260 comp rx=0 tx=0).stop 2023-11-27T15:22:49.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.015+0000 7fe7a83ff700 1 -- 172.21.15.133:0/1997895134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7a0074b30 msgr2=0x7fe7a01988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.015+0000 7fe7a83ff700 1 --2- 172.21.15.133:0/1997895134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7a0074b30 0x7fe7a01988d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe79000e970 tx=0x7fe79000ec80 comp rx=0 tx=0).stop 2023-11-27T15:22:49.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.015+0000 7fe7a83ff700 1 -- 172.21.15.133:0/1997895134 shutdown_connections 2023-11-27T15:22:49.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.015+0000 7fe7a83ff700 1 --2- 172.21.15.133:0/1997895134 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe78c0a1150 0x7fe78c0a3600 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.015+0000 7fe7a83ff700 1 --2- 172.21.15.133:0/1997895134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7a0108180 0x7fe7a019f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.015+0000 7fe7a83ff700 1 --2- 172.21.15.133:0/1997895134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe7a0073550 0x7fe7a0198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.015+0000 7fe7a83ff700 1 --2- 172.21.15.133:0/1997895134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe7a0074b30 0x7fe7a01988d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.016+0000 7fe7a83ff700 1 -- 172.21.15.133:0/1997895134 >> 172.21.15.133:0/1997895134 conn(0x7fe7a00fb8a0 msgr2=0x7fe7a0104080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:49.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.016+0000 7fe7a83ff700 1 -- 172.21.15.133:0/1997895134 shutdown_connections 2023-11-27T15:22:49.021 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.016+0000 7fe7a83ff700 1 -- 172.21.15.133:0/1997895134 wait complete. 2023-11-27T15:22:49.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:48 smithi133 ceph-mon[97860]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-11-27T15:22:49.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:48 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2352538182' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:49.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:48 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1744968436' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:22:49.039 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-27T15:22:49.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:48 smithi177 ceph-mon[131504]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-11-27T15:22:49.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:48 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2352538182' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:49.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:48 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1744968436' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:22:49.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:48 smithi174 ceph-mon[131110]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-11-27T15:22:49.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:48 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2352538182' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:22:49.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:48 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1744968436' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:22:49.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.237+0000 7f0d63dff700 1 -- 172.21.15.133:0/421579304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d5c1002d0 msgr2=0x7f0d5c1006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.237+0000 7f0d63dff700 1 --2- 172.21.15.133:0/421579304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d5c1002d0 0x7f0d5c1006e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009990 tx=0x7f0d58009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.238+0000 7f0d63dff700 1 -- 172.21.15.133:0/421579304 shutdown_connections 2023-11-27T15:22:49.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.238+0000 7f0d63dff700 1 --2- 172.21.15.133:0/421579304 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d5c101e50 0x7f0d5c108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.238+0000 7f0d63dff700 1 --2- 172.21.15.133:0/421579304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d5c1002d0 0x7f0d5c1006e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.238+0000 7f0d63dff700 1 --2- 172.21.15.133:0/421579304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d5c1014c0 0x7f0d5c101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.238+0000 7f0d63dff700 1 -- 172.21.15.133:0/421579304 >> 172.21.15.133:0/421579304 conn(0x7f0d5c0fb8b0 msgr2=0x7f0d5c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:49.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.238+0000 7f0d63dff700 1 -- 172.21.15.133:0/421579304 shutdown_connections 2023-11-27T15:22:49.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.238+0000 7f0d63dff700 1 -- 172.21.15.133:0/421579304 wait complete. 2023-11-27T15:22:49.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.239+0000 7f0d63dff700 1 Processor -- start 2023-11-27T15:22:49.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.239+0000 7f0d63dff700 1 -- start start 2023-11-27T15:22:49.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.239+0000 7f0d63dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d5c1002d0 0x7f0d5c0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:49.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.239+0000 7f0d63dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d5c1014c0 0x7f0d5c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:49.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.239+0000 7f0d63dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d5c101e50 0x7f0d5c073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:49.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.239+0000 7f0d63dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c073bc0 con 0x7f0d5c1014c0 2023-11-27T15:22:49.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.239+0000 7f0d63dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c073d30 con 0x7f0d5c1002d0 2023-11-27T15:22:49.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.239+0000 7f0d63dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c1a0100 con 0x7f0d5c101e50 2023-11-27T15:22:49.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.240+0000 7f0d61b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d5c1002d0 0x7f0d5c0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:49.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.240+0000 7f0d61b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d5c1002d0 0x7f0d5c0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:45260/0 (socket says 172.21.15.133:45260) 2023-11-27T15:22:49.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.240+0000 7f0d61b9b700 1 -- 172.21.15.133:0/2735088085 learned_addr learned my addr 172.21.15.133:0/2735088085 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:49.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.240+0000 7f0d6139a700 1 --2- 172.21.15.133:0/2735088085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d5c1014c0 0x7f0d5c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:49.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.240+0000 7f0d6239c700 1 --2- 172.21.15.133:0/2735088085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d5c101e50 0x7f0d5c073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:49.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.240+0000 7f0d61b9b700 1 -- 172.21.15.133:0/2735088085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d5c101e50 msgr2=0x7f0d5c073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.240+0000 7f0d61b9b700 1 --2- 172.21.15.133:0/2735088085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d5c101e50 0x7f0d5c073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.240+0000 7f0d61b9b700 1 -- 172.21.15.133:0/2735088085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d5c1014c0 msgr2=0x7f0d5c072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.240+0000 7f0d61b9b700 1 --2- 172.21.15.133:0/2735088085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d5c1014c0 0x7f0d5c072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.240+0000 7f0d61b9b700 1 -- 172.21.15.133:0/2735088085 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c009520 con 0x7f0d5c1002d0 2023-11-27T15:22:49.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.240+0000 7f0d61b9b700 1 --2- 172.21.15.133:0/2735088085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d5c1002d0 0x7f0d5c0748a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0d58004730 tx=0x7f0d58004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:49.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.240+0000 7f0d52ffd700 1 -- 172.21.15.133:0/2735088085 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5801d070 con 0x7f0d5c1002d0 2023-11-27T15:22:49.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.241+0000 7f0d52ffd700 1 -- 172.21.15.133:0/2735088085 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0d58021410 con 0x7f0d5c1002d0 2023-11-27T15:22:49.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.241+0000 7f0d52ffd700 1 -- 172.21.15.133:0/2735088085 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d58017650 con 0x7f0d5c1002d0 2023-11-27T15:22:49.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.241+0000 7f0d63dff700 1 -- 172.21.15.133:0/2735088085 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d580095f0 con 0x7f0d5c1002d0 2023-11-27T15:22:49.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.241+0000 7f0d63dff700 1 -- 172.21.15.133:0/2735088085 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5c10bca0 con 0x7f0d5c1002d0 2023-11-27T15:22:49.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.242+0000 7f0d63dff700 1 -- 172.21.15.133:0/2735088085 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d5c067b00 con 0x7f0d5c1002d0 2023-11-27T15:22:49.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.244+0000 7f0d52ffd700 1 -- 172.21.15.133:0/2735088085 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0d58021580 con 0x7f0d5c1002d0 2023-11-27T15:22:49.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.245+0000 7f0d52ffd700 1 --2- 172.21.15.133:0/2735088085 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0d480a11d0 0x7f0d480a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:49.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.245+0000 7f0d52ffd700 1 -- 172.21.15.133:0/2735088085 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f0d580ce6e0 con 0x7f0d5c1002d0 2023-11-27T15:22:49.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.245+0000 7f0d6139a700 1 --2- 172.21.15.133:0/2735088085 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0d480a11d0 0x7f0d480a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:49.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.246+0000 7f0d6139a700 1 --2- 172.21.15.133:0/2735088085 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0d480a11d0 0x7f0d480a3680 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009df0 tx=0x7f0d4c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:49.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.247+0000 7f0d52ffd700 1 -- 172.21.15.133:0/2735088085 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0d58097b10 con 0x7f0d5c1002d0 2023-11-27T15:22:49.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.448+0000 7f0d63dff700 1 -- 172.21.15.133:0/2735088085 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0d5c107200 con 0x7f0d5c1002d0 2023-11-27T15:22:49.452 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.449+0000 7f0d52ffd700 1 -- 172.21.15.133:0/2735088085 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0d5809b130 con 0x7f0d5c1002d0 2023-11-27T15:22:49.453 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:22:49.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.451+0000 7f0d63dff700 1 -- 172.21.15.133:0/2735088085 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0d480a11d0 msgr2=0x7f0d480a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.451+0000 7f0d63dff700 1 --2- 172.21.15.133:0/2735088085 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0d480a11d0 0x7f0d480a3680 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009df0 tx=0x7f0d4c009260 comp rx=0 tx=0).stop 2023-11-27T15:22:49.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.451+0000 7f0d63dff700 1 -- 172.21.15.133:0/2735088085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d5c1002d0 msgr2=0x7f0d5c0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.455 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.451+0000 7f0d63dff700 1 --2- 172.21.15.133:0/2735088085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d5c1002d0 0x7f0d5c0748a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0d58004730 tx=0x7f0d58004760 comp rx=0 tx=0).stop 2023-11-27T15:22:49.456 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.451+0000 7f0d63dff700 1 -- 172.21.15.133:0/2735088085 shutdown_connections 2023-11-27T15:22:49.456 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.451+0000 7f0d63dff700 1 --2- 172.21.15.133:0/2735088085 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0d480a11d0 0x7f0d480a3680 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.456 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.451+0000 7f0d63dff700 1 --2- 172.21.15.133:0/2735088085 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0d5c101e50 0x7f0d5c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.456 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.451+0000 7f0d63dff700 1 --2- 172.21.15.133:0/2735088085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d5c1002d0 0x7f0d5c0748a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.456 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.451+0000 7f0d63dff700 1 --2- 172.21.15.133:0/2735088085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d5c1014c0 0x7f0d5c072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.456 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.451+0000 7f0d63dff700 1 -- 172.21.15.133:0/2735088085 >> 172.21.15.133:0/2735088085 conn(0x7f0d5c0fb8b0 msgr2=0x7f0d5c0fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:49.456 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.451+0000 7f0d63dff700 1 -- 172.21.15.133:0/2735088085 shutdown_connections 2023-11-27T15:22:49.457 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.451+0000 7f0d63dff700 1 -- 172.21.15.133:0/2735088085 wait complete. 2023-11-27T15:22:49.474 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:22:49.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.667+0000 7f73ea3ff700 1 -- 172.21.15.133:0/3805434922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73e4101480 msgr2=0x7f73e41018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.667+0000 7f73ea3ff700 1 --2- 172.21.15.133:0/3805434922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73e4101480 0x7f73e41018d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009990 tx=0x7f73cc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.667+0000 7f73ea3ff700 1 -- 172.21.15.133:0/3805434922 shutdown_connections 2023-11-27T15:22:49.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.667+0000 7f73ea3ff700 1 --2- 172.21.15.133:0/3805434922 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73e4101e10 0x7f73e41086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.667+0000 7f73ea3ff700 1 --2- 172.21.15.133:0/3805434922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73e4100290 0x7f73e41006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.667+0000 7f73ea3ff700 1 --2- 172.21.15.133:0/3805434922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73e4101480 0x7f73e41018d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.667+0000 7f73ea3ff700 1 -- 172.21.15.133:0/3805434922 >> 172.21.15.133:0/3805434922 conn(0x7f73e40fb870 msgr2=0x7f73e40fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:49.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.667+0000 7f73ea3ff700 1 -- 172.21.15.133:0/3805434922 shutdown_connections 2023-11-27T15:22:49.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.667+0000 7f73ea3ff700 1 -- 172.21.15.133:0/3805434922 wait complete. 2023-11-27T15:22:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.668+0000 7f73ea3ff700 1 Processor -- start 2023-11-27T15:22:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.668+0000 7f73ea3ff700 1 -- start start 2023-11-27T15:22:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.668+0000 7f73ea3ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73e4100290 0x7f73e4198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.668+0000 7f73ea3ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73e4101480 0x7f73e4198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.668+0000 7f73ea3ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73e4101e10 0x7f73e419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:49.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.668+0000 7f73ea3ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e419fd80 con 0x7f73e4101480 2023-11-27T15:22:49.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.668+0000 7f73ea3ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e419fec0 con 0x7f73e4101e10 2023-11-27T15:22:49.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.668+0000 7f73ea3ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e41a0050 con 0x7f73e4100290 2023-11-27T15:22:49.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.668+0000 7f73dbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73e4101480 0x7f73e4198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:49.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.668+0000 7f73dbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73e4101480 0x7f73e4198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60576/0 (socket says 172.21.15.133:60576) 2023-11-27T15:22:49.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.668+0000 7f73dbfff700 1 -- 172.21.15.133:0/135775035 learned_addr learned my addr 172.21.15.133:0/135775035 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:49.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.668+0000 7f73e899c700 1 --2- 172.21.15.133:0/135775035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73e4101e10 0x7f73e419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:49.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.669+0000 7f73e3fff700 1 --2- 172.21.15.133:0/135775035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73e4100290 0x7f73e4198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:49.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.669+0000 7f73dbfff700 1 -- 172.21.15.133:0/135775035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73e4100290 msgr2=0x7f73e4198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.669+0000 7f73dbfff700 1 --2- 172.21.15.133:0/135775035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73e4100290 0x7f73e4198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.669+0000 7f73dbfff700 1 -- 172.21.15.133:0/135775035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73e4101e10 msgr2=0x7f73e419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.669+0000 7f73dbfff700 1 --2- 172.21.15.133:0/135775035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73e4101e10 0x7f73e419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.669+0000 7f73dbfff700 1 -- 172.21.15.133:0/135775035 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73cc0095f0 con 0x7f73e4101480 2023-11-27T15:22:49.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.669+0000 7f73dbfff700 1 --2- 172.21.15.133:0/135775035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73e4101480 0x7f73e4198740 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00b620 tx=0x7f73cc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:49.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.669+0000 7f73e1ffb700 1 -- 172.21.15.133:0/135775035 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc01d070 con 0x7f73e4101480 2023-11-27T15:22:49.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.669+0000 7f73ea3ff700 1 -- 172.21.15.133:0/135775035 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e410bc50 con 0x7f73e4101480 2023-11-27T15:22:49.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.669+0000 7f73e1ffb700 1 -- 172.21.15.133:0/135775035 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73cc022470 con 0x7f73e4101480 2023-11-27T15:22:49.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.669+0000 7f73e1ffb700 1 -- 172.21.15.133:0/135775035 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc017650 con 0x7f73e4101480 2023-11-27T15:22:49.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.670+0000 7f73ea3ff700 1 -- 172.21.15.133:0/135775035 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e410bac0 con 0x7f73e4101480 2023-11-27T15:22:49.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.671+0000 7f73ea3ff700 1 -- 172.21.15.133:0/135775035 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e41928e0 con 0x7f73e4101480 2023-11-27T15:22:49.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.673+0000 7f73e1ffb700 1 -- 172.21.15.133:0/135775035 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f73cc0225e0 con 0x7f73e4101480 2023-11-27T15:22:49.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.674+0000 7f73e1ffb700 1 --2- 172.21.15.133:0/135775035 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f73d00a11d0 0x7f73d00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:49.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.674+0000 7f73e1ffb700 1 -- 172.21.15.133:0/135775035 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f73cc0ce1c0 con 0x7f73e4101480 2023-11-27T15:22:49.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.674+0000 7f73e3fff700 1 --2- 172.21.15.133:0/135775035 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f73d00a11d0 0x7f73d00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:49.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.675+0000 7f73e3fff700 1 --2- 172.21.15.133:0/135775035 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f73d00a11d0 0x7f73d00a3680 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f73e41012e0 tx=0x7f73d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:49.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.677+0000 7f73e1ffb700 1 -- 172.21.15.133:0/135775035 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f73cc0975f0 con 0x7f73e4101480 2023-11-27T15:22:49.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.922+0000 7f73ea3ff700 1 -- 172.21.15.133:0/135775035 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f73e402cfe0 con 0x7f73e4101480 2023-11-27T15:22:49.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.922+0000 7f73e1ffb700 1 -- 172.21.15.133:0/135775035 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10829 (secure 0 0 0) 0x7f73cc09ac10 con 0x7f73e4101480 2023-11-27T15:22:49.927 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:49.928 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14493,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2381290259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2381290259},{"type":"v1","addr":"172.21.15.133:6839","nonce":2381290259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1704875895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1704875895},{"type":"v1","addr":"172.21.15.133:6841","nonce":1704875895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:20:33.201023+0000","modified":"2023-11-27T15:21:04.794526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24521,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3492215689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3492215689},{"type":"v1","addr":"172.21.15.177:6837","nonce":3492215689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6839/2781994447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2781994447},{"type":"v1","addr":"172.21.15.174:6839","nonce":2781994447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:22:49.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.925+0000 7f73ea3ff700 1 -- 172.21.15.133:0/135775035 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f73d00a11d0 msgr2=0x7f73d00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.925+0000 7f73ea3ff700 1 --2- 172.21.15.133:0/135775035 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f73d00a11d0 0x7f73d00a3680 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f73e41012e0 tx=0x7f73d4009260 comp rx=0 tx=0).stop 2023-11-27T15:22:49.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.925+0000 7f73ea3ff700 1 -- 172.21.15.133:0/135775035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73e4101480 msgr2=0x7f73e4198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:49.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.925+0000 7f73ea3ff700 1 --2- 172.21.15.133:0/135775035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73e4101480 0x7f73e4198740 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00b620 tx=0x7f73cc005740 comp rx=0 tx=0).stop 2023-11-27T15:22:49.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.925+0000 7f73ea3ff700 1 -- 172.21.15.133:0/135775035 shutdown_connections 2023-11-27T15:22:49.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.925+0000 7f73ea3ff700 1 --2- 172.21.15.133:0/135775035 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f73d00a11d0 0x7f73d00a3680 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.925+0000 7f73ea3ff700 1 --2- 172.21.15.133:0/135775035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f73e4101e10 0x7f73e419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.925+0000 7f73ea3ff700 1 --2- 172.21.15.133:0/135775035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f73e4100290 0x7f73e4198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.925+0000 7f73ea3ff700 1 --2- 172.21.15.133:0/135775035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f73e4101480 0x7f73e4198740 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:49.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.925+0000 7f73ea3ff700 1 -- 172.21.15.133:0/135775035 >> 172.21.15.133:0/135775035 conn(0x7f73e40fb870 msgr2=0x7f73e4106da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:49.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.925+0000 7f73ea3ff700 1 -- 172.21.15.133:0/135775035 shutdown_connections 2023-11-27T15:22:49.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:49.925+0000 7f73ea3ff700 1 -- 172.21.15.133:0/135775035 wait complete. 2023-11-27T15:22:49.933 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 23 2023-11-27T15:22:49.948 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:22:50.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:49 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1997895134' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:22:50.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:49 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2735088085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:22:50.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.141+0000 7f5ea5bff700 1 -- 172.21.15.133:0/3831524390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ea01002b0 msgr2=0x7f5ea01006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.145 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.141+0000 7f5ea5bff700 1 --2- 172.21.15.133:0/3831524390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ea01002b0 0x7f5ea01006c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5e90009990 tx=0x7f5e90009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.141+0000 7f5ea5bff700 1 -- 172.21.15.133:0/3831524390 shutdown_connections 2023-11-27T15:22:50.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.141+0000 7f5ea5bff700 1 --2- 172.21.15.133:0/3831524390 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ea0101e30 0x7f5ea01086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.141+0000 7f5ea5bff700 1 --2- 172.21.15.133:0/3831524390 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ea01002b0 0x7f5ea01006c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.141+0000 7f5ea5bff700 1 --2- 172.21.15.133:0/3831524390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ea01014a0 0x7f5ea01018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.141+0000 7f5ea5bff700 1 -- 172.21.15.133:0/3831524390 >> 172.21.15.133:0/3831524390 conn(0x7f5ea00fb890 msgr2=0x7f5ea00fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:50.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.141+0000 7f5ea5bff700 1 -- 172.21.15.133:0/3831524390 shutdown_connections 2023-11-27T15:22:50.146 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.141+0000 7f5ea5bff700 1 -- 172.21.15.133:0/3831524390 wait complete. 2023-11-27T15:22:50.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.142+0000 7f5ea5bff700 1 Processor -- start 2023-11-27T15:22:50.148 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.142+0000 7f5ea5bff700 1 -- start start 2023-11-27T15:22:50.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.143+0000 7f5ea5bff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ea01002b0 0x7f5ea00748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.143+0000 7f5ea5bff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ea01014a0 0x7f5ea0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.143+0000 7f5ea5bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ea0101e30 0x7f5ea0073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.143+0000 7f5ea5bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea0073bc0 con 0x7f5ea0101e30 2023-11-27T15:22:50.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.143+0000 7f5ea5bff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea0073d30 con 0x7f5ea01002b0 2023-11-27T15:22:50.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.143+0000 7f5ea5bff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea019fff0 con 0x7f5ea01014a0 2023-11-27T15:22:50.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.143+0000 7f5e9f7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ea01002b0 0x7f5ea00748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.143+0000 7f5e9ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ea0101e30 0x7f5ea0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.143+0000 7f5e97fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ea01014a0 0x7f5ea0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.149 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.143+0000 7f5e9ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ea0101e30 0x7f5ea0073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60588/0 (socket says 172.21.15.133:60588) 2023-11-27T15:22:50.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.143+0000 7f5e9ffff700 1 -- 172.21.15.133:0/1964605857 learned_addr learned my addr 172.21.15.133:0/1964605857 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:50.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.143+0000 7f5e9f7fe700 1 -- 172.21.15.133:0/1964605857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ea01014a0 msgr2=0x7f5ea0072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.143+0000 7f5e9f7fe700 1 --2- 172.21.15.133:0/1964605857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ea01014a0 0x7f5ea0072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.143+0000 7f5e9f7fe700 1 -- 172.21.15.133:0/1964605857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ea0101e30 msgr2=0x7f5ea0073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.143+0000 7f5e9f7fe700 1 --2- 172.21.15.133:0/1964605857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ea0101e30 0x7f5ea0073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.143+0000 7f5e9f7fe700 1 -- 172.21.15.133:0/1964605857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e98009c40 con 0x7f5ea01002b0 2023-11-27T15:22:50.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.143+0000 7f5e9ffff700 1 --2- 172.21.15.133:0/1964605857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ea0101e30 0x7f5ea0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:50.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.143+0000 7f5e97fff700 1 --2- 172.21.15.133:0/1964605857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ea01014a0 0x7f5ea0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:50.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.144+0000 7f5e9f7fe700 1 --2- 172.21.15.133:0/1964605857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ea01002b0 0x7f5ea00748a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5e900098e0 tx=0x7f5e90009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:50.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.144+0000 7f5e9d7fa700 1 -- 172.21.15.133:0/1964605857 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9001d070 con 0x7f5ea01002b0 2023-11-27T15:22:50.150 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.144+0000 7f5e9d7fa700 1 -- 172.21.15.133:0/1964605857 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e90021410 con 0x7f5ea01002b0 2023-11-27T15:22:50.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.144+0000 7f5ea5bff700 1 -- 172.21.15.133:0/1964605857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e900095f0 con 0x7f5ea01002b0 2023-11-27T15:22:50.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.144+0000 7f5e9d7fa700 1 -- 172.21.15.133:0/1964605857 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e90005720 con 0x7f5ea01002b0 2023-11-27T15:22:50.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.144+0000 7f5ea5bff700 1 -- 172.21.15.133:0/1964605857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea010bc00 con 0x7f5ea01002b0 2023-11-27T15:22:50.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.145+0000 7f5ea5bff700 1 -- 172.21.15.133:0/1964605857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea010bd90 con 0x7f5ea01002b0 2023-11-27T15:22:50.151 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.147+0000 7f5e9d7fa700 1 -- 172.21.15.133:0/1964605857 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5e90021580 con 0x7f5ea01002b0 2023-11-27T15:22:50.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.148+0000 7f5e9d7fa700 1 --2- 172.21.15.133:0/1964605857 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5e8c0a1150 0x7f5e8c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.148+0000 7f5e9d7fa700 1 -- 172.21.15.133:0/1964605857 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f5e900ce6d0 con 0x7f5ea01002b0 2023-11-27T15:22:50.152 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.148+0000 7f5e97fff700 1 --2- 172.21.15.133:0/1964605857 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5e8c0a1150 0x7f5e8c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:49 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1997895134' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:22:50.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:49 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2735088085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:22:50.153 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.149+0000 7f5e97fff700 1 --2- 172.21.15.133:0/1964605857 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5e8c0a1150 0x7f5e8c0a3600 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5e88011490 tx=0x7f5e88009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:50.154 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.150+0000 7f5e9d7fa700 1 -- 172.21.15.133:0/1964605857 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5e90097b00 con 0x7f5ea01002b0 2023-11-27T15:22:50.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:49 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1997895134' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:22:50.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:49 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2735088085' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:22:50.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.350+0000 7f5ea5bff700 1 -- 172.21.15.133:0/1964605857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ea01a0b20 con 0x7f5ea01002b0 2023-11-27T15:22:50.355 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.352+0000 7f5e9d7fa700 1 -- 172.21.15.133:0/1964605857 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+24616 (secure 0 0 0) 0x7f5e9009b120 con 0x7f5ea01002b0 2023-11-27T15:22:50.356 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:22:50.356 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":92,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:22:02.807005+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:22:50.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.354+0000 7f5ea5bff700 1 -- 172.21.15.133:0/1964605857 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5e8c0a1150 msgr2=0x7f5e8c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.354+0000 7f5ea5bff700 1 --2- 172.21.15.133:0/1964605857 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5e8c0a1150 0x7f5e8c0a3600 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5e88011490 tx=0x7f5e88009260 comp rx=0 tx=0).stop 2023-11-27T15:22:50.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.354+0000 7f5ea5bff700 1 -- 172.21.15.133:0/1964605857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ea01002b0 msgr2=0x7f5ea00748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.354+0000 7f5ea5bff700 1 --2- 172.21.15.133:0/1964605857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ea01002b0 0x7f5ea00748a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5e900098e0 tx=0x7f5e90009580 comp rx=0 tx=0).stop 2023-11-27T15:22:50.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.354+0000 7f5ea5bff700 1 -- 172.21.15.133:0/1964605857 shutdown_connections 2023-11-27T15:22:50.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.354+0000 7f5ea5bff700 1 --2- 172.21.15.133:0/1964605857 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5e8c0a1150 0x7f5e8c0a3600 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.354+0000 7f5ea5bff700 1 --2- 172.21.15.133:0/1964605857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5ea0101e30 0x7f5ea0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.354+0000 7f5ea5bff700 1 --2- 172.21.15.133:0/1964605857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ea01002b0 0x7f5ea00748a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.354+0000 7f5ea5bff700 1 --2- 172.21.15.133:0/1964605857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5ea01014a0 0x7f5ea0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.354+0000 7f5ea5bff700 1 -- 172.21.15.133:0/1964605857 >> 172.21.15.133:0/1964605857 conn(0x7f5ea00fb890 msgr2=0x7f5ea00fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:50.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.354+0000 7f5ea5bff700 1 -- 172.21.15.133:0/1964605857 shutdown_connections 2023-11-27T15:22:50.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.354+0000 7f5ea5bff700 1 -- 172.21.15.133:0/1964605857 wait complete. 2023-11-27T15:22:50.378 DEBUG:tasks.fwd_scrub:Yielding 2023-11-27T15:22:50.378 INFO:teuthology.run_tasks:Running task background_exec... 2023-11-27T15:22:50.388 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi133.front.sepia.ceph.com 2023-11-27T15:22:50.388 DEBUG:teuthology.orchestra.run.smithi133:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-11-27T15:22:50.389 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-11-27T15:22:50.389 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:22:50.389 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:22:50.393 INFO:teuthology.run_tasks:Running task workunit... 2023-11-27T15:22:50.404 INFO:tasks.workunit:Pulling workunits from ref 5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-27T15:22:50.404 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-11-27T15:22:50.404 INFO:tasks.workunit:timeout=3h 2023-11-27T15:22:50.404 INFO:tasks.workunit:cleanup=True 2023-11-27T15:22:50.405 DEBUG:teuthology.orchestra.run.smithi133:> stat -- /home/ubuntu/cephtest/mnt.0 2023-11-27T15:22:50.453 INFO:teuthology.orchestra.run.smithi133.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-11-27T15:22:50.453 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-11-27T15:22:50.453 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 79h/121d Inode: 1099511628283 Links: 2 2023-11-27T15:22:50.453 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-27T15:22:50.453 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:cephfs_t:s0 2023-11-27T15:22:50.454 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2023-11-27 15:21:06.263588831 +0000 2023-11-27T15:22:50.454 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2023-11-27 15:21:06.263588831 +0000 2023-11-27T15:22:50.454 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2023-11-27 15:22:40.527284066 +0000 2023-11-27T15:22:50.454 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2023-11-27 15:21:06.263588831 +0000 2023-11-27T15:22:50.454 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-11-27T15:22:50.455 DEBUG:teuthology.orchestra.run.smithi133:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-11-27T15:22:50.574 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/chrisphoffman/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 5e4a743ad7fd1422ebd6c4dc950a21a303279207 2023-11-27T15:22:50.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.598+0000 7f9588dff700 1 -- 172.21.15.133:0/1583236467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9584073550 msgr2=0x7f95840739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.598+0000 7f9588dff700 1 --2- 172.21.15.133:0/1583236467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9584073550 0x7f95840739c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f95780098f0 tx=0x7f9578009c00 comp rx=0 tx=0).stop 2023-11-27T15:22:50.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.598+0000 7f9588dff700 1 -- 172.21.15.133:0/1583236467 shutdown_connections 2023-11-27T15:22:50.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.598+0000 7f9588dff700 1 --2- 172.21.15.133:0/1583236467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f958410c8a0 0x7f958410ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.598+0000 7f9588dff700 1 --2- 172.21.15.133:0/1583236467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9584074b30 0x7f9584072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.598+0000 7f9588dff700 1 --2- 172.21.15.133:0/1583236467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9584073550 0x7f95840739c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.598+0000 7f9588dff700 1 -- 172.21.15.133:0/1583236467 >> 172.21.15.133:0/1583236467 conn(0x7f95840fb8b0 msgr2=0x7f95840fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:50.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.598+0000 7f9588dff700 1 -- 172.21.15.133:0/1583236467 shutdown_connections 2023-11-27T15:22:50.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.599+0000 7f9588dff700 1 -- 172.21.15.133:0/1583236467 wait complete. 2023-11-27T15:22:50.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.600+0000 7f9588dff700 1 Processor -- start 2023-11-27T15:22:50.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.600+0000 7f9588dff700 1 -- start start 2023-11-27T15:22:50.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.600+0000 7f9588dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9584073550 0x7f958419c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.600+0000 7f9588dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9584074b30 0x7f958419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.600+0000 7f9588dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f958410c8a0 0x7f95841a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.600+0000 7f9588dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95841a42c0 con 0x7f9584074b30 2023-11-27T15:22:50.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.600+0000 7f9588dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95841a4400 con 0x7f9584073550 2023-11-27T15:22:50.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.600+0000 7f9588dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95841a4590 con 0x7f958410c8a0 2023-11-27T15:22:50.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.600+0000 7f958259c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9584073550 0x7f958419c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.600+0000 7f9581d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9584074b30 0x7f958419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.600+0000 7f958259c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9584073550 0x7f958419c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:45310/0 (socket says 172.21.15.133:45310) 2023-11-27T15:22:50.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.600+0000 7f958259c700 1 -- 172.21.15.133:0/482816948 learned_addr learned my addr 172.21.15.133:0/482816948 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:50.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.600+0000 7f9582d9d700 1 --2- 172.21.15.133:0/482816948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f958410c8a0 0x7f95841a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.601+0000 7f958259c700 1 -- 172.21.15.133:0/482816948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f958410c8a0 msgr2=0x7f95841a3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.601+0000 7f958259c700 1 --2- 172.21.15.133:0/482816948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f958410c8a0 0x7f95841a3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.601+0000 7f958259c700 1 -- 172.21.15.133:0/482816948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9584074b30 msgr2=0x7f958419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.601+0000 7f958259c700 1 --2- 172.21.15.133:0/482816948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9584074b30 0x7f958419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.601+0000 7f958259c700 1 -- 172.21.15.133:0/482816948 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9578009520 con 0x7f9584073550 2023-11-27T15:22:50.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.601+0000 7f9581d9b700 1 --2- 172.21.15.133:0/482816948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9584074b30 0x7f958419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:50.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.601+0000 7f958259c700 1 --2- 172.21.15.133:0/482816948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9584073550 0x7f958419c740 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f957400e970 tx=0x7f957400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:50.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.601+0000 7f95737fe700 1 -- 172.21.15.133:0/482816948 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9574009790 con 0x7f9584073550 2023-11-27T15:22:50.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.601+0000 7f9588dff700 1 -- 172.21.15.133:0/482816948 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95841122b0 con 0x7f9584073550 2023-11-27T15:22:50.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.601+0000 7f95737fe700 1 -- 172.21.15.133:0/482816948 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f957400c7e0 con 0x7f9584073550 2023-11-27T15:22:50.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.601+0000 7f95737fe700 1 -- 172.21.15.133:0/482816948 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95740054e0 con 0x7f9584073550 2023-11-27T15:22:50.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.602+0000 7f9588dff700 1 -- 172.21.15.133:0/482816948 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9584112120 con 0x7f9584073550 2023-11-27T15:22:50.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.605+0000 7f95737fe700 1 -- 172.21.15.133:0/482816948 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f957400c950 con 0x7f9584073550 2023-11-27T15:22:50.609 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.606+0000 7f959c766b80 1 Processor -- start 2023-11-27T15:22:50.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.606+0000 7f959c766b80 1 -- start start 2023-11-27T15:22:50.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.607+0000 7f959c766b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e1fc80 0x563ef9e20090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.607+0000 7f959c766b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e205d0 0x563ef9e20bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.607+0000 7f959c766b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e21190 0x563ef9e21600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.607+0000 7f959c766b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef9d250e0 con 0x563ef9e21190 2023-11-27T15:22:50.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.607+0000 7f959c766b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef9d77920 con 0x563ef9e205d0 2023-11-27T15:22:50.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.607+0000 7f959c766b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef9d70db0 con 0x563ef9e1fc80 2023-11-27T15:22:50.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.607+0000 7f95737fe700 1 --2- 172.21.15.133:0/482816948 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f956c0a1150 0x7f956c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.607+0000 7f95737fe700 1 -- 172.21.15.133:0/482816948 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f9574014070 con 0x7f9584073550 2023-11-27T15:22:50.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.607+0000 7f958259c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e205d0 0x563ef9e20bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.607+0000 7f9582d9d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e1fc80 0x563ef9e20090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.607+0000 7f958259c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e205d0 0x563ef9e20bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:45314/0 (socket says 172.21.15.133:45314) 2023-11-27T15:22:50.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.607+0000 7f958259c700 1 -- 172.21.15.133:0/2085409524 learned_addr learned my addr 172.21.15.133:0/2085409524 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:50.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.607+0000 7f9581d9b700 1 --2- 172.21.15.133:0/2085409524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e21190 0x563ef9e21600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.607+0000 7f9582d9d700 1 --2- 172.21.15.133:0/482816948 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f956c0a1150 0x7f956c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.608+0000 7f958259c700 1 -- 172.21.15.133:0/2085409524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e1fc80 msgr2=0x563ef9e20090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:50.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.608+0000 7f958259c700 1 --2- 172.21.15.133:0/2085409524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e1fc80 0x563ef9e20090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.608+0000 7f958259c700 1 -- 172.21.15.133:0/2085409524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e21190 msgr2=0x563ef9e21600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:50.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.608+0000 7f958259c700 1 --2- 172.21.15.133:0/2085409524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e21190 0x563ef9e21600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.608+0000 7f958259c700 1 -- 172.21.15.133:0/2085409524 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ef9bf26b0 con 0x563ef9e205d0 2023-11-27T15:22:50.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.608+0000 7f9582d9d700 1 --2- 172.21.15.133:0/2085409524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e1fc80 0x563ef9e20090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:50.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.608+0000 7f958259c700 1 --2- 172.21.15.133:0/2085409524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e205d0 0x563ef9e20bb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9574005950 tx=0x7f9574099e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebf6b980764224d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:50.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.608+0000 7f9582d9d700 1 --2- 172.21.15.133:0/482816948 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f956c0a1150 0x7f956c0a3600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f957c0060b0 tx=0x7f957c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:50.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.608+0000 7f9570ff9700 1 -- 172.21.15.133:0/2085409524 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957409df60 con 0x563ef9e205d0 2023-11-27T15:22:50.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.608+0000 7f9570ff9700 1 -- 172.21.15.133:0/2085409524 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95740a2380 con 0x563ef9e205d0 2023-11-27T15:22:50.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.608+0000 7f9570ff9700 1 -- 172.21.15.133:0/2085409524 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95740a9580 con 0x563ef9e205d0 2023-11-27T15:22:50.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.609+0000 7f959c766b80 1 -- 172.21.15.133:0/2085409524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e205d0 msgr2=0x563ef9e20bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.609+0000 7f959c766b80 1 --2- 172.21.15.133:0/2085409524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e205d0 0x563ef9e20bb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9574005950 tx=0x7f9574099e10 comp rx=0 tx=0).stop 2023-11-27T15:22:50.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.609+0000 7f959c766b80 1 -- 172.21.15.133:0/2085409524 shutdown_connections 2023-11-27T15:22:50.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.609+0000 7f959c766b80 1 --2- 172.21.15.133:0/2085409524 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e21190 0x563ef9e21600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.609+0000 7f959c766b80 1 --2- 172.21.15.133:0/2085409524 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e1fc80 0x563ef9e20090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.609+0000 7f959c766b80 1 --2- 172.21.15.133:0/2085409524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e205d0 0x563ef9e20bb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.609+0000 7f959c766b80 1 -- 172.21.15.133:0/2085409524 >> 172.21.15.133:0/2085409524 conn(0x563ef9e153e0 msgr2=0x563ef9e1a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:50.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.609+0000 7f959c766b80 1 -- 172.21.15.133:0/2085409524 shutdown_connections 2023-11-27T15:22:50.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.609+0000 7f959c766b80 1 -- 172.21.15.133:0/2085409524 wait complete. 2023-11-27T15:22:50.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.610+0000 7f959c766b80 1 -- start start 2023-11-27T15:22:50.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.610+0000 7f959c766b80 1 Processor -- start 2023-11-27T15:22:50.619 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.610+0000 7f959c766b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e1fc80 0x563ef9e29690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.610+0000 7f959c766b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e205d0 0x563ef9e29bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.610+0000 7f959c766b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e21190 0x563ef9e30c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.610+0000 7f959c766b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef9d12960 con 0x563ef9e1fc80 2023-11-27T15:22:50.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.610+0000 7f959c766b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef9d12600 con 0x563ef9e21190 2023-11-27T15:22:50.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.610+0000 7f959c766b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef9bf49a0 con 0x563ef9e205d0 2023-11-27T15:22:50.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.611+0000 7f959c766b80 10 client.0 _ll_register_callbacks cb 0x563ef9d30800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:22:50.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.611+0000 7f958259c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e1fc80 0x563ef9e29690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.611+0000 7f9581d9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e205d0 0x563ef9e29bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.611+0000 7f958259c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e1fc80 0x563ef9e29690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60636/0 (socket says 172.21.15.133:60636) 2023-11-27T15:22:50.620 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.611+0000 7f958259c700 1 -- 172.21.15.133:0/4249767786 learned_addr learned my addr 172.21.15.133:0/4249767786 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:50.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.611+0000 7f9582d9d700 1 --2- 172.21.15.133:0/4249767786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e21190 0x563ef9e30c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.611+0000 7f9581d9b700 1 -- 172.21.15.133:0/4249767786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e21190 msgr2=0x563ef9e30c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:50.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.611+0000 7f9581d9b700 1 --2- 172.21.15.133:0/4249767786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e21190 0x563ef9e30c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.611+0000 7f9581d9b700 1 -- 172.21.15.133:0/4249767786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e1fc80 msgr2=0x563ef9e29690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:50.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.611+0000 7f9581d9b700 1 --2- 172.21.15.133:0/4249767786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e1fc80 0x563ef9e29690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.611+0000 7f9581d9b700 1 -- 172.21.15.133:0/4249767786 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9584112120 con 0x563ef9e205d0 2023-11-27T15:22:50.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.612+0000 7f9581d9b700 1 --2- 172.21.15.133:0/4249767786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e205d0 0x563ef9e29bd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9578000c00 tx=0x7f957800b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2238d291782df8f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:50.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.612+0000 7f9569ffb700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:22:50.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.612+0000 7f9569ffb700 1 -- 172.21.15.133:0/4249767786 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957801c040 con 0x563ef9e205d0 2023-11-27T15:22:50.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.612+0000 7f959c766b80 1 -- 172.21.15.133:0/4249767786 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ef9e12ba0 con 0x563ef9e205d0 2023-11-27T15:22:50.621 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.612+0000 7f9569ffb700 1 -- 172.21.15.133:0/4249767786 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f957800bbc0 con 0x563ef9e205d0 2023-11-27T15:22:50.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.612+0000 7f9569ffb700 1 -- 172.21.15.133:0/4249767786 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9578005240 con 0x563ef9e205d0 2023-11-27T15:22:50.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.612+0000 7f959c766b80 10 client.25474 fetch_fsmap learned FSMap version 23 2023-11-27T15:22:50.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.612+0000 7f959c766b80 1 -- 172.21.15.133:0/4249767786 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563ef9d37210 con 0x563ef9e205d0 2023-11-27T15:22:50.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.613+0000 7f9569ffb700 1 -- 172.21.15.133:0/4249767786 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563ef9e12ba0 con 0x563ef9e205d0 2023-11-27T15:22:50.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.613+0000 7f9569ffb700 1 -- 172.21.15.133:0/4249767786 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9578015020 con 0x563ef9e205d0 2023-11-27T15:22:50.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.613+0000 7f959c766b80 10 client.25474 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:22:50.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.613+0000 7f959c766b80 10 client.25474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:22:50.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.613+0000 7f959c766b80 20 client.25474 populate_metadata read hostname 'smithi133' 2023-11-27T15:22:50.622 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.613+0000 7f959c766b80 1 --2- 172.21.15.133:0/4249767786 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x563ef9e35430 0x563ef9e37840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.613+0000 7f959c766b80 4 client.25474 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:22:50.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.613+0000 7f959c766b80 1 -- 172.21.15.133:0/4249767786 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ef9d33120 con 0x563ef9e35430 2023-11-27T15:22:50.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.613+0000 7f958259c700 1 --2- 172.21.15.133:0/4249767786 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x563ef9e35430 0x563ef9e37840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.614+0000 7f958259c700 1 --2- 172.21.15.133:0/4249767786 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x563ef9e35430 0x563ef9e37840 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=278ba825e56ab8d9 server_cookie=64ebebb2e5d028dc in_seq=0 out_seq=0 2023-11-27T15:22:50.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.614+0000 7f9569ffb700 10 client.25474 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:22:50.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.615+0000 7f9569ffb700 1 -- 172.21.15.133:0/4249767786 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563ef9d33120 con 0x563ef9e35430 2023-11-27T15:22:50.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.615+0000 7f9569ffb700 10 client.25474 handle_command_reply: tid=0 2023-11-27T15:22:50.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.615+0000 7f959c766b80 1 client.25474 shutdown 2023-11-27T15:22:50.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.615+0000 7f959c766b80 1 -- 172.21.15.133:0/4249767786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e205d0 msgr2=0x563ef9e29bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.615+0000 7f959c766b80 1 --2- 172.21.15.133:0/4249767786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e205d0 0x563ef9e29bd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9578000c00 tx=0x7f957800b790 comp rx=0 tx=0).stop 2023-11-27T15:22:50.623 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.615+0000 7f959c766b80 1 -- 172.21.15.133:0/4249767786 shutdown_connections 2023-11-27T15:22:50.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.615+0000 7f959c766b80 1 --2- 172.21.15.133:0/4249767786 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x563ef9e35430 0x563ef9e37840 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.615+0000 7f959c766b80 1 --2- 172.21.15.133:0/4249767786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e21190 0x563ef9e30c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.615+0000 7f959c766b80 1 --2- 172.21.15.133:0/4249767786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e1fc80 0x563ef9e29690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.615+0000 7f9569ffb700 0 client.25474 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:22:50.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.615+0000 7f959c766b80 1 --2- 172.21.15.133:0/4249767786 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e205d0 0x563ef9e29bd0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.615+0000 7f959c766b80 1 -- 172.21.15.133:0/4249767786 >> 172.21.15.133:0/4249767786 conn(0x563ef9e153e0 msgr2=0x563ef9e19e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:50.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.615+0000 7f959c766b80 1 -- 172.21.15.133:0/4249767786 shutdown_connections 2023-11-27T15:22:50.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.615+0000 7f959c766b80 1 -- 172.21.15.133:0/4249767786 wait complete. 2023-11-27T15:22:50.624 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.615+0000 7f959c766b80 20 client.25474 trim_cache size 0 max 16384 2023-11-27T15:22:50.625 INFO:tasks.workunit.client.0.smithi133.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-11-27T15:22:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.633+0000 7f959c766b80 1 Processor -- start 2023-11-27T15:22:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.633+0000 7f959c766b80 1 -- start start 2023-11-27T15:22:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.633+0000 7f959c766b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e1fc80 0x563ef9e29b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.633+0000 7f959c766b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e205d0 0x563ef9e30bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.633+0000 7f959c766b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e21190 0x563ef9e195d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.633+0000 7f959c766b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef9e21010 con 0x563ef9e1fc80 2023-11-27T15:22:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.633+0000 7f959c766b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef9e28b40 con 0x563ef9e205d0 2023-11-27T15:22:50.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.633+0000 7f959c766b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef9e29200 con 0x563ef9e21190 2023-11-27T15:22:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.634+0000 7f958259c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e1fc80 0x563ef9e29b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.634+0000 7f9581d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e205d0 0x563ef9e30bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.634+0000 7f9581d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e205d0 0x563ef9e30bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:45340/0 (socket says 172.21.15.133:45340) 2023-11-27T15:22:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.634+0000 7f9582d9d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e21190 0x563ef9e195d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.634+0000 7f958259c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e1fc80 0x563ef9e29b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60646/0 (socket says 172.21.15.133:60646) 2023-11-27T15:22:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.634+0000 7f9581d9b700 1 -- 172.21.15.133:0/2420942400 learned_addr learned my addr 172.21.15.133:0/2420942400 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.634+0000 7f9581d9b700 1 -- 172.21.15.133:0/2420942400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e21190 msgr2=0x563ef9e195d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.634+0000 7f9581d9b700 1 --2- 172.21.15.133:0/2420942400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e21190 0x563ef9e195d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.634+0000 7f9581d9b700 1 -- 172.21.15.133:0/2420942400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e1fc80 msgr2=0x563ef9e29b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.634+0000 7f9581d9b700 1 --2- 172.21.15.133:0/2420942400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e1fc80 0x563ef9e29b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.634+0000 7f9581d9b700 1 -- 172.21.15.133:0/2420942400 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ef9d36f60 con 0x563ef9e205d0 2023-11-27T15:22:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.634+0000 7f9581d9b700 1 --2- 172.21.15.133:0/2420942400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e205d0 0x563ef9e30bd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9578005fa0 tx=0x7f9578009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ca286ab71e50f19 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:50.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.634+0000 7f95717fa700 1 -- 172.21.15.133:0/2420942400 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957801c070 con 0x563ef9e205d0 2023-11-27T15:22:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.635+0000 7f95717fa700 1 -- 172.21.15.133:0/2420942400 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f957801d040 con 0x563ef9e205d0 2023-11-27T15:22:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.635+0000 7f95717fa700 1 -- 172.21.15.133:0/2420942400 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578005110 con 0x563ef9e205d0 2023-11-27T15:22:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.635+0000 7f959c766b80 1 -- 172.21.15.133:0/2420942400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e205d0 msgr2=0x563ef9e30bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.635+0000 7f959c766b80 1 --2- 172.21.15.133:0/2420942400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e205d0 0x563ef9e30bd0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9578005fa0 tx=0x7f9578009c00 comp rx=0 tx=0).stop 2023-11-27T15:22:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.635+0000 7f959c766b80 1 -- 172.21.15.133:0/2420942400 shutdown_connections 2023-11-27T15:22:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.635+0000 7f959c766b80 1 --2- 172.21.15.133:0/2420942400 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e21190 0x563ef9e195d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.635+0000 7f959c766b80 1 --2- 172.21.15.133:0/2420942400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e1fc80 0x563ef9e29b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.635+0000 7f959c766b80 1 --2- 172.21.15.133:0/2420942400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e205d0 0x563ef9e30bd0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.635+0000 7f959c766b80 1 -- 172.21.15.133:0/2420942400 >> 172.21.15.133:0/2420942400 conn(0x563ef9e153e0 msgr2=0x563ef9e29440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.635+0000 7f959c766b80 1 -- 172.21.15.133:0/2420942400 shutdown_connections 2023-11-27T15:22:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.635+0000 7f959c766b80 1 -- 172.21.15.133:0/2420942400 wait complete. 2023-11-27T15:22:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.636+0000 7f959c766b80 1 -- start start 2023-11-27T15:22:50.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.636+0000 7f959c766b80 1 Processor -- start 2023-11-27T15:22:50.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.636+0000 7f959c766b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e1fc80 0x563ef9e30830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.636+0000 7f959c766b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e205d0 0x563ef9e2d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.636+0000 7f959c766b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e21190 0x563ef9e2d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.636+0000 7f959c766b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef9e2dee0 con 0x563ef9e1fc80 2023-11-27T15:22:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.636+0000 7f959c766b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef9e2e050 con 0x563ef9e21190 2023-11-27T15:22:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.636+0000 7f959c766b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ef9e892e0 con 0x563ef9e205d0 2023-11-27T15:22:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.636+0000 7f959c766b80 10 client.0 _ll_register_callbacks cb 0x7f957800f7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:22:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.637+0000 7f9581d9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e205d0 0x563ef9e2d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.637+0000 7f958259c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e1fc80 0x563ef9e30830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.637+0000 7f9581d9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e205d0 0x563ef9e2d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:46386/0 (socket says 172.21.15.133:46386) 2023-11-27T15:22:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.637+0000 7f9581d9b700 1 -- 172.21.15.133:0/4033789808 learned_addr learned my addr 172.21.15.133:0/4033789808 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:50.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.637+0000 7f9582d9d700 1 --2- 172.21.15.133:0/4033789808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e21190 0x563ef9e2d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.637+0000 7f9581d9b700 1 -- 172.21.15.133:0/4033789808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e21190 msgr2=0x563ef9e2d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:50.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.637+0000 7f9581d9b700 1 --2- 172.21.15.133:0/4033789808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e21190 0x563ef9e2d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.637+0000 7f9581d9b700 1 -- 172.21.15.133:0/4033789808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e1fc80 msgr2=0x563ef9e30830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:50.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.637+0000 7f9581d9b700 1 --2- 172.21.15.133:0/4033789808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e1fc80 0x563ef9e30830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.637+0000 7f9581d9b700 1 -- 172.21.15.133:0/4033789808 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ef9d37210 con 0x563ef9e205d0 2023-11-27T15:22:50.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.637+0000 7f958259c700 1 --2- 172.21.15.133:0/4033789808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e1fc80 0x563ef9e30830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:50.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.637+0000 7f9581d9b700 1 --2- 172.21.15.133:0/4033789808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e205d0 0x563ef9e2d460 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f957800fa40 tx=0x7f9578022610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bf0fc2124aec6f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:50.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.637+0000 7f9569ffb700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:22:50.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.638+0000 7f9569ffb700 1 -- 172.21.15.133:0/4033789808 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957801c040 con 0x563ef9e205d0 2023-11-27T15:22:50.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.638+0000 7f959c766b80 1 -- 172.21.15.133:0/4033789808 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95740a2380 con 0x563ef9e205d0 2023-11-27T15:22:50.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.638+0000 7f9569ffb700 1 -- 172.21.15.133:0/4033789808 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f9578000e40 con 0x563ef9e205d0 2023-11-27T15:22:50.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.638+0000 7f9569ffb700 1 -- 172.21.15.133:0/4033789808 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95780228b0 con 0x563ef9e205d0 2023-11-27T15:22:50.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.638+0000 7f959c766b80 10 client.25480 fetch_fsmap learned FSMap version 23 2023-11-27T15:22:50.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.638+0000 7f959c766b80 1 -- 172.21.15.133:0/4033789808 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563ef9b87ce0 con 0x563ef9e205d0 2023-11-27T15:22:50.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.638+0000 7f9569ffb700 1 -- 172.21.15.133:0/4033789808 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95740a2380 con 0x563ef9e205d0 2023-11-27T15:22:50.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.638+0000 7f9569ffb700 1 -- 172.21.15.133:0/4033789808 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9578015040 con 0x563ef9e205d0 2023-11-27T15:22:50.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.638+0000 7f959c766b80 10 client.25480 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:22:50.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.638+0000 7f959c766b80 10 client.25480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:22:50.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.638+0000 7f959c766b80 20 client.25480 populate_metadata read hostname 'smithi133' 2023-11-27T15:22:50.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.638+0000 7f959c766b80 1 --2- 172.21.15.133:0/4033789808 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x563ef9e8a830 0x563ef9e8cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.638+0000 7f959c766b80 4 client.25480 mds_command: new command op to 24560 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:22:50.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.638+0000 7f959c766b80 1 -- 172.21.15.133:0/4033789808 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563ef9d3f6e0 con 0x563ef9e8a830 2023-11-27T15:22:50.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.639+0000 7f958259c700 1 --2- 172.21.15.133:0/4033789808 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x563ef9e8a830 0x563ef9e8cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.639+0000 7f958259c700 1 --2- 172.21.15.133:0/4033789808 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x563ef9e8a830 0x563ef9e8cc40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4e21e9e7f0dcfe server_cookie=b8fd2ac30612f04f in_seq=0 out_seq=0 2023-11-27T15:22:50.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.639+0000 7f9569ffb700 10 client.25480 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:22:50.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.640+0000 7f9569ffb700 1 -- 172.21.15.133:0/4033789808 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563ef9d3f6e0 con 0x563ef9e8a830 2023-11-27T15:22:50.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.640+0000 7f9569ffb700 10 client.25480 handle_command_reply: tid=0 2023-11-27T15:22:50.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.640+0000 7f959c766b80 1 client.25480 shutdown 2023-11-27T15:22:50.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.640+0000 7f959c766b80 1 -- 172.21.15.133:0/4033789808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e205d0 msgr2=0x563ef9e2d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.640+0000 7f959c766b80 1 --2- 172.21.15.133:0/4033789808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e205d0 0x563ef9e2d460 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f957800fa40 tx=0x7f9578022610 comp rx=0 tx=0).stop 2023-11-27T15:22:50.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.640+0000 7f959c766b80 1 -- 172.21.15.133:0/4033789808 shutdown_connections 2023-11-27T15:22:50.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.640+0000 7f959c766b80 1 --2- 172.21.15.133:0/4033789808 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x563ef9e8a830 0x563ef9e8cc40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.640+0000 7f959c766b80 1 --2- 172.21.15.133:0/4033789808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ef9e21190 0x563ef9e2d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.640+0000 7f959c766b80 1 --2- 172.21.15.133:0/4033789808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563ef9e1fc80 0x563ef9e30830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.640+0000 7f9569ffb700 0 client.25480 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:22:50.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.640+0000 7f959c766b80 1 --2- 172.21.15.133:0/4033789808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563ef9e205d0 0x563ef9e2d460 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.640+0000 7f959c766b80 1 -- 172.21.15.133:0/4033789808 >> 172.21.15.133:0/4033789808 conn(0x563ef9e153e0 msgr2=0x563ef9e1e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:50.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.641+0000 7f959c766b80 1 -- 172.21.15.133:0/4033789808 shutdown_connections 2023-11-27T15:22:50.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.641+0000 7f959c766b80 1 -- 172.21.15.133:0/4033789808 wait complete. 2023-11-27T15:22:50.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.641+0000 7f959c766b80 20 client.25480 trim_cache size 0 max 16384 2023-11-27T15:22:50.649 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:22:50.649 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2023-11-27T15:22:50.650 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "b8e77da0-8857-4646-b89b-418462eaa3f5", 2023-11-27T15:22:50.650 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2023-11-27T15:22:50.650 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:22:50.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.641+0000 7f9588dff700 1 -- 172.21.15.133:0/482816948 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f956c0a1150 msgr2=0x7f956c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.641+0000 7f9588dff700 1 --2- 172.21.15.133:0/482816948 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f956c0a1150 0x7f956c0a3600 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f957c0060b0 tx=0x7f957c00b040 comp rx=0 tx=0).stop 2023-11-27T15:22:50.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.641+0000 7f9588dff700 1 -- 172.21.15.133:0/482816948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9584073550 msgr2=0x7f958419c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.641+0000 7f9588dff700 1 --2- 172.21.15.133:0/482816948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9584073550 0x7f958419c740 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f957400e970 tx=0x7f957400ec80 comp rx=0 tx=0).stop 2023-11-27T15:22:50.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.641+0000 7f9588dff700 1 -- 172.21.15.133:0/482816948 shutdown_connections 2023-11-27T15:22:50.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.641+0000 7f9588dff700 1 --2- 172.21.15.133:0/482816948 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f956c0a1150 0x7f956c0a3600 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.641+0000 7f9588dff700 1 --2- 172.21.15.133:0/482816948 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f958410c8a0 0x7f95841a3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.641+0000 7f9588dff700 1 --2- 172.21.15.133:0/482816948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9584073550 0x7f958419c740 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.642+0000 7f9588dff700 1 --2- 172.21.15.133:0/482816948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9584074b30 0x7f958419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.642+0000 7f9588dff700 1 -- 172.21.15.133:0/482816948 >> 172.21.15.133:0/482816948 conn(0x7f95840fb8b0 msgr2=0x7f95840fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:50.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.642+0000 7f9588dff700 1 -- 172.21.15.133:0/482816948 shutdown_connections 2023-11-27T15:22:50.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.642+0000 7f9588dff700 1 -- 172.21.15.133:0/482816948 wait complete. 2023-11-27T15:22:50.664 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:22:50.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.903+0000 7fb0d2bff700 1 -- 172.21.15.133:0/2718787536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0cc1002c0 msgr2=0x7fb0cc1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.903+0000 7fb0d2bff700 1 --2- 172.21.15.133:0/2718787536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0cc1002c0 0x7fb0cc1006d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0009990 tx=0x7fb0c0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.903+0000 7fb0d2bff700 1 -- 172.21.15.133:0/2718787536 shutdown_connections 2023-11-27T15:22:50.907 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.903+0000 7fb0d2bff700 1 --2- 172.21.15.133:0/2718787536 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0cc101e40 0x7fb0cc108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.903+0000 7fb0d2bff700 1 --2- 172.21.15.133:0/2718787536 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0cc1002c0 0x7fb0cc1006d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.903+0000 7fb0d2bff700 1 --2- 172.21.15.133:0/2718787536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0cc1014b0 0x7fb0cc101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.903+0000 7fb0d2bff700 1 -- 172.21.15.133:0/2718787536 >> 172.21.15.133:0/2718787536 conn(0x7fb0cc0fb8a0 msgr2=0x7fb0cc0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:50.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.903+0000 7fb0d2bff700 1 -- 172.21.15.133:0/2718787536 shutdown_connections 2023-11-27T15:22:50.908 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.903+0000 7fb0d2bff700 1 -- 172.21.15.133:0/2718787536 wait complete. 2023-11-27T15:22:50.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.904+0000 7fb0d2bff700 1 Processor -- start 2023-11-27T15:22:50.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.904+0000 7fb0d2bff700 1 -- start start 2023-11-27T15:22:50.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.904+0000 7fb0d2bff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0cc1002c0 0x7fb0cc196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.905+0000 7fb0d2bff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0cc1014b0 0x7fb0cc1965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.905+0000 7fb0d2bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0cc101e40 0x7fb0cc19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.905+0000 7fb0d2bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc19dc10 con 0x7fb0cc101e40 2023-11-27T15:22:50.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.905+0000 7fb0d2bff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc19dd50 con 0x7fb0cc1002c0 2023-11-27T15:22:50.910 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.905+0000 7fb0d2bff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0cc19dee0 con 0x7fb0cc1014b0 2023-11-27T15:22:50.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.905+0000 7fb0d119c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0cc101e40 0x7fb0cc19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.905+0000 7fb0d099b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0cc1002c0 0x7fb0cc196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.905+0000 7fb0d099b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0cc1002c0 0x7fb0cc196090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:45376/0 (socket says 172.21.15.133:45376) 2023-11-27T15:22:50.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.905+0000 7fb0d119c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0cc101e40 0x7fb0cc19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60674/0 (socket says 172.21.15.133:60674) 2023-11-27T15:22:50.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.905+0000 7fb0d099b700 1 -- 172.21.15.133:0/1161028433 learned_addr learned my addr 172.21.15.133:0/1161028433 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:50.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.905+0000 7fb0cbfff700 1 --2- 172.21.15.133:0/1161028433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0cc1014b0 0x7fb0cc1965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.905+0000 7fb0d099b700 1 -- 172.21.15.133:0/1161028433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0cc1014b0 msgr2=0x7fb0cc1965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.905+0000 7fb0d099b700 1 --2- 172.21.15.133:0/1161028433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0cc1014b0 0x7fb0cc1965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.905+0000 7fb0d099b700 1 -- 172.21.15.133:0/1161028433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0cc101e40 msgr2=0x7fb0cc19d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.905+0000 7fb0d099b700 1 --2- 172.21.15.133:0/1161028433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0cc101e40 0x7fb0cc19d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.911 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.905+0000 7fb0d099b700 1 -- 172.21.15.133:0/1161028433 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c00095f0 con 0x7fb0cc1002c0 2023-11-27T15:22:50.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.905+0000 7fb0d119c700 1 --2- 172.21.15.133:0/1161028433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0cc101e40 0x7fb0cc19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:22:50.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.906+0000 7fb0d099b700 1 --2- 172.21.15.133:0/1161028433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0cc1002c0 0x7fb0cc196090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0005740 tx=0x7fb0c0005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:50.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.906+0000 7fb0c9ffb700 1 -- 172.21.15.133:0/1161028433 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c001d070 con 0x7fb0cc1002c0 2023-11-27T15:22:50.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.906+0000 7fb0d2bff700 1 -- 172.21.15.133:0/1161028433 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0cc10bc90 con 0x7fb0cc1002c0 2023-11-27T15:22:50.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.906+0000 7fb0c9ffb700 1 -- 172.21.15.133:0/1161028433 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0c0021410 con 0x7fb0cc1002c0 2023-11-27T15:22:50.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.906+0000 7fb0c9ffb700 1 -- 172.21.15.133:0/1161028433 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0017650 con 0x7fb0cc1002c0 2023-11-27T15:22:50.912 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.906+0000 7fb0d2bff700 1 -- 172.21.15.133:0/1161028433 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0cc10bb00 con 0x7fb0cc1002c0 2023-11-27T15:22:50.914 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.910+0000 7fb0c9ffb700 1 -- 172.21.15.133:0/1161028433 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb0c0021580 con 0x7fb0cc1002c0 2023-11-27T15:22:50.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.911+0000 7fb0e6463b80 1 Processor -- start 2023-11-27T15:22:50.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.911+0000 7fb0e6463b80 1 -- start start 2023-11-27T15:22:50.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.912+0000 7fb0e6463b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb9e4fc00 0x55efb9e50010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.912+0000 7fb0e6463b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e50550 0x55efb9e50b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.912+0000 7fb0e6463b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 0x55efb9e51580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.916 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.912+0000 7fb0e6463b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb9d55060 con 0x55efb9e50550 2023-11-27T15:22:50.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.912+0000 7fb0e6463b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb9da78a0 con 0x55efb9e4fc00 2023-11-27T15:22:50.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.912+0000 7fb0e6463b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb9da0d30 con 0x55efb9e51110 2023-11-27T15:22:50.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.912+0000 7fb0c9ffb700 1 --2- 172.21.15.133:0/1161028433 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb0bc0a11d0 0x7fb0bc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.912+0000 7fb0d099b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e50550 0x55efb9e50b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.912+0000 7fb0d099b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e50550 0x55efb9e50b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60690/0 (socket says 172.21.15.133:60690) 2023-11-27T15:22:50.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.912+0000 7fb0d099b700 1 -- 172.21.15.133:0/2339053936 learned_addr learned my addr 172.21.15.133:0/2339053936 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:50.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.912+0000 7fb0cbfff700 1 --2- 172.21.15.133:0/2339053936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 0x55efb9e51580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.912+0000 7fb0d119c700 1 --2- 172.21.15.133:0/2339053936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb9e4fc00 0x55efb9e50010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.912+0000 7fb0c9ffb700 1 -- 172.21.15.133:0/1161028433 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb0c00cdc50 con 0x7fb0cc1002c0 2023-11-27T15:22:50.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.912+0000 7fb0d119c700 1 --2- 172.21.15.133:0/1161028433 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb0bc0a11d0 0x7fb0bc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.912+0000 7fb0cbfff700 1 -- 172.21.15.133:0/2339053936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb9e4fc00 msgr2=0x55efb9e50010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:50.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.912+0000 7fb0cbfff700 1 --2- 172.21.15.133:0/2339053936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb9e4fc00 0x55efb9e50010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.912+0000 7fb0cbfff700 1 -- 172.21.15.133:0/2339053936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e50550 msgr2=0x55efb9e50b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:50.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.912+0000 7fb0cbfff700 1 --2- 172.21.15.133:0/2339053936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e50550 0x55efb9e50b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.912+0000 7fb0cbfff700 1 -- 172.21.15.133:0/2339053936 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efb9c22630 con 0x55efb9e51110 2023-11-27T15:22:50.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.913+0000 7fb0cbfff700 1 --2- 172.21.15.133:0/2339053936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 0x55efb9e51580 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009cb0 tx=0x7fb0b8011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e01d148129e9a84f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:50.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.913+0000 7fb0d119c700 1 --2- 172.21.15.133:0/1161028433 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb0bc0a11d0 0x7fb0bc0a3680 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4009ec0 tx=0x7fb0c400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:50.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.913+0000 7fb0b6ffd700 1 -- 172.21.15.133:0/2339053936 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8011d20 con 0x55efb9e51110 2023-11-27T15:22:50.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.913+0000 7fb0b6ffd700 1 -- 172.21.15.133:0/2339053936 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0b8011e80 con 0x55efb9e51110 2023-11-27T15:22:50.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.913+0000 7fb0b6ffd700 1 -- 172.21.15.133:0/2339053936 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8005280 con 0x55efb9e51110 2023-11-27T15:22:50.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.914+0000 7fb0e6463b80 1 -- 172.21.15.133:0/2339053936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 msgr2=0x55efb9e51580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.914+0000 7fb0e6463b80 1 --2- 172.21.15.133:0/2339053936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 0x55efb9e51580 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009cb0 tx=0x7fb0b8011440 comp rx=0 tx=0).stop 2023-11-27T15:22:50.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.914+0000 7fb0e6463b80 1 -- 172.21.15.133:0/2339053936 shutdown_connections 2023-11-27T15:22:50.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.914+0000 7fb0e6463b80 1 --2- 172.21.15.133:0/2339053936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 0x55efb9e51580 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.914+0000 7fb0e6463b80 1 --2- 172.21.15.133:0/2339053936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb9e4fc00 0x55efb9e50010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.914+0000 7fb0e6463b80 1 --2- 172.21.15.133:0/2339053936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e50550 0x55efb9e50b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.914+0000 7fb0e6463b80 1 -- 172.21.15.133:0/2339053936 >> 172.21.15.133:0/2339053936 conn(0x55efb9e45360 msgr2=0x55efb9e4a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:50.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.914+0000 7fb0e6463b80 1 -- 172.21.15.133:0/2339053936 shutdown_connections 2023-11-27T15:22:50.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.914+0000 7fb0e6463b80 1 -- 172.21.15.133:0/2339053936 wait complete. 2023-11-27T15:22:50.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.914+0000 7fb0e6463b80 1 -- start start 2023-11-27T15:22:50.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.915+0000 7fb0e6463b80 1 Processor -- start 2023-11-27T15:22:50.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.915+0000 7fb0e6463b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e4fc00 0x55efb9e59500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.915+0000 7fb0e6463b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb9e50550 0x55efb9e59a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.915+0000 7fb0e6463b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 0x55efb9e60a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.915+0000 7fb0e6463b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb9d428e0 con 0x55efb9e4fc00 2023-11-27T15:22:50.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.915+0000 7fb0e6463b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb9d42580 con 0x55efb9e50550 2023-11-27T15:22:50.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.915+0000 7fb0e6463b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb9c24920 con 0x55efb9e51110 2023-11-27T15:22:50.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.915+0000 7fb0e6463b80 10 client.0 _ll_register_callbacks cb 0x55efb9d60780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:22:50.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.915+0000 7fb0d099b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e4fc00 0x55efb9e59500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.915+0000 7fb0cbfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb9e50550 0x55efb9e59a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.915+0000 7fb0d119c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 0x55efb9e60a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.915+0000 7fb0d099b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e4fc00 0x55efb9e59500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60694/0 (socket says 172.21.15.133:60694) 2023-11-27T15:22:50.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.915+0000 7fb0d099b700 1 -- 172.21.15.133:0/3074401993 learned_addr learned my addr 172.21.15.133:0/3074401993 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:50.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.916+0000 7fb0d099b700 1 -- 172.21.15.133:0/3074401993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 msgr2=0x55efb9e60a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:50.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.916+0000 7fb0d099b700 1 --2- 172.21.15.133:0/3074401993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 0x55efb9e60a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.916+0000 7fb0d099b700 1 -- 172.21.15.133:0/3074401993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb9e50550 msgr2=0x55efb9e59a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:50.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.916+0000 7fb0d099b700 1 --2- 172.21.15.133:0/3074401993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb9e50550 0x55efb9e59a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.916+0000 7fb0d099b700 1 -- 172.21.15.133:0/3074401993 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b80098e0 con 0x55efb9e4fc00 2023-11-27T15:22:50.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.916+0000 7fb0d099b700 1 --2- 172.21.15.133:0/3074401993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e4fc00 0x55efb9e59500 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000b620 tx=0x7fb0c0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2701a5859e2dd466 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:50.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.916+0000 7fb0abfff700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:22:50.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.916+0000 7fb0abfff700 1 -- 172.21.15.133:0/3074401993 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c009a5b0 con 0x55efb9e4fc00 2023-11-27T15:22:50.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.916+0000 7fb0abfff700 1 -- 172.21.15.133:0/3074401993 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb0c009aec0 con 0x55efb9e4fc00 2023-11-27T15:22:50.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.916+0000 7fb0e6463b80 1 -- 172.21.15.133:0/3074401993 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55efb9d630a0 con 0x55efb9e4fc00 2023-11-27T15:22:50.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.916+0000 7fb0abfff700 1 -- 172.21.15.133:0/3074401993 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0c009a820 con 0x55efb9e4fc00 2023-11-27T15:22:50.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.916+0000 7fb0e6463b80 10 client.15459 fetch_fsmap learned FSMap version 23 2023-11-27T15:22:50.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.916+0000 7fb0e6463b80 1 -- 172.21.15.133:0/3074401993 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55efb9d67190 con 0x55efb9e4fc00 2023-11-27T15:22:50.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.917+0000 7fb0abfff700 1 -- 172.21.15.133:0/3074401993 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55efb9d630a0 con 0x55efb9e4fc00 2023-11-27T15:22:50.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.917+0000 7fb0abfff700 1 -- 172.21.15.133:0/3074401993 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb0c00a2df0 con 0x55efb9e4fc00 2023-11-27T15:22:50.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.917+0000 7fb0e6463b80 10 client.15459 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:22:50.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.917+0000 7fb0e6463b80 10 client.15459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:22:50.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.917+0000 7fb0e6463b80 20 client.15459 populate_metadata read hostname 'smithi133' 2023-11-27T15:22:50.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.917+0000 7fb0e6463b80 1 --2- 172.21.15.133:0/3074401993 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55efb9e64fa0 0x55efb9e673b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.917+0000 7fb0e6463b80 4 client.15459 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:22:50.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.917+0000 7fb0e6463b80 1 -- 172.21.15.133:0/3074401993 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55efb9d6f660 con 0x55efb9e64fa0 2023-11-27T15:22:50.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.917+0000 7fb0cbfff700 1 --2- 172.21.15.133:0/3074401993 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55efb9e64fa0 0x55efb9e673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.918+0000 7fb0cbfff700 1 --2- 172.21.15.133:0/3074401993 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55efb9e64fa0 0x55efb9e673b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=460827416a732bde server_cookie=275f4a8524ae4521 in_seq=0 out_seq=0 2023-11-27T15:22:50.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.918+0000 7fb0abfff700 10 client.15459 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:22:50.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.919+0000 7fb0abfff700 1 -- 172.21.15.133:0/3074401993 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55efb9d6f660 con 0x55efb9e64fa0 2023-11-27T15:22:50.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.919+0000 7fb0abfff700 10 client.15459 handle_command_reply: tid=0 2023-11-27T15:22:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.919+0000 7fb0e6463b80 1 client.15459 shutdown 2023-11-27T15:22:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.919+0000 7fb0e6463b80 1 -- 172.21.15.133:0/3074401993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e4fc00 msgr2=0x55efb9e59500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.919+0000 7fb0e6463b80 1 --2- 172.21.15.133:0/3074401993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e4fc00 0x55efb9e59500 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000b620 tx=0x7fb0c0009580 comp rx=0 tx=0).stop 2023-11-27T15:22:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.919+0000 7fb0e6463b80 1 -- 172.21.15.133:0/3074401993 shutdown_connections 2023-11-27T15:22:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.919+0000 7fb0e6463b80 1 --2- 172.21.15.133:0/3074401993 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55efb9e64fa0 0x55efb9e673b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.919+0000 7fb0e6463b80 1 --2- 172.21.15.133:0/3074401993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 0x55efb9e60a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.919+0000 7fb0e6463b80 1 --2- 172.21.15.133:0/3074401993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e4fc00 0x55efb9e59500 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.919+0000 7fb0e6463b80 1 --2- 172.21.15.133:0/3074401993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb9e50550 0x55efb9e59a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.919+0000 7fb0e6463b80 1 -- 172.21.15.133:0/3074401993 >> 172.21.15.133:0/3074401993 conn(0x55efb9e45360 msgr2=0x55efb9e49de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.919+0000 7fb0abfff700 0 client.15459 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:22:50.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.919+0000 7fb0e6463b80 1 -- 172.21.15.133:0/3074401993 shutdown_connections 2023-11-27T15:22:50.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.919+0000 7fb0e6463b80 1 -- 172.21.15.133:0/3074401993 wait complete. 2023-11-27T15:22:50.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.920+0000 7fb0e6463b80 20 client.15459 trim_cache size 0 max 16384 2023-11-27T15:22:50.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.937+0000 7fb0e6463b80 1 Processor -- start 2023-11-27T15:22:50.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.937+0000 7fb0e6463b80 1 -- start start 2023-11-27T15:22:50.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.937+0000 7fb0e6463b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb9e4fc00 0x55efb9e573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.937+0000 7fb0e6463b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e50550 0x55efb9e57930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.937+0000 7fb0e6463b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 0x55efb9e60a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.937+0000 7fb0e6463b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb9e51a00 con 0x55efb9e50550 2023-11-27T15:22:50.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.937+0000 7fb0e6463b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb9e589b0 con 0x55efb9e4fc00 2023-11-27T15:22:50.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.937+0000 7fb0e6463b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb9e4b550 con 0x55efb9e51110 2023-11-27T15:22:50.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.937+0000 7fb0cbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e50550 0x55efb9e57930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.938+0000 7fb0cbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e50550 0x55efb9e57930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60702/0 (socket says 172.21.15.133:60702) 2023-11-27T15:22:50.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.938+0000 7fb0cbfff700 1 -- 172.21.15.133:0/356823191 learned_addr learned my addr 172.21.15.133:0/356823191 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:50.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.938+0000 7fb0d119c700 1 --2- 172.21.15.133:0/356823191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 0x55efb9e60a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.938+0000 7fb0d099b700 1 --2- 172.21.15.133:0/356823191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb9e4fc00 0x55efb9e573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.938+0000 7fb0cbfff700 1 -- 172.21.15.133:0/356823191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 msgr2=0x55efb9e60a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:50.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.938+0000 7fb0cbfff700 1 --2- 172.21.15.133:0/356823191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 0x55efb9e60a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.938+0000 7fb0cbfff700 1 -- 172.21.15.133:0/356823191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb9e4fc00 msgr2=0x55efb9e573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:50.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.938+0000 7fb0cbfff700 1 --2- 172.21.15.133:0/356823191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb9e4fc00 0x55efb9e573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.938+0000 7fb0cbfff700 1 -- 172.21.15.133:0/356823191 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efb9d66ee0 con 0x55efb9e50550 2023-11-27T15:22:50.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.938+0000 7fb0cbfff700 1 --2- 172.21.15.133:0/356823191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e50550 0x55efb9e57930 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009cb0 tx=0x7fb0b8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fb088d37f2a6c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:50.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.938+0000 7fb0b77fe700 1 -- 172.21.15.133:0/356823191 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8004870 con 0x55efb9e50550 2023-11-27T15:22:50.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.938+0000 7fb0b77fe700 1 -- 172.21.15.133:0/356823191 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0b8004b90 con 0x55efb9e50550 2023-11-27T15:22:50.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.938+0000 7fb0b77fe700 1 -- 172.21.15.133:0/356823191 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b80043b0 con 0x55efb9e50550 2023-11-27T15:22:50.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.939+0000 7fb0e6463b80 1 -- 172.21.15.133:0/356823191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e50550 msgr2=0x55efb9e57930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.939+0000 7fb0e6463b80 1 --2- 172.21.15.133:0/356823191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e50550 0x55efb9e57930 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009cb0 tx=0x7fb0b8004ab0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.939+0000 7fb0e6463b80 1 -- 172.21.15.133:0/356823191 shutdown_connections 2023-11-27T15:22:50.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.939+0000 7fb0e6463b80 1 --2- 172.21.15.133:0/356823191 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 0x55efb9e60a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.939+0000 7fb0e6463b80 1 --2- 172.21.15.133:0/356823191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb9e4fc00 0x55efb9e573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.939+0000 7fb0e6463b80 1 --2- 172.21.15.133:0/356823191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e50550 0x55efb9e57930 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.939+0000 7fb0e6463b80 1 -- 172.21.15.133:0/356823191 >> 172.21.15.133:0/356823191 conn(0x55efb9e45360 msgr2=0x55efb9e53f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:50.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.939+0000 7fb0e6463b80 1 -- 172.21.15.133:0/356823191 shutdown_connections 2023-11-27T15:22:50.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.939+0000 7fb0e6463b80 1 -- 172.21.15.133:0/356823191 wait complete. 2023-11-27T15:22:50.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.939+0000 7fb0e6463b80 1 -- start start 2023-11-27T15:22:50.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.940+0000 7fb0e6463b80 1 Processor -- start 2023-11-27T15:22:50.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.940+0000 7fb0e6463b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb9e4fc00 0x55efb9e5c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.940+0000 7fb0e6463b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e50550 0x55efb9e5ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.940+0000 7fb0e6463b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 0x55efb9e5fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.940+0000 7fb0e6463b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb9e5ffd0 con 0x55efb9e50550 2023-11-27T15:22:50.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.940+0000 7fb0e6463b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb9e60140 con 0x55efb9e4fc00 2023-11-27T15:22:50.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.940+0000 7fb0e6463b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efb9e60300 con 0x55efb9e51110 2023-11-27T15:22:50.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.940+0000 7fb0e6463b80 10 client.0 _ll_register_callbacks cb 0x7fb0c00057e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:22:50.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.940+0000 7fb0d119c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 0x55efb9e5fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.940+0000 7fb0d119c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 0x55efb9e5fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:46432/0 (socket says 172.21.15.133:46432) 2023-11-27T15:22:50.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.940+0000 7fb0d119c700 1 -- 172.21.15.133:0/3058421023 learned_addr learned my addr 172.21.15.133:0/3058421023 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:22:50.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.940+0000 7fb0cbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e50550 0x55efb9e5ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.940+0000 7fb0d099b700 1 --2- 172.21.15.133:0/3058421023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb9e4fc00 0x55efb9e5c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.941+0000 7fb0d119c700 1 -- 172.21.15.133:0/3058421023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb9e4fc00 msgr2=0x55efb9e5c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:50.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.941+0000 7fb0d119c700 1 --2- 172.21.15.133:0/3058421023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb9e4fc00 0x55efb9e5c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.941+0000 7fb0d119c700 1 -- 172.21.15.133:0/3058421023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e50550 msgr2=0x55efb9e5ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:22:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.941+0000 7fb0d119c700 1 --2- 172.21.15.133:0/3058421023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e50550 0x55efb9e5ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.941+0000 7fb0d119c700 1 -- 172.21.15.133:0/3058421023 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b801d040 con 0x55efb9e51110 2023-11-27T15:22:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.941+0000 7fb0d119c700 1 --2- 172.21.15.133:0/3058421023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 0x55efb9e5fa90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4007e70 tx=0x7fb0c4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92282d984ef4990c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:22:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.941+0000 7fb0abfff700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:22:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.941+0000 7fb0abfff700 1 -- 172.21.15.133:0/3058421023 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4027030 con 0x55efb9e51110 2023-11-27T15:22:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.941+0000 7fb0e6463b80 1 -- 172.21.15.133:0/3058421023 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55efb9e42b20 con 0x55efb9e51110 2023-11-27T15:22:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.941+0000 7fb0abfff700 1 -- 172.21.15.133:0/3058421023 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb0c4023d60 con 0x55efb9e51110 2023-11-27T15:22:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.942+0000 7fb0abfff700 1 -- 172.21.15.133:0/3058421023 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0c4003a00 con 0x55efb9e51110 2023-11-27T15:22:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.942+0000 7fb0e6463b80 10 client.25498 fetch_fsmap learned FSMap version 23 2023-11-27T15:22:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.942+0000 7fb0e6463b80 1 -- 172.21.15.133:0/3058421023 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55efb9bb7c60 con 0x55efb9e51110 2023-11-27T15:22:50.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.942+0000 7fb0abfff700 1 -- 172.21.15.133:0/3058421023 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55efb9e42b20 con 0x55efb9e51110 2023-11-27T15:22:50.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.942+0000 7fb0abfff700 1 -- 172.21.15.133:0/3058421023 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb0c4035070 con 0x55efb9e51110 2023-11-27T15:22:50.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.942+0000 7fb0e6463b80 10 client.25498 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:22:50.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.942+0000 7fb0e6463b80 10 client.25498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:22:50.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.942+0000 7fb0e6463b80 20 client.25498 populate_metadata read hostname 'smithi133' 2023-11-27T15:22:50.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.942+0000 7fb0e6463b80 1 --2- 172.21.15.133:0/3058421023 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55efb9eb9bd0 0x55efb9ebbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:22:50.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.942+0000 7fb0e6463b80 4 client.25498 mds_command: new command op to 24560 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:22:50.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.942+0000 7fb0e6463b80 1 -- 172.21.15.133:0/3058421023 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55efb9beff70 con 0x55efb9eb9bd0 2023-11-27T15:22:50.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.943+0000 7fb0d099b700 1 --2- 172.21.15.133:0/3058421023 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55efb9eb9bd0 0x55efb9ebbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:22:50.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.943+0000 7fb0abfff700 10 client.25498 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:22:50.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.943+0000 7fb0d099b700 1 --2- 172.21.15.133:0/3058421023 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55efb9eb9bd0 0x55efb9ebbfe0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25d03479790a01b5 server_cookie=49622adf725214fa in_seq=0 out_seq=0 2023-11-27T15:22:50.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.943+0000 7fb0abfff700 1 -- 172.21.15.133:0/3058421023 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55efb9beff70 con 0x55efb9eb9bd0 2023-11-27T15:22:50.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.943+0000 7fb0abfff700 10 client.25498 handle_command_reply: tid=0 2023-11-27T15:22:50.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.943+0000 7fb0e6463b80 1 client.25498 shutdown 2023-11-27T15:22:50.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.943+0000 7fb0e6463b80 1 -- 172.21.15.133:0/3058421023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 msgr2=0x55efb9e5fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.943+0000 7fb0e6463b80 1 --2- 172.21.15.133:0/3058421023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 0x55efb9e5fa90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4007e70 tx=0x7fb0c4004ad0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.943+0000 7fb0e6463b80 1 -- 172.21.15.133:0/3058421023 shutdown_connections 2023-11-27T15:22:50.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.943+0000 7fb0e6463b80 1 --2- 172.21.15.133:0/3058421023 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55efb9eb9bd0 0x55efb9ebbfe0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.943+0000 7fb0e6463b80 1 --2- 172.21.15.133:0/3058421023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efb9e51110 0x55efb9e5fa90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.943+0000 7fb0abfff700 0 client.25498 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:22:50.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.943+0000 7fb0e6463b80 1 --2- 172.21.15.133:0/3058421023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb9e4fc00 0x55efb9e5c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.943+0000 7fb0e6463b80 1 --2- 172.21.15.133:0/3058421023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efb9e50550 0x55efb9e5ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.943+0000 7fb0e6463b80 1 -- 172.21.15.133:0/3058421023 >> 172.21.15.133:0/3058421023 conn(0x55efb9e45360 msgr2=0x55efb9e53f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:50.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.944+0000 7fb0e6463b80 1 -- 172.21.15.133:0/3058421023 shutdown_connections 2023-11-27T15:22:50.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.944+0000 7fb0e6463b80 1 -- 172.21.15.133:0/3058421023 wait complete. 2023-11-27T15:22:50.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.944+0000 7fb0e6463b80 20 client.25498 trim_cache size 0 max 16384 2023-11-27T15:22:50.953 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:22:50.953 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-27T15:22:50.953 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-11-27T15:22:50.953 INFO:teuthology.orchestra.run.smithi133.stdout: "b8e77da0-8857-4646-b89b-418462eaa3f5": { 2023-11-27T15:22:50.953 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-11-27T15:22:50.953 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "b8e77da0-8857-4646-b89b-418462eaa3f5", 2023-11-27T15:22:50.953 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-11-27T15:22:50.953 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:22:50.953 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:22:50.953 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:22:50.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.944+0000 7fb0d2bff700 1 -- 172.21.15.133:0/1161028433 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb0bc0a11d0 msgr2=0x7fb0bc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.944+0000 7fb0d2bff700 1 --2- 172.21.15.133:0/1161028433 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb0bc0a11d0 0x7fb0bc0a3680 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4009ec0 tx=0x7fb0c400b040 comp rx=0 tx=0).stop 2023-11-27T15:22:50.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.944+0000 7fb0d2bff700 1 -- 172.21.15.133:0/1161028433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0cc1002c0 msgr2=0x7fb0cc196090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:22:50.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.944+0000 7fb0d2bff700 1 --2- 172.21.15.133:0/1161028433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0cc1002c0 0x7fb0cc196090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0005740 tx=0x7fb0c0005770 comp rx=0 tx=0).stop 2023-11-27T15:22:50.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.944+0000 7fb0d2bff700 1 -- 172.21.15.133:0/1161028433 shutdown_connections 2023-11-27T15:22:50.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.944+0000 7fb0d2bff700 1 --2- 172.21.15.133:0/1161028433 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb0bc0a11d0 0x7fb0bc0a3680 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.944+0000 7fb0d2bff700 1 --2- 172.21.15.133:0/1161028433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb0cc101e40 0x7fb0cc19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.944+0000 7fb0d2bff700 1 --2- 172.21.15.133:0/1161028433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0cc1002c0 0x7fb0cc196090 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.944+0000 7fb0d2bff700 1 --2- 172.21.15.133:0/1161028433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb0cc1014b0 0x7fb0cc1965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:22:50.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.944+0000 7fb0d2bff700 1 -- 172.21.15.133:0/1161028433 >> 172.21.15.133:0/1161028433 conn(0x7fb0cc0fb8a0 msgr2=0x7fb0cc104990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:22:50.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.944+0000 7fb0d2bff700 1 -- 172.21.15.133:0/1161028433 shutdown_connections 2023-11-27T15:22:50.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:22:50.944+0000 7fb0d2bff700 1 -- 172.21.15.133:0/1161028433 wait complete. 2023-11-27T15:22:50.968 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:50 smithi133 ceph-mon[97860]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 6 op/s 2023-11-27T15:22:50.968 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:50 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/135775035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:50.968 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:50 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1964605857' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:50.971 INFO:tasks.cephfs.filesystem:scrub status for tag:b8e77da0-8857-4646-b89b-418462eaa3f5 - {'path': '/', 'tag': 'b8e77da0-8857-4646-b89b-418462eaa3f5', 'options': 'recursive,force'} 2023-11-27T15:22:51.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:50 smithi177 ceph-mon[131504]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 6 op/s 2023-11-27T15:22:51.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:50 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/135775035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:51.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:50 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1964605857' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:51.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:50 smithi174 ceph-mon[131110]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 6 op/s 2023-11-27T15:22:51.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:50 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/135775035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:22:51.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:50 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1964605857' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:22:52.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:51 smithi174 ceph-mon[131110]: scrub queued for path: / 2023-11-27T15:22:52.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:51 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:22:52.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:51 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:22:52.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:51 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:22:52.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:51 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:22:52.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:51 smithi133 ceph-mon[97860]: scrub queued for path: / 2023-11-27T15:22:52.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:51 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:22:52.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:51 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:22:52.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:51 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:22:52.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:51 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:22:52.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:51 smithi177 ceph-mon[131504]: scrub queued for path: / 2023-11-27T15:22:52.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:51 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:22:52.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:51 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:22:52.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:51 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:22:52.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:51 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:22:53.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:52 smithi174 ceph-mon[131110]: pgmap v347: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2023-11-27T15:22:53.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:52 smithi133 ceph-mon[97860]: pgmap v347: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2023-11-27T15:22:53.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:52 smithi177 ceph-mon[131504]: pgmap v347: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2023-11-27T15:22:55.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:54 smithi177 ceph-mon[131504]: pgmap v348: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2023-11-27T15:22:55.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:54 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:22:55.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:54 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:22:55.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:54 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:22:55.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:54 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:22:55.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:54 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:22:55.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:54 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:22:55.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:54 smithi174 ceph-mon[131110]: pgmap v348: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2023-11-27T15:22:55.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:54 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:22:55.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:54 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:22:55.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:54 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:22:55.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:54 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:22:55.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:54 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:22:55.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:54 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:22:55.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:54 smithi133 ceph-mon[97860]: pgmap v348: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2023-11-27T15:22:55.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:54 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:22:55.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:54 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:22:55.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:54 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:22:55.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:54 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:22:55.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:54 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:22:55.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:54 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:22:56.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:55 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:56.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:55 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:22:56.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:55 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:22:56.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:55 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:56.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:55 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:22:56.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:55 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:22:56.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:55 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:22:56.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:55 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:22:56.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:55 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:22:57.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:56 smithi177 ceph-mon[131504]: pgmap v349: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-11-27T15:22:57.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:56 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:22:57.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:56 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:22:57.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:56 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:22:57.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:56 smithi177 ceph-mon[131504]: Service mds.all has no related daemon already placed. 2023-11-27T15:22:57.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:56 smithi174 ceph-mon[131110]: pgmap v349: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-11-27T15:22:57.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:56 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:22:57.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:56 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:22:57.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:56 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:22:57.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:56 smithi174 ceph-mon[131110]: Service mds.all has no related daemon already placed. 2023-11-27T15:22:57.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:56 smithi133 ceph-mon[97860]: pgmap v349: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2023-11-27T15:22:57.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:56 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:22:57.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:56 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:22:57.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:56 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:22:57.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:56 smithi133 ceph-mon[97860]: Service mds.all has no related daemon already placed. 2023-11-27T15:22:59.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:22:58 smithi133 ceph-mon[97860]: pgmap v350: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2023-11-27T15:22:59.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:22:58 smithi177 ceph-mon[131504]: pgmap v350: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2023-11-27T15:22:59.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:22:58 smithi174 ceph-mon[131110]: pgmap v350: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2023-11-27T15:23:01.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:00 smithi177 ceph-mon[131504]: pgmap v351: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 6 op/s 2023-11-27T15:23:01.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:00 smithi174 ceph-mon[131110]: pgmap v351: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 6 op/s 2023-11-27T15:23:01.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:00 smithi133 ceph-mon[97860]: pgmap v351: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 6 op/s 2023-11-27T15:23:02.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:01 smithi177 ceph-mon[131504]: scrub summary: idle 2023-11-27T15:23:02.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:01 smithi174 ceph-mon[131110]: scrub summary: idle 2023-11-27T15:23:02.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:01 smithi133 ceph-mon[97860]: scrub summary: idle 2023-11-27T15:23:03.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:02 smithi177 ceph-mon[131504]: pgmap v352: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 7 op/s 2023-11-27T15:23:03.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:02 smithi174 ceph-mon[131110]: pgmap v352: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 7 op/s 2023-11-27T15:23:03.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:02 smithi133 ceph-mon[97860]: pgmap v352: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 853 B/s wr, 7 op/s 2023-11-27T15:23:03.654 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.649+0000 7fa8979ff700 1 -- 172.21.15.133:0/1782653825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa890100270 msgr2=0x7fa890100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.649+0000 7fa8979ff700 1 --2- 172.21.15.133:0/1782653825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa890100270 0x7fa890100680 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa88c009990 tx=0x7fa88c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.649+0000 7fa8979ff700 1 -- 172.21.15.133:0/1782653825 shutdown_connections 2023-11-27T15:23:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.649+0000 7fa8979ff700 1 --2- 172.21.15.133:0/1782653825 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa890101df0 0x7fa8901086b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.649+0000 7fa8979ff700 1 --2- 172.21.15.133:0/1782653825 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa890100270 0x7fa890100680 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.649+0000 7fa8979ff700 1 --2- 172.21.15.133:0/1782653825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa890101460 0x7fa8901018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.649+0000 7fa8979ff700 1 -- 172.21.15.133:0/1782653825 >> 172.21.15.133:0/1782653825 conn(0x7fa8900fb890 msgr2=0x7fa8900fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.649+0000 7fa8979ff700 1 -- 172.21.15.133:0/1782653825 shutdown_connections 2023-11-27T15:23:03.655 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.649+0000 7fa8979ff700 1 -- 172.21.15.133:0/1782653825 wait complete. 2023-11-27T15:23:03.657 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.650+0000 7fa8979ff700 1 Processor -- start 2023-11-27T15:23:03.657 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.650+0000 7fa8979ff700 1 -- start start 2023-11-27T15:23:03.657 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.650+0000 7fa8979ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa890100270 0x7fa890198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:03.657 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.650+0000 7fa8979ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa890101460 0x7fa8901986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:03.657 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.651+0000 7fa8979ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa890101df0 0x7fa89019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:03.658 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.651+0000 7fa8979ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89019fd10 con 0x7fa890101460 2023-11-27T15:23:03.658 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.651+0000 7fa8979ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89019fe50 con 0x7fa890100270 2023-11-27T15:23:03.658 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.651+0000 7fa8979ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89019ffe0 con 0x7fa890101df0 2023-11-27T15:23:03.658 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.651+0000 7fa894f9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa890101460 0x7fa8901986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:03.658 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.651+0000 7fa894f9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa890101460 0x7fa8901986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34272/0 (socket says 172.21.15.133:34272) 2023-11-27T15:23:03.658 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.651+0000 7fa894f9a700 1 -- 172.21.15.133:0/1003419355 learned_addr learned my addr 172.21.15.133:0/1003419355 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:03.658 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.651+0000 7fa89579b700 1 --2- 172.21.15.133:0/1003419355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa890100270 0x7fa890198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:03.658 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.651+0000 7fa895f9c700 1 --2- 172.21.15.133:0/1003419355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa890101df0 0x7fa89019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:03.658 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.651+0000 7fa894f9a700 1 -- 172.21.15.133:0/1003419355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa890101df0 msgr2=0x7fa89019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:03.658 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.651+0000 7fa894f9a700 1 --2- 172.21.15.133:0/1003419355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa890101df0 0x7fa89019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.658 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.651+0000 7fa894f9a700 1 -- 172.21.15.133:0/1003419355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa890100270 msgr2=0x7fa890198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:03.658 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.651+0000 7fa894f9a700 1 --2- 172.21.15.133:0/1003419355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa890100270 0x7fa890198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.659 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.651+0000 7fa894f9a700 1 -- 172.21.15.133:0/1003419355 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88c0095f0 con 0x7fa890101460 2023-11-27T15:23:03.659 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.652+0000 7fa894f9a700 1 --2- 172.21.15.133:0/1003419355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa890101460 0x7fa8901986d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa88000e9a0 tx=0x7fa88000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:03.659 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.652+0000 7fa8867fc700 1 -- 172.21.15.133:0/1003419355 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880009790 con 0x7fa890101460 2023-11-27T15:23:03.659 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.652+0000 7fa8867fc700 1 -- 172.21.15.133:0/1003419355 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa880010430 con 0x7fa890101460 2023-11-27T15:23:03.659 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.652+0000 7fa8979ff700 1 -- 172.21.15.133:0/1003419355 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa89010bbc0 con 0x7fa890101460 2023-11-27T15:23:03.659 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.652+0000 7fa8867fc700 1 -- 172.21.15.133:0/1003419355 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880018680 con 0x7fa890101460 2023-11-27T15:23:03.659 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.652+0000 7fa8979ff700 1 -- 172.21.15.133:0/1003419355 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa89010ba30 con 0x7fa890101460 2023-11-27T15:23:03.660 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.655+0000 7fa8867fc700 1 -- 172.21.15.133:0/1003419355 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa8800105a0 con 0x7fa890101460 2023-11-27T15:23:03.661 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.657+0000 7fa8ab233b80 1 Processor -- start 2023-11-27T15:23:03.661 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.657+0000 7fa8867fc700 1 --2- 172.21.15.133:0/1003419355 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa87c0a1150 0x7fa87c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:03.663 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.657+0000 7fa8867fc700 1 -- 172.21.15.133:0/1003419355 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa880014070 con 0x7fa890101460 2023-11-27T15:23:03.663 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.657+0000 7fa8ab233b80 1 -- start start 2023-11-27T15:23:03.663 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.657+0000 7fa895f9c700 1 --2- 172.21.15.133:0/1003419355 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa87c0a1150 0x7fa87c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:03.663 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.657+0000 7fa8ab233b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628627e56b0 0x562862989870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:03.663 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.657+0000 7fa8ab233b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562862989db0 0x56286298a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:03.663 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.657+0000 7fa8ab233b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56286298ab10 0x56286298af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:03.663 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.657+0000 7fa8ab233b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286288f000 con 0x56286298ab10 2023-11-27T15:23:03.663 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.657+0000 7fa8ab233b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628628e1840 con 0x562862989db0 2023-11-27T15:23:03.663 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.657+0000 7fa8ab233b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628628dacd0 con 0x5628627e56b0 2023-11-27T15:23:03.663 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.658+0000 7fa894f9a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562862989db0 0x56286298a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:03.663 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.658+0000 7fa895f9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56286298ab10 0x56286298af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:03.664 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.658+0000 7fa89579b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628627e56b0 0x562862989870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:03.664 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.658+0000 7fa894f9a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562862989db0 0x56286298a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:52948/0 (socket says 172.21.15.133:52948) 2023-11-27T15:23:03.664 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.658+0000 7fa89579b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628627e56b0 0x562862989870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:54822/0 (socket says 172.21.15.133:54822) 2023-11-27T15:23:03.664 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.658+0000 7fa894f9a700 1 -- 172.21.15.133:0/3477497276 learned_addr learned my addr 172.21.15.133:0/3477497276 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:03.664 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.658+0000 7fa895f9c700 1 --2- 172.21.15.133:0/1003419355 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa87c0a1150 0x7fa87c0a3600 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa88800f590 tx=0x7fa888009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:03.664 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.658+0000 7fa894f9a700 1 -- 172.21.15.133:0/3477497276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628627e56b0 msgr2=0x562862989870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:03.664 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.658+0000 7fa894f9a700 1 --2- 172.21.15.133:0/3477497276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628627e56b0 0x562862989870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.664 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.658+0000 7fa894f9a700 1 -- 172.21.15.133:0/3477497276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56286298ab10 msgr2=0x56286298af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:03.664 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.658+0000 7fa894f9a700 1 --2- 172.21.15.133:0/3477497276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56286298ab10 0x56286298af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.664 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.658+0000 7fa894f9a700 1 -- 172.21.15.133:0/3477497276 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56286275c5d0 con 0x562862989db0 2023-11-27T15:23:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.659+0000 7fa894f9a700 1 --2- 172.21.15.133:0/3477497276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562862989db0 0x56286298a5d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa8800962b0 tx=0x7fa880099e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2bd8223e4637310 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.659+0000 7fa87b7fe700 1 -- 172.21.15.133:0/3477497276 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8800a2200 con 0x562862989db0 2023-11-27T15:23:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.659+0000 7fa87b7fe700 1 -- 172.21.15.133:0/3477497276 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa880098150 con 0x562862989db0 2023-11-27T15:23:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.659+0000 7fa87b7fe700 1 -- 172.21.15.133:0/3477497276 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8800a1f70 con 0x562862989db0 2023-11-27T15:23:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.660+0000 7fa8ab233b80 1 -- 172.21.15.133:0/3477497276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562862989db0 msgr2=0x56286298a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.660+0000 7fa8ab233b80 1 --2- 172.21.15.133:0/3477497276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562862989db0 0x56286298a5d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa8800962b0 tx=0x7fa880099e80 comp rx=0 tx=0).stop 2023-11-27T15:23:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.660+0000 7fa8ab233b80 1 -- 172.21.15.133:0/3477497276 shutdown_connections 2023-11-27T15:23:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.660+0000 7fa8ab233b80 1 --2- 172.21.15.133:0/3477497276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56286298ab10 0x56286298af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.660+0000 7fa8ab233b80 1 --2- 172.21.15.133:0/3477497276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628627e56b0 0x562862989870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.660+0000 7fa8ab233b80 1 --2- 172.21.15.133:0/3477497276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562862989db0 0x56286298a5d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.660+0000 7fa8ab233b80 1 -- 172.21.15.133:0/3477497276 >> 172.21.15.133:0/3477497276 conn(0x56286297ef50 msgr2=0x562862984100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.660+0000 7fa8ab233b80 1 -- 172.21.15.133:0/3477497276 shutdown_connections 2023-11-27T15:23:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.660+0000 7fa8ab233b80 1 -- 172.21.15.133:0/3477497276 wait complete. 2023-11-27T15:23:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.660+0000 7fa8ab233b80 1 -- start start 2023-11-27T15:23:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.661+0000 7fa8ab233b80 1 Processor -- start 2023-11-27T15:23:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.661+0000 7fa8ab233b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628627e56b0 0x562862992cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.661+0000 7fa8ab233b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562862989db0 0x562862993200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.661+0000 7fa8ab233b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56286298ab10 0x56286299a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.661+0000 7fa8ab233b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286287c880 con 0x562862989db0 2023-11-27T15:23:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.661+0000 7fa8ab233b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286287c520 con 0x56286298ab10 2023-11-27T15:23:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.661+0000 7fa8ab233b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286275e8c0 con 0x5628627e56b0 2023-11-27T15:23:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.661+0000 7fa8ab233b80 10 client.0 _ll_register_callbacks cb 0x56286289a720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.662+0000 7fa894f9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562862989db0 0x562862993200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.662+0000 7fa89579b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628627e56b0 0x562862992cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.662+0000 7fa89579b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628627e56b0 0x562862992cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:54826/0 (socket says 172.21.15.133:54826) 2023-11-27T15:23:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.662+0000 7fa894f9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562862989db0 0x562862993200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34300/0 (socket says 172.21.15.133:34300) 2023-11-27T15:23:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.662+0000 7fa89579b700 1 -- 172.21.15.133:0/2380007095 learned_addr learned my addr 172.21.15.133:0/2380007095 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.662+0000 7fa89579b700 1 -- 172.21.15.133:0/2380007095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56286298ab10 msgr2=0x56286299a240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-27T15:23:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.662+0000 7fa89579b700 1 --2- 172.21.15.133:0/2380007095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56286298ab10 0x56286299a240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.662+0000 7fa89579b700 1 -- 172.21.15.133:0/2380007095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562862989db0 msgr2=0x562862993200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.662+0000 7fa89579b700 1 --2- 172.21.15.133:0/2380007095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562862989db0 0x562862993200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.662+0000 7fa89579b700 1 -- 172.21.15.133:0/2380007095 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa89010ba30 con 0x5628627e56b0 2023-11-27T15:23:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.662+0000 7fa894f9a700 1 --2- 172.21.15.133:0/2380007095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562862989db0 0x562862993200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.662+0000 7fa89579b700 1 --2- 172.21.15.133:0/2380007095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628627e56b0 0x562862992cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa8901012c0 tx=0x7fa88c009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=972accee993faf0b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.662+0000 7fa878ff9700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:23:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.663+0000 7fa878ff9700 1 -- 172.21.15.133:0/2380007095 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c01d070 con 0x5628627e56b0 2023-11-27T15:23:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.663+0000 7fa8ab233b80 1 -- 172.21.15.133:0/2380007095 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56286297c710 con 0x5628627e56b0 2023-11-27T15:23:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.663+0000 7fa878ff9700 1 -- 172.21.15.133:0/2380007095 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa88c004500 con 0x5628627e56b0 2023-11-27T15:23:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.663+0000 7fa878ff9700 1 -- 172.21.15.133:0/2380007095 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa88c0056f0 con 0x5628627e56b0 2023-11-27T15:23:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.663+0000 7fa8ab233b80 10 client.25510 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.663+0000 7fa8ab233b80 1 -- 172.21.15.133:0/2380007095 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5628628a1130 con 0x5628627e56b0 2023-11-27T15:23:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.663+0000 7fa878ff9700 1 -- 172.21.15.133:0/2380007095 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56286297c710 con 0x5628627e56b0 2023-11-27T15:23:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.663+0000 7fa878ff9700 1 -- 172.21.15.133:0/2380007095 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa88c00f7b0 con 0x5628627e56b0 2023-11-27T15:23:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.663+0000 7fa8ab233b80 10 client.25510 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.663+0000 7fa8ab233b80 10 client.25510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.663+0000 7fa8ab233b80 20 client.25510 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.664+0000 7fa8ab233b80 1 --2- 172.21.15.133:0/2380007095 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56286299eac0 0x5628629a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.664+0000 7fa8ab233b80 4 client.25510 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.664+0000 7fa8ab233b80 1 -- 172.21.15.133:0/2380007095 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56286289d040 con 0x56286299eac0 2023-11-27T15:23:03.672 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.664+0000 7fa894f9a700 1 --2- 172.21.15.133:0/2380007095 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56286299eac0 0x5628629a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.664+0000 7fa894f9a700 1 --2- 172.21.15.133:0/2380007095 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56286299eac0 0x5628629a0ed0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f470339c12eb309 server_cookie=57ab48bdfdbda172 in_seq=0 out_seq=0 2023-11-27T15:23:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.664+0000 7fa878ff9700 10 client.25510 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.665+0000 7fa878ff9700 1 -- 172.21.15.133:0/2380007095 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56286289d040 con 0x56286299eac0 2023-11-27T15:23:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.665+0000 7fa878ff9700 10 client.25510 handle_command_reply: tid=0 2023-11-27T15:23:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.665+0000 7fa8ab233b80 1 client.25510 shutdown 2023-11-27T15:23:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.665+0000 7fa8ab233b80 1 -- 172.21.15.133:0/2380007095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628627e56b0 msgr2=0x562862992cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.665+0000 7fa8ab233b80 1 --2- 172.21.15.133:0/2380007095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628627e56b0 0x562862992cc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa8901012c0 tx=0x7fa88c009570 comp rx=0 tx=0).stop 2023-11-27T15:23:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.665+0000 7fa8ab233b80 1 -- 172.21.15.133:0/2380007095 shutdown_connections 2023-11-27T15:23:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.665+0000 7fa8ab233b80 1 --2- 172.21.15.133:0/2380007095 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56286299eac0 0x5628629a0ed0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.674 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.665+0000 7fa8ab233b80 1 --2- 172.21.15.133:0/2380007095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56286298ab10 0x56286299a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.675 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.665+0000 7fa8ab233b80 1 --2- 172.21.15.133:0/2380007095 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628627e56b0 0x562862992cc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.675 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.665+0000 7fa8ab233b80 1 --2- 172.21.15.133:0/2380007095 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562862989db0 0x562862993200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.675 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.665+0000 7fa878ff9700 0 client.25510 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:03.675 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.665+0000 7fa8ab233b80 1 -- 172.21.15.133:0/2380007095 >> 172.21.15.133:0/2380007095 conn(0x56286297ef50 msgr2=0x5628629837e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:03.675 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.666+0000 7fa8ab233b80 1 -- 172.21.15.133:0/2380007095 shutdown_connections 2023-11-27T15:23:03.675 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.666+0000 7fa8ab233b80 1 -- 172.21.15.133:0/2380007095 wait complete. 2023-11-27T15:23:03.675 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.666+0000 7fa8ab233b80 20 client.25510 trim_cache size 0 max 16384 2023-11-27T15:23:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.681+0000 7fa8ab233b80 1 Processor -- start 2023-11-27T15:23:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.681+0000 7fa8ab233b80 1 -- start start 2023-11-27T15:23:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.681+0000 7fa8ab233b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628627e56b0 0x56286298da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.682+0000 7fa8ab233b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562862989db0 0x56286298df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.682+0000 7fa8ab233b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56286298ab10 0x56286299a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.682+0000 7fa8ab233b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562862992890 con 0x56286298ab10 2023-11-27T15:23:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.682+0000 7fa8ab233b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56286297ebc0 con 0x562862989db0 2023-11-27T15:23:03.687 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.682+0000 7fa8ab233b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x562862990ce0 con 0x5628627e56b0 2023-11-27T15:23:03.688 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.682+0000 7fa895f9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56286298ab10 0x56286299a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:03.688 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.682+0000 7fa895f9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56286298ab10 0x56286299a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34306/0 (socket says 172.21.15.133:34306) 2023-11-27T15:23:03.688 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.682+0000 7fa895f9c700 1 -- 172.21.15.133:0/1918654741 learned_addr learned my addr 172.21.15.133:0/1918654741 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:03.688 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.682+0000 7fa89579b700 1 --2- 172.21.15.133:0/1918654741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628627e56b0 0x56286298da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:03.688 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.682+0000 7fa894f9a700 1 --2- 172.21.15.133:0/1918654741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562862989db0 0x56286298df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:03.688 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.682+0000 7fa89579b700 1 -- 172.21.15.133:0/1918654741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562862989db0 msgr2=0x56286298df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:03.688 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.682+0000 7fa89579b700 1 --2- 172.21.15.133:0/1918654741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562862989db0 0x56286298df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.688 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.682+0000 7fa89579b700 1 -- 172.21.15.133:0/1918654741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56286298ab10 msgr2=0x56286299a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:03.688 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.682+0000 7fa89579b700 1 --2- 172.21.15.133:0/1918654741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56286298ab10 0x56286299a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.689 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.682+0000 7fa89579b700 1 -- 172.21.15.133:0/1918654741 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628628a0e80 con 0x5628627e56b0 2023-11-27T15:23:03.689 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.682+0000 7fa895f9c700 1 --2- 172.21.15.133:0/1918654741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56286298ab10 0x56286299a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:03.689 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.683+0000 7fa89579b700 1 --2- 172.21.15.133:0/1918654741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628627e56b0 0x56286298da00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00b680 tx=0x7fa88c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2bba640b054db43 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:03.689 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.683+0000 7fa87bfff700 1 -- 172.21.15.133:0/1918654741 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c01d070 con 0x5628627e56b0 2023-11-27T15:23:03.689 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.683+0000 7fa87bfff700 1 -- 172.21.15.133:0/1918654741 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa88c01e040 con 0x5628627e56b0 2023-11-27T15:23:03.689 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.683+0000 7fa87bfff700 1 -- 172.21.15.133:0/1918654741 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c0047f0 con 0x5628627e56b0 2023-11-27T15:23:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.684+0000 7fa8ab233b80 1 -- 172.21.15.133:0/1918654741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628627e56b0 msgr2=0x56286298da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.684+0000 7fa8ab233b80 1 --2- 172.21.15.133:0/1918654741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628627e56b0 0x56286298da00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00b680 tx=0x7fa88c009310 comp rx=0 tx=0).stop 2023-11-27T15:23:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.684+0000 7fa8ab233b80 1 -- 172.21.15.133:0/1918654741 shutdown_connections 2023-11-27T15:23:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.684+0000 7fa8ab233b80 1 --2- 172.21.15.133:0/1918654741 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56286298ab10 0x56286299a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.684+0000 7fa8ab233b80 1 --2- 172.21.15.133:0/1918654741 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5628627e56b0 0x56286298da00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.684+0000 7fa8ab233b80 1 --2- 172.21.15.133:0/1918654741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562862989db0 0x56286298df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.684+0000 7fa8ab233b80 1 -- 172.21.15.133:0/1918654741 >> 172.21.15.133:0/1918654741 conn(0x56286297ef50 msgr2=0x5628629919d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.684+0000 7fa8ab233b80 1 -- 172.21.15.133:0/1918654741 shutdown_connections 2023-11-27T15:23:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.684+0000 7fa8ab233b80 1 -- 172.21.15.133:0/1918654741 wait complete. 2023-11-27T15:23:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.684+0000 7fa8ab233b80 1 -- start start 2023-11-27T15:23:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.684+0000 7fa8ab233b80 1 Processor -- start 2023-11-27T15:23:03.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.685+0000 7fa8ab233b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5628627e56b0 0x562862998440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.685+0000 7fa8ab233b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562862989db0 0x562862998980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.685+0000 7fa8ab233b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56286298ab10 0x562862998f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.685+0000 7fa8ab233b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562862999650 con 0x5628627e56b0 2023-11-27T15:23:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.685+0000 7fa8ab233b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628629997c0 con 0x562862989db0 2023-11-27T15:23:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.685+0000 7fa8ab233b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x562862999980 con 0x56286298ab10 2023-11-27T15:23:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.685+0000 7fa8ab233b80 10 client.0 _ll_register_callbacks cb 0x7fa88c00bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.685+0000 7fa89579b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5628627e56b0 0x562862998440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.685+0000 7fa89579b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5628627e56b0 0x562862998440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34310/0 (socket says 172.21.15.133:34310) 2023-11-27T15:23:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.685+0000 7fa89579b700 1 -- 172.21.15.133:0/2096602067 learned_addr learned my addr 172.21.15.133:0/2096602067 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.685+0000 7fa894f9a700 1 --2- 172.21.15.133:0/2096602067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562862989db0 0x562862998980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.685+0000 7fa895f9c700 1 --2- 172.21.15.133:0/2096602067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56286298ab10 0x562862998f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.685+0000 7fa89579b700 1 -- 172.21.15.133:0/2096602067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56286298ab10 msgr2=0x562862998f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.685+0000 7fa89579b700 1 --2- 172.21.15.133:0/2096602067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56286298ab10 0x562862998f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.685+0000 7fa89579b700 1 -- 172.21.15.133:0/2096602067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562862989db0 msgr2=0x562862998980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:03.693 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.685+0000 7fa89579b700 1 --2- 172.21.15.133:0/2096602067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562862989db0 0x562862998980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.685+0000 7fa89579b700 1 -- 172.21.15.133:0/2096602067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa888019540 con 0x5628627e56b0 2023-11-27T15:23:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.686+0000 7fa89579b700 1 --2- 172.21.15.133:0/2096602067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5628627e56b0 0x562862998440 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00c010 tx=0x7fa88c0094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70fb8ebae2c2af31 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.686+0000 7fa878ff9700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:23:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.686+0000 7fa878ff9700 1 -- 172.21.15.133:0/2096602067 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c01d070 con 0x5628627e56b0 2023-11-27T15:23:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.686+0000 7fa8ab233b80 1 -- 172.21.15.133:0/2096602067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa880098150 con 0x5628627e56b0 2023-11-27T15:23:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.686+0000 7fa878ff9700 1 -- 172.21.15.133:0/2096602067 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa88c03c020 con 0x5628627e56b0 2023-11-27T15:23:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.686+0000 7fa878ff9700 1 -- 172.21.15.133:0/2096602067 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa88c00f670 con 0x5628627e56b0 2023-11-27T15:23:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.686+0000 7fa8ab233b80 10 client.15483 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.686+0000 7fa8ab233b80 1 -- 172.21.15.133:0/2096602067 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5628628a1130 con 0x5628627e56b0 2023-11-27T15:23:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.687+0000 7fa878ff9700 1 -- 172.21.15.133:0/2096602067 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa880098150 con 0x5628627e56b0 2023-11-27T15:23:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.687+0000 7fa878ff9700 1 -- 172.21.15.133:0/2096602067 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa88c04b070 con 0x5628627e56b0 2023-11-27T15:23:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.687+0000 7fa8ab233b80 10 client.15483 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.687+0000 7fa8ab233b80 10 client.15483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.687+0000 7fa8ab233b80 20 client.15483 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.687+0000 7fa8ab233b80 1 --2- 172.21.15.133:0/2096602067 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5628629f2ff0 0x5628629f5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.687+0000 7fa8ab233b80 4 client.15483 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:23:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.687+0000 7fa8ab233b80 1 -- 172.21.15.133:0/2096602067 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5628628a9600 con 0x5628629f2ff0 2023-11-27T15:23:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.687+0000 7fa894f9a700 1 --2- 172.21.15.133:0/2096602067 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5628629f2ff0 0x5628629f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.688+0000 7fa878ff9700 10 client.15483 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.688+0000 7fa894f9a700 1 --2- 172.21.15.133:0/2096602067 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5628629f2ff0 0x5628629f5400 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a8de74b6a612d46 server_cookie=e88fee8de0b15fbc in_seq=0 out_seq=0 2023-11-27T15:23:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.701+0000 7fa878ff9700 1 -- 172.21.15.133:0/2096602067 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5628628a9600 con 0x5628629f2ff0 2023-11-27T15:23:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.701+0000 7fa878ff9700 10 client.15483 handle_command_reply: tid=0 2023-11-27T15:23:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.701+0000 7fa8ab233b80 1 client.15483 shutdown 2023-11-27T15:23:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.701+0000 7fa8ab233b80 1 -- 172.21.15.133:0/2096602067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5628627e56b0 msgr2=0x562862998440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.701+0000 7fa8ab233b80 1 --2- 172.21.15.133:0/2096602067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5628627e56b0 0x562862998440 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa88c00c010 tx=0x7fa88c0094b0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.701+0000 7fa8ab233b80 1 -- 172.21.15.133:0/2096602067 shutdown_connections 2023-11-27T15:23:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.701+0000 7fa8ab233b80 1 --2- 172.21.15.133:0/2096602067 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5628629f2ff0 0x5628629f5400 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.701+0000 7fa8ab233b80 1 --2- 172.21.15.133:0/2096602067 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56286298ab10 0x562862998f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.701+0000 7fa8ab233b80 1 --2- 172.21.15.133:0/2096602067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5628627e56b0 0x562862998440 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.701+0000 7fa8ab233b80 1 --2- 172.21.15.133:0/2096602067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562862989db0 0x562862998980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.707 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.701+0000 7fa8ab233b80 1 -- 172.21.15.133:0/2096602067 >> 172.21.15.133:0/2096602067 conn(0x56286297ef50 msgr2=0x56286298da00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.701+0000 7fa878ff9700 0 client.15483 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.702+0000 7fa8ab233b80 1 -- 172.21.15.133:0/2096602067 shutdown_connections 2023-11-27T15:23:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.702+0000 7fa8ab233b80 1 -- 172.21.15.133:0/2096602067 wait complete. 2023-11-27T15:23:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.702+0000 7fa8ab233b80 20 client.15483 trim_cache size 0 max 16384 2023-11-27T15:23:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:23:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:23:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:23:03.708 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:23:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.702+0000 7fa8979ff700 1 -- 172.21.15.133:0/1003419355 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa87c0a1150 msgr2=0x7fa87c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.702+0000 7fa8979ff700 1 --2- 172.21.15.133:0/1003419355 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa87c0a1150 0x7fa87c0a3600 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa88800f590 tx=0x7fa888009920 comp rx=0 tx=0).stop 2023-11-27T15:23:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.702+0000 7fa8979ff700 1 -- 172.21.15.133:0/1003419355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa890101460 msgr2=0x7fa8901986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.702+0000 7fa8979ff700 1 --2- 172.21.15.133:0/1003419355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa890101460 0x7fa8901986d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa88000e9a0 tx=0x7fa88000ecb0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.703+0000 7fa8979ff700 1 -- 172.21.15.133:0/1003419355 shutdown_connections 2023-11-27T15:23:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.703+0000 7fa8979ff700 1 --2- 172.21.15.133:0/1003419355 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa87c0a1150 0x7fa87c0a3600 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.703+0000 7fa8979ff700 1 --2- 172.21.15.133:0/1003419355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa890101df0 0x7fa89019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.703+0000 7fa8979ff700 1 --2- 172.21.15.133:0/1003419355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa890100270 0x7fa890198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.709 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.703+0000 7fa8979ff700 1 --2- 172.21.15.133:0/1003419355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa890101460 0x7fa8901986d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.703+0000 7fa8979ff700 1 -- 172.21.15.133:0/1003419355 >> 172.21.15.133:0/1003419355 conn(0x7fa8900fb890 msgr2=0x7fa890106ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.703+0000 7fa8979ff700 1 -- 172.21.15.133:0/1003419355 shutdown_connections 2023-11-27T15:23:03.710 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:03.703+0000 7fa8979ff700 1 -- 172.21.15.133:0/1003419355 wait complete. 2023-11-27T15:23:05.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:04 smithi177 ceph-mon[131504]: pgmap v353: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:23:05.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:04 smithi174 ceph-mon[131110]: pgmap v353: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:23:05.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:04 smithi133 ceph-mon[97860]: pgmap v353: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2023-11-27T15:23:07.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:06 smithi177 ceph-mon[131504]: pgmap v354: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2023-11-27T15:23:07.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:06 smithi174 ceph-mon[131110]: pgmap v354: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2023-11-27T15:23:07.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:06 smithi133 ceph-mon[97860]: pgmap v354: 129 pgs: 129 active+clean; 651 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2023-11-27T15:23:09.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:08 smithi133 ceph-mon[97860]: pgmap v355: 129 pgs: 129 active+clean; 652 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-11-27T15:23:09.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:08 smithi177 ceph-mon[131504]: pgmap v355: 129 pgs: 129 active+clean; 652 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-11-27T15:23:09.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:08 smithi174 ceph-mon[131110]: pgmap v355: 129 pgs: 129 active+clean; 652 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-11-27T15:23:11.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:10 smithi177 ceph-mon[131504]: pgmap v356: 129 pgs: 129 active+clean; 652 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2023-11-27T15:23:11.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:10 smithi174 ceph-mon[131110]: pgmap v356: 129 pgs: 129 active+clean; 652 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2023-11-27T15:23:11.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:10 smithi133 ceph-mon[97860]: pgmap v356: 129 pgs: 129 active+clean; 652 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2023-11-27T15:23:13.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:12 smithi177 ceph-mon[131504]: pgmap v357: 129 pgs: 129 active+clean; 652 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-11-27T15:23:13.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:12 smithi174 ceph-mon[131110]: pgmap v357: 129 pgs: 129 active+clean; 652 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-11-27T15:23:13.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:12 smithi133 ceph-mon[97860]: pgmap v357: 129 pgs: 129 active+clean; 652 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2023-11-27T15:23:15.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:14 smithi177 ceph-mon[131504]: pgmap v358: 129 pgs: 129 active+clean; 652 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2023-11-27T15:23:15.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:14 smithi174 ceph-mon[131110]: pgmap v358: 129 pgs: 129 active+clean; 652 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2023-11-27T15:23:15.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:14 smithi133 ceph-mon[97860]: pgmap v358: 129 pgs: 129 active+clean; 652 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2023-11-27T15:23:16.941 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.937+0000 7f47bffff700 1 -- 172.21.15.133:0/1473420854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b8101dc0 msgr2=0x7f47b8108680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.937+0000 7f47bffff700 1 --2- 172.21.15.133:0/1473420854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b8101dc0 0x7f47b8108680 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009f90 tx=0x7f47b000b330 comp rx=0 tx=0).stop 2023-11-27T15:23:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.937+0000 7f47bffff700 1 -- 172.21.15.133:0/1473420854 shutdown_connections 2023-11-27T15:23:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.937+0000 7f47bffff700 1 --2- 172.21.15.133:0/1473420854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b8101dc0 0x7f47b8108680 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.937+0000 7f47bffff700 1 --2- 172.21.15.133:0/1473420854 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47b8100240 0x7f47b8100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.937+0000 7f47bffff700 1 --2- 172.21.15.133:0/1473420854 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47b8101430 0x7f47b8101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.937+0000 7f47bffff700 1 -- 172.21.15.133:0/1473420854 >> 172.21.15.133:0/1473420854 conn(0x7f47b80fb840 msgr2=0x7f47b80fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.938+0000 7f47bffff700 1 -- 172.21.15.133:0/1473420854 shutdown_connections 2023-11-27T15:23:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.938+0000 7f47bffff700 1 -- 172.21.15.133:0/1473420854 wait complete. 2023-11-27T15:23:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.939+0000 7f47bffff700 1 Processor -- start 2023-11-27T15:23:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.940+0000 7f47bffff700 1 -- start start 2023-11-27T15:23:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.940+0000 7f47bffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47b8100240 0x7f47b80748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.940+0000 7f47bffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b8101430 0x7f47b8072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.940+0000 7f47bffff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47b8101dc0 0x7f47b8073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.940+0000 7f47bffff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b8073bc0 con 0x7f47b8100240 2023-11-27T15:23:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.940+0000 7f47bffff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b8073d30 con 0x7f47b8101430 2023-11-27T15:23:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.940+0000 7f47bffff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b81a0150 con 0x7f47b8101dc0 2023-11-27T15:23:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.940+0000 7f47bdd9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47b8100240 0x7f47b80748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.940+0000 7f47bd59a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b8101430 0x7f47b8072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.941+0000 7f47bdd9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47b8100240 0x7f47b80748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51194/0 (socket says 172.21.15.133:51194) 2023-11-27T15:23:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.941+0000 7f47bdd9b700 1 -- 172.21.15.133:0/3348996009 learned_addr learned my addr 172.21.15.133:0/3348996009 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.941+0000 7f47be59c700 1 --2- 172.21.15.133:0/3348996009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47b8101dc0 0x7f47b8073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.941+0000 7f47bd59a700 1 -- 172.21.15.133:0/3348996009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47b8101dc0 msgr2=0x7f47b8073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.941+0000 7f47bd59a700 1 --2- 172.21.15.133:0/3348996009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47b8101dc0 0x7f47b8073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.941+0000 7f47bd59a700 1 -- 172.21.15.133:0/3348996009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47b8100240 msgr2=0x7f47b80748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.941+0000 7f47bd59a700 1 --2- 172.21.15.133:0/3348996009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47b8100240 0x7f47b80748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.941+0000 7f47bd59a700 1 -- 172.21.15.133:0/3348996009 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b40095f0 con 0x7f47b8101430 2023-11-27T15:23:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.941+0000 7f47bdd9b700 1 --2- 172.21.15.133:0/3348996009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47b8100240 0x7f47b80748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.941+0000 7f47bd59a700 1 --2- 172.21.15.133:0/3348996009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b8101430 0x7f47b8072ee0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f47a800e9f0 tx=0x7f47a800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.942+0000 7f47aeffd700 1 -- 172.21.15.133:0/3348996009 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a800c9c0 con 0x7f47b8101430 2023-11-27T15:23:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.942+0000 7f47aeffd700 1 -- 172.21.15.133:0/3348996009 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47a8010430 con 0x7f47b8101430 2023-11-27T15:23:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.942+0000 7f47aeffd700 1 -- 172.21.15.133:0/3348996009 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8018680 con 0x7f47b8101430 2023-11-27T15:23:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.942+0000 7f47bffff700 1 -- 172.21.15.133:0/3348996009 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b0009c40 con 0x7f47b8101430 2023-11-27T15:23:16.947 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.942+0000 7f47bffff700 1 -- 172.21.15.133:0/3348996009 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b810bc20 con 0x7f47b8101430 2023-11-27T15:23:16.951 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.947+0000 7f47aeffd700 1 -- 172.21.15.133:0/3348996009 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f47a8010860 con 0x7f47b8101430 2023-11-27T15:23:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.949+0000 7f47d38d2b80 1 Processor -- start 2023-11-27T15:23:16.953 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.950+0000 7f47aeffd700 1 --2- 172.21.15.133:0/3348996009 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f47a40a1150 0x7f47a40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:16.954 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.950+0000 7f47d38d2b80 1 -- start start 2023-11-27T15:23:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.950+0000 7f47aeffd700 1 -- 172.21.15.133:0/3348996009 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f47a8014070 con 0x7f47b8101430 2023-11-27T15:23:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.950+0000 7f47d38d2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede80486b0 0x55ede81f5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.950+0000 7f47d38d2b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f55c0 0x55ede81f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.950+0000 7f47d38d2b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ede81f6320 0x55ede81f6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.950+0000 7f47d38d2b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede80f2000 con 0x55ede81f55c0 2023-11-27T15:23:16.955 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.950+0000 7f47d38d2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede8144840 con 0x55ede80486b0 2023-11-27T15:23:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.950+0000 7f47d38d2b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede813dcd0 con 0x55ede81f6320 2023-11-27T15:23:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.951+0000 7f47bdd9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede80486b0 0x55ede81f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.951+0000 7f47bdd9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede80486b0 0x55ede81f5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:52188/0 (socket says 172.21.15.133:52188) 2023-11-27T15:23:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.951+0000 7f47bdd9b700 1 -- 172.21.15.133:0/2055103276 learned_addr learned my addr 172.21.15.133:0/2055103276 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.951+0000 7f47be59c700 1 --2- 172.21.15.133:0/2055103276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ede81f6320 0x55ede81f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.951+0000 7f47be59c700 1 --2- 172.21.15.133:0/3348996009 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f47a40a1150 0x7f47a40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.951+0000 7f47bd59a700 1 --2- 172.21.15.133:0/2055103276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f55c0 0x55ede81f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.951+0000 7f47bdd9b700 1 -- 172.21.15.133:0/2055103276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ede81f6320 msgr2=0x55ede81f6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.951+0000 7f47bdd9b700 1 --2- 172.21.15.133:0/2055103276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ede81f6320 0x55ede81f6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.951+0000 7f47bdd9b700 1 -- 172.21.15.133:0/2055103276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f55c0 msgr2=0x55ede81f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:16.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.951+0000 7f47bdd9b700 1 --2- 172.21.15.133:0/2055103276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f55c0 0x55ede81f5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.951+0000 7f47bdd9b700 1 -- 172.21.15.133:0/2055103276 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ede7fbf5d0 con 0x55ede80486b0 2023-11-27T15:23:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.951+0000 7f47bd59a700 1 --2- 172.21.15.133:0/2055103276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f55c0 0x55ede81f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:23:16.957 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.951+0000 7f47bdd9b700 1 --2- 172.21.15.133:0/2055103276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede80486b0 0x55ede81f5080 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f47b40118b0 tx=0x7f47b4011bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8399cc043ddd57b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.952+0000 7f47a3fff700 1 -- 172.21.15.133:0/2055103276 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4011250 con 0x55ede80486b0 2023-11-27T15:23:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.952+0000 7f47a3fff700 1 -- 172.21.15.133:0/2055103276 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47b4004cf0 con 0x55ede80486b0 2023-11-27T15:23:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.952+0000 7f47a3fff700 1 -- 172.21.15.133:0/2055103276 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b400f950 con 0x55ede80486b0 2023-11-27T15:23:16.958 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.953+0000 7f47d38d2b80 1 -- 172.21.15.133:0/2055103276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede80486b0 msgr2=0x55ede81f5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.953+0000 7f47be59c700 1 --2- 172.21.15.133:0/3348996009 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f47a40a1150 0x7f47a40a3600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009c10 tx=0x7f47b0009ac0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.953+0000 7f47d38d2b80 1 --2- 172.21.15.133:0/2055103276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede80486b0 0x55ede81f5080 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f47b40118b0 tx=0x7f47b4011bc0 comp rx=0 tx=0).stop 2023-11-27T15:23:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.953+0000 7f47d38d2b80 1 -- 172.21.15.133:0/2055103276 shutdown_connections 2023-11-27T15:23:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.953+0000 7f47d38d2b80 1 --2- 172.21.15.133:0/2055103276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ede81f6320 0x55ede81f6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.953+0000 7f47d38d2b80 1 --2- 172.21.15.133:0/2055103276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede80486b0 0x55ede81f5080 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.953+0000 7f47d38d2b80 1 --2- 172.21.15.133:0/2055103276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f55c0 0x55ede81f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.953+0000 7f47d38d2b80 1 -- 172.21.15.133:0/2055103276 >> 172.21.15.133:0/2055103276 conn(0x55ede81e1f50 msgr2=0x55ede81e7100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.953+0000 7f47d38d2b80 1 -- 172.21.15.133:0/2055103276 shutdown_connections 2023-11-27T15:23:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.953+0000 7f47d38d2b80 1 -- 172.21.15.133:0/2055103276 wait complete. 2023-11-27T15:23:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.954+0000 7f47d38d2b80 1 -- start start 2023-11-27T15:23:16.959 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.954+0000 7f47d38d2b80 1 Processor -- start 2023-11-27T15:23:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.955+0000 7f47d38d2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede80486b0 0x55ede81fe470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.955+0000 7f47d38d2b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f55c0 0x55ede81fe9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.955+0000 7f47d38d2b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ede81f6320 0x55ede82059f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.955+0000 7f47d38d2b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede80df880 con 0x55ede81f55c0 2023-11-27T15:23:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.955+0000 7f47d38d2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede80df520 con 0x55ede80486b0 2023-11-27T15:23:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.955+0000 7f47d38d2b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede7fc18c0 con 0x55ede81f6320 2023-11-27T15:23:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.955+0000 7f47d38d2b80 10 client.0 _ll_register_callbacks cb 0x55ede80fd720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.955+0000 7f47bd59a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f55c0 0x55ede81fe9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.956+0000 7f47bdd9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede80486b0 0x55ede81fe470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:16.962 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.956+0000 7f47bd59a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f55c0 0x55ede81fe9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51210/0 (socket says 172.21.15.133:51210) 2023-11-27T15:23:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.956+0000 7f47bd59a700 1 -- 172.21.15.133:0/2623898743 learned_addr learned my addr 172.21.15.133:0/2623898743 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.956+0000 7f47bdd9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede80486b0 0x55ede81fe470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:52190/0 (socket says 172.21.15.133:52190) 2023-11-27T15:23:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.956+0000 7f47be59c700 1 --2- 172.21.15.133:0/2623898743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ede81f6320 0x55ede82059f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.956+0000 7f47bdd9b700 1 -- 172.21.15.133:0/2623898743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ede81f6320 msgr2=0x55ede82059f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.956+0000 7f47bdd9b700 1 --2- 172.21.15.133:0/2623898743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ede81f6320 0x55ede82059f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.956+0000 7f47bdd9b700 1 -- 172.21.15.133:0/2623898743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f55c0 msgr2=0x55ede81fe9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.956+0000 7f47bdd9b700 1 --2- 172.21.15.133:0/2623898743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f55c0 0x55ede81fe9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.956+0000 7f47bdd9b700 1 -- 172.21.15.133:0/2623898743 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b40114e0 con 0x55ede80486b0 2023-11-27T15:23:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.956+0000 7f47bd59a700 1 --2- 172.21.15.133:0/2623898743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f55c0 0x55ede81fe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:16.963 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.957+0000 7f47bdd9b700 1 --2- 172.21.15.133:0/2623898743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede80486b0 0x55ede81fe470 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f47b401b040 tx=0x7f47b4005090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ee3799f14ff9102 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.957+0000 7f47a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:23:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.957+0000 7f47a17fa700 1 -- 172.21.15.133:0/2623898743 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4004b60 con 0x55ede80486b0 2023-11-27T15:23:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.957+0000 7f47a17fa700 1 -- 172.21.15.133:0/2623898743 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f47b400f6f0 con 0x55ede80486b0 2023-11-27T15:23:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.957+0000 7f47d38d2b80 1 -- 172.21.15.133:0/2623898743 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ede81df710 con 0x55ede80486b0 2023-11-27T15:23:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.958+0000 7f47a17fa700 1 -- 172.21.15.133:0/2623898743 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47b4005700 con 0x55ede80486b0 2023-11-27T15:23:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.958+0000 7f47d38d2b80 10 client.25541 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.958+0000 7f47d38d2b80 1 -- 172.21.15.133:0/2623898743 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ede8104130 con 0x55ede80486b0 2023-11-27T15:23:16.964 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.958+0000 7f47a17fa700 1 -- 172.21.15.133:0/2623898743 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47b400fd70 con 0x55ede80486b0 2023-11-27T15:23:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.959+0000 7f47a17fa700 1 -- 172.21.15.133:0/2623898743 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f47b4021040 con 0x55ede80486b0 2023-11-27T15:23:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.959+0000 7f47d38d2b80 10 client.25541 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.959+0000 7f47d38d2b80 10 client.25541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.959+0000 7f47d38d2b80 20 client.25541 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.959+0000 7f47d38d2b80 1 --2- 172.21.15.133:0/2623898743 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ede8209fc0 0x55ede820c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.959+0000 7f47d38d2b80 4 client.25541 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.959+0000 7f47d38d2b80 1 -- 172.21.15.133:0/2623898743 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ede81df710 con 0x55ede8209fc0 2023-11-27T15:23:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.960+0000 7f47bd59a700 1 --2- 172.21.15.133:0/2623898743 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ede8209fc0 0x55ede820c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.960+0000 7f47bd59a700 1 --2- 172.21.15.133:0/2623898743 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ede8209fc0 0x55ede820c410 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15cda12f86ec3e3f server_cookie=61767fc74fa570bd in_seq=0 out_seq=0 2023-11-27T15:23:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.960+0000 7f47a17fa700 10 client.25541 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.961+0000 7f47a17fa700 1 -- 172.21.15.133:0/2623898743 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ede81df710 con 0x55ede8209fc0 2023-11-27T15:23:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.961+0000 7f47a17fa700 10 client.25541 handle_command_reply: tid=0 2023-11-27T15:23:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.961+0000 7f47d38d2b80 1 client.25541 shutdown 2023-11-27T15:23:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.961+0000 7f47d38d2b80 1 -- 172.21.15.133:0/2623898743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede80486b0 msgr2=0x55ede81fe470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.961+0000 7f47d38d2b80 1 --2- 172.21.15.133:0/2623898743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede80486b0 0x55ede81fe470 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f47b401b040 tx=0x7f47b4005090 comp rx=0 tx=0).stop 2023-11-27T15:23:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.961+0000 7f47d38d2b80 1 -- 172.21.15.133:0/2623898743 shutdown_connections 2023-11-27T15:23:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.961+0000 7f47d38d2b80 1 --2- 172.21.15.133:0/2623898743 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ede8209fc0 0x55ede820c410 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:16.967 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.961+0000 7f47d38d2b80 1 --2- 172.21.15.133:0/2623898743 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ede81f6320 0x55ede82059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.961+0000 7f47d38d2b80 1 --2- 172.21.15.133:0/2623898743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede80486b0 0x55ede81fe470 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.961+0000 7f47d38d2b80 1 --2- 172.21.15.133:0/2623898743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f55c0 0x55ede81fe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.961+0000 7f47d38d2b80 1 -- 172.21.15.133:0/2623898743 >> 172.21.15.133:0/2623898743 conn(0x55ede81e1f50 msgr2=0x55ede81e67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.961+0000 7f47a17fa700 0 client.25541 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.962+0000 7f47d38d2b80 1 -- 172.21.15.133:0/2623898743 shutdown_connections 2023-11-27T15:23:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.962+0000 7f47d38d2b80 1 -- 172.21.15.133:0/2623898743 wait complete. 2023-11-27T15:23:16.968 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.962+0000 7f47d38d2b80 20 client.25541 trim_cache size 0 max 16384 2023-11-27T15:23:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.991+0000 7f47d38d2b80 1 Processor -- start 2023-11-27T15:23:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.991+0000 7f47d38d2b80 1 -- start start 2023-11-27T15:23:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.992+0000 7f47d38d2b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ede80486b0 0x55ede81fc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.992+0000 7f47d38d2b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f55c0 0x55ede81fcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.992+0000 7f47d38d2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede81f6320 0x55ede82059b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.992+0000 7f47d38d2b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede81e7fe0 con 0x55ede81f55c0 2023-11-27T15:23:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.992+0000 7f47d38d2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede81df250 con 0x55ede81f6320 2023-11-27T15:23:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.992+0000 7f47d38d2b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede81f5c30 con 0x55ede80486b0 2023-11-27T15:23:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.992+0000 7f47bdd9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ede80486b0 0x55ede81fc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.992+0000 7f47bdd9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ede80486b0 0x55ede81fc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:36238/0 (socket says 172.21.15.133:36238) 2023-11-27T15:23:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.992+0000 7f47bdd9b700 1 -- 172.21.15.133:0/1814283791 learned_addr learned my addr 172.21.15.133:0/1814283791 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.992+0000 7f47be59c700 1 --2- 172.21.15.133:0/1814283791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede81f6320 0x55ede82059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.992+0000 7f47bd59a700 1 --2- 172.21.15.133:0/1814283791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f55c0 0x55ede81fcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.993+0000 7f47bdd9b700 1 -- 172.21.15.133:0/1814283791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede81f6320 msgr2=0x55ede82059b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.993+0000 7f47bdd9b700 1 --2- 172.21.15.133:0/1814283791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede81f6320 0x55ede82059b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.993+0000 7f47bdd9b700 1 -- 172.21.15.133:0/1814283791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f55c0 msgr2=0x55ede81fcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.993+0000 7f47bdd9b700 1 --2- 172.21.15.133:0/1814283791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f55c0 0x55ede81fcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.993+0000 7f47bdd9b700 1 -- 172.21.15.133:0/1814283791 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ede8103e80 con 0x55ede80486b0 2023-11-27T15:23:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.993+0000 7f47bd59a700 1 --2- 172.21.15.133:0/1814283791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f55c0 0x55ede81fcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.993+0000 7f47bdd9b700 1 --2- 172.21.15.133:0/1814283791 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ede80486b0 0x55ede81fc6c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f47b4011be0 tx=0x7f47b4001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84df88b2cb932eef server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.994+0000 7f47acff9700 1 -- 172.21.15.133:0/1814283791 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b40055f0 con 0x55ede80486b0 2023-11-27T15:23:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.994+0000 7f47acff9700 1 -- 172.21.15.133:0/1814283791 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47b4015070 con 0x55ede80486b0 2023-11-27T15:23:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.994+0000 7f47acff9700 1 -- 172.21.15.133:0/1814283791 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b400f6c0 con 0x55ede80486b0 2023-11-27T15:23:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.995+0000 7f47d38d2b80 1 -- 172.21.15.133:0/1814283791 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ede80486b0 msgr2=0x55ede81fc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.995+0000 7f47d38d2b80 1 --2- 172.21.15.133:0/1814283791 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ede80486b0 0x55ede81fc6c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f47b4011be0 tx=0x7f47b4001e50 comp rx=0 tx=0).stop 2023-11-27T15:23:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.995+0000 7f47d38d2b80 1 -- 172.21.15.133:0/1814283791 shutdown_connections 2023-11-27T15:23:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.995+0000 7f47d38d2b80 1 --2- 172.21.15.133:0/1814283791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede81f6320 0x55ede82059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.995+0000 7f47d38d2b80 1 --2- 172.21.15.133:0/1814283791 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ede80486b0 0x55ede81fc6c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.995+0000 7f47d38d2b80 1 --2- 172.21.15.133:0/1814283791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f55c0 0x55ede81fcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.995+0000 7f47d38d2b80 1 -- 172.21.15.133:0/1814283791 >> 172.21.15.133:0/1814283791 conn(0x55ede81e1f50 msgr2=0x55ede8210db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.995+0000 7f47d38d2b80 1 -- 172.21.15.133:0/1814283791 shutdown_connections 2023-11-27T15:23:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.995+0000 7f47d38d2b80 1 -- 172.21.15.133:0/1814283791 wait complete. 2023-11-27T15:23:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.995+0000 7f47d38d2b80 1 -- start start 2023-11-27T15:23:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.996+0000 7f47d38d2b80 1 Processor -- start 2023-11-27T15:23:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.996+0000 7f47d38d2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede80486b0 0x55ede8201500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.996+0000 7f47d38d2b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ede81f55c0 0x55ede8201a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.996+0000 7f47d38d2b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f6320 0x55ede8204a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.996+0000 7f47d38d2b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede8204fa0 con 0x55ede81f6320 2023-11-27T15:23:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.996+0000 7f47d38d2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede8205110 con 0x55ede80486b0 2023-11-27T15:23:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.996+0000 7f47d38d2b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ede82052d0 con 0x55ede81f55c0 2023-11-27T15:23:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.996+0000 7f47d38d2b80 10 client.0 _ll_register_callbacks cb 0x7f47b4011700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.997+0000 7f47bdd9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede80486b0 0x55ede8201500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.997+0000 7f47bdd9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede80486b0 0x55ede8201500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:52196/0 (socket says 172.21.15.133:52196) 2023-11-27T15:23:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.997+0000 7f47bdd9b700 1 -- 172.21.15.133:0/252762043 learned_addr learned my addr 172.21.15.133:0/252762043 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.997+0000 7f47be59c700 1 --2- 172.21.15.133:0/252762043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f6320 0x55ede8204a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.997+0000 7f47bdd9b700 1 -- 172.21.15.133:0/252762043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ede81f55c0 msgr2=0x55ede8201a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.997+0000 7f47bdd9b700 1 --2- 172.21.15.133:0/252762043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ede81f55c0 0x55ede8201a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.997+0000 7f47bdd9b700 1 -- 172.21.15.133:0/252762043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f6320 msgr2=0x55ede8204a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.997+0000 7f47bdd9b700 1 --2- 172.21.15.133:0/252762043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f6320 0x55ede8204a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.997+0000 7f47bdd9b700 1 -- 172.21.15.133:0/252762043 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b810bc20 con 0x55ede80486b0 2023-11-27T15:23:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.997+0000 7f47bdd9b700 1 --2- 172.21.15.133:0/252762043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede80486b0 0x55ede8201500 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f47b4005f50 tx=0x7f47b400ff60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a99fcb53cd5b2f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.997+0000 7f47a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:23:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.998+0000 7f47a17fa700 1 -- 172.21.15.133:0/252762043 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b400f6c0 con 0x55ede80486b0 2023-11-27T15:23:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.998+0000 7f47d38d2b80 1 -- 172.21.15.133:0/252762043 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47b4004cf0 con 0x55ede80486b0 2023-11-27T15:23:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.998+0000 7f47a17fa700 1 -- 172.21.15.133:0/252762043 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f47b4023680 con 0x55ede80486b0 2023-11-27T15:23:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.998+0000 7f47a17fa700 1 -- 172.21.15.133:0/252762043 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47b4023840 con 0x55ede80486b0 2023-11-27T15:23:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.998+0000 7f47d38d2b80 10 client.25547 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.998+0000 7f47d38d2b80 1 -- 172.21.15.133:0/252762043 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ede8104130 con 0x55ede80486b0 2023-11-27T15:23:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.998+0000 7f47a17fa700 1 -- 172.21.15.133:0/252762043 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47b4004cf0 con 0x55ede80486b0 2023-11-27T15:23:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.999+0000 7f47a17fa700 1 -- 172.21.15.133:0/252762043 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f47b4021070 con 0x55ede80486b0 2023-11-27T15:23:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.999+0000 7f47d38d2b80 10 client.25547 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.999+0000 7f47d38d2b80 10 client.25547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.999+0000 7f47d38d2b80 20 client.25547 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.999+0000 7f47d38d2b80 1 --2- 172.21.15.133:0/252762043 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ede825e7f0 0x55ede8260c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.999+0000 7f47d38d2b80 4 client.25547 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:23:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.999+0000 7f47d38d2b80 1 -- 172.21.15.133:0/252762043 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ede8100040 con 0x55ede825e7f0 2023-11-27T15:23:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:16.999+0000 7f47bd59a700 1 --2- 172.21.15.133:0/252762043 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ede825e7f0 0x55ede8260c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.000+0000 7f47bd59a700 1 --2- 172.21.15.133:0/252762043 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ede825e7f0 0x55ede8260c00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac7f577786942d2c server_cookie=6dfceea9393f30ea in_seq=0 out_seq=0 2023-11-27T15:23:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.000+0000 7f47a17fa700 10 client.25547 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.004+0000 7f47a17fa700 1 -- 172.21.15.133:0/252762043 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ede8100040 con 0x55ede825e7f0 2023-11-27T15:23:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.004+0000 7f47a17fa700 10 client.25547 handle_command_reply: tid=0 2023-11-27T15:23:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.004+0000 7f47d38d2b80 1 client.25547 shutdown 2023-11-27T15:23:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.004+0000 7f47d38d2b80 1 -- 172.21.15.133:0/252762043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede80486b0 msgr2=0x55ede8201500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.004+0000 7f47d38d2b80 1 --2- 172.21.15.133:0/252762043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede80486b0 0x55ede8201500 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f47b4005f50 tx=0x7f47b400ff60 comp rx=0 tx=0).stop 2023-11-27T15:23:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.007+0000 7f47d38d2b80 1 -- 172.21.15.133:0/252762043 shutdown_connections 2023-11-27T15:23:17.010 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.007+0000 7f47d38d2b80 1 --2- 172.21.15.133:0/252762043 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ede825e7f0 0x55ede8260c00 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.007+0000 7f47d38d2b80 1 --2- 172.21.15.133:0/252762043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ede81f6320 0x55ede8204a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.007+0000 7f47d38d2b80 1 --2- 172.21.15.133:0/252762043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ede80486b0 0x55ede8201500 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.007+0000 7f47d38d2b80 1 --2- 172.21.15.133:0/252762043 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ede81f55c0 0x55ede8201a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.007+0000 7f47d38d2b80 1 -- 172.21.15.133:0/252762043 >> 172.21.15.133:0/252762043 conn(0x55ede81e1f50 msgr2=0x55ede81fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.007+0000 7f47a17fa700 0 client.25547 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.007+0000 7f47d38d2b80 1 -- 172.21.15.133:0/252762043 shutdown_connections 2023-11-27T15:23:17.011 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.007+0000 7f47d38d2b80 1 -- 172.21.15.133:0/252762043 wait complete. 2023-11-27T15:23:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.008+0000 7f47d38d2b80 20 client.25547 trim_cache size 0 max 16384 2023-11-27T15:23:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:23:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:23:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:23:17.012 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:23:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.009+0000 7f47bffff700 1 -- 172.21.15.133:0/3348996009 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f47a40a1150 msgr2=0x7f47a40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.009+0000 7f47bffff700 1 --2- 172.21.15.133:0/3348996009 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f47a40a1150 0x7f47a40a3600 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f47b0009c10 tx=0x7f47b0009ac0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.009+0000 7f47bffff700 1 -- 172.21.15.133:0/3348996009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b8101430 msgr2=0x7f47b8072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.009+0000 7f47bffff700 1 --2- 172.21.15.133:0/3348996009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b8101430 0x7f47b8072ee0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f47a800e9f0 tx=0x7f47a800ed00 comp rx=0 tx=0).stop 2023-11-27T15:23:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.010+0000 7f47bffff700 1 -- 172.21.15.133:0/3348996009 shutdown_connections 2023-11-27T15:23:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.010+0000 7f47bffff700 1 --2- 172.21.15.133:0/3348996009 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f47a40a1150 0x7f47a40a3600 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.010+0000 7f47bffff700 1 --2- 172.21.15.133:0/3348996009 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f47b8101dc0 0x7f47b8073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.010+0000 7f47bffff700 1 --2- 172.21.15.133:0/3348996009 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47b8100240 0x7f47b80748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.010+0000 7f47bffff700 1 --2- 172.21.15.133:0/3348996009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b8101430 0x7f47b8072ee0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.010+0000 7f47bffff700 1 -- 172.21.15.133:0/3348996009 >> 172.21.15.133:0/3348996009 conn(0x7f47b80fb840 msgr2=0x7f47b8104870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.010+0000 7f47bffff700 1 -- 172.21.15.133:0/3348996009 shutdown_connections 2023-11-27T15:23:17.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:17.010+0000 7f47bffff700 1 -- 172.21.15.133:0/3348996009 wait complete. 2023-11-27T15:23:17.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:16 smithi177 ceph-mon[131504]: pgmap v359: 129 pgs: 129 active+clean; 652 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2023-11-27T15:23:17.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:16 smithi174 ceph-mon[131110]: pgmap v359: 129 pgs: 129 active+clean; 652 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2023-11-27T15:23:17.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:16 smithi133 ceph-mon[97860]: pgmap v359: 129 pgs: 129 active+clean; 652 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2023-11-27T15:23:19.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:18 smithi133 ceph-mon[97860]: pgmap v360: 129 pgs: 129 active+clean; 652 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 8 op/s 2023-11-27T15:23:19.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:18 smithi177 ceph-mon[131504]: pgmap v360: 129 pgs: 129 active+clean; 652 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 8 op/s 2023-11-27T15:23:19.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:18 smithi174 ceph-mon[131110]: pgmap v360: 129 pgs: 129 active+clean; 652 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 8 op/s 2023-11-27T15:23:20.971 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:23:21.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:20 smithi177 ceph-mon[131504]: pgmap v361: 129 pgs: 129 active+clean; 652 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-11-27T15:23:21.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:20 smithi174 ceph-mon[131110]: pgmap v361: 129 pgs: 129 active+clean; 652 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-11-27T15:23:21.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.236+0000 7f37395ff700 1 -- 172.21.15.133:0/1158812386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3734107b30 msgr2=0x7f3734107fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.236+0000 7f37395ff700 1 --2- 172.21.15.133:0/1158812386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3734107b30 0x7f3734107fa0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3724009990 tx=0x7f3724009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.237+0000 7f37395ff700 1 -- 172.21.15.133:0/1158812386 shutdown_connections 2023-11-27T15:23:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.237+0000 7f37395ff700 1 --2- 172.21.15.133:0/1158812386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37341084e0 0x7f373410ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.237+0000 7f37395ff700 1 --2- 172.21.15.133:0/1158812386 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3734106cc0 0x7f37341070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.237+0000 7f37395ff700 1 --2- 172.21.15.133:0/1158812386 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3734107b30 0x7f3734107fa0 secure :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3724009990 tx=0x7f3724009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.237+0000 7f37395ff700 1 -- 172.21.15.133:0/1158812386 >> 172.21.15.133:0/1158812386 conn(0x7f37340759e0 msgr2=0x7f3734077e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.237+0000 7f37395ff700 1 -- 172.21.15.133:0/1158812386 shutdown_connections 2023-11-27T15:23:21.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.237+0000 7f37395ff700 1 -- 172.21.15.133:0/1158812386 wait complete. 2023-11-27T15:23:21.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.239+0000 7f37395ff700 1 Processor -- start 2023-11-27T15:23:21.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.239+0000 7f37395ff700 1 -- start start 2023-11-27T15:23:21.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.240+0000 7f37395ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3734106cc0 0x7f373407e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.240+0000 7f37395ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37341084e0 0x7f373407e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.240+0000 7f37395ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37341aa840 0x7f37341acc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.240+0000 7f37327fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37341084e0 0x7f373407e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:21.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.240+0000 7f3732ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3734106cc0 0x7f373407e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:21.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.240+0000 7f37327fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37341084e0 0x7f373407e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:52204/0 (socket says 172.21.15.133:52204) 2023-11-27T15:23:21.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.240+0000 7f3732ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3734106cc0 0x7f373407e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:36282/0 (socket says 172.21.15.133:36282) 2023-11-27T15:23:21.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.241+0000 7f37395ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373407f040 con 0x7f37341aa840 2023-11-27T15:23:21.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.241+0000 7f37395ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37341ad160 con 0x7f37341084e0 2023-11-27T15:23:21.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.241+0000 7f37327fc700 1 -- 172.21.15.133:0/1942158653 learned_addr learned my addr 172.21.15.133:0/1942158653 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:21.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.241+0000 7f37337fe700 1 --2- 172.21.15.133:0/1942158653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37341aa840 0x7f37341acc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:21.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.241+0000 7f37395ff700 1 -- 172.21.15.133:0/1942158653 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37341ad2f0 con 0x7f3734106cc0 2023-11-27T15:23:21.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.242+0000 7f37327fc700 1 -- 172.21.15.133:0/1942158653 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3734106cc0 msgr2=0x7f373407e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:21.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.242+0000 7f37327fc700 1 --2- 172.21.15.133:0/1942158653 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3734106cc0 0x7f373407e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.242+0000 7f37327fc700 1 -- 172.21.15.133:0/1942158653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37341aa840 msgr2=0x7f37341acc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:21.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.242+0000 7f37327fc700 1 --2- 172.21.15.133:0/1942158653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37341aa840 0x7f37341acc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.242+0000 7f37327fc700 1 -- 172.21.15.133:0/1942158653 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f372c009c40 con 0x7f37341084e0 2023-11-27T15:23:21.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.242+0000 7f37327fc700 1 --2- 172.21.15.133:0/1942158653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37341084e0 0x7f373407e930 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f372400b680 tx=0x7f3724005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:21.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.242+0000 7f371bfff700 1 -- 172.21.15.133:0/1942158653 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372401d070 con 0x7f37341084e0 2023-11-27T15:23:21.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.242+0000 7f371bfff700 1 -- 172.21.15.133:0/1942158653 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3724021410 con 0x7f37341084e0 2023-11-27T15:23:21.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.242+0000 7f371bfff700 1 -- 172.21.15.133:0/1942158653 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724017650 con 0x7f37341084e0 2023-11-27T15:23:21.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.242+0000 7f37395ff700 1 -- 172.21.15.133:0/1942158653 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37240095f0 con 0x7f37341084e0 2023-11-27T15:23:21.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.243+0000 7f37395ff700 1 -- 172.21.15.133:0/1942158653 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3734112330 con 0x7f37341084e0 2023-11-27T15:23:21.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.248+0000 7f371bfff700 1 -- 172.21.15.133:0/1942158653 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f37240217f0 con 0x7f37341084e0 2023-11-27T15:23:21.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.251+0000 7f371bfff700 1 --2- 172.21.15.133:0/1942158653 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f371c0a11d0 0x7f371c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.251+0000 7f371bfff700 1 -- 172.21.15.133:0/1942158653 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f37240cdee0 con 0x7f37341084e0 2023-11-27T15:23:21.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.252+0000 7f3732ffd700 1 --2- 172.21.15.133:0/1942158653 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f371c0a11d0 0x7f371c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:21.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.252+0000 7f374cffcb80 1 Processor -- start 2023-11-27T15:23:21.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.253+0000 7f3732ffd700 1 --2- 172.21.15.133:0/1942158653 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f371c0a11d0 0x7f371c0a3680 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f372800ac50 tx=0x7f372800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:21.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.253+0000 7f374cffcb80 1 -- start start 2023-11-27T15:23:21.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.253+0000 7f374cffcb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f56c00 0x560cf9f57010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.253+0000 7f374cffcb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cf9f57550 0x560cf9f57b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.253+0000 7f374cffcb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cf9f58110 0x560cf9f58580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.253+0000 7f374cffcb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf9e5c060 con 0x560cf9f56c00 2023-11-27T15:23:21.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.253+0000 7f374cffcb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf9eae8a0 con 0x560cf9f58110 2023-11-27T15:23:21.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.253+0000 7f374cffcb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf9ea7d30 con 0x560cf9f57550 2023-11-27T15:23:21.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.254+0000 7f37337fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cf9f58110 0x560cf9f58580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:21.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.254+0000 7f37337fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cf9f58110 0x560cf9f58580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:52206/0 (socket says 172.21.15.133:52206) 2023-11-27T15:23:21.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.254+0000 7f37337fe700 1 -- 172.21.15.133:0/270498503 learned_addr learned my addr 172.21.15.133:0/270498503 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:21.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.254+0000 7f3732ffd700 1 --2- 172.21.15.133:0/270498503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f56c00 0x560cf9f57010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:21.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.254+0000 7f37327fc700 1 --2- 172.21.15.133:0/270498503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cf9f57550 0x560cf9f57b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:21.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.254+0000 7f37337fe700 1 -- 172.21.15.133:0/270498503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cf9f57550 msgr2=0x560cf9f57b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:21.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.254+0000 7f37337fe700 1 --2- 172.21.15.133:0/270498503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cf9f57550 0x560cf9f57b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.254+0000 7f37337fe700 1 -- 172.21.15.133:0/270498503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f56c00 msgr2=0x560cf9f57010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:21.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.254+0000 7f37337fe700 1 --2- 172.21.15.133:0/270498503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f56c00 0x560cf9f57010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.254+0000 7f37337fe700 1 -- 172.21.15.133:0/270498503 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cf9d29630 con 0x560cf9f58110 2023-11-27T15:23:21.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.254+0000 7f3732ffd700 1 --2- 172.21.15.133:0/270498503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f56c00 0x560cf9f57010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:21.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.255+0000 7f37337fe700 1 --2- 172.21.15.133:0/270498503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cf9f58110 0x560cf9f58580 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f372c00e730 tx=0x7f372c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c62345c252e7cbce server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.255+0000 7f37197fa700 1 -- 172.21.15.133:0/270498503 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c00e040 con 0x560cf9f58110 2023-11-27T15:23:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.255+0000 7f37197fa700 1 -- 172.21.15.133:0/270498503 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f372c00f040 con 0x560cf9f58110 2023-11-27T15:23:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.255+0000 7f37197fa700 1 -- 172.21.15.133:0/270498503 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c007700 con 0x560cf9f58110 2023-11-27T15:23:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.256+0000 7f374cffcb80 1 -- 172.21.15.133:0/270498503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cf9f58110 msgr2=0x560cf9f58580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.256+0000 7f374cffcb80 1 --2- 172.21.15.133:0/270498503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cf9f58110 0x560cf9f58580 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f372c00e730 tx=0x7f372c009920 comp rx=0 tx=0).stop 2023-11-27T15:23:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.257+0000 7f374cffcb80 1 -- 172.21.15.133:0/270498503 shutdown_connections 2023-11-27T15:23:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.257+0000 7f374cffcb80 1 --2- 172.21.15.133:0/270498503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cf9f58110 0x560cf9f58580 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.257+0000 7f374cffcb80 1 --2- 172.21.15.133:0/270498503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f56c00 0x560cf9f57010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.257+0000 7f374cffcb80 1 --2- 172.21.15.133:0/270498503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cf9f57550 0x560cf9f57b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.257+0000 7f374cffcb80 1 -- 172.21.15.133:0/270498503 >> 172.21.15.133:0/270498503 conn(0x560cf9f4c360 msgr2=0x560cf9f51670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:21.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.257+0000 7f374cffcb80 1 -- 172.21.15.133:0/270498503 shutdown_connections 2023-11-27T15:23:21.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.257+0000 7f374cffcb80 1 -- 172.21.15.133:0/270498503 wait complete. 2023-11-27T15:23:21.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.257+0000 7f374cffcb80 1 -- start start 2023-11-27T15:23:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.258+0000 7f374cffcb80 1 Processor -- start 2023-11-27T15:23:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.258+0000 7f374cffcb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cf9f56c00 0x560cf9f60500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.258+0000 7f374cffcb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cf9f57550 0x560cf9f60a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.258+0000 7f374cffcb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f58110 0x560cf9f67a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.258+0000 7f374cffcb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf9e498e0 con 0x560cf9f58110 2023-11-27T15:23:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.258+0000 7f374cffcb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf9e49580 con 0x560cf9f57550 2023-11-27T15:23:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.258+0000 7f374cffcb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf9d2b920 con 0x560cf9f56c00 2023-11-27T15:23:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.258+0000 7f374cffcb80 10 client.0 _ll_register_callbacks cb 0x560cf9e67780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.259+0000 7f37337fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f58110 0x560cf9f67a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.259+0000 7f37337fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f58110 0x560cf9f67a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51266/0 (socket says 172.21.15.133:51266) 2023-11-27T15:23:21.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.259+0000 7f37337fe700 1 -- 172.21.15.133:0/3773094078 learned_addr learned my addr 172.21.15.133:0/3773094078 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:21.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.259+0000 7f37337fe700 1 -- 172.21.15.133:0/3773094078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cf9f56c00 msgr2=0x560cf9f60500 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-27T15:23:21.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.259+0000 7f37337fe700 1 --2- 172.21.15.133:0/3773094078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cf9f56c00 0x560cf9f60500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.260+0000 7f37337fe700 1 -- 172.21.15.133:0/3773094078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cf9f57550 msgr2=0x560cf9f60a40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-27T15:23:21.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.260+0000 7f37337fe700 1 --2- 172.21.15.133:0/3773094078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cf9f57550 0x560cf9f60a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.260+0000 7f37337fe700 1 -- 172.21.15.133:0/3773094078 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f372c00e410 con 0x560cf9f58110 2023-11-27T15:23:21.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.260+0000 7f37337fe700 1 --2- 172.21.15.133:0/3773094078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f58110 0x560cf9f67a80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f372c00e170 tx=0x7f372c0043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff877e4632dfe97 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:21.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.260+0000 7f3716ff5700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:23:21.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.260+0000 7f3716ff5700 1 -- 172.21.15.133:0/3773094078 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c003c00 con 0x560cf9f58110 2023-11-27T15:23:21.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.261+0000 7f3716ff5700 1 -- 172.21.15.133:0/3773094078 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f372c003d60 con 0x560cf9f58110 2023-11-27T15:23:21.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.261+0000 7f374cffcb80 1 -- 172.21.15.133:0/3773094078 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560cf9f49b20 con 0x560cf9f58110 2023-11-27T15:23:21.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.261+0000 7f3716ff5700 1 -- 172.21.15.133:0/3773094078 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f372c007500 con 0x560cf9f58110 2023-11-27T15:23:21.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.262+0000 7f374cffcb80 10 client.15510 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:21.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.262+0000 7f374cffcb80 1 -- 172.21.15.133:0/3773094078 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560cf9e6e190 con 0x560cf9f58110 2023-11-27T15:23:21.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.262+0000 7f3716ff5700 1 -- 172.21.15.133:0/3773094078 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f372c030020 con 0x560cf9f58110 2023-11-27T15:23:21.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.263+0000 7f3716ff5700 1 -- 172.21.15.133:0/3773094078 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f372c028870 con 0x560cf9f58110 2023-11-27T15:23:21.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.263+0000 7f374cffcb80 10 client.15510 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:21.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.263+0000 7f374cffcb80 10 client.15510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:21.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.263+0000 7f374cffcb80 20 client.15510 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:21.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.263+0000 7f374cffcb80 1 --2- 172.21.15.133:0/3773094078 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x560cf9f6c1c0 0x560cf9f6e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.263+0000 7f374cffcb80 4 client.15510 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:21.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.263+0000 7f374cffcb80 1 -- 172.21.15.133:0/3773094078 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560cf9e6a0a0 con 0x560cf9f6c1c0 2023-11-27T15:23:21.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.264+0000 7f3732ffd700 1 --2- 172.21.15.133:0/3773094078 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x560cf9f6c1c0 0x560cf9f6e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:21.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.264+0000 7f3716ff5700 10 client.15510 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:21.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.264+0000 7f3732ffd700 1 --2- 172.21.15.133:0/3773094078 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x560cf9f6c1c0 0x560cf9f6e5d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ca3dedaa26b6be server_cookie=ab75907f38ef7ec6 in_seq=0 out_seq=0 2023-11-27T15:23:21.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.265+0000 7f3716ff5700 1 -- 172.21.15.133:0/3773094078 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560cf9e6a0a0 con 0x560cf9f6c1c0 2023-11-27T15:23:21.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.265+0000 7f3716ff5700 10 client.15510 handle_command_reply: tid=0 2023-11-27T15:23:21.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.265+0000 7f374cffcb80 1 client.15510 shutdown 2023-11-27T15:23:21.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.266+0000 7f374cffcb80 1 -- 172.21.15.133:0/3773094078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f58110 msgr2=0x560cf9f67a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:21.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.266+0000 7f374cffcb80 1 --2- 172.21.15.133:0/3773094078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f58110 0x560cf9f67a80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f372c00e170 tx=0x7f372c0043f0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.266+0000 7f374cffcb80 1 -- 172.21.15.133:0/3773094078 shutdown_connections 2023-11-27T15:23:21.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.266+0000 7f374cffcb80 1 --2- 172.21.15.133:0/3773094078 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x560cf9f6c1c0 0x560cf9f6e5d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.266+0000 7f374cffcb80 1 --2- 172.21.15.133:0/3773094078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f58110 0x560cf9f67a80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.266+0000 7f374cffcb80 1 --2- 172.21.15.133:0/3773094078 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cf9f56c00 0x560cf9f60500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.266+0000 7f374cffcb80 1 --2- 172.21.15.133:0/3773094078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cf9f57550 0x560cf9f60a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.266+0000 7f374cffcb80 1 -- 172.21.15.133:0/3773094078 >> 172.21.15.133:0/3773094078 conn(0x560cf9f4c360 msgr2=0x560cf9f50de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:21.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.266+0000 7f3716ff5700 0 client.15510 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:21.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.266+0000 7f374cffcb80 1 -- 172.21.15.133:0/3773094078 shutdown_connections 2023-11-27T15:23:21.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.266+0000 7f374cffcb80 1 -- 172.21.15.133:0/3773094078 wait complete. 2023-11-27T15:23:21.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.267+0000 7f374cffcb80 20 client.15510 trim_cache size 0 max 16384 2023-11-27T15:23:21.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:20 smithi133 ceph-mon[97860]: pgmap v361: 129 pgs: 129 active+clean; 652 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-11-27T15:23:21.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.300+0000 7f374cffcb80 1 Processor -- start 2023-11-27T15:23:21.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.300+0000 7f374cffcb80 1 -- start start 2023-11-27T15:23:21.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.300+0000 7f374cffcb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f56c00 0x560cf9f5b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.300+0000 7f374cffcb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cf9f57550 0x560cf9f67a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.300+0000 7f374cffcb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cf9f58110 0x560cf9f5b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.300+0000 7f374cffcb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf9f57f90 con 0x560cf9f56c00 2023-11-27T15:23:21.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.300+0000 7f374cffcb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf9f5f9b0 con 0x560cf9f57550 2023-11-27T15:23:21.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.300+0000 7f374cffcb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf9f6bd30 con 0x560cf9f58110 2023-11-27T15:23:21.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.301+0000 7f3732ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f56c00 0x560cf9f5b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:21.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.301+0000 7f3732ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f56c00 0x560cf9f5b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51268/0 (socket says 172.21.15.133:51268) 2023-11-27T15:23:21.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.301+0000 7f3732ffd700 1 -- 172.21.15.133:0/1934640793 learned_addr learned my addr 172.21.15.133:0/1934640793 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:21.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.301+0000 7f37337fe700 1 --2- 172.21.15.133:0/1934640793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cf9f58110 0x560cf9f5b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:21.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.301+0000 7f3732ffd700 1 -- 172.21.15.133:0/1934640793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cf9f58110 msgr2=0x560cf9f5b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:21.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.301+0000 7f3732ffd700 1 --2- 172.21.15.133:0/1934640793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cf9f58110 0x560cf9f5b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.301+0000 7f3732ffd700 1 -- 172.21.15.133:0/1934640793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cf9f57550 msgr2=0x560cf9f67a40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-27T15:23:21.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.301+0000 7f3732ffd700 1 --2- 172.21.15.133:0/1934640793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cf9f57550 0x560cf9f67a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.301+0000 7f3732ffd700 1 -- 172.21.15.133:0/1934640793 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cf9e6dee0 con 0x560cf9f56c00 2023-11-27T15:23:21.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.301+0000 7f3732ffd700 1 --2- 172.21.15.133:0/1934640793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f56c00 0x560cf9f5b010 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3728004380 tx=0x7f3728012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b959ad8c77d94bd server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:21.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.302+0000 7f3719ffb700 1 -- 172.21.15.133:0/1934640793 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3728024070 con 0x560cf9f56c00 2023-11-27T15:23:21.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.302+0000 7f3719ffb700 1 -- 172.21.15.133:0/1934640793 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37280243d0 con 0x560cf9f56c00 2023-11-27T15:23:21.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.302+0000 7f3719ffb700 1 -- 172.21.15.133:0/1934640793 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3728024c90 con 0x560cf9f56c00 2023-11-27T15:23:21.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.303+0000 7f374cffcb80 1 -- 172.21.15.133:0/1934640793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f56c00 msgr2=0x560cf9f5b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:21.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.303+0000 7f374cffcb80 1 --2- 172.21.15.133:0/1934640793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f56c00 0x560cf9f5b010 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3728004380 tx=0x7f3728012000 comp rx=0 tx=0).stop 2023-11-27T15:23:21.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.303+0000 7f374cffcb80 1 -- 172.21.15.133:0/1934640793 shutdown_connections 2023-11-27T15:23:21.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.303+0000 7f374cffcb80 1 --2- 172.21.15.133:0/1934640793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cf9f58110 0x560cf9f5b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.303+0000 7f374cffcb80 1 --2- 172.21.15.133:0/1934640793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f56c00 0x560cf9f5b010 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.303+0000 7f374cffcb80 1 --2- 172.21.15.133:0/1934640793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cf9f57550 0x560cf9f67a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.303+0000 7f374cffcb80 1 -- 172.21.15.133:0/1934640793 >> 172.21.15.133:0/1934640793 conn(0x560cf9f4c360 msgr2=0x560cf9f68370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:21.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.303+0000 7f374cffcb80 1 -- 172.21.15.133:0/1934640793 shutdown_connections 2023-11-27T15:23:21.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.303+0000 7f374cffcb80 1 -- 172.21.15.133:0/1934640793 wait complete. 2023-11-27T15:23:21.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.304+0000 7f374cffcb80 1 -- start start 2023-11-27T15:23:21.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.304+0000 7f374cffcb80 1 Processor -- start 2023-11-27T15:23:21.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.305+0000 7f374cffcb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cf9f56c00 0x560cf9f63660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.305+0000 7f374cffcb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cf9f57550 0x560cf9f63ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.305+0000 7f374cffcb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f58110 0x560cf9f66bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.305+0000 7f374cffcb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf9f67100 con 0x560cf9f58110 2023-11-27T15:23:21.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.305+0000 7f374cffcb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf9f67270 con 0x560cf9f57550 2023-11-27T15:23:21.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.305+0000 7f374cffcb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cf9f67430 con 0x560cf9f56c00 2023-11-27T15:23:21.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.305+0000 7f374cffcb80 10 client.0 _ll_register_callbacks cb 0x7f372800a980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:21.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.305+0000 7f37337fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f58110 0x560cf9f66bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:21.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.305+0000 7f37337fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f58110 0x560cf9f66bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51278/0 (socket says 172.21.15.133:51278) 2023-11-27T15:23:21.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.305+0000 7f37337fe700 1 -- 172.21.15.133:0/2561491983 learned_addr learned my addr 172.21.15.133:0/2561491983 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:21.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.306+0000 7f3732ffd700 1 --2- 172.21.15.133:0/2561491983 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cf9f56c00 0x560cf9f63660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:21.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.306+0000 7f37337fe700 1 -- 172.21.15.133:0/2561491983 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cf9f56c00 msgr2=0x560cf9f63660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:21.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.306+0000 7f37337fe700 1 --2- 172.21.15.133:0/2561491983 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cf9f56c00 0x560cf9f63660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.306+0000 7f37337fe700 1 -- 172.21.15.133:0/2561491983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cf9f57550 msgr2=0x560cf9f63ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-27T15:23:21.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.306+0000 7f37337fe700 1 --2- 172.21.15.133:0/2561491983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cf9f57550 0x560cf9f63ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.306+0000 7f37337fe700 1 -- 172.21.15.133:0/2561491983 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3728004030 con 0x560cf9f58110 2023-11-27T15:23:21.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.306+0000 7f37337fe700 1 --2- 172.21.15.133:0/2561491983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f58110 0x560cf9f66bc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f372c028bd0 tx=0x7f372c009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d51bdd48ddc31e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:21.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.306+0000 7f3716ff5700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:23:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.307+0000 7f3716ff5700 1 -- 172.21.15.133:0/2561491983 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372c028470 con 0x560cf9f58110 2023-11-27T15:23:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.307+0000 7f3716ff5700 1 -- 172.21.15.133:0/2561491983 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f372c031080 con 0x560cf9f58110 2023-11-27T15:23:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.307+0000 7f374cffcb80 1 -- 172.21.15.133:0/2561491983 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f372c00f040 con 0x560cf9f58110 2023-11-27T15:23:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.307+0000 7f3716ff5700 1 -- 172.21.15.133:0/2561491983 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f372c0285d0 con 0x560cf9f58110 2023-11-27T15:23:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.307+0000 7f374cffcb80 10 client.15516 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.307+0000 7f374cffcb80 1 -- 172.21.15.133:0/2561491983 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560cf9cbec60 con 0x560cf9f58110 2023-11-27T15:23:21.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.308+0000 7f3716ff5700 1 -- 172.21.15.133:0/2561491983 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f372c00f040 con 0x560cf9f58110 2023-11-27T15:23:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.308+0000 7f3716ff5700 1 -- 172.21.15.133:0/2561491983 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f372c007bc0 con 0x560cf9f58110 2023-11-27T15:23:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.308+0000 7f374cffcb80 10 client.15516 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.308+0000 7f374cffcb80 10 client.15516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.308+0000 7f374cffcb80 20 client.15516 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.308+0000 7f374cffcb80 1 --2- 172.21.15.133:0/2561491983 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x560cf9fc0b90 0x560cf9fc2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.308+0000 7f374cffcb80 4 client.15516 mds_command: new command op to 24560 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:23:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.308+0000 7f374cffcb80 1 -- 172.21.15.133:0/2561491983 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560cf9e76660 con 0x560cf9fc0b90 2023-11-27T15:23:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.308+0000 7f3732ffd700 1 --2- 172.21.15.133:0/2561491983 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x560cf9fc0b90 0x560cf9fc2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.309+0000 7f3732ffd700 1 --2- 172.21.15.133:0/2561491983 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x560cf9fc0b90 0x560cf9fc2fa0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f33a4b31a357dc6f server_cookie=82f2b9070eda62e3 in_seq=0 out_seq=0 2023-11-27T15:23:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.309+0000 7f3716ff5700 10 client.15516 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.309+0000 7f3716ff5700 1 -- 172.21.15.133:0/2561491983 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560cf9e76660 con 0x560cf9fc0b90 2023-11-27T15:23:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.309+0000 7f3716ff5700 10 client.15516 handle_command_reply: tid=0 2023-11-27T15:23:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.309+0000 7f374cffcb80 1 client.15516 shutdown 2023-11-27T15:23:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.310+0000 7f374cffcb80 1 -- 172.21.15.133:0/2561491983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f58110 msgr2=0x560cf9f66bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:21.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.310+0000 7f374cffcb80 1 --2- 172.21.15.133:0/2561491983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f58110 0x560cf9f66bc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f372c028bd0 tx=0x7f372c009ae0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.310+0000 7f374cffcb80 1 -- 172.21.15.133:0/2561491983 shutdown_connections 2023-11-27T15:23:21.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.310+0000 7f374cffcb80 1 --2- 172.21.15.133:0/2561491983 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x560cf9fc0b90 0x560cf9fc2fa0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.310+0000 7f374cffcb80 1 --2- 172.21.15.133:0/2561491983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560cf9f58110 0x560cf9f66bc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.310+0000 7f374cffcb80 1 --2- 172.21.15.133:0/2561491983 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560cf9f56c00 0x560cf9f63660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.310+0000 7f374cffcb80 1 --2- 172.21.15.133:0/2561491983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cf9f57550 0x560cf9f63ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.310+0000 7f374cffcb80 1 -- 172.21.15.133:0/2561491983 >> 172.21.15.133:0/2561491983 conn(0x560cf9f4c360 msgr2=0x560cf9f5b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:21.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.310+0000 7f3716ff5700 0 client.15516 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:21.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.310+0000 7f374cffcb80 1 -- 172.21.15.133:0/2561491983 shutdown_connections 2023-11-27T15:23:21.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.310+0000 7f374cffcb80 1 -- 172.21.15.133:0/2561491983 wait complete. 2023-11-27T15:23:21.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.311+0000 7f374cffcb80 20 client.15516 trim_cache size 0 max 16384 2023-11-27T15:23:21.316 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:23:21.316 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2023-11-27T15:23:21.316 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2023-11-27T15:23:21.316 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:23:21.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.311+0000 7f37395ff700 1 -- 172.21.15.133:0/1942158653 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f371c0a11d0 msgr2=0x7f371c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:21.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.311+0000 7f37395ff700 1 --2- 172.21.15.133:0/1942158653 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f371c0a11d0 0x7f371c0a3680 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f372800ac50 tx=0x7f372800a380 comp rx=0 tx=0).stop 2023-11-27T15:23:21.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.311+0000 7f37395ff700 1 -- 172.21.15.133:0/1942158653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37341084e0 msgr2=0x7f373407e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:21.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.311+0000 7f37395ff700 1 --2- 172.21.15.133:0/1942158653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37341084e0 0x7f373407e930 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f372400b680 tx=0x7f3724005740 comp rx=0 tx=0).stop 2023-11-27T15:23:21.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.312+0000 7f37395ff700 1 -- 172.21.15.133:0/1942158653 shutdown_connections 2023-11-27T15:23:21.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.312+0000 7f37395ff700 1 --2- 172.21.15.133:0/1942158653 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f371c0a11d0 0x7f371c0a3680 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.312+0000 7f37395ff700 1 --2- 172.21.15.133:0/1942158653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37341aa840 0x7f37341acc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.312+0000 7f37395ff700 1 --2- 172.21.15.133:0/1942158653 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3734106cc0 0x7f373407e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.312+0000 7f37395ff700 1 --2- 172.21.15.133:0/1942158653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37341084e0 0x7f373407e930 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.312+0000 7f37395ff700 1 -- 172.21.15.133:0/1942158653 >> 172.21.15.133:0/1942158653 conn(0x7f37340759e0 msgr2=0x7f373410b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:21.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.312+0000 7f37395ff700 1 -- 172.21.15.133:0/1942158653 shutdown_connections 2023-11-27T15:23:21.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.312+0000 7f37395ff700 1 -- 172.21.15.133:0/1942158653 wait complete. 2023-11-27T15:23:21.343 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:23:21.344 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:23:21.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.672+0000 7fa35d79d700 1 -- 172.21.15.133:0/1861999935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa358100bc0 msgr2=0x7fa358102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:21.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.672+0000 7fa35d79d700 1 --2- 172.21.15.133:0/1861999935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa358100bc0 0x7fa358102fa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa3480098f0 tx=0x7fa348009c00 comp rx=0 tx=0).stop 2023-11-27T15:23:21.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.672+0000 7fa35d79d700 1 -- 172.21.15.133:0/1861999935 shutdown_connections 2023-11-27T15:23:21.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.672+0000 7fa35d79d700 1 --2- 172.21.15.133:0/1861999935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa358105e00 0x7fa3581081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.672+0000 7fa35d79d700 1 --2- 172.21.15.133:0/1861999935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa358100bc0 0x7fa358102fa0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.672+0000 7fa35d79d700 1 --2- 172.21.15.133:0/1861999935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3581034e0 0x7fa3581058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.672+0000 7fa35d79d700 1 -- 172.21.15.133:0/1861999935 >> 172.21.15.133:0/1861999935 conn(0x7fa3580fa620 msgr2=0x7fa3580fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:21.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.672+0000 7fa35d79d700 1 -- 172.21.15.133:0/1861999935 shutdown_connections 2023-11-27T15:23:21.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.672+0000 7fa35d79d700 1 -- 172.21.15.133:0/1861999935 wait complete. 2023-11-27T15:23:21.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.674+0000 7fa35d79d700 1 Processor -- start 2023-11-27T15:23:21.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.674+0000 7fa35d79d700 1 -- start start 2023-11-27T15:23:21.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.674+0000 7fa35d79d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa358100bc0 0x7fa358198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.674+0000 7fa35d79d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3581034e0 0x7fa358198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.674+0000 7fa35d79d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa358105e00 0x7fa35819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.675+0000 7fa357fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa358100bc0 0x7fa358198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:21.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.675+0000 7fa357fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa358100bc0 0x7fa358198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:36342/0 (socket says 172.21.15.133:36342) 2023-11-27T15:23:21.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.675+0000 7fa35d79d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35819fba0 con 0x7fa3581034e0 2023-11-27T15:23:21.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.675+0000 7fa35d79d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35819fce0 con 0x7fa358105e00 2023-11-27T15:23:21.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.675+0000 7fa35d79d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35819fe70 con 0x7fa358100bc0 2023-11-27T15:23:21.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.675+0000 7fa357fff700 1 -- 172.21.15.133:0/2795154589 learned_addr learned my addr 172.21.15.133:0/2795154589 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:21.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.675+0000 7fa3577fe700 1 --2- 172.21.15.133:0/2795154589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3581034e0 0x7fa358198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:21.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.675+0000 7fa35cf9c700 1 --2- 172.21.15.133:0/2795154589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa358105e00 0x7fa35819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:21.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.675+0000 7fa357fff700 1 -- 172.21.15.133:0/2795154589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa358105e00 msgr2=0x7fa35819f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:21.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.676+0000 7fa357fff700 1 --2- 172.21.15.133:0/2795154589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa358105e00 0x7fa35819f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.676+0000 7fa357fff700 1 -- 172.21.15.133:0/2795154589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3581034e0 msgr2=0x7fa358198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:21.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.676+0000 7fa357fff700 1 --2- 172.21.15.133:0/2795154589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3581034e0 0x7fa358198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:21.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.676+0000 7fa357fff700 1 -- 172.21.15.133:0/2795154589 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa348009520 con 0x7fa358100bc0 2023-11-27T15:23:21.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.676+0000 7fa35cf9c700 1 --2- 172.21.15.133:0/2795154589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa358105e00 0x7fa35819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:23:21.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.676+0000 7fa3577fe700 1 --2- 172.21.15.133:0/2795154589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3581034e0 0x7fa358198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:21.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.676+0000 7fa357fff700 1 --2- 172.21.15.133:0/2795154589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa358100bc0 0x7fa358198020 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa348000c00 tx=0x7fa34800b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:21.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.676+0000 7fa3557fa700 1 -- 172.21.15.133:0/2795154589 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34801c070 con 0x7fa358100bc0 2023-11-27T15:23:21.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.676+0000 7fa3557fa700 1 -- 172.21.15.133:0/2795154589 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa348005040 con 0x7fa358100bc0 2023-11-27T15:23:21.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.676+0000 7fa35d79d700 1 -- 172.21.15.133:0/2795154589 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35810ba60 con 0x7fa358100bc0 2023-11-27T15:23:21.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.676+0000 7fa3557fa700 1 -- 172.21.15.133:0/2795154589 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa348003ab0 con 0x7fa358100bc0 2023-11-27T15:23:21.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.676+0000 7fa35d79d700 1 -- 172.21.15.133:0/2795154589 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35810b8d0 con 0x7fa358100bc0 2023-11-27T15:23:21.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.678+0000 7fa35d79d700 1 -- 172.21.15.133:0/2795154589 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa358107970 con 0x7fa358100bc0 2023-11-27T15:23:21.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.682+0000 7fa3557fa700 1 -- 172.21.15.133:0/2795154589 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa348020030 con 0x7fa358100bc0 2023-11-27T15:23:21.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.684+0000 7fa3557fa700 1 --2- 172.21.15.133:0/2795154589 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa3400a11d0 0x7fa3400a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:21.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.684+0000 7fa3557fa700 1 -- 172.21.15.133:0/2795154589 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa3480cde60 con 0x7fa358100bc0 2023-11-27T15:23:21.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.684+0000 7fa3577fe700 1 --2- 172.21.15.133:0/2795154589 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa3400a11d0 0x7fa3400a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:21.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.684+0000 7fa3557fa700 1 -- 172.21.15.133:0/2795154589 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa34801f280 con 0x7fa358100bc0 2023-11-27T15:23:21.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:21.685+0000 7fa3577fe700 1 --2- 172.21.15.133:0/2795154589 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa3400a11d0 0x7fa3400a3680 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa34c005fd0 tx=0x7fa34c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:22.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.024+0000 7fa35d79d700 1 -- 172.21.15.133:0/2795154589 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa358067980 con 0x7fa358100bc0 2023-11-27T15:23:22.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.025+0000 7fa3557fa700 1 -- 172.21.15.133:0/2795154589 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10829 (secure 0 0 0) 0x7fa3480051b0 con 0x7fa358100bc0 2023-11-27T15:23:22.029 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:23:22.029 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14493,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2381290259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2381290259},{"type":"v1","addr":"172.21.15.133:6839","nonce":2381290259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1704875895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1704875895},{"type":"v1","addr":"172.21.15.133:6841","nonce":1704875895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:20:33.201023+0000","modified":"2023-11-27T15:21:04.794526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24521,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3492215689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3492215689},{"type":"v1","addr":"172.21.15.177:6837","nonce":3492215689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6839/2781994447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2781994447},{"type":"v1","addr":"172.21.15.174:6839","nonce":2781994447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:23:22.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.028+0000 7fa35d79d700 1 -- 172.21.15.133:0/2795154589 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa3400a11d0 msgr2=0x7fa3400a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.028+0000 7fa35d79d700 1 --2- 172.21.15.133:0/2795154589 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa3400a11d0 0x7fa3400a3680 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa34c005fd0 tx=0x7fa34c009310 comp rx=0 tx=0).stop 2023-11-27T15:23:22.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.028+0000 7fa35d79d700 1 -- 172.21.15.133:0/2795154589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa358100bc0 msgr2=0x7fa358198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.028+0000 7fa35d79d700 1 --2- 172.21.15.133:0/2795154589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa358100bc0 0x7fa358198020 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa348000c00 tx=0x7fa34800b610 comp rx=0 tx=0).stop 2023-11-27T15:23:22.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.028+0000 7fa35d79d700 1 -- 172.21.15.133:0/2795154589 shutdown_connections 2023-11-27T15:23:22.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.028+0000 7fa35d79d700 1 --2- 172.21.15.133:0/2795154589 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa3400a11d0 0x7fa3400a3680 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.028+0000 7fa35d79d700 1 --2- 172.21.15.133:0/2795154589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa358105e00 0x7fa35819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.028+0000 7fa35d79d700 1 --2- 172.21.15.133:0/2795154589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa358100bc0 0x7fa358198020 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.028+0000 7fa35d79d700 1 --2- 172.21.15.133:0/2795154589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3581034e0 0x7fa358198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.028+0000 7fa35d79d700 1 -- 172.21.15.133:0/2795154589 >> 172.21.15.133:0/2795154589 conn(0x7fa3580fa620 msgr2=0x7fa3580ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:22.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.028+0000 7fa35d79d700 1 -- 172.21.15.133:0/2795154589 shutdown_connections 2023-11-27T15:23:22.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.028+0000 7fa35d79d700 1 -- 172.21.15.133:0/2795154589 wait complete. 2023-11-27T15:23:22.033 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 23 2023-11-27T15:23:22.052 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:23:22.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.259+0000 7f4187dff700 1 -- 172.21.15.133:0/4155042841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41801014b0 msgr2=0x7f4180101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.259+0000 7f4187dff700 1 --2- 172.21.15.133:0/4155042841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41801014b0 0x7f4180101900 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4170009990 tx=0x7f4170009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.259+0000 7f4187dff700 1 -- 172.21.15.133:0/4155042841 shutdown_connections 2023-11-27T15:23:22.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.259+0000 7f4187dff700 1 --2- 172.21.15.133:0/4155042841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4180101e40 0x7f4180108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.259+0000 7f4187dff700 1 --2- 172.21.15.133:0/4155042841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41801002c0 0x7f41801006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.259+0000 7f4187dff700 1 --2- 172.21.15.133:0/4155042841 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41801014b0 0x7f4180101900 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.259+0000 7f4187dff700 1 -- 172.21.15.133:0/4155042841 >> 172.21.15.133:0/4155042841 conn(0x7f41800fb8a0 msgr2=0x7f41800fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:22.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.260+0000 7f4187dff700 1 -- 172.21.15.133:0/4155042841 shutdown_connections 2023-11-27T15:23:22.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.260+0000 7f4187dff700 1 -- 172.21.15.133:0/4155042841 wait complete. 2023-11-27T15:23:22.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.261+0000 7f4187dff700 1 Processor -- start 2023-11-27T15:23:22.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.261+0000 7f4187dff700 1 -- start start 2023-11-27T15:23:22.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.262+0000 7f4187dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41801002c0 0x7f4180198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.262+0000 7f4187dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41801014b0 0x7f4180198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.262+0000 7f4187dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4180101e40 0x7f418019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.262+0000 7f4187dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418019fdc0 con 0x7f41801002c0 2023-11-27T15:23:22.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.262+0000 7f4187dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418019ff00 con 0x7f4180101e40 2023-11-27T15:23:22.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.262+0000 7f4187dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41801a0090 con 0x7f41801014b0 2023-11-27T15:23:22.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.262+0000 7f4185b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41801002c0 0x7f4180198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.262+0000 7f418639c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4180101e40 0x7f418019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.262+0000 7f418639c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4180101e40 0x7f418019f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:52258/0 (socket says 172.21.15.133:52258) 2023-11-27T15:23:22.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.262+0000 7f4185b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41801002c0 0x7f4180198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51322/0 (socket says 172.21.15.133:51322) 2023-11-27T15:23:22.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.262+0000 7f418639c700 1 -- 172.21.15.133:0/684475975 learned_addr learned my addr 172.21.15.133:0/684475975 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:22.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.263+0000 7f418539a700 1 --2- 172.21.15.133:0/684475975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41801014b0 0x7f4180198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.263+0000 7f418639c700 1 -- 172.21.15.133:0/684475975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41801014b0 msgr2=0x7f4180198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.263+0000 7f418639c700 1 --2- 172.21.15.133:0/684475975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41801014b0 0x7f4180198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.263+0000 7f418639c700 1 -- 172.21.15.133:0/684475975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41801002c0 msgr2=0x7f4180198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.263+0000 7f418639c700 1 --2- 172.21.15.133:0/684475975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41801002c0 0x7f4180198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.263+0000 7f418639c700 1 -- 172.21.15.133:0/684475975 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41700095f0 con 0x7f4180101e40 2023-11-27T15:23:22.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.263+0000 7f418639c700 1 --2- 172.21.15.133:0/684475975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4180101e40 0x7f418019f7f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f417800ef30 tx=0x7f417800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:22.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.264+0000 7f4176ffd700 1 -- 172.21.15.133:0/684475975 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4178018070 con 0x7f4180101e40 2023-11-27T15:23:22.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.264+0000 7f4176ffd700 1 -- 172.21.15.133:0/684475975 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f417800f040 con 0x7f4180101e40 2023-11-27T15:23:22.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.264+0000 7f4176ffd700 1 -- 172.21.15.133:0/684475975 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417800cd00 con 0x7f4180101e40 2023-11-27T15:23:22.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.264+0000 7f4187dff700 1 -- 172.21.15.133:0/684475975 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f418010bc70 con 0x7f4180101e40 2023-11-27T15:23:22.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.264+0000 7f4187dff700 1 -- 172.21.15.133:0/684475975 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418010bae0 con 0x7f4180101e40 2023-11-27T15:23:22.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.268+0000 7f4176ffd700 1 -- 172.21.15.133:0/684475975 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4178003a00 con 0x7f4180101e40 2023-11-27T15:23:22.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.269+0000 7f4176ffd700 1 --2- 172.21.15.133:0/684475975 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f416c0a1150 0x7f416c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.269+0000 7f4176ffd700 1 -- 172.21.15.133:0/684475975 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f41780cc7a0 con 0x7f4180101e40 2023-11-27T15:23:22.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.270+0000 7f419b6d7b80 1 Processor -- start 2023-11-27T15:23:22.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.270+0000 7f4185b9b700 1 --2- 172.21.15.133:0/684475975 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f416c0a1150 0x7f416c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.270+0000 7f419b6d7b80 1 -- start start 2023-11-27T15:23:22.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.271+0000 7f4185b9b700 1 --2- 172.21.15.133:0/684475975 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f416c0a1150 0x7f416c0a3600 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4180101310 tx=0x7f417c007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:22.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.271+0000 7f419b6d7b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc044c00 0x55aafc045010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.272+0000 7f419b6d7b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc045550 0x55aafc045b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.272+0000 7f419b6d7b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc046110 0x55aafc046580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.272+0000 7f419b6d7b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafbf4a060 con 0x55aafc045550 2023-11-27T15:23:22.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.272+0000 7f419b6d7b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafbf9c8a0 con 0x55aafc044c00 2023-11-27T15:23:22.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.272+0000 7f419b6d7b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafbf95d30 con 0x55aafc046110 2023-11-27T15:23:22.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.272+0000 7f4185b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc044c00 0x55aafc045010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.272+0000 7f4185b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc044c00 0x55aafc045010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:52268/0 (socket says 172.21.15.133:52268) 2023-11-27T15:23:22.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.272+0000 7f4185b9b700 1 -- 172.21.15.133:0/2586498276 learned_addr learned my addr 172.21.15.133:0/2586498276 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:22.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.272+0000 7f418639c700 1 --2- 172.21.15.133:0/2586498276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc046110 0x55aafc046580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.272+0000 7f418539a700 1 --2- 172.21.15.133:0/2586498276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc045550 0x55aafc045b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.272+0000 7f4185b9b700 1 -- 172.21.15.133:0/2586498276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc046110 msgr2=0x55aafc046580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:22.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.272+0000 7f4185b9b700 1 --2- 172.21.15.133:0/2586498276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc046110 0x55aafc046580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.273+0000 7f4185b9b700 1 -- 172.21.15.133:0/2586498276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc045550 msgr2=0x55aafc045b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:22.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.273+0000 7f4185b9b700 1 --2- 172.21.15.133:0/2586498276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc045550 0x55aafc045b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.273+0000 7f4185b9b700 1 -- 172.21.15.133:0/2586498276 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aafbe17630 con 0x55aafc044c00 2023-11-27T15:23:22.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.273+0000 7f418639c700 1 --2- 172.21.15.133:0/2586498276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc046110 0x55aafc046580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:22.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.273+0000 7f418539a700 1 --2- 172.21.15.133:0/2586498276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc045550 0x55aafc045b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:22.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.273+0000 7f4185b9b700 1 --2- 172.21.15.133:0/2586498276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc044c00 0x55aafc045010 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f417c010e50 tx=0x7f417c013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9262937741f728be server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:22.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.273+0000 7f416bfff700 1 -- 172.21.15.133:0/2586498276 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c011db0 con 0x55aafc044c00 2023-11-27T15:23:22.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.273+0000 7f416bfff700 1 -- 172.21.15.133:0/2586498276 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f417c015070 con 0x55aafc044c00 2023-11-27T15:23:22.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.273+0000 7f416bfff700 1 -- 172.21.15.133:0/2586498276 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c003d30 con 0x55aafc044c00 2023-11-27T15:23:22.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.274+0000 7f419b6d7b80 1 -- 172.21.15.133:0/2586498276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc044c00 msgr2=0x55aafc045010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.274+0000 7f419b6d7b80 1 --2- 172.21.15.133:0/2586498276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc044c00 0x55aafc045010 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f417c010e50 tx=0x7f417c013000 comp rx=0 tx=0).stop 2023-11-27T15:23:22.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.275+0000 7f419b6d7b80 1 -- 172.21.15.133:0/2586498276 shutdown_connections 2023-11-27T15:23:22.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.275+0000 7f419b6d7b80 1 --2- 172.21.15.133:0/2586498276 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc046110 0x55aafc046580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.275+0000 7f419b6d7b80 1 --2- 172.21.15.133:0/2586498276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc044c00 0x55aafc045010 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.275+0000 7f419b6d7b80 1 --2- 172.21.15.133:0/2586498276 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc045550 0x55aafc045b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.275+0000 7f419b6d7b80 1 -- 172.21.15.133:0/2586498276 >> 172.21.15.133:0/2586498276 conn(0x55aafc03a360 msgr2=0x55aafc03f670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:22.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.275+0000 7f419b6d7b80 1 -- 172.21.15.133:0/2586498276 shutdown_connections 2023-11-27T15:23:22.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.275+0000 7f419b6d7b80 1 -- 172.21.15.133:0/2586498276 wait complete. 2023-11-27T15:23:22.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.276+0000 7f419b6d7b80 1 -- start start 2023-11-27T15:23:22.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.276+0000 7f419b6d7b80 1 Processor -- start 2023-11-27T15:23:22.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.277+0000 7f419b6d7b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc044c00 0x55aafc04e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.277+0000 7f419b6d7b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc045550 0x55aafc04eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.277+0000 7f419b6d7b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc046110 0x55aafc055b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.277+0000 7f419b6d7b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafbf378e0 con 0x55aafc046110 2023-11-27T15:23:22.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.277+0000 7f419b6d7b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafbf37580 con 0x55aafc044c00 2023-11-27T15:23:22.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.277+0000 7f419b6d7b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafbe19920 con 0x55aafc045550 2023-11-27T15:23:22.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.277+0000 7f419b6d7b80 10 client.0 _ll_register_callbacks cb 0x55aafbf55780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:22.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.277+0000 7f4185b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc044c00 0x55aafc04e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.277+0000 7f4185b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc044c00 0x55aafc04e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:52270/0 (socket says 172.21.15.133:52270) 2023-11-27T15:23:22.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.277+0000 7f4185b9b700 1 -- 172.21.15.133:0/744076733 learned_addr learned my addr 172.21.15.133:0/744076733 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:22.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.277+0000 7f418539a700 1 --2- 172.21.15.133:0/744076733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc045550 0x55aafc04eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.277+0000 7f418639c700 1 --2- 172.21.15.133:0/744076733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc046110 0x55aafc055b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.278+0000 7f4185b9b700 1 -- 172.21.15.133:0/744076733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc045550 msgr2=0x55aafc04eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:22.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.278+0000 7f4185b9b700 1 --2- 172.21.15.133:0/744076733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc045550 0x55aafc04eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.278+0000 7f4185b9b700 1 -- 172.21.15.133:0/744076733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc046110 msgr2=0x55aafc055b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:22.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.278+0000 7f4185b9b700 1 --2- 172.21.15.133:0/744076733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc046110 0x55aafc055b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.278+0000 7f4185b9b700 1 -- 172.21.15.133:0/744076733 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417c010b30 con 0x55aafc044c00 2023-11-27T15:23:22.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.278+0000 7f418639c700 1 --2- 172.21.15.133:0/744076733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc046110 0x55aafc055b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:22.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.278+0000 7f418539a700 1 --2- 172.21.15.133:0/744076733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc045550 0x55aafc04eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:22.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.278+0000 7f4185b9b700 1 --2- 172.21.15.133:0/744076733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc044c00 0x55aafc04e5d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f417c007750 tx=0x7f417c011de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17895d2e1863d317 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:22.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.278+0000 7f41697fa700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:23:22.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.278+0000 7f41697fa700 1 -- 172.21.15.133:0/744076733 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c005460 con 0x55aafc044c00 2023-11-27T15:23:22.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.279+0000 7f41697fa700 1 -- 172.21.15.133:0/744076733 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f417c017070 con 0x55aafc044c00 2023-11-27T15:23:22.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.279+0000 7f419b6d7b80 1 -- 172.21.15.133:0/744076733 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aafc037b20 con 0x55aafc044c00 2023-11-27T15:23:22.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.279+0000 7f41697fa700 1 -- 172.21.15.133:0/744076733 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f417c0055c0 con 0x55aafc044c00 2023-11-27T15:23:22.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.279+0000 7f419b6d7b80 10 client.25571 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:22.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.279+0000 7f419b6d7b80 1 -- 172.21.15.133:0/744076733 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55aafbf5c190 con 0x55aafc044c00 2023-11-27T15:23:22.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.280+0000 7f41697fa700 1 -- 172.21.15.133:0/744076733 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aafc037b20 con 0x55aafc044c00 2023-11-27T15:23:22.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.280+0000 7f41697fa700 1 -- 172.21.15.133:0/744076733 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f417c024070 con 0x55aafc044c00 2023-11-27T15:23:22.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.280+0000 7f419b6d7b80 10 client.25571 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:22.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.280+0000 7f419b6d7b80 10 client.25571 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-11-27T15:23:22.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.280+0000 7f419b6d7b80 20 client.25571 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:22.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.280+0000 7f419b6d7b80 1 --2- 172.21.15.133:0/744076733 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55aafc05a2f0 0x55aafc05c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.281+0000 7f419b6d7b80 4 client.25571 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:22.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.281+0000 7f419b6d7b80 1 -- 172.21.15.133:0/744076733 --> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aafbf580a0 con 0x55aafc05a2f0 2023-11-27T15:23:22.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.281+0000 7f418539a700 1 --2- 172.21.15.133:0/744076733 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55aafc05a2f0 0x55aafc05c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.282+0000 7f41697fa700 10 client.25571 ms_handle_connect on v2:172.21.15.177:6838/1463687598 2023-11-27T15:23:22.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.282+0000 7f418539a700 1 --2- 172.21.15.133:0/744076733 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55aafc05a2f0 0x55aafc05c700 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1303a01159f58eb7 server_cookie=9960b6c0d37dcb4 in_seq=0 out_seq=0 2023-11-27T15:23:22.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.282+0000 7f41697fa700 1 -- 172.21.15.133:0/744076733 <== mds.2 v2:172.21.15.177:6838/1463687598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55aafbf580a0 con 0x55aafc05a2f0 2023-11-27T15:23:22.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.282+0000 7f41697fa700 10 client.25571 handle_command_reply: tid=0 2023-11-27T15:23:22.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.283+0000 7f419b6d7b80 1 client.25571 shutdown 2023-11-27T15:23:22.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.283+0000 7f419b6d7b80 1 -- 172.21.15.133:0/744076733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc044c00 msgr2=0x55aafc04e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.283+0000 7f419b6d7b80 1 --2- 172.21.15.133:0/744076733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc044c00 0x55aafc04e5d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f417c007750 tx=0x7f417c011de0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.283+0000 7f419b6d7b80 1 -- 172.21.15.133:0/744076733 shutdown_connections 2023-11-27T15:23:22.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.283+0000 7f419b6d7b80 1 --2- 172.21.15.133:0/744076733 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55aafc05a2f0 0x55aafc05c700 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.283+0000 7f419b6d7b80 1 --2- 172.21.15.133:0/744076733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc046110 0x55aafc055b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.283+0000 7f41697fa700 0 client.25571 ms_handle_reset on v2:172.21.15.177:6838/1463687598 2023-11-27T15:23:22.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.283+0000 7f419b6d7b80 1 --2- 172.21.15.133:0/744076733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc044c00 0x55aafc04e5d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.283+0000 7f419b6d7b80 1 --2- 172.21.15.133:0/744076733 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc045550 0x55aafc04eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.283+0000 7f419b6d7b80 1 -- 172.21.15.133:0/744076733 >> 172.21.15.133:0/744076733 conn(0x55aafc03a360 msgr2=0x55aafc03ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:22.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.284+0000 7f419b6d7b80 1 -- 172.21.15.133:0/744076733 shutdown_connections 2023-11-27T15:23:22.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.284+0000 7f419b6d7b80 1 -- 172.21.15.133:0/744076733 wait complete. 2023-11-27T15:23:22.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.284+0000 7f419b6d7b80 20 client.25571 trim_cache size 0 max 16384 2023-11-27T15:23:22.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.309+0000 7f419b6d7b80 1 Processor -- start 2023-11-27T15:23:22.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.310+0000 7f419b6d7b80 1 -- start start 2023-11-27T15:23:22.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.310+0000 7f419b6d7b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc044c00 0x55aafc0490a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.310+0000 7f419b6d7b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc045550 0x55aafc0495e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.310+0000 7f419b6d7b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc046110 0x55aafc055b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.310+0000 7f419b6d7b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafc045f90 con 0x55aafc046110 2023-11-27T15:23:22.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.310+0000 7f419b6d7b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafc04da80 con 0x55aafc045550 2023-11-27T15:23:22.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.310+0000 7f419b6d7b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafc040550 con 0x55aafc044c00 2023-11-27T15:23:22.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.310+0000 7f4185b9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc044c00 0x55aafc0490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.310+0000 7f4185b9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc044c00 0x55aafc0490a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:36384/0 (socket says 172.21.15.133:36384) 2023-11-27T15:23:22.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.310+0000 7f4185b9b700 1 -- 172.21.15.133:0/3001211000 learned_addr learned my addr 172.21.15.133:0/3001211000 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:22.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.311+0000 7f418639c700 1 --2- 172.21.15.133:0/3001211000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc046110 0x55aafc055b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.311+0000 7f418539a700 1 --2- 172.21.15.133:0/3001211000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc045550 0x55aafc0495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.311+0000 7f4185b9b700 1 -- 172.21.15.133:0/3001211000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc045550 msgr2=0x55aafc0495e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:22.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.311+0000 7f4185b9b700 1 --2- 172.21.15.133:0/3001211000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc045550 0x55aafc0495e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.311+0000 7f4185b9b700 1 -- 172.21.15.133:0/3001211000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc046110 msgr2=0x55aafc055b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:22.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.311+0000 7f4185b9b700 1 --2- 172.21.15.133:0/3001211000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc046110 0x55aafc055b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.311+0000 7f4185b9b700 1 -- 172.21.15.133:0/3001211000 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aafbf5bee0 con 0x55aafc044c00 2023-11-27T15:23:22.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.311+0000 7f418639c700 1 --2- 172.21.15.133:0/3001211000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc046110 0x55aafc055b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:22.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.311+0000 7f418539a700 1 --2- 172.21.15.133:0/3001211000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc045550 0x55aafc0495e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:22.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.312+0000 7f4185b9b700 1 --2- 172.21.15.133:0/3001211000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc044c00 0x55aafc0490a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f417c007720 tx=0x7f417c003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6e8b45c5391eefd server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:22.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.312+0000 7f4174bff700 1 -- 172.21.15.133:0/3001211000 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c011a00 con 0x55aafc044c00 2023-11-27T15:23:22.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.312+0000 7f4174bff700 1 -- 172.21.15.133:0/3001211000 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f417c026440 con 0x55aafc044c00 2023-11-27T15:23:22.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.312+0000 7f4174bff700 1 -- 172.21.15.133:0/3001211000 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c01e680 con 0x55aafc044c00 2023-11-27T15:23:22.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.313+0000 7f419b6d7b80 1 -- 172.21.15.133:0/3001211000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc044c00 msgr2=0x55aafc0490a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.313+0000 7f419b6d7b80 1 --2- 172.21.15.133:0/3001211000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc044c00 0x55aafc0490a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f417c007720 tx=0x7f417c003680 comp rx=0 tx=0).stop 2023-11-27T15:23:22.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.313+0000 7f419b6d7b80 1 -- 172.21.15.133:0/3001211000 shutdown_connections 2023-11-27T15:23:22.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.313+0000 7f419b6d7b80 1 --2- 172.21.15.133:0/3001211000 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc046110 0x55aafc055b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.313+0000 7f419b6d7b80 1 --2- 172.21.15.133:0/3001211000 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc044c00 0x55aafc0490a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.313+0000 7f419b6d7b80 1 --2- 172.21.15.133:0/3001211000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc045550 0x55aafc0495e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.313+0000 7f419b6d7b80 1 -- 172.21.15.133:0/3001211000 >> 172.21.15.133:0/3001211000 conn(0x55aafc03a360 msgr2=0x55aafc04d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:22.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.313+0000 7f419b6d7b80 1 -- 172.21.15.133:0/3001211000 shutdown_connections 2023-11-27T15:23:22.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.313+0000 7f419b6d7b80 1 -- 172.21.15.133:0/3001211000 wait complete. 2023-11-27T15:23:22.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.313+0000 7f419b6d7b80 1 -- start start 2023-11-27T15:23:22.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.314+0000 7f419b6d7b80 1 Processor -- start 2023-11-27T15:23:22.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.314+0000 7f419b6d7b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc044c00 0x55aafc0530b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.314+0000 7f419b6d7b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc045550 0x55aafc0535f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.314+0000 7f419b6d7b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc046110 0x55aafc054600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.314+0000 7f419b6d7b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafc054b40 con 0x55aafc046110 2023-11-27T15:23:22.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.314+0000 7f419b6d7b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafc054cb0 con 0x55aafc044c00 2023-11-27T15:23:22.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.314+0000 7f419b6d7b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafc054e70 con 0x55aafc045550 2023-11-27T15:23:22.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.314+0000 7f419b6d7b80 10 client.0 _ll_register_callbacks cb 0x55aafbf55780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:22.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.315+0000 7f4185b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc044c00 0x55aafc0530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.315+0000 7f4185b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc044c00 0x55aafc0530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:52290/0 (socket says 172.21.15.133:52290) 2023-11-27T15:23:22.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.315+0000 7f4185b9b700 1 -- 172.21.15.133:0/2831713434 learned_addr learned my addr 172.21.15.133:0/2831713434 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:22.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.315+0000 7f418539a700 1 --2- 172.21.15.133:0/2831713434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc045550 0x55aafc0535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.315+0000 7f4185b9b700 1 -- 172.21.15.133:0/2831713434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc045550 msgr2=0x55aafc0535f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:22.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.315+0000 7f4185b9b700 1 --2- 172.21.15.133:0/2831713434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc045550 0x55aafc0535f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.315+0000 7f4185b9b700 1 -- 172.21.15.133:0/2831713434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc046110 msgr2=0x55aafc054600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:22.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.315+0000 7f418639c700 1 --2- 172.21.15.133:0/2831713434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc046110 0x55aafc054600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.315+0000 7f4185b9b700 1 --2- 172.21.15.133:0/2831713434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc046110 0x55aafc054600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.315+0000 7f4185b9b700 1 -- 172.21.15.133:0/2831713434 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418010bc70 con 0x55aafc044c00 2023-11-27T15:23:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.315+0000 7f418539a700 1 --2- 172.21.15.133:0/2831713434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc045550 0x55aafc0535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.315+0000 7f418639c700 1 --2- 172.21.15.133:0/2831713434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc046110 0x55aafc054600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:23:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.316+0000 7f4185b9b700 1 --2- 172.21.15.133:0/2831713434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc044c00 0x55aafc0530b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f417c005a80 tx=0x7f417c007230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6051f85249c568bf server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.316+0000 7f41697fa700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:23:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.316+0000 7f41697fa700 1 -- 172.21.15.133:0/2831713434 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c005460 con 0x55aafc044c00 2023-11-27T15:23:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.316+0000 7f41697fa700 1 -- 172.21.15.133:0/2831713434 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f417c017070 con 0x55aafc044c00 2023-11-27T15:23:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.316+0000 7f419b6d7b80 1 -- 172.21.15.133:0/2831713434 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f417c015070 con 0x55aafc044c00 2023-11-27T15:23:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.316+0000 7f41697fa700 1 -- 172.21.15.133:0/2831713434 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f417c0055c0 con 0x55aafc044c00 2023-11-27T15:23:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.317+0000 7f419b6d7b80 10 client.25577 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:22.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.317+0000 7f419b6d7b80 1 -- 172.21.15.133:0/2831713434 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55aafbf5c190 con 0x55aafc044c00 2023-11-27T15:23:22.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.317+0000 7f41697fa700 1 -- 172.21.15.133:0/2831713434 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f417c015070 con 0x55aafc044c00 2023-11-27T15:23:22.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.317+0000 7f41697fa700 1 -- 172.21.15.133:0/2831713434 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f417c011720 con 0x55aafc044c00 2023-11-27T15:23:22.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.317+0000 7f419b6d7b80 10 client.25577 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:22.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.317+0000 7f419b6d7b80 10 client.25577 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-11-27T15:23:22.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.317+0000 7f419b6d7b80 20 client.25577 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:22.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.317+0000 7f419b6d7b80 1 --2- 172.21.15.133:0/2831713434 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55aafc05a2f0 0x55aafc0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.317+0000 7f419b6d7b80 4 client.25577 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:23:22.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.317+0000 7f419b6d7b80 1 -- 172.21.15.133:0/2831713434 --> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aafbf64660 con 0x55aafc05a2f0 2023-11-27T15:23:22.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.318+0000 7f418539a700 1 --2- 172.21.15.133:0/2831713434 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55aafc05a2f0 0x55aafc0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.318+0000 7f418539a700 1 --2- 172.21.15.133:0/2831713434 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55aafc05a2f0 0x55aafc0b0530 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c8dac82503ac2836 server_cookie=91aa5efb49dc8829 in_seq=0 out_seq=0 2023-11-27T15:23:22.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.318+0000 7f41697fa700 10 client.25577 ms_handle_connect on v2:172.21.15.177:6838/1463687598 2023-11-27T15:23:22.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.318+0000 7f41697fa700 1 -- 172.21.15.133:0/2831713434 <== mds.2 v2:172.21.15.177:6838/1463687598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aafbf64660 con 0x55aafc05a2f0 2023-11-27T15:23:22.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.318+0000 7f41697fa700 10 client.25577 handle_command_reply: tid=0 2023-11-27T15:23:22.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.318+0000 7f419b6d7b80 1 client.25577 shutdown 2023-11-27T15:23:22.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.319+0000 7f419b6d7b80 1 -- 172.21.15.133:0/2831713434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc044c00 msgr2=0x55aafc0530b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.319+0000 7f419b6d7b80 1 --2- 172.21.15.133:0/2831713434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc044c00 0x55aafc0530b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f417c005a80 tx=0x7f417c007230 comp rx=0 tx=0).stop 2023-11-27T15:23:22.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.319+0000 7f419b6d7b80 1 -- 172.21.15.133:0/2831713434 shutdown_connections 2023-11-27T15:23:22.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.319+0000 7f419b6d7b80 1 --2- 172.21.15.133:0/2831713434 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55aafc05a2f0 0x55aafc0b0530 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.319+0000 7f419b6d7b80 1 --2- 172.21.15.133:0/2831713434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55aafc046110 0x55aafc054600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.319+0000 7f419b6d7b80 1 --2- 172.21.15.133:0/2831713434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aafc044c00 0x55aafc0530b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.319+0000 7f419b6d7b80 1 --2- 172.21.15.133:0/2831713434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55aafc045550 0x55aafc0535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.319+0000 7f419b6d7b80 1 -- 172.21.15.133:0/2831713434 >> 172.21.15.133:0/2831713434 conn(0x55aafc03a360 msgr2=0x55aafc0490a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:22.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.319+0000 7f41697fa700 0 client.25577 ms_handle_reset on v2:172.21.15.177:6838/1463687598 2023-11-27T15:23:22.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.319+0000 7f419b6d7b80 1 -- 172.21.15.133:0/2831713434 shutdown_connections 2023-11-27T15:23:22.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.319+0000 7f419b6d7b80 1 -- 172.21.15.133:0/2831713434 wait complete. 2023-11-27T15:23:22.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.319+0000 7f419b6d7b80 20 client.25577 trim_cache size 0 max 16384 2023-11-27T15:23:22.326 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2023-11-27T15:23:22.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.320+0000 7f4187dff700 1 -- 172.21.15.133:0/684475975 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f416c0a1150 msgr2=0x7f416c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.320+0000 7f4187dff700 1 --2- 172.21.15.133:0/684475975 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f416c0a1150 0x7f416c0a3600 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4180101310 tx=0x7f417c007400 comp rx=0 tx=0).stop 2023-11-27T15:23:22.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.320+0000 7f4187dff700 1 -- 172.21.15.133:0/684475975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4180101e40 msgr2=0x7f418019f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.320+0000 7f4187dff700 1 --2- 172.21.15.133:0/684475975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4180101e40 0x7f418019f7f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f417800ef30 tx=0x7f417800c3a0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.320+0000 7f4187dff700 1 -- 172.21.15.133:0/684475975 shutdown_connections 2023-11-27T15:23:22.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.320+0000 7f4187dff700 1 --2- 172.21.15.133:0/684475975 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f416c0a1150 0x7f416c0a3600 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.320+0000 7f4187dff700 1 --2- 172.21.15.133:0/684475975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4180101e40 0x7f418019f7f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.320+0000 7f4187dff700 1 --2- 172.21.15.133:0/684475975 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41801002c0 0x7f4180198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.320+0000 7f4187dff700 1 --2- 172.21.15.133:0/684475975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41801014b0 0x7f4180198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.320+0000 7f4187dff700 1 -- 172.21.15.133:0/684475975 >> 172.21.15.133:0/684475975 conn(0x7f41800fb8a0 msgr2=0x7f4180106d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:22.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.320+0000 7f4187dff700 1 -- 172.21.15.133:0/684475975 shutdown_connections 2023-11-27T15:23:22.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.320+0000 7f4187dff700 1 -- 172.21.15.133:0/684475975 wait complete. 2023-11-27T15:23:22.342 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:23:22.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.550+0000 7f76d57ff700 1 -- 172.21.15.133:0/3310359027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76d01014b0 msgr2=0x7f76d0101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.550+0000 7f76d57ff700 1 --2- 172.21.15.133:0/3310359027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76d01014b0 0x7f76d0101900 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009990 tx=0x7f76c4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.554 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.550+0000 7f76d57ff700 1 -- 172.21.15.133:0/3310359027 shutdown_connections 2023-11-27T15:23:22.555 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.550+0000 7f76d57ff700 1 --2- 172.21.15.133:0/3310359027 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d0101e40 0x7f76d0108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.555 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.550+0000 7f76d57ff700 1 --2- 172.21.15.133:0/3310359027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76d01002c0 0x7f76d01006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.555 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.551+0000 7f76d57ff700 1 --2- 172.21.15.133:0/3310359027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76d01014b0 0x7f76d0101900 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.555 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.551+0000 7f76d57ff700 1 -- 172.21.15.133:0/3310359027 >> 172.21.15.133:0/3310359027 conn(0x7f76d00fb8a0 msgr2=0x7f76d00fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:22.555 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.551+0000 7f76d57ff700 1 -- 172.21.15.133:0/3310359027 shutdown_connections 2023-11-27T15:23:22.555 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.551+0000 7f76d57ff700 1 -- 172.21.15.133:0/3310359027 wait complete. 2023-11-27T15:23:22.556 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.552+0000 7f76d57ff700 1 Processor -- start 2023-11-27T15:23:22.556 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.553+0000 7f76d57ff700 1 -- start start 2023-11-27T15:23:22.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.553+0000 7f76d57ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76d01002c0 0x7f76d00748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.553+0000 7f76d57ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d01014b0 0x7f76d0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.553+0000 7f76d57ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76d0101e40 0x7f76d0073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.553+0000 7f76d57ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d0073bc0 con 0x7f76d01014b0 2023-11-27T15:23:22.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.553+0000 7f76d57ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d0073d30 con 0x7f76d01002c0 2023-11-27T15:23:22.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.553+0000 7f76d57ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d01a00b0 con 0x7f76d0101e40 2023-11-27T15:23:22.558 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.553+0000 7f76cf7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76d0101e40 0x7f76d0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.553+0000 7f76cf7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76d0101e40 0x7f76d0073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:36416/0 (socket says 172.21.15.133:36416) 2023-11-27T15:23:22.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.553+0000 7f76cf7fe700 1 -- 172.21.15.133:0/285033621 learned_addr learned my addr 172.21.15.133:0/285033621 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:22.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.554+0000 7f76ce7fc700 1 --2- 172.21.15.133:0/285033621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d01014b0 0x7f76d0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.554+0000 7f76ceffd700 1 --2- 172.21.15.133:0/285033621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76d01002c0 0x7f76d00748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.554+0000 7f76cf7fe700 1 -- 172.21.15.133:0/285033621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76d01002c0 msgr2=0x7f76d00748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.554+0000 7f76cf7fe700 1 --2- 172.21.15.133:0/285033621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76d01002c0 0x7f76d00748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.554+0000 7f76cf7fe700 1 -- 172.21.15.133:0/285033621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d01014b0 msgr2=0x7f76d0072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.554+0000 7f76cf7fe700 1 --2- 172.21.15.133:0/285033621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d01014b0 0x7f76d0072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.554+0000 7f76cf7fe700 1 -- 172.21.15.133:0/285033621 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c40095f0 con 0x7f76d0101e40 2023-11-27T15:23:22.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.554+0000 7f76ce7fc700 1 --2- 172.21.15.133:0/285033621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d01014b0 0x7f76d0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:22.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.555+0000 7f76cf7fe700 1 --2- 172.21.15.133:0/285033621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76d0101e40 0x7f76d0073420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f76c800b970 tx=0x7f76c800bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:22.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.555+0000 7f76b7fff700 1 -- 172.21.15.133:0/285033621 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c800c930 con 0x7f76d0101e40 2023-11-27T15:23:22.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.555+0000 7f76b7fff700 1 -- 172.21.15.133:0/285033621 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76c8014440 con 0x7f76d0101e40 2023-11-27T15:23:22.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.555+0000 7f76b7fff700 1 -- 172.21.15.133:0/285033621 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c80075d0 con 0x7f76d0101e40 2023-11-27T15:23:22.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.555+0000 7f76d57ff700 1 -- 172.21.15.133:0/285033621 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d010bca0 con 0x7f76d0101e40 2023-11-27T15:23:22.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.555+0000 7f76d57ff700 1 -- 172.21.15.133:0/285033621 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d010bb10 con 0x7f76d0101e40 2023-11-27T15:23:22.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.560+0000 7f76b7fff700 1 -- 172.21.15.133:0/285033621 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f76c80145b0 con 0x7f76d0101e40 2023-11-27T15:23:22.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.561+0000 7f76e9029b80 1 Processor -- start 2023-11-27T15:23:22.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.562+0000 7f76e9029b80 1 -- start start 2023-11-27T15:23:22.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.562+0000 7f76b7fff700 1 --2- 172.21.15.133:0/285033621 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f76b80a1250 0x7f76b80a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.562+0000 7f76b7fff700 1 -- 172.21.15.133:0/285033621 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f76c8006e40 con 0x7f76d0101e40 2023-11-27T15:23:22.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.563+0000 7f76ce7fc700 1 --2- 172.21.15.133:0/285033621 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f76b80a1250 0x7f76b80a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.563+0000 7f76e9029b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c8954a0 0x56498c8958b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.563+0000 7f76e9029b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c896330 0x56498c8967a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.563+0000 7f76e9029b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56498c896ce0 0x56498c89d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.563+0000 7f76e9029b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498c792060 con 0x56498c896330 2023-11-27T15:23:22.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.563+0000 7f76e9029b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498c7e48a0 con 0x56498c8954a0 2023-11-27T15:23:22.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.563+0000 7f76e9029b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498c7ddd30 con 0x56498c896ce0 2023-11-27T15:23:22.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.563+0000 7f76ce7fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c896330 0x56498c8967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.564+0000 7f76cf7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56498c896ce0 0x56498c89d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.564+0000 7f76ce7fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c896330 0x56498c8967a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51384/0 (socket says 172.21.15.133:51384) 2023-11-27T15:23:22.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.564+0000 7f76ce7fc700 1 -- 172.21.15.133:0/569405956 learned_addr learned my addr 172.21.15.133:0/569405956 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:22.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.564+0000 7f76ceffd700 1 --2- 172.21.15.133:0/569405956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c8954a0 0x56498c8958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.564+0000 7f76ce7fc700 1 --2- 172.21.15.133:0/285033621 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f76b80a1250 0x7f76b80a3700 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f76c400b2e0 tx=0x7f76c4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:22.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.564+0000 7f76ce7fc700 1 -- 172.21.15.133:0/569405956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56498c896ce0 msgr2=0x56498c89d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:22.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.564+0000 7f76ce7fc700 1 --2- 172.21.15.133:0/569405956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56498c896ce0 0x56498c89d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.564+0000 7f76ce7fc700 1 -- 172.21.15.133:0/569405956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c8954a0 msgr2=0x56498c8958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:22.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.564+0000 7f76ce7fc700 1 --2- 172.21.15.133:0/569405956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c8954a0 0x56498c8958b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.564+0000 7f76ce7fc700 1 -- 172.21.15.133:0/569405956 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56498c65f630 con 0x56498c896330 2023-11-27T15:23:22.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.564+0000 7f76ce7fc700 1 --2- 172.21.15.133:0/569405956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c896330 0x56498c8967a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f76c4004610 tx=0x7f76c4026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc3db53e7dbd1b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:22.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.565+0000 7f76b57fa700 1 -- 172.21.15.133:0/569405956 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c400fe40 con 0x56498c896330 2023-11-27T15:23:22.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.565+0000 7f76b57fa700 1 -- 172.21.15.133:0/569405956 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76c401e040 con 0x56498c896330 2023-11-27T15:23:22.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.565+0000 7f76b57fa700 1 -- 172.21.15.133:0/569405956 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c402f830 con 0x56498c896330 2023-11-27T15:23:22.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.566+0000 7f76e9029b80 1 -- 172.21.15.133:0/569405956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c896330 msgr2=0x56498c8967a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.566+0000 7f76e9029b80 1 --2- 172.21.15.133:0/569405956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c896330 0x56498c8967a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f76c4004610 tx=0x7f76c4026000 comp rx=0 tx=0).stop 2023-11-27T15:23:22.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.566+0000 7f76e9029b80 1 -- 172.21.15.133:0/569405956 shutdown_connections 2023-11-27T15:23:22.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.566+0000 7f76e9029b80 1 --2- 172.21.15.133:0/569405956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56498c896ce0 0x56498c89d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.566+0000 7f76e9029b80 1 --2- 172.21.15.133:0/569405956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c8954a0 0x56498c8958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.566+0000 7f76e9029b80 1 --2- 172.21.15.133:0/569405956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c896330 0x56498c8967a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.566+0000 7f76e9029b80 1 -- 172.21.15.133:0/569405956 >> 172.21.15.133:0/569405956 conn(0x56498c882360 msgr2=0x56498c887670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:22.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.566+0000 7f76e9029b80 1 -- 172.21.15.133:0/569405956 shutdown_connections 2023-11-27T15:23:22.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.566+0000 7f76e9029b80 1 -- 172.21.15.133:0/569405956 wait complete. 2023-11-27T15:23:22.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.567+0000 7f76e9029b80 1 -- start start 2023-11-27T15:23:22.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.567+0000 7f76e9029b80 1 Processor -- start 2023-11-27T15:23:22.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.568+0000 7f76e9029b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c8954a0 0x56498c89edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.568+0000 7f76e9029b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56498c896330 0x56498c89f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.568+0000 7f76e9029b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c896ce0 0x56498c8a6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.568+0000 7f76e9029b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498c77f8e0 con 0x56498c8954a0 2023-11-27T15:23:22.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.568+0000 7f76e9029b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498c77f580 con 0x56498c896ce0 2023-11-27T15:23:22.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.568+0000 7f76e9029b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498c661920 con 0x56498c896330 2023-11-27T15:23:22.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.568+0000 7f76e9029b80 10 client.0 _ll_register_callbacks cb 0x56498c79d780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:22.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.568+0000 7f76cf7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c896ce0 0x56498c8a6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.569+0000 7f76cf7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c896ce0 0x56498c8a6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:52340/0 (socket says 172.21.15.133:52340) 2023-11-27T15:23:22.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.569+0000 7f76cf7fe700 1 -- 172.21.15.133:0/2287337036 learned_addr learned my addr 172.21.15.133:0/2287337036 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:22.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.569+0000 7f76ceffd700 1 --2- 172.21.15.133:0/2287337036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c8954a0 0x56498c89edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.569+0000 7f76cf7fe700 1 -- 172.21.15.133:0/2287337036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56498c896330 msgr2=0x56498c89f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:22.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.569+0000 7f76cf7fe700 1 --2- 172.21.15.133:0/2287337036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56498c896330 0x56498c89f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.569+0000 7f76cf7fe700 1 -- 172.21.15.133:0/2287337036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c8954a0 msgr2=0x56498c89edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:22.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.569+0000 7f76cf7fe700 1 --2- 172.21.15.133:0/2287337036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c8954a0 0x56498c89edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.569+0000 7f76cf7fe700 1 -- 172.21.15.133:0/2287337036 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d0109030 con 0x56498c896ce0 2023-11-27T15:23:22.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.569+0000 7f76ceffd700 1 --2- 172.21.15.133:0/2287337036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c8954a0 0x56498c89edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:23:22.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.569+0000 7f76cf7fe700 1 --2- 172.21.15.133:0/2287337036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c896ce0 0x56498c8a6340 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f76c800bfd0 tx=0x7f76c8014ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5279bebc23ccb18a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:22.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.569+0000 7f76b2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:23:22.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.570+0000 7f76b2ff5700 1 -- 172.21.15.133:0/2287337036 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c8084070 con 0x56498c896ce0 2023-11-27T15:23:22.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.570+0000 7f76e9029b80 1 -- 172.21.15.133:0/2287337036 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56498c87fb20 con 0x56498c896ce0 2023-11-27T15:23:22.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.570+0000 7f76b2ff5700 1 -- 172.21.15.133:0/2287337036 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f76c809d370 con 0x56498c896ce0 2023-11-27T15:23:22.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.570+0000 7f76b2ff5700 1 -- 172.21.15.133:0/2287337036 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76c809d170 con 0x56498c896ce0 2023-11-27T15:23:22.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.570+0000 7f76e9029b80 10 client.25586 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:22.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.570+0000 7f76e9029b80 1 -- 172.21.15.133:0/2287337036 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56498c7a4190 con 0x56498c896ce0 2023-11-27T15:23:22.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.570+0000 7f76b2ff5700 1 -- 172.21.15.133:0/2287337036 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76c80b5030 con 0x56498c896ce0 2023-11-27T15:23:22.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.571+0000 7f76b2ff5700 1 -- 172.21.15.133:0/2287337036 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f76c80b1d20 con 0x56498c896ce0 2023-11-27T15:23:22.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.571+0000 7f76e9029b80 10 client.25586 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:22.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.571+0000 7f76e9029b80 10 client.25586 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:23:22.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.571+0000 7f76e9029b80 20 client.25586 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:22.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.571+0000 7f76e9029b80 1 --2- 172.21.15.133:0/2287337036 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56498c8aabc0 0x56498c8acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.571+0000 7f76e9029b80 4 client.25586 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:22.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.571+0000 7f76e9029b80 1 -- 172.21.15.133:0/2287337036 --> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56498c7a00a0 con 0x56498c8aabc0 2023-11-27T15:23:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.572+0000 7f76ceffd700 1 --2- 172.21.15.133:0/2287337036 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56498c8aabc0 0x56498c8acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.572+0000 7f76ceffd700 1 --2- 172.21.15.133:0/2287337036 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56498c8aabc0 0x56498c8acfd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39fd7741774bbc37 server_cookie=ffc84b256e60526 in_seq=0 out_seq=0 2023-11-27T15:23:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.572+0000 7f76b2ff5700 10 client.25586 ms_handle_connect on v2:172.21.15.174:6836/1955090548 2023-11-27T15:23:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.573+0000 7f76b2ff5700 1 -- 172.21.15.133:0/2287337036 <== mds.1 v2:172.21.15.174:6836/1955090548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56498c7a00a0 con 0x56498c8aabc0 2023-11-27T15:23:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.573+0000 7f76b2ff5700 10 client.25586 handle_command_reply: tid=0 2023-11-27T15:23:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.573+0000 7f76e9029b80 1 client.25586 shutdown 2023-11-27T15:23:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.574+0000 7f76e9029b80 1 -- 172.21.15.133:0/2287337036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c896ce0 msgr2=0x56498c8a6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.574+0000 7f76e9029b80 1 --2- 172.21.15.133:0/2287337036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c896ce0 0x56498c8a6340 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f76c800bfd0 tx=0x7f76c8014ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.574+0000 7f76e9029b80 1 -- 172.21.15.133:0/2287337036 shutdown_connections 2023-11-27T15:23:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.574+0000 7f76e9029b80 1 --2- 172.21.15.133:0/2287337036 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56498c8aabc0 0x56498c8acfd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.574+0000 7f76e9029b80 1 --2- 172.21.15.133:0/2287337036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c896ce0 0x56498c8a6340 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.574+0000 7f76b2ff5700 0 client.25586 ms_handle_reset on v2:172.21.15.174:6836/1955090548 2023-11-27T15:23:22.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.574+0000 7f76e9029b80 1 --2- 172.21.15.133:0/2287337036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c8954a0 0x56498c89edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.574+0000 7f76e9029b80 1 --2- 172.21.15.133:0/2287337036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56498c896330 0x56498c89f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.574+0000 7f76e9029b80 1 -- 172.21.15.133:0/2287337036 >> 172.21.15.133:0/2287337036 conn(0x56498c882360 msgr2=0x56498c886de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:22.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.574+0000 7f76e9029b80 1 -- 172.21.15.133:0/2287337036 shutdown_connections 2023-11-27T15:23:22.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.574+0000 7f76e9029b80 1 -- 172.21.15.133:0/2287337036 wait complete. 2023-11-27T15:23:22.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.574+0000 7f76e9029b80 20 client.25586 trim_cache size 0 max 16384 2023-11-27T15:23:22.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.600+0000 7f76e9029b80 1 Processor -- start 2023-11-27T15:23:22.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.600+0000 7f76e9029b80 1 -- start start 2023-11-27T15:23:22.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.600+0000 7f76e9029b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c8954a0 0x56498c8b1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.601+0000 7f76e9029b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56498c896330 0x56498c8a6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.601+0000 7f76e9029b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c896ce0 0x56498c899bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.601+0000 7f76e9029b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498c8aa9a0 con 0x56498c8954a0 2023-11-27T15:23:22.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.601+0000 7f76e9029b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498c89e960 con 0x56498c896ce0 2023-11-27T15:23:22.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.601+0000 7f76e9029b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498c888ef0 con 0x56498c896330 2023-11-27T15:23:22.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.601+0000 7f76ceffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c8954a0 0x56498c8b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.601+0000 7f76ce7fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56498c896330 0x56498c8a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.601+0000 7f76ceffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c8954a0 0x56498c8b1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51404/0 (socket says 172.21.15.133:51404) 2023-11-27T15:23:22.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.601+0000 7f76ceffd700 1 -- 172.21.15.133:0/1549049717 learned_addr learned my addr 172.21.15.133:0/1549049717 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:22.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.601+0000 7f76cf7fe700 1 --2- 172.21.15.133:0/1549049717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c896ce0 0x56498c899bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.601+0000 7f76ce7fc700 1 -- 172.21.15.133:0/1549049717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c896ce0 msgr2=0x56498c899bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:22.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.601+0000 7f76ce7fc700 1 --2- 172.21.15.133:0/1549049717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c896ce0 0x56498c899bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.601+0000 7f76ce7fc700 1 -- 172.21.15.133:0/1549049717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c8954a0 msgr2=0x56498c8b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:22.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.601+0000 7f76ce7fc700 1 --2- 172.21.15.133:0/1549049717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c8954a0 0x56498c8b1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.606 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.601+0000 7f76ce7fc700 1 -- 172.21.15.133:0/1549049717 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56498c7a3ee0 con 0x56498c896330 2023-11-27T15:23:22.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.602+0000 7f76ce7fc700 1 --2- 172.21.15.133:0/1549049717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56498c896330 0x56498c8a6300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f76c400fbd0 tx=0x7f76c40093a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2d4459150f95bc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:22.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.602+0000 7f76b5ffb700 1 -- 172.21.15.133:0/1549049717 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c403c070 con 0x56498c896330 2023-11-27T15:23:22.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.602+0000 7f76b5ffb700 1 -- 172.21.15.133:0/1549049717 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76c4004c90 con 0x56498c896330 2023-11-27T15:23:22.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.602+0000 7f76b5ffb700 1 -- 172.21.15.133:0/1549049717 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c4005610 con 0x56498c896330 2023-11-27T15:23:22.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.603+0000 7f76e9029b80 1 -- 172.21.15.133:0/1549049717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56498c896330 msgr2=0x56498c8a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.603+0000 7f76e9029b80 1 --2- 172.21.15.133:0/1549049717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56498c896330 0x56498c8a6300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f76c400fbd0 tx=0x7f76c40093a0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.603+0000 7f76e9029b80 1 -- 172.21.15.133:0/1549049717 shutdown_connections 2023-11-27T15:23:22.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.603+0000 7f76e9029b80 1 --2- 172.21.15.133:0/1549049717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c896ce0 0x56498c899bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.603+0000 7f76e9029b80 1 --2- 172.21.15.133:0/1549049717 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c8954a0 0x56498c8b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.603+0000 7f76e9029b80 1 --2- 172.21.15.133:0/1549049717 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56498c896330 0x56498c8a6300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.603+0000 7f76e9029b80 1 -- 172.21.15.133:0/1549049717 >> 172.21.15.133:0/1549049717 conn(0x56498c882360 msgr2=0x56498c883830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:22.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.603+0000 7f76e9029b80 1 -- 172.21.15.133:0/1549049717 shutdown_connections 2023-11-27T15:23:22.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.603+0000 7f76e9029b80 1 -- 172.21.15.133:0/1549049717 wait complete. 2023-11-27T15:23:22.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.604+0000 7f76e9029b80 1 -- start start 2023-11-27T15:23:22.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.604+0000 7f76e9029b80 1 Processor -- start 2023-11-27T15:23:22.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.605+0000 7f76e9029b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c8954a0 0x56498c8a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.605+0000 7f76e9029b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56498c896330 0x56498c8a3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.605+0000 7f76e9029b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c896ce0 0x56498c8a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.605+0000 7f76e9029b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498c8a5450 con 0x56498c8954a0 2023-11-27T15:23:22.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.605+0000 7f76e9029b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498c8a55c0 con 0x56498c896ce0 2023-11-27T15:23:22.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.605+0000 7f76e9029b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56498c8a5780 con 0x56498c896330 2023-11-27T15:23:22.610 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.605+0000 7f76e9029b80 10 client.0 _ll_register_callbacks cb 0x7f76c8098e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:22.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.605+0000 7f76ceffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c8954a0 0x56498c8a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.605+0000 7f76cf7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c896ce0 0x56498c8a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.605+0000 7f76ceffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c8954a0 0x56498c8a39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:51408/0 (socket says 172.21.15.133:51408) 2023-11-27T15:23:22.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.605+0000 7f76cf7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c896ce0 0x56498c8a4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:52368/0 (socket says 172.21.15.133:52368) 2023-11-27T15:23:22.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.605+0000 7f76ceffd700 1 -- 172.21.15.133:0/2455838947 learned_addr learned my addr 172.21.15.133:0/2455838947 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:22.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.605+0000 7f76ce7fc700 1 --2- 172.21.15.133:0/2455838947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56498c896330 0x56498c8a3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.606+0000 7f76cf7fe700 1 -- 172.21.15.133:0/2455838947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56498c896330 msgr2=0x56498c8a3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:22.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.606+0000 7f76cf7fe700 1 --2- 172.21.15.133:0/2455838947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56498c896330 0x56498c8a3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.606+0000 7f76cf7fe700 1 -- 172.21.15.133:0/2455838947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c8954a0 msgr2=0x56498c8a39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:22.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.606+0000 7f76cf7fe700 1 --2- 172.21.15.133:0/2455838947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c8954a0 0x56498c8a39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.606+0000 7f76cf7fe700 1 -- 172.21.15.133:0/2455838947 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56498c65f630 con 0x56498c896ce0 2023-11-27T15:23:22.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.606+0000 7f76ceffd700 1 --2- 172.21.15.133:0/2455838947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c8954a0 0x56498c8a39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:22.611 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.606+0000 7f76cf7fe700 1 --2- 172.21.15.133:0/2455838947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c896ce0 0x56498c8a4f10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f76c8095db0 tx=0x7f76c8014b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e0ddcd36803c8c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:22.612 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.606+0000 7f76b2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:23:22.613 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.607+0000 7f76b2ff5700 1 -- 172.21.15.133:0/2455838947 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c8095680 con 0x56498c896ce0 2023-11-27T15:23:22.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.607+0000 7f76b2ff5700 1 -- 172.21.15.133:0/2455838947 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f76c809d370 con 0x56498c896ce0 2023-11-27T15:23:22.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.607+0000 7f76e9029b80 1 -- 172.21.15.133:0/2455838947 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76c401e040 con 0x56498c896ce0 2023-11-27T15:23:22.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.608+0000 7f76b2ff5700 1 -- 172.21.15.133:0/2455838947 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76c80a1790 con 0x56498c896ce0 2023-11-27T15:23:22.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.608+0000 7f76e9029b80 10 client.25592 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:22.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.608+0000 7f76e9029b80 1 -- 172.21.15.133:0/2455838947 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56498c5f4c60 con 0x56498c896ce0 2023-11-27T15:23:22.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.608+0000 7f76b2ff5700 1 -- 172.21.15.133:0/2455838947 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76c401e040 con 0x56498c896ce0 2023-11-27T15:23:22.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.608+0000 7f76b2ff5700 1 -- 172.21.15.133:0/2455838947 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f76c80995a0 con 0x56498c896ce0 2023-11-27T15:23:22.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.608+0000 7f76e9029b80 10 client.25592 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:22.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.608+0000 7f76e9029b80 10 client.25592 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:23:22.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.608+0000 7f76e9029b80 20 client.25592 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:22.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.608+0000 7f76e9029b80 1 --2- 172.21.15.133:0/2455838947 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56498c8fefd0 0x56498c9013e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.608+0000 7f76e9029b80 4 client.25592 mds_command: new command op to 24521 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:23:22.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.608+0000 7f76e9029b80 1 -- 172.21.15.133:0/2455838947 --> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56498c7ac660 con 0x56498c8fefd0 2023-11-27T15:23:22.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.608+0000 7f76ceffd700 1 --2- 172.21.15.133:0/2455838947 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56498c8fefd0 0x56498c9013e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.609+0000 7f76ceffd700 1 --2- 172.21.15.133:0/2455838947 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56498c8fefd0 0x56498c9013e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fdc28fce69d8139f server_cookie=be3e13d7c21c14c5 in_seq=0 out_seq=0 2023-11-27T15:23:22.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.609+0000 7f76b2ff5700 10 client.25592 ms_handle_connect on v2:172.21.15.174:6836/1955090548 2023-11-27T15:23:22.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.609+0000 7f76b2ff5700 1 -- 172.21.15.133:0/2455838947 <== mds.1 v2:172.21.15.174:6836/1955090548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56498c7ac660 con 0x56498c8fefd0 2023-11-27T15:23:22.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.609+0000 7f76b2ff5700 10 client.25592 handle_command_reply: tid=0 2023-11-27T15:23:22.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.609+0000 7f76e9029b80 1 client.25592 shutdown 2023-11-27T15:23:22.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.610+0000 7f76e9029b80 1 -- 172.21.15.133:0/2455838947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c896ce0 msgr2=0x56498c8a4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.610+0000 7f76e9029b80 1 --2- 172.21.15.133:0/2455838947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c896ce0 0x56498c8a4f10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f76c8095db0 tx=0x7f76c8014b00 comp rx=0 tx=0).stop 2023-11-27T15:23:22.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.610+0000 7f76e9029b80 1 -- 172.21.15.133:0/2455838947 shutdown_connections 2023-11-27T15:23:22.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.610+0000 7f76e9029b80 1 --2- 172.21.15.133:0/2455838947 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56498c8fefd0 0x56498c9013e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.610+0000 7f76b2ff5700 0 client.25592 ms_handle_reset on v2:172.21.15.174:6836/1955090548 2023-11-27T15:23:22.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.610+0000 7f76e9029b80 1 --2- 172.21.15.133:0/2455838947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56498c896ce0 0x56498c8a4f10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.610+0000 7f76e9029b80 1 --2- 172.21.15.133:0/2455838947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56498c8954a0 0x56498c8a39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.610+0000 7f76e9029b80 1 --2- 172.21.15.133:0/2455838947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56498c896330 0x56498c8a3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.610+0000 7f76e9029b80 1 -- 172.21.15.133:0/2455838947 >> 172.21.15.133:0/2455838947 conn(0x56498c882360 msgr2=0x56498c8b1970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:22.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.610+0000 7f76e9029b80 1 -- 172.21.15.133:0/2455838947 shutdown_connections 2023-11-27T15:23:22.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.610+0000 7f76e9029b80 1 -- 172.21.15.133:0/2455838947 wait complete. 2023-11-27T15:23:22.616 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.610+0000 7f76e9029b80 20 client.25592 trim_cache size 0 max 16384 2023-11-27T15:23:22.616 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2023-11-27T15:23:22.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.611+0000 7f76d57ff700 1 -- 172.21.15.133:0/285033621 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f76b80a1250 msgr2=0x7f76b80a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.611+0000 7f76d57ff700 1 --2- 172.21.15.133:0/285033621 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f76b80a1250 0x7f76b80a3700 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f76c400b2e0 tx=0x7f76c4005fb0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.611+0000 7f76d57ff700 1 -- 172.21.15.133:0/285033621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76d0101e40 msgr2=0x7f76d0073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.611+0000 7f76d57ff700 1 --2- 172.21.15.133:0/285033621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76d0101e40 0x7f76d0073420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f76c800b970 tx=0x7f76c800bc80 comp rx=0 tx=0).stop 2023-11-27T15:23:22.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.611+0000 7f76d57ff700 1 -- 172.21.15.133:0/285033621 shutdown_connections 2023-11-27T15:23:22.617 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.611+0000 7f76d57ff700 1 --2- 172.21.15.133:0/285033621 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f76b80a1250 0x7f76b80a3700 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.611+0000 7f76d57ff700 1 --2- 172.21.15.133:0/285033621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76d0101e40 0x7f76d0073420 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.611+0000 7f76d57ff700 1 --2- 172.21.15.133:0/285033621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76d01002c0 0x7f76d00748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.611+0000 7f76d57ff700 1 --2- 172.21.15.133:0/285033621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76d01014b0 0x7f76d0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.611+0000 7f76d57ff700 1 -- 172.21.15.133:0/285033621 >> 172.21.15.133:0/285033621 conn(0x7f76d00fb8a0 msgr2=0x7f76d00fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:22.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.611+0000 7f76d57ff700 1 -- 172.21.15.133:0/285033621 shutdown_connections 2023-11-27T15:23:22.618 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.611+0000 7f76d57ff700 1 -- 172.21.15.133:0/285033621 wait complete. 2023-11-27T15:23:22.643 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:23:22.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.957+0000 7fd970d9d700 1 -- 172.21.15.133:0/444779035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd96c100050 msgr2=0x7fd96c100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.957+0000 7fd970d9d700 1 --2- 172.21.15.133:0/444779035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd96c100050 0x7fd96c100460 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd95c009990 tx=0x7fd95c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.957+0000 7fd970d9d700 1 -- 172.21.15.133:0/444779035 shutdown_connections 2023-11-27T15:23:22.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.957+0000 7fd970d9d700 1 --2- 172.21.15.133:0/444779035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd96c101c90 0x7fd96c108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.957+0000 7fd970d9d700 1 --2- 172.21.15.133:0/444779035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd96c100050 0x7fd96c100460 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.957+0000 7fd970d9d700 1 --2- 172.21.15.133:0/444779035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd96c1012e0 0x7fd96c101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.957+0000 7fd970d9d700 1 -- 172.21.15.133:0/444779035 >> 172.21.15.133:0/444779035 conn(0x7fd96c0fb650 msgr2=0x7fd96c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:22.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.957+0000 7fd970d9d700 1 -- 172.21.15.133:0/444779035 shutdown_connections 2023-11-27T15:23:22.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.957+0000 7fd970d9d700 1 -- 172.21.15.133:0/444779035 wait complete. 2023-11-27T15:23:22.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.959+0000 7fd970d9d700 1 Processor -- start 2023-11-27T15:23:22.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.959+0000 7fd970d9d700 1 -- start start 2023-11-27T15:23:22.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.959+0000 7fd970d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd96c100050 0x7fd96c195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.959+0000 7fd970d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd96c1012e0 0x7fd96c196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.959+0000 7fd970d9d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd96c101c90 0x7fd96c19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.959+0000 7fd970d9d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c19d9c0 con 0x7fd96c1012e0 2023-11-27T15:23:22.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.959+0000 7fd970d9d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c19db00 con 0x7fd96c100050 2023-11-27T15:23:22.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.959+0000 7fd970d9d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c19dc90 con 0x7fd96c101c90 2023-11-27T15:23:22.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.960+0000 7fd96b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd96c100050 0x7fd96c195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.960+0000 7fd96b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd96c100050 0x7fd96c195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:47774/0 (socket says 172.21.15.133:47774) 2023-11-27T15:23:22.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.960+0000 7fd96b7fe700 1 -- 172.21.15.133:0/2676458402 learned_addr learned my addr 172.21.15.133:0/2676458402 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:22.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.960+0000 7fd96bfff700 1 --2- 172.21.15.133:0/2676458402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd96c101c90 0x7fd96c19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.960+0000 7fd96affd700 1 --2- 172.21.15.133:0/2676458402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd96c1012e0 0x7fd96c196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.960+0000 7fd96b7fe700 1 -- 172.21.15.133:0/2676458402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd96c101c90 msgr2=0x7fd96c19d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.960+0000 7fd96b7fe700 1 --2- 172.21.15.133:0/2676458402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd96c101c90 0x7fd96c19d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.960+0000 7fd96b7fe700 1 -- 172.21.15.133:0/2676458402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd96c1012e0 msgr2=0x7fd96c196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.960+0000 7fd96b7fe700 1 --2- 172.21.15.133:0/2676458402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd96c1012e0 0x7fd96c196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.960+0000 7fd96b7fe700 1 -- 172.21.15.133:0/2676458402 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd960009520 con 0x7fd96c100050 2023-11-27T15:23:22.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.960+0000 7fd96affd700 1 --2- 172.21.15.133:0/2676458402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd96c1012e0 0x7fd96c196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:22.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.960+0000 7fd96bfff700 1 --2- 172.21.15.133:0/2676458402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd96c101c90 0x7fd96c19d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:22.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.961+0000 7fd96b7fe700 1 --2- 172.21.15.133:0/2676458402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd96c100050 0x7fd96c195e40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd95c004730 tx=0x7fd95c005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:22.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.961+0000 7fd968ff9700 1 -- 172.21.15.133:0/2676458402 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c01d070 con 0x7fd96c100050 2023-11-27T15:23:22.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.961+0000 7fd968ff9700 1 -- 172.21.15.133:0/2676458402 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd95c004cd0 con 0x7fd96c100050 2023-11-27T15:23:22.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.961+0000 7fd968ff9700 1 -- 172.21.15.133:0/2676458402 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0055f0 con 0x7fd96c100050 2023-11-27T15:23:22.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.961+0000 7fd970d9d700 1 -- 172.21.15.133:0/2676458402 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd95c0095f0 con 0x7fd96c100050 2023-11-27T15:23:22.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.961+0000 7fd970d9d700 1 -- 172.21.15.133:0/2676458402 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96c10ba00 con 0x7fd96c100050 2023-11-27T15:23:22.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.967+0000 7fd968ff9700 1 -- 172.21.15.133:0/2676458402 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd95c00fc70 con 0x7fd96c100050 2023-11-27T15:23:22.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.967+0000 7fd98594bb80 1 Processor -- start 2023-11-27T15:23:22.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.968+0000 7fd98594bb80 1 -- start start 2023-11-27T15:23:22.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.968+0000 7fd98594bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a036f0 0x560308a03b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.968+0000 7fd98594bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a04540 0x560308a049b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.968+0000 7fd98594bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560308a04ef0 0x560308a0b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.968+0000 7fd98594bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560308900060 con 0x560308a04540 2023-11-27T15:23:22.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.968+0000 7fd98594bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603089528a0 con 0x560308a04ef0 2023-11-27T15:23:22.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.968+0000 7fd98594bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56030894bd30 con 0x560308a036f0 2023-11-27T15:23:22.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.969+0000 7fd96b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a04540 0x560308a049b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.969+0000 7fd96bfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a036f0 0x560308a03b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.969+0000 7fd96b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a04540 0x560308a049b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60774/0 (socket says 172.21.15.133:60774) 2023-11-27T15:23:22.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.969+0000 7fd96bfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a036f0 0x560308a03b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:57896/0 (socket says 172.21.15.133:57896) 2023-11-27T15:23:22.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.969+0000 7fd96b7fe700 1 -- 172.21.15.133:0/472650713 learned_addr learned my addr 172.21.15.133:0/472650713 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:22.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.969+0000 7fd96affd700 1 --2- 172.21.15.133:0/472650713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560308a04ef0 0x560308a0b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.969+0000 7fd96b7fe700 1 -- 172.21.15.133:0/472650713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a036f0 msgr2=0x560308a03b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:22.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.969+0000 7fd96b7fe700 1 --2- 172.21.15.133:0/472650713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a036f0 0x560308a03b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.969+0000 7fd96b7fe700 1 -- 172.21.15.133:0/472650713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560308a04ef0 msgr2=0x560308a0b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:22.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.969+0000 7fd96b7fe700 1 --2- 172.21.15.133:0/472650713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560308a04ef0 0x560308a0b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.969+0000 7fd96b7fe700 1 -- 172.21.15.133:0/472650713 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603087cd630 con 0x560308a04540 2023-11-27T15:23:22.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.970+0000 7fd96b7fe700 1 --2- 172.21.15.133:0/472650713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a04540 0x560308a049b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd95c0d6290 tx=0x7fd95c0d1f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79cd3b1f3a95faca server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:22.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.970+0000 7fd968ff9700 1 --2- 172.21.15.133:0/2676458402 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9540a11d0 0x7fd9540a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.970+0000 7fd951ffb700 1 -- 172.21.15.133:0/472650713 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0976a0 con 0x560308a04540 2023-11-27T15:23:22.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.970+0000 7fd951ffb700 1 -- 172.21.15.133:0/472650713 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd95c09b130 con 0x560308a04540 2023-11-27T15:23:22.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.970+0000 7fd951ffb700 1 -- 172.21.15.133:0/472650713 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c003d80 con 0x560308a04540 2023-11-27T15:23:22.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.971+0000 7fd968ff9700 1 -- 172.21.15.133:0/2676458402 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd95c0ce270 con 0x7fd96c100050 2023-11-27T15:23:22.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.971+0000 7fd96bfff700 1 --2- 172.21.15.133:0/2676458402 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9540a11d0 0x7fd9540a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.971+0000 7fd98594bb80 1 -- 172.21.15.133:0/472650713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a04540 msgr2=0x560308a049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.971+0000 7fd98594bb80 1 --2- 172.21.15.133:0/472650713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a04540 0x560308a049b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd95c0d6290 tx=0x7fd95c0d1f90 comp rx=0 tx=0).stop 2023-11-27T15:23:22.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.972+0000 7fd98594bb80 1 -- 172.21.15.133:0/472650713 shutdown_connections 2023-11-27T15:23:22.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.972+0000 7fd98594bb80 1 --2- 172.21.15.133:0/472650713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560308a04ef0 0x560308a0b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.972+0000 7fd98594bb80 1 --2- 172.21.15.133:0/472650713 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a036f0 0x560308a03b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.972+0000 7fd98594bb80 1 --2- 172.21.15.133:0/472650713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a04540 0x560308a049b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.972+0000 7fd98594bb80 1 -- 172.21.15.133:0/472650713 >> 172.21.15.133:0/472650713 conn(0x5603089f05b0 msgr2=0x5603089f5860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:22.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.972+0000 7fd96bfff700 1 --2- 172.21.15.133:0/2676458402 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9540a11d0 0x7fd9540a3680 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd964009e50 tx=0x7fd964009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:22.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.972+0000 7fd98594bb80 1 -- 172.21.15.133:0/472650713 shutdown_connections 2023-11-27T15:23:22.977 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.972+0000 7fd98594bb80 1 -- 172.21.15.133:0/472650713 wait complete. 2023-11-27T15:23:22.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.973+0000 7fd98594bb80 1 -- start start 2023-11-27T15:23:22.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.973+0000 7fd98594bb80 1 Processor -- start 2023-11-27T15:23:22.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.974+0000 7fd98594bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a036f0 0x560308a0cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.974+0000 7fd98594bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a04540 0x560308a0d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.974+0000 7fd98594bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560308a04ef0 0x560308a14170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.974+0000 7fd98594bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603088ed8e0 con 0x560308a04540 2023-11-27T15:23:22.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.974+0000 7fd98594bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603088ed580 con 0x560308a04ef0 2023-11-27T15:23:22.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.974+0000 7fd98594bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603087cf920 con 0x560308a036f0 2023-11-27T15:23:22.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.974+0000 7fd98594bb80 10 client.0 _ll_register_callbacks cb 0x56030890b780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:22.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.974+0000 7fd96b7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a036f0 0x560308a0cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.974+0000 7fd96b7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a036f0 0x560308a0cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:57898/0 (socket says 172.21.15.133:57898) 2023-11-27T15:23:22.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.974+0000 7fd96b7fe700 1 -- 172.21.15.133:0/3229924008 learned_addr learned my addr 172.21.15.133:0/3229924008 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:22.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.975+0000 7fd96b7fe700 1 -- 172.21.15.133:0/3229924008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560308a04ef0 msgr2=0x560308a14170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-27T15:23:22.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.975+0000 7fd96b7fe700 1 --2- 172.21.15.133:0/3229924008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560308a04ef0 0x560308a14170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.975+0000 7fd96b7fe700 1 -- 172.21.15.133:0/3229924008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a04540 msgr2=0x560308a0d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:22.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.975+0000 7fd96b7fe700 1 --2- 172.21.15.133:0/3229924008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a04540 0x560308a0d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.975+0000 7fd96b7fe700 1 -- 172.21.15.133:0/3229924008 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96c10ba00 con 0x560308a036f0 2023-11-27T15:23:22.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.975+0000 7fd96b7fe700 1 --2- 172.21.15.133:0/3229924008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a036f0 0x560308a0cbf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd95c0d6260 tx=0x7fd95c005050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e228914722150da4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:22.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.975+0000 7fd94affd700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:23:22.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.975+0000 7fd94affd700 1 -- 172.21.15.133:0/3229924008 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0d17a0 con 0x560308a036f0 2023-11-27T15:23:22.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.975+0000 7fd94affd700 1 -- 172.21.15.133:0/3229924008 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd95c028540 con 0x560308a036f0 2023-11-27T15:23:22.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.976+0000 7fd98594bb80 1 -- 172.21.15.133:0/3229924008 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5603089edd70 con 0x560308a036f0 2023-11-27T15:23:22.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.976+0000 7fd94affd700 1 -- 172.21.15.133:0/3229924008 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd95c019040 con 0x560308a036f0 2023-11-27T15:23:22.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.976+0000 7fd98594bb80 10 client.25591 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:22.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.976+0000 7fd98594bb80 1 -- 172.21.15.133:0/3229924008 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560308912190 con 0x560308a036f0 2023-11-27T15:23:22.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.976+0000 7fd94affd700 1 -- 172.21.15.133:0/3229924008 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5603089edd70 con 0x560308a036f0 2023-11-27T15:23:22.983 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.976+0000 7fd94affd700 1 -- 172.21.15.133:0/3229924008 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd95c0a10d0 con 0x560308a036f0 2023-11-27T15:23:22.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.977+0000 7fd98594bb80 10 client.25591 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:22.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.977+0000 7fd98594bb80 10 client.25591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:22.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.977+0000 7fd98594bb80 20 client.25591 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:22.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.977+0000 7fd98594bb80 1 --2- 172.21.15.133:0/3229924008 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x560308a18570 0x560308a1a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:22.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.977+0000 7fd98594bb80 4 client.25591 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:22.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.977+0000 7fd98594bb80 1 -- 172.21.15.133:0/3229924008 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56030890e0a0 con 0x560308a18570 2023-11-27T15:23:22.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.977+0000 7fd96affd700 1 --2- 172.21.15.133:0/3229924008 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x560308a18570 0x560308a1a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:22.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.978+0000 7fd96affd700 1 --2- 172.21.15.133:0/3229924008 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x560308a18570 0x560308a1a980 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2441c830a4755802 server_cookie=ed855a8065dfb06b in_seq=0 out_seq=0 2023-11-27T15:23:22.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.978+0000 7fd94affd700 10 client.25591 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:22.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.978+0000 7fd94affd700 1 -- 172.21.15.133:0/3229924008 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56030890e0a0 con 0x560308a18570 2023-11-27T15:23:22.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.979+0000 7fd94affd700 10 client.25591 handle_command_reply: tid=0 2023-11-27T15:23:22.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.979+0000 7fd98594bb80 1 client.25591 shutdown 2023-11-27T15:23:22.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.979+0000 7fd98594bb80 1 -- 172.21.15.133:0/3229924008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a036f0 msgr2=0x560308a0cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:22.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.979+0000 7fd98594bb80 1 --2- 172.21.15.133:0/3229924008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a036f0 0x560308a0cbf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd95c0d6260 tx=0x7fd95c005050 comp rx=0 tx=0).stop 2023-11-27T15:23:22.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.979+0000 7fd98594bb80 1 -- 172.21.15.133:0/3229924008 shutdown_connections 2023-11-27T15:23:22.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.979+0000 7fd98594bb80 1 --2- 172.21.15.133:0/3229924008 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x560308a18570 0x560308a1a980 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.979+0000 7fd98594bb80 1 --2- 172.21.15.133:0/3229924008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560308a04ef0 0x560308a14170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.979+0000 7fd98594bb80 1 --2- 172.21.15.133:0/3229924008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a036f0 0x560308a0cbf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.979+0000 7fd98594bb80 1 --2- 172.21.15.133:0/3229924008 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a04540 0x560308a0d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:22.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.979+0000 7fd98594bb80 1 -- 172.21.15.133:0/3229924008 >> 172.21.15.133:0/3229924008 conn(0x5603089f05b0 msgr2=0x5603089f5030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:22.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.979+0000 7fd94affd700 0 client.25591 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:22.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.979+0000 7fd98594bb80 1 -- 172.21.15.133:0/3229924008 shutdown_connections 2023-11-27T15:23:22.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.979+0000 7fd98594bb80 1 -- 172.21.15.133:0/3229924008 wait complete. 2023-11-27T15:23:22.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:22.980+0000 7fd98594bb80 20 client.25591 trim_cache size 0 max 16384 2023-11-27T15:23:23.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.006+0000 7fd98594bb80 1 Processor -- start 2023-11-27T15:23:23.010 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.007+0000 7fd98594bb80 1 -- start start 2023-11-27T15:23:23.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.008+0000 7fd98594bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560308a036f0 0x560308a07ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:23.011 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.008+0000 7fd98594bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a04540 0x560308a14130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:23.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.008+0000 7fd98594bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a04ef0 0x560308a08020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:23.012 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.008+0000 7fd98594bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603089f00e0 con 0x560308a04540 2023-11-27T15:23:23.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.008+0000 7fd98594bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56030890e0d0 con 0x560308a036f0 2023-11-27T15:23:23.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.008+0000 7fd98594bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603089f6350 con 0x560308a04ef0 2023-11-27T15:23:23.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.008+0000 7fd96affd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a04540 0x560308a14130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:23.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.008+0000 7fd96affd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a04540 0x560308a14130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60796/0 (socket says 172.21.15.133:60796) 2023-11-27T15:23:23.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.008+0000 7fd96affd700 1 -- 172.21.15.133:0/947203124 learned_addr learned my addr 172.21.15.133:0/947203124 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:23.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.008+0000 7fd96b7fe700 1 --2- 172.21.15.133:0/947203124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560308a036f0 0x560308a07ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:23.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.008+0000 7fd96bfff700 1 --2- 172.21.15.133:0/947203124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a04ef0 0x560308a08020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:23.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.008+0000 7fd96b7fe700 1 -- 172.21.15.133:0/947203124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a04ef0 msgr2=0x560308a08020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:23.013 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.008+0000 7fd96b7fe700 1 --2- 172.21.15.133:0/947203124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a04ef0 0x560308a08020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.009+0000 7fd96b7fe700 1 -- 172.21.15.133:0/947203124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a04540 msgr2=0x560308a14130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:23.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.009+0000 7fd96b7fe700 1 --2- 172.21.15.133:0/947203124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a04540 0x560308a14130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.009+0000 7fd96b7fe700 1 -- 172.21.15.133:0/947203124 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560308911ee0 con 0x560308a036f0 2023-11-27T15:23:23.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.009+0000 7fd96affd700 1 --2- 172.21.15.133:0/947203124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a04540 0x560308a14130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:23.014 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.009+0000 7fd96b7fe700 1 --2- 172.21.15.133:0/947203124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560308a036f0 0x560308a07ae0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd95c0095c0 tx=0x7fd95c0d1f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a05e68a7aadaa89b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:23.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.009+0000 7fd9527fc700 1 -- 172.21.15.133:0/947203124 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0d17a0 con 0x560308a036f0 2023-11-27T15:23:23.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.009+0000 7fd9527fc700 1 -- 172.21.15.133:0/947203124 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd95c0976a0 con 0x560308a036f0 2023-11-27T15:23:23.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.009+0000 7fd9527fc700 1 -- 172.21.15.133:0/947203124 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c09b360 con 0x560308a036f0 2023-11-27T15:23:23.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.010+0000 7fd98594bb80 1 -- 172.21.15.133:0/947203124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560308a036f0 msgr2=0x560308a07ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:23.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.010+0000 7fd98594bb80 1 --2- 172.21.15.133:0/947203124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560308a036f0 0x560308a07ae0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd95c0095c0 tx=0x7fd95c0d1f90 comp rx=0 tx=0).stop 2023-11-27T15:23:23.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.010+0000 7fd98594bb80 1 -- 172.21.15.133:0/947203124 shutdown_connections 2023-11-27T15:23:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.010+0000 7fd98594bb80 1 --2- 172.21.15.133:0/947203124 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a04ef0 0x560308a08020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.010+0000 7fd98594bb80 1 --2- 172.21.15.133:0/947203124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560308a036f0 0x560308a07ae0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.010+0000 7fd98594bb80 1 --2- 172.21.15.133:0/947203124 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a04540 0x560308a14130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.010+0000 7fd98594bb80 1 -- 172.21.15.133:0/947203124 >> 172.21.15.133:0/947203124 conn(0x5603089f05b0 msgr2=0x5603089f5780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.010+0000 7fd98594bb80 1 -- 172.21.15.133:0/947203124 shutdown_connections 2023-11-27T15:23:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.011+0000 7fd98594bb80 1 -- 172.21.15.133:0/947203124 wait complete. 2023-11-27T15:23:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.011+0000 7fd98594bb80 1 -- start start 2023-11-27T15:23:23.016 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.011+0000 7fd98594bb80 1 Processor -- start 2023-11-27T15:23:23.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.012+0000 7fd98594bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a036f0 0x560308a113f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:23.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.012+0000 7fd98594bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560308a04540 0x560308a11930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:23.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.012+0000 7fd98594bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a04ef0 0x560308a12940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:23.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.012+0000 7fd98594bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560308a12e80 con 0x560308a036f0 2023-11-27T15:23:23.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.012+0000 7fd98594bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x560308a12ff0 con 0x560308a04540 2023-11-27T15:23:23.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.012+0000 7fd98594bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560308a131b0 con 0x560308a04ef0 2023-11-27T15:23:23.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.012+0000 7fd98594bb80 10 client.0 _ll_register_callbacks cb 0x7fd9600097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:23.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.012+0000 7fd96b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a036f0 0x560308a113f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:23.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.012+0000 7fd96b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a036f0 0x560308a113f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60802/0 (socket says 172.21.15.133:60802) 2023-11-27T15:23:23.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.013+0000 7fd96b7fe700 1 -- 172.21.15.133:0/344735716 learned_addr learned my addr 172.21.15.133:0/344735716 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:23.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.013+0000 7fd96bfff700 1 --2- 172.21.15.133:0/344735716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a04ef0 0x560308a12940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:23.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.013+0000 7fd96b7fe700 1 -- 172.21.15.133:0/344735716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a04ef0 msgr2=0x560308a12940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:23.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.013+0000 7fd96b7fe700 1 --2- 172.21.15.133:0/344735716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a04ef0 0x560308a12940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.013+0000 7fd96b7fe700 1 -- 172.21.15.133:0/344735716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560308a04540 msgr2=0x560308a11930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:23.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.013+0000 7fd96b7fe700 1 --2- 172.21.15.133:0/344735716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560308a04540 0x560308a11930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.013+0000 7fd96b7fe700 1 -- 172.21.15.133:0/344735716 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd960009c60 con 0x560308a036f0 2023-11-27T15:23:23.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.013+0000 7fd96b7fe700 1 --2- 172.21.15.133:0/344735716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a036f0 0x560308a113f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd95c09b2f0 tx=0x7fd95c005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48060e676b2070e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:23.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.013+0000 7fd94affd700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:23:23.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.014+0000 7fd94affd700 1 -- 172.21.15.133:0/344735716 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0d17a0 con 0x560308a036f0 2023-11-27T15:23:23.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.014+0000 7fd94affd700 1 -- 172.21.15.133:0/344735716 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd95c0289d0 con 0x560308a036f0 2023-11-27T15:23:23.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.014+0000 7fd98594bb80 1 -- 172.21.15.133:0/344735716 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd95c09b130 con 0x560308a036f0 2023-11-27T15:23:23.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.014+0000 7fd94affd700 1 -- 172.21.15.133:0/344735716 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd95c09ab10 con 0x560308a036f0 2023-11-27T15:23:23.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.014+0000 7fd98594bb80 10 client.15564 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:23.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.014+0000 7fd98594bb80 1 -- 172.21.15.133:0/344735716 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560308912190 con 0x560308a036f0 2023-11-27T15:23:23.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.015+0000 7fd94affd700 1 -- 172.21.15.133:0/344735716 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd95c09b130 con 0x560308a036f0 2023-11-27T15:23:23.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.015+0000 7fd94affd700 1 -- 172.21.15.133:0/344735716 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fd95c097a70 con 0x560308a036f0 2023-11-27T15:23:23.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.015+0000 7fd98594bb80 10 client.15564 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:23.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.015+0000 7fd98594bb80 10 client.15564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:23.022 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.015+0000 7fd98594bb80 20 client.15564 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:23.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.015+0000 7fd98594bb80 1 --2- 172.21.15.133:0/344735716 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x560308a6c440 0x560308a6e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:23.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.015+0000 7fd98594bb80 4 client.15564 mds_command: new command op to 24560 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:23:23.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.015+0000 7fd98594bb80 1 -- 172.21.15.133:0/344735716 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56030891a660 con 0x560308a6c440 2023-11-27T15:23:23.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.016+0000 7fd96affd700 1 --2- 172.21.15.133:0/344735716 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x560308a6c440 0x560308a6e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:23.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.016+0000 7fd96affd700 1 --2- 172.21.15.133:0/344735716 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x560308a6c440 0x560308a6e820 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d732f19c466059f8 server_cookie=1b9edbb5c65a0298 in_seq=0 out_seq=0 2023-11-27T15:23:23.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.016+0000 7fd94affd700 10 client.15564 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:23.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.016+0000 7fd94affd700 1 -- 172.21.15.133:0/344735716 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56030891a660 con 0x560308a6c440 2023-11-27T15:23:23.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.016+0000 7fd94affd700 10 client.15564 handle_command_reply: tid=0 2023-11-27T15:23:23.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.016+0000 7fd98594bb80 1 client.15564 shutdown 2023-11-27T15:23:23.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.017+0000 7fd98594bb80 1 -- 172.21.15.133:0/344735716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a036f0 msgr2=0x560308a113f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:23.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.017+0000 7fd98594bb80 1 --2- 172.21.15.133:0/344735716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a036f0 0x560308a113f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd95c09b2f0 tx=0x7fd95c005050 comp rx=0 tx=0).stop 2023-11-27T15:23:23.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.017+0000 7fd98594bb80 1 -- 172.21.15.133:0/344735716 shutdown_connections 2023-11-27T15:23:23.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.017+0000 7fd98594bb80 1 --2- 172.21.15.133:0/344735716 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x560308a6c440 0x560308a6e820 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.017+0000 7fd98594bb80 1 --2- 172.21.15.133:0/344735716 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560308a04ef0 0x560308a12940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.017+0000 7fd98594bb80 1 --2- 172.21.15.133:0/344735716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560308a036f0 0x560308a113f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.017+0000 7fd94affd700 0 client.15564 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:23.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.017+0000 7fd98594bb80 1 --2- 172.21.15.133:0/344735716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560308a04540 0x560308a11930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.017+0000 7fd98594bb80 1 -- 172.21.15.133:0/344735716 >> 172.21.15.133:0/344735716 conn(0x5603089f05b0 msgr2=0x560308a07ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:23.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.017+0000 7fd98594bb80 1 -- 172.21.15.133:0/344735716 shutdown_connections 2023-11-27T15:23:23.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.018+0000 7fd98594bb80 1 -- 172.21.15.133:0/344735716 wait complete. 2023-11-27T15:23:23.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.018+0000 7fd98594bb80 20 client.15564 trim_cache size 0 max 16384 2023-11-27T15:23:23.025 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2023-11-27T15:23:23.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.019+0000 7fd970d9d700 1 -- 172.21.15.133:0/2676458402 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9540a11d0 msgr2=0x7fd9540a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:23.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.019+0000 7fd970d9d700 1 --2- 172.21.15.133:0/2676458402 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9540a11d0 0x7fd9540a3680 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd964009e50 tx=0x7fd964009040 comp rx=0 tx=0).stop 2023-11-27T15:23:23.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.019+0000 7fd970d9d700 1 -- 172.21.15.133:0/2676458402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd96c100050 msgr2=0x7fd96c195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:23.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.019+0000 7fd970d9d700 1 --2- 172.21.15.133:0/2676458402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd96c100050 0x7fd96c195e40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd95c004730 tx=0x7fd95c005ef0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.019+0000 7fd970d9d700 1 -- 172.21.15.133:0/2676458402 shutdown_connections 2023-11-27T15:23:23.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.019+0000 7fd970d9d700 1 --2- 172.21.15.133:0/2676458402 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd9540a11d0 0x7fd9540a3680 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.019+0000 7fd970d9d700 1 --2- 172.21.15.133:0/2676458402 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd96c101c90 0x7fd96c19d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.019+0000 7fd970d9d700 1 --2- 172.21.15.133:0/2676458402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd96c100050 0x7fd96c195e40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.019+0000 7fd970d9d700 1 --2- 172.21.15.133:0/2676458402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd96c1012e0 0x7fd96c196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:23.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.019+0000 7fd970d9d700 1 -- 172.21.15.133:0/2676458402 >> 172.21.15.133:0/2676458402 conn(0x7fd96c0fb650 msgr2=0x7fd96c1047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:23.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.019+0000 7fd970d9d700 1 -- 172.21.15.133:0/2676458402 shutdown_connections 2023-11-27T15:23:23.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:23.019+0000 7fd970d9d700 1 -- 172.21.15.133:0/2676458402 wait complete. 2023-11-27T15:23:23.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:22 smithi133 ceph-mon[97860]: pgmap v362: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:23:23.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:22 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2795154589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:23.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:22 smithi177 ceph-mon[131504]: pgmap v362: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:23:23.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:22 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2795154589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:23.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:22 smithi174 ceph-mon[131110]: pgmap v362: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:23:23.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:22 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2795154589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:23:24.052 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:23:24.052 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:23:24.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.286+0000 7f62f8b9d700 1 -- 172.21.15.133:0/1621152267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62f4107ed0 msgr2=0x7f62f410a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.286+0000 7f62f8b9d700 1 --2- 172.21.15.133:0/1621152267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62f4107ed0 0x7f62f410a2f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009f90 tx=0x7f62ec00b330 comp rx=0 tx=0).stop 2023-11-27T15:23:24.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.286+0000 7f62f8b9d700 1 -- 172.21.15.133:0/1621152267 shutdown_connections 2023-11-27T15:23:24.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.286+0000 7f62f8b9d700 1 --2- 172.21.15.133:0/1621152267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62f4107ed0 0x7f62f410a2f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.286+0000 7f62f8b9d700 1 --2- 172.21.15.133:0/1621152267 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62f4074a90 0x7f62f4072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.286+0000 7f62f8b9d700 1 --2- 172.21.15.133:0/1621152267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62f40734c0 0x7f62f4073930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.286+0000 7f62f8b9d700 1 -- 172.21.15.133:0/1621152267 >> 172.21.15.133:0/1621152267 conn(0x7f62f40fb650 msgr2=0x7f62f40fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:24.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.287+0000 7f62f8b9d700 1 -- 172.21.15.133:0/1621152267 shutdown_connections 2023-11-27T15:23:24.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.287+0000 7f62f8b9d700 1 -- 172.21.15.133:0/1621152267 wait complete. 2023-11-27T15:23:24.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.288+0000 7f62f8b9d700 1 Processor -- start 2023-11-27T15:23:24.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.288+0000 7f62f8b9d700 1 -- start start 2023-11-27T15:23:24.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.289+0000 7f62f8b9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62f40734c0 0x7f62f4198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.289+0000 7f62f8b9d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62f4074a90 0x7f62f41985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.289+0000 7f62f8b9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62f4107ed0 0x7f62f419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.289+0000 7f62f8b9d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f419fc00 con 0x7f62f4107ed0 2023-11-27T15:23:24.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.289+0000 7f62f8b9d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f419fd40 con 0x7f62f40734c0 2023-11-27T15:23:24.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.289+0000 7f62f8b9d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f419fed0 con 0x7f62f4074a90 2023-11-27T15:23:24.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.289+0000 7f62f37fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62f40734c0 0x7f62f4198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.289+0000 7f62f37fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62f40734c0 0x7f62f4198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:47814/0 (socket says 172.21.15.133:47814) 2023-11-27T15:23:24.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.289+0000 7f62f3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62f4107ed0 0x7f62f419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.289+0000 7f62f37fe700 1 -- 172.21.15.133:0/3125166433 learned_addr learned my addr 172.21.15.133:0/3125166433 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:24.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.289+0000 7f62f2ffd700 1 --2- 172.21.15.133:0/3125166433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62f4074a90 0x7f62f41985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.294 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.289+0000 7f62f37fe700 1 -- 172.21.15.133:0/3125166433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62f4074a90 msgr2=0x7f62f41985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.289+0000 7f62f37fe700 1 --2- 172.21.15.133:0/3125166433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62f4074a90 0x7f62f41985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.290+0000 7f62f37fe700 1 -- 172.21.15.133:0/3125166433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62f4107ed0 msgr2=0x7f62f419f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.290+0000 7f62f37fe700 1 --2- 172.21.15.133:0/3125166433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62f4107ed0 0x7f62f419f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.290+0000 7f62f37fe700 1 -- 172.21.15.133:0/3125166433 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e8009520 con 0x7f62f40734c0 2023-11-27T15:23:24.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.290+0000 7f62f37fe700 1 --2- 172.21.15.133:0/3125166433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62f40734c0 0x7f62f4198080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f62e400ebf0 tx=0x7f62e400ef00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:24.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.290+0000 7f62f0ff9700 1 -- 172.21.15.133:0/3125166433 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4009a10 con 0x7f62f40734c0 2023-11-27T15:23:24.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.290+0000 7f62f0ff9700 1 -- 172.21.15.133:0/3125166433 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62e400cd40 con 0x7f62f40734c0 2023-11-27T15:23:24.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.290+0000 7f62f0ff9700 1 -- 172.21.15.133:0/3125166433 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4018820 con 0x7f62f40734c0 2023-11-27T15:23:24.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.290+0000 7f62f8b9d700 1 -- 172.21.15.133:0/3125166433 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e40095f0 con 0x7f62f40734c0 2023-11-27T15:23:24.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.290+0000 7f62f8b9d700 1 -- 172.21.15.133:0/3125166433 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ec009c40 con 0x7f62f40734c0 2023-11-27T15:23:24.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.295+0000 7f630d635b80 1 Processor -- start 2023-11-27T15:23:24.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.295+0000 7f630d635b80 1 -- start start 2023-11-27T15:23:24.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.295+0000 7f630d635b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d9178f10 0x5619d9179320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.295+0000 7f630d635b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d9179860 0x5619d9179da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.295+0000 7f630d635b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d917a330 0x5619d917a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.295+0000 7f630d635b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d907e0e0 con 0x5619d9178f10 2023-11-27T15:23:24.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.295+0000 7f630d635b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d90d0920 con 0x5619d917a330 2023-11-27T15:23:24.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.295+0000 7f630d635b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d90c9db0 con 0x5619d9179860 2023-11-27T15:23:24.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.296+0000 7f62f3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d9178f10 0x5619d9179320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.296+0000 7f62f3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d9178f10 0x5619d9179320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60844/0 (socket says 172.21.15.133:60844) 2023-11-27T15:23:24.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.296+0000 7f62f3fff700 1 -- 172.21.15.133:0/2167615494 learned_addr learned my addr 172.21.15.133:0/2167615494 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:24.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.296+0000 7f62f2ffd700 1 --2- 172.21.15.133:0/2167615494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d917a330 0x5619d917a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.296+0000 7f62f3fff700 1 -- 172.21.15.133:0/2167615494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d9179860 msgr2=0x5619d9179da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-27T15:23:24.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.296+0000 7f62f3fff700 1 --2- 172.21.15.133:0/2167615494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d9179860 0x5619d9179da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.296+0000 7f62f3fff700 1 -- 172.21.15.133:0/2167615494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d917a330 msgr2=0x5619d917a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:24.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.297+0000 7f62f3fff700 1 --2- 172.21.15.133:0/2167615494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d917a330 0x5619d917a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.297+0000 7f62f3fff700 1 -- 172.21.15.133:0/2167615494 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619d8f4b6b0 con 0x5619d9178f10 2023-11-27T15:23:24.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.297+0000 7f62f0ff9700 1 -- 172.21.15.133:0/3125166433 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f62e4018980 con 0x7f62f40734c0 2023-11-27T15:23:24.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.297+0000 7f62f3fff700 1 --2- 172.21.15.133:0/2167615494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d9178f10 0x5619d9179320 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009ba0 tx=0x7f62ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f560177499cb3301 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:24.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.297+0000 7f62d9ffb700 1 -- 172.21.15.133:0/2167615494 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec00e040 con 0x5619d9178f10 2023-11-27T15:23:24.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.297+0000 7f62d9ffb700 1 -- 172.21.15.133:0/2167615494 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62ec0076a0 con 0x5619d9178f10 2023-11-27T15:23:24.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.297+0000 7f62d9ffb700 1 -- 172.21.15.133:0/2167615494 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec00be20 con 0x5619d9178f10 2023-11-27T15:23:24.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.298+0000 7f630d635b80 1 -- 172.21.15.133:0/2167615494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d9178f10 msgr2=0x5619d9179320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.298+0000 7f630d635b80 1 --2- 172.21.15.133:0/2167615494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d9178f10 0x5619d9179320 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009ba0 tx=0x7f62ec009bd0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.298+0000 7f630d635b80 1 -- 172.21.15.133:0/2167615494 shutdown_connections 2023-11-27T15:23:24.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.298+0000 7f630d635b80 1 --2- 172.21.15.133:0/2167615494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d917a330 0x5619d917a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.298+0000 7f630d635b80 1 --2- 172.21.15.133:0/2167615494 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d9178f10 0x5619d9179320 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.298+0000 7f630d635b80 1 --2- 172.21.15.133:0/2167615494 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d9179860 0x5619d9179da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.298+0000 7f630d635b80 1 -- 172.21.15.133:0/2167615494 >> 172.21.15.133:0/2167615494 conn(0x5619d916e630 msgr2=0x5619d91738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:24.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.299+0000 7f630d635b80 1 -- 172.21.15.133:0/2167615494 shutdown_connections 2023-11-27T15:23:24.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.299+0000 7f630d635b80 1 -- 172.21.15.133:0/2167615494 wait complete. 2023-11-27T15:23:24.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.299+0000 7f630d635b80 1 -- start start 2023-11-27T15:23:24.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.299+0000 7f62f0ff9700 1 --2- 172.21.15.133:0/3125166433 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f62dc0a1060 0x7f62dc0a3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.299+0000 7f630d635b80 1 Processor -- start 2023-11-27T15:23:24.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.300+0000 7f62f3fff700 1 --2- 172.21.15.133:0/3125166433 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f62dc0a1060 0x7f62dc0a3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.300+0000 7f62f0ff9700 1 -- 172.21.15.133:0/3125166433 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f62e4014070 con 0x7f62f40734c0 2023-11-27T15:23:24.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.300+0000 7f630d635b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d9178f10 0x5619d9182820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.300+0000 7f630d635b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d9179860 0x5619d9182d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.300+0000 7f630d635b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d917a330 0x5619d919add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.300+0000 7f630d635b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d906b960 con 0x5619d9179860 2023-11-27T15:23:24.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.300+0000 7f630d635b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d906b600 con 0x5619d917a330 2023-11-27T15:23:24.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.300+0000 7f630d635b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d8f4d9a0 con 0x5619d9178f10 2023-11-27T15:23:24.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.300+0000 7f630d635b80 10 client.0 _ll_register_callbacks cb 0x5619d9089800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:24.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.300+0000 7f62f37fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d9178f10 0x5619d9182820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.300+0000 7f62f37fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d9178f10 0x5619d9182820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:57936/0 (socket says 172.21.15.133:57936) 2023-11-27T15:23:24.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.300+0000 7f62f37fe700 1 -- 172.21.15.133:0/488779036 learned_addr learned my addr 172.21.15.133:0/488779036 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:24.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.301+0000 7f62f37fe700 1 -- 172.21.15.133:0/488779036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d917a330 msgr2=0x5619d919add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:24.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.301+0000 7f62f37fe700 1 --2- 172.21.15.133:0/488779036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d917a330 0x5619d919add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.301+0000 7f62f37fe700 1 -- 172.21.15.133:0/488779036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d9179860 msgr2=0x5619d9182d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:24.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.301+0000 7f62f2ffd700 1 --2- 172.21.15.133:0/488779036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d9179860 0x5619d9182d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.301+0000 7f62f37fe700 1 --2- 172.21.15.133:0/488779036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d9179860 0x5619d9182d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.301+0000 7f62f37fe700 1 -- 172.21.15.133:0/488779036 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f410afb0 con 0x5619d9178f10 2023-11-27T15:23:24.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.301+0000 7f62f2ffd700 1 --2- 172.21.15.133:0/488779036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d9179860 0x5619d9182d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:23:24.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.301+0000 7f62f37fe700 1 --2- 172.21.15.133:0/488779036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d9178f10 0x5619d9182820 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f62e409a370 tx=0x7f62e409a920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb553ac1c3b70f6d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:24.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.301+0000 7f62d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:23:24.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.301+0000 7f62f3fff700 1 --2- 172.21.15.133:0/3125166433 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f62dc0a1060 0x7f62dc0a3510 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f62ec003ed0 tx=0x7f62ec009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:24.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.301+0000 7f62d2ffd700 1 -- 172.21.15.133:0/488779036 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e40cd170 con 0x5619d9178f10 2023-11-27T15:23:24.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.301+0000 7f630d635b80 1 -- 172.21.15.133:0/488779036 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619d916bdf0 con 0x5619d9178f10 2023-11-27T15:23:24.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.301+0000 7f62d2ffd700 1 -- 172.21.15.133:0/488779036 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f62e409e410 con 0x5619d9178f10 2023-11-27T15:23:24.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.302+0000 7f62d2ffd700 1 -- 172.21.15.133:0/488779036 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62e40a2830 con 0x5619d9178f10 2023-11-27T15:23:24.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.302+0000 7f630d635b80 10 client.25606 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:24.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.302+0000 7f630d635b80 1 -- 172.21.15.133:0/488779036 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5619d9090210 con 0x5619d9178f10 2023-11-27T15:23:24.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.302+0000 7f62d2ffd700 1 -- 172.21.15.133:0/488779036 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5619d916bdf0 con 0x5619d9178f10 2023-11-27T15:23:24.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.302+0000 7f62d2ffd700 1 -- 172.21.15.133:0/488779036 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f62e40b8a10 con 0x5619d9178f10 2023-11-27T15:23:24.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.302+0000 7f630d635b80 10 client.25606 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:24.308 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.302+0000 7f630d635b80 10 client.25606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:24.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.302+0000 7f630d635b80 20 client.25606 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:24.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.302+0000 7f630d635b80 1 --2- 172.21.15.133:0/488779036 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5619d919f610 0x5619d91a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.302+0000 7f630d635b80 4 client.25606 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:24.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.302+0000 7f630d635b80 1 -- 172.21.15.133:0/488779036 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619d908c120 con 0x5619d919f610 2023-11-27T15:23:24.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.303+0000 7f62f2ffd700 1 --2- 172.21.15.133:0/488779036 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5619d919f610 0x5619d91a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.303+0000 7f62f2ffd700 1 --2- 172.21.15.133:0/488779036 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5619d919f610 0x5619d91a1a20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afd20c334ef8e4a server_cookie=799b22e289b3c1f4 in_seq=0 out_seq=0 2023-11-27T15:23:24.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.303+0000 7f62d2ffd700 10 client.25606 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:24.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.304+0000 7f62d2ffd700 1 -- 172.21.15.133:0/488779036 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5619d908c120 con 0x5619d919f610 2023-11-27T15:23:24.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.304+0000 7f62d2ffd700 10 client.25606 handle_command_reply: tid=0 2023-11-27T15:23:24.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.304+0000 7f630d635b80 1 client.25606 shutdown 2023-11-27T15:23:24.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.304+0000 7f630d635b80 1 -- 172.21.15.133:0/488779036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d9178f10 msgr2=0x5619d9182820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.304+0000 7f630d635b80 1 --2- 172.21.15.133:0/488779036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d9178f10 0x5619d9182820 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f62e409a370 tx=0x7f62e409a920 comp rx=0 tx=0).stop 2023-11-27T15:23:24.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.304+0000 7f630d635b80 1 -- 172.21.15.133:0/488779036 shutdown_connections 2023-11-27T15:23:24.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.304+0000 7f630d635b80 1 --2- 172.21.15.133:0/488779036 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5619d919f610 0x5619d91a1a20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.304+0000 7f630d635b80 1 --2- 172.21.15.133:0/488779036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d917a330 0x5619d919add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.304+0000 7f630d635b80 1 --2- 172.21.15.133:0/488779036 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d9178f10 0x5619d9182820 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.304+0000 7f62d2ffd700 0 client.25606 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:24.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.304+0000 7f630d635b80 1 --2- 172.21.15.133:0/488779036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d9179860 0x5619d9182d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.304+0000 7f630d635b80 1 -- 172.21.15.133:0/488779036 >> 172.21.15.133:0/488779036 conn(0x5619d916e630 msgr2=0x5619d91730b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:24.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.305+0000 7f630d635b80 1 -- 172.21.15.133:0/488779036 shutdown_connections 2023-11-27T15:23:24.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.305+0000 7f630d635b80 1 -- 172.21.15.133:0/488779036 wait complete. 2023-11-27T15:23:24.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.305+0000 7f630d635b80 20 client.25606 trim_cache size 0 max 16384 2023-11-27T15:23:24.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.322+0000 7f630d635b80 1 Processor -- start 2023-11-27T15:23:24.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.323+0000 7f630d635b80 1 -- start start 2023-11-27T15:23:24.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.323+0000 7f630d635b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d9178f10 0x5619d91af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.323+0000 7f630d635b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d9179860 0x5619d917d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.323+0000 7f630d635b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d917a330 0x5619d917d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.323+0000 7f630d635b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d908c120 con 0x5619d917a330 2023-11-27T15:23:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.323+0000 7f630d635b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d919caa0 con 0x5619d9178f10 2023-11-27T15:23:24.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.323+0000 7f630d635b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d91839c0 con 0x5619d9179860 2023-11-27T15:23:24.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.323+0000 7f62f37fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d9178f10 0x5619d91af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.323+0000 7f62f37fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d9178f10 0x5619d91af2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:47834/0 (socket says 172.21.15.133:47834) 2023-11-27T15:23:24.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.323+0000 7f62f37fe700 1 -- 172.21.15.133:0/3007514859 learned_addr learned my addr 172.21.15.133:0/3007514859 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:24.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.323+0000 7f62f3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d917a330 0x5619d917d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.323+0000 7f62f2ffd700 1 --2- 172.21.15.133:0/3007514859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d9179860 0x5619d917d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.324+0000 7f62f37fe700 1 -- 172.21.15.133:0/3007514859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d9179860 msgr2=0x5619d917d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:24.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.324+0000 7f62f37fe700 1 --2- 172.21.15.133:0/3007514859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d9179860 0x5619d917d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.324+0000 7f62f37fe700 1 -- 172.21.15.133:0/3007514859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d917a330 msgr2=0x5619d917d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:24.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.324+0000 7f62f37fe700 1 --2- 172.21.15.133:0/3007514859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d917a330 0x5619d917d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.324+0000 7f62f37fe700 1 -- 172.21.15.133:0/3007514859 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619d917a9d0 con 0x5619d9178f10 2023-11-27T15:23:24.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.324+0000 7f62f2ffd700 1 --2- 172.21.15.133:0/3007514859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d9179860 0x5619d917d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:24.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.324+0000 7f62f3fff700 1 --2- 172.21.15.133:0/3007514859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d917a330 0x5619d917d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:24.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.324+0000 7f62f37fe700 1 --2- 172.21.15.133:0/3007514859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d9178f10 0x5619d91af2a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f62e400c660 tx=0x7f62e4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=204b33bb5136497a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:24.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.324+0000 7f62da7fc700 1 -- 172.21.15.133:0/3007514859 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4099730 con 0x5619d9178f10 2023-11-27T15:23:24.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.324+0000 7f62da7fc700 1 -- 172.21.15.133:0/3007514859 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62e40a2830 con 0x5619d9178f10 2023-11-27T15:23:24.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.324+0000 7f62da7fc700 1 -- 172.21.15.133:0/3007514859 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e40b8b30 con 0x5619d9178f10 2023-11-27T15:23:24.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.325+0000 7f630d635b80 1 -- 172.21.15.133:0/3007514859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d9178f10 msgr2=0x5619d91af2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.325+0000 7f630d635b80 1 --2- 172.21.15.133:0/3007514859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d9178f10 0x5619d91af2a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f62e400c660 tx=0x7f62e4000bc0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.325+0000 7f630d635b80 1 -- 172.21.15.133:0/3007514859 shutdown_connections 2023-11-27T15:23:24.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.325+0000 7f630d635b80 1 --2- 172.21.15.133:0/3007514859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d917a330 0x5619d917d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.325+0000 7f630d635b80 1 --2- 172.21.15.133:0/3007514859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d9178f10 0x5619d91af2a0 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f62e400c660 tx=0x7f62e4000bc0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.325+0000 7f630d635b80 1 --2- 172.21.15.133:0/3007514859 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d9179860 0x5619d917d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.325+0000 7f630d635b80 1 -- 172.21.15.133:0/3007514859 >> 172.21.15.133:0/3007514859 conn(0x5619d916e630 msgr2=0x5619d91806c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:24.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.325+0000 7f630d635b80 1 -- 172.21.15.133:0/3007514859 shutdown_connections 2023-11-27T15:23:24.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.325+0000 7f630d635b80 1 -- 172.21.15.133:0/3007514859 wait complete. 2023-11-27T15:23:24.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.326+0000 7f630d635b80 1 -- start start 2023-11-27T15:23:24.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.326+0000 7f630d635b80 1 Processor -- start 2023-11-27T15:23:24.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.327+0000 7f630d635b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d9179860 0x5619d91af310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.327+0000 7f630d635b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d917a330 0x5619d91ad920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.327+0000 7f630d635b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d91ade60 0x5619d91ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.327+0000 7f630d635b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d91f3c50 con 0x5619d91ade60 2023-11-27T15:23:24.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.327+0000 7f630d635b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d91f3d90 con 0x5619d9179860 2023-11-27T15:23:24.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.327+0000 7f630d635b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619d91f3ed0 con 0x5619d917a330 2023-11-27T15:23:24.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.327+0000 7f630d635b80 10 client.0 _ll_register_callbacks cb 0x7f62e409a2a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:24.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.327+0000 7f62f37fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d9179860 0x5619d91af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.327+0000 7f62f37fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d9179860 0x5619d91af310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:47850/0 (socket says 172.21.15.133:47850) 2023-11-27T15:23:24.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.327+0000 7f62f37fe700 1 -- 172.21.15.133:0/2966112665 learned_addr learned my addr 172.21.15.133:0/2966112665 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:24.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.327+0000 7f62f2ffd700 1 --2- 172.21.15.133:0/2966112665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d917a330 0x5619d91ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.327+0000 7f62f3fff700 1 --2- 172.21.15.133:0/2966112665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d91ade60 0x5619d91ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.328+0000 7f62f37fe700 1 -- 172.21.15.133:0/2966112665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d917a330 msgr2=0x5619d91ad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:24.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.328+0000 7f62f37fe700 1 --2- 172.21.15.133:0/2966112665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d917a330 0x5619d91ad920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.328+0000 7f62f37fe700 1 -- 172.21.15.133:0/2966112665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d91ade60 msgr2=0x5619d91ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:24.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.328+0000 7f62f37fe700 1 --2- 172.21.15.133:0/2966112665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d91ade60 0x5619d91ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.328+0000 7f62f37fe700 1 -- 172.21.15.133:0/2966112665 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619d9090210 con 0x5619d9179860 2023-11-27T15:23:24.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.328+0000 7f62f3fff700 1 --2- 172.21.15.133:0/2966112665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d91ade60 0x5619d91ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:23:24.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.328+0000 7f62f37fe700 1 --2- 172.21.15.133:0/2966112665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d9179860 0x5619d91af310 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f62e40968e0 tx=0x7f62e4098990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4874054e506ddf0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:24.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.328+0000 7f62d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:23:24.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.328+0000 7f62d2ffd700 1 -- 172.21.15.133:0/2966112665 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e409e120 con 0x5619d9179860 2023-11-27T15:23:24.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.328+0000 7f62d2ffd700 1 -- 172.21.15.133:0/2966112665 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f62e409e410 con 0x5619d9179860 2023-11-27T15:23:24.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.328+0000 7f630d635b80 1 -- 172.21.15.133:0/2966112665 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619d90986e0 con 0x5619d9179860 2023-11-27T15:23:24.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.329+0000 7f62d2ffd700 1 -- 172.21.15.133:0/2966112665 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62e40a2830 con 0x5619d9179860 2023-11-27T15:23:24.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.329+0000 7f630d635b80 10 client.25619 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:24.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.329+0000 7f630d635b80 1 -- 172.21.15.133:0/2966112665 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5619d8ef7e60 con 0x5619d9179860 2023-11-27T15:23:24.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.329+0000 7f62d2ffd700 1 -- 172.21.15.133:0/2966112665 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5619d90986e0 con 0x5619d9179860 2023-11-27T15:23:24.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.329+0000 7f62d2ffd700 1 -- 172.21.15.133:0/2966112665 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f62e409b3d0 con 0x5619d9179860 2023-11-27T15:23:24.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.329+0000 7f630d635b80 10 client.25619 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:24.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.329+0000 7f630d635b80 10 client.25619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:24.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.329+0000 7f630d635b80 20 client.25619 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:24.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.329+0000 7f630d635b80 1 --2- 172.21.15.133:0/2966112665 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5619d91f5320 0x5619d91f7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.329+0000 7f630d635b80 4 client.25619 mds_command: new command op to 24560 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:23:24.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.329+0000 7f630d635b80 1 -- 172.21.15.133:0/2966112665 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5619d8f18ff0 con 0x5619d91f5320 2023-11-27T15:23:24.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.329+0000 7f62f2ffd700 1 --2- 172.21.15.133:0/2966112665 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5619d91f5320 0x5619d91f7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.330+0000 7f62d2ffd700 10 client.25619 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:24.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.330+0000 7f62f2ffd700 1 --2- 172.21.15.133:0/2966112665 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5619d91f5320 0x5619d91f7730 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca5098c1b97b517 server_cookie=fd884a44876a5b7f in_seq=0 out_seq=0 2023-11-27T15:23:24.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.331+0000 7f62d2ffd700 1 -- 172.21.15.133:0/2966112665 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5619d8f18ff0 con 0x5619d91f5320 2023-11-27T15:23:24.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.331+0000 7f62d2ffd700 10 client.25619 handle_command_reply: tid=0 2023-11-27T15:23:24.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.331+0000 7f630d635b80 1 client.25619 shutdown 2023-11-27T15:23:24.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.331+0000 7f630d635b80 1 -- 172.21.15.133:0/2966112665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d9179860 msgr2=0x5619d91af310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.331+0000 7f630d635b80 1 --2- 172.21.15.133:0/2966112665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d9179860 0x5619d91af310 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f62e40968e0 tx=0x7f62e4098990 comp rx=0 tx=0).stop 2023-11-27T15:23:24.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.332+0000 7f630d635b80 1 -- 172.21.15.133:0/2966112665 shutdown_connections 2023-11-27T15:23:24.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.332+0000 7f630d635b80 1 --2- 172.21.15.133:0/2966112665 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5619d91f5320 0x5619d91f7730 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.332+0000 7f630d635b80 1 --2- 172.21.15.133:0/2966112665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5619d91ade60 0x5619d91ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.332+0000 7f630d635b80 1 --2- 172.21.15.133:0/2966112665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d9179860 0x5619d91af310 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.332+0000 7f630d635b80 1 --2- 172.21.15.133:0/2966112665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5619d917a330 0x5619d91ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.332+0000 7f630d635b80 1 -- 172.21.15.133:0/2966112665 >> 172.21.15.133:0/2966112665 conn(0x5619d916e630 msgr2=0x5619d917d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:24.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.332+0000 7f62d2ffd700 0 client.25619 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:24.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.332+0000 7f630d635b80 1 -- 172.21.15.133:0/2966112665 shutdown_connections 2023-11-27T15:23:24.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.332+0000 7f630d635b80 1 -- 172.21.15.133:0/2966112665 wait complete. 2023-11-27T15:23:24.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.332+0000 7f630d635b80 20 client.25619 trim_cache size 0 max 16384 2023-11-27T15:23:24.339 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:23:24.339 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2023-11-27T15:23:24.339 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "2f111e46-b73b-4444-b4f8-e196a6e85e69", 2023-11-27T15:23:24.339 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2023-11-27T15:23:24.339 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:23:24.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.333+0000 7f62f8b9d700 1 -- 172.21.15.133:0/3125166433 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f62dc0a1060 msgr2=0x7f62dc0a3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.333+0000 7f62f8b9d700 1 --2- 172.21.15.133:0/3125166433 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f62dc0a1060 0x7f62dc0a3510 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f62ec003ed0 tx=0x7f62ec009920 comp rx=0 tx=0).stop 2023-11-27T15:23:24.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.333+0000 7f62f8b9d700 1 -- 172.21.15.133:0/3125166433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62f40734c0 msgr2=0x7f62f4198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.333+0000 7f62f8b9d700 1 --2- 172.21.15.133:0/3125166433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62f40734c0 0x7f62f4198080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f62e400ebf0 tx=0x7f62e400ef00 comp rx=0 tx=0).stop 2023-11-27T15:23:24.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.333+0000 7f62f8b9d700 1 -- 172.21.15.133:0/3125166433 shutdown_connections 2023-11-27T15:23:24.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.333+0000 7f62f8b9d700 1 --2- 172.21.15.133:0/3125166433 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f62dc0a1060 0x7f62dc0a3510 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.333+0000 7f62f8b9d700 1 --2- 172.21.15.133:0/3125166433 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62f4107ed0 0x7f62f419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.333+0000 7f62f8b9d700 1 --2- 172.21.15.133:0/3125166433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62f40734c0 0x7f62f4198080 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.333+0000 7f62f8b9d700 1 --2- 172.21.15.133:0/3125166433 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62f4074a90 0x7f62f41985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.333+0000 7f62f8b9d700 1 -- 172.21.15.133:0/3125166433 >> 172.21.15.133:0/3125166433 conn(0x7f62f40fb650 msgr2=0x7f62f410a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:24.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.334+0000 7f62f8b9d700 1 -- 172.21.15.133:0/3125166433 shutdown_connections 2023-11-27T15:23:24.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.334+0000 7f62f8b9d700 1 -- 172.21.15.133:0/3125166433 wait complete. 2023-11-27T15:23:24.356 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:23:24.572 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.568+0000 7f60b99ff700 1 -- 172.21.15.133:0/2576587647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60b4101e50 msgr2=0x7f60b4108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.572 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.568+0000 7f60b99ff700 1 --2- 172.21.15.133:0/2576587647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60b4101e50 0x7f60b4108710 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00b0a0 tx=0x7f60ac00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.572 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.568+0000 7f60b99ff700 1 -- 172.21.15.133:0/2576587647 shutdown_connections 2023-11-27T15:23:24.572 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.568+0000 7f60b99ff700 1 --2- 172.21.15.133:0/2576587647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60b4101e50 0x7f60b4108710 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.572 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.568+0000 7f60b99ff700 1 --2- 172.21.15.133:0/2576587647 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60b41002d0 0x7f60b41006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.572 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.568+0000 7f60b99ff700 1 --2- 172.21.15.133:0/2576587647 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60b41014c0 0x7f60b4101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.572 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.568+0000 7f60b99ff700 1 -- 172.21.15.133:0/2576587647 >> 172.21.15.133:0/2576587647 conn(0x7f60b40fb8b0 msgr2=0x7f60b40fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:24.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.568+0000 7f60b99ff700 1 -- 172.21.15.133:0/2576587647 shutdown_connections 2023-11-27T15:23:24.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.568+0000 7f60b99ff700 1 -- 172.21.15.133:0/2576587647 wait complete. 2023-11-27T15:23:24.573 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.569+0000 7f60b99ff700 1 Processor -- start 2023-11-27T15:23:24.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.570+0000 7f60b99ff700 1 -- start start 2023-11-27T15:23:24.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.570+0000 7f60b99ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60b41002d0 0x7f60b4198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.570+0000 7f60b99ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60b41014c0 0x7f60b4198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.570+0000 7f60b99ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60b4101e50 0x7f60b419f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.570+0000 7f60b99ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b419feb0 con 0x7f60b41002d0 2023-11-27T15:23:24.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.570+0000 7f60b99ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b419fff0 con 0x7f60b4101e50 2023-11-27T15:23:24.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.570+0000 7f60b99ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b41a0180 con 0x7f60b41014c0 2023-11-27T15:23:24.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.570+0000 7f60b27fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60b41014c0 0x7f60b4198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.570+0000 7f60b37fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60b4101e50 0x7f60b419f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.570+0000 7f60b27fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60b41014c0 0x7f60b4198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:57968/0 (socket says 172.21.15.133:57968) 2023-11-27T15:23:24.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.570+0000 7f60b27fc700 1 -- 172.21.15.133:0/3525192350 learned_addr learned my addr 172.21.15.133:0/3525192350 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:24.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.571+0000 7f60b2ffd700 1 --2- 172.21.15.133:0/3525192350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60b41002d0 0x7f60b4198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.571+0000 7f60b27fc700 1 -- 172.21.15.133:0/3525192350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60b4101e50 msgr2=0x7f60b419f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.571+0000 7f60b27fc700 1 --2- 172.21.15.133:0/3525192350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60b4101e50 0x7f60b419f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.571+0000 7f60b27fc700 1 -- 172.21.15.133:0/3525192350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60b41002d0 msgr2=0x7f60b4198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.571+0000 7f60b27fc700 1 --2- 172.21.15.133:0/3525192350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60b41002d0 0x7f60b4198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.571+0000 7f60b27fc700 1 -- 172.21.15.133:0/3525192350 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac009d00 con 0x7f60b41014c0 2023-11-27T15:23:24.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.571+0000 7f60b27fc700 1 --2- 172.21.15.133:0/3525192350 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60b41014c0 0x7f60b4198870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f60a800d980 tx=0x7f60a800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:24.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.572+0000 7f60b89fd700 1 -- 172.21.15.133:0/3525192350 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a8004220 con 0x7f60b41014c0 2023-11-27T15:23:24.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.572+0000 7f60b89fd700 1 -- 172.21.15.133:0/3525192350 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60a8004cf0 con 0x7f60b41014c0 2023-11-27T15:23:24.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.572+0000 7f60b89fd700 1 -- 172.21.15.133:0/3525192350 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a8005720 con 0x7f60b41014c0 2023-11-27T15:23:24.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.572+0000 7f60b99ff700 1 -- 172.21.15.133:0/3525192350 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b410bc60 con 0x7f60b41014c0 2023-11-27T15:23:24.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.573+0000 7f60b99ff700 1 -- 172.21.15.133:0/3525192350 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b410bad0 con 0x7f60b41014c0 2023-11-27T15:23:24.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.577+0000 7f60cd3cdb80 1 Processor -- start 2023-11-27T15:23:24.581 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.577+0000 7f60b89fd700 1 -- 172.21.15.133:0/3525192350 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f60a8004410 con 0x7f60b41014c0 2023-11-27T15:23:24.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.578+0000 7f60cd3cdb80 1 -- start start 2023-11-27T15:23:24.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.578+0000 7f60cd3cdb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d0549c00 0x5594d054a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.582 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.578+0000 7f60cd3cdb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d054a550 0x5594d054ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.578+0000 7f60cd3cdb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054b110 0x5594d054b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.578+0000 7f60cd3cdb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d044f060 con 0x5594d054b110 2023-11-27T15:23:24.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.578+0000 7f60cd3cdb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d04a18a0 con 0x5594d054a550 2023-11-27T15:23:24.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.578+0000 7f60cd3cdb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d049ad30 con 0x5594d0549c00 2023-11-27T15:23:24.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.579+0000 7f60b37fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d0549c00 0x5594d054a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.579+0000 7f60b27fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054b110 0x5594d054b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.579+0000 7f60b37fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d0549c00 0x5594d054a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:57974/0 (socket says 172.21.15.133:57974) 2023-11-27T15:23:24.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.579+0000 7f60b37fe700 1 -- 172.21.15.133:0/1934145493 learned_addr learned my addr 172.21.15.133:0/1934145493 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:24.583 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.579+0000 7f60b2ffd700 1 --2- 172.21.15.133:0/1934145493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d054a550 0x5594d054ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.579+0000 7f60b2ffd700 1 -- 172.21.15.133:0/1934145493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d0549c00 msgr2=0x5594d054a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:24.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.579+0000 7f60b2ffd700 1 --2- 172.21.15.133:0/1934145493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d0549c00 0x5594d054a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.579+0000 7f60b2ffd700 1 -- 172.21.15.133:0/1934145493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054b110 msgr2=0x5594d054b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:24.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.579+0000 7f60b2ffd700 1 --2- 172.21.15.133:0/1934145493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054b110 0x5594d054b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.584 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.579+0000 7f60b2ffd700 1 -- 172.21.15.133:0/1934145493 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594d031c630 con 0x5594d054a550 2023-11-27T15:23:24.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.579+0000 7f60b27fc700 1 --2- 172.21.15.133:0/1934145493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054b110 0x5594d054b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:24.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.580+0000 7f60b2ffd700 1 --2- 172.21.15.133:0/1934145493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d054a550 0x5594d054ab30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f60b4101320 tx=0x7f60a400ce90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b2d8f662b8d44a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:24.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.580+0000 7f60b89fd700 1 --2- 172.21.15.133:0/3525192350 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f609c0a11d0 0x7f609c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.580+0000 7f60b89fd700 1 -- 172.21.15.133:0/3525192350 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f60a80cc830 con 0x7f60b41014c0 2023-11-27T15:23:24.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.580+0000 7f6099ffb700 1 -- 172.21.15.133:0/1934145493 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a400ed10 con 0x5594d054a550 2023-11-27T15:23:24.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.580+0000 7f6099ffb700 1 -- 172.21.15.133:0/1934145493 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60a400ee70 con 0x5594d054a550 2023-11-27T15:23:24.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.580+0000 7f6099ffb700 1 -- 172.21.15.133:0/1934145493 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a4010690 con 0x5594d054a550 2023-11-27T15:23:24.585 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.580+0000 7f60b37fe700 1 --2- 172.21.15.133:0/3525192350 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f609c0a11d0 0x7f609c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.581+0000 7f60b37fe700 1 --2- 172.21.15.133:0/3525192350 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f609c0a11d0 0x7f609c0a3680 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f60ac015000 tx=0x7f60ac00b7d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:24.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.581+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/1934145493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d054a550 msgr2=0x5594d054ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.581+0000 7f60cd3cdb80 1 --2- 172.21.15.133:0/1934145493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d054a550 0x5594d054ab30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f60b4101320 tx=0x7f60a400ce90 comp rx=0 tx=0).stop 2023-11-27T15:23:24.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.581+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/1934145493 shutdown_connections 2023-11-27T15:23:24.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.581+0000 7f60cd3cdb80 1 --2- 172.21.15.133:0/1934145493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054b110 0x5594d054b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.581+0000 7f60cd3cdb80 1 --2- 172.21.15.133:0/1934145493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d0549c00 0x5594d054a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.581+0000 7f60cd3cdb80 1 --2- 172.21.15.133:0/1934145493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d054a550 0x5594d054ab30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.581+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/1934145493 >> 172.21.15.133:0/1934145493 conn(0x5594d053f360 msgr2=0x5594d0544670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:24.586 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.582+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/1934145493 shutdown_connections 2023-11-27T15:23:24.587 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.582+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/1934145493 wait complete. 2023-11-27T15:23:24.587 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.582+0000 7f60cd3cdb80 1 -- start start 2023-11-27T15:23:24.587 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.583+0000 7f60cd3cdb80 1 Processor -- start 2023-11-27T15:23:24.587 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.584+0000 7f60cd3cdb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d0549c00 0x5594d0553620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.588 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.584+0000 7f60cd3cdb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d054a550 0x5594d0553b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.585+0000 7f60b2ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d0549c00 0x5594d0553620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.585+0000 7f60b2ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d0549c00 0x5594d0553620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:57986/0 (socket says 172.21.15.133:57986) 2023-11-27T15:23:24.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.585+0000 7f60b2ffd700 1 -- 172.21.15.133:0/1288633563 learned_addr learned my addr 172.21.15.133:0/1288633563 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:24.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.585+0000 7f60b27fc700 1 --2- 172.21.15.133:0/1288633563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d054a550 0x5594d0553b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.585+0000 7f60cd3cdb80 1 --2- 172.21.15.133:0/1288633563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054b110 0x5594d055aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.585+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/1288633563 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d043c8e0 con 0x5594d054b110 2023-11-27T15:23:24.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.585+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/1288633563 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d043c580 con 0x5594d054a550 2023-11-27T15:23:24.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.585+0000 7f60b37fe700 1 --2- 172.21.15.133:0/1288633563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054b110 0x5594d055aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.585+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/1288633563 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d031e920 con 0x5594d0549c00 2023-11-27T15:23:24.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.586+0000 7f60cd3cdb80 10 client.0 _ll_register_callbacks cb 0x5594d045a780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:24.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.586+0000 7f60b37fe700 1 -- 172.21.15.133:0/1288633563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d0549c00 msgr2=0x5594d0553620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:24.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.586+0000 7f60b37fe700 1 --2- 172.21.15.133:0/1288633563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d0549c00 0x5594d0553620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.586+0000 7f60b37fe700 1 -- 172.21.15.133:0/1288633563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d054a550 msgr2=0x5594d0553b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:24.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.586+0000 7f60b37fe700 1 --2- 172.21.15.133:0/1288633563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d054a550 0x5594d0553b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.586+0000 7f60b37fe700 1 -- 172.21.15.133:0/1288633563 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a400c810 con 0x5594d054b110 2023-11-27T15:23:24.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.586+0000 7f60b2ffd700 1 --2- 172.21.15.133:0/1288633563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d0549c00 0x5594d0553620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:24.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.587+0000 7f60b37fe700 1 --2- 172.21.15.133:0/1288633563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054b110 0x5594d055aba0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00d010 tx=0x7f60ac00b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca3055dd2e05eda server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:24.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.587+0000 7f6092ffd700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:23:24.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.587+0000 7f6092ffd700 1 -- 172.21.15.133:0/1288633563 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac026070 con 0x5594d054b110 2023-11-27T15:23:24.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.587+0000 7f6092ffd700 1 -- 172.21.15.133:0/1288633563 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f60ac005250 con 0x5594d054b110 2023-11-27T15:23:24.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.588+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/1288633563 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5594d053cb20 con 0x5594d054b110 2023-11-27T15:23:24.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.588+0000 7f6092ffd700 1 -- 172.21.15.133:0/1288633563 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60ac00e040 con 0x5594d054b110 2023-11-27T15:23:24.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.588+0000 7f60cd3cdb80 10 client.15588 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:24.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.588+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/1288633563 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5594d0461190 con 0x5594d054b110 2023-11-27T15:23:24.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.589+0000 7f6092ffd700 1 -- 172.21.15.133:0/1288633563 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5594d053cb20 con 0x5594d054b110 2023-11-27T15:23:24.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.589+0000 7f6092ffd700 1 -- 172.21.15.133:0/1288633563 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f60ac02c7c0 con 0x5594d054b110 2023-11-27T15:23:24.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.589+0000 7f60cd3cdb80 10 client.15588 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:24.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.589+0000 7f60cd3cdb80 10 client.15588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:24.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.589+0000 7f60cd3cdb80 20 client.15588 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:24.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.589+0000 7f60cd3cdb80 1 --2- 172.21.15.133:0/1288633563 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5594d055f430 0x5594d0561840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.589+0000 7f60cd3cdb80 4 client.15588 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:24.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.589+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/1288633563 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594d045d0a0 con 0x5594d055f430 2023-11-27T15:23:24.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.590+0000 7f60b2ffd700 1 --2- 172.21.15.133:0/1288633563 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5594d055f430 0x5594d0561840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.590+0000 7f60b2ffd700 1 --2- 172.21.15.133:0/1288633563 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5594d055f430 0x5594d0561840 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4d5252677a2cb05 server_cookie=d4e8b7a3a1412aa4 in_seq=0 out_seq=0 2023-11-27T15:23:24.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.590+0000 7f6092ffd700 10 client.15588 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:24.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.591+0000 7f6092ffd700 1 -- 172.21.15.133:0/1288633563 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5594d045d0a0 con 0x5594d055f430 2023-11-27T15:23:24.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.591+0000 7f6092ffd700 10 client.15588 handle_command_reply: tid=0 2023-11-27T15:23:24.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.591+0000 7f60cd3cdb80 1 client.15588 shutdown 2023-11-27T15:23:24.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.592+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/1288633563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054b110 msgr2=0x5594d055aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.592+0000 7f60cd3cdb80 1 --2- 172.21.15.133:0/1288633563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054b110 0x5594d055aba0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00d010 tx=0x7f60ac00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.592+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/1288633563 shutdown_connections 2023-11-27T15:23:24.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.592+0000 7f60cd3cdb80 1 --2- 172.21.15.133:0/1288633563 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5594d055f430 0x5594d0561840 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.592+0000 7f60cd3cdb80 1 --2- 172.21.15.133:0/1288633563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054b110 0x5594d055aba0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.592+0000 7f6092ffd700 0 client.15588 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:24.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.592+0000 7f60cd3cdb80 1 --2- 172.21.15.133:0/1288633563 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d0549c00 0x5594d0553620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.592+0000 7f60cd3cdb80 1 --2- 172.21.15.133:0/1288633563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d054a550 0x5594d0553b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.592+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/1288633563 >> 172.21.15.133:0/1288633563 conn(0x5594d053f360 msgr2=0x5594d0543de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:24.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.593+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/1288633563 shutdown_connections 2023-11-27T15:23:24.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.593+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/1288633563 wait complete. 2023-11-27T15:23:24.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.593+0000 7f60cd3cdb80 20 client.15588 trim_cache size 0 max 16384 2023-11-27T15:23:24.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.626+0000 7f60cd3cdb80 1 Processor -- start 2023-11-27T15:23:24.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.626+0000 7f60cd3cdb80 1 -- start start 2023-11-27T15:23:24.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.626+0000 7f60cd3cdb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d0549c00 0x5594d054b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.626+0000 7f60cd3cdb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054a550 0x5594d055ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.626+0000 7f60cd3cdb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d054b110 0x5594d05514f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.626+0000 7f60cd3cdb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d0546180 con 0x5594d054a550 2023-11-27T15:23:24.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.626+0000 7f60cd3cdb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d0545580 con 0x5594d054b110 2023-11-27T15:23:24.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.626+0000 7f60cd3cdb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d05512a0 con 0x5594d0549c00 2023-11-27T15:23:24.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.627+0000 7f60b2ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d0549c00 0x5594d054b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.627+0000 7f60b2ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d0549c00 0x5594d054b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:57994/0 (socket says 172.21.15.133:57994) 2023-11-27T15:23:24.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.627+0000 7f60b2ffd700 1 -- 172.21.15.133:0/2111994631 learned_addr learned my addr 172.21.15.133:0/2111994631 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:24.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.627+0000 7f60b27fc700 1 --2- 172.21.15.133:0/2111994631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054a550 0x5594d055ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.627+0000 7f60b37fe700 1 --2- 172.21.15.133:0/2111994631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d054b110 0x5594d05514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.627+0000 7f60b2ffd700 1 -- 172.21.15.133:0/2111994631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d054b110 msgr2=0x5594d05514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:24.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.627+0000 7f60b2ffd700 1 --2- 172.21.15.133:0/2111994631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d054b110 0x5594d05514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.627+0000 7f60b2ffd700 1 -- 172.21.15.133:0/2111994631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054a550 msgr2=0x5594d055ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:24.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.627+0000 7f60b2ffd700 1 --2- 172.21.15.133:0/2111994631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054a550 0x5594d055ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.627+0000 7f60b2ffd700 1 -- 172.21.15.133:0/2111994631 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594d0460ee0 con 0x5594d0549c00 2023-11-27T15:23:24.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.627+0000 7f60b2ffd700 1 --2- 172.21.15.133:0/2111994631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d0549c00 0x5594d054b4e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f60a400e760 tx=0x7f60a400c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c525251efed790f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:24.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.628+0000 7f609a7fc700 1 -- 172.21.15.133:0/2111994631 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a4020040 con 0x5594d0549c00 2023-11-27T15:23:24.632 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.628+0000 7f609a7fc700 1 -- 172.21.15.133:0/2111994631 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60a4021040 con 0x5594d0549c00 2023-11-27T15:23:24.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.628+0000 7f609a7fc700 1 -- 172.21.15.133:0/2111994631 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a4004290 con 0x5594d0549c00 2023-11-27T15:23:24.633 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.628+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/2111994631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d0549c00 msgr2=0x5594d054b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.628+0000 7f60cd3cdb80 1 --2- 172.21.15.133:0/2111994631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d0549c00 0x5594d054b4e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f60a400e760 tx=0x7f60a400c4b0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.629+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/2111994631 shutdown_connections 2023-11-27T15:23:24.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.629+0000 7f60cd3cdb80 1 --2- 172.21.15.133:0/2111994631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d054b110 0x5594d05514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.629+0000 7f60cd3cdb80 1 --2- 172.21.15.133:0/2111994631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d0549c00 0x5594d054b4e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.629+0000 7f60cd3cdb80 1 --2- 172.21.15.133:0/2111994631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054a550 0x5594d055ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.629+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/2111994631 >> 172.21.15.133:0/2111994631 conn(0x5594d053f360 msgr2=0x5594d05661c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:24.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.629+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/2111994631 shutdown_connections 2023-11-27T15:23:24.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.629+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/2111994631 wait complete. 2023-11-27T15:23:24.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.629+0000 7f60cd3cdb80 1 -- start start 2023-11-27T15:23:24.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.629+0000 7f60cd3cdb80 1 Processor -- start 2023-11-27T15:23:24.634 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.630+0000 7f60cd3cdb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d0549c00 0x5594d05564b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.635 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.630+0000 7f60cd3cdb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054a550 0x5594d05569f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.635 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.630+0000 7f60cd3cdb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d054b110 0x5594d0559a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.635 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.630+0000 7f60cd3cdb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d0559f50 con 0x5594d054a550 2023-11-27T15:23:24.635 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.630+0000 7f60cd3cdb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d055a0c0 con 0x5594d0549c00 2023-11-27T15:23:24.635 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.630+0000 7f60cd3cdb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594d055a280 con 0x5594d054b110 2023-11-27T15:23:24.635 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.630+0000 7f60cd3cdb80 10 client.0 _ll_register_callbacks cb 0x7f60ac0090d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:24.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.630+0000 7f60b2ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d0549c00 0x5594d05564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.630+0000 7f60b37fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d054b110 0x5594d0559a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.630+0000 7f60b27fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054a550 0x5594d05569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.630+0000 7f60b37fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d054b110 0x5594d0559a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:58004/0 (socket says 172.21.15.133:58004) 2023-11-27T15:23:24.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.630+0000 7f60b37fe700 1 -- 172.21.15.133:0/2929423986 learned_addr learned my addr 172.21.15.133:0/2929423986 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:24.636 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.630+0000 7f60b2ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d0549c00 0x5594d05564b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:47890/0 (socket says 172.21.15.133:47890) 2023-11-27T15:23:24.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.631+0000 7f60b37fe700 1 -- 172.21.15.133:0/2929423986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d0549c00 msgr2=0x5594d05564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:24.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.631+0000 7f60b37fe700 1 --2- 172.21.15.133:0/2929423986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d0549c00 0x5594d05564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.631+0000 7f60b37fe700 1 -- 172.21.15.133:0/2929423986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054a550 msgr2=0x5594d05569f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:24.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.631+0000 7f60b37fe700 1 --2- 172.21.15.133:0/2929423986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054a550 0x5594d05569f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.631+0000 7f60b37fe700 1 -- 172.21.15.133:0/2929423986 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5594d031c630 con 0x5594d054b110 2023-11-27T15:23:24.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.631+0000 7f60b27fc700 1 --2- 172.21.15.133:0/2929423986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054a550 0x5594d05569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:24.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.631+0000 7f60b37fe700 1 --2- 172.21.15.133:0/2929423986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d054b110 0x5594d0559a10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f60ac02cc70 tx=0x7f60ac0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c0c8cf6a7c3933c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:24.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.631+0000 7f6092ffd700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:23:24.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.631+0000 7f6092ffd700 1 -- 172.21.15.133:0/2929423986 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac026040 con 0x5594d054b110 2023-11-27T15:23:24.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.631+0000 7f6092ffd700 1 -- 172.21.15.133:0/2929423986 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f60ac005560 con 0x5594d054b110 2023-11-27T15:23:24.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.632+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/2929423986 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60a400ee70 con 0x5594d054b110 2023-11-27T15:23:24.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.632+0000 7f6092ffd700 1 -- 172.21.15.133:0/2929423986 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60ac005720 con 0x5594d054b110 2023-11-27T15:23:24.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.632+0000 7f60cd3cdb80 10 client.25627 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:24.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.632+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/2929423986 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5594d02b1c60 con 0x5594d054b110 2023-11-27T15:23:24.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.632+0000 7f6092ffd700 1 -- 172.21.15.133:0/2929423986 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60a400ee70 con 0x5594d054b110 2023-11-27T15:23:24.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.633+0000 7f6092ffd700 1 -- 172.21.15.133:0/2929423986 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f60ac005880 con 0x5594d054b110 2023-11-27T15:23:24.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.633+0000 7f60cd3cdb80 10 client.25627 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:24.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.633+0000 7f60cd3cdb80 10 client.25627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:24.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.633+0000 7f60cd3cdb80 20 client.25627 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:24.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.633+0000 7f60cd3cdb80 1 --2- 172.21.15.133:0/2929423986 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5594d055f430 0x5594d05b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:24.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.633+0000 7f60cd3cdb80 4 client.25627 mds_command: new command op to 24560 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:23:24.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.633+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/2929423986 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5594d0469660 con 0x5594d055f430 2023-11-27T15:23:24.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.634+0000 7f60b2ffd700 1 --2- 172.21.15.133:0/2929423986 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5594d055f430 0x5594d05b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:24.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.634+0000 7f6092ffd700 10 client.25627 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:24.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.634+0000 7f60b2ffd700 1 --2- 172.21.15.133:0/2929423986 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5594d055f430 0x5594d05b5b20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e1f16065d427466 server_cookie=6efc71c943347930 in_seq=0 out_seq=0 2023-11-27T15:23:24.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.634+0000 7f6092ffd700 1 -- 172.21.15.133:0/2929423986 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5594d0469660 con 0x5594d055f430 2023-11-27T15:23:24.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.634+0000 7f6092ffd700 10 client.25627 handle_command_reply: tid=0 2023-11-27T15:23:24.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.634+0000 7f60cd3cdb80 1 client.25627 shutdown 2023-11-27T15:23:24.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.635+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/2929423986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d054b110 msgr2=0x5594d0559a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.635+0000 7f60cd3cdb80 1 --2- 172.21.15.133:0/2929423986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d054b110 0x5594d0559a10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f60ac02cc70 tx=0x7f60ac0099e0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.635+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/2929423986 shutdown_connections 2023-11-27T15:23:24.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.635+0000 7f60cd3cdb80 1 --2- 172.21.15.133:0/2929423986 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5594d055f430 0x5594d05b5b20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.635+0000 7f60cd3cdb80 1 --2- 172.21.15.133:0/2929423986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5594d054b110 0x5594d0559a10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.635+0000 7f60cd3cdb80 1 --2- 172.21.15.133:0/2929423986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5594d0549c00 0x5594d05564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.635+0000 7f6092ffd700 0 client.25627 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:24.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.635+0000 7f60cd3cdb80 1 --2- 172.21.15.133:0/2929423986 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5594d054a550 0x5594d05569f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.635+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/2929423986 >> 172.21.15.133:0/2929423986 conn(0x5594d053f360 msgr2=0x5594d055b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:24.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.635+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/2929423986 shutdown_connections 2023-11-27T15:23:24.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.635+0000 7f60cd3cdb80 1 -- 172.21.15.133:0/2929423986 wait complete. 2023-11-27T15:23:24.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.635+0000 7f60cd3cdb80 20 client.25627 trim_cache size 0 max 16384 2023-11-27T15:23:24.642 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:23:24.642 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-27T15:23:24.642 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-11-27T15:23:24.642 INFO:teuthology.orchestra.run.smithi133.stdout: "2f111e46-b73b-4444-b4f8-e196a6e85e69": { 2023-11-27T15:23:24.642 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-11-27T15:23:24.642 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "2f111e46-b73b-4444-b4f8-e196a6e85e69", 2023-11-27T15:23:24.642 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-11-27T15:23:24.642 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:23:24.642 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:23:24.642 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:23:24.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.636+0000 7f60b99ff700 1 -- 172.21.15.133:0/3525192350 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f609c0a11d0 msgr2=0x7f609c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.636+0000 7f60b99ff700 1 --2- 172.21.15.133:0/3525192350 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f609c0a11d0 0x7f609c0a3680 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f60ac015000 tx=0x7f60ac00b7d0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.636+0000 7f60b99ff700 1 -- 172.21.15.133:0/3525192350 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60b41014c0 msgr2=0x7f60b4198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:24.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.636+0000 7f60b99ff700 1 --2- 172.21.15.133:0/3525192350 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60b41014c0 0x7f60b4198870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f60a800d980 tx=0x7f60a800dc90 comp rx=0 tx=0).stop 2023-11-27T15:23:24.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.636+0000 7f60b99ff700 1 -- 172.21.15.133:0/3525192350 shutdown_connections 2023-11-27T15:23:24.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.636+0000 7f60b99ff700 1 --2- 172.21.15.133:0/3525192350 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f609c0a11d0 0x7f609c0a3680 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.636+0000 7f60b99ff700 1 --2- 172.21.15.133:0/3525192350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60b4101e50 0x7f60b419f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.636+0000 7f60b99ff700 1 --2- 172.21.15.133:0/3525192350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f60b41002d0 0x7f60b4198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.636+0000 7f60b99ff700 1 --2- 172.21.15.133:0/3525192350 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60b41014c0 0x7f60b4198870 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:24.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.636+0000 7f60b99ff700 1 -- 172.21.15.133:0/3525192350 >> 172.21.15.133:0/3525192350 conn(0x7f60b40fb8b0 msgr2=0x7f60b40fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:24.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.636+0000 7f60b99ff700 1 -- 172.21.15.133:0/3525192350 shutdown_connections 2023-11-27T15:23:24.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:24.636+0000 7f60b99ff700 1 -- 172.21.15.133:0/3525192350 wait complete. 2023-11-27T15:23:24.666 INFO:tasks.cephfs.filesystem:scrub status for tag:2f111e46-b73b-4444-b4f8-e196a6e85e69 - {'path': '/', 'tag': '2f111e46-b73b-4444-b4f8-e196a6e85e69', 'options': 'recursive,force'} 2023-11-27T15:23:25.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:24 smithi177 ceph-mon[131504]: pgmap v363: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-27T15:23:25.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:24 smithi177 ceph-mon[131504]: scrub queued for path: / 2023-11-27T15:23:25.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:24 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:23:25.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:24 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:23:25.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:24 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:23:25.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:24 smithi174 ceph-mon[131110]: pgmap v363: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-27T15:23:25.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:24 smithi174 ceph-mon[131110]: scrub queued for path: / 2023-11-27T15:23:25.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:24 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:23:25.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:24 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:23:25.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:24 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:23:25.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:24 smithi133 ceph-mon[97860]: pgmap v363: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-11-27T15:23:25.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:24 smithi133 ceph-mon[97860]: scrub queued for path: / 2023-11-27T15:23:25.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:24 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:23:25.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:24 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:23:25.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:24 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:23:27.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:26 smithi177 ceph-mon[131504]: pgmap v364: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-27T15:23:27.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:26 smithi177 ceph-mon[131504]: scrub summary: idle 2023-11-27T15:23:27.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:26 smithi174 ceph-mon[131110]: pgmap v364: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-27T15:23:27.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:26 smithi174 ceph-mon[131110]: scrub summary: idle 2023-11-27T15:23:27.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:26 smithi133 ceph-mon[97860]: pgmap v364: 129 pgs: 129 active+clean; 653 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2023-11-27T15:23:27.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:26 smithi133 ceph-mon[97860]: scrub summary: idle 2023-11-27T15:23:29.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:28 smithi133 ceph-mon[97860]: pgmap v365: 129 pgs: 129 active+clean; 662 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-11-27T15:23:29.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:28 smithi177 ceph-mon[131504]: pgmap v365: 129 pgs: 129 active+clean; 662 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-11-27T15:23:29.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:28 smithi174 ceph-mon[131110]: pgmap v365: 129 pgs: 129 active+clean; 662 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2023-11-27T15:23:30.176 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.172+0000 7f4701dff700 1 -- 172.21.15.133:0/1273322747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46fc074b30 msgr2=0x7f46fc072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.172+0000 7f4701dff700 1 --2- 172.21.15.133:0/1273322747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46fc074b30 0x7f46fc072f80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009990 tx=0x7f46ec009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.172+0000 7f4701dff700 1 -- 172.21.15.133:0/1273322747 shutdown_connections 2023-11-27T15:23:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.172+0000 7f4701dff700 1 --2- 172.21.15.133:0/1273322747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46fc10c810 0x7f46fc10ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.172+0000 7f4701dff700 1 --2- 172.21.15.133:0/1273322747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46fc074b30 0x7f46fc072f80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.172+0000 7f4701dff700 1 --2- 172.21.15.133:0/1273322747 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46fc073550 0x7f46fc0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.172+0000 7f4701dff700 1 -- 172.21.15.133:0/1273322747 >> 172.21.15.133:0/1273322747 conn(0x7f46fc0fb840 msgr2=0x7f46fc0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.173+0000 7f4701dff700 1 -- 172.21.15.133:0/1273322747 shutdown_connections 2023-11-27T15:23:30.177 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.173+0000 7f4701dff700 1 -- 172.21.15.133:0/1273322747 wait complete. 2023-11-27T15:23:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.173+0000 7f4701dff700 1 Processor -- start 2023-11-27T15:23:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.174+0000 7f4701dff700 1 -- start start 2023-11-27T15:23:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.174+0000 7f4701dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46fc073550 0x7f46fc19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.174+0000 7f4701dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46fc074b30 0x7f46fc19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.174+0000 7f4701dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46fc10c810 0x7f46fc1a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.179 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.174+0000 7f4701dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc1a41a0 con 0x7f46fc074b30 2023-11-27T15:23:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.174+0000 7f4701dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc1a42e0 con 0x7f46fc10c810 2023-11-27T15:23:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.174+0000 7f4701dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc1a4470 con 0x7f46fc073550 2023-11-27T15:23:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.174+0000 7f46faffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46fc074b30 0x7f46fc19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.174+0000 7f46fb7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46fc073550 0x7f46fc19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.174+0000 7f46faffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46fc074b30 0x7f46fc19cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60974/0 (socket says 172.21.15.133:60974) 2023-11-27T15:23:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.174+0000 7f46faffd700 1 -- 172.21.15.133:0/2563110298 learned_addr learned my addr 172.21.15.133:0/2563110298 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.174+0000 7f46fbfff700 1 --2- 172.21.15.133:0/2563110298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46fc10c810 0x7f46fc1a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.175+0000 7f46faffd700 1 -- 172.21.15.133:0/2563110298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46fc073550 msgr2=0x7f46fc19c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.175+0000 7f46faffd700 1 --2- 172.21.15.133:0/2563110298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46fc073550 0x7f46fc19c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.175+0000 7f46faffd700 1 -- 172.21.15.133:0/2563110298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46fc10c810 msgr2=0x7f46fc1a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.175+0000 7f46faffd700 1 --2- 172.21.15.133:0/2563110298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46fc10c810 0x7f46fc1a3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.175+0000 7f46faffd700 1 -- 172.21.15.133:0/2563110298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ec0095f0 con 0x7f46fc074b30 2023-11-27T15:23:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.175+0000 7f46faffd700 1 --2- 172.21.15.133:0/2563110298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46fc074b30 0x7f46fc19cb60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f46f000e970 tx=0x7f46f000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.175+0000 7f46f8ff9700 1 -- 172.21.15.133:0/2563110298 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0009790 con 0x7f46fc074b30 2023-11-27T15:23:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.175+0000 7f46f8ff9700 1 -- 172.21.15.133:0/2563110298 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46f000c7e0 con 0x7f46fc074b30 2023-11-27T15:23:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.175+0000 7f4701dff700 1 -- 172.21.15.133:0/2563110298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46fc112200 con 0x7f46fc074b30 2023-11-27T15:23:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.175+0000 7f46f8ff9700 1 -- 172.21.15.133:0/2563110298 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0003c70 con 0x7f46fc074b30 2023-11-27T15:23:30.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.175+0000 7f4701dff700 1 -- 172.21.15.133:0/2563110298 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46fc112070 con 0x7f46fc074b30 2023-11-27T15:23:30.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.178+0000 7f46f8ff9700 1 -- 172.21.15.133:0/2563110298 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f46f000cc50 con 0x7f46fc074b30 2023-11-27T15:23:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.180+0000 7f471575ab80 1 Processor -- start 2023-11-27T15:23:30.183 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.180+0000 7f471575ab80 1 -- start start 2023-11-27T15:23:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.180+0000 7f46f8ff9700 1 --2- 172.21.15.133:0/2563110298 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f46e40a1150 0x7f46e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.180+0000 7f471575ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7097496b0 0x55c7098ed870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.180+0000 7f471575ab80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7098eddb0 0x55c7098ee5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.180+0000 7f471575ab80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7098eeb10 0x55c7098eef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.180+0000 7f471575ab80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7097f3000 con 0x55c7098eddb0 2023-11-27T15:23:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.180+0000 7f471575ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c709845840 con 0x55c7097496b0 2023-11-27T15:23:30.185 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.180+0000 7f471575ab80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c70983ecd0 con 0x55c7098eeb10 2023-11-27T15:23:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.180+0000 7f46f8ff9700 1 -- 172.21.15.133:0/2563110298 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f46f0014070 con 0x7f46fc074b30 2023-11-27T15:23:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.180+0000 7f46fb7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7097496b0 0x55c7098ed870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.180+0000 7f46faffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7098eddb0 0x55c7098ee5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.181+0000 7f46fb7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7097496b0 0x55c7098ed870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:47918/0 (socket says 172.21.15.133:47918) 2023-11-27T15:23:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.181+0000 7f46fb7fe700 1 -- 172.21.15.133:0/982262404 learned_addr learned my addr 172.21.15.133:0/982262404 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.181+0000 7f46fbfff700 1 --2- 172.21.15.133:0/2563110298 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f46e40a1150 0x7f46e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.181+0000 7f46fbfff700 1 --2- 172.21.15.133:0/982262404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7098eeb10 0x55c7098eef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.181+0000 7f46fb7fe700 1 -- 172.21.15.133:0/982262404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7098eeb10 msgr2=0x55c7098eef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.181+0000 7f46fb7fe700 1 --2- 172.21.15.133:0/982262404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7098eeb10 0x55c7098eef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.181+0000 7f46fb7fe700 1 -- 172.21.15.133:0/982262404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7098eddb0 msgr2=0x55c7098ee5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.181+0000 7f46fb7fe700 1 --2- 172.21.15.133:0/982262404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7098eddb0 0x55c7098ee5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.181+0000 7f46fb7fe700 1 -- 172.21.15.133:0/982262404 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7096c05d0 con 0x55c7097496b0 2023-11-27T15:23:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.181+0000 7f46fbfff700 1 --2- 172.21.15.133:0/982262404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7098eeb10 0x55c7098eef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.181+0000 7f46fb7fe700 1 --2- 172.21.15.133:0/982262404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7097496b0 0x55c7098ed870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009990 tx=0x7f46ec005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba16e88b83a166e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.181+0000 7f46fbfff700 1 --2- 172.21.15.133:0/2563110298 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f46e40a1150 0x7f46e40a3600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009ec0 tx=0x7f46f4009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.182+0000 7f46e1ffb700 1 -- 172.21.15.133:0/982262404 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec01d070 con 0x55c7097496b0 2023-11-27T15:23:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.182+0000 7f46e1ffb700 1 -- 172.21.15.133:0/982262404 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46ec01e040 con 0x55c7097496b0 2023-11-27T15:23:30.187 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.182+0000 7f46e1ffb700 1 -- 172.21.15.133:0/982262404 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec004980 con 0x55c7097496b0 2023-11-27T15:23:30.190 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.182+0000 7f471575ab80 1 -- 172.21.15.133:0/982262404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7097496b0 msgr2=0x55c7098ed870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:30.190 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.182+0000 7f471575ab80 1 --2- 172.21.15.133:0/982262404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7097496b0 0x55c7098ed870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009990 tx=0x7f46ec005ea0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.190 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.183+0000 7f471575ab80 1 -- 172.21.15.133:0/982262404 shutdown_connections 2023-11-27T15:23:30.190 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.183+0000 7f471575ab80 1 --2- 172.21.15.133:0/982262404 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7098eeb10 0x55c7098eef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.183+0000 7f471575ab80 1 --2- 172.21.15.133:0/982262404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7097496b0 0x55c7098ed870 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.183+0000 7f471575ab80 1 --2- 172.21.15.133:0/982262404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7098eddb0 0x55c7098ee5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.183+0000 7f471575ab80 1 -- 172.21.15.133:0/982262404 >> 172.21.15.133:0/982262404 conn(0x55c7098e2f50 msgr2=0x55c7098e8100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.183+0000 7f471575ab80 1 -- 172.21.15.133:0/982262404 shutdown_connections 2023-11-27T15:23:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.183+0000 7f471575ab80 1 -- 172.21.15.133:0/982262404 wait complete. 2023-11-27T15:23:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.183+0000 7f471575ab80 1 -- start start 2023-11-27T15:23:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.184+0000 7f471575ab80 1 Processor -- start 2023-11-27T15:23:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.184+0000 7f471575ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7097496b0 0x55c7098f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.184+0000 7f471575ab80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7098eddb0 0x55c7098f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.184+0000 7f471575ab80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7098eeb10 0x55c7098fe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.184+0000 7f471575ab80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7097e0880 con 0x55c7098eeb10 2023-11-27T15:23:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.184+0000 7f471575ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7097e0520 con 0x55c7097496b0 2023-11-27T15:23:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.184+0000 7f471575ab80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7096c28c0 con 0x55c7098eddb0 2023-11-27T15:23:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.184+0000 7f471575ab80 10 client.0 _ll_register_callbacks cb 0x55c7097fe720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.185+0000 7f46fbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7098eeb10 0x55c7098fe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.185+0000 7f46fb7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7097496b0 0x55c7098f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.185+0000 7f46fbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7098eeb10 0x55c7098fe120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60994/0 (socket says 172.21.15.133:60994) 2023-11-27T15:23:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.185+0000 7f46fbfff700 1 -- 172.21.15.133:0/3647360689 learned_addr learned my addr 172.21.15.133:0/3647360689 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.185+0000 7f46faffd700 1 --2- 172.21.15.133:0/3647360689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7098eddb0 0x55c7098f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.185+0000 7f46fb7fe700 1 -- 172.21.15.133:0/3647360689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7098eddb0 msgr2=0x55c7098f70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.185+0000 7f46fb7fe700 1 --2- 172.21.15.133:0/3647360689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7098eddb0 0x55c7098f70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.185+0000 7f46fb7fe700 1 -- 172.21.15.133:0/3647360689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7098eeb10 msgr2=0x55c7098fe120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:30.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.185+0000 7f46fb7fe700 1 --2- 172.21.15.133:0/3647360689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7098eeb10 0x55c7098fe120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.185+0000 7f46fb7fe700 1 -- 172.21.15.133:0/3647360689 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46fc10f5a0 con 0x55c7097496b0 2023-11-27T15:23:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.185+0000 7f46fb7fe700 1 --2- 172.21.15.133:0/3647360689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7097496b0 0x55c7098f6ba0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0098e0 tx=0x7f46ec022630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b20bdd1b09957ea0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.185+0000 7f46daffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:23:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.186+0000 7f46daffd700 1 -- 172.21.15.133:0/3647360689 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec01d030 con 0x55c7097496b0 2023-11-27T15:23:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.186+0000 7f46daffd700 1 -- 172.21.15.133:0/3647360689 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f46ec0058e0 con 0x55c7097496b0 2023-11-27T15:23:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.186+0000 7f471575ab80 1 -- 172.21.15.133:0/3647360689 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7098e0710 con 0x55c7097496b0 2023-11-27T15:23:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.186+0000 7f46daffd700 1 -- 172.21.15.133:0/3647360689 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46ec022c70 con 0x55c7097496b0 2023-11-27T15:23:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.186+0000 7f471575ab80 10 client.25649 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.186+0000 7f471575ab80 1 -- 172.21.15.133:0/3647360689 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c709805130 con 0x55c7097496b0 2023-11-27T15:23:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.186+0000 7f46daffd700 1 -- 172.21.15.133:0/3647360689 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c7098e0710 con 0x55c7097496b0 2023-11-27T15:23:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.186+0000 7f46daffd700 1 -- 172.21.15.133:0/3647360689 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f46ec02c400 con 0x55c7097496b0 2023-11-27T15:23:30.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.187+0000 7f471575ab80 10 client.25649 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.187+0000 7f471575ab80 10 client.25649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.187+0000 7f471575ab80 20 client.25649 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.187+0000 7f471575ab80 1 --2- 172.21.15.133:0/3647360689 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c709902450 0x55c709904830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.187+0000 7f471575ab80 4 client.25649 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.187+0000 7f471575ab80 1 -- 172.21.15.133:0/3647360689 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c709801040 con 0x55c709902450 2023-11-27T15:23:30.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.187+0000 7f46faffd700 1 --2- 172.21.15.133:0/3647360689 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c709902450 0x55c709904830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.187+0000 7f46faffd700 1 --2- 172.21.15.133:0/3647360689 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c709902450 0x55c709904830 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65c975a68371e648 server_cookie=6cc6479f67f90a6d in_seq=0 out_seq=0 2023-11-27T15:23:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.187+0000 7f46daffd700 10 client.25649 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.188+0000 7f46daffd700 1 -- 172.21.15.133:0/3647360689 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c709801040 con 0x55c709902450 2023-11-27T15:23:30.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.188+0000 7f46daffd700 10 client.25649 handle_command_reply: tid=0 2023-11-27T15:23:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.188+0000 7f471575ab80 1 client.25649 shutdown 2023-11-27T15:23:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.188+0000 7f471575ab80 1 -- 172.21.15.133:0/3647360689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7097496b0 msgr2=0x55c7098f6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.188+0000 7f471575ab80 1 --2- 172.21.15.133:0/3647360689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7097496b0 0x55c7098f6ba0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0098e0 tx=0x7f46ec022630 comp rx=0 tx=0).stop 2023-11-27T15:23:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.188+0000 7f471575ab80 1 -- 172.21.15.133:0/3647360689 shutdown_connections 2023-11-27T15:23:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.188+0000 7f471575ab80 1 --2- 172.21.15.133:0/3647360689 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c709902450 0x55c709904830 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.188+0000 7f471575ab80 1 --2- 172.21.15.133:0/3647360689 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7098eeb10 0x55c7098fe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.188+0000 7f471575ab80 1 --2- 172.21.15.133:0/3647360689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7097496b0 0x55c7098f6ba0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.188+0000 7f471575ab80 1 --2- 172.21.15.133:0/3647360689 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7098eddb0 0x55c7098f70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.188+0000 7f46daffd700 0 client.25649 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.188+0000 7f471575ab80 1 -- 172.21.15.133:0/3647360689 >> 172.21.15.133:0/3647360689 conn(0x55c7098e2f50 msgr2=0x55c7098e77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.188+0000 7f471575ab80 1 -- 172.21.15.133:0/3647360689 shutdown_connections 2023-11-27T15:23:30.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.189+0000 7f471575ab80 1 -- 172.21.15.133:0/3647360689 wait complete. 2023-11-27T15:23:30.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.189+0000 7f471575ab80 20 client.25649 trim_cache size 0 max 16384 2023-11-27T15:23:30.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.204+0000 7f471575ab80 1 Processor -- start 2023-11-27T15:23:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.204+0000 7f471575ab80 1 -- start start 2023-11-27T15:23:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.204+0000 7f471575ab80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7097496b0 0x55c709909180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.204+0000 7f471575ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7098eddb0 0x55c7098fe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.204+0000 7f471575ab80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7098eeb10 0x55c7098f1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.204+0000 7f471575ab80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7098f4ba0 con 0x55c7098eeb10 2023-11-27T15:23:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.204+0000 7f471575ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7098f2490 con 0x55c7098eddb0 2023-11-27T15:23:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.204+0000 7f471575ab80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7099096c0 con 0x55c7097496b0 2023-11-27T15:23:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.205+0000 7f46fbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7098eeb10 0x55c7098f1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.205+0000 7f46faffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7098eddb0 0x55c7098fe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.205+0000 7f46faffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7098eddb0 0x55c7098fe0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:47934/0 (socket says 172.21.15.133:47934) 2023-11-27T15:23:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.205+0000 7f46fbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7098eeb10 0x55c7098f1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32774/0 (socket says 172.21.15.133:32774) 2023-11-27T15:23:30.209 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.205+0000 7f46faffd700 1 -- 172.21.15.133:0/1334940752 learned_addr learned my addr 172.21.15.133:0/1334940752 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.205+0000 7f46fb7fe700 1 --2- 172.21.15.133:0/1334940752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7097496b0 0x55c709909180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.205+0000 7f46faffd700 1 -- 172.21.15.133:0/1334940752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7097496b0 msgr2=0x55c709909180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.205+0000 7f46faffd700 1 --2- 172.21.15.133:0/1334940752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7097496b0 0x55c709909180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.205+0000 7f46faffd700 1 -- 172.21.15.133:0/1334940752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7098eeb10 msgr2=0x55c7098f1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.205+0000 7f46faffd700 1 --2- 172.21.15.133:0/1334940752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7098eeb10 0x55c7098f1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.210 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.205+0000 7f46faffd700 1 -- 172.21.15.133:0/1334940752 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c709804e80 con 0x55c7098eddb0 2023-11-27T15:23:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.205+0000 7f46faffd700 1 --2- 172.21.15.133:0/1334940752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7098eddb0 0x55c7098fe0e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f46f00ca5a0 tx=0x7f46f0096320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67d55b2db1565667 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.206+0000 7f46e27fc700 1 -- 172.21.15.133:0/1334940752 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0099eb0 con 0x55c7098eddb0 2023-11-27T15:23:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.206+0000 7f46e27fc700 1 -- 172.21.15.133:0/1334940752 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46f009dee0 con 0x55c7098eddb0 2023-11-27T15:23:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.206+0000 7f46e27fc700 1 -- 172.21.15.133:0/1334940752 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f00802f0 con 0x55c7098eddb0 2023-11-27T15:23:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.206+0000 7f471575ab80 1 -- 172.21.15.133:0/1334940752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7098eddb0 msgr2=0x55c7098fe0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.206+0000 7f471575ab80 1 --2- 172.21.15.133:0/1334940752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7098eddb0 0x55c7098fe0e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f46f00ca5a0 tx=0x7f46f0096320 comp rx=0 tx=0).stop 2023-11-27T15:23:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.206+0000 7f471575ab80 1 -- 172.21.15.133:0/1334940752 shutdown_connections 2023-11-27T15:23:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.206+0000 7f471575ab80 1 --2- 172.21.15.133:0/1334940752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7098eeb10 0x55c7098f1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.206+0000 7f471575ab80 1 --2- 172.21.15.133:0/1334940752 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7097496b0 0x55c709909180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.212 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.206+0000 7f471575ab80 1 --2- 172.21.15.133:0/1334940752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7098eddb0 0x55c7098fe0e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.206+0000 7f471575ab80 1 -- 172.21.15.133:0/1334940752 >> 172.21.15.133:0/1334940752 conn(0x55c7098e2f50 msgr2=0x55c7098e8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.206+0000 7f471575ab80 1 -- 172.21.15.133:0/1334940752 shutdown_connections 2023-11-27T15:23:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.207+0000 7f471575ab80 1 -- 172.21.15.133:0/1334940752 wait complete. 2023-11-27T15:23:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.207+0000 7f471575ab80 1 -- start start 2023-11-27T15:23:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.207+0000 7f471575ab80 1 Processor -- start 2023-11-27T15:23:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.208+0000 7f471575ab80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7097496b0 0x55c7098fa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.208+0000 7f471575ab80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7098eddb0 0x55c7098fa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.208+0000 7f471575ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7098eeb10 0x55c7098fd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.208+0000 7f471575ab80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7098fdc60 con 0x55c7097496b0 2023-11-27T15:23:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.208+0000 7f471575ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7098fddd0 con 0x55c7098eeb10 2023-11-27T15:23:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.208+0000 7f471575ab80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7098fdf90 con 0x55c7098eddb0 2023-11-27T15:23:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.208+0000 7f471575ab80 10 client.0 _ll_register_callbacks cb 0x7f46f0095ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:30.213 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.208+0000 7f46fb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7097496b0 0x55c7098fa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.208+0000 7f46faffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7098eddb0 0x55c7098fa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.208+0000 7f46faffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7098eddb0 0x55c7098fa700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:58032/0 (socket says 172.21.15.133:58032) 2023-11-27T15:23:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.208+0000 7f46fb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7097496b0 0x55c7098fa1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:32786/0 (socket says 172.21.15.133:32786) 2023-11-27T15:23:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.208+0000 7f46faffd700 1 -- 172.21.15.133:0/2584219035 learned_addr learned my addr 172.21.15.133:0/2584219035 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.208+0000 7f46fbfff700 1 --2- 172.21.15.133:0/2584219035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7098eeb10 0x55c7098fd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.209+0000 7f46faffd700 1 -- 172.21.15.133:0/2584219035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7098eeb10 msgr2=0x55c7098fd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.209+0000 7f46faffd700 1 --2- 172.21.15.133:0/2584219035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7098eeb10 0x55c7098fd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.209+0000 7f46faffd700 1 -- 172.21.15.133:0/2584219035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7097496b0 msgr2=0x55c7098fa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.209+0000 7f46faffd700 1 --2- 172.21.15.133:0/2584219035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7097496b0 0x55c7098fa1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.209+0000 7f46faffd700 1 -- 172.21.15.133:0/2584219035 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46fc112070 con 0x55c7098eddb0 2023-11-27T15:23:30.214 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.209+0000 7f46fbfff700 1 --2- 172.21.15.133:0/2584219035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7098eeb10 0x55c7098fd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.209+0000 7f46faffd700 1 --2- 172.21.15.133:0/2584219035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7098eddb0 0x55c7098fa700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f46f000efd0 tx=0x7f46f00a2440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c83258e6220ec253 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.209+0000 7f46daffd700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:23:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.209+0000 7f46daffd700 1 -- 172.21.15.133:0/2584219035 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f007c070 con 0x55c7098eddb0 2023-11-27T15:23:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.209+0000 7f471575ab80 1 -- 172.21.15.133:0/2584219035 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46ec01e040 con 0x55c7098eddb0 2023-11-27T15:23:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.209+0000 7f46daffd700 1 -- 172.21.15.133:0/2584219035 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f46f009a110 con 0x55c7098eddb0 2023-11-27T15:23:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.209+0000 7f46daffd700 1 -- 172.21.15.133:0/2584219035 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46f009a2d0 con 0x55c7098eddb0 2023-11-27T15:23:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.209+0000 7f471575ab80 10 client.25645 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.209+0000 7f471575ab80 1 -- 172.21.15.133:0/2584219035 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c709655c00 con 0x55c7098eddb0 2023-11-27T15:23:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.209+0000 7f46daffd700 1 -- 172.21.15.133:0/2584219035 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46ec01e040 con 0x55c7098eddb0 2023-11-27T15:23:30.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.210+0000 7f46daffd700 1 -- 172.21.15.133:0/2584219035 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f46f00adbb0 con 0x55c7098eddb0 2023-11-27T15:23:30.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.210+0000 7f471575ab80 10 client.25645 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:30.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.210+0000 7f471575ab80 10 client.25645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:30.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.210+0000 7f471575ab80 20 client.25645 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:30.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.210+0000 7f471575ab80 1 --2- 172.21.15.133:0/2584219035 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c709957310 0x55c709959720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:30.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.210+0000 7f471575ab80 4 client.25645 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:23:30.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.210+0000 7f471575ab80 1 -- 172.21.15.133:0/2584219035 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c70980d600 con 0x55c709957310 2023-11-27T15:23:30.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.210+0000 7f46fb7fe700 1 --2- 172.21.15.133:0/2584219035 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c709957310 0x55c709959720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:30.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.210+0000 7f46daffd700 10 client.25645 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:30.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.210+0000 7f46fb7fe700 1 --2- 172.21.15.133:0/2584219035 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c709957310 0x55c709959720 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e12ccf1b04f2265 server_cookie=b3325c6411d8a5e8 in_seq=0 out_seq=0 2023-11-27T15:23:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.218+0000 7f46daffd700 1 -- 172.21.15.133:0/2584219035 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c70980d600 con 0x55c709957310 2023-11-27T15:23:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.218+0000 7f46daffd700 10 client.25645 handle_command_reply: tid=0 2023-11-27T15:23:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.218+0000 7f471575ab80 1 client.25645 shutdown 2023-11-27T15:23:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.218+0000 7f471575ab80 1 -- 172.21.15.133:0/2584219035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7098eddb0 msgr2=0x55c7098fa700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.218+0000 7f471575ab80 1 --2- 172.21.15.133:0/2584219035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7098eddb0 0x55c7098fa700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f46f000efd0 tx=0x7f46f00a2440 comp rx=0 tx=0).stop 2023-11-27T15:23:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.218+0000 7f471575ab80 1 -- 172.21.15.133:0/2584219035 shutdown_connections 2023-11-27T15:23:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.218+0000 7f471575ab80 1 --2- 172.21.15.133:0/2584219035 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c709957310 0x55c709959720 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.218+0000 7f471575ab80 1 --2- 172.21.15.133:0/2584219035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7098eeb10 0x55c7098fd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.218+0000 7f471575ab80 1 --2- 172.21.15.133:0/2584219035 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c7097496b0 0x55c7098fa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.218+0000 7f471575ab80 1 --2- 172.21.15.133:0/2584219035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c7098eddb0 0x55c7098fa700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.218+0000 7f46daffd700 0 client.25645 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.218+0000 7f471575ab80 1 -- 172.21.15.133:0/2584219035 >> 172.21.15.133:0/2584219035 conn(0x55c7098e2f50 msgr2=0x55c7098f1920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.218+0000 7f471575ab80 1 -- 172.21.15.133:0/2584219035 shutdown_connections 2023-11-27T15:23:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.218+0000 7f471575ab80 1 -- 172.21.15.133:0/2584219035 wait complete. 2023-11-27T15:23:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.218+0000 7f471575ab80 20 client.25645 trim_cache size 0 max 16384 2023-11-27T15:23:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:23:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:23:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:23:30.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:23:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.219+0000 7f4701dff700 1 -- 172.21.15.133:0/2563110298 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f46e40a1150 msgr2=0x7f46e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.219+0000 7f4701dff700 1 --2- 172.21.15.133:0/2563110298 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f46e40a1150 0x7f46e40a3600 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009ec0 tx=0x7f46f4009920 comp rx=0 tx=0).stop 2023-11-27T15:23:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.219+0000 7f4701dff700 1 -- 172.21.15.133:0/2563110298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46fc074b30 msgr2=0x7f46fc19cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.219+0000 7f4701dff700 1 --2- 172.21.15.133:0/2563110298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46fc074b30 0x7f46fc19cb60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f46f000e970 tx=0x7f46f000ec80 comp rx=0 tx=0).stop 2023-11-27T15:23:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.219+0000 7f4701dff700 1 -- 172.21.15.133:0/2563110298 shutdown_connections 2023-11-27T15:23:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.219+0000 7f4701dff700 1 --2- 172.21.15.133:0/2563110298 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f46e40a1150 0x7f46e40a3600 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.219+0000 7f4701dff700 1 --2- 172.21.15.133:0/2563110298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46fc10c810 0x7f46fc1a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.219+0000 7f4701dff700 1 --2- 172.21.15.133:0/2563110298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f46fc073550 0x7f46fc19c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.219+0000 7f4701dff700 1 --2- 172.21.15.133:0/2563110298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46fc074b30 0x7f46fc19cb60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.219+0000 7f4701dff700 1 -- 172.21.15.133:0/2563110298 >> 172.21.15.133:0/2563110298 conn(0x7f46fc0fb840 msgr2=0x7f46fc0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.219+0000 7f4701dff700 1 -- 172.21.15.133:0/2563110298 shutdown_connections 2023-11-27T15:23:30.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:30.219+0000 7f4701dff700 1 -- 172.21.15.133:0/2563110298 wait complete. 2023-11-27T15:23:31.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:30 smithi177 ceph-mon[131504]: pgmap v366: 129 pgs: 129 active+clean; 662 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-11-27T15:23:31.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:30 smithi174 ceph-mon[131110]: pgmap v366: 129 pgs: 129 active+clean; 662 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-11-27T15:23:31.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:30 smithi133 ceph-mon[97860]: pgmap v366: 129 pgs: 129 active+clean; 662 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-11-27T15:23:32.432 INFO:tasks.workunit.client.0.smithi133.stderr:Updating files: 74% (8952/12072) Updating files: 75% (9054/12072) Updating files: 76% (9175/12072) Updating files: 77% (9296/12072) Updating files: 78% (9417/12072) Updating files: 79% (9537/12072) Updating files: 80% (9658/12072) Updating files: 81% (9779/12072) Updating files: 82% (9900/12072) Updating files: 83% (10020/12072) Updating files: 84% (10141/12072) Updating files: 85% (10262/12072) Updating files: 86% (10382/12072) Updating files: 87% (10503/12072) Updating files: 88% (10624/12072) Updating files: 89% (10745/12072) Updating files: 90% (10865/12072) Updating files: 91% (10986/12072) Updating files: 92% (11107/12072) Updating files: 93% (11227/12072) Updating files: 94% (11348/12072) Updating files: 95% (11469/12072) Updating files: 96% (11590/12072) Updating files: 97% (11710/12072) Updating files: 98% (11831/12072) Updating files: 99% (11952/12072) Updating files: 100% (12072/12072) Updating files: 100% (12072/12072), done. 2023-11-27T15:23:33.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:32 smithi177 ceph-mon[131504]: pgmap v367: 129 pgs: 129 active+clean; 663 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2023-11-27T15:23:33.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:32 smithi174 ceph-mon[131110]: pgmap v367: 129 pgs: 129 active+clean; 663 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2023-11-27T15:23:33.244 INFO:tasks.workunit.client.0.smithi133.stderr:Note: switching to '5e4a743ad7fd1422ebd6c4dc950a21a303279207'. 2023-11-27T15:23:33.244 INFO:tasks.workunit.client.0.smithi133.stderr: 2023-11-27T15:23:33.244 INFO:tasks.workunit.client.0.smithi133.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-11-27T15:23:33.244 INFO:tasks.workunit.client.0.smithi133.stderr:changes and commit them, and you can discard any commits you make in this 2023-11-27T15:23:33.244 INFO:tasks.workunit.client.0.smithi133.stderr:state without impacting any branches by switching back to a branch. 2023-11-27T15:23:33.244 INFO:tasks.workunit.client.0.smithi133.stderr: 2023-11-27T15:23:33.244 INFO:tasks.workunit.client.0.smithi133.stderr:If you want to create a new branch to retain commits you create, you may 2023-11-27T15:23:33.244 INFO:tasks.workunit.client.0.smithi133.stderr:do so (now or later) by using -c with the switch command. Example: 2023-11-27T15:23:33.244 INFO:tasks.workunit.client.0.smithi133.stderr: 2023-11-27T15:23:33.245 INFO:tasks.workunit.client.0.smithi133.stderr: git switch -c 2023-11-27T15:23:33.245 INFO:tasks.workunit.client.0.smithi133.stderr: 2023-11-27T15:23:33.245 INFO:tasks.workunit.client.0.smithi133.stderr:Or undo this operation with: 2023-11-27T15:23:33.245 INFO:tasks.workunit.client.0.smithi133.stderr: 2023-11-27T15:23:33.245 INFO:tasks.workunit.client.0.smithi133.stderr: git switch - 2023-11-27T15:23:33.245 INFO:tasks.workunit.client.0.smithi133.stderr: 2023-11-27T15:23:33.245 INFO:tasks.workunit.client.0.smithi133.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-11-27T15:23:33.245 INFO:tasks.workunit.client.0.smithi133.stderr: 2023-11-27T15:23:33.245 INFO:tasks.workunit.client.0.smithi133.stderr:HEAD is now at 5e4a743ad7f doc/cephfs: Update docs to match remove functionality and respective vxattrs 2023-11-27T15:23:33.254 DEBUG:teuthology.orchestra.run.smithi133:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-11-27T15:23:33.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:32 smithi133 ceph-mon[97860]: pgmap v367: 129 pgs: 129 active+clean; 663 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2023-11-27T15:23:33.305 INFO:tasks.workunit.client.0.smithi133.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-11-27T15:23:33.310 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-11-27T15:23:33.310 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-11-27T15:23:33.900 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-11-27T15:23:33.981 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-11-27T15:23:34.046 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-11-27T15:23:34.050 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-11-27T15:23:34.050 INFO:tasks.workunit.client.0.smithi133.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-11-27T15:23:34.114 INFO:tasks.workunit.client.0.smithi133.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-11-27T15:23:34.121 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-11-27T15:23:34.121 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-11-27T15:23:34.204 INFO:tasks.workunit:Running workunits matching fs/misc on client.0... 2023-11-27T15:23:34.205 INFO:tasks.workunit:Running workunit fs/misc/acl.sh... 2023-11-27T15:23:34.206 DEBUG:teuthology.orchestra.run.smithi133:workunit test fs/misc/acl.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/acl.sh 2023-11-27T15:23:34.301 INFO:tasks.workunit.client.0.smithi133.stderr:+ set -e 2023-11-27T15:23:34.301 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir -p testdir 2023-11-27T15:23:34.305 INFO:tasks.workunit.client.0.smithi133.stderr:+ cd testdir 2023-11-27T15:23:34.305 INFO:tasks.workunit.client.0.smithi133.stderr:+ set +e 2023-11-27T15:23:34.305 INFO:tasks.workunit.client.0.smithi133.stderr:+ setfacl -d -m u:nobody:rw . 2023-11-27T15:23:34.332 INFO:tasks.workunit.client.0.smithi133.stderr:+ test 0 '!=' 0 2023-11-27T15:23:34.332 INFO:tasks.workunit.client.0.smithi133.stderr:+ set -e 2023-11-27T15:23:34.332 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=0 2023-11-27T15:23:34.332 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 0 -lt 100 ']' 2023-11-27T15:23:34.332 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 0 + 1 2023-11-27T15:23:34.334 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=1 2023-11-27T15:23:34.334 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:34.339 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:34.339 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:34.345 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:34.346 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:34.346 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:35.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:34 smithi177 ceph-mon[131504]: pgmap v368: 129 pgs: 129 active+clean; 663 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-27T15:23:35.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:34 smithi174 ceph-mon[131110]: pgmap v368: 129 pgs: 129 active+clean; 663 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-27T15:23:35.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:34 smithi133 ceph-mon[97860]: pgmap v368: 129 pgs: 129 active+clean; 663 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-11-27T15:23:37.063 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:37.063 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:37.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:36 smithi177 ceph-mon[131504]: pgmap v369: 129 pgs: 129 active+clean; 663 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2023-11-27T15:23:37.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:36 smithi174 ceph-mon[131110]: pgmap v369: 129 pgs: 129 active+clean; 663 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2023-11-27T15:23:37.240 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:37.240 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:37.255 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:37.255 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:37.255 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 1 -lt 100 ']' 2023-11-27T15:23:37.256 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 1 + 1 2023-11-27T15:23:37.274 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=2 2023-11-27T15:23:37.274 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:37.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:36 smithi133 ceph-mon[97860]: pgmap v369: 129 pgs: 129 active+clean; 663 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2023-11-27T15:23:37.310 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:37.311 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:37.313 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:37.314 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:37.314 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:37.829 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:37.829 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:37.922 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:37.922 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:37.938 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:37.938 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:37.938 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -lt 100 ']' 2023-11-27T15:23:37.938 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 2 + 1 2023-11-27T15:23:37.996 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=3 2023-11-27T15:23:37.996 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:38.048 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:38.048 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:38.050 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:38.051 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:38.051 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:38.462 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:38.462 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:38.569 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:38.569 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:38.588 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:38.588 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:38.588 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 3 -lt 100 ']' 2023-11-27T15:23:38.588 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 3 + 1 2023-11-27T15:23:38.607 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=4 2023-11-27T15:23:38.607 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:38.637 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:38.637 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:38.640 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:38.640 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:38.641 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:39.024 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:38 smithi133 ceph-mon[97860]: pgmap v370: 129 pgs: 129 active+clean; 676 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2023-11-27T15:23:39.086 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:39.087 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:39.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:38 smithi177 ceph-mon[131504]: pgmap v370: 129 pgs: 129 active+clean; 676 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2023-11-27T15:23:39.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:38 smithi174 ceph-mon[131110]: pgmap v370: 129 pgs: 129 active+clean; 676 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.8 KiB/s wr, 9 op/s 2023-11-27T15:23:39.264 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:39.264 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:39.278 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:39.278 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:39.278 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 4 -lt 100 ']' 2023-11-27T15:23:39.278 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 4 + 1 2023-11-27T15:23:39.299 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=5 2023-11-27T15:23:39.299 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:39.338 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:39.339 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:39.341 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:39.342 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:39.342 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:39.803 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:39.804 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:39.940 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:39.940 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:39.954 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:39.954 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:39.954 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 5 -lt 100 ']' 2023-11-27T15:23:39.954 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 5 + 1 2023-11-27T15:23:39.974 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=6 2023-11-27T15:23:39.974 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:40.003 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:40.003 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:40.006 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:40.006 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:40.007 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:40.645 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:40.645 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:40.781 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:40.781 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:40.811 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:40.811 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:40.812 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 6 -lt 100 ']' 2023-11-27T15:23:40.812 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 6 + 1 2023-11-27T15:23:40.832 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=7 2023-11-27T15:23:40.832 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:40.861 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:40.861 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:40.864 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:40.865 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:40.865 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:41.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:40 smithi177 ceph-mon[131504]: pgmap v371: 129 pgs: 129 active+clean; 676 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2023-11-27T15:23:41.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:40 smithi174 ceph-mon[131110]: pgmap v371: 129 pgs: 129 active+clean; 676 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2023-11-27T15:23:41.153 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:40 smithi133 ceph-mon[97860]: pgmap v371: 129 pgs: 129 active+clean; 676 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2023-11-27T15:23:41.361 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:41.361 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:41.464 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:41.464 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:41.487 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:41.487 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:41.487 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 7 -lt 100 ']' 2023-11-27T15:23:41.487 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 7 + 1 2023-11-27T15:23:41.506 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=8 2023-11-27T15:23:41.507 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:41.541 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:41.542 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:41.544 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:41.545 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:41.545 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:42.070 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:42.070 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:42.171 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:42.171 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:42.203 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:42.204 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:42.204 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 8 -lt 100 ']' 2023-11-27T15:23:42.204 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 8 + 1 2023-11-27T15:23:42.221 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=9 2023-11-27T15:23:42.221 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:42.266 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:42.267 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:42.269 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:42.270 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:42.271 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:42.795 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:42.795 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:42.906 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:42.906 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:42.919 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:42.920 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:42.920 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 9 -lt 100 ']' 2023-11-27T15:23:42.920 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 9 + 1 2023-11-27T15:23:42.940 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=10 2023-11-27T15:23:42.940 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:42.968 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:42.969 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:42.971 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:42.971 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:42.972 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:43.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:42 smithi177 ceph-mon[131504]: pgmap v372: 129 pgs: 129 active+clean; 676 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2023-11-27T15:23:43.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:42 smithi174 ceph-mon[131110]: pgmap v372: 129 pgs: 129 active+clean; 676 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2023-11-27T15:23:43.250 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:42 smithi133 ceph-mon[97860]: pgmap v372: 129 pgs: 129 active+clean; 676 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2023-11-27T15:23:43.436 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:43.436 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:43.573 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:43.573 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:43.603 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:43.603 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:43.603 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 10 -lt 100 ']' 2023-11-27T15:23:43.603 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 10 + 1 2023-11-27T15:23:43.645 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=11 2023-11-27T15:23:43.645 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:43.678 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:43.678 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:43.680 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:43.681 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:43.681 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:44.492 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:44.492 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:44.713 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:44.714 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:44.736 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:44.737 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:44.737 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 11 -lt 100 ']' 2023-11-27T15:23:44.737 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 11 + 1 2023-11-27T15:23:44.773 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=12 2023-11-27T15:23:44.773 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:44.803 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:44.803 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:44.805 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:44.806 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:44.807 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:45.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:44 smithi177 ceph-mon[131504]: pgmap v373: 129 pgs: 129 active+clean; 676 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-11-27T15:23:45.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:44 smithi174 ceph-mon[131110]: pgmap v373: 129 pgs: 129 active+clean; 676 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-11-27T15:23:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.215+0000 7f3ff03ff700 1 -- 172.21.15.133:0/1058445613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fe8103670 msgr2=0x7f3fe8103ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.215+0000 7f3ff03ff700 1 --2- 172.21.15.133:0/1058445613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fe8103670 0x7f3fe8103ac0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009990 tx=0x7f3fd8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.215+0000 7f3ff03ff700 1 -- 172.21.15.133:0/1058445613 shutdown_connections 2023-11-27T15:23:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.215+0000 7f3ff03ff700 1 --2- 172.21.15.133:0/1058445613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fe8104000 0x7f3fe810a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.215+0000 7f3ff03ff700 1 --2- 172.21.15.133:0/1058445613 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fe8102480 0x7f3fe8102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.215+0000 7f3ff03ff700 1 --2- 172.21.15.133:0/1058445613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fe8103670 0x7f3fe8103ac0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.215+0000 7f3ff03ff700 1 -- 172.21.15.133:0/1058445613 >> 172.21.15.133:0/1058445613 conn(0x7f3fe80fda80 msgr2=0x7f3fe80ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.215+0000 7f3ff03ff700 1 -- 172.21.15.133:0/1058445613 shutdown_connections 2023-11-27T15:23:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.215+0000 7f3ff03ff700 1 -- 172.21.15.133:0/1058445613 wait complete. 2023-11-27T15:23:45.221 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:44 smithi133 ceph-mon[97860]: pgmap v373: 129 pgs: 129 active+clean; 676 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-11-27T15:23:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.217+0000 7f3ff03ff700 1 Processor -- start 2023-11-27T15:23:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.217+0000 7f3ff03ff700 1 -- start start 2023-11-27T15:23:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.217+0000 7f3ff03ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fe8102480 0x7f3fe8198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.217+0000 7f3ff03ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fe8103670 0x7f3fe8198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.217+0000 7f3ff03ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fe8104000 0x7f3fe819f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.217+0000 7f3ff03ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe819fdc0 con 0x7f3fe8104000 2023-11-27T15:23:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.217+0000 7f3ff03ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe819ff00 con 0x7f3fe8102480 2023-11-27T15:23:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.217+0000 7f3ff03ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe81a0090 con 0x7f3fe8103670 2023-11-27T15:23:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.218+0000 7f3fee19b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fe8102480 0x7f3fe8198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.218+0000 7f3fee99c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fe8104000 0x7f3fe819f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.218+0000 7f3fee19b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fe8102480 0x7f3fe8198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:57422/0 (socket says 172.21.15.133:57422) 2023-11-27T15:23:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.218+0000 7f3fee19b700 1 -- 172.21.15.133:0/3556818651 learned_addr learned my addr 172.21.15.133:0/3556818651 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.218+0000 7f3fed99a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fe8103670 0x7f3fe8198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.218+0000 7f3fee19b700 1 -- 172.21.15.133:0/3556818651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fe8103670 msgr2=0x7f3fe8198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.218+0000 7f3fee19b700 1 --2- 172.21.15.133:0/3556818651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fe8103670 0x7f3fe8198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.218+0000 7f3fee19b700 1 -- 172.21.15.133:0/3556818651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fe8104000 msgr2=0x7f3fe819f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.218+0000 7f3fee19b700 1 --2- 172.21.15.133:0/3556818651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fe8104000 0x7f3fe819f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.218+0000 7f3fee19b700 1 -- 172.21.15.133:0/3556818651 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd80095f0 con 0x7f3fe8102480 2023-11-27T15:23:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.218+0000 7f3fee19b700 1 --2- 172.21.15.133:0/3556818651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fe8102480 0x7f3fe8198240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400e970 tx=0x7f3fe400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.219+0000 7f3fdf7fe700 1 -- 172.21.15.133:0/3556818651 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4009790 con 0x7f3fe8102480 2023-11-27T15:23:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.219+0000 7f3fdf7fe700 1 -- 172.21.15.133:0/3556818651 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fe400c7e0 con 0x7f3fe8102480 2023-11-27T15:23:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.219+0000 7f3ff03ff700 1 -- 172.21.15.133:0/3556818651 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe810de30 con 0x7f3fe8102480 2023-11-27T15:23:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.219+0000 7f3fdf7fe700 1 -- 172.21.15.133:0/3556818651 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4003c70 con 0x7f3fe8102480 2023-11-27T15:23:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.219+0000 7f3ff03ff700 1 -- 172.21.15.133:0/3556818651 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe810dca0 con 0x7f3fe8102480 2023-11-27T15:23:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.222+0000 7f3fdf7fe700 1 -- 172.21.15.133:0/3556818651 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3fe400cc50 con 0x7f3fe8102480 2023-11-27T15:23:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.224+0000 7f3fdf7fe700 1 --2- 172.21.15.133:0/3556818651 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3fd40a1150 0x7f3fd40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.224+0000 7f3fdf7fe700 1 -- 172.21.15.133:0/3556818651 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f3fe4014070 con 0x7f3fe8102480 2023-11-27T15:23:45.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.224+0000 7f3fed99a700 1 --2- 172.21.15.133:0/3556818651 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3fd40a1150 0x7f3fd40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.225+0000 7f3fed99a700 1 --2- 172.21.15.133:0/3556818651 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3fd40a1150 0x7f3fd40a3600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800b2e0 tx=0x7f3fd8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:45.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.300+0000 7f4003dc3b80 1 Processor -- start 2023-11-27T15:23:45.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.300+0000 7f4003dc3b80 1 -- start start 2023-11-27T15:23:45.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.300+0000 7f4003dc3b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfcf56b0 0x5605dfe99870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.300+0000 7f4003dc3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfe99db0 0x5605dfe9a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.300+0000 7f4003dc3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe9ab10 0x5605dfe9af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.300+0000 7f4003dc3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605dfd9f000 con 0x5605dfe99db0 2023-11-27T15:23:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.300+0000 7f4003dc3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605dfdf1840 con 0x5605dfe9ab10 2023-11-27T15:23:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.300+0000 7f4003dc3b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605dfdeacd0 con 0x5605dfcf56b0 2023-11-27T15:23:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.301+0000 7f3fed99a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfe99db0 0x5605dfe9a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.301+0000 7f3fed99a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfe99db0 0x5605dfe9a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57992/0 (socket says 172.21.15.133:57992) 2023-11-27T15:23:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.301+0000 7f3fed99a700 1 -- 172.21.15.133:0/2960505806 learned_addr learned my addr 172.21.15.133:0/2960505806 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:45.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.301+0000 7f3fee19b700 1 --2- 172.21.15.133:0/2960505806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfcf56b0 0x5605dfe99870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.301+0000 7f3fee99c700 1 --2- 172.21.15.133:0/2960505806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe9ab10 0x5605dfe9af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.301+0000 7f3fee19b700 1 -- 172.21.15.133:0/2960505806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe9ab10 msgr2=0x5605dfe9af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.301+0000 7f3fee19b700 1 --2- 172.21.15.133:0/2960505806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe9ab10 0x5605dfe9af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.301+0000 7f3fee19b700 1 -- 172.21.15.133:0/2960505806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfe99db0 msgr2=0x5605dfe9a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:45.309 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.301+0000 7f3fee19b700 1 --2- 172.21.15.133:0/2960505806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfe99db0 0x5605dfe9a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.301+0000 7f3fee19b700 1 -- 172.21.15.133:0/2960505806 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605dfc6c5d0 con 0x5605dfcf56b0 2023-11-27T15:23:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.302+0000 7f3fee19b700 1 --2- 172.21.15.133:0/2960505806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfcf56b0 0x5605dfe99870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4005950 tx=0x7f3fe4099890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7f2e80bd8f8b5e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.302+0000 7f3fdcbfe700 1 -- 172.21.15.133:0/2960505806 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe40b1070 con 0x5605dfcf56b0 2023-11-27T15:23:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.302+0000 7f3fdcbfe700 1 -- 172.21.15.133:0/2960505806 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fe409dee0 con 0x5605dfcf56b0 2023-11-27T15:23:45.310 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.302+0000 7f3fdcbfe700 1 -- 172.21.15.133:0/2960505806 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe409a330 con 0x5605dfcf56b0 2023-11-27T15:23:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.303+0000 7f4003dc3b80 1 -- 172.21.15.133:0/2960505806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfcf56b0 msgr2=0x5605dfe99870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.303+0000 7f4003dc3b80 1 --2- 172.21.15.133:0/2960505806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfcf56b0 0x5605dfe99870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4005950 tx=0x7f3fe4099890 comp rx=0 tx=0).stop 2023-11-27T15:23:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.303+0000 7f4003dc3b80 1 -- 172.21.15.133:0/2960505806 shutdown_connections 2023-11-27T15:23:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.303+0000 7f4003dc3b80 1 --2- 172.21.15.133:0/2960505806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe9ab10 0x5605dfe9af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.303+0000 7f4003dc3b80 1 --2- 172.21.15.133:0/2960505806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfcf56b0 0x5605dfe99870 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.311 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.303+0000 7f4003dc3b80 1 --2- 172.21.15.133:0/2960505806 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfe99db0 0x5605dfe9a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.303+0000 7f4003dc3b80 1 -- 172.21.15.133:0/2960505806 >> 172.21.15.133:0/2960505806 conn(0x5605dfe8ef50 msgr2=0x5605dfe94100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.303+0000 7f4003dc3b80 1 -- 172.21.15.133:0/2960505806 shutdown_connections 2023-11-27T15:23:45.312 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.303+0000 7f4003dc3b80 1 -- 172.21.15.133:0/2960505806 wait complete. 2023-11-27T15:23:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.310+0000 7f4003dc3b80 1 -- start start 2023-11-27T15:23:45.314 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.310+0000 7f4003dc3b80 1 Processor -- start 2023-11-27T15:23:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.311+0000 7f4003dc3b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfcf56b0 0x5605dfea2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.311+0000 7f4003dc3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe99db0 0x5605dfea3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.311+0000 7f4003dc3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfe9ab10 0x5605dfeaa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.311+0000 7f4003dc3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605dfd8c880 con 0x5605dfe9ab10 2023-11-27T15:23:45.316 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.311+0000 7f4003dc3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605dfd8c520 con 0x5605dfe99db0 2023-11-27T15:23:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.311+0000 7f4003dc3b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605dfc6e8c0 con 0x5605dfcf56b0 2023-11-27T15:23:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.311+0000 7f4003dc3b80 10 client.0 _ll_register_callbacks cb 0x5605dfdaa720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.311+0000 7f3fee99c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfe9ab10 0x5605dfeaa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.311+0000 7f3fee99c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfe9ab10 0x5605dfeaa240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58002/0 (socket says 172.21.15.133:58002) 2023-11-27T15:23:45.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.311+0000 7f3fee19b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfcf56b0 0x5605dfea2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.311+0000 7f3fee99c700 1 -- 172.21.15.133:0/2439714454 learned_addr learned my addr 172.21.15.133:0/2439714454 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.311+0000 7f3fed99a700 1 --2- 172.21.15.133:0/2439714454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe99db0 0x5605dfea3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.312+0000 7f3fee99c700 1 -- 172.21.15.133:0/2439714454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfcf56b0 msgr2=0x5605dfea2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.312+0000 7f3fee99c700 1 --2- 172.21.15.133:0/2439714454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfcf56b0 0x5605dfea2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.312+0000 7f3fee99c700 1 -- 172.21.15.133:0/2439714454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe99db0 msgr2=0x5605dfea3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.312+0000 7f3fee99c700 1 --2- 172.21.15.133:0/2439714454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe99db0 0x5605dfea3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.312+0000 7f3fee99c700 1 -- 172.21.15.133:0/2439714454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe810dca0 con 0x5605dfe9ab10 2023-11-27T15:23:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.312+0000 7f3fed99a700 1 --2- 172.21.15.133:0/2439714454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe99db0 0x5605dfea3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.312+0000 7f3fee19b700 1 --2- 172.21.15.133:0/2439714454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfcf56b0 0x5605dfea2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.312+0000 7f3fee99c700 1 --2- 172.21.15.133:0/2439714454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfe9ab10 0x5605dfeaa240 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0011f60 tx=0x7f3fe0014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98cb9528582f3f7b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:45.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.312+0000 7f3fd1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:23:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.312+0000 7f3fd1ffb700 1 -- 172.21.15.133:0/2439714454 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe000e040 con 0x5605dfe9ab10 2023-11-27T15:23:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.312+0000 7f4003dc3b80 1 -- 172.21.15.133:0/2439714454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605dfe8c710 con 0x5605dfe9ab10 2023-11-27T15:23:45.320 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.312+0000 7f3fd1ffb700 1 -- 172.21.15.133:0/2439714454 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f3fe0014ba0 con 0x5605dfe9ab10 2023-11-27T15:23:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.325+0000 7f3fd1ffb700 1 -- 172.21.15.133:0/2439714454 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fe0014d60 con 0x5605dfe9ab10 2023-11-27T15:23:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.325+0000 7f4003dc3b80 10 client.15624 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.325+0000 7f4003dc3b80 1 -- 172.21.15.133:0/2439714454 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5605dfdb1130 con 0x5605dfe9ab10 2023-11-27T15:23:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.325+0000 7f3fd1ffb700 1 -- 172.21.15.133:0/2439714454 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5605dfe8c710 con 0x5605dfe9ab10 2023-11-27T15:23:45.329 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.325+0000 7f3fd1ffb700 1 -- 172.21.15.133:0/2439714454 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3fe001c020 con 0x5605dfe9ab10 2023-11-27T15:23:45.330 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.325+0000 7f4003dc3b80 10 client.15624 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.329+0000 7f4003dc3b80 10 client.15624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.329+0000 7f4003dc3b80 20 client.15624 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.329+0000 7f4003dc3b80 1 --2- 172.21.15.133:0/2439714454 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5605dfeaea80 0x5605dfeb0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.333 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.329+0000 7f4003dc3b80 4 client.15624 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:45.334 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.329+0000 7f4003dc3b80 1 -- 172.21.15.133:0/2439714454 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605dfdad040 con 0x5605dfeaea80 2023-11-27T15:23:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.331+0000 7f3fee19b700 1 --2- 172.21.15.133:0/2439714454 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5605dfeaea80 0x5605dfeb0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.332+0000 7f3fee19b700 1 --2- 172.21.15.133:0/2439714454 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5605dfeaea80 0x5605dfeb0e90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e9355e09b8a0b97 server_cookie=d1851d6c2e8c2c9f in_seq=0 out_seq=0 2023-11-27T15:23:45.335 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.332+0000 7f3fd1ffb700 10 client.15624 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:45.336 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.333+0000 7f3fd1ffb700 1 -- 172.21.15.133:0/2439714454 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5605dfdad040 con 0x5605dfeaea80 2023-11-27T15:23:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.333+0000 7f3fd1ffb700 10 client.15624 handle_command_reply: tid=0 2023-11-27T15:23:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.333+0000 7f4003dc3b80 1 client.15624 shutdown 2023-11-27T15:23:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.333+0000 7f4003dc3b80 1 -- 172.21.15.133:0/2439714454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfe9ab10 msgr2=0x5605dfeaa240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.333+0000 7f4003dc3b80 1 --2- 172.21.15.133:0/2439714454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfe9ab10 0x5605dfeaa240 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0011f60 tx=0x7f3fe0014600 comp rx=0 tx=0).stop 2023-11-27T15:23:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.333+0000 7f4003dc3b80 1 -- 172.21.15.133:0/2439714454 shutdown_connections 2023-11-27T15:23:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.333+0000 7f4003dc3b80 1 --2- 172.21.15.133:0/2439714454 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5605dfeaea80 0x5605dfeb0e90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.333+0000 7f4003dc3b80 1 --2- 172.21.15.133:0/2439714454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfe9ab10 0x5605dfeaa240 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.333+0000 7f4003dc3b80 1 --2- 172.21.15.133:0/2439714454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfcf56b0 0x5605dfea2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.333+0000 7f4003dc3b80 1 --2- 172.21.15.133:0/2439714454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe99db0 0x5605dfea3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.337 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.333+0000 7f4003dc3b80 1 -- 172.21.15.133:0/2439714454 >> 172.21.15.133:0/2439714454 conn(0x5605dfe8ef50 msgr2=0x5605dfe937e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.333+0000 7f3fd1ffb700 0 client.15624 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.333+0000 7f4003dc3b80 1 -- 172.21.15.133:0/2439714454 shutdown_connections 2023-11-27T15:23:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.334+0000 7f4003dc3b80 1 -- 172.21.15.133:0/2439714454 wait complete. 2023-11-27T15:23:45.338 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.334+0000 7f4003dc3b80 20 client.15624 trim_cache size 0 max 16384 2023-11-27T15:23:45.361 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.357+0000 7f4003dc3b80 1 Processor -- start 2023-11-27T15:23:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.357+0000 7f4003dc3b80 1 -- start start 2023-11-27T15:23:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.358+0000 7f4003dc3b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfcf56b0 0x5605dfea0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.358+0000 7f4003dc3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfe99db0 0x5605dfea1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.358+0000 7f4003dc3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe9ab10 0x5605dfeaa200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.358+0000 7f4003dc3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605dfea2830 con 0x5605dfe99db0 2023-11-27T15:23:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.358+0000 7f4003dc3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605dfe8c250 con 0x5605dfe9ab10 2023-11-27T15:23:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.358+0000 7f4003dc3b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605dfe9a420 con 0x5605dfcf56b0 2023-11-27T15:23:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.358+0000 7f3fed99a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfe99db0 0x5605dfea1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.358+0000 7f3fed99a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfe99db0 0x5605dfea1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58006/0 (socket says 172.21.15.133:58006) 2023-11-27T15:23:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.358+0000 7f3fed99a700 1 -- 172.21.15.133:0/3746220238 learned_addr learned my addr 172.21.15.133:0/3746220238 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.358+0000 7f3fee19b700 1 --2- 172.21.15.133:0/3746220238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfcf56b0 0x5605dfea0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.362 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.358+0000 7f3fee99c700 1 --2- 172.21.15.133:0/3746220238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe9ab10 0x5605dfeaa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.358+0000 7f3fed99a700 1 -- 172.21.15.133:0/3746220238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfcf56b0 msgr2=0x5605dfea0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.358+0000 7f3fed99a700 1 --2- 172.21.15.133:0/3746220238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfcf56b0 0x5605dfea0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.358+0000 7f3fed99a700 1 -- 172.21.15.133:0/3746220238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe9ab10 msgr2=0x5605dfeaa200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.358+0000 7f3fed99a700 1 --2- 172.21.15.133:0/3746220238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe9ab10 0x5605dfeaa200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.358+0000 7f3fed99a700 1 -- 172.21.15.133:0/3746220238 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605dfdb0e80 con 0x5605dfe99db0 2023-11-27T15:23:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.358+0000 7f3fee99c700 1 --2- 172.21.15.133:0/3746220238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe9ab10 0x5605dfeaa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.359+0000 7f3fed99a700 1 --2- 172.21.15.133:0/3746220238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfe99db0 0x5605dfea1390 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80095c0 tx=0x7f3fd8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11d05596c22c92d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.359+0000 7f3fdd3ff700 1 -- 172.21.15.133:0/3746220238 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd800bea0 con 0x5605dfe99db0 2023-11-27T15:23:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.359+0000 7f3fdd3ff700 1 -- 172.21.15.133:0/3746220238 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fd801e040 con 0x5605dfe99db0 2023-11-27T15:23:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.359+0000 7f3fdd3ff700 1 -- 172.21.15.133:0/3746220238 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd800fe80 con 0x5605dfe99db0 2023-11-27T15:23:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.360+0000 7f4003dc3b80 1 -- 172.21.15.133:0/3746220238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfe99db0 msgr2=0x5605dfea1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.360+0000 7f4003dc3b80 1 --2- 172.21.15.133:0/3746220238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfe99db0 0x5605dfea1390 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80095c0 tx=0x7f3fd8020000 comp rx=0 tx=0).stop 2023-11-27T15:23:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.360+0000 7f4003dc3b80 1 -- 172.21.15.133:0/3746220238 shutdown_connections 2023-11-27T15:23:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.360+0000 7f4003dc3b80 1 --2- 172.21.15.133:0/3746220238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe9ab10 0x5605dfeaa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.360+0000 7f4003dc3b80 1 --2- 172.21.15.133:0/3746220238 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfcf56b0 0x5605dfea0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.367 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.360+0000 7f4003dc3b80 1 --2- 172.21.15.133:0/3746220238 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfe99db0 0x5605dfea1390 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.360+0000 7f4003dc3b80 1 -- 172.21.15.133:0/3746220238 >> 172.21.15.133:0/3746220238 conn(0x5605dfe8ef50 msgr2=0x5605dfea19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.360+0000 7f4003dc3b80 1 -- 172.21.15.133:0/3746220238 shutdown_connections 2023-11-27T15:23:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.360+0000 7f4003dc3b80 1 -- 172.21.15.133:0/3746220238 wait complete. 2023-11-27T15:23:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.360+0000 7f4003dc3b80 1 -- start start 2023-11-27T15:23:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.360+0000 7f4003dc3b80 1 Processor -- start 2023-11-27T15:23:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.361+0000 7f4003dc3b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfcf56b0 0x5605dfea5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.361+0000 7f4003dc3b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfe99db0 0x5605dfea6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.368 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.361+0000 7f4003dc3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe9ab10 0x5605dfea91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.361+0000 7f4003dc3b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605dfea96e0 con 0x5605dfcf56b0 2023-11-27T15:23:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.361+0000 7f4003dc3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605dfea9850 con 0x5605dfe9ab10 2023-11-27T15:23:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.361+0000 7f4003dc3b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605dfea9a10 con 0x5605dfe99db0 2023-11-27T15:23:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.361+0000 7f4003dc3b80 10 client.0 _ll_register_callbacks cb 0x7f3fe00118c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.361+0000 7f3fee19b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfcf56b0 0x5605dfea5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.361+0000 7f3fee19b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfcf56b0 0x5605dfea5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58010/0 (socket says 172.21.15.133:58010) 2023-11-27T15:23:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.361+0000 7f3fee19b700 1 -- 172.21.15.133:0/1871069452 learned_addr learned my addr 172.21.15.133:0/1871069452 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.361+0000 7f3fee99c700 1 --2- 172.21.15.133:0/1871069452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe9ab10 0x5605dfea91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.361+0000 7f3fed99a700 1 --2- 172.21.15.133:0/1871069452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfe99db0 0x5605dfea6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.361+0000 7f3fee19b700 1 -- 172.21.15.133:0/1871069452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfe99db0 msgr2=0x5605dfea6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.361+0000 7f3fee19b700 1 --2- 172.21.15.133:0/1871069452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfe99db0 0x5605dfea6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.361+0000 7f3fee19b700 1 -- 172.21.15.133:0/1871069452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe9ab10 msgr2=0x5605dfea91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.361+0000 7f3fee19b700 1 --2- 172.21.15.133:0/1871069452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe9ab10 0x5605dfea91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.361+0000 7f3fee19b700 1 -- 172.21.15.133:0/1871069452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605dfdb1130 con 0x5605dfcf56b0 2023-11-27T15:23:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.361+0000 7f3fed99a700 1 --2- 172.21.15.133:0/1871069452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfe99db0 0x5605dfea6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:23:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.362+0000 7f3fee19b700 1 --2- 172.21.15.133:0/1871069452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfcf56b0 0x5605dfea5c40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400efd0 tx=0x7f3fe40092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c92c05a1adaab18 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.362+0000 7f3fd1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:23:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.362+0000 7f3fd1ffb700 1 -- 172.21.15.133:0/1871069452 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe40b1040 con 0x5605dfcf56b0 2023-11-27T15:23:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.362+0000 7f3fd1ffb700 1 -- 172.21.15.133:0/1871069452 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f3fe409a110 con 0x5605dfcf56b0 2023-11-27T15:23:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.362+0000 7f4003dc3b80 1 -- 172.21.15.133:0/1871069452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fe409dee0 con 0x5605dfcf56b0 2023-11-27T15:23:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.362+0000 7f3fd1ffb700 1 -- 172.21.15.133:0/1871069452 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fe40b82f0 con 0x5605dfcf56b0 2023-11-27T15:23:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.362+0000 7f4003dc3b80 10 client.15630 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.362+0000 7f4003dc3b80 1 -- 172.21.15.133:0/1871069452 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f3fe810b1f0 con 0x5605dfcf56b0 2023-11-27T15:23:45.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.362+0000 7f3fd1ffb700 1 -- 172.21.15.133:0/1871069452 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fe409dee0 con 0x5605dfcf56b0 2023-11-27T15:23:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.362+0000 7f3fd1ffb700 1 -- 172.21.15.133:0/1871069452 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3fe40a1e70 con 0x5605dfcf56b0 2023-11-27T15:23:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.362+0000 7f4003dc3b80 10 client.15630 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.362+0000 7f4003dc3b80 10 client.15630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.362+0000 7f4003dc3b80 20 client.15630 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.362+0000 7f4003dc3b80 1 --2- 172.21.15.133:0/1871069452 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5605dff031c0 0x5605dff055d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.362+0000 7f4003dc3b80 4 client.15630 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:23:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.362+0000 7f4003dc3b80 1 -- 172.21.15.133:0/1871069452 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5605dfdb9600 con 0x5605dff031c0 2023-11-27T15:23:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.363+0000 7f3fed99a700 1 --2- 172.21.15.133:0/1871069452 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5605dff031c0 0x5605dff055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.363+0000 7f3fd1ffb700 10 client.15630 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:45.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.363+0000 7f3fed99a700 1 --2- 172.21.15.133:0/1871069452 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5605dff031c0 0x5605dff055d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=806bfce561a29877 server_cookie=d3b55e1681961fa7 in_seq=0 out_seq=0 2023-11-27T15:23:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.385+0000 7f3fd1ffb700 1 -- 172.21.15.133:0/1871069452 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5605dfdb9600 con 0x5605dff031c0 2023-11-27T15:23:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.385+0000 7f3fd1ffb700 10 client.15630 handle_command_reply: tid=0 2023-11-27T15:23:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.385+0000 7f4003dc3b80 1 client.15630 shutdown 2023-11-27T15:23:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.386+0000 7f4003dc3b80 1 -- 172.21.15.133:0/1871069452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfcf56b0 msgr2=0x5605dfea5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.386+0000 7f4003dc3b80 1 --2- 172.21.15.133:0/1871069452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfcf56b0 0x5605dfea5c40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400efd0 tx=0x7f3fe40092d0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.386+0000 7f4003dc3b80 1 -- 172.21.15.133:0/1871069452 shutdown_connections 2023-11-27T15:23:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.386+0000 7f4003dc3b80 1 --2- 172.21.15.133:0/1871069452 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5605dff031c0 0x5605dff055d0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.386+0000 7f4003dc3b80 1 --2- 172.21.15.133:0/1871069452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5605dfe9ab10 0x5605dfea91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.386+0000 7f4003dc3b80 1 --2- 172.21.15.133:0/1871069452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5605dfcf56b0 0x5605dfea5c40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.386+0000 7f3fd1ffb700 0 client.15630 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.386+0000 7f4003dc3b80 1 --2- 172.21.15.133:0/1871069452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5605dfe99db0 0x5605dfea6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.386+0000 7f4003dc3b80 1 -- 172.21.15.133:0/1871069452 >> 172.21.15.133:0/1871069452 conn(0x5605dfe8ef50 msgr2=0x5605dfea0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.386+0000 7f4003dc3b80 1 -- 172.21.15.133:0/1871069452 shutdown_connections 2023-11-27T15:23:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.386+0000 7f4003dc3b80 1 -- 172.21.15.133:0/1871069452 wait complete. 2023-11-27T15:23:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.386+0000 7f4003dc3b80 20 client.15630 trim_cache size 0 max 16384 2023-11-27T15:23:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:23:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:23:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:23:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:23:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.387+0000 7f3ff03ff700 1 -- 172.21.15.133:0/3556818651 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3fd40a1150 msgr2=0x7f3fd40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.387+0000 7f3ff03ff700 1 --2- 172.21.15.133:0/3556818651 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3fd40a1150 0x7f3fd40a3600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800b2e0 tx=0x7f3fd8005fb0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.387+0000 7f3ff03ff700 1 -- 172.21.15.133:0/3556818651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fe8102480 msgr2=0x7f3fe8198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.387+0000 7f3ff03ff700 1 --2- 172.21.15.133:0/3556818651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fe8102480 0x7f3fe8198240 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400e970 tx=0x7f3fe400ec80 comp rx=0 tx=0).stop 2023-11-27T15:23:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.387+0000 7f3ff03ff700 1 -- 172.21.15.133:0/3556818651 shutdown_connections 2023-11-27T15:23:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.387+0000 7f3ff03ff700 1 --2- 172.21.15.133:0/3556818651 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3fd40a1150 0x7f3fd40a3600 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.387+0000 7f3ff03ff700 1 --2- 172.21.15.133:0/3556818651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3fe8104000 0x7f3fe819f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.387+0000 7f3ff03ff700 1 --2- 172.21.15.133:0/3556818651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3fe8102480 0x7f3fe8198240 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.387+0000 7f3ff03ff700 1 --2- 172.21.15.133:0/3556818651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3fe8103670 0x7f3fe8198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.387+0000 7f3ff03ff700 1 -- 172.21.15.133:0/3556818651 >> 172.21.15.133:0/3556818651 conn(0x7f3fe80fda80 msgr2=0x7f3fe80ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.387+0000 7f3ff03ff700 1 -- 172.21.15.133:0/3556818651 shutdown_connections 2023-11-27T15:23:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:23:45.387+0000 7f3ff03ff700 1 -- 172.21.15.133:0/3556818651 wait complete. 2023-11-27T15:23:45.561 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:45.561 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:45.671 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:45.671 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:45.684 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:45.684 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:45.685 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 12 -lt 100 ']' 2023-11-27T15:23:45.685 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 12 + 1 2023-11-27T15:23:45.706 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=13 2023-11-27T15:23:45.706 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:45.744 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:45.744 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:45.747 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:45.747 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:45.748 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:46.277 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:46.277 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:46.438 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:46.438 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:46.487 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:46.487 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:46.487 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 13 -lt 100 ']' 2023-11-27T15:23:46.488 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 13 + 1 2023-11-27T15:23:46.506 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=14 2023-11-27T15:23:46.506 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:46.535 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:46.536 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:46.538 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:46.539 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:46.539 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:47.052 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:47.052 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:47.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:46 smithi177 ceph-mon[131504]: pgmap v374: 129 pgs: 129 active+clean; 676 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-27T15:23:47.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:46 smithi174 ceph-mon[131110]: pgmap v374: 129 pgs: 129 active+clean; 676 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-27T15:23:47.229 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:47.229 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:47.261 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:47.261 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:47.261 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 14 -lt 100 ']' 2023-11-27T15:23:47.262 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 14 + 1 2023-11-27T15:23:47.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:46 smithi133 ceph-mon[97860]: pgmap v374: 129 pgs: 129 active+clean; 676 KiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-11-27T15:23:47.294 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=15 2023-11-27T15:23:47.294 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:47.325 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:47.325 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:47.327 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:47.328 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:47.328 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:47.794 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:47.794 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:47.913 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:47.913 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:47.926 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:47.926 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:47.926 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 15 -lt 100 ']' 2023-11-27T15:23:47.927 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 15 + 1 2023-11-27T15:23:47.947 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=16 2023-11-27T15:23:47.947 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:47.976 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:47.977 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:47.979 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:47.980 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:47.980 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:48.460 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:48.460 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:48.655 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:48.655 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:48.683 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:48.683 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:48.683 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 16 -lt 100 ']' 2023-11-27T15:23:48.683 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 16 + 1 2023-11-27T15:23:48.705 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=17 2023-11-27T15:23:48.705 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:48.736 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:48.736 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:48.739 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:48.739 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:48.739 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:49.019 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:48 smithi133 ceph-mon[97860]: pgmap v375: 129 pgs: 129 active+clean; 790 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2023-11-27T15:23:49.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:48 smithi177 ceph-mon[131504]: pgmap v375: 129 pgs: 129 active+clean; 790 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2023-11-27T15:23:49.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:48 smithi174 ceph-mon[131110]: pgmap v375: 129 pgs: 129 active+clean; 790 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 10 op/s 2023-11-27T15:23:49.201 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:49.202 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:49.338 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:49.338 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:49.352 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:49.352 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:49.352 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 17 -lt 100 ']' 2023-11-27T15:23:49.352 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 17 + 1 2023-11-27T15:23:49.372 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=18 2023-11-27T15:23:49.372 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:49.401 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:49.401 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:49.404 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:49.404 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:49.405 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:49.852 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:49.852 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:49.971 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:49.971 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:49.986 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:49.987 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:49.987 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 18 -lt 100 ']' 2023-11-27T15:23:49.987 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 18 + 1 2023-11-27T15:23:50.005 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=19 2023-11-27T15:23:50.005 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:50.041 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:50.041 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:50.044 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:50.044 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:50.044 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:50.509 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:50.510 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:50.629 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:50.629 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:50.655 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:50.655 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:50.655 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 19 -lt 100 ']' 2023-11-27T15:23:50.655 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 19 + 1 2023-11-27T15:23:50.702 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=20 2023-11-27T15:23:50.702 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:50.782 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:50.782 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:50.784 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:50.785 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:50.785 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:51.077 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:50 smithi133 ceph-mon[97860]: pgmap v376: 129 pgs: 129 active+clean; 790 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2023-11-27T15:23:51.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:50 smithi177 ceph-mon[131504]: pgmap v376: 129 pgs: 129 active+clean; 790 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2023-11-27T15:23:51.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:50 smithi174 ceph-mon[131110]: pgmap v376: 129 pgs: 129 active+clean; 790 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2023-11-27T15:23:51.268 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:51.268 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:51.403 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:51.403 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:51.427 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:51.427 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:51.427 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 20 -lt 100 ']' 2023-11-27T15:23:51.428 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 20 + 1 2023-11-27T15:23:51.460 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=21 2023-11-27T15:23:51.460 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:51.491 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:51.491 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:51.493 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:51.494 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:51.494 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:52.034 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:52.035 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:52.169 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:52.169 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:52.201 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:52.201 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:52.201 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 21 -lt 100 ']' 2023-11-27T15:23:52.201 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 21 + 1 2023-11-27T15:23:52.235 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=22 2023-11-27T15:23:52.235 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:52.318 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:52.319 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:52.321 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:52.321 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:52.322 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:52.876 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:52.876 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:53.012 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:53.012 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:53.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:52 smithi133 ceph-mon[97860]: pgmap v377: 129 pgs: 129 active+clean; 810 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2023-11-27T15:23:53.044 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:53.044 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:53.045 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 22 -lt 100 ']' 2023-11-27T15:23:53.045 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 22 + 1 2023-11-27T15:23:53.068 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=23 2023-11-27T15:23:53.068 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:53.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:52 smithi177 ceph-mon[131504]: pgmap v377: 129 pgs: 129 active+clean; 810 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2023-11-27T15:23:53.091 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:53.091 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:53.094 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:53.094 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:53.095 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:53.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:52 smithi174 ceph-mon[131110]: pgmap v377: 129 pgs: 129 active+clean; 810 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 15 op/s 2023-11-27T15:23:53.576 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:53.576 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:53.695 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:53.695 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:53.708 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:53.709 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:53.709 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 23 -lt 100 ']' 2023-11-27T15:23:53.709 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 23 + 1 2023-11-27T15:23:53.729 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=24 2023-11-27T15:23:53.729 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:53.763 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:53.763 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:53.766 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:53.766 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:53.767 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:54.284 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:54.284 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:54.432 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:54.433 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:54.443 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:54.443 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:54.443 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 24 -lt 100 ']' 2023-11-27T15:23:54.443 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 24 + 1 2023-11-27T15:23:54.471 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=25 2023-11-27T15:23:54.471 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:54.517 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:54.517 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:54.519 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:54.520 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:54.520 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:54.667 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:23:55.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:54 smithi177 ceph-mon[131504]: pgmap v378: 129 pgs: 129 active+clean; 810 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-11-27T15:23:55.113 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:54 smithi133 ceph-mon[97860]: pgmap v378: 129 pgs: 129 active+clean; 810 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-11-27T15:23:55.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:54 smithi174 ceph-mon[131110]: pgmap v378: 129 pgs: 129 active+clean; 810 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-11-27T15:23:55.192 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:55.192 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:55.311 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:55.311 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:55.323 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:55.323 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:55.323 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 25 -lt 100 ']' 2023-11-27T15:23:55.324 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 25 + 1 2023-11-27T15:23:55.351 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=26 2023-11-27T15:23:55.351 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:55.375 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:55.376 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:55.378 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:55.379 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:55.379 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:56.089 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:56.089 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:56.227 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:56.227 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:56.277 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:56.277 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:56.277 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 26 -lt 100 ']' 2023-11-27T15:23:56.278 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 26 + 1 2023-11-27T15:23:56.326 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=27 2023-11-27T15:23:56.326 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:56.346 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:56.346 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:56.349 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:56.349 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:56.349 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:57.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:56 smithi177 ceph-mon[131504]: pgmap v379: 129 pgs: 129 active+clean; 810 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-11-27T15:23:57.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:56 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:23:57.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:56 smithi174 ceph-mon[131110]: pgmap v379: 129 pgs: 129 active+clean; 810 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-11-27T15:23:57.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:56 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:23:57.170 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:56 smithi133 ceph-mon[97860]: pgmap v379: 129 pgs: 129 active+clean; 810 KiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-11-27T15:23:57.170 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:56 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:23:57.309 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:57.310 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:57.512 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:57.512 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:57.533 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:57.534 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:57.534 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 27 -lt 100 ']' 2023-11-27T15:23:57.534 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 27 + 1 2023-11-27T15:23:57.587 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=28 2023-11-27T15:23:57.587 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:57.699 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:57.699 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:57.701 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:57.702 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:57.702 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:57.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.812+0000 7fbd979ff700 1 -- 172.21.15.133:0/2432230864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd901002d0 msgr2=0x7fbd901006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:57.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.812+0000 7fbd979ff700 1 --2- 172.21.15.133:0/2432230864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd901002d0 0x7fbd901006e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009990 tx=0x7fbd8c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.812+0000 7fbd979ff700 1 -- 172.21.15.133:0/2432230864 shutdown_connections 2023-11-27T15:23:57.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.812+0000 7fbd979ff700 1 --2- 172.21.15.133:0/2432230864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd90101e50 0x7fbd90108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.812+0000 7fbd979ff700 1 --2- 172.21.15.133:0/2432230864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd901002d0 0x7fbd901006e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.812+0000 7fbd979ff700 1 --2- 172.21.15.133:0/2432230864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd901014c0 0x7fbd90101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.818 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.812+0000 7fbd979ff700 1 -- 172.21.15.133:0/2432230864 >> 172.21.15.133:0/2432230864 conn(0x7fbd900fb8b0 msgr2=0x7fbd900fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:57.818 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.812+0000 7fbd979ff700 1 -- 172.21.15.133:0/2432230864 shutdown_connections 2023-11-27T15:23:57.818 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.812+0000 7fbd979ff700 1 -- 172.21.15.133:0/2432230864 wait complete. 2023-11-27T15:23:57.820 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.813+0000 7fbd979ff700 1 Processor -- start 2023-11-27T15:23:57.820 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.813+0000 7fbd979ff700 1 -- start start 2023-11-27T15:23:57.820 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.813+0000 7fbd979ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd901002d0 0x7fbd90198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.820 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.813+0000 7fbd979ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd901014c0 0x7fbd901987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.820 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.813+0000 7fbd979ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd90101e50 0x7fbd9019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.813+0000 7fbd979ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9019fe10 con 0x7fbd901014c0 2023-11-27T15:23:57.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.813+0000 7fbd979ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9019ff50 con 0x7fbd901002d0 2023-11-27T15:23:57.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.813+0000 7fbd979ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd901a00e0 con 0x7fbd90101e50 2023-11-27T15:23:57.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.814+0000 7fbd94f9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd901014c0 0x7fbd901987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.814+0000 7fbd9579b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd901002d0 0x7fbd90198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.814+0000 7fbd9579b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd901002d0 0x7fbd90198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:55432/0 (socket says 172.21.15.133:55432) 2023-11-27T15:23:57.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.814+0000 7fbd94f9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd901014c0 0x7fbd901987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60840/0 (socket says 172.21.15.133:60840) 2023-11-27T15:23:57.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.814+0000 7fbd9579b700 1 -- 172.21.15.133:0/288221084 learned_addr learned my addr 172.21.15.133:0/288221084 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:57.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.814+0000 7fbd95f9c700 1 --2- 172.21.15.133:0/288221084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd90101e50 0x7fbd9019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.814+0000 7fbd9579b700 1 -- 172.21.15.133:0/288221084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd90101e50 msgr2=0x7fbd9019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:57.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.814+0000 7fbd9579b700 1 --2- 172.21.15.133:0/288221084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd90101e50 0x7fbd9019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.814+0000 7fbd9579b700 1 -- 172.21.15.133:0/288221084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd901014c0 msgr2=0x7fbd901987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:57.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.814+0000 7fbd9579b700 1 --2- 172.21.15.133:0/288221084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd901014c0 0x7fbd901987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.814+0000 7fbd9579b700 1 -- 172.21.15.133:0/288221084 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c0095f0 con 0x7fbd901002d0 2023-11-27T15:23:57.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.814+0000 7fbd94f9a700 1 --2- 172.21.15.133:0/288221084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd901014c0 0x7fbd901987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:57.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.814+0000 7fbd9579b700 1 --2- 172.21.15.133:0/288221084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd901002d0 0x7fbd90198290 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c005740 tx=0x7fbd8c005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:57.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.815+0000 7fbd867fc700 1 -- 172.21.15.133:0/288221084 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c01d070 con 0x7fbd901002d0 2023-11-27T15:23:57.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.815+0000 7fbd979ff700 1 -- 172.21.15.133:0/288221084 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9010bca0 con 0x7fbd901002d0 2023-11-27T15:23:57.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.815+0000 7fbd867fc700 1 -- 172.21.15.133:0/288221084 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd8c021410 con 0x7fbd901002d0 2023-11-27T15:23:57.823 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.815+0000 7fbd867fc700 1 -- 172.21.15.133:0/288221084 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c017650 con 0x7fbd901002d0 2023-11-27T15:23:57.823 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.815+0000 7fbd979ff700 1 -- 172.21.15.133:0/288221084 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9010bb10 con 0x7fbd901002d0 2023-11-27T15:23:57.824 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.818+0000 7fbd867fc700 1 -- 172.21.15.133:0/288221084 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fbd8c021580 con 0x7fbd901002d0 2023-11-27T15:23:57.824 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.819+0000 7fbd867fc700 1 --2- 172.21.15.133:0/288221084 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbd7c0a1150 0x7fbd7c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.824 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.819+0000 7fbd867fc700 1 -- 172.21.15.133:0/288221084 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbd8c0cdde0 con 0x7fbd901002d0 2023-11-27T15:23:57.824 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.819+0000 7fbd94f9a700 1 --2- 172.21.15.133:0/288221084 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbd7c0a1150 0x7fbd7c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.824 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.820+0000 7fbd94f9a700 1 --2- 172.21.15.133:0/288221084 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbd7c0a1150 0x7fbd7c0a3600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbd80011490 tx=0x7fbd80009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:57.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.928+0000 7fbdab3f5b80 1 Processor -- start 2023-11-27T15:23:57.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.928+0000 7fbdab3f5b80 1 -- start start 2023-11-27T15:23:57.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.929+0000 7fbdab3f5b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9afc00 0x56369c9b0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.929+0000 7fbdab3f5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9b0550 0x56369c9b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.929+0000 7fbdab3f5b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b1110 0x56369c9b1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.929+0000 7fbdab3f5b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369c8b5060 con 0x56369c9afc00 2023-11-27T15:23:57.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.929+0000 7fbdab3f5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369c9078a0 con 0x56369c9b0550 2023-11-27T15:23:57.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.929+0000 7fbdab3f5b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369c900d30 con 0x56369c9b1110 2023-11-27T15:23:57.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.929+0000 7fbd9579b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9afc00 0x56369c9b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.929+0000 7fbd95f9c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b1110 0x56369c9b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.929+0000 7fbd9579b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9afc00 0x56369c9b0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60850/0 (socket says 172.21.15.133:60850) 2023-11-27T15:23:57.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.929+0000 7fbd9579b700 1 -- 172.21.15.133:0/1853798547 learned_addr learned my addr 172.21.15.133:0/1853798547 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:57.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.929+0000 7fbd94f9a700 1 --2- 172.21.15.133:0/1853798547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9b0550 0x56369c9b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.935 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.929+0000 7fbd95f9c700 1 -- 172.21.15.133:0/1853798547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9b0550 msgr2=0x56369c9b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:57.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.929+0000 7fbd95f9c700 1 --2- 172.21.15.133:0/1853798547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9b0550 0x56369c9b0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.929+0000 7fbd95f9c700 1 -- 172.21.15.133:0/1853798547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9afc00 msgr2=0x56369c9b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:57.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.929+0000 7fbd95f9c700 1 --2- 172.21.15.133:0/1853798547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9afc00 0x56369c9b0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.929+0000 7fbd95f9c700 1 -- 172.21.15.133:0/1853798547 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56369c782630 con 0x56369c9b1110 2023-11-27T15:23:57.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.929+0000 7fbd9579b700 1 --2- 172.21.15.133:0/1853798547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9afc00 0x56369c9b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:57.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.930+0000 7fbd95f9c700 1 --2- 172.21.15.133:0/1853798547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b1110 0x56369c9b1580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbd88011580 tx=0x7fbd88009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ddba365861c994e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:57.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.930+0000 7fbd7b7fe700 1 -- 172.21.15.133:0/1853798547 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8800e040 con 0x56369c9b1110 2023-11-27T15:23:57.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.930+0000 7fbd7b7fe700 1 -- 172.21.15.133:0/1853798547 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd8800f040 con 0x56369c9b1110 2023-11-27T15:23:57.936 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.930+0000 7fbd7b7fe700 1 -- 172.21.15.133:0/1853798547 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd88004430 con 0x56369c9b1110 2023-11-27T15:23:57.939 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.931+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/1853798547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b1110 msgr2=0x56369c9b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:57.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.931+0000 7fbdab3f5b80 1 --2- 172.21.15.133:0/1853798547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b1110 0x56369c9b1580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbd88011580 tx=0x7fbd88009920 comp rx=0 tx=0).stop 2023-11-27T15:23:57.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.931+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/1853798547 shutdown_connections 2023-11-27T15:23:57.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.931+0000 7fbdab3f5b80 1 --2- 172.21.15.133:0/1853798547 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b1110 0x56369c9b1580 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.931+0000 7fbdab3f5b80 1 --2- 172.21.15.133:0/1853798547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9afc00 0x56369c9b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.931+0000 7fbdab3f5b80 1 --2- 172.21.15.133:0/1853798547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9b0550 0x56369c9b0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.931+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/1853798547 >> 172.21.15.133:0/1853798547 conn(0x56369c9a5360 msgr2=0x56369c9aa670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:57.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.931+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/1853798547 shutdown_connections 2023-11-27T15:23:57.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.931+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/1853798547 wait complete. 2023-11-27T15:23:57.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.931+0000 7fbdab3f5b80 1 -- start start 2023-11-27T15:23:57.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.931+0000 7fbdab3f5b80 1 Processor -- start 2023-11-27T15:23:57.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.932+0000 7fbdab3f5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9afc00 0x56369c9b95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.940 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.932+0000 7fbdab3f5b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b0550 0x56369c9b9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.932+0000 7fbdab3f5b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9b1110 0x56369c9c0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.932+0000 7fbdab3f5b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369c8a28e0 con 0x56369c9b1110 2023-11-27T15:23:57.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.932+0000 7fbdab3f5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369c8a2580 con 0x56369c9afc00 2023-11-27T15:23:57.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.932+0000 7fbdab3f5b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369c784920 con 0x56369c9b0550 2023-11-27T15:23:57.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.932+0000 7fbdab3f5b80 10 client.0 _ll_register_callbacks cb 0x56369c8c0780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:57.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.932+0000 7fbd9579b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9afc00 0x56369c9b95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.932+0000 7fbd94f9a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b0550 0x56369c9b9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.932+0000 7fbd95f9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9b1110 0x56369c9c0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.932+0000 7fbd9579b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9afc00 0x56369c9b95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:55464/0 (socket says 172.21.15.133:55464) 2023-11-27T15:23:57.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.932+0000 7fbd9579b700 1 -- 172.21.15.133:0/217716602 learned_addr learned my addr 172.21.15.133:0/217716602 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:57.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.933+0000 7fbd94f9a700 1 -- 172.21.15.133:0/217716602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9afc00 msgr2=0x56369c9b95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:57.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.933+0000 7fbd94f9a700 1 --2- 172.21.15.133:0/217716602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9afc00 0x56369c9b95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.941 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.933+0000 7fbd94f9a700 1 -- 172.21.15.133:0/217716602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9b1110 msgr2=0x56369c9c0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:57.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.933+0000 7fbd94f9a700 1 --2- 172.21.15.133:0/217716602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9b1110 0x56369c9c0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.933+0000 7fbd94f9a700 1 -- 172.21.15.133:0/217716602 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd88011230 con 0x56369c9b0550 2023-11-27T15:23:57.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.933+0000 7fbd9579b700 1 --2- 172.21.15.133:0/217716602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9afc00 0x56369c9b95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:57.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.933+0000 7fbd94f9a700 1 --2- 172.21.15.133:0/217716602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b0550 0x56369c9b9b00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbd80011be0 tx=0x7fbd800047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=907dbd861c56ba7d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:57.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.933+0000 7fbd78ff9700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:23:57.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.933+0000 7fbd78ff9700 1 -- 172.21.15.133:0/217716602 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8001fa10 con 0x56369c9b0550 2023-11-27T15:23:57.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.933+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/217716602 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56369c9a2b20 con 0x56369c9b0550 2023-11-27T15:23:57.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.933+0000 7fbd78ff9700 1 -- 172.21.15.133:0/217716602 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbd8001fb70 con 0x56369c9b0550 2023-11-27T15:23:57.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.933+0000 7fbd78ff9700 1 -- 172.21.15.133:0/217716602 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd8001fd70 con 0x56369c9b0550 2023-11-27T15:23:57.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.933+0000 7fbdab3f5b80 10 client.25675 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:57.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.933+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/217716602 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56369c8c7190 con 0x56369c9b0550 2023-11-27T15:23:57.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.934+0000 7fbd78ff9700 1 -- 172.21.15.133:0/217716602 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56369c9a2b20 con 0x56369c9b0550 2023-11-27T15:23:57.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.934+0000 7fbd78ff9700 1 -- 172.21.15.133:0/217716602 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbd800273e0 con 0x56369c9b0550 2023-11-27T15:23:57.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.934+0000 7fbdab3f5b80 10 client.25675 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:57.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.934+0000 7fbdab3f5b80 10 client.25675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:57.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.934+0000 7fbdab3f5b80 20 client.25675 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:57.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.934+0000 7fbdab3f5b80 1 --2- 172.21.15.133:0/217716602 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56369c9c5410 0x56369c9c7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.934+0000 7fbdab3f5b80 4 client.25675 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:23:57.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.934+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/217716602 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56369c8c30a0 con 0x56369c9c5410 2023-11-27T15:23:57.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.934+0000 7fbd9579b700 1 --2- 172.21.15.133:0/217716602 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56369c9c5410 0x56369c9c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.935+0000 7fbd78ff9700 10 client.25675 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:57.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.935+0000 7fbd9579b700 1 --2- 172.21.15.133:0/217716602 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56369c9c5410 0x56369c9c7820 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5909213568ca982 server_cookie=859791102d0d7333 in_seq=0 out_seq=0 2023-11-27T15:23:57.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.936+0000 7fbd78ff9700 1 -- 172.21.15.133:0/217716602 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56369c8c30a0 con 0x56369c9c5410 2023-11-27T15:23:57.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.936+0000 7fbd78ff9700 10 client.25675 handle_command_reply: tid=0 2023-11-27T15:23:57.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.936+0000 7fbdab3f5b80 1 client.25675 shutdown 2023-11-27T15:23:57.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.936+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/217716602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b0550 msgr2=0x56369c9b9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:57.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.936+0000 7fbdab3f5b80 1 --2- 172.21.15.133:0/217716602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b0550 0x56369c9b9b00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbd80011be0 tx=0x7fbd800047c0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.936+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/217716602 shutdown_connections 2023-11-27T15:23:57.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.936+0000 7fbdab3f5b80 1 --2- 172.21.15.133:0/217716602 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56369c9c5410 0x56369c9c7820 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.936+0000 7fbdab3f5b80 1 --2- 172.21.15.133:0/217716602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9b1110 0x56369c9c0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.936+0000 7fbdab3f5b80 1 --2- 172.21.15.133:0/217716602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9afc00 0x56369c9b95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.936+0000 7fbd78ff9700 0 client.25675 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:57.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.936+0000 7fbdab3f5b80 1 --2- 172.21.15.133:0/217716602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b0550 0x56369c9b9b00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.936+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/217716602 >> 172.21.15.133:0/217716602 conn(0x56369c9a5360 msgr2=0x56369c9a9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:57.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.936+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/217716602 shutdown_connections 2023-11-27T15:23:57.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.936+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/217716602 wait complete. 2023-11-27T15:23:57.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.936+0000 7fbdab3f5b80 20 client.25675 trim_cache size 0 max 16384 2023-11-27T15:23:57.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.953+0000 7fbdab3f5b80 1 Processor -- start 2023-11-27T15:23:57.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.954+0000 7fbdab3f5b80 1 -- start start 2023-11-27T15:23:57.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.954+0000 7fbdab3f5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9afc00 0x56369c9b7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.954+0000 7fbdab3f5b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9b0550 0x56369c9b7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.954+0000 7fbdab3f5b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b1110 0x56369c9c0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.954+0000 7fbdab3f5b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369c9c0fa0 con 0x56369c9b0550 2023-11-27T15:23:57.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.954+0000 7fbdab3f5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369c9b0920 con 0x56369c9afc00 2023-11-27T15:23:57.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.954+0000 7fbdab3f5b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369c9b71c0 con 0x56369c9b1110 2023-11-27T15:23:57.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.954+0000 7fbd94f9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9b0550 0x56369c9b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.954+0000 7fbd94f9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9b0550 0x56369c9b7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60868/0 (socket says 172.21.15.133:60868) 2023-11-27T15:23:57.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.954+0000 7fbd94f9a700 1 -- 172.21.15.133:0/4154525838 learned_addr learned my addr 172.21.15.133:0/4154525838 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:57.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.954+0000 7fbd9579b700 1 --2- 172.21.15.133:0/4154525838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9afc00 0x56369c9b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.954+0000 7fbd95f9c700 1 --2- 172.21.15.133:0/4154525838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b1110 0x56369c9c0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.954+0000 7fbd94f9a700 1 -- 172.21.15.133:0/4154525838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b1110 msgr2=0x56369c9c0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:57.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.954+0000 7fbd94f9a700 1 --2- 172.21.15.133:0/4154525838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b1110 0x56369c9c0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.954+0000 7fbd94f9a700 1 -- 172.21.15.133:0/4154525838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9afc00 msgr2=0x56369c9b7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:57.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.954+0000 7fbd94f9a700 1 --2- 172.21.15.133:0/4154525838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9afc00 0x56369c9b7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.955+0000 7fbd94f9a700 1 -- 172.21.15.133:0/4154525838 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56369c8c6ee0 con 0x56369c9b0550 2023-11-27T15:23:57.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.955+0000 7fbd94f9a700 1 --2- 172.21.15.133:0/4154525838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9b0550 0x56369c9b7950 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbd800097c0 tx=0x7fbd800040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907f68ba776e152c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:57.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.955+0000 7fbd7bfff700 1 -- 172.21.15.133:0/4154525838 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd800273e0 con 0x56369c9b0550 2023-11-27T15:23:57.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.955+0000 7fbd7bfff700 1 -- 172.21.15.133:0/4154525838 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd80027540 con 0x56369c9b0550 2023-11-27T15:23:57.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.955+0000 7fbd7bfff700 1 -- 172.21.15.133:0/4154525838 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80028440 con 0x56369c9b0550 2023-11-27T15:23:57.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.956+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/4154525838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9b0550 msgr2=0x56369c9b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:57.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.956+0000 7fbdab3f5b80 1 --2- 172.21.15.133:0/4154525838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9b0550 0x56369c9b7950 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbd800097c0 tx=0x7fbd800040e0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.956+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/4154525838 shutdown_connections 2023-11-27T15:23:57.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.956+0000 7fbdab3f5b80 1 --2- 172.21.15.133:0/4154525838 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b1110 0x56369c9c0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.956+0000 7fbdab3f5b80 1 --2- 172.21.15.133:0/4154525838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9afc00 0x56369c9b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.956+0000 7fbdab3f5b80 1 --2- 172.21.15.133:0/4154525838 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9b0550 0x56369c9b7950 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.956+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/4154525838 >> 172.21.15.133:0/4154525838 conn(0x56369c9a5360 msgr2=0x56369c9cc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:57.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.956+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/4154525838 shutdown_connections 2023-11-27T15:23:57.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.956+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/4154525838 wait complete. 2023-11-27T15:23:57.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.956+0000 7fbdab3f5b80 1 -- start start 2023-11-27T15:23:57.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.956+0000 7fbdab3f5b80 1 Processor -- start 2023-11-27T15:23:57.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.957+0000 7fbdab3f5b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9afc00 0x56369c9bc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.957+0000 7fbdab3f5b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b0550 0x56369c9bcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.957+0000 7fbdab3f5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9b1110 0x56369c9bff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.957+0000 7fbdab3f5b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369c9c0470 con 0x56369c9afc00 2023-11-27T15:23:57.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.957+0000 7fbdab3f5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369c9c05e0 con 0x56369c9b1110 2023-11-27T15:23:57.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.957+0000 7fbdab3f5b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369c9c07a0 con 0x56369c9b0550 2023-11-27T15:23:57.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.957+0000 7fbdab3f5b80 10 client.0 _ll_register_callbacks cb 0x7fbd80003680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:23:57.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.957+0000 7fbd9579b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9afc00 0x56369c9bc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.957+0000 7fbd9579b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9afc00 0x56369c9bc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60882/0 (socket says 172.21.15.133:60882) 2023-11-27T15:23:57.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.957+0000 7fbd9579b700 1 -- 172.21.15.133:0/788468239 learned_addr learned my addr 172.21.15.133:0/788468239 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:23:57.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.957+0000 7fbd94f9a700 1 --2- 172.21.15.133:0/788468239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b0550 0x56369c9bcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.957+0000 7fbd95f9c700 1 --2- 172.21.15.133:0/788468239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9b1110 0x56369c9bff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.957+0000 7fbd9579b700 1 -- 172.21.15.133:0/788468239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b0550 msgr2=0x56369c9bcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:57.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.957+0000 7fbd9579b700 1 --2- 172.21.15.133:0/788468239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b0550 0x56369c9bcf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.957+0000 7fbd9579b700 1 -- 172.21.15.133:0/788468239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9b1110 msgr2=0x56369c9bff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:23:57.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.957+0000 7fbd9579b700 1 --2- 172.21.15.133:0/788468239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9b1110 0x56369c9bff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.957+0000 7fbd9579b700 1 -- 172.21.15.133:0/788468239 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56369c8c7190 con 0x56369c9afc00 2023-11-27T15:23:57.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.957+0000 7fbd95f9c700 1 --2- 172.21.15.133:0/788468239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9b1110 0x56369c9bff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:23:57.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.957+0000 7fbd94f9a700 1 --2- 172.21.15.133:0/788468239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b0550 0x56369c9bcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:23:57.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.958+0000 7fbd9579b700 1 --2- 172.21.15.133:0/788468239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9afc00 0x56369c9bc9d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c097a60 tx=0x7fbd8c0cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90cf7c7e08e66f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:23:57.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.958+0000 7fbd78ff9700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:23:57.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.958+0000 7fbd78ff9700 1 -- 172.21.15.133:0/788468239 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c09f020 con 0x56369c9afc00 2023-11-27T15:23:57.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.958+0000 7fbd78ff9700 1 -- 172.21.15.133:0/788468239 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbd8c0a33b0 con 0x56369c9afc00 2023-11-27T15:23:57.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.958+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/788468239 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd8800f040 con 0x56369c9afc00 2023-11-27T15:23:57.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.958+0000 7fbd78ff9700 1 -- 172.21.15.133:0/788468239 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd8c0b92c0 con 0x56369c9afc00 2023-11-27T15:23:57.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.958+0000 7fbdab3f5b80 10 client.15648 fetch_fsmap learned FSMap version 23 2023-11-27T15:23:57.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.958+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/788468239 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56369c717c60 con 0x56369c9afc00 2023-11-27T15:23:57.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.958+0000 7fbd78ff9700 1 -- 172.21.15.133:0/788468239 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd8800f040 con 0x56369c9afc00 2023-11-27T15:23:57.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.958+0000 7fbd78ff9700 1 -- 172.21.15.133:0/788468239 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbd8c0a2f20 con 0x56369c9afc00 2023-11-27T15:23:57.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.958+0000 7fbdab3f5b80 10 client.15648 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:23:57.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.958+0000 7fbdab3f5b80 10 client.15648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:23:57.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.958+0000 7fbdab3f5b80 20 client.15648 populate_metadata read hostname 'smithi133' 2023-11-27T15:23:57.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.958+0000 7fbdab3f5b80 1 --2- 172.21.15.133:0/788468239 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56369ca1a0d0 0x56369ca1c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:23:57.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.958+0000 7fbdab3f5b80 4 client.15648 mds_command: new command op to 24560 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:23:57.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.958+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/788468239 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56369c8cf660 con 0x56369ca1a0d0 2023-11-27T15:23:57.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.959+0000 7fbd94f9a700 1 --2- 172.21.15.133:0/788468239 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56369ca1a0d0 0x56369ca1c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:23:57.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.959+0000 7fbd94f9a700 1 --2- 172.21.15.133:0/788468239 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56369ca1a0d0 0x56369ca1c4e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ed33c4bc0eb15b6 server_cookie=fcd4f3668e1fe641 in_seq=0 out_seq=0 2023-11-27T15:23:57.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.959+0000 7fbd78ff9700 10 client.15648 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:57.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.959+0000 7fbd78ff9700 1 -- 172.21.15.133:0/788468239 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56369c8cf660 con 0x56369ca1a0d0 2023-11-27T15:23:57.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.959+0000 7fbd78ff9700 10 client.15648 handle_command_reply: tid=0 2023-11-27T15:23:57.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.959+0000 7fbdab3f5b80 1 client.15648 shutdown 2023-11-27T15:23:57.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.960+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/788468239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9afc00 msgr2=0x56369c9bc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:57.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.960+0000 7fbdab3f5b80 1 --2- 172.21.15.133:0/788468239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9afc00 0x56369c9bc9d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c097a60 tx=0x7fbd8c0cd040 comp rx=0 tx=0).stop 2023-11-27T15:23:57.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.960+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/788468239 shutdown_connections 2023-11-27T15:23:57.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.960+0000 7fbdab3f5b80 1 --2- 172.21.15.133:0/788468239 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56369ca1a0d0 0x56369ca1c4e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.960+0000 7fbdab3f5b80 1 --2- 172.21.15.133:0/788468239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369c9b1110 0x56369c9bff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.960+0000 7fbdab3f5b80 1 --2- 172.21.15.133:0/788468239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56369c9afc00 0x56369c9bc9d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.960+0000 7fbdab3f5b80 1 --2- 172.21.15.133:0/788468239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56369c9b0550 0x56369c9bcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.960+0000 7fbd78ff9700 0 client.15648 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:23:57.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.960+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/788468239 >> 172.21.15.133:0/788468239 conn(0x56369c9a5360 msgr2=0x56369c9b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:57.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.960+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/788468239 shutdown_connections 2023-11-27T15:23:57.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.960+0000 7fbdab3f5b80 1 -- 172.21.15.133:0/788468239 wait complete. 2023-11-27T15:23:57.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.960+0000 7fbdab3f5b80 20 client.15648 trim_cache size 0 max 16384 2023-11-27T15:23:57.969 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:23:57.970 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2023-11-27T15:23:57.970 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2023-11-27T15:23:57.970 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:23:57.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.961+0000 7fbd979ff700 1 -- 172.21.15.133:0/288221084 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbd7c0a1150 msgr2=0x7fbd7c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:57.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.961+0000 7fbd979ff700 1 --2- 172.21.15.133:0/288221084 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbd7c0a1150 0x7fbd7c0a3600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbd80011490 tx=0x7fbd80009260 comp rx=0 tx=0).stop 2023-11-27T15:23:57.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.961+0000 7fbd979ff700 1 -- 172.21.15.133:0/288221084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd901002d0 msgr2=0x7fbd90198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:23:57.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.961+0000 7fbd979ff700 1 --2- 172.21.15.133:0/288221084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd901002d0 0x7fbd90198290 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c005740 tx=0x7fbd8c005770 comp rx=0 tx=0).stop 2023-11-27T15:23:57.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.961+0000 7fbd979ff700 1 -- 172.21.15.133:0/288221084 shutdown_connections 2023-11-27T15:23:57.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.961+0000 7fbd979ff700 1 --2- 172.21.15.133:0/288221084 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbd7c0a1150 0x7fbd7c0a3600 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.961+0000 7fbd979ff700 1 --2- 172.21.15.133:0/288221084 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbd90101e50 0x7fbd9019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.961+0000 7fbd979ff700 1 --2- 172.21.15.133:0/288221084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd901002d0 0x7fbd90198290 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.961+0000 7fbd979ff700 1 --2- 172.21.15.133:0/288221084 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd901014c0 0x7fbd901987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:23:57.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.961+0000 7fbd979ff700 1 -- 172.21.15.133:0/288221084 >> 172.21.15.133:0/288221084 conn(0x7fbd900fb8b0 msgr2=0x7fbd90106db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:23:57.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.961+0000 7fbd979ff700 1 -- 172.21.15.133:0/288221084 shutdown_connections 2023-11-27T15:23:57.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:23:57.961+0000 7fbd979ff700 1 -- 172.21.15.133:0/288221084 wait complete. 2023-11-27T15:23:58.005 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:23:58.005 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:23:58.583 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:58.583 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:58.750 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:58.750 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:58.764 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:58.764 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:58.765 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 28 -lt 100 ']' 2023-11-27T15:23:58.765 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 28 + 1 2023-11-27T15:23:58.801 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=29 2023-11-27T15:23:58.801 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:58.842 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:58.842 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:58.844 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:58.845 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:58.845 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:23:59.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:23:58 smithi133 ceph-mon[97860]: pgmap v380: 129 pgs: 129 active+clean; 891 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2023-11-27T15:23:59.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:23:58 smithi177 ceph-mon[131504]: pgmap v380: 129 pgs: 129 active+clean; 891 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2023-11-27T15:23:59.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:23:58 smithi174 ceph-mon[131110]: pgmap v380: 129 pgs: 129 active+clean; 891 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2023-11-27T15:23:59.433 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:59.433 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:59.549 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:23:59.549 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:23:59.575 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:59.575 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:23:59.575 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 29 -lt 100 ']' 2023-11-27T15:23:59.576 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 29 + 1 2023-11-27T15:23:59.809 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=30 2023-11-27T15:23:59.810 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:23:59.854 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:23:59.854 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:23:59.857 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:23:59.857 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:23:59.858 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:00.700 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:00.700 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:00.969 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:00.969 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:01.042 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:01.042 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:01.042 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 30 -lt 100 ']' 2023-11-27T15:24:01.042 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 30 + 1 2023-11-27T15:24:01.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:00 smithi177 ceph-mon[131504]: pgmap v381: 129 pgs: 129 active+clean; 891 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 11 op/s 2023-11-27T15:24:01.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:00 smithi174 ceph-mon[131110]: pgmap v381: 129 pgs: 129 active+clean; 891 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 11 op/s 2023-11-27T15:24:01.193 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=31 2023-11-27T15:24:01.193 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:01.211 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:01.211 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:01.214 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:01.214 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:01.215 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:01.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:00 smithi133 ceph-mon[97860]: pgmap v381: 129 pgs: 129 active+clean; 891 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 11 op/s 2023-11-27T15:24:02.511 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:02.512 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:02.648 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:02.648 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:02.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.662+0000 7f097c9ff700 1 -- 172.21.15.133:0/2715631090 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0978100db0 msgr2=0x7f0978107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.662+0000 7f097c9ff700 1 --2- 172.21.15.133:0/2715631090 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0978100db0 0x7f0978107650 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f097000b0a0 tx=0x7f097000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.662+0000 7f097c9ff700 1 -- 172.21.15.133:0/2715631090 shutdown_connections 2023-11-27T15:24:02.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.662+0000 7f097c9ff700 1 --2- 172.21.15.133:0/2715631090 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0978100db0 0x7f0978107650 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.662+0000 7f097c9ff700 1 --2- 172.21.15.133:0/2715631090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09780ff170 0x7f09780ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.662+0000 7f097c9ff700 1 --2- 172.21.15.133:0/2715631090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0978100400 0x7f0978100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.662+0000 7f097c9ff700 1 -- 172.21.15.133:0/2715631090 >> 172.21.15.133:0/2715631090 conn(0x7f09780fa750 msgr2=0x7f09780fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:02.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.662+0000 7f097c9ff700 1 -- 172.21.15.133:0/2715631090 shutdown_connections 2023-11-27T15:24:02.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.662+0000 7f097c9ff700 1 -- 172.21.15.133:0/2715631090 wait complete. 2023-11-27T15:24:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.662+0000 7fbde15ff700 1 -- 172.21.15.133:0/3154099474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbddc1002c0 msgr2=0x7fbddc1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.662+0000 7fbde15ff700 1 --2- 172.21.15.133:0/3154099474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbddc1002c0 0x7fbddc1006d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009990 tx=0x7fbdcc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.662+0000 7fbde15ff700 1 -- 172.21.15.133:0/3154099474 shutdown_connections 2023-11-27T15:24:02.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.662+0000 7fbde15ff700 1 --2- 172.21.15.133:0/3154099474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbddc101e40 0x7fbddc108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.662+0000 7fbde15ff700 1 --2- 172.21.15.133:0/3154099474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbddc1002c0 0x7fbddc1006d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.662+0000 7fbde15ff700 1 --2- 172.21.15.133:0/3154099474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbddc1014b0 0x7fbddc101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.662+0000 7fbde15ff700 1 -- 172.21.15.133:0/3154099474 >> 172.21.15.133:0/3154099474 conn(0x7fbddc0fb8e0 msgr2=0x7fbddc0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.663+0000 7fbde15ff700 1 -- 172.21.15.133:0/3154099474 shutdown_connections 2023-11-27T15:24:02.671 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.663+0000 7fbde15ff700 1 -- 172.21.15.133:0/3154099474 wait complete. 2023-11-27T15:24:02.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.663+0000 7f097c9ff700 1 Processor -- start 2023-11-27T15:24:02.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.664+0000 7f097c9ff700 1 -- start start 2023-11-27T15:24:02.672 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.664+0000 7fbde15ff700 1 Processor -- start 2023-11-27T15:24:02.673 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.664+0000 7f097c9ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09780ff170 0x7f09781960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.664+0000 7f097c9ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0978100400 0x7f09781965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.664+0000 7f097c9ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0978100db0 0x7f097819d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.664+0000 7f097c9ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097819dc30 con 0x7f0978100400 2023-11-27T15:24:02.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.664+0000 7f097c9ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097819dd70 con 0x7f0978100db0 2023-11-27T15:24:02.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.664+0000 7f097c9ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097819df00 con 0x7f09780ff170 2023-11-27T15:24:02.675 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.664+0000 7fbde15ff700 1 -- start start 2023-11-27T15:24:02.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.665+0000 7f0975d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0978100400 0x7f09781965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.665+0000 7f097659c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09780ff170 0x7f09781960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.665+0000 7f0976d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0978100db0 0x7f097819d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.665+0000 7f097659c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09780ff170 0x7f09781960b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:47728/0 (socket says 172.21.15.133:47728) 2023-11-27T15:24:02.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.665+0000 7f097659c700 1 -- 172.21.15.133:0/2552643694 learned_addr learned my addr 172.21.15.133:0/2552643694 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:02.677 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.665+0000 7fbde15ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbddc1002c0 0x7fbddc0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.665+0000 7fbde15ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbddc1014b0 0x7fbddc072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.665+0000 7fbde15ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbddc101e40 0x7fbddc073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.665+0000 7fbde15ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc073bc0 con 0x7fbddc1014b0 2023-11-27T15:24:02.678 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.665+0000 7fbde15ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc073d30 con 0x7fbddc1002c0 2023-11-27T15:24:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.665+0000 7fbde15ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc1a0070 con 0x7fbddc101e40 2023-11-27T15:24:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.665+0000 7fbddaffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbddc1002c0 0x7fbddc0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.665+0000 7fbddaffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbddc1002c0 0x7fbddc0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:55518/0 (socket says 172.21.15.133:55518) 2023-11-27T15:24:02.679 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.665+0000 7fbddaffd700 1 -- 172.21.15.133:0/192312834 learned_addr learned my addr 172.21.15.133:0/192312834 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:02.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.665+0000 7f097659c700 1 -- 172.21.15.133:0/2552643694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0978100db0 msgr2=0x7f097819d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.665+0000 7f097659c700 1 --2- 172.21.15.133:0/2552643694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0978100db0 0x7f097819d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.665+0000 7f097659c700 1 -- 172.21.15.133:0/2552643694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0978100400 msgr2=0x7f09781965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.665+0000 7f097659c700 1 --2- 172.21.15.133:0/2552643694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0978100400 0x7f09781965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.665+0000 7f097659c700 1 -- 172.21.15.133:0/2552643694 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0968009520 con 0x7f09780ff170 2023-11-27T15:24:02.681 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.665+0000 7fbddb7fe700 1 --2- 172.21.15.133:0/192312834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbddc101e40 0x7fbddc073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.665+0000 7f0976d9d700 1 --2- 172.21.15.133:0/2552643694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0978100db0 0x7f097819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:02.682 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.665+0000 7fbdd3fff700 1 --2- 172.21.15.133:0/192312834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbddc1014b0 0x7fbddc072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.665+0000 7f097659c700 1 --2- 172.21.15.133:0/2552643694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09780ff170 0x7f09781960b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f096800eb30 tx=0x7f096800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:02.683 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.666+0000 7fbddaffd700 1 -- 172.21.15.133:0/192312834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbddc101e40 msgr2=0x7fbddc073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.666+0000 7fbddaffd700 1 --2- 172.21.15.133:0/192312834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbddc101e40 0x7fbddc073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.666+0000 7fbddaffd700 1 -- 172.21.15.133:0/192312834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbddc1014b0 msgr2=0x7fbddc072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.666+0000 7fbddaffd700 1 --2- 172.21.15.133:0/192312834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbddc1014b0 0x7fbddc072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.684 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.666+0000 7fbddaffd700 1 -- 172.21.15.133:0/192312834 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc4009520 con 0x7fbddc1002c0 2023-11-27T15:24:02.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.666+0000 7f09677fe700 1 -- 172.21.15.133:0/2552643694 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0968009880 con 0x7f09780ff170 2023-11-27T15:24:02.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.666+0000 7f097c9ff700 1 -- 172.21.15.133:0/2552643694 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0970009d00 con 0x7f09780ff170 2023-11-27T15:24:02.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.666+0000 7f09677fe700 1 -- 172.21.15.133:0/2552643694 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09680044e0 con 0x7f09780ff170 2023-11-27T15:24:02.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.666+0000 7f09677fe700 1 -- 172.21.15.133:0/2552643694 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0968003d80 con 0x7f09780ff170 2023-11-27T15:24:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.666+0000 7fbddaffd700 1 --2- 172.21.15.133:0/192312834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbddc1002c0 0x7fbddc0748a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc004730 tx=0x7fbdcc004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.666+0000 7fbdd8ff9700 1 -- 172.21.15.133:0/192312834 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc01d070 con 0x7fbddc1002c0 2023-11-27T15:24:02.686 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.666+0000 7fbdd8ff9700 1 -- 172.21.15.133:0/192312834 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdcc021410 con 0x7fbddc1002c0 2023-11-27T15:24:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.666+0000 7fbdd8ff9700 1 -- 172.21.15.133:0/192312834 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc017650 con 0x7fbddc1002c0 2023-11-27T15:24:02.687 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.667+0000 7fbde15ff700 1 -- 172.21.15.133:0/192312834 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdcc0095f0 con 0x7fbddc1002c0 2023-11-27T15:24:02.687 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.666+0000 7f097c9ff700 1 -- 172.21.15.133:0/2552643694 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097810ab70 con 0x7f09780ff170 2023-11-27T15:24:02.688 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.667+0000 7fbde15ff700 1 -- 172.21.15.133:0/192312834 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbddc10bc00 con 0x7fbddc1002c0 2023-11-27T15:24:02.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.667+0000 7f097c9ff700 1 -- 172.21.15.133:0/2552643694 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0978067a60 con 0x7f09780ff170 2023-11-27T15:24:02.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.669+0000 7f09677fe700 1 -- 172.21.15.133:0/2552643694 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0968003f20 con 0x7f09780ff170 2023-11-27T15:24:02.689 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.671+0000 7fbdd8ff9700 1 -- 172.21.15.133:0/192312834 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fbdcc021580 con 0x7fbddc1002c0 2023-11-27T15:24:02.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.671+0000 7f09677fe700 1 --2- 172.21.15.133:0/2552643694 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f09600a11d0 0x7f09600a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.671+0000 7f09677fe700 1 -- 172.21.15.133:0/2552643694 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f0968014070 con 0x7f09780ff170 2023-11-27T15:24:02.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.671+0000 7f0975d9b700 1 --2- 172.21.15.133:0/2552643694 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f09600a11d0 0x7f09600a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.672+0000 7f0975d9b700 1 --2- 172.21.15.133:0/2552643694 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f09600a11d0 0x7f09600a3680 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f096c009be0 tx=0x7f096c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.672+0000 7fbdd8ff9700 1 --2- 172.21.15.133:0/192312834 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbdc80a11d0 0x7fbdc80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.672+0000 7fbdd8ff9700 1 -- 172.21.15.133:0/192312834 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbdcc0cdfc0 con 0x7fbddc1002c0 2023-11-27T15:24:02.691 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.673+0000 7fbdd3fff700 1 --2- 172.21.15.133:0/192312834 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbdc80a11d0 0x7fbdc80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.692 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.673+0000 7fbdd3fff700 1 --2- 172.21.15.133:0/192312834 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbdc80a11d0 0x7fbdc80a3680 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009df0 tx=0x7fbdc4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:02.692 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:02.692 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:02.693 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 31 -lt 100 ']' 2023-11-27T15:24:02.693 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 31 + 1 2023-11-27T15:24:02.712 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.707+0000 7f09677fe700 1 -- 172.21.15.133:0/2552643694 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0968096530 con 0x7f09780ff170 2023-11-27T15:24:02.742 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=32 2023-11-27T15:24:02.742 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.773+0000 7fbdf4ef9b80 1 Processor -- start 2023-11-27T15:24:02.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.773+0000 7fbdf4ef9b80 1 -- start start 2023-11-27T15:24:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.773+0000 7fbdf4ef9b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89dcf6b0 0x55ba89f73870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.773+0000 7fbdf4ef9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89f73db0 0x55ba89f745d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.773+0000 7fbdf4ef9b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 0x55ba89f74f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.773+0000 7fbdf4ef9b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba89e79000 con 0x55ba89f74b10 2023-11-27T15:24:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.773+0000 7fbdf4ef9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba89ecb840 con 0x55ba89f73db0 2023-11-27T15:24:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.773+0000 7fbdf4ef9b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba89ec4cd0 con 0x55ba89dcf6b0 2023-11-27T15:24:02.780 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.774+0000 7fbddb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 0x55ba89f74f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.774+0000 7fbddaffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89dcf6b0 0x55ba89f73870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.774+0000 7fbdd3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89f73db0 0x55ba89f745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.774+0000 7fbdd3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89f73db0 0x55ba89f745d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:55534/0 (socket says 172.21.15.133:55534) 2023-11-27T15:24:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.774+0000 7fbddaffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89dcf6b0 0x55ba89f73870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:47756/0 (socket says 172.21.15.133:47756) 2023-11-27T15:24:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.774+0000 7fbdd3fff700 1 -- 172.21.15.133:0/716258557 learned_addr learned my addr 172.21.15.133:0/716258557 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.774+0000 7fbdd3fff700 1 -- 172.21.15.133:0/716258557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89dcf6b0 msgr2=0x55ba89f73870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:02.781 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.774+0000 7fbdd3fff700 1 --2- 172.21.15.133:0/716258557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89dcf6b0 0x55ba89f73870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.774+0000 7fbdd3fff700 1 -- 172.21.15.133:0/716258557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 msgr2=0x55ba89f74f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.774+0000 7fbdd3fff700 1 --2- 172.21.15.133:0/716258557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 0x55ba89f74f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.774+0000 7fbdd3fff700 1 -- 172.21.15.133:0/716258557 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba89d465d0 con 0x55ba89f73db0 2023-11-27T15:24:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.774+0000 7fbddb7fe700 1 --2- 172.21.15.133:0/716258557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 0x55ba89f74f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.774+0000 7fbdd3fff700 1 --2- 172.21.15.133:0/716258557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89f73db0 0x55ba89f745d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40094f0 tx=0x7fbdc4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=254b060a2bc78671 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.775+0000 7fbdd17fa700 1 -- 172.21.15.133:0/716258557 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4026630 con 0x55ba89f73db0 2023-11-27T15:24:02.782 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.775+0000 7fbdd17fa700 1 -- 172.21.15.133:0/716258557 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdc4021070 con 0x55ba89f73db0 2023-11-27T15:24:02.783 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.775+0000 7fbdd17fa700 1 -- 172.21.15.133:0/716258557 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc401d700 con 0x55ba89f73db0 2023-11-27T15:24:02.788 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.775+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/716258557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89f73db0 msgr2=0x55ba89f745d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.775+0000 7fbdf4ef9b80 1 --2- 172.21.15.133:0/716258557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89f73db0 0x55ba89f745d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40094f0 tx=0x7fbdc4014000 comp rx=0 tx=0).stop 2023-11-27T15:24:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.776+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/716258557 shutdown_connections 2023-11-27T15:24:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.776+0000 7fbdf4ef9b80 1 --2- 172.21.15.133:0/716258557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 0x55ba89f74f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.776+0000 7fbdf4ef9b80 1 --2- 172.21.15.133:0/716258557 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89dcf6b0 0x55ba89f73870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.776+0000 7fbdf4ef9b80 1 --2- 172.21.15.133:0/716258557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89f73db0 0x55ba89f745d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.776+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/716258557 >> 172.21.15.133:0/716258557 conn(0x55ba89f68f50 msgr2=0x55ba89f6e100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.776+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/716258557 shutdown_connections 2023-11-27T15:24:02.789 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.776+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/716258557 wait complete. 2023-11-27T15:24:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.776+0000 7fbdf4ef9b80 1 -- start start 2023-11-27T15:24:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.777+0000 7fbdf4ef9b80 1 Processor -- start 2023-11-27T15:24:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.777+0000 7fbdf4ef9b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89dcf6b0 0x55ba89f7ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.777+0000 7fbdf4ef9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89f73db0 0x55ba89f7d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.777+0000 7fbdf4ef9b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 0x55ba89f84230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.777+0000 7fbdf4ef9b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba89e66880 con 0x55ba89f74b10 2023-11-27T15:24:02.790 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.777+0000 7fbdf4ef9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba89e66520 con 0x55ba89f73db0 2023-11-27T15:24:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.777+0000 7fbdf4ef9b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba89d488c0 con 0x55ba89dcf6b0 2023-11-27T15:24:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.777+0000 7fbdf4ef9b80 10 client.0 _ll_register_callbacks cb 0x55ba89e84720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.778+0000 7fbddb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 0x55ba89f84230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.778+0000 7fbddb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 0x55ba89f84230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60928/0 (socket says 172.21.15.133:60928) 2023-11-27T15:24:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.778+0000 7fbddb7fe700 1 -- 172.21.15.133:0/1962940474 learned_addr learned my addr 172.21.15.133:0/1962940474 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.778+0000 7fbdd3fff700 1 --2- 172.21.15.133:0/1962940474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89f73db0 0x55ba89f7d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.778+0000 7fbddaffd700 1 --2- 172.21.15.133:0/1962940474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89dcf6b0 0x55ba89f7ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.791 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.778+0000 7fbddb7fe700 1 -- 172.21.15.133:0/1962940474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89dcf6b0 msgr2=0x55ba89f7ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.778+0000 7fbddb7fe700 1 --2- 172.21.15.133:0/1962940474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89dcf6b0 0x55ba89f7ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.778+0000 7fbddb7fe700 1 -- 172.21.15.133:0/1962940474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89f73db0 msgr2=0x55ba89f7d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.778+0000 7fbddb7fe700 1 --2- 172.21.15.133:0/1962940474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89f73db0 0x55ba89f7d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.778+0000 7fbddb7fe700 1 -- 172.21.15.133:0/1962940474 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc4017070 con 0x55ba89f74b10 2023-11-27T15:24:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.778+0000 7fbddb7fe700 1 --2- 172.21.15.133:0/1962940474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 0x55ba89f84230 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4011ef0 tx=0x7fbdd4014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3864ff81feb147 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.778+0000 7fbdc27fc700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:24:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.779+0000 7fbdc27fc700 1 -- 172.21.15.133:0/1962940474 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd400e040 con 0x55ba89f74b10 2023-11-27T15:24:02.792 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.779+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/1962940474 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba89f66710 con 0x55ba89f74b10 2023-11-27T15:24:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.779+0000 7fbdc27fc700 1 -- 172.21.15.133:0/1962940474 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbdd401a070 con 0x55ba89f74b10 2023-11-27T15:24:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.779+0000 7fbdc27fc700 1 -- 172.21.15.133:0/1962940474 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdd40093a0 con 0x55ba89f74b10 2023-11-27T15:24:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.779+0000 7fbdf4ef9b80 10 client.15666 fetch_fsmap learned FSMap version 23 2023-11-27T15:24:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.779+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/1962940474 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ba89e8b130 con 0x55ba89f74b10 2023-11-27T15:24:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.779+0000 7fbdc27fc700 1 -- 172.21.15.133:0/1962940474 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba89f66710 con 0x55ba89f74b10 2023-11-27T15:24:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.779+0000 7fbdc27fc700 1 -- 172.21.15.133:0/1962940474 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbdd4004500 con 0x55ba89f74b10 2023-11-27T15:24:02.793 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.780+0000 7fbdf4ef9b80 10 client.15666 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:24:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.780+0000 7fbdf4ef9b80 10 client.15666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.780+0000 7fbdf4ef9b80 20 client.15666 populate_metadata read hostname 'smithi133' 2023-11-27T15:24:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.780+0000 7fbdf4ef9b80 1 --2- 172.21.15.133:0/1962940474 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ba89f88b10 0x55ba89f8af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.780+0000 7fbdf4ef9b80 4 client.15666 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.780+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/1962940474 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba89e87040 con 0x55ba89f88b10 2023-11-27T15:24:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.780+0000 7fbddaffd700 1 --2- 172.21.15.133:0/1962940474 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ba89f88b10 0x55ba89f8af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.780+0000 7fbddaffd700 1 --2- 172.21.15.133:0/1962940474 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ba89f88b10 0x55ba89f8af20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37714bf900a11d25 server_cookie=d5a941635f663cea in_seq=0 out_seq=0 2023-11-27T15:24:02.794 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.780+0000 7fbdc27fc700 10 client.15666 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.781+0000 7fbdc27fc700 1 -- 172.21.15.133:0/1962940474 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ba89e87040 con 0x55ba89f88b10 2023-11-27T15:24:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.781+0000 7fbdc27fc700 10 client.15666 handle_command_reply: tid=0 2023-11-27T15:24:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.781+0000 7fbdf4ef9b80 1 client.15666 shutdown 2023-11-27T15:24:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.781+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/1962940474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 msgr2=0x55ba89f84230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.781+0000 7fbdf4ef9b80 1 --2- 172.21.15.133:0/1962940474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 0x55ba89f84230 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4011ef0 tx=0x7fbdd4014600 comp rx=0 tx=0).stop 2023-11-27T15:24:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.782+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/1962940474 shutdown_connections 2023-11-27T15:24:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.782+0000 7fbdf4ef9b80 1 --2- 172.21.15.133:0/1962940474 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ba89f88b10 0x55ba89f8af20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.782+0000 7fbdf4ef9b80 1 --2- 172.21.15.133:0/1962940474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 0x55ba89f84230 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.782+0000 7fbdf4ef9b80 1 --2- 172.21.15.133:0/1962940474 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89dcf6b0 0x55ba89f7ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.795 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.782+0000 7fbdf4ef9b80 1 --2- 172.21.15.133:0/1962940474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89f73db0 0x55ba89f7d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.782+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/1962940474 >> 172.21.15.133:0/1962940474 conn(0x55ba89f68f50 msgr2=0x55ba89f6d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.782+0000 7fbdc27fc700 0 client.15666 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.782+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/1962940474 shutdown_connections 2023-11-27T15:24:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.782+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/1962940474 wait complete. 2023-11-27T15:24:02.796 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.782+0000 7fbdf4ef9b80 20 client.15666 trim_cache size 0 max 16384 2023-11-27T15:24:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.797+0000 7fbdf4ef9b80 1 Processor -- start 2023-11-27T15:24:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.797+0000 7fbdf4ef9b80 1 -- start start 2023-11-27T15:24:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.797+0000 7fbdf4ef9b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89dcf6b0 0x55ba89f7ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.803 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.797+0000 7fbdf4ef9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89f73db0 0x55ba89f7b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.797+0000 7fbdf4ef9b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 0x55ba89f84150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.797+0000 7fbdf4ef9b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba89f6afd0 con 0x55ba89f74b10 2023-11-27T15:24:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.797+0000 7fbdf4ef9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba89f74420 con 0x55ba89f73db0 2023-11-27T15:24:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.797+0000 7fbdf4ef9b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba89f85f20 con 0x55ba89dcf6b0 2023-11-27T15:24:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.797+0000 7fbddb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 0x55ba89f84150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.797+0000 7fbddaffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89dcf6b0 0x55ba89f7ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.798+0000 7fbddaffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89dcf6b0 0x55ba89f7ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:47768/0 (socket says 172.21.15.133:47768) 2023-11-27T15:24:02.804 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.798+0000 7fbddaffd700 1 -- 172.21.15.133:0/4283763241 learned_addr learned my addr 172.21.15.133:0/4283763241 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.798+0000 7fbddb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 0x55ba89f84150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60944/0 (socket says 172.21.15.133:60944) 2023-11-27T15:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.798+0000 7fbdd3fff700 1 --2- 172.21.15.133:0/4283763241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89f73db0 0x55ba89f7b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.798+0000 7fbddaffd700 1 -- 172.21.15.133:0/4283763241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89f73db0 msgr2=0x55ba89f7b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.798+0000 7fbddaffd700 1 --2- 172.21.15.133:0/4283763241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89f73db0 0x55ba89f7b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.798+0000 7fbddaffd700 1 -- 172.21.15.133:0/4283763241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 msgr2=0x55ba89f84150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.798+0000 7fbddaffd700 1 --2- 172.21.15.133:0/4283763241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 0x55ba89f84150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.798+0000 7fbddaffd700 1 -- 172.21.15.133:0/4283763241 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba89e8ae80 con 0x55ba89dcf6b0 2023-11-27T15:24:02.805 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.798+0000 7fbddaffd700 1 --2- 172.21.15.133:0/4283763241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89dcf6b0 0x55ba89f7ae50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc0cb770 tx=0x7fbdcc0cd040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=483e21d8ad48e2c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.798+0000 7fbdd1ffb700 1 -- 172.21.15.133:0/4283763241 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc09f140 con 0x55ba89dcf6b0 2023-11-27T15:24:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.798+0000 7fbdd1ffb700 1 -- 172.21.15.133:0/4283763241 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdcc0a34d0 con 0x55ba89dcf6b0 2023-11-27T15:24:02.806 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.798+0000 7fbdd1ffb700 1 -- 172.21.15.133:0/4283763241 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc09b5d0 con 0x55ba89dcf6b0 2023-11-27T15:24:02.809 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.799+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/4283763241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89dcf6b0 msgr2=0x55ba89f7ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.799+0000 7fbdf4ef9b80 1 --2- 172.21.15.133:0/4283763241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89dcf6b0 0x55ba89f7ae50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc0cb770 tx=0x7fbdcc0cd040 comp rx=0 tx=0).stop 2023-11-27T15:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.799+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/4283763241 shutdown_connections 2023-11-27T15:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.799+0000 7fbdf4ef9b80 1 --2- 172.21.15.133:0/4283763241 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 0x55ba89f84150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.799+0000 7fbdf4ef9b80 1 --2- 172.21.15.133:0/4283763241 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89dcf6b0 0x55ba89f7ae50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.799+0000 7fbdf4ef9b80 1 --2- 172.21.15.133:0/4283763241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89f73db0 0x55ba89f7b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.799+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/4283763241 >> 172.21.15.133:0/4283763241 conn(0x55ba89f68f50 msgr2=0x55ba89f8f880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.799+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/4283763241 shutdown_connections 2023-11-27T15:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.799+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/4283763241 wait complete. 2023-11-27T15:24:02.810 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.800+0000 7fbdf4ef9b80 1 -- start start 2023-11-27T15:24:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.800+0000 7fbdf4ef9b80 1 Processor -- start 2023-11-27T15:24:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.800+0000 7fbdf4ef9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89dcf6b0 0x55ba89f7f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.800+0000 7fbdf4ef9b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89f73db0 0x55ba89f7fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.800+0000 7fbdf4ef9b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 0x55ba89f82ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.800+0000 7fbdf4ef9b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba89f831e0 con 0x55ba89f74b10 2023-11-27T15:24:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.800+0000 7fbdf4ef9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba89f83350 con 0x55ba89dcf6b0 2023-11-27T15:24:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.800+0000 7fbdf4ef9b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba89f83510 con 0x55ba89f73db0 2023-11-27T15:24:02.811 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.800+0000 7fbdf4ef9b80 10 client.0 _ll_register_callbacks cb 0x7fbdd4011820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.800+0000 7fbddaffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89dcf6b0 0x55ba89f7f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.800+0000 7fbddaffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89dcf6b0 0x55ba89f7f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:55566/0 (socket says 172.21.15.133:55566) 2023-11-27T15:24:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.800+0000 7fbddaffd700 1 -- 172.21.15.133:0/1264792577 learned_addr learned my addr 172.21.15.133:0/1264792577 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.801+0000 7fbddb7fe700 1 --2- 172.21.15.133:0/1264792577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 0x55ba89f82ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.801+0000 7fbdd3fff700 1 --2- 172.21.15.133:0/1264792577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89f73db0 0x55ba89f7fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.801+0000 7fbddaffd700 1 -- 172.21.15.133:0/1264792577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89f73db0 msgr2=0x55ba89f7fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.801+0000 7fbddaffd700 1 --2- 172.21.15.133:0/1264792577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89f73db0 0x55ba89f7fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.801+0000 7fbddaffd700 1 -- 172.21.15.133:0/1264792577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 msgr2=0x55ba89f82ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:02.812 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.801+0000 7fbddaffd700 1 --2- 172.21.15.133:0/1264792577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 0x55ba89f82ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.801+0000 7fbddaffd700 1 -- 172.21.15.133:0/1264792577 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbddc10bc00 con 0x55ba89dcf6b0 2023-11-27T15:24:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.801+0000 7fbddb7fe700 1 --2- 172.21.15.133:0/1264792577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 0x55ba89f82ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.801+0000 7fbddaffd700 1 --2- 172.21.15.133:0/1264792577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89dcf6b0 0x55ba89f7f740 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc09ae10 tx=0x7fbdcc021a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a4ef97f8480b9fe server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.801+0000 7fbdc27fc700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:24:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.801+0000 7fbdc27fc700 1 -- 172.21.15.133:0/1264792577 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc0cb470 con 0x55ba89dcf6b0 2023-11-27T15:24:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.801+0000 7fbdc27fc700 1 -- 172.21.15.133:0/1264792577 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fbdcc09a600 con 0x55ba89dcf6b0 2023-11-27T15:24:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.801+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/1264792577 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbdc4021070 con 0x55ba89dcf6b0 2023-11-27T15:24:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.801+0000 7fbdc27fc700 1 -- 172.21.15.133:0/1264792577 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdcc09a800 con 0x55ba89dcf6b0 2023-11-27T15:24:02.813 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.801+0000 7fbdf4ef9b80 10 client.25712 fetch_fsmap learned FSMap version 23 2023-11-27T15:24:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.801+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/1264792577 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ba89cdbc00 con 0x55ba89dcf6b0 2023-11-27T15:24:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.802+0000 7fbdc27fc700 1 -- 172.21.15.133:0/1264792577 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdc4021070 con 0x55ba89dcf6b0 2023-11-27T15:24:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.802+0000 7fbdc27fc700 1 -- 172.21.15.133:0/1264792577 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbdcc0b91d0 con 0x55ba89dcf6b0 2023-11-27T15:24:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.802+0000 7fbdf4ef9b80 10 client.25712 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:24:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.802+0000 7fbdf4ef9b80 10 client.25712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.802+0000 7fbdf4ef9b80 20 client.25712 populate_metadata read hostname 'smithi133' 2023-11-27T15:24:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.802+0000 7fbdf4ef9b80 1 --2- 172.21.15.133:0/1264792577 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ba89fdcde0 0x55ba89fdf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.802+0000 7fbdf4ef9b80 4 client.25712 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:24:02.814 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.802+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/1264792577 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ba89e93600 con 0x55ba89fdcde0 2023-11-27T15:24:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.802+0000 7fbdd3fff700 1 --2- 172.21.15.133:0/1264792577 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ba89fdcde0 0x55ba89fdf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.803+0000 7fbdc27fc700 10 client.25712 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:02.815 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.803+0000 7fbdd3fff700 1 --2- 172.21.15.133:0/1264792577 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ba89fdcde0 0x55ba89fdf1f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a7052ec7ee850d1 server_cookie=678fb121f72f4c4d in_seq=0 out_seq=0 2023-11-27T15:24:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.819+0000 7fbdc27fc700 1 -- 172.21.15.133:0/1264792577 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ba89e93600 con 0x55ba89fdcde0 2023-11-27T15:24:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.819+0000 7fbdc27fc700 10 client.25712 handle_command_reply: tid=0 2023-11-27T15:24:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.820+0000 7fbdf4ef9b80 1 client.25712 shutdown 2023-11-27T15:24:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.820+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/1264792577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89dcf6b0 msgr2=0x55ba89f7f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.825 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.820+0000 7fbdf4ef9b80 1 --2- 172.21.15.133:0/1264792577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89dcf6b0 0x55ba89f7f740 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc09ae10 tx=0x7fbdcc021a70 comp rx=0 tx=0).stop 2023-11-27T15:24:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.820+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/1264792577 shutdown_connections 2023-11-27T15:24:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.820+0000 7fbdf4ef9b80 1 --2- 172.21.15.133:0/1264792577 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ba89fdcde0 0x55ba89fdf1f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.820+0000 7fbdf4ef9b80 1 --2- 172.21.15.133:0/1264792577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ba89f74b10 0x55ba89f82ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.820+0000 7fbdf4ef9b80 1 --2- 172.21.15.133:0/1264792577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba89dcf6b0 0x55ba89f7f740 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.820+0000 7fbdf4ef9b80 1 --2- 172.21.15.133:0/1264792577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ba89f73db0 0x55ba89f7fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.820+0000 7fbdc27fc700 0 client.25712 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.820+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/1264792577 >> 172.21.15.133:0/1264792577 conn(0x55ba89f68f50 msgr2=0x55ba89f7ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.820+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/1264792577 shutdown_connections 2023-11-27T15:24:02.826 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.820+0000 7fbdf4ef9b80 1 -- 172.21.15.133:0/1264792577 wait complete. 2023-11-27T15:24:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.820+0000 7fbdf4ef9b80 20 client.25712 trim_cache size 0 max 16384 2023-11-27T15:24:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:24:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:24:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:24:02.827 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:24:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.821+0000 7fbde15ff700 1 -- 172.21.15.133:0/192312834 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbdc80a11d0 msgr2=0x7fbdc80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.821+0000 7fbde15ff700 1 --2- 172.21.15.133:0/192312834 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbdc80a11d0 0x7fbdc80a3680 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009df0 tx=0x7fbdc4009260 comp rx=0 tx=0).stop 2023-11-27T15:24:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.821+0000 7fbde15ff700 1 -- 172.21.15.133:0/192312834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbddc1002c0 msgr2=0x7fbddc0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.821+0000 7fbde15ff700 1 --2- 172.21.15.133:0/192312834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbddc1002c0 0x7fbddc0748a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc004730 tx=0x7fbdcc004760 comp rx=0 tx=0).stop 2023-11-27T15:24:02.828 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.821+0000 7fbde15ff700 1 -- 172.21.15.133:0/192312834 shutdown_connections 2023-11-27T15:24:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.821+0000 7fbde15ff700 1 --2- 172.21.15.133:0/192312834 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fbdc80a11d0 0x7fbdc80a3680 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.821+0000 7fbde15ff700 1 --2- 172.21.15.133:0/192312834 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbddc101e40 0x7fbddc073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.821+0000 7fbde15ff700 1 --2- 172.21.15.133:0/192312834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbddc1002c0 0x7fbddc0748a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.821+0000 7fbde15ff700 1 --2- 172.21.15.133:0/192312834 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbddc1014b0 0x7fbddc072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.821+0000 7fbde15ff700 1 -- 172.21.15.133:0/192312834 >> 172.21.15.133:0/192312834 conn(0x7fbddc0fb8e0 msgr2=0x7fbddc1049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.821+0000 7fbde15ff700 1 -- 172.21.15.133:0/192312834 shutdown_connections 2023-11-27T15:24:02.829 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:02.821+0000 7fbde15ff700 1 -- 172.21.15.133:0/192312834 wait complete. 2023-11-27T15:24:02.830 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:02.830 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:02.831 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:02.832 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:02.833 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:02.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.969+0000 7f097c9ff700 1 -- 172.21.15.133:0/2552643694 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f097819eac0 con 0x7f09780ff170 2023-11-27T15:24:02.975 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.971+0000 7f09677fe700 1 -- 172.21.15.133:0/2552643694 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10829 (secure 0 0 0) 0x7f0968099b50 con 0x7f09780ff170 2023-11-27T15:24:02.975 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:24:02.976 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14493,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2381290259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2381290259},{"type":"v1","addr":"172.21.15.133:6839","nonce":2381290259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1704875895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1704875895},{"type":"v1","addr":"172.21.15.133:6841","nonce":1704875895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:20:33.201023+0000","modified":"2023-11-27T15:21:04.794526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24521,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3492215689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3492215689},{"type":"v1","addr":"172.21.15.177:6837","nonce":3492215689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6839/2781994447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2781994447},{"type":"v1","addr":"172.21.15.174:6839","nonce":2781994447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:24:02.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.973+0000 7f097c9ff700 1 -- 172.21.15.133:0/2552643694 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f09600a11d0 msgr2=0x7f09600a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.973+0000 7f097c9ff700 1 --2- 172.21.15.133:0/2552643694 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f09600a11d0 0x7f09600a3680 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f096c009be0 tx=0x7f096c009260 comp rx=0 tx=0).stop 2023-11-27T15:24:02.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.973+0000 7f097c9ff700 1 -- 172.21.15.133:0/2552643694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09780ff170 msgr2=0x7f09781960b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:02.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.973+0000 7f097c9ff700 1 --2- 172.21.15.133:0/2552643694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09780ff170 0x7f09781960b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f096800eb30 tx=0x7f096800ee40 comp rx=0 tx=0).stop 2023-11-27T15:24:02.978 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.973+0000 7f097c9ff700 1 -- 172.21.15.133:0/2552643694 shutdown_connections 2023-11-27T15:24:02.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.973+0000 7f097c9ff700 1 --2- 172.21.15.133:0/2552643694 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f09600a11d0 0x7f09600a3680 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.973+0000 7f097c9ff700 1 --2- 172.21.15.133:0/2552643694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0978100db0 0x7f097819d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.973+0000 7f097c9ff700 1 --2- 172.21.15.133:0/2552643694 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09780ff170 0x7f09781960b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.973+0000 7f097c9ff700 1 --2- 172.21.15.133:0/2552643694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0978100400 0x7f09781965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:02.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.973+0000 7f097c9ff700 1 -- 172.21.15.133:0/2552643694 >> 172.21.15.133:0/2552643694 conn(0x7f09780fa750 msgr2=0x7f0978067e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:02.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.973+0000 7f097c9ff700 1 -- 172.21.15.133:0/2552643694 shutdown_connections 2023-11-27T15:24:02.979 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:02.973+0000 7f097c9ff700 1 -- 172.21.15.133:0/2552643694 wait complete. 2023-11-27T15:24:02.980 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 23 2023-11-27T15:24:02.999 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:24:03.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:02 smithi177 ceph-mon[131504]: pgmap v382: 129 pgs: 129 active+clean; 952 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-11-27T15:24:03.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:02 smithi174 ceph-mon[131110]: pgmap v382: 129 pgs: 129 active+clean; 952 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-11-27T15:24:03.199 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:02 smithi133 ceph-mon[97860]: pgmap v382: 129 pgs: 129 active+clean; 952 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-11-27T15:24:03.466 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:03.466 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:03.490 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:03.490 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:03.523 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:03.523 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:03.523 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 32 -lt 100 ']' 2023-11-27T15:24:03.524 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 32 + 1 2023-11-27T15:24:03.575 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=33 2023-11-27T15:24:03.575 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:03.582 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:03.583 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:03.586 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:03.590 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:03.590 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:03.982 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:03.983 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:04.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:03 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2552643694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:24:04.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:03 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:24:04.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:03 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:24:04.083 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:04.083 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:04.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:03 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2552643694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:24:04.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:03 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:24:04.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:03 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:24:04.149 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:04.149 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:04.149 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 33 -lt 100 ']' 2023-11-27T15:24:04.149 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 33 + 1 2023-11-27T15:24:04.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:03 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2552643694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:24:04.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:03 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:24:04.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:03 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:24:04.208 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=34 2023-11-27T15:24:04.208 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:04.273 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:04.273 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:04.275 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:04.276 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:04.276 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:05.245 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:05 smithi133 ceph-mon[97860]: pgmap v383: 129 pgs: 129 active+clean; 952 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 8 op/s 2023-11-27T15:24:05.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:05 smithi174 ceph-mon[131110]: pgmap v383: 129 pgs: 129 active+clean; 952 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 8 op/s 2023-11-27T15:24:05.407 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:05.408 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:05.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:05 smithi177 ceph-mon[131504]: pgmap v383: 129 pgs: 129 active+clean; 952 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 8 op/s 2023-11-27T15:24:05.810 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:05.810 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:06.232 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:06.233 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:06.233 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 34 -lt 100 ']' 2023-11-27T15:24:06.233 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 34 + 1 2023-11-27T15:24:06.533 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=35 2023-11-27T15:24:06.534 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:07.216 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:07.216 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:07.219 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:07.219 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:07.220 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:07.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:07 smithi174 ceph-mon[131110]: pgmap v384: 129 pgs: 129 active+clean; 952 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-11-27T15:24:07.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.461+0000 7f3f6b7ff700 1 -- 172.21.15.133:0/547394885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f641002c0 msgr2=0x7f3f641006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:07.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.461+0000 7f3f6b7ff700 1 --2- 172.21.15.133:0/547394885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f641002c0 0x7f3f641006d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3f60009990 tx=0x7f3f60009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.461+0000 7f3f6b7ff700 1 -- 172.21.15.133:0/547394885 shutdown_connections 2023-11-27T15:24:07.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.461+0000 7f3f6b7ff700 1 --2- 172.21.15.133:0/547394885 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f64101e40 0x7f3f64108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.461+0000 7f3f6b7ff700 1 --2- 172.21.15.133:0/547394885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f641002c0 0x7f3f641006d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.461+0000 7f3f6b7ff700 1 --2- 172.21.15.133:0/547394885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f641014b0 0x7f3f64101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.461+0000 7f3f6b7ff700 1 -- 172.21.15.133:0/547394885 >> 172.21.15.133:0/547394885 conn(0x7f3f640fb8a0 msgr2=0x7f3f640fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:07.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.462+0000 7f3f6b7ff700 1 -- 172.21.15.133:0/547394885 shutdown_connections 2023-11-27T15:24:07.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.462+0000 7f3f6b7ff700 1 -- 172.21.15.133:0/547394885 wait complete. 2023-11-27T15:24:07.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.463+0000 7f3f6b7ff700 1 Processor -- start 2023-11-27T15:24:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.463+0000 7f3f6b7ff700 1 -- start start 2023-11-27T15:24:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.463+0000 7f3f6b7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f641002c0 0x7f3f64198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.463+0000 7f3f6b7ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f641014b0 0x7f3f64198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.463+0000 7f3f6b7ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f64101e40 0x7f3f6419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.463+0000 7f3f6b7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6419fd90 con 0x7f3f641002c0 2023-11-27T15:24:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.463+0000 7f3f6b7ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6419fed0 con 0x7f3f641014b0 2023-11-27T15:24:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.463+0000 7f3f6b7ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f641a0060 con 0x7f3f64101e40 2023-11-27T15:24:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.464+0000 7f3f6959b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f641002c0 0x7f3f64198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:07.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.464+0000 7f3f6959b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f641002c0 0x7f3f64198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35572/0 (socket says 172.21.15.133:35572) 2023-11-27T15:24:07.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.464+0000 7f3f6959b700 1 -- 172.21.15.133:0/262497313 learned_addr learned my addr 172.21.15.133:0/262497313 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:07.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.464+0000 7f3f69d9c700 1 --2- 172.21.15.133:0/262497313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f64101e40 0x7f3f6419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:07.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.464+0000 7f3f68d9a700 1 --2- 172.21.15.133:0/262497313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f641014b0 0x7f3f64198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:07.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.464+0000 7f3f6959b700 1 -- 172.21.15.133:0/262497313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f64101e40 msgr2=0x7f3f6419f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:07.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.464+0000 7f3f6959b700 1 --2- 172.21.15.133:0/262497313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f64101e40 0x7f3f6419f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.464+0000 7f3f6959b700 1 -- 172.21.15.133:0/262497313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f641014b0 msgr2=0x7f3f64198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:07.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.464+0000 7f3f6959b700 1 --2- 172.21.15.133:0/262497313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f641014b0 0x7f3f64198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.464+0000 7f3f6959b700 1 -- 172.21.15.133:0/262497313 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f600095f0 con 0x7f3f641002c0 2023-11-27T15:24:07.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.464+0000 7f3f69d9c700 1 --2- 172.21.15.133:0/262497313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f64101e40 0x7f3f6419f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:07.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.465+0000 7f3f6959b700 1 --2- 172.21.15.133:0/262497313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f641002c0 0x7f3f64198210 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3f60005740 tx=0x7f3f60005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:07.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.465+0000 7f3f5a7fc700 1 -- 172.21.15.133:0/262497313 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6001d070 con 0x7f3f641002c0 2023-11-27T15:24:07.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.465+0000 7f3f5a7fc700 1 -- 172.21.15.133:0/262497313 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f60021410 con 0x7f3f641002c0 2023-11-27T15:24:07.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.465+0000 7f3f5a7fc700 1 -- 172.21.15.133:0/262497313 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f60017650 con 0x7f3f641002c0 2023-11-27T15:24:07.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.465+0000 7f3f6b7ff700 1 -- 172.21.15.133:0/262497313 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6410bc90 con 0x7f3f641002c0 2023-11-27T15:24:07.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.465+0000 7f3f6b7ff700 1 -- 172.21.15.133:0/262497313 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6410bb00 con 0x7f3f641002c0 2023-11-27T15:24:07.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.468+0000 7f3f5a7fc700 1 -- 172.21.15.133:0/262497313 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3f60021580 con 0x7f3f641002c0 2023-11-27T15:24:07.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.470+0000 7f3f5a7fc700 1 --2- 172.21.15.133:0/262497313 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3f500a1150 0x7f3f500a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:07.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.470+0000 7f3f5a7fc700 1 -- 172.21.15.133:0/262497313 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f3f600cdde0 con 0x7f3f641002c0 2023-11-27T15:24:07.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.470+0000 7f3f68d9a700 1 --2- 172.21.15.133:0/262497313 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3f500a1150 0x7f3f500a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:07.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.471+0000 7f3f68d9a700 1 --2- 172.21.15.133:0/262497313 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3f500a1150 0x7f3f500a3600 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009c60 tx=0x7f3f54009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:07.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:07 smithi133 ceph-mon[97860]: pgmap v384: 129 pgs: 129 active+clean; 952 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-11-27T15:24:07.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:07 smithi177 ceph-mon[131504]: pgmap v384: 129 pgs: 129 active+clean; 952 KiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 11 op/s 2023-11-27T15:24:07.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.653+0000 7f3f7f125b80 1 Processor -- start 2023-11-27T15:24:07.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.653+0000 7f3f7f125b80 1 -- start start 2023-11-27T15:24:07.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.654+0000 7f3f7f125b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f1ffc00 0x557e7f200010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:07.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.654+0000 7f3f7f125b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f200550 0x557e7f200b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:07.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.654+0000 7f3f7f125b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f201110 0x557e7f201580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:07.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.654+0000 7f3f7f125b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7f105060 con 0x557e7f200550 2023-11-27T15:24:07.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.654+0000 7f3f7f125b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7f1578a0 con 0x557e7f1ffc00 2023-11-27T15:24:07.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.654+0000 7f3f7f125b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7f150d30 con 0x557e7f201110 2023-11-27T15:24:07.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.654+0000 7f3f68d9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f200550 0x557e7f200b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:07.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.654+0000 7f3f68d9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f200550 0x557e7f200b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35580/0 (socket says 172.21.15.133:35580) 2023-11-27T15:24:07.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.654+0000 7f3f68d9a700 1 -- 172.21.15.133:0/478402833 learned_addr learned my addr 172.21.15.133:0/478402833 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:07.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.654+0000 7f3f69d9c700 1 --2- 172.21.15.133:0/478402833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f201110 0x557e7f201580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:07.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.654+0000 7f3f6959b700 1 --2- 172.21.15.133:0/478402833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f1ffc00 0x557e7f200010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:07.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.655+0000 7f3f68d9a700 1 -- 172.21.15.133:0/478402833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f201110 msgr2=0x557e7f201580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:07.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.655+0000 7f3f68d9a700 1 --2- 172.21.15.133:0/478402833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f201110 0x557e7f201580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.655+0000 7f3f68d9a700 1 -- 172.21.15.133:0/478402833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f1ffc00 msgr2=0x557e7f200010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:07.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.655+0000 7f3f68d9a700 1 --2- 172.21.15.133:0/478402833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f1ffc00 0x557e7f200010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.655+0000 7f3f68d9a700 1 -- 172.21.15.133:0/478402833 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e7efd2630 con 0x557e7f200550 2023-11-27T15:24:07.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.655+0000 7f3f68d9a700 1 --2- 172.21.15.133:0/478402833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f200550 0x557e7f200b30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400e4e0 tx=0x7f3f54014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b969c5b8b8e7016 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:07.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.656+0000 7f3f4f7fe700 1 -- 172.21.15.133:0/478402833 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5400ee00 con 0x557e7f200550 2023-11-27T15:24:07.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.656+0000 7f3f4f7fe700 1 -- 172.21.15.133:0/478402833 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f54021070 con 0x557e7f200550 2023-11-27T15:24:07.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.656+0000 7f3f4f7fe700 1 -- 172.21.15.133:0/478402833 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5401d810 con 0x557e7f200550 2023-11-27T15:24:07.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.657+0000 7f3f7f125b80 1 -- 172.21.15.133:0/478402833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f200550 msgr2=0x557e7f200b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:07.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.657+0000 7f3f7f125b80 1 --2- 172.21.15.133:0/478402833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f200550 0x557e7f200b30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400e4e0 tx=0x7f3f54014000 comp rx=0 tx=0).stop 2023-11-27T15:24:07.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.657+0000 7f3f7f125b80 1 -- 172.21.15.133:0/478402833 shutdown_connections 2023-11-27T15:24:07.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.657+0000 7f3f7f125b80 1 --2- 172.21.15.133:0/478402833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f201110 0x557e7f201580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.657+0000 7f3f7f125b80 1 --2- 172.21.15.133:0/478402833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f1ffc00 0x557e7f200010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.657+0000 7f3f7f125b80 1 --2- 172.21.15.133:0/478402833 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f200550 0x557e7f200b30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.657+0000 7f3f7f125b80 1 -- 172.21.15.133:0/478402833 >> 172.21.15.133:0/478402833 conn(0x557e7f1f5360 msgr2=0x557e7f1fa670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:07.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.657+0000 7f3f7f125b80 1 -- 172.21.15.133:0/478402833 shutdown_connections 2023-11-27T15:24:07.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.657+0000 7f3f7f125b80 1 -- 172.21.15.133:0/478402833 wait complete. 2023-11-27T15:24:07.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.658+0000 7f3f7f125b80 1 -- start start 2023-11-27T15:24:07.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.658+0000 7f3f7f125b80 1 Processor -- start 2023-11-27T15:24:07.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.659+0000 7f3f7f125b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f1ffc00 0x557e7f209590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.659+0000 7f3f7f125b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f200550 0x557e7f209ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.659+0000 7f3f7f125b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f201110 0x557e7f210b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.659+0000 7f3f7f125b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7f0f28e0 con 0x557e7f201110 2023-11-27T15:24:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.659+0000 7f3f7f125b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7f0f2580 con 0x557e7f1ffc00 2023-11-27T15:24:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.659+0000 7f3f7f125b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7efd4920 con 0x557e7f200550 2023-11-27T15:24:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.659+0000 7f3f7f125b80 10 client.0 _ll_register_callbacks cb 0x557e7f110780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.659+0000 7f3f6959b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f1ffc00 0x557e7f209590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.660+0000 7f3f69d9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f201110 0x557e7f210b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.660+0000 7f3f6959b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f1ffc00 0x557e7f209590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:37370/0 (socket says 172.21.15.133:37370) 2023-11-27T15:24:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.660+0000 7f3f6959b700 1 -- 172.21.15.133:0/2650745888 learned_addr learned my addr 172.21.15.133:0/2650745888 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:07.666 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.660+0000 7f3f68d9a700 1 --2- 172.21.15.133:0/2650745888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f200550 0x557e7f209ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:07.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.660+0000 7f3f6959b700 1 -- 172.21.15.133:0/2650745888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f200550 msgr2=0x557e7f209ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:07.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.660+0000 7f3f6959b700 1 --2- 172.21.15.133:0/2650745888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f200550 0x557e7f209ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.660+0000 7f3f6959b700 1 -- 172.21.15.133:0/2650745888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f201110 msgr2=0x557e7f210b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:07.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.660+0000 7f3f6959b700 1 --2- 172.21.15.133:0/2650745888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f201110 0x557e7f210b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.660+0000 7f3f6959b700 1 -- 172.21.15.133:0/2650745888 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f54017070 con 0x557e7f1ffc00 2023-11-27T15:24:07.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.660+0000 7f3f6959b700 1 --2- 172.21.15.133:0/2650745888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f1ffc00 0x557e7f209590 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3f60021830 tx=0x7f3f60021b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=107fa3f815cda043 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:07.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.660+0000 7f3f4cff9700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:24:07.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.661+0000 7f3f4cff9700 1 -- 172.21.15.133:0/2650745888 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6009a6e0 con 0x557e7f1ffc00 2023-11-27T15:24:07.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.661+0000 7f3f7f125b80 1 -- 172.21.15.133:0/2650745888 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e7f1f2b20 con 0x557e7f1ffc00 2023-11-27T15:24:07.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.661+0000 7f3f4cff9700 1 -- 172.21.15.133:0/2650745888 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f3f6009f020 con 0x557e7f1ffc00 2023-11-27T15:24:07.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.661+0000 7f3f4cff9700 1 -- 172.21.15.133:0/2650745888 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f600a33b0 con 0x557e7f1ffc00 2023-11-27T15:24:07.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.661+0000 7f3f7f125b80 10 client.25724 fetch_fsmap learned FSMap version 23 2023-11-27T15:24:07.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.661+0000 7f3f7f125b80 1 -- 172.21.15.133:0/2650745888 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557e7f117190 con 0x557e7f1ffc00 2023-11-27T15:24:07.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.661+0000 7f3f4cff9700 1 -- 172.21.15.133:0/2650745888 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e7f1f2b20 con 0x557e7f1ffc00 2023-11-27T15:24:07.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.661+0000 7f3f4cff9700 1 -- 172.21.15.133:0/2650745888 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3f600b91d0 con 0x557e7f1ffc00 2023-11-27T15:24:07.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.661+0000 7f3f7f125b80 10 client.25724 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:24:07.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.661+0000 7f3f7f125b80 10 client.25724 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-11-27T15:24:07.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.661+0000 7f3f7f125b80 20 client.25724 populate_metadata read hostname 'smithi133' 2023-11-27T15:24:07.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.661+0000 7f3f7f125b80 1 --2- 172.21.15.133:0/2650745888 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x557e7f214f40 0x557e7f217350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:07.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.661+0000 7f3f7f125b80 4 client.25724 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:07.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.661+0000 7f3f7f125b80 1 -- 172.21.15.133:0/2650745888 --> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e7f1130a0 con 0x557e7f214f40 2023-11-27T15:24:07.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.662+0000 7f3f68d9a700 1 --2- 172.21.15.133:0/2650745888 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x557e7f214f40 0x557e7f217350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:07.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.662+0000 7f3f68d9a700 1 --2- 172.21.15.133:0/2650745888 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x557e7f214f40 0x557e7f217350 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff1e576445818860 server_cookie=e797a7a78ca4ec9c in_seq=0 out_seq=0 2023-11-27T15:24:07.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.662+0000 7f3f4cff9700 10 client.25724 ms_handle_connect on v2:172.21.15.177:6838/1463687598 2023-11-27T15:24:07.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.663+0000 7f3f4cff9700 1 -- 172.21.15.133:0/2650745888 <== mds.2 v2:172.21.15.177:6838/1463687598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557e7f1130a0 con 0x557e7f214f40 2023-11-27T15:24:07.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.663+0000 7f3f4cff9700 10 client.25724 handle_command_reply: tid=0 2023-11-27T15:24:07.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.663+0000 7f3f7f125b80 1 client.25724 shutdown 2023-11-27T15:24:07.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.663+0000 7f3f7f125b80 1 -- 172.21.15.133:0/2650745888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f1ffc00 msgr2=0x557e7f209590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:07.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.663+0000 7f3f7f125b80 1 --2- 172.21.15.133:0/2650745888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f1ffc00 0x557e7f209590 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3f60021830 tx=0x7f3f60021b60 comp rx=0 tx=0).stop 2023-11-27T15:24:07.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.663+0000 7f3f7f125b80 1 -- 172.21.15.133:0/2650745888 shutdown_connections 2023-11-27T15:24:07.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.663+0000 7f3f7f125b80 1 --2- 172.21.15.133:0/2650745888 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x557e7f214f40 0x557e7f217350 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.663+0000 7f3f7f125b80 1 --2- 172.21.15.133:0/2650745888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f201110 0x557e7f210b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.663+0000 7f3f7f125b80 1 --2- 172.21.15.133:0/2650745888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f1ffc00 0x557e7f209590 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.663+0000 7f3f4cff9700 0 client.25724 ms_handle_reset on v2:172.21.15.177:6838/1463687598 2023-11-27T15:24:07.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.663+0000 7f3f7f125b80 1 --2- 172.21.15.133:0/2650745888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f200550 0x557e7f209ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.663+0000 7f3f7f125b80 1 -- 172.21.15.133:0/2650745888 >> 172.21.15.133:0/2650745888 conn(0x557e7f1f5360 msgr2=0x557e7f1f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:07.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.664+0000 7f3f7f125b80 1 -- 172.21.15.133:0/2650745888 shutdown_connections 2023-11-27T15:24:07.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.664+0000 7f3f7f125b80 1 -- 172.21.15.133:0/2650745888 wait complete. 2023-11-27T15:24:07.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.664+0000 7f3f7f125b80 20 client.25724 trim_cache size 0 max 16384 2023-11-27T15:24:07.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.679+0000 7f3f7f125b80 1 Processor -- start 2023-11-27T15:24:07.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.679+0000 7f3f7f125b80 1 -- start start 2023-11-27T15:24:07.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.679+0000 7f3f7f125b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f1ffc00 0x557e7f2014e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:07.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.679+0000 7f3f7f125b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f200550 0x557e7f210ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:07.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.679+0000 7f3f7f125b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f201110 0x557e7f21bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:07.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.679+0000 7f3f7f125b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7f208a40 con 0x557e7f200550 2023-11-27T15:24:07.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.679+0000 7f3f7f125b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7f207d20 con 0x557e7f201110 2023-11-27T15:24:07.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.679+0000 7f3f7f125b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7f1fb160 con 0x557e7f1ffc00 2023-11-27T15:24:07.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.679+0000 7f3f6959b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f1ffc00 0x557e7f2014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:07.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.679+0000 7f3f68d9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f200550 0x557e7f210ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:07.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.679+0000 7f3f6959b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f1ffc00 0x557e7f2014e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:56642/0 (socket says 172.21.15.133:56642) 2023-11-27T15:24:07.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.679+0000 7f3f6959b700 1 -- 172.21.15.133:0/1504008156 learned_addr learned my addr 172.21.15.133:0/1504008156 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:07.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.679+0000 7f3f69d9c700 1 --2- 172.21.15.133:0/1504008156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f201110 0x557e7f21bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:07.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.680+0000 7f3f6959b700 1 -- 172.21.15.133:0/1504008156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f201110 msgr2=0x557e7f21bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:07.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.680+0000 7f3f6959b700 1 --2- 172.21.15.133:0/1504008156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f201110 0x557e7f21bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.680+0000 7f3f6959b700 1 -- 172.21.15.133:0/1504008156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f200550 msgr2=0x557e7f210ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:07.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.680+0000 7f3f6959b700 1 --2- 172.21.15.133:0/1504008156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f200550 0x557e7f210ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.680+0000 7f3f6959b700 1 -- 172.21.15.133:0/1504008156 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e7f116ee0 con 0x557e7f1ffc00 2023-11-27T15:24:07.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.680+0000 7f3f6959b700 1 --2- 172.21.15.133:0/1504008156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f1ffc00 0x557e7f2014e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3f60099340 tx=0x7f3f60009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ec985300568b2c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:07.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.680+0000 7f3f4ffff700 1 -- 172.21.15.133:0/1504008156 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f600cb080 con 0x557e7f1ffc00 2023-11-27T15:24:07.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.680+0000 7f3f4ffff700 1 -- 172.21.15.133:0/1504008156 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f6009f020 con 0x557e7f1ffc00 2023-11-27T15:24:07.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.680+0000 7f3f4ffff700 1 -- 172.21.15.133:0/1504008156 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6009a740 con 0x557e7f1ffc00 2023-11-27T15:24:07.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.681+0000 7f3f7f125b80 1 -- 172.21.15.133:0/1504008156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f1ffc00 msgr2=0x557e7f2014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:07.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.681+0000 7f3f7f125b80 1 --2- 172.21.15.133:0/1504008156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f1ffc00 0x557e7f2014e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3f60099340 tx=0x7f3f60009530 comp rx=0 tx=0).stop 2023-11-27T15:24:07.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.681+0000 7f3f7f125b80 1 -- 172.21.15.133:0/1504008156 shutdown_connections 2023-11-27T15:24:07.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.681+0000 7f3f7f125b80 1 --2- 172.21.15.133:0/1504008156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f201110 0x557e7f21bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.681+0000 7f3f7f125b80 1 --2- 172.21.15.133:0/1504008156 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f1ffc00 0x557e7f2014e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.681+0000 7f3f7f125b80 1 --2- 172.21.15.133:0/1504008156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f200550 0x557e7f210ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.681+0000 7f3f7f125b80 1 -- 172.21.15.133:0/1504008156 >> 172.21.15.133:0/1504008156 conn(0x557e7f1f5360 msgr2=0x557e7f1f6830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:07.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.681+0000 7f3f7f125b80 1 -- 172.21.15.133:0/1504008156 shutdown_connections 2023-11-27T15:24:07.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.681+0000 7f3f7f125b80 1 -- 172.21.15.133:0/1504008156 wait complete. 2023-11-27T15:24:07.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.682+0000 7f3f7f125b80 1 -- start start 2023-11-27T15:24:07.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.682+0000 7f3f7f125b80 1 Processor -- start 2023-11-27T15:24:07.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.682+0000 7f3f7f125b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f1ffc00 0x557e7f20cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:07.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.682+0000 7f3f7f125b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f200550 0x557e7f20d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:07.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.683+0000 7f3f7f125b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f201110 0x557e7f20fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:07.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.683+0000 7f3f7f125b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7f2101e0 con 0x557e7f200550 2023-11-27T15:24:07.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.683+0000 7f3f7f125b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7f210350 con 0x557e7f1ffc00 2023-11-27T15:24:07.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.683+0000 7f3f7f125b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e7f210510 con 0x557e7f201110 2023-11-27T15:24:07.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.683+0000 7f3f7f125b80 10 client.0 _ll_register_callbacks cb 0x7f3f60097330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:07.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.683+0000 7f3f68d9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f200550 0x557e7f20d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:07.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.683+0000 7f3f6959b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f1ffc00 0x557e7f20cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:07.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.683+0000 7f3f68d9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f200550 0x557e7f20d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35600/0 (socket says 172.21.15.133:35600) 2023-11-27T15:24:07.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.683+0000 7f3f68d9a700 1 -- 172.21.15.133:0/67692619 learned_addr learned my addr 172.21.15.133:0/67692619 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:07.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.683+0000 7f3f69d9c700 1 --2- 172.21.15.133:0/67692619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f201110 0x557e7f20fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:07.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.683+0000 7f3f6959b700 1 -- 172.21.15.133:0/67692619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f201110 msgr2=0x557e7f20fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:07.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.683+0000 7f3f6959b700 1 --2- 172.21.15.133:0/67692619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f201110 0x557e7f20fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.683+0000 7f3f6959b700 1 -- 172.21.15.133:0/67692619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f200550 msgr2=0x557e7f20d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:07.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.683+0000 7f3f6959b700 1 --2- 172.21.15.133:0/67692619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f200550 0x557e7f20d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.683+0000 7f3f6959b700 1 -- 172.21.15.133:0/67692619 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e7f117190 con 0x557e7f1ffc00 2023-11-27T15:24:07.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.683+0000 7f3f68d9a700 1 --2- 172.21.15.133:0/67692619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f200550 0x557e7f20d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:07.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.684+0000 7f3f6959b700 1 --2- 172.21.15.133:0/67692619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f1ffc00 0x557e7f20cc60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3f600a3110 tx=0x7f3f60021b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2abf837adba4a1b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:07.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.684+0000 7f3f4cff9700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:24:07.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.684+0000 7f3f4cff9700 1 -- 172.21.15.133:0/67692619 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f600a33b0 con 0x557e7f1ffc00 2023-11-27T15:24:07.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.684+0000 7f3f4cff9700 1 -- 172.21.15.133:0/67692619 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f3f600cb080 con 0x557e7f1ffc00 2023-11-27T15:24:07.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.684+0000 7f3f7f125b80 1 -- 172.21.15.133:0/67692619 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f54021070 con 0x557e7f1ffc00 2023-11-27T15:24:07.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.688+0000 7f3f4cff9700 1 -- 172.21.15.133:0/67692619 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f600cb280 con 0x557e7f1ffc00 2023-11-27T15:24:07.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.688+0000 7f3f7f125b80 10 client.25730 fetch_fsmap learned FSMap version 23 2023-11-27T15:24:07.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.688+0000 7f3f7f125b80 1 -- 172.21.15.133:0/67692619 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557e7ef67c60 con 0x557e7f1ffc00 2023-11-27T15:24:07.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.689+0000 7f3f4cff9700 1 -- 172.21.15.133:0/67692619 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f54021070 con 0x557e7f1ffc00 2023-11-27T15:24:07.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.689+0000 7f3f4cff9700 1 -- 172.21.15.133:0/67692619 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3f600cb9f0 con 0x557e7f1ffc00 2023-11-27T15:24:07.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.689+0000 7f3f7f125b80 10 client.25730 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:24:07.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.689+0000 7f3f7f125b80 10 client.25730 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-11-27T15:24:07.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.689+0000 7f3f7f125b80 20 client.25730 populate_metadata read hostname 'smithi133' 2023-11-27T15:24:07.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.689+0000 7f3f7f125b80 1 --2- 172.21.15.133:0/67692619 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x557e7f269410 0x557e7f26b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:07.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.689+0000 7f3f7f125b80 4 client.25730 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:24:07.694 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.689+0000 7f3f7f125b80 1 -- 172.21.15.133:0/67692619 --> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557e7f11f660 con 0x557e7f269410 2023-11-27T15:24:07.694 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.689+0000 7f3f68d9a700 1 --2- 172.21.15.133:0/67692619 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x557e7f269410 0x557e7f26b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:07.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.690+0000 7f3f68d9a700 1 --2- 172.21.15.133:0/67692619 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x557e7f269410 0x557e7f26b7f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9e86b406c147227 server_cookie=b5f56029292718a in_seq=0 out_seq=0 2023-11-27T15:24:07.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.690+0000 7f3f4cff9700 10 client.25730 ms_handle_connect on v2:172.21.15.177:6838/1463687598 2023-11-27T15:24:07.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.690+0000 7f3f4cff9700 1 -- 172.21.15.133:0/67692619 <== mds.2 v2:172.21.15.177:6838/1463687598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557e7f11f660 con 0x557e7f269410 2023-11-27T15:24:07.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.690+0000 7f3f4cff9700 10 client.25730 handle_command_reply: tid=0 2023-11-27T15:24:07.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.690+0000 7f3f7f125b80 1 client.25730 shutdown 2023-11-27T15:24:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.690+0000 7f3f7f125b80 1 -- 172.21.15.133:0/67692619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f1ffc00 msgr2=0x557e7f20cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.690+0000 7f3f7f125b80 1 --2- 172.21.15.133:0/67692619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f1ffc00 0x557e7f20cc60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3f600a3110 tx=0x7f3f60021b60 comp rx=0 tx=0).stop 2023-11-27T15:24:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.690+0000 7f3f7f125b80 1 -- 172.21.15.133:0/67692619 shutdown_connections 2023-11-27T15:24:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.690+0000 7f3f7f125b80 1 --2- 172.21.15.133:0/67692619 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x557e7f269410 0x557e7f26b7f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.690+0000 7f3f7f125b80 1 --2- 172.21.15.133:0/67692619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557e7f201110 0x557e7f20fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.690+0000 7f3f7f125b80 1 --2- 172.21.15.133:0/67692619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e7f1ffc00 0x557e7f20cc60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.690+0000 7f3f7f125b80 1 --2- 172.21.15.133:0/67692619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557e7f200550 0x557e7f20d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.690+0000 7f3f4cff9700 0 client.25730 ms_handle_reset on v2:172.21.15.177:6838/1463687598 2023-11-27T15:24:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.690+0000 7f3f7f125b80 1 -- 172.21.15.133:0/67692619 >> 172.21.15.133:0/67692619 conn(0x557e7f1f5360 msgr2=0x557e7f21bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.690+0000 7f3f7f125b80 1 -- 172.21.15.133:0/67692619 shutdown_connections 2023-11-27T15:24:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.690+0000 7f3f7f125b80 1 -- 172.21.15.133:0/67692619 wait complete. 2023-11-27T15:24:07.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.691+0000 7f3f7f125b80 20 client.25730 trim_cache size 0 max 16384 2023-11-27T15:24:07.697 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2023-11-27T15:24:07.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.691+0000 7f3f6b7ff700 1 -- 172.21.15.133:0/262497313 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3f500a1150 msgr2=0x7f3f500a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:07.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.691+0000 7f3f6b7ff700 1 --2- 172.21.15.133:0/262497313 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3f500a1150 0x7f3f500a3600 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009c60 tx=0x7f3f54009260 comp rx=0 tx=0).stop 2023-11-27T15:24:07.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.691+0000 7f3f6b7ff700 1 -- 172.21.15.133:0/262497313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f641002c0 msgr2=0x7f3f64198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:07.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.691+0000 7f3f6b7ff700 1 --2- 172.21.15.133:0/262497313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f641002c0 0x7f3f64198210 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3f60005740 tx=0x7f3f60005770 comp rx=0 tx=0).stop 2023-11-27T15:24:07.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.691+0000 7f3f6b7ff700 1 -- 172.21.15.133:0/262497313 shutdown_connections 2023-11-27T15:24:07.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.691+0000 7f3f6b7ff700 1 --2- 172.21.15.133:0/262497313 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3f500a1150 0x7f3f500a3600 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.691+0000 7f3f6b7ff700 1 --2- 172.21.15.133:0/262497313 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3f64101e40 0x7f3f6419f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.691+0000 7f3f6b7ff700 1 --2- 172.21.15.133:0/262497313 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f641002c0 0x7f3f64198210 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.698 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.691+0000 7f3f6b7ff700 1 --2- 172.21.15.133:0/262497313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f641014b0 0x7f3f64198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:07.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.691+0000 7f3f6b7ff700 1 -- 172.21.15.133:0/262497313 >> 172.21.15.133:0/262497313 conn(0x7f3f640fb8a0 msgr2=0x7f3f64106d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:07.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.692+0000 7f3f6b7ff700 1 -- 172.21.15.133:0/262497313 shutdown_connections 2023-11-27T15:24:07.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:07.692+0000 7f3f6b7ff700 1 -- 172.21.15.133:0/262497313 wait complete. 2023-11-27T15:24:07.723 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:24:08.949 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:08 smithi133 ceph-mon[97860]: pgmap v385: 129 pgs: 129 active+clean; 1.0 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-11-27T15:24:09.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:08 smithi177 ceph-mon[131504]: pgmap v385: 129 pgs: 129 active+clean; 1.0 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-11-27T15:24:09.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:08 smithi174 ceph-mon[131110]: pgmap v385: 129 pgs: 129 active+clean; 1.0 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 KiB/s wr, 14 op/s 2023-11-27T15:24:09.407 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:09.407 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:09.559 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:09.560 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:09.582 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:09.582 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:09.583 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 35 -lt 100 ']' 2023-11-27T15:24:09.583 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 35 + 1 2023-11-27T15:24:09.632 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=36 2023-11-27T15:24:09.632 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:09.659 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:09.660 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:09.662 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:09.662 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:09.663 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:10.457 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:10.457 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:10.593 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:10.593 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:10.656 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:10.657 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:10.657 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 36 -lt 100 ']' 2023-11-27T15:24:10.657 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 36 + 1 2023-11-27T15:24:10.698 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=37 2023-11-27T15:24:10.699 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:10.743 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:10.744 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:10.746 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:10.747 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:10.747 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:11.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:10 smithi177 ceph-mon[131504]: pgmap v386: 129 pgs: 129 active+clean; 1.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-11-27T15:24:11.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:10 smithi174 ceph-mon[131110]: pgmap v386: 129 pgs: 129 active+clean; 1.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-11-27T15:24:11.207 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:10 smithi133 ceph-mon[97860]: pgmap v386: 129 pgs: 129 active+clean; 1.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 11 op/s 2023-11-27T15:24:11.406 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:11.407 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:11.633 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:11.633 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:11.716 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:11.716 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:11.717 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 37 -lt 100 ']' 2023-11-27T15:24:11.717 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 37 + 1 2023-11-27T15:24:11.765 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=38 2023-11-27T15:24:11.765 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:11.800 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:11.801 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:11.803 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:11.803 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:11.804 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:12.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:11 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:24:12.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:11 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:24:12.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:11 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:24:12.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:11 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:24:12.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:11 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:24:12.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:11 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:24:12.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:11 smithi177 ceph-mon[131504]: Service mds.all has no related daemon already placed. 2023-11-27T15:24:12.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:11 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:24:12.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:11 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:24:12.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:11 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:24:12.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:11 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:24:12.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:11 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:24:12.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:11 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:24:12.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:11 smithi174 ceph-mon[131110]: Service mds.all has no related daemon already placed. 2023-11-27T15:24:12.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:11 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:24:12.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:11 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:24:12.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:11 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:24:12.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:11 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:24:12.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:11 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:24:12.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:11 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:24:12.165 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:11 smithi133 ceph-mon[97860]: Service mds.all has no related daemon already placed. 2023-11-27T15:24:12.381 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:12.381 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:12.625 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:12.626 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:12.689 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:12.689 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:12.689 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 38 -lt 100 ']' 2023-11-27T15:24:12.690 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 38 + 1 2023-11-27T15:24:12.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.718+0000 7fb1dc5ff700 1 -- 172.21.15.133:0/2673930030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1d4101e40 msgr2=0x7fb1d4108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:12.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.718+0000 7fb1dc5ff700 1 --2- 172.21.15.133:0/2673930030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1d4101e40 0x7fb1d4108700 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc00b0a0 tx=0x7fb1cc00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.718+0000 7fb1dc5ff700 1 -- 172.21.15.133:0/2673930030 shutdown_connections 2023-11-27T15:24:12.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.718+0000 7fb1dc5ff700 1 --2- 172.21.15.133:0/2673930030 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1d4101e40 0x7fb1d4108700 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.718+0000 7fb1dc5ff700 1 --2- 172.21.15.133:0/2673930030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1d41002c0 0x7fb1d41006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.718+0000 7fb1dc5ff700 1 --2- 172.21.15.133:0/2673930030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1d41014b0 0x7fb1d4101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.718+0000 7fb1dc5ff700 1 -- 172.21.15.133:0/2673930030 >> 172.21.15.133:0/2673930030 conn(0x7fb1d40fb8a0 msgr2=0x7fb1d40fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:12.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.718+0000 7fb1dc5ff700 1 -- 172.21.15.133:0/2673930030 shutdown_connections 2023-11-27T15:24:12.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.718+0000 7fb1dc5ff700 1 -- 172.21.15.133:0/2673930030 wait complete. 2023-11-27T15:24:12.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.720+0000 7fb1dc5ff700 1 Processor -- start 2023-11-27T15:24:12.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.720+0000 7fb1dc5ff700 1 -- start start 2023-11-27T15:24:12.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.720+0000 7fb1dc5ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1d41002c0 0x7fb1d40748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.720+0000 7fb1dc5ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1d41014b0 0x7fb1d4072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.720+0000 7fb1dc5ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1d4101e40 0x7fb1d4073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.721+0000 7fb1dc5ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d4073bc0 con 0x7fb1d41002c0 2023-11-27T15:24:12.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.721+0000 7fb1dc5ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d4073d30 con 0x7fb1d41014b0 2023-11-27T15:24:12.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.721+0000 7fb1dc5ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d41a0230 con 0x7fb1d4101e40 2023-11-27T15:24:12.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.721+0000 7fb1da39b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1d41002c0 0x7fb1d40748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.721+0000 7fb1da39b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1d41002c0 0x7fb1d40748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35622/0 (socket says 172.21.15.133:35622) 2023-11-27T15:24:12.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.721+0000 7fb1da39b700 1 -- 172.21.15.133:0/3244944506 learned_addr learned my addr 172.21.15.133:0/3244944506 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:12.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.721+0000 7fb1d9b9a700 1 --2- 172.21.15.133:0/3244944506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1d41014b0 0x7fb1d4072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.721+0000 7fb1dab9c700 1 --2- 172.21.15.133:0/3244944506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1d4101e40 0x7fb1d4073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.721+0000 7fb1da39b700 1 -- 172.21.15.133:0/3244944506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1d4101e40 msgr2=0x7fb1d4073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:12.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.721+0000 7fb1da39b700 1 --2- 172.21.15.133:0/3244944506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1d4101e40 0x7fb1d4073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.721+0000 7fb1da39b700 1 -- 172.21.15.133:0/3244944506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1d41014b0 msgr2=0x7fb1d4072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:12.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.721+0000 7fb1da39b700 1 --2- 172.21.15.133:0/3244944506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1d41014b0 0x7fb1d4072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.721+0000 7fb1da39b700 1 -- 172.21.15.133:0/3244944506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc009d00 con 0x7fb1d41002c0 2023-11-27T15:24:12.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.721+0000 7fb1da39b700 1 --2- 172.21.15.133:0/3244944506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1d41002c0 0x7fb1d40748a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000e970 tx=0x7fb1d000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:12.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.722+0000 7fb1cb7fe700 1 -- 172.21.15.133:0/3244944506 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0009790 con 0x7fb1d41002c0 2023-11-27T15:24:12.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.722+0000 7fb1cb7fe700 1 -- 172.21.15.133:0/3244944506 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1d00044e0 con 0x7fb1d41002c0 2023-11-27T15:24:12.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.722+0000 7fb1dc5ff700 1 -- 172.21.15.133:0/3244944506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d410bc90 con 0x7fb1d41002c0 2023-11-27T15:24:12.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.722+0000 7fb1cb7fe700 1 -- 172.21.15.133:0/3244944506 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0003e30 con 0x7fb1d41002c0 2023-11-27T15:24:12.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.722+0000 7fb1dc5ff700 1 -- 172.21.15.133:0/3244944506 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d410bb00 con 0x7fb1d41002c0 2023-11-27T15:24:12.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.725+0000 7fb1cb7fe700 1 -- 172.21.15.133:0/3244944506 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb1d0003f90 con 0x7fb1d41002c0 2023-11-27T15:24:12.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.726+0000 7fb1cb7fe700 1 --2- 172.21.15.133:0/3244944506 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb1c00a1150 0x7fb1c00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.726+0000 7fb1cb7fe700 1 -- 172.21.15.133:0/3244944506 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb1d0014070 con 0x7fb1d41002c0 2023-11-27T15:24:12.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.727+0000 7fb1d9b9a700 1 --2- 172.21.15.133:0/3244944506 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb1c00a1150 0x7fb1c00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.730 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.727+0000 7fb1d9b9a700 1 --2- 172.21.15.133:0/3244944506 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb1c00a1150 0x7fb1c00a3600 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4009bd0 tx=0x7fb1c4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:12.748 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=39 2023-11-27T15:24:12.749 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:12.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.802+0000 7fb1eff0db80 1 Processor -- start 2023-11-27T15:24:12.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.802+0000 7fb1eff0db80 1 -- start start 2023-11-27T15:24:12.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.802+0000 7fb1eff0db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997254c00 0x560997255010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.802+0000 7fb1eff0db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997255550 0x560997255b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.802+0000 7fb1eff0db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560997256110 0x560997256580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.802+0000 7fb1eff0db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56099715a060 con 0x560997255550 2023-11-27T15:24:12.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.802+0000 7fb1eff0db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609971ac8a0 con 0x560997254c00 2023-11-27T15:24:12.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.802+0000 7fb1eff0db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609971a5d30 con 0x560997256110 2023-11-27T15:24:12.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.803+0000 7fb1d9b9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997255550 0x560997255b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.803+0000 7fb1da39b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997254c00 0x560997255010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.803+0000 7fb1da39b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997254c00 0x560997255010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:37418/0 (socket says 172.21.15.133:37418) 2023-11-27T15:24:12.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.803+0000 7fb1da39b700 1 -- 172.21.15.133:0/1517918101 learned_addr learned my addr 172.21.15.133:0/1517918101 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:12.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.803+0000 7fb1dab9c700 1 --2- 172.21.15.133:0/1517918101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560997256110 0x560997256580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.803+0000 7fb1da39b700 1 -- 172.21.15.133:0/1517918101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560997256110 msgr2=0x560997256580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:12.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.803+0000 7fb1da39b700 1 --2- 172.21.15.133:0/1517918101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560997256110 0x560997256580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.803+0000 7fb1da39b700 1 -- 172.21.15.133:0/1517918101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997255550 msgr2=0x560997255b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:12.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.803+0000 7fb1da39b700 1 --2- 172.21.15.133:0/1517918101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997255550 0x560997255b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.803+0000 7fb1da39b700 1 -- 172.21.15.133:0/1517918101 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560997027630 con 0x560997254c00 2023-11-27T15:24:12.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.803+0000 7fb1da39b700 1 --2- 172.21.15.133:0/1517918101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997254c00 0x560997255010 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0099fa0 tx=0x7fb1d0099bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b58b19b2a86fbb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:12.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.804+0000 7fb1c8bfe700 1 -- 172.21.15.133:0/1517918101 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d009a010 con 0x560997254c00 2023-11-27T15:24:12.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.804+0000 7fb1c8bfe700 1 -- 172.21.15.133:0/1517918101 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1d009e040 con 0x560997254c00 2023-11-27T15:24:12.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.804+0000 7fb1c8bfe700 1 -- 172.21.15.133:0/1517918101 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d009a4d0 con 0x560997254c00 2023-11-27T15:24:12.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.804+0000 7fb1eff0db80 1 -- 172.21.15.133:0/1517918101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997254c00 msgr2=0x560997255010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:12.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.804+0000 7fb1eff0db80 1 --2- 172.21.15.133:0/1517918101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997254c00 0x560997255010 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0099fa0 tx=0x7fb1d0099bd0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.805+0000 7fb1eff0db80 1 -- 172.21.15.133:0/1517918101 shutdown_connections 2023-11-27T15:24:12.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.805+0000 7fb1eff0db80 1 --2- 172.21.15.133:0/1517918101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560997256110 0x560997256580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.805+0000 7fb1eff0db80 1 --2- 172.21.15.133:0/1517918101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997254c00 0x560997255010 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.805+0000 7fb1eff0db80 1 --2- 172.21.15.133:0/1517918101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997255550 0x560997255b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.805+0000 7fb1eff0db80 1 -- 172.21.15.133:0/1517918101 >> 172.21.15.133:0/1517918101 conn(0x56099724a360 msgr2=0x56099724f670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:12.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.805+0000 7fb1eff0db80 1 -- 172.21.15.133:0/1517918101 shutdown_connections 2023-11-27T15:24:12.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.805+0000 7fb1eff0db80 1 -- 172.21.15.133:0/1517918101 wait complete. 2023-11-27T15:24:12.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.805+0000 7fb1eff0db80 1 -- start start 2023-11-27T15:24:12.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.805+0000 7fb1eff0db80 1 Processor -- start 2023-11-27T15:24:12.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.806+0000 7fb1eff0db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560997254c00 0x56099725e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.806+0000 7fb1eff0db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997255550 0x56099725eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.806+0000 7fb1eff0db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997256110 0x560997265b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.806+0000 7fb1eff0db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609971478e0 con 0x560997255550 2023-11-27T15:24:12.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.806+0000 7fb1eff0db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x560997147580 con 0x560997256110 2023-11-27T15:24:12.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.806+0000 7fb1eff0db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560997029920 con 0x560997254c00 2023-11-27T15:24:12.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.806+0000 7fb1eff0db80 10 client.0 _ll_register_callbacks cb 0x560997165780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:12.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.806+0000 7fb1d9b9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997255550 0x56099725eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.806+0000 7fb1d9b9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997255550 0x56099725eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35642/0 (socket says 172.21.15.133:35642) 2023-11-27T15:24:12.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.806+0000 7fb1d9b9a700 1 -- 172.21.15.133:0/4209118822 learned_addr learned my addr 172.21.15.133:0/4209118822 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:12.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.806+0000 7fb1da39b700 1 --2- 172.21.15.133:0/4209118822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560997254c00 0x56099725e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.806+0000 7fb1dab9c700 1 --2- 172.21.15.133:0/4209118822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997256110 0x560997265b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.806+0000 7fb1d9b9a700 1 -- 172.21.15.133:0/4209118822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560997254c00 msgr2=0x56099725e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:12.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.807+0000 7fb1d9b9a700 1 --2- 172.21.15.133:0/4209118822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560997254c00 0x56099725e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.807+0000 7fb1d9b9a700 1 -- 172.21.15.133:0/4209118822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997256110 msgr2=0x560997265b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:12.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.807+0000 7fb1d9b9a700 1 --2- 172.21.15.133:0/4209118822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997256110 0x560997265b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.807+0000 7fb1d9b9a700 1 -- 172.21.15.133:0/4209118822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d410bb00 con 0x560997255550 2023-11-27T15:24:12.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.807+0000 7fb1da39b700 1 --2- 172.21.15.133:0/4209118822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560997254c00 0x56099725e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:12.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.807+0000 7fb1dab9c700 1 --2- 172.21.15.133:0/4209118822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997256110 0x560997265b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:12.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.807+0000 7fb1d9b9a700 1 --2- 172.21.15.133:0/4209118822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997255550 0x56099725eb20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4011920 tx=0x7fb1c40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e2971552d2205d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:12.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.807+0000 7fb1bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:24:12.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.807+0000 7fb1bdffb700 1 -- 172.21.15.133:0/4209118822 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c401f840 con 0x560997255550 2023-11-27T15:24:12.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.807+0000 7fb1bdffb700 1 -- 172.21.15.133:0/4209118822 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb1c4017070 con 0x560997255550 2023-11-27T15:24:12.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.807+0000 7fb1eff0db80 1 -- 172.21.15.133:0/4209118822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560997247b20 con 0x560997255550 2023-11-27T15:24:12.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.807+0000 7fb1bdffb700 1 -- 172.21.15.133:0/4209118822 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1c4011e90 con 0x560997255550 2023-11-27T15:24:12.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.807+0000 7fb1eff0db80 10 client.15702 fetch_fsmap learned FSMap version 23 2023-11-27T15:24:12.814 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.807+0000 7fb1eff0db80 1 -- 172.21.15.133:0/4209118822 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56099716c190 con 0x560997255550 2023-11-27T15:24:12.816 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.808+0000 7fb1bdffb700 1 -- 172.21.15.133:0/4209118822 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560997247b20 con 0x560997255550 2023-11-27T15:24:12.816 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.808+0000 7fb1bdffb700 1 -- 172.21.15.133:0/4209118822 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb1c4028c50 con 0x560997255550 2023-11-27T15:24:12.816 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.808+0000 7fb1eff0db80 10 client.15702 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:24:12.816 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.808+0000 7fb1eff0db80 10 client.15702 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:24:12.816 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.808+0000 7fb1eff0db80 20 client.15702 populate_metadata read hostname 'smithi133' 2023-11-27T15:24:12.816 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.808+0000 7fb1eff0db80 1 --2- 172.21.15.133:0/4209118822 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56099726a440 0x56099726c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.816 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.808+0000 7fb1eff0db80 4 client.15702 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:12.816 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.808+0000 7fb1eff0db80 1 -- 172.21.15.133:0/4209118822 --> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609971680a0 con 0x56099726a440 2023-11-27T15:24:12.816 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.808+0000 7fb1da39b700 1 --2- 172.21.15.133:0/4209118822 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56099726a440 0x56099726c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.816 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.809+0000 7fb1bdffb700 10 client.15702 ms_handle_connect on v2:172.21.15.174:6836/1955090548 2023-11-27T15:24:12.816 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.809+0000 7fb1da39b700 1 --2- 172.21.15.133:0/4209118822 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56099726a440 0x56099726c850 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ab7e473330f2ee3 server_cookie=b9fa3202c9ecc9d3 in_seq=0 out_seq=0 2023-11-27T15:24:12.816 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.809+0000 7fb1bdffb700 1 -- 172.21.15.133:0/4209118822 <== mds.1 v2:172.21.15.174:6836/1955090548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5609971680a0 con 0x56099726a440 2023-11-27T15:24:12.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.810+0000 7fb1bdffb700 10 client.15702 handle_command_reply: tid=0 2023-11-27T15:24:12.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.810+0000 7fb1eff0db80 1 client.15702 shutdown 2023-11-27T15:24:12.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.810+0000 7fb1eff0db80 1 -- 172.21.15.133:0/4209118822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997255550 msgr2=0x56099725eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:12.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.810+0000 7fb1eff0db80 1 --2- 172.21.15.133:0/4209118822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997255550 0x56099725eb20 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4011920 tx=0x7fb1c40047c0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.810+0000 7fb1eff0db80 1 -- 172.21.15.133:0/4209118822 shutdown_connections 2023-11-27T15:24:12.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.810+0000 7fb1eff0db80 1 --2- 172.21.15.133:0/4209118822 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56099726a440 0x56099726c850 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.810+0000 7fb1eff0db80 1 --2- 172.21.15.133:0/4209118822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997256110 0x560997265b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.810+0000 7fb1eff0db80 1 --2- 172.21.15.133:0/4209118822 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560997254c00 0x56099725e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.810+0000 7fb1eff0db80 1 --2- 172.21.15.133:0/4209118822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997255550 0x56099725eb20 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.810+0000 7fb1bdffb700 0 client.15702 ms_handle_reset on v2:172.21.15.174:6836/1955090548 2023-11-27T15:24:12.817 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.810+0000 7fb1eff0db80 1 -- 172.21.15.133:0/4209118822 >> 172.21.15.133:0/4209118822 conn(0x56099724a360 msgr2=0x56099724ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:12.818 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.810+0000 7fb1eff0db80 1 -- 172.21.15.133:0/4209118822 shutdown_connections 2023-11-27T15:24:12.818 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.810+0000 7fb1eff0db80 1 -- 172.21.15.133:0/4209118822 wait complete. 2023-11-27T15:24:12.818 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.810+0000 7fb1eff0db80 20 client.15702 trim_cache size 0 max 16384 2023-11-27T15:24:12.823 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:12.824 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:12.826 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:12.826 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:12.827 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:12.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.825+0000 7fb1eff0db80 1 Processor -- start 2023-11-27T15:24:12.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.825+0000 7fb1eff0db80 1 -- start start 2023-11-27T15:24:12.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.825+0000 7fb1eff0db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997254c00 0x56099725e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.825+0000 7fb1eff0db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560997255550 0x560997265b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.825+0000 7fb1eff0db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997256110 0x56099725eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.825+0000 7fb1eff0db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560997251140 con 0x560997256110 2023-11-27T15:24:12.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.825+0000 7fb1eff0db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56099725da90 con 0x560997254c00 2023-11-27T15:24:12.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.825+0000 7fb1eff0db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560997250b00 con 0x560997255550 2023-11-27T15:24:12.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.826+0000 7fb1dab9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997256110 0x56099725eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.826+0000 7fb1dab9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997256110 0x56099725eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35646/0 (socket says 172.21.15.133:35646) 2023-11-27T15:24:12.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.826+0000 7fb1da39b700 1 --2- 172.21.15.133:0/2310722902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997254c00 0x56099725e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.826+0000 7fb1dab9c700 1 -- 172.21.15.133:0/2310722902 learned_addr learned my addr 172.21.15.133:0/2310722902 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:12.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.826+0000 7fb1d9b9a700 1 --2- 172.21.15.133:0/2310722902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560997255550 0x560997265b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.826+0000 7fb1dab9c700 1 -- 172.21.15.133:0/2310722902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560997255550 msgr2=0x560997265b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:12.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.826+0000 7fb1dab9c700 1 --2- 172.21.15.133:0/2310722902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560997255550 0x560997265b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.826+0000 7fb1dab9c700 1 -- 172.21.15.133:0/2310722902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997254c00 msgr2=0x56099725e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:12.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.826+0000 7fb1dab9c700 1 --2- 172.21.15.133:0/2310722902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997254c00 0x56099725e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.826+0000 7fb1dab9c700 1 -- 172.21.15.133:0/2310722902 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56099716bee0 con 0x560997256110 2023-11-27T15:24:12.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.827+0000 7fb1dab9c700 1 --2- 172.21.15.133:0/2310722902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997256110 0x56099725eb20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0079f0 tx=0x7fb1cc009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12951557290404dc server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:12.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.827+0000 7fb1c93ff700 1 -- 172.21.15.133:0/2310722902 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc00e040 con 0x560997256110 2023-11-27T15:24:12.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.827+0000 7fb1c93ff700 1 -- 172.21.15.133:0/2310722902 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1cc020070 con 0x560997256110 2023-11-27T15:24:12.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.827+0000 7fb1c93ff700 1 -- 172.21.15.133:0/2310722902 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc004cb0 con 0x560997256110 2023-11-27T15:24:12.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.827+0000 7fb1eff0db80 1 -- 172.21.15.133:0/2310722902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997256110 msgr2=0x56099725eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:12.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.827+0000 7fb1eff0db80 1 --2- 172.21.15.133:0/2310722902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997256110 0x56099725eb20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0079f0 tx=0x7fb1cc009300 comp rx=0 tx=0).stop 2023-11-27T15:24:12.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.828+0000 7fb1eff0db80 1 -- 172.21.15.133:0/2310722902 shutdown_connections 2023-11-27T15:24:12.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.828+0000 7fb1eff0db80 1 --2- 172.21.15.133:0/2310722902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997256110 0x56099725eb20 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.828+0000 7fb1eff0db80 1 --2- 172.21.15.133:0/2310722902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997254c00 0x56099725e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.828+0000 7fb1eff0db80 1 --2- 172.21.15.133:0/2310722902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560997255550 0x560997265b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.828+0000 7fb1eff0db80 1 -- 172.21.15.133:0/2310722902 >> 172.21.15.133:0/2310722902 conn(0x56099724a360 msgr2=0x56099725d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:12.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.828+0000 7fb1eff0db80 1 -- 172.21.15.133:0/2310722902 shutdown_connections 2023-11-27T15:24:12.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.828+0000 7fb1eff0db80 1 -- 172.21.15.133:0/2310722902 wait complete. 2023-11-27T15:24:12.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.828+0000 7fb1eff0db80 1 -- start start 2023-11-27T15:24:12.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.828+0000 7fb1eff0db80 1 Processor -- start 2023-11-27T15:24:12.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.829+0000 7fb1eff0db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997254c00 0x560997261b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.829+0000 7fb1eff0db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997255550 0x5609972620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.829+0000 7fb1eff0db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560997256110 0x5609972650c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.829+0000 7fb1eff0db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x560997265600 con 0x560997254c00 2023-11-27T15:24:12.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.829+0000 7fb1eff0db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x560997265770 con 0x560997255550 2023-11-27T15:24:12.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.829+0000 7fb1eff0db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560997265930 con 0x560997256110 2023-11-27T15:24:12.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.829+0000 7fb1eff0db80 10 client.0 _ll_register_callbacks cb 0x560997165780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:12.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.829+0000 7fb1da39b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997254c00 0x560997261b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.829+0000 7fb1da39b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997254c00 0x560997261b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:35652/0 (socket says 172.21.15.133:35652) 2023-11-27T15:24:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.829+0000 7fb1da39b700 1 -- 172.21.15.133:0/2916157267 learned_addr learned my addr 172.21.15.133:0/2916157267 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.829+0000 7fb1d9b9a700 1 --2- 172.21.15.133:0/2916157267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997255550 0x5609972620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.829+0000 7fb1dab9c700 1 --2- 172.21.15.133:0/2916157267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560997256110 0x5609972650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.829+0000 7fb1da39b700 1 -- 172.21.15.133:0/2916157267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560997256110 msgr2=0x5609972650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.829+0000 7fb1da39b700 1 --2- 172.21.15.133:0/2916157267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560997256110 0x5609972650c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.829+0000 7fb1da39b700 1 -- 172.21.15.133:0/2916157267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997255550 msgr2=0x5609972620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.829+0000 7fb1da39b700 1 --2- 172.21.15.133:0/2916157267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997255550 0x5609972620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.829+0000 7fb1da39b700 1 -- 172.21.15.133:0/2916157267 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d4109030 con 0x560997254c00 2023-11-27T15:24:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.829+0000 7fb1d9b9a700 1 --2- 172.21.15.133:0/2916157267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997255550 0x5609972620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.830+0000 7fb1da39b700 1 --2- 172.21.15.133:0/2916157267 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997254c00 0x560997261b60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0096a80 tx=0x7fb1d0099a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efc69475ed6c913 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.830+0000 7fb1bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:24:12.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.830+0000 7fb1bdffb700 1 -- 172.21.15.133:0/2916157267 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d00a2460 con 0x560997254c00 2023-11-27T15:24:12.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.830+0000 7fb1bdffb700 1 -- 172.21.15.133:0/2916157267 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb1d0098150 con 0x560997254c00 2023-11-27T15:24:12.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.830+0000 7fb1eff0db80 1 -- 172.21.15.133:0/2916157267 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1d009e040 con 0x560997254c00 2023-11-27T15:24:12.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.830+0000 7fb1bdffb700 1 -- 172.21.15.133:0/2916157267 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1d00984b0 con 0x560997254c00 2023-11-27T15:24:12.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.830+0000 7fb1eff0db80 10 client.15708 fetch_fsmap learned FSMap version 23 2023-11-27T15:24:12.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.830+0000 7fb1eff0db80 1 -- 172.21.15.133:0/2916157267 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560996fbcc60 con 0x560997254c00 2023-11-27T15:24:12.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.830+0000 7fb1bdffb700 1 -- 172.21.15.133:0/2916157267 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1d009e040 con 0x560997254c00 2023-11-27T15:24:12.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.830+0000 7fb1bdffb700 1 -- 172.21.15.133:0/2916157267 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb1d00a1fd0 con 0x560997254c00 2023-11-27T15:24:12.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.830+0000 7fb1eff0db80 10 client.15708 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:24:12.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.830+0000 7fb1eff0db80 10 client.15708 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:24:12.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.830+0000 7fb1eff0db80 20 client.15708 populate_metadata read hostname 'smithi133' 2023-11-27T15:24:12.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.830+0000 7fb1eff0db80 1 --2- 172.21.15.133:0/2916157267 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x5609972bf0e0 0x5609972c14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:12.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.831+0000 7fb1eff0db80 4 client.15708 mds_command: new command op to 24521 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:24:12.839 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.831+0000 7fb1eff0db80 1 -- 172.21.15.133:0/2916157267 --> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560997174660 con 0x5609972bf0e0 2023-11-27T15:24:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.831+0000 7fb1d9b9a700 1 --2- 172.21.15.133:0/2916157267 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x5609972bf0e0 0x5609972c14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.831+0000 7fb1d9b9a700 1 --2- 172.21.15.133:0/2916157267 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x5609972bf0e0 0x5609972c14f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4019bcbc33060ab1 server_cookie=ae38a046d665696c in_seq=0 out_seq=0 2023-11-27T15:24:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.831+0000 7fb1bdffb700 10 client.15708 ms_handle_connect on v2:172.21.15.174:6836/1955090548 2023-11-27T15:24:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.831+0000 7fb1bdffb700 1 -- 172.21.15.133:0/2916157267 <== mds.1 v2:172.21.15.174:6836/1955090548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560997174660 con 0x5609972bf0e0 2023-11-27T15:24:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.831+0000 7fb1bdffb700 10 client.15708 handle_command_reply: tid=0 2023-11-27T15:24:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.832+0000 7fb1eff0db80 1 client.15708 shutdown 2023-11-27T15:24:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.832+0000 7fb1eff0db80 1 -- 172.21.15.133:0/2916157267 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997254c00 msgr2=0x560997261b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.832+0000 7fb1eff0db80 1 --2- 172.21.15.133:0/2916157267 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997254c00 0x560997261b60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0096a80 tx=0x7fb1d0099a70 comp rx=0 tx=0).stop 2023-11-27T15:24:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.832+0000 7fb1eff0db80 1 -- 172.21.15.133:0/2916157267 shutdown_connections 2023-11-27T15:24:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.832+0000 7fb1eff0db80 1 --2- 172.21.15.133:0/2916157267 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560997256110 0x5609972650c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.832+0000 7fb1eff0db80 1 --2- 172.21.15.133:0/2916157267 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560997254c00 0x560997261b60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.832+0000 7fb1eff0db80 1 --2- 172.21.15.133:0/2916157267 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x5609972bf0e0 0x5609972c14f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.832+0000 7fb1eff0db80 1 --2- 172.21.15.133:0/2916157267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560997255550 0x5609972620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.832+0000 7fb1bdffb700 0 client.15708 ms_handle_reset on v2:172.21.15.174:6836/1955090548 2023-11-27T15:24:12.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.832+0000 7fb1eff0db80 1 -- 172.21.15.133:0/2916157267 >> 172.21.15.133:0/2916157267 conn(0x56099724a360 msgr2=0x560997258fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:12.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.832+0000 7fb1eff0db80 1 -- 172.21.15.133:0/2916157267 shutdown_connections 2023-11-27T15:24:12.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.832+0000 7fb1eff0db80 1 -- 172.21.15.133:0/2916157267 wait complete. 2023-11-27T15:24:12.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.832+0000 7fb1eff0db80 20 client.15708 trim_cache size 0 max 16384 2023-11-27T15:24:12.841 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2023-11-27T15:24:12.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.833+0000 7fb1dc5ff700 1 -- 172.21.15.133:0/3244944506 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb1c00a1150 msgr2=0x7fb1c00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:12.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.833+0000 7fb1dc5ff700 1 --2- 172.21.15.133:0/3244944506 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb1c00a1150 0x7fb1c00a3600 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4009bd0 tx=0x7fb1c4009260 comp rx=0 tx=0).stop 2023-11-27T15:24:12.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.833+0000 7fb1dc5ff700 1 -- 172.21.15.133:0/3244944506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1d41002c0 msgr2=0x7fb1d40748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:12.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.833+0000 7fb1dc5ff700 1 --2- 172.21.15.133:0/3244944506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1d41002c0 0x7fb1d40748a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000e970 tx=0x7fb1d000ed30 comp rx=0 tx=0).stop 2023-11-27T15:24:12.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.833+0000 7fb1dc5ff700 1 -- 172.21.15.133:0/3244944506 shutdown_connections 2023-11-27T15:24:12.842 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.833+0000 7fb1dc5ff700 1 --2- 172.21.15.133:0/3244944506 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb1c00a1150 0x7fb1c00a3600 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.833+0000 7fb1dc5ff700 1 --2- 172.21.15.133:0/3244944506 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb1d4101e40 0x7fb1d4073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.833+0000 7fb1dc5ff700 1 --2- 172.21.15.133:0/3244944506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1d41002c0 0x7fb1d40748a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.833+0000 7fb1dc5ff700 1 --2- 172.21.15.133:0/3244944506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1d41014b0 0x7fb1d4072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:12.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.833+0000 7fb1dc5ff700 1 -- 172.21.15.133:0/3244944506 >> 172.21.15.133:0/3244944506 conn(0x7fb1d40fb8a0 msgr2=0x7fb1d41048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:12.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.833+0000 7fb1dc5ff700 1 -- 172.21.15.133:0/3244944506 shutdown_connections 2023-11-27T15:24:12.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:12.833+0000 7fb1dc5ff700 1 -- 172.21.15.133:0/3244944506 wait complete. 2023-11-27T15:24:12.902 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:24:13.481 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:13.481 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:13.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:13 smithi133 ceph-mon[97860]: pgmap v387: 129 pgs: 129 active+clean; 1.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2023-11-27T15:24:13.575 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:13.575 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:13.597 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:13.597 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:13.597 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 39 -lt 100 ']' 2023-11-27T15:24:13.597 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 39 + 1 2023-11-27T15:24:13.623 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=40 2023-11-27T15:24:13.623 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:13.652 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:13 smithi174 ceph-mon[131110]: pgmap v387: 129 pgs: 129 active+clean; 1.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2023-11-27T15:24:13.663 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:13.664 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:13.669 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:13.669 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:13.669 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:13.838 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:13 smithi177 ceph-mon[131504]: pgmap v387: 129 pgs: 129 active+clean; 1.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 13 op/s 2023-11-27T15:24:14.284 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:14.989 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:14.989 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:14.989 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:14.989 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:14.989 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:14.989 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 40 -lt 100 ']' 2023-11-27T15:24:14.989 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 40 + 1 2023-11-27T15:24:14.989 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=41 2023-11-27T15:24:14.990 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:14.990 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:14.990 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:14.990 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:14.990 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:14.990 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:15.472 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:15.472 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:15.652 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:15 smithi174 ceph-mon[131110]: pgmap v388: 129 pgs: 129 active+clean; 1.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-11-27T15:24:15.685 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:15.685 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:15.697 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:15.697 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:15.697 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 41 -lt 100 ']' 2023-11-27T15:24:15.698 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 41 + 1 2023-11-27T15:24:15.773 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=42 2023-11-27T15:24:15.773 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:15.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:15 smithi133 ceph-mon[97860]: pgmap v388: 129 pgs: 129 active+clean; 1.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-11-27T15:24:15.837 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:15 smithi177 ceph-mon[131504]: pgmap v388: 129 pgs: 129 active+clean; 1.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-11-27T15:24:15.930 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:15.931 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:15.933 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:15.934 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:15.934 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:15.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.952+0000 7ff217bff700 1 -- 172.21.15.133:0/1804555544 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff210073550 msgr2=0x7ff2100739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:15.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.952+0000 7ff217bff700 1 --2- 172.21.15.133:0/1804555544 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff210073550 0x7ff2100739c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff2000098f0 tx=0x7ff200009c00 comp rx=0 tx=0).stop 2023-11-27T15:24:15.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.952+0000 7ff217bff700 1 -- 172.21.15.133:0/1804555544 shutdown_connections 2023-11-27T15:24:15.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.952+0000 7ff217bff700 1 --2- 172.21.15.133:0/1804555544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff210108180 0x7ff21010a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:15.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.952+0000 7ff217bff700 1 --2- 172.21.15.133:0/1804555544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff210074b30 0x7ff210072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:15.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.952+0000 7ff217bff700 1 --2- 172.21.15.133:0/1804555544 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff210073550 0x7ff2100739c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:15.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.952+0000 7ff217bff700 1 -- 172.21.15.133:0/1804555544 >> 172.21.15.133:0/1804555544 conn(0x7ff2100fb8a0 msgr2=0x7ff2100fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:15.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.952+0000 7ff217bff700 1 -- 172.21.15.133:0/1804555544 shutdown_connections 2023-11-27T15:24:15.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.952+0000 7ff217bff700 1 -- 172.21.15.133:0/1804555544 wait complete. 2023-11-27T15:24:15.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.953+0000 7ff217bff700 1 Processor -- start 2023-11-27T15:24:15.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.954+0000 7ff217bff700 1 -- start start 2023-11-27T15:24:15.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.954+0000 7ff217bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff210073550 0x7ff2101982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:15.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.954+0000 7ff217bff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff210074b30 0x7ff210198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:15.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.954+0000 7ff217bff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff210108180 0x7ff21019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:15.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.954+0000 7ff217bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21019fe40 con 0x7ff210073550 2023-11-27T15:24:15.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.954+0000 7ff217bff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21019ff80 con 0x7ff210108180 2023-11-27T15:24:15.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.954+0000 7ff217bff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2101a0110 con 0x7ff210074b30 2023-11-27T15:24:15.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.954+0000 7ff21519a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff210074b30 0x7ff210198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:15.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.954+0000 7ff21599b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff210073550 0x7ff2101982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:15.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.954+0000 7ff21599b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff210073550 0x7ff2101982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58130/0 (socket says 172.21.15.133:58130) 2023-11-27T15:24:15.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.954+0000 7ff21599b700 1 -- 172.21.15.133:0/1475107037 learned_addr learned my addr 172.21.15.133:0/1475107037 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:15.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.954+0000 7ff21519a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff210074b30 0x7ff210198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:52104/0 (socket says 172.21.15.133:52104) 2023-11-27T15:24:15.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.954+0000 7ff21619c700 1 --2- 172.21.15.133:0/1475107037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff210108180 0x7ff21019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:15.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.955+0000 7ff21599b700 1 -- 172.21.15.133:0/1475107037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff210074b30 msgr2=0x7ff210198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:15.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.955+0000 7ff21599b700 1 --2- 172.21.15.133:0/1475107037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff210074b30 0x7ff210198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:15.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.955+0000 7ff21599b700 1 -- 172.21.15.133:0/1475107037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff210108180 msgr2=0x7ff21019f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:15.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.955+0000 7ff21599b700 1 --2- 172.21.15.133:0/1475107037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff210108180 0x7ff21019f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:15.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.955+0000 7ff21599b700 1 -- 172.21.15.133:0/1475107037 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff200009520 con 0x7ff210073550 2023-11-27T15:24:15.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.955+0000 7ff21599b700 1 --2- 172.21.15.133:0/1475107037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff210073550 0x7ff2101982c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff20c00e970 tx=0x7ff20c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:15.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.955+0000 7ff206ffd700 1 -- 172.21.15.133:0/1475107037 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20c009790 con 0x7ff210073550 2023-11-27T15:24:15.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.955+0000 7ff217bff700 1 -- 172.21.15.133:0/1475107037 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff21010de50 con 0x7ff210073550 2023-11-27T15:24:15.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.956+0000 7ff206ffd700 1 -- 172.21.15.133:0/1475107037 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff20c00c7e0 con 0x7ff210073550 2023-11-27T15:24:15.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.956+0000 7ff206ffd700 1 -- 172.21.15.133:0/1475107037 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20c0054e0 con 0x7ff210073550 2023-11-27T15:24:15.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.956+0000 7ff217bff700 1 -- 172.21.15.133:0/1475107037 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21010dcc0 con 0x7ff210073550 2023-11-27T15:24:15.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.959+0000 7ff206ffd700 1 -- 172.21.15.133:0/1475107037 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff20c00c950 con 0x7ff210073550 2023-11-27T15:24:15.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.960+0000 7ff206ffd700 1 --2- 172.21.15.133:0/1475107037 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff1fc0a1150 0x7ff1fc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:15.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.961+0000 7ff206ffd700 1 -- 172.21.15.133:0/1475107037 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ff20c014070 con 0x7ff210073550 2023-11-27T15:24:15.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.961+0000 7ff21519a700 1 --2- 172.21.15.133:0/1475107037 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff1fc0a1150 0x7ff1fc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:15.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:15.961+0000 7ff21519a700 1 --2- 172.21.15.133:0/1475107037 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff1fc0a1150 0x7ff1fc0a3600 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff2000098c0 tx=0x7ff200019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:16.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.101+0000 7ff22b60cb80 1 Processor -- start 2023-11-27T15:24:16.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.102+0000 7ff22b60cb80 1 -- start start 2023-11-27T15:24:16.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.102+0000 7ff22b60cb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e51aa4c00 0x558e51aa5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:16.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.102+0000 7ff22b60cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa5550 0x558e51aa5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:16.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.102+0000 7ff22b60cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa6110 0x558e51aa6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:16.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.102+0000 7ff22b60cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e519aa060 con 0x558e51aa5550 2023-11-27T15:24:16.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.102+0000 7ff22b60cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e519fc8a0 con 0x558e51aa6110 2023-11-27T15:24:16.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.102+0000 7ff22b60cb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e519f5d30 con 0x558e51aa4c00 2023-11-27T15:24:16.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.102+0000 7ff21519a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa5550 0x558e51aa5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:16.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.103+0000 7ff21519a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa5550 0x558e51aa5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58134/0 (socket says 172.21.15.133:58134) 2023-11-27T15:24:16.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.103+0000 7ff21519a700 1 -- 172.21.15.133:0/1567222748 learned_addr learned my addr 172.21.15.133:0/1567222748 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:16.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.103+0000 7ff21619c700 1 --2- 172.21.15.133:0/1567222748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa6110 0x558e51aa6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:16.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.103+0000 7ff21599b700 1 --2- 172.21.15.133:0/1567222748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e51aa4c00 0x558e51aa5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:16.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.103+0000 7ff21619c700 1 -- 172.21.15.133:0/1567222748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e51aa4c00 msgr2=0x558e51aa5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:16.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.103+0000 7ff21619c700 1 --2- 172.21.15.133:0/1567222748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e51aa4c00 0x558e51aa5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.103+0000 7ff21619c700 1 -- 172.21.15.133:0/1567222748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa5550 msgr2=0x558e51aa5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:16.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.103+0000 7ff21619c700 1 --2- 172.21.15.133:0/1567222748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa5550 0x558e51aa5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.103+0000 7ff21619c700 1 -- 172.21.15.133:0/1567222748 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e51877630 con 0x558e51aa6110 2023-11-27T15:24:16.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.103+0000 7ff21619c700 1 --2- 172.21.15.133:0/1567222748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa6110 0x558e51aa6580 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff208011580 tx=0x7ff2080099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d72d4a085ffd2d2a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:16.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.104+0000 7ff1fbfff700 1 -- 172.21.15.133:0/1567222748 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20800e040 con 0x558e51aa6110 2023-11-27T15:24:16.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.104+0000 7ff1fbfff700 1 -- 172.21.15.133:0/1567222748 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff20800f040 con 0x558e51aa6110 2023-11-27T15:24:16.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.104+0000 7ff1fbfff700 1 -- 172.21.15.133:0/1567222748 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2080043b0 con 0x558e51aa6110 2023-11-27T15:24:16.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.105+0000 7ff22b60cb80 1 -- 172.21.15.133:0/1567222748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa6110 msgr2=0x558e51aa6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:16.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.105+0000 7ff22b60cb80 1 --2- 172.21.15.133:0/1567222748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa6110 0x558e51aa6580 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff208011580 tx=0x7ff2080099e0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.105+0000 7ff22b60cb80 1 -- 172.21.15.133:0/1567222748 shutdown_connections 2023-11-27T15:24:16.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.105+0000 7ff22b60cb80 1 --2- 172.21.15.133:0/1567222748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa6110 0x558e51aa6580 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.105+0000 7ff22b60cb80 1 --2- 172.21.15.133:0/1567222748 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e51aa4c00 0x558e51aa5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.105+0000 7ff22b60cb80 1 --2- 172.21.15.133:0/1567222748 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa5550 0x558e51aa5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.105+0000 7ff22b60cb80 1 -- 172.21.15.133:0/1567222748 >> 172.21.15.133:0/1567222748 conn(0x558e51a9a360 msgr2=0x558e51a9f670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:16.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.105+0000 7ff22b60cb80 1 -- 172.21.15.133:0/1567222748 shutdown_connections 2023-11-27T15:24:16.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.105+0000 7ff22b60cb80 1 -- 172.21.15.133:0/1567222748 wait complete. 2023-11-27T15:24:16.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.106+0000 7ff22b60cb80 1 -- start start 2023-11-27T15:24:16.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.106+0000 7ff22b60cb80 1 Processor -- start 2023-11-27T15:24:16.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.106+0000 7ff22b60cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa4c00 0x558e51aae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:16.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.106+0000 7ff22b60cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa5550 0x558e51aaeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:16.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.106+0000 7ff22b60cb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e51aa6110 0x558e51ab5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:16.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.106+0000 7ff22b60cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e519978e0 con 0x558e51aa4c00 2023-11-27T15:24:16.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.106+0000 7ff22b60cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e51997580 con 0x558e51aa5550 2023-11-27T15:24:16.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.106+0000 7ff22b60cb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e51879920 con 0x558e51aa6110 2023-11-27T15:24:16.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.106+0000 7ff22b60cb80 10 client.0 _ll_register_callbacks cb 0x558e519b5780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:16.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.107+0000 7ff21599b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa4c00 0x558e51aae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:16.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.107+0000 7ff21599b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa4c00 0x558e51aae5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58138/0 (socket says 172.21.15.133:58138) 2023-11-27T15:24:16.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.107+0000 7ff21599b700 1 -- 172.21.15.133:0/353665031 learned_addr learned my addr 172.21.15.133:0/353665031 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:16.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.107+0000 7ff21519a700 1 --2- 172.21.15.133:0/353665031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa5550 0x558e51aaeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:16.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.107+0000 7ff21619c700 1 --2- 172.21.15.133:0/353665031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e51aa6110 0x558e51ab5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:16.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.107+0000 7ff21519a700 1 -- 172.21.15.133:0/353665031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e51aa6110 msgr2=0x558e51ab5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:16.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.107+0000 7ff21519a700 1 --2- 172.21.15.133:0/353665031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e51aa6110 0x558e51ab5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.107+0000 7ff21519a700 1 -- 172.21.15.133:0/353665031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa4c00 msgr2=0x558e51aae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:16.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.107+0000 7ff21519a700 1 --2- 172.21.15.133:0/353665031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa4c00 0x558e51aae5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.107+0000 7ff21519a700 1 -- 172.21.15.133:0/353665031 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff208011230 con 0x558e51aa5550 2023-11-27T15:24:16.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.107+0000 7ff21599b700 1 --2- 172.21.15.133:0/353665031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa4c00 0x558e51aae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:16.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.107+0000 7ff21619c700 1 --2- 172.21.15.133:0/353665031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e51aa6110 0x558e51ab5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:16.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.108+0000 7ff21519a700 1 --2- 172.21.15.133:0/353665031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa5550 0x558e51aaeb20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff20000fa40 tx=0x7ff200009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31eae20d1f92b542 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:16.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.108+0000 7ff1f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:24:16.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.108+0000 7ff1f97fa700 1 -- 172.21.15.133:0/353665031 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20002b040 con 0x558e51aa5550 2023-11-27T15:24:16.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.108+0000 7ff1f97fa700 1 -- 172.21.15.133:0/353665031 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ff2000054d0 con 0x558e51aa5550 2023-11-27T15:24:16.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.108+0000 7ff22b60cb80 1 -- 172.21.15.133:0/353665031 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e51a97b20 con 0x558e51aa5550 2023-11-27T15:24:16.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.108+0000 7ff1f97fa700 1 -- 172.21.15.133:0/353665031 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2000056b0 con 0x558e51aa5550 2023-11-27T15:24:16.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.108+0000 7ff22b60cb80 10 client.25760 fetch_fsmap learned FSMap version 23 2023-11-27T15:24:16.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.108+0000 7ff22b60cb80 1 -- 172.21.15.133:0/353665031 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558e519bc190 con 0x558e51aa5550 2023-11-27T15:24:16.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.109+0000 7ff1f97fa700 1 -- 172.21.15.133:0/353665031 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558e51a97b20 con 0x558e51aa5550 2023-11-27T15:24:16.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.109+0000 7ff1f97fa700 1 -- 172.21.15.133:0/353665031 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff200037070 con 0x558e51aa5550 2023-11-27T15:24:16.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.109+0000 7ff22b60cb80 10 client.25760 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:24:16.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.109+0000 7ff22b60cb80 10 client.25760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:16.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.109+0000 7ff22b60cb80 20 client.25760 populate_metadata read hostname 'smithi133' 2023-11-27T15:24:16.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.109+0000 7ff22b60cb80 1 --2- 172.21.15.133:0/353665031 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x558e51aba340 0x558e51abc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:16.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.109+0000 7ff22b60cb80 4 client.25760 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:16.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.109+0000 7ff22b60cb80 1 -- 172.21.15.133:0/353665031 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e519b80a0 con 0x558e51aba340 2023-11-27T15:24:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.109+0000 7ff21599b700 1 --2- 172.21.15.133:0/353665031 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x558e51aba340 0x558e51abc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.110+0000 7ff1f97fa700 10 client.25760 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.110+0000 7ff21599b700 1 --2- 172.21.15.133:0/353665031 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x558e51aba340 0x558e51abc750 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2152e2308bb4d900 server_cookie=dfc21b98023e7998 in_seq=0 out_seq=0 2023-11-27T15:24:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.110+0000 7ff1f97fa700 1 -- 172.21.15.133:0/353665031 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558e519b80a0 con 0x558e51aba340 2023-11-27T15:24:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.110+0000 7ff1f97fa700 10 client.25760 handle_command_reply: tid=0 2023-11-27T15:24:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.111+0000 7ff22b60cb80 1 client.25760 shutdown 2023-11-27T15:24:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.111+0000 7ff22b60cb80 1 -- 172.21.15.133:0/353665031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa5550 msgr2=0x558e51aaeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.111+0000 7ff22b60cb80 1 --2- 172.21.15.133:0/353665031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa5550 0x558e51aaeb20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff20000fa40 tx=0x7ff200009c00 comp rx=0 tx=0).stop 2023-11-27T15:24:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.111+0000 7ff22b60cb80 1 -- 172.21.15.133:0/353665031 shutdown_connections 2023-11-27T15:24:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.111+0000 7ff22b60cb80 1 --2- 172.21.15.133:0/353665031 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x558e51aba340 0x558e51abc750 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.111+0000 7ff22b60cb80 1 --2- 172.21.15.133:0/353665031 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e51aa6110 0x558e51ab5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.111+0000 7ff22b60cb80 1 --2- 172.21.15.133:0/353665031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa4c00 0x558e51aae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.111+0000 7ff1f97fa700 0 client.25760 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:16.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.111+0000 7ff22b60cb80 1 --2- 172.21.15.133:0/353665031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa5550 0x558e51aaeb20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.111+0000 7ff22b60cb80 1 -- 172.21.15.133:0/353665031 >> 172.21.15.133:0/353665031 conn(0x558e51a9a360 msgr2=0x558e51a9ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:16.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.111+0000 7ff22b60cb80 1 -- 172.21.15.133:0/353665031 shutdown_connections 2023-11-27T15:24:16.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.111+0000 7ff22b60cb80 1 -- 172.21.15.133:0/353665031 wait complete. 2023-11-27T15:24:16.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.111+0000 7ff22b60cb80 20 client.25760 trim_cache size 0 max 16384 2023-11-27T15:24:16.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.125+0000 7ff22b60cb80 1 Processor -- start 2023-11-27T15:24:16.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.125+0000 7ff22b60cb80 1 -- start start 2023-11-27T15:24:16.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.126+0000 7ff22b60cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa4c00 0x558e51aa64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:16.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.126+0000 7ff22b60cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa5550 0x558e51ab5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:16.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.126+0000 7ff22b60cb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e51aa6110 0x558e51ac10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:16.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.126+0000 7ff22b60cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e51aada90 con 0x558e51aa5550 2023-11-27T15:24:16.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.126+0000 7ff22b60cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e51aae150 con 0x558e51aa4c00 2023-11-27T15:24:16.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.126+0000 7ff22b60cb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e51aacd20 con 0x558e51aa6110 2023-11-27T15:24:16.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.126+0000 7ff21599b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa4c00 0x558e51aa64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:16.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.126+0000 7ff21519a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa5550 0x558e51ab5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:16.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.126+0000 7ff21599b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa4c00 0x558e51aa64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:56668/0 (socket says 172.21.15.133:56668) 2023-11-27T15:24:16.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.126+0000 7ff21599b700 1 -- 172.21.15.133:0/727042966 learned_addr learned my addr 172.21.15.133:0/727042966 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:16.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.126+0000 7ff21619c700 1 --2- 172.21.15.133:0/727042966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e51aa6110 0x558e51ac10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:16.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.126+0000 7ff21599b700 1 -- 172.21.15.133:0/727042966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e51aa6110 msgr2=0x558e51ac10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:16.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.126+0000 7ff21599b700 1 --2- 172.21.15.133:0/727042966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e51aa6110 0x558e51ac10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.126+0000 7ff21599b700 1 -- 172.21.15.133:0/727042966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa5550 msgr2=0x558e51ab5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:16.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.126+0000 7ff21599b700 1 --2- 172.21.15.133:0/727042966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa5550 0x558e51ab5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.126+0000 7ff21599b700 1 -- 172.21.15.133:0/727042966 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e519bbee0 con 0x558e51aa4c00 2023-11-27T15:24:16.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.127+0000 7ff21599b700 1 --2- 172.21.15.133:0/727042966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa4c00 0x558e51aa64e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff20c00efd0 tx=0x7ff20c0cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbd3c7f7ea8f27aa server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:16.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.127+0000 7ff204bff700 1 -- 172.21.15.133:0/727042966 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20c0a23d0 con 0x558e51aa4c00 2023-11-27T15:24:16.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.127+0000 7ff204bff700 1 -- 172.21.15.133:0/727042966 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff20c016e20 con 0x558e51aa4c00 2023-11-27T15:24:16.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.127+0000 7ff204bff700 1 -- 172.21.15.133:0/727042966 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20c099470 con 0x558e51aa4c00 2023-11-27T15:24:16.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.128+0000 7ff22b60cb80 1 -- 172.21.15.133:0/727042966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa4c00 msgr2=0x558e51aa64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.128+0000 7ff22b60cb80 1 --2- 172.21.15.133:0/727042966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa4c00 0x558e51aa64e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff20c00efd0 tx=0x7ff20c0cc040 comp rx=0 tx=0).stop 2023-11-27T15:24:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.128+0000 7ff22b60cb80 1 -- 172.21.15.133:0/727042966 shutdown_connections 2023-11-27T15:24:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.128+0000 7ff22b60cb80 1 --2- 172.21.15.133:0/727042966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e51aa6110 0x558e51ac10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.128+0000 7ff22b60cb80 1 --2- 172.21.15.133:0/727042966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa4c00 0x558e51aa64e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.128+0000 7ff22b60cb80 1 --2- 172.21.15.133:0/727042966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa5550 0x558e51ab5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.128+0000 7ff22b60cb80 1 -- 172.21.15.133:0/727042966 >> 172.21.15.133:0/727042966 conn(0x558e51a9a360 msgr2=0x558e51a9f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.128+0000 7ff22b60cb80 1 -- 172.21.15.133:0/727042966 shutdown_connections 2023-11-27T15:24:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.128+0000 7ff22b60cb80 1 -- 172.21.15.133:0/727042966 wait complete. 2023-11-27T15:24:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.128+0000 7ff22b60cb80 1 -- start start 2023-11-27T15:24:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.128+0000 7ff22b60cb80 1 Processor -- start 2023-11-27T15:24:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.129+0000 7ff22b60cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa4c00 0x558e51ab1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:16.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.129+0000 7ff22b60cb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e51aa5550 0x558e51ab1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.129+0000 7ff22b60cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa6110 0x558e51ab4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.129+0000 7ff22b60cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e51ab50a0 con 0x558e51aa6110 2023-11-27T15:24:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.129+0000 7ff22b60cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e51ab5210 con 0x558e51aa4c00 2023-11-27T15:24:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.129+0000 7ff22b60cb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e51ab53d0 con 0x558e51aa5550 2023-11-27T15:24:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.129+0000 7ff22b60cb80 10 client.0 _ll_register_callbacks cb 0x7ff20000b890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.129+0000 7ff21599b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa4c00 0x558e51ab1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.129+0000 7ff21619c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa6110 0x558e51ab4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.129+0000 7ff21519a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e51aa5550 0x558e51ab1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.129+0000 7ff21599b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa4c00 0x558e51ab1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:56672/0 (socket says 172.21.15.133:56672) 2023-11-27T15:24:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.129+0000 7ff21599b700 1 -- 172.21.15.133:0/2282457174 learned_addr learned my addr 172.21.15.133:0/2282457174 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.129+0000 7ff21619c700 1 --2- 172.21.15.133:0/2282457174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa6110 0x558e51ab4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58156/0 (socket says 172.21.15.133:58156) 2023-11-27T15:24:16.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.130+0000 7ff21599b700 1 -- 172.21.15.133:0/2282457174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e51aa5550 msgr2=0x558e51ab1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.130+0000 7ff21599b700 1 --2- 172.21.15.133:0/2282457174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e51aa5550 0x558e51ab1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.130+0000 7ff21599b700 1 -- 172.21.15.133:0/2282457174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa6110 msgr2=0x558e51ab4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.130+0000 7ff21599b700 1 --2- 172.21.15.133:0/2282457174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa6110 0x558e51ab4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.130+0000 7ff21599b700 1 -- 172.21.15.133:0/2282457174 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21010dcc0 con 0x558e51aa4c00 2023-11-27T15:24:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.130+0000 7ff21599b700 1 --2- 172.21.15.133:0/2282457174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa4c00 0x558e51ab1600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0a1ea0 tx=0x7ff20c016f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46b4f66f8a6b99db server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.130+0000 7ff1f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:24:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.130+0000 7ff1f97fa700 1 -- 172.21.15.133:0/2282457174 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20c005640 con 0x558e51aa4c00 2023-11-27T15:24:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.130+0000 7ff22b60cb80 1 -- 172.21.15.133:0/2282457174 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff20800f040 con 0x558e51aa4c00 2023-11-27T15:24:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.130+0000 7ff1f97fa700 1 -- 172.21.15.133:0/2282457174 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ff20c0ca1b0 con 0x558e51aa4c00 2023-11-27T15:24:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.130+0000 7ff1f97fa700 1 -- 172.21.15.133:0/2282457174 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff20c0ca370 con 0x558e51aa4c00 2023-11-27T15:24:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.130+0000 7ff22b60cb80 10 client.25766 fetch_fsmap learned FSMap version 23 2023-11-27T15:24:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.130+0000 7ff22b60cb80 1 -- 172.21.15.133:0/2282457174 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558e5180cc60 con 0x558e51aa4c00 2023-11-27T15:24:16.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.131+0000 7ff1f97fa700 1 -- 172.21.15.133:0/2282457174 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff20800f040 con 0x558e51aa4c00 2023-11-27T15:24:16.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.131+0000 7ff1f97fa700 1 -- 172.21.15.133:0/2282457174 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff20c094b90 con 0x558e51aa4c00 2023-11-27T15:24:16.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.131+0000 7ff22b60cb80 10 client.25766 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:24:16.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.131+0000 7ff22b60cb80 10 client.25766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:16.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.131+0000 7ff22b60cb80 20 client.25766 populate_metadata read hostname 'smithi133' 2023-11-27T15:24:16.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.131+0000 7ff22b60cb80 1 --2- 172.21.15.133:0/2282457174 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x558e51b0ea10 0x558e51b10e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:16.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.131+0000 7ff22b60cb80 4 client.25766 mds_command: new command op to 24560 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:24:16.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.131+0000 7ff22b60cb80 1 -- 172.21.15.133:0/2282457174 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558e519c4660 con 0x558e51b0ea10 2023-11-27T15:24:16.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.131+0000 7ff21519a700 1 --2- 172.21.15.133:0/2282457174 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x558e51b0ea10 0x558e51b10e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:16.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.132+0000 7ff1f97fa700 10 client.25766 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:16.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.132+0000 7ff21519a700 1 --2- 172.21.15.133:0/2282457174 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x558e51b0ea10 0x558e51b10e20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=592c61e07ceb8e76 server_cookie=ccc064de233e24af in_seq=0 out_seq=0 2023-11-27T15:24:16.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.132+0000 7ff1f97fa700 1 -- 172.21.15.133:0/2282457174 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558e519c4660 con 0x558e51b0ea10 2023-11-27T15:24:16.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.132+0000 7ff1f97fa700 10 client.25766 handle_command_reply: tid=0 2023-11-27T15:24:16.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.132+0000 7ff22b60cb80 1 client.25766 shutdown 2023-11-27T15:24:16.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.132+0000 7ff22b60cb80 1 -- 172.21.15.133:0/2282457174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa4c00 msgr2=0x558e51ab1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:16.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.132+0000 7ff22b60cb80 1 --2- 172.21.15.133:0/2282457174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa4c00 0x558e51ab1600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0a1ea0 tx=0x7ff20c016f90 comp rx=0 tx=0).stop 2023-11-27T15:24:16.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.132+0000 7ff22b60cb80 1 -- 172.21.15.133:0/2282457174 shutdown_connections 2023-11-27T15:24:16.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.132+0000 7ff22b60cb80 1 --2- 172.21.15.133:0/2282457174 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x558e51b0ea10 0x558e51b10e20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.132+0000 7ff22b60cb80 1 --2- 172.21.15.133:0/2282457174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558e51aa6110 0x558e51ab4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.132+0000 7ff22b60cb80 1 --2- 172.21.15.133:0/2282457174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e51aa4c00 0x558e51ab1600 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.132+0000 7ff1f97fa700 0 client.25766 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:16.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.132+0000 7ff22b60cb80 1 --2- 172.21.15.133:0/2282457174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558e51aa5550 0x558e51ab1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.132+0000 7ff22b60cb80 1 -- 172.21.15.133:0/2282457174 >> 172.21.15.133:0/2282457174 conn(0x558e51a9a360 msgr2=0x558e51aa8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:16.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.132+0000 7ff22b60cb80 1 -- 172.21.15.133:0/2282457174 shutdown_connections 2023-11-27T15:24:16.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.132+0000 7ff22b60cb80 1 -- 172.21.15.133:0/2282457174 wait complete. 2023-11-27T15:24:16.141 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.133+0000 7ff22b60cb80 20 client.25766 trim_cache size 0 max 16384 2023-11-27T15:24:16.141 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2023-11-27T15:24:16.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.133+0000 7ff217bff700 1 -- 172.21.15.133:0/1475107037 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff1fc0a1150 msgr2=0x7ff1fc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:16.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.133+0000 7ff217bff700 1 --2- 172.21.15.133:0/1475107037 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff1fc0a1150 0x7ff1fc0a3600 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff2000098c0 tx=0x7ff200019040 comp rx=0 tx=0).stop 2023-11-27T15:24:16.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.133+0000 7ff217bff700 1 -- 172.21.15.133:0/1475107037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff210073550 msgr2=0x7ff2101982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:16.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.133+0000 7ff217bff700 1 --2- 172.21.15.133:0/1475107037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff210073550 0x7ff2101982c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff20c00e970 tx=0x7ff20c00ec80 comp rx=0 tx=0).stop 2023-11-27T15:24:16.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.133+0000 7ff217bff700 1 -- 172.21.15.133:0/1475107037 shutdown_connections 2023-11-27T15:24:16.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.133+0000 7ff217bff700 1 --2- 172.21.15.133:0/1475107037 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff1fc0a1150 0x7ff1fc0a3600 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.133+0000 7ff217bff700 1 --2- 172.21.15.133:0/1475107037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff210108180 0x7ff21019f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.133+0000 7ff217bff700 1 --2- 172.21.15.133:0/1475107037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff210073550 0x7ff2101982c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.133+0000 7ff217bff700 1 --2- 172.21.15.133:0/1475107037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff210074b30 0x7ff210198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:16.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.133+0000 7ff217bff700 1 -- 172.21.15.133:0/1475107037 >> 172.21.15.133:0/1475107037 conn(0x7ff2100fb8a0 msgr2=0x7ff2100fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:16.142 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.134+0000 7ff217bff700 1 -- 172.21.15.133:0/1475107037 shutdown_connections 2023-11-27T15:24:16.143 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:16.134+0000 7ff217bff700 1 -- 172.21.15.133:0/1475107037 wait complete. 2023-11-27T15:24:16.680 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:16.680 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:16.812 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:16.812 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:16.855 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:16.855 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:16.855 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 42 -lt 100 ']' 2023-11-27T15:24:16.855 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 42 + 1 2023-11-27T15:24:16.922 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=43 2023-11-27T15:24:16.923 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:17.007 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:17.007 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:17.010 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:17.010 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:17.011 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:17.202 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:24:17.202 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:24:17.651 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:17 smithi133 ceph-mon[97860]: pgmap v389: 129 pgs: 129 active+clean; 1.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-11-27T15:24:17.652 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:17 smithi174 ceph-mon[131110]: pgmap v389: 129 pgs: 129 active+clean; 1.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-11-27T15:24:17.796 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:17.797 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:17.837 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:17 smithi177 ceph-mon[131504]: pgmap v389: 129 pgs: 129 active+clean; 1.0 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 11 op/s 2023-11-27T15:24:17.864 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:17.865 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:17.889 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:17.889 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:17.889 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 43 -lt 100 ']' 2023-11-27T15:24:17.889 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 43 + 1 2023-11-27T15:24:17.973 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=44 2023-11-27T15:24:17.973 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:17.992 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:17.992 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:17.994 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:17.995 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:17.995 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.418+0000 7f2bf79ff700 1 -- 172.21.15.133:0/2157088763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf0101e40 msgr2=0x7f2bf0108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.418+0000 7f2bf79ff700 1 --2- 172.21.15.133:0/2157088763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf0101e40 0x7f2bf0108700 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2be800b0a0 tx=0x7f2be800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.418+0000 7f2bf79ff700 1 -- 172.21.15.133:0/2157088763 shutdown_connections 2023-11-27T15:24:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.418+0000 7f2bf79ff700 1 --2- 172.21.15.133:0/2157088763 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf0101e40 0x7f2bf0108700 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.418+0000 7f2bf79ff700 1 --2- 172.21.15.133:0/2157088763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bf01002c0 0x7f2bf01006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.418+0000 7f2bf79ff700 1 --2- 172.21.15.133:0/2157088763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bf01014b0 0x7f2bf0101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.418+0000 7f2bf79ff700 1 -- 172.21.15.133:0/2157088763 >> 172.21.15.133:0/2157088763 conn(0x7f2bf00fb8e0 msgr2=0x7f2bf00fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.418+0000 7f2bf79ff700 1 -- 172.21.15.133:0/2157088763 shutdown_connections 2023-11-27T15:24:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.418+0000 7f2bf79ff700 1 -- 172.21.15.133:0/2157088763 wait complete. 2023-11-27T15:24:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.419+0000 7f2bf79ff700 1 Processor -- start 2023-11-27T15:24:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.420+0000 7f2bf79ff700 1 -- start start 2023-11-27T15:24:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.420+0000 7f2bf79ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf01002c0 0x7f2bf00748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.420+0000 7f2bf79ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bf01014b0 0x7f2bf0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.420+0000 7f2bf79ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bf0101e40 0x7f2bf0073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.420+0000 7f2bf79ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf0073bc0 con 0x7f2bf01002c0 2023-11-27T15:24:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.420+0000 7f2bf79ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf0073d30 con 0x7f2bf01014b0 2023-11-27T15:24:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.420+0000 7f2bf79ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf01a0160 con 0x7f2bf0101e40 2023-11-27T15:24:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.420+0000 7f2bf579b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf01002c0 0x7f2bf00748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.420+0000 7f2bf579b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf01002c0 0x7f2bf00748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58180/0 (socket says 172.21.15.133:58180) 2023-11-27T15:24:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.420+0000 7f2bf579b700 1 -- 172.21.15.133:0/3083401779 learned_addr learned my addr 172.21.15.133:0/3083401779 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.420+0000 7f2bf5f9c700 1 --2- 172.21.15.133:0/3083401779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bf0101e40 0x7f2bf0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.420+0000 7f2bf4f9a700 1 --2- 172.21.15.133:0/3083401779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bf01014b0 0x7f2bf0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.421+0000 7f2bf579b700 1 -- 172.21.15.133:0/3083401779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bf0101e40 msgr2=0x7f2bf0073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.421+0000 7f2bf579b700 1 --2- 172.21.15.133:0/3083401779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bf0101e40 0x7f2bf0073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.421+0000 7f2bf579b700 1 -- 172.21.15.133:0/3083401779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bf01014b0 msgr2=0x7f2bf0072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.421+0000 7f2bf579b700 1 --2- 172.21.15.133:0/3083401779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bf01014b0 0x7f2bf0072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.421+0000 7f2bf579b700 1 -- 172.21.15.133:0/3083401779 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be8009d00 con 0x7f2bf01002c0 2023-11-27T15:24:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.421+0000 7f2bf579b700 1 --2- 172.21.15.133:0/3083401779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf01002c0 0x7f2bf00748a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00e970 tx=0x7f2bec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.421+0000 7f2be67fc700 1 -- 172.21.15.133:0/3083401779 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec009790 con 0x7f2bf01002c0 2023-11-27T15:24:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.421+0000 7f2be67fc700 1 -- 172.21.15.133:0/3083401779 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bec00c7e0 con 0x7f2bf01002c0 2023-11-27T15:24:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.421+0000 7f2bf79ff700 1 -- 172.21.15.133:0/3083401779 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf010bc30 con 0x7f2bf01002c0 2023-11-27T15:24:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.421+0000 7f2be67fc700 1 -- 172.21.15.133:0/3083401779 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec003c70 con 0x7f2bf01002c0 2023-11-27T15:24:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.421+0000 7f2bf79ff700 1 -- 172.21.15.133:0/3083401779 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf010baa0 con 0x7f2bf01002c0 2023-11-27T15:24:18.428 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.424+0000 7f2be67fc700 1 -- 172.21.15.133:0/3083401779 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2bec00cc50 con 0x7f2bf01002c0 2023-11-27T15:24:18.429 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.426+0000 7f2be67fc700 1 --2- 172.21.15.133:0/3083401779 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2bdc0a1150 0x7f2bdc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.426+0000 7f2be67fc700 1 -- 172.21.15.133:0/3083401779 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f2bec014070 con 0x7f2bf01002c0 2023-11-27T15:24:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.426+0000 7f2bf4f9a700 1 --2- 172.21.15.133:0/3083401779 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2bdc0a1150 0x7f2bdc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.430 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.427+0000 7f2bf4f9a700 1 --2- 172.21.15.133:0/3083401779 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2bdc0a1150 0x7f2bdc0a3600 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009c60 tx=0x7f2be0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.517+0000 7f2c0b239b80 1 Processor -- start 2023-11-27T15:24:18.521 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.518+0000 7f2c0b239b80 1 -- start start 2023-11-27T15:24:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.518+0000 7f2c0b239b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde55ac6b0 0x55dde5750870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.518+0000 7f2c0b239b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 0x55dde57515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.518+0000 7f2c0b239b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde5751b10 0x55dde5751f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.518+0000 7f2c0b239b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde5656000 con 0x55dde55ac6b0 2023-11-27T15:24:18.523 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.518+0000 7f2c0b239b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde56a8840 con 0x55dde5751b10 2023-11-27T15:24:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.518+0000 7f2c0b239b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde56a1cd0 con 0x55dde5750db0 2023-11-27T15:24:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.518+0000 7f2bf579b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde55ac6b0 0x55dde5750870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.519+0000 7f2bf579b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde55ac6b0 0x55dde5750870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58186/0 (socket says 172.21.15.133:58186) 2023-11-27T15:24:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.519+0000 7f2bf579b700 1 -- 172.21.15.133:0/906979502 learned_addr learned my addr 172.21.15.133:0/906979502 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.519+0000 7f2bf4f9a700 1 --2- 172.21.15.133:0/906979502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 0x55dde57515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.519+0000 7f2bf5f9c700 1 --2- 172.21.15.133:0/906979502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde5751b10 0x55dde5751f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.519+0000 7f2bf579b700 1 -- 172.21.15.133:0/906979502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 msgr2=0x55dde57515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.519+0000 7f2bf579b700 1 --2- 172.21.15.133:0/906979502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 0x55dde57515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.519+0000 7f2bf579b700 1 -- 172.21.15.133:0/906979502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde5751b10 msgr2=0x55dde5751f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.519+0000 7f2bf579b700 1 --2- 172.21.15.133:0/906979502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde5751b10 0x55dde5751f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.519+0000 7f2bf579b700 1 -- 172.21.15.133:0/906979502 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dde55235d0 con 0x55dde55ac6b0 2023-11-27T15:24:18.524 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.519+0000 7f2bf579b700 1 --2- 172.21.15.133:0/906979502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde55ac6b0 0x55dde5750870 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2bec099e40 tx=0x7f2bec099890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd719bef25a6155e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.519+0000 7f2bdb7fe700 1 -- 172.21.15.133:0/906979502 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec0b1070 con 0x55dde55ac6b0 2023-11-27T15:24:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.519+0000 7f2bdb7fe700 1 -- 172.21.15.133:0/906979502 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bec0a2300 con 0x55dde55ac6b0 2023-11-27T15:24:18.525 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.519+0000 7f2bdb7fe700 1 -- 172.21.15.133:0/906979502 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec09a370 con 0x55dde55ac6b0 2023-11-27T15:24:18.528 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.520+0000 7f2c0b239b80 1 -- 172.21.15.133:0/906979502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde55ac6b0 msgr2=0x55dde5750870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.520+0000 7f2c0b239b80 1 --2- 172.21.15.133:0/906979502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde55ac6b0 0x55dde5750870 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2bec099e40 tx=0x7f2bec099890 comp rx=0 tx=0).stop 2023-11-27T15:24:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.520+0000 7f2c0b239b80 1 -- 172.21.15.133:0/906979502 shutdown_connections 2023-11-27T15:24:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.520+0000 7f2c0b239b80 1 --2- 172.21.15.133:0/906979502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde5751b10 0x55dde5751f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.520+0000 7f2c0b239b80 1 --2- 172.21.15.133:0/906979502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde55ac6b0 0x55dde5750870 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.520+0000 7f2c0b239b80 1 --2- 172.21.15.133:0/906979502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 0x55dde57515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.520+0000 7f2c0b239b80 1 -- 172.21.15.133:0/906979502 >> 172.21.15.133:0/906979502 conn(0x55dde5745f50 msgr2=0x55dde574b100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.520+0000 7f2c0b239b80 1 -- 172.21.15.133:0/906979502 shutdown_connections 2023-11-27T15:24:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.520+0000 7f2c0b239b80 1 -- 172.21.15.133:0/906979502 wait complete. 2023-11-27T15:24:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.521+0000 7f2c0b239b80 1 -- start start 2023-11-27T15:24:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.521+0000 7f2c0b239b80 1 Processor -- start 2023-11-27T15:24:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.522+0000 7f2c0b239b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde55ac6b0 0x55dde5759cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.522+0000 7f2c0b239b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 0x55dde575a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.522+0000 7f2c0b239b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde5751b10 0x55dde5761240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.522+0000 7f2c0b239b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde5643880 con 0x55dde55ac6b0 2023-11-27T15:24:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.522+0000 7f2c0b239b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde5643520 con 0x55dde5751b10 2023-11-27T15:24:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.522+0000 7f2c0b239b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde55258c0 con 0x55dde5750db0 2023-11-27T15:24:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.522+0000 7f2c0b239b80 10 client.0 _ll_register_callbacks cb 0x55dde5661720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.522+0000 7f2bf579b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde55ac6b0 0x55dde5759cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.522+0000 7f2bf579b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde55ac6b0 0x55dde5759cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58196/0 (socket says 172.21.15.133:58196) 2023-11-27T15:24:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.522+0000 7f2bf579b700 1 -- 172.21.15.133:0/453685314 learned_addr learned my addr 172.21.15.133:0/453685314 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.522+0000 7f2bf4f9a700 1 --2- 172.21.15.133:0/453685314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 0x55dde575a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.522+0000 7f2bf5f9c700 1 --2- 172.21.15.133:0/453685314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde5751b10 0x55dde5761240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.522+0000 7f2bf579b700 1 -- 172.21.15.133:0/453685314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 msgr2=0x55dde575a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.522+0000 7f2bf579b700 1 --2- 172.21.15.133:0/453685314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 0x55dde575a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.522+0000 7f2bf579b700 1 -- 172.21.15.133:0/453685314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde5751b10 msgr2=0x55dde5761240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.522+0000 7f2bf579b700 1 --2- 172.21.15.133:0/453685314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde5751b10 0x55dde5761240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.522+0000 7f2bf579b700 1 -- 172.21.15.133:0/453685314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf010baa0 con 0x55dde55ac6b0 2023-11-27T15:24:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.523+0000 7f2bf579b700 1 --2- 172.21.15.133:0/453685314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde55ac6b0 0x55dde5759cc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2bec096920 tx=0x7f2bec09a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b91a87ee164aa7 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.523+0000 7f2bd8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:24:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.523+0000 7f2bd8ff9700 1 -- 172.21.15.133:0/453685314 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec0b1070 con 0x55dde55ac6b0 2023-11-27T15:24:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.523+0000 7f2bd8ff9700 1 -- 172.21.15.133:0/453685314 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f2bec097ff0 con 0x55dde55ac6b0 2023-11-27T15:24:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.523+0000 7f2c0b239b80 1 -- 172.21.15.133:0/453685314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dde5743710 con 0x55dde55ac6b0 2023-11-27T15:24:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.523+0000 7f2bd8ff9700 1 -- 172.21.15.133:0/453685314 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bec096730 con 0x55dde55ac6b0 2023-11-27T15:24:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.523+0000 7f2c0b239b80 10 client.15738 fetch_fsmap learned FSMap version 23 2023-11-27T15:24:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.523+0000 7f2c0b239b80 1 -- 172.21.15.133:0/453685314 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55dde5668130 con 0x55dde55ac6b0 2023-11-27T15:24:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.523+0000 7f2bd8ff9700 1 -- 172.21.15.133:0/453685314 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dde5743710 con 0x55dde55ac6b0 2023-11-27T15:24:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.524+0000 7f2bd8ff9700 1 -- 172.21.15.133:0/453685314 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2bec0adbb0 con 0x55dde55ac6b0 2023-11-27T15:24:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.524+0000 7f2c0b239b80 10 client.15738 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:24:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.524+0000 7f2c0b239b80 10 client.15738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.524+0000 7f2c0b239b80 20 client.15738 populate_metadata read hostname 'smithi133' 2023-11-27T15:24:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.524+0000 7f2c0b239b80 1 --2- 172.21.15.133:0/453685314 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55dde57656e0 0x55dde5767af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.524+0000 7f2c0b239b80 4 client.15738 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.524+0000 7f2c0b239b80 1 -- 172.21.15.133:0/453685314 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dde5664040 con 0x55dde57656e0 2023-11-27T15:24:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.524+0000 7f2bf4f9a700 1 --2- 172.21.15.133:0/453685314 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55dde57656e0 0x55dde5767af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.525+0000 7f2bd8ff9700 10 client.15738 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.525+0000 7f2bf4f9a700 1 --2- 172.21.15.133:0/453685314 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55dde57656e0 0x55dde5767af0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=798162a854e48316 server_cookie=ce5bf23f338da75f in_seq=0 out_seq=0 2023-11-27T15:24:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.525+0000 7f2bd8ff9700 1 -- 172.21.15.133:0/453685314 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dde5664040 con 0x55dde57656e0 2023-11-27T15:24:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.526+0000 7f2bd8ff9700 10 client.15738 handle_command_reply: tid=0 2023-11-27T15:24:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.526+0000 7f2c0b239b80 1 client.15738 shutdown 2023-11-27T15:24:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.526+0000 7f2c0b239b80 1 -- 172.21.15.133:0/453685314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde55ac6b0 msgr2=0x55dde5759cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.526+0000 7f2c0b239b80 1 --2- 172.21.15.133:0/453685314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde55ac6b0 0x55dde5759cc0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2bec096920 tx=0x7f2bec09a340 comp rx=0 tx=0).stop 2023-11-27T15:24:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.526+0000 7f2c0b239b80 1 -- 172.21.15.133:0/453685314 shutdown_connections 2023-11-27T15:24:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.526+0000 7f2c0b239b80 1 --2- 172.21.15.133:0/453685314 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55dde57656e0 0x55dde5767af0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.526+0000 7f2c0b239b80 1 --2- 172.21.15.133:0/453685314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde5751b10 0x55dde5761240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.526+0000 7f2bd8ff9700 0 client.15738 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.526+0000 7f2c0b239b80 1 --2- 172.21.15.133:0/453685314 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde55ac6b0 0x55dde5759cc0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.526+0000 7f2c0b239b80 1 --2- 172.21.15.133:0/453685314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 0x55dde575a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.526+0000 7f2c0b239b80 1 -- 172.21.15.133:0/453685314 >> 172.21.15.133:0/453685314 conn(0x55dde5745f50 msgr2=0x55dde574a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.526+0000 7f2c0b239b80 1 -- 172.21.15.133:0/453685314 shutdown_connections 2023-11-27T15:24:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.526+0000 7f2c0b239b80 1 -- 172.21.15.133:0/453685314 wait complete. 2023-11-27T15:24:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.526+0000 7f2c0b239b80 20 client.15738 trim_cache size 0 max 16384 2023-11-27T15:24:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.541+0000 7f2c0b239b80 1 Processor -- start 2023-11-27T15:24:18.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.541+0000 7f2c0b239b80 1 -- start start 2023-11-27T15:24:18.546 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.542+0000 7f2c0b239b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde55ac6b0 0x55dde5751ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.542+0000 7f2c0b239b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 0x55dde5761200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.542+0000 7f2c0b239b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde5751b10 0x55dde57589d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.542+0000 7f2c0b239b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde574c930 con 0x55dde5751b10 2023-11-27T15:24:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.542+0000 7f2c0b239b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde574cb70 con 0x55dde55ac6b0 2023-11-27T15:24:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.542+0000 7f2c0b239b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde5757c40 con 0x55dde5750db0 2023-11-27T15:24:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.542+0000 7f2bf4f9a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 0x55dde5761200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.542+0000 7f2bf5f9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde5751b10 0x55dde57589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.542+0000 7f2bf4f9a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 0x55dde5761200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:52188/0 (socket says 172.21.15.133:52188) 2023-11-27T15:24:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.542+0000 7f2bf4f9a700 1 -- 172.21.15.133:0/701138143 learned_addr learned my addr 172.21.15.133:0/701138143 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.542+0000 7f2bf579b700 1 --2- 172.21.15.133:0/701138143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde55ac6b0 0x55dde5751ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.542+0000 7f2bf4f9a700 1 -- 172.21.15.133:0/701138143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde55ac6b0 msgr2=0x55dde5751ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.542+0000 7f2bf4f9a700 1 --2- 172.21.15.133:0/701138143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde55ac6b0 0x55dde5751ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.542+0000 7f2bf4f9a700 1 -- 172.21.15.133:0/701138143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde5751b10 msgr2=0x55dde57589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.542+0000 7f2bf4f9a700 1 --2- 172.21.15.133:0/701138143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde5751b10 0x55dde57589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.542+0000 7f2bf4f9a700 1 -- 172.21.15.133:0/701138143 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dde5667e80 con 0x55dde5750db0 2023-11-27T15:24:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.542+0000 7f2bf579b700 1 --2- 172.21.15.133:0/701138143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde55ac6b0 0x55dde5751ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.543+0000 7f2bf4f9a700 1 --2- 172.21.15.133:0/701138143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 0x55dde5761200 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55dde5768400 tx=0x7f2be0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39b782eb170ad76e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.543+0000 7f2bdbfff700 1 -- 172.21.15.133:0/701138143 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be0022030 con 0x55dde5750db0 2023-11-27T15:24:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.543+0000 7f2bdbfff700 1 -- 172.21.15.133:0/701138143 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2be0023040 con 0x55dde5750db0 2023-11-27T15:24:18.548 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.543+0000 7f2bdbfff700 1 -- 172.21.15.133:0/701138143 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be0029650 con 0x55dde5750db0 2023-11-27T15:24:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.544+0000 7f2c0b239b80 1 -- 172.21.15.133:0/701138143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 msgr2=0x55dde5761200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.544+0000 7f2c0b239b80 1 --2- 172.21.15.133:0/701138143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 0x55dde5761200 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55dde5768400 tx=0x7f2be0014000 comp rx=0 tx=0).stop 2023-11-27T15:24:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.544+0000 7f2c0b239b80 1 -- 172.21.15.133:0/701138143 shutdown_connections 2023-11-27T15:24:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.544+0000 7f2c0b239b80 1 --2- 172.21.15.133:0/701138143 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde5751b10 0x55dde57589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.544+0000 7f2c0b239b80 1 --2- 172.21.15.133:0/701138143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde55ac6b0 0x55dde5751ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.544+0000 7f2c0b239b80 1 --2- 172.21.15.133:0/701138143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 0x55dde5761200 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.544+0000 7f2c0b239b80 1 -- 172.21.15.133:0/701138143 >> 172.21.15.133:0/701138143 conn(0x55dde5745f50 msgr2=0x55dde576c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.544+0000 7f2c0b239b80 1 -- 172.21.15.133:0/701138143 shutdown_connections 2023-11-27T15:24:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.544+0000 7f2c0b239b80 1 -- 172.21.15.133:0/701138143 wait complete. 2023-11-27T15:24:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.544+0000 7f2c0b239b80 1 -- start start 2023-11-27T15:24:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.544+0000 7f2c0b239b80 1 Processor -- start 2023-11-27T15:24:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.545+0000 7f2c0b239b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde55ac6b0 0x55dde575cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.545+0000 7f2c0b239b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 0x55dde575d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.545+0000 7f2c0b239b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde5751b10 0x55dde5760310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.545+0000 7f2c0b239b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde5760850 con 0x55dde55ac6b0 2023-11-27T15:24:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.545+0000 7f2c0b239b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde57609c0 con 0x55dde5751b10 2023-11-27T15:24:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.545+0000 7f2c0b239b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dde5760b80 con 0x55dde5750db0 2023-11-27T15:24:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.545+0000 7f2c0b239b80 10 client.0 _ll_register_callbacks cb 0x7f2be0005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.545+0000 7f2bf579b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde55ac6b0 0x55dde575cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.545+0000 7f2bf579b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde55ac6b0 0x55dde575cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58206/0 (socket says 172.21.15.133:58206) 2023-11-27T15:24:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.545+0000 7f2bf579b700 1 -- 172.21.15.133:0/4055928876 learned_addr learned my addr 172.21.15.133:0/4055928876 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.545+0000 7f2bf4f9a700 1 --2- 172.21.15.133:0/4055928876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 0x55dde575d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.545+0000 7f2bf5f9c700 1 --2- 172.21.15.133:0/4055928876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde5751b10 0x55dde5760310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.545+0000 7f2bf579b700 1 -- 172.21.15.133:0/4055928876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 msgr2=0x55dde575d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.545+0000 7f2bf579b700 1 --2- 172.21.15.133:0/4055928876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 0x55dde575d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.545+0000 7f2bf579b700 1 -- 172.21.15.133:0/4055928876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde5751b10 msgr2=0x55dde5760310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.545+0000 7f2bf579b700 1 --2- 172.21.15.133:0/4055928876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde5751b10 0x55dde5760310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.545+0000 7f2bf579b700 1 -- 172.21.15.133:0/4055928876 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be0017070 con 0x55dde55ac6b0 2023-11-27T15:24:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.545+0000 7f2bf5f9c700 1 --2- 172.21.15.133:0/4055928876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde5751b10 0x55dde5760310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.546+0000 7f2bf579b700 1 --2- 172.21.15.133:0/4055928876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde55ac6b0 0x55dde575cdb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0b28e0 tx=0x7f2bec003710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89ac78070d82d60 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.546+0000 7f2bd8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:24:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.546+0000 7f2bd8ff9700 1 -- 172.21.15.133:0/4055928876 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec0b1020 con 0x55dde55ac6b0 2023-11-27T15:24:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.546+0000 7f2bd8ff9700 1 -- 172.21.15.133:0/4055928876 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f2bec095e40 con 0x55dde55ac6b0 2023-11-27T15:24:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.546+0000 7f2c0b239b80 1 -- 172.21.15.133:0/4055928876 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2bec0a2300 con 0x55dde55ac6b0 2023-11-27T15:24:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.546+0000 7f2bd8ff9700 1 -- 172.21.15.133:0/4055928876 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bec09a3b0 con 0x55dde55ac6b0 2023-11-27T15:24:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.546+0000 7f2c0b239b80 10 client.15744 fetch_fsmap learned FSMap version 23 2023-11-27T15:24:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.546+0000 7f2c0b239b80 1 -- 172.21.15.133:0/4055928876 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55dde54b8c00 con 0x55dde55ac6b0 2023-11-27T15:24:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.546+0000 7f2bd8ff9700 1 -- 172.21.15.133:0/4055928876 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2bec07c080 con 0x55dde55ac6b0 2023-11-27T15:24:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.547+0000 7f2bd8ff9700 1 -- 172.21.15.133:0/4055928876 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2bec0ca2b0 con 0x55dde55ac6b0 2023-11-27T15:24:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.547+0000 7f2c0b239b80 10 client.15744 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:24:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.547+0000 7f2c0b239b80 10 client.15744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.547+0000 7f2c0b239b80 20 client.15744 populate_metadata read hostname 'smithi133' 2023-11-27T15:24:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.547+0000 7f2c0b239b80 1 --2- 172.21.15.133:0/4055928876 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55dde57b9f20 0x55dde57bc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.547+0000 7f2c0b239b80 4 client.15744 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:24:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.547+0000 7f2c0b239b80 1 -- 172.21.15.133:0/4055928876 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dde5670600 con 0x55dde57b9f20 2023-11-27T15:24:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.547+0000 7f2bf4f9a700 1 --2- 172.21.15.133:0/4055928876 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55dde57b9f20 0x55dde57bc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.547+0000 7f2bf4f9a700 1 --2- 172.21.15.133:0/4055928876 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55dde57b9f20 0x55dde57bc300 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b067c40017704a6 server_cookie=7aff66993f55fb48 in_seq=0 out_seq=0 2023-11-27T15:24:18.555 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.547+0000 7f2bd8ff9700 10 client.15744 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.569+0000 7f2bd8ff9700 1 -- 172.21.15.133:0/4055928876 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dde5670600 con 0x55dde57b9f20 2023-11-27T15:24:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.569+0000 7f2bd8ff9700 10 client.15744 handle_command_reply: tid=0 2023-11-27T15:24:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.569+0000 7f2c0b239b80 1 client.15744 shutdown 2023-11-27T15:24:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.569+0000 7f2c0b239b80 1 -- 172.21.15.133:0/4055928876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde55ac6b0 msgr2=0x55dde575cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.569+0000 7f2c0b239b80 1 --2- 172.21.15.133:0/4055928876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde55ac6b0 0x55dde575cdb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2bec0b28e0 tx=0x7f2bec003710 comp rx=0 tx=0).stop 2023-11-27T15:24:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.569+0000 7f2c0b239b80 1 -- 172.21.15.133:0/4055928876 shutdown_connections 2023-11-27T15:24:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.569+0000 7f2c0b239b80 1 --2- 172.21.15.133:0/4055928876 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55dde57b9f20 0x55dde57bc300 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.569+0000 7f2c0b239b80 1 --2- 172.21.15.133:0/4055928876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dde5751b10 0x55dde5760310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.569+0000 7f2c0b239b80 1 --2- 172.21.15.133:0/4055928876 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55dde55ac6b0 0x55dde575cdb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.569+0000 7f2bd8ff9700 0 client.15744 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.569+0000 7f2c0b239b80 1 --2- 172.21.15.133:0/4055928876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55dde5750db0 0x55dde575d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.569+0000 7f2c0b239b80 1 -- 172.21.15.133:0/4055928876 >> 172.21.15.133:0/4055928876 conn(0x55dde5745f50 msgr2=0x55dde57589d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.569+0000 7f2c0b239b80 1 -- 172.21.15.133:0/4055928876 shutdown_connections 2023-11-27T15:24:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.569+0000 7f2c0b239b80 1 -- 172.21.15.133:0/4055928876 wait complete. 2023-11-27T15:24:18.574 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.570+0000 7f2c0b239b80 20 client.15744 trim_cache size 0 max 16384 2023-11-27T15:24:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:24:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:24:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:24:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:24:18.575 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.570+0000 7f2bf79ff700 1 -- 172.21.15.133:0/3083401779 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2bdc0a1150 msgr2=0x7f2bdc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.570+0000 7f2bf79ff700 1 --2- 172.21.15.133:0/3083401779 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2bdc0a1150 0x7f2bdc0a3600 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2be0009c60 tx=0x7f2be0009260 comp rx=0 tx=0).stop 2023-11-27T15:24:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.570+0000 7f2bf79ff700 1 -- 172.21.15.133:0/3083401779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf01002c0 msgr2=0x7f2bf00748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.570+0000 7f2bf79ff700 1 --2- 172.21.15.133:0/3083401779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf01002c0 0x7f2bf00748a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00e970 tx=0x7f2bec00ec80 comp rx=0 tx=0).stop 2023-11-27T15:24:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.570+0000 7f2bf79ff700 1 -- 172.21.15.133:0/3083401779 shutdown_connections 2023-11-27T15:24:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.570+0000 7f2bf79ff700 1 --2- 172.21.15.133:0/3083401779 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2bdc0a1150 0x7f2bdc0a3600 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.570+0000 7f2bf79ff700 1 --2- 172.21.15.133:0/3083401779 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2bf0101e40 0x7f2bf0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.570+0000 7f2bf79ff700 1 --2- 172.21.15.133:0/3083401779 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bf01002c0 0x7f2bf00748a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.570+0000 7f2bf79ff700 1 --2- 172.21.15.133:0/3083401779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bf01014b0 0x7f2bf0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.570+0000 7f2bf79ff700 1 -- 172.21.15.133:0/3083401779 >> 172.21.15.133:0/3083401779 conn(0x7f2bf00fb8e0 msgr2=0x7f2bf0104920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.570+0000 7f2bf79ff700 1 -- 172.21.15.133:0/3083401779 shutdown_connections 2023-11-27T15:24:18.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:18.570+0000 7f2bf79ff700 1 -- 172.21.15.133:0/3083401779 wait complete. 2023-11-27T15:24:18.846 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:18.847 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:18.932 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:18.932 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:18.951 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:18.952 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:18.952 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 44 -lt 100 ']' 2023-11-27T15:24:18.952 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 44 + 1 2023-11-27T15:24:19.005 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=45 2023-11-27T15:24:19.006 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:19.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:18 smithi133 ceph-mon[97860]: pgmap v390: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 10 op/s 2023-11-27T15:24:19.028 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:19.028 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:19.031 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:19.031 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:19.032 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:19.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:18 smithi177 ceph-mon[131504]: pgmap v390: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 10 op/s 2023-11-27T15:24:19.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:18 smithi174 ceph-mon[131110]: pgmap v390: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 15 KiB/s wr, 10 op/s 2023-11-27T15:24:19.721 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:19.722 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:19.883 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:19.883 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:19.912 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:19.912 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:19.912 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 45 -lt 100 ']' 2023-11-27T15:24:19.913 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 45 + 1 2023-11-27T15:24:19.963 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=46 2023-11-27T15:24:19.963 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:20.070 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:20.070 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:20.072 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:20.073 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:20.073 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:21.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:20 smithi177 ceph-mon[131504]: pgmap v391: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2023-11-27T15:24:21.105 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:21.105 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:21.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:20 smithi174 ceph-mon[131110]: pgmap v391: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2023-11-27T15:24:21.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:20 smithi133 ceph-mon[97860]: pgmap v391: 129 pgs: 129 active+clean; 1.1 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 7.2 KiB/s wr, 9 op/s 2023-11-27T15:24:21.332 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:21.333 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:21.355 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:21.355 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:21.355 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 46 -lt 100 ']' 2023-11-27T15:24:21.356 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 46 + 1 2023-11-27T15:24:21.388 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=47 2023-11-27T15:24:21.388 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:21.511 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:21.511 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:21.514 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:21.515 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:21.515 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:21.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.801+0000 7fc20e1ff700 1 -- 172.21.15.133:0/3453942969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2081002c0 msgr2=0x7fc2081006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:21.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.801+0000 7fc20e1ff700 1 --2- 172.21.15.133:0/3453942969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2081002c0 0x7fc2081006d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009990 tx=0x7fc1f8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.805 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.801+0000 7fc20e1ff700 1 -- 172.21.15.133:0/3453942969 shutdown_connections 2023-11-27T15:24:21.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.801+0000 7fc20e1ff700 1 --2- 172.21.15.133:0/3453942969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc208101e40 0x7fc208108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.801+0000 7fc20e1ff700 1 --2- 172.21.15.133:0/3453942969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2081002c0 0x7fc2081006d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.801+0000 7fc20e1ff700 1 --2- 172.21.15.133:0/3453942969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2081014b0 0x7fc208101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.801+0000 7fc20e1ff700 1 -- 172.21.15.133:0/3453942969 >> 172.21.15.133:0/3453942969 conn(0x7fc2080fb8a0 msgr2=0x7fc2080fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:21.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.801+0000 7fc20e1ff700 1 -- 172.21.15.133:0/3453942969 shutdown_connections 2023-11-27T15:24:21.806 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.801+0000 7fc20e1ff700 1 -- 172.21.15.133:0/3453942969 wait complete. 2023-11-27T15:24:21.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.803+0000 7fc20e1ff700 1 Processor -- start 2023-11-27T15:24:21.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.803+0000 7fc20e1ff700 1 -- start start 2023-11-27T15:24:21.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.803+0000 7fc20e1ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2081002c0 0x7fc2080748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:21.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.803+0000 7fc20e1ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2081014b0 0x7fc208072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:21.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.804+0000 7fc20e1ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc208101e40 0x7fc208073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:21.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.804+0000 7fc20e1ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc208073bc0 con 0x7fc2081002c0 2023-11-27T15:24:21.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.804+0000 7fc20e1ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc208073d30 con 0x7fc2081014b0 2023-11-27T15:24:21.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.804+0000 7fc20e1ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2081a0100 con 0x7fc208101e40 2023-11-27T15:24:21.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.804+0000 7fc2077fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2081002c0 0x7fc2080748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:21.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.804+0000 7fc2077fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2081002c0 0x7fc2080748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58216/0 (socket says 172.21.15.133:58216) 2023-11-27T15:24:21.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.804+0000 7fc2077fe700 1 -- 172.21.15.133:0/1337573337 learned_addr learned my addr 172.21.15.133:0/1337573337 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:21.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.804+0000 7fc207fff700 1 --2- 172.21.15.133:0/1337573337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc208101e40 0x7fc208073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:21.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.804+0000 7fc206ffd700 1 --2- 172.21.15.133:0/1337573337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2081014b0 0x7fc208072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:21.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.804+0000 7fc2077fe700 1 -- 172.21.15.133:0/1337573337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc208101e40 msgr2=0x7fc208073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:21.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.804+0000 7fc2077fe700 1 --2- 172.21.15.133:0/1337573337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc208101e40 0x7fc208073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.804+0000 7fc2077fe700 1 -- 172.21.15.133:0/1337573337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2081014b0 msgr2=0x7fc208072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:21.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.804+0000 7fc2077fe700 1 --2- 172.21.15.133:0/1337573337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2081014b0 0x7fc208072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.804+0000 7fc2077fe700 1 -- 172.21.15.133:0/1337573337 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f80095f0 con 0x7fc2081002c0 2023-11-27T15:24:21.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.805+0000 7fc2077fe700 1 --2- 172.21.15.133:0/1337573337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2081002c0 0x7fc2080748a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8004730 tx=0x7fc1f8004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:21.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.805+0000 7fc204ff9700 1 -- 172.21.15.133:0/1337573337 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f801d070 con 0x7fc2081002c0 2023-11-27T15:24:21.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.805+0000 7fc204ff9700 1 -- 172.21.15.133:0/1337573337 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1f8021410 con 0x7fc2081002c0 2023-11-27T15:24:21.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.805+0000 7fc204ff9700 1 -- 172.21.15.133:0/1337573337 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8017650 con 0x7fc2081002c0 2023-11-27T15:24:21.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.805+0000 7fc20e1ff700 1 -- 172.21.15.133:0/1337573337 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20810bc90 con 0x7fc2081002c0 2023-11-27T15:24:21.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.805+0000 7fc20e1ff700 1 -- 172.21.15.133:0/1337573337 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20810bb00 con 0x7fc2081002c0 2023-11-27T15:24:21.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.808+0000 7fc204ff9700 1 -- 172.21.15.133:0/1337573337 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fc1f8021580 con 0x7fc2081002c0 2023-11-27T15:24:21.815 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.810+0000 7fc204ff9700 1 --2- 172.21.15.133:0/1337573337 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc1f00a11d0 0x7fc1f00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:21.815 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.810+0000 7fc204ff9700 1 -- 172.21.15.133:0/1337573337 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc1f80cdfe0 con 0x7fc2081002c0 2023-11-27T15:24:21.815 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.810+0000 7fc206ffd700 1 --2- 172.21.15.133:0/1337573337 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc1f00a11d0 0x7fc1f00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:21.815 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.811+0000 7fc206ffd700 1 --2- 172.21.15.133:0/1337573337 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc1f00a11d0 0x7fc1f00a3680 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009c60 tx=0x7fc1fc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:21.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.884+0000 7fc221c0ab80 1 Processor -- start 2023-11-27T15:24:21.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.884+0000 7fc221c0ab80 1 -- start start 2023-11-27T15:24:21.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.884+0000 7fc221c0ab80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50cc80 0x5618eb50d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:21.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.884+0000 7fc221c0ab80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50d5d0 0x5618eb50dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:21.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.884+0000 7fc221c0ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50e190 0x5618eb50e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:21.888 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.884+0000 7fc221c0ab80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618eb4120e0 con 0x5618eb50cc80 2023-11-27T15:24:21.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.884+0000 7fc221c0ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618eb464920 con 0x5618eb50e190 2023-11-27T15:24:21.889 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.884+0000 7fc221c0ab80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618eb45ddb0 con 0x5618eb50d5d0 2023-11-27T15:24:21.890 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.885+0000 7fc2077fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50cc80 0x5618eb50d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:21.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.885+0000 7fc2077fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50cc80 0x5618eb50d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58222/0 (socket says 172.21.15.133:58222) 2023-11-27T15:24:21.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.885+0000 7fc2077fe700 1 -- 172.21.15.133:0/2008103490 learned_addr learned my addr 172.21.15.133:0/2008103490 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:21.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.885+0000 7fc206ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50d5d0 0x5618eb50dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:21.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.885+0000 7fc207fff700 1 --2- 172.21.15.133:0/2008103490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50e190 0x5618eb50e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:21.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.885+0000 7fc2077fe700 1 -- 172.21.15.133:0/2008103490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50d5d0 msgr2=0x5618eb50dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:21.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.885+0000 7fc2077fe700 1 --2- 172.21.15.133:0/2008103490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50d5d0 0x5618eb50dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.885+0000 7fc2077fe700 1 -- 172.21.15.133:0/2008103490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50e190 msgr2=0x5618eb50e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:21.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.885+0000 7fc2077fe700 1 --2- 172.21.15.133:0/2008103490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50e190 0x5618eb50e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.885+0000 7fc2077fe700 1 -- 172.21.15.133:0/2008103490 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618eb2df6b0 con 0x5618eb50cc80 2023-11-27T15:24:21.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.885+0000 7fc206ffd700 1 --2- 172.21.15.133:0/2008103490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50d5d0 0x5618eb50dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:21.891 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.886+0000 7fc2077fe700 1 --2- 172.21.15.133:0/2008103490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50cc80 0x5618eb50d090 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc1f809ae10 tx=0x7fc1f809acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660222f89484f636 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:21.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.886+0000 7fc1edffb700 1 -- 172.21.15.133:0/2008103490 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f809f140 con 0x5618eb50cc80 2023-11-27T15:24:21.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.886+0000 7fc1edffb700 1 -- 172.21.15.133:0/2008103490 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1f80a34d0 con 0x5618eb50cc80 2023-11-27T15:24:21.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.886+0000 7fc1edffb700 1 -- 172.21.15.133:0/2008103490 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f809a820 con 0x5618eb50cc80 2023-11-27T15:24:21.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.887+0000 7fc221c0ab80 1 -- 172.21.15.133:0/2008103490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50cc80 msgr2=0x5618eb50d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:21.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.887+0000 7fc221c0ab80 1 --2- 172.21.15.133:0/2008103490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50cc80 0x5618eb50d090 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc1f809ae10 tx=0x7fc1f809acd0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.887+0000 7fc221c0ab80 1 -- 172.21.15.133:0/2008103490 shutdown_connections 2023-11-27T15:24:21.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.887+0000 7fc221c0ab80 1 --2- 172.21.15.133:0/2008103490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50e190 0x5618eb50e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.887+0000 7fc221c0ab80 1 --2- 172.21.15.133:0/2008103490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50cc80 0x5618eb50d090 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.887+0000 7fc221c0ab80 1 --2- 172.21.15.133:0/2008103490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50d5d0 0x5618eb50dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.887+0000 7fc221c0ab80 1 -- 172.21.15.133:0/2008103490 >> 172.21.15.133:0/2008103490 conn(0x5618eb5023e0 msgr2=0x5618eb5076f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:21.892 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.887+0000 7fc221c0ab80 1 -- 172.21.15.133:0/2008103490 shutdown_connections 2023-11-27T15:24:21.893 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.887+0000 7fc221c0ab80 1 -- 172.21.15.133:0/2008103490 wait complete. 2023-11-27T15:24:21.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.888+0000 7fc221c0ab80 1 -- start start 2023-11-27T15:24:21.895 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.888+0000 7fc221c0ab80 1 Processor -- start 2023-11-27T15:24:21.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.888+0000 7fc221c0ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50cc80 0x5618eb516310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:21.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.888+0000 7fc221c0ab80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50d5d0 0x5618eb516850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:21.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.888+0000 7fc221c0ab80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50e190 0x5618eb51d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:21.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.888+0000 7fc221c0ab80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618eb3ff960 con 0x5618eb50e190 2023-11-27T15:24:21.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.888+0000 7fc221c0ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618eb3ff600 con 0x5618eb50cc80 2023-11-27T15:24:21.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.888+0000 7fc221c0ab80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618eb2e19a0 con 0x5618eb50d5d0 2023-11-27T15:24:21.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.888+0000 7fc221c0ab80 10 client.0 _ll_register_callbacks cb 0x5618eb41d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:21.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.889+0000 7fc206ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50d5d0 0x5618eb516850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:21.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.889+0000 7fc206ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50d5d0 0x5618eb516850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:52238/0 (socket says 172.21.15.133:52238) 2023-11-27T15:24:21.896 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.889+0000 7fc206ffd700 1 -- 172.21.15.133:0/2734140603 learned_addr learned my addr 172.21.15.133:0/2734140603 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:21.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.889+0000 7fc207fff700 1 --2- 172.21.15.133:0/2734140603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50e190 0x5618eb51d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:21.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.889+0000 7fc2077fe700 1 --2- 172.21.15.133:0/2734140603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50cc80 0x5618eb516310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:21.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.889+0000 7fc2077fe700 1 -- 172.21.15.133:0/2734140603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50d5d0 msgr2=0x5618eb516850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:21.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.889+0000 7fc2077fe700 1 --2- 172.21.15.133:0/2734140603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50d5d0 0x5618eb516850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.889+0000 7fc2077fe700 1 -- 172.21.15.133:0/2734140603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50e190 msgr2=0x5618eb51d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:21.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.889+0000 7fc2077fe700 1 --2- 172.21.15.133:0/2734140603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50e190 0x5618eb51d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.889+0000 7fc2077fe700 1 -- 172.21.15.133:0/2734140603 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20810bb00 con 0x5618eb50cc80 2023-11-27T15:24:21.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.889+0000 7fc206ffd700 1 --2- 172.21.15.133:0/2734140603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50d5d0 0x5618eb516850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:21.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.890+0000 7fc2077fe700 1 --2- 172.21.15.133:0/2734140603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50cc80 0x5618eb516310 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8097b80 tx=0x7fc1f809b290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b207f51188cf68b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:21.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.890+0000 7fc1e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:24:21.897 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.890+0000 7fc1e6ffd700 1 -- 172.21.15.133:0/2734140603 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f809b3f0 con 0x5618eb50cc80 2023-11-27T15:24:21.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.890+0000 7fc1e6ffd700 1 -- 172.21.15.133:0/2734140603 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc1f809f140 con 0x5618eb50cc80 2023-11-27T15:24:21.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.890+0000 7fc221c0ab80 1 -- 172.21.15.133:0/2734140603 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618eb4ffba0 con 0x5618eb50cc80 2023-11-27T15:24:21.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.890+0000 7fc1e6ffd700 1 -- 172.21.15.133:0/2734140603 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1f80b92c0 con 0x5618eb50cc80 2023-11-27T15:24:21.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.890+0000 7fc221c0ab80 10 client.25796 fetch_fsmap learned FSMap version 23 2023-11-27T15:24:21.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.890+0000 7fc221c0ab80 1 -- 172.21.15.133:0/2734140603 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5618eb424210 con 0x5618eb50cc80 2023-11-27T15:24:21.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.890+0000 7fc1e6ffd700 1 -- 172.21.15.133:0/2734140603 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5618eb4ffba0 con 0x5618eb50cc80 2023-11-27T15:24:21.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.891+0000 7fc1e6ffd700 1 -- 172.21.15.133:0/2734140603 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc1f8097820 con 0x5618eb50cc80 2023-11-27T15:24:21.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.891+0000 7fc221c0ab80 10 client.25796 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:24:21.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.891+0000 7fc221c0ab80 10 client.25796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:21.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.891+0000 7fc221c0ab80 20 client.25796 populate_metadata read hostname 'smithi133' 2023-11-27T15:24:21.898 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.891+0000 7fc221c0ab80 1 --2- 172.21.15.133:0/2734140603 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5618eb521bc0 0x5618eb523fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:21.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.891+0000 7fc221c0ab80 4 client.25796 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:21.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.891+0000 7fc221c0ab80 1 -- 172.21.15.133:0/2734140603 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618eb420120 con 0x5618eb521bc0 2023-11-27T15:24:21.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.891+0000 7fc206ffd700 1 --2- 172.21.15.133:0/2734140603 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5618eb521bc0 0x5618eb523fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:21.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.892+0000 7fc206ffd700 1 --2- 172.21.15.133:0/2734140603 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5618eb521bc0 0x5618eb523fa0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a84fa01b660ae07 server_cookie=1d6f13d5d9429db4 in_seq=0 out_seq=0 2023-11-27T15:24:21.899 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.892+0000 7fc1e6ffd700 10 client.25796 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:21.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.892+0000 7fc1e6ffd700 1 -- 172.21.15.133:0/2734140603 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5618eb420120 con 0x5618eb521bc0 2023-11-27T15:24:21.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.892+0000 7fc1e6ffd700 10 client.25796 handle_command_reply: tid=0 2023-11-27T15:24:21.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.892+0000 7fc221c0ab80 1 client.25796 shutdown 2023-11-27T15:24:21.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.893+0000 7fc221c0ab80 1 -- 172.21.15.133:0/2734140603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50cc80 msgr2=0x5618eb516310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:21.900 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.893+0000 7fc221c0ab80 1 --2- 172.21.15.133:0/2734140603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50cc80 0x5618eb516310 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8097b80 tx=0x7fc1f809b290 comp rx=0 tx=0).stop 2023-11-27T15:24:21.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.893+0000 7fc221c0ab80 1 -- 172.21.15.133:0/2734140603 shutdown_connections 2023-11-27T15:24:21.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.893+0000 7fc221c0ab80 1 --2- 172.21.15.133:0/2734140603 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5618eb521bc0 0x5618eb523fa0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.893+0000 7fc221c0ab80 1 --2- 172.21.15.133:0/2734140603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50e190 0x5618eb51d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.893+0000 7fc221c0ab80 1 --2- 172.21.15.133:0/2734140603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50cc80 0x5618eb516310 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.893+0000 7fc1e6ffd700 0 client.25796 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:21.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.893+0000 7fc221c0ab80 1 --2- 172.21.15.133:0/2734140603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50d5d0 0x5618eb516850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.893+0000 7fc221c0ab80 1 -- 172.21.15.133:0/2734140603 >> 172.21.15.133:0/2734140603 conn(0x5618eb5023e0 msgr2=0x5618eb506e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:21.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.893+0000 7fc221c0ab80 1 -- 172.21.15.133:0/2734140603 shutdown_connections 2023-11-27T15:24:21.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.893+0000 7fc221c0ab80 1 -- 172.21.15.133:0/2734140603 wait complete. 2023-11-27T15:24:21.901 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.893+0000 7fc221c0ab80 20 client.25796 trim_cache size 0 max 16384 2023-11-27T15:24:21.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.913+0000 7fc221c0ab80 1 Processor -- start 2023-11-27T15:24:21.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.913+0000 7fc221c0ab80 1 -- start start 2023-11-27T15:24:21.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.913+0000 7fc221c0ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50cc80 0x5618eb511040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:21.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.913+0000 7fc221c0ab80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50d5d0 0x5618eb51d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:21.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.913+0000 7fc221c0ab80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50e190 0x5618eb511580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:21.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.913+0000 7fc221c0ab80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618eb50e050 con 0x5618eb50d5d0 2023-11-27T15:24:21.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.913+0000 7fc221c0ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618eb5091f0 con 0x5618eb50cc80 2023-11-27T15:24:21.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.913+0000 7fc221c0ab80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618eb50d9a0 con 0x5618eb50e190 2023-11-27T15:24:21.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.914+0000 7fc206ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50d5d0 0x5618eb51d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:21.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.914+0000 7fc206ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50d5d0 0x5618eb51d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58234/0 (socket says 172.21.15.133:58234) 2023-11-27T15:24:21.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.914+0000 7fc206ffd700 1 -- 172.21.15.133:0/1503848895 learned_addr learned my addr 172.21.15.133:0/1503848895 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:21.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.914+0000 7fc2077fe700 1 --2- 172.21.15.133:0/1503848895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50cc80 0x5618eb511040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:21.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.914+0000 7fc207fff700 1 --2- 172.21.15.133:0/1503848895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50e190 0x5618eb511580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:21.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.914+0000 7fc206ffd700 1 -- 172.21.15.133:0/1503848895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50e190 msgr2=0x5618eb511580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:21.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.914+0000 7fc206ffd700 1 --2- 172.21.15.133:0/1503848895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50e190 0x5618eb511580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.914+0000 7fc206ffd700 1 -- 172.21.15.133:0/1503848895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50cc80 msgr2=0x5618eb511040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:21.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.914+0000 7fc206ffd700 1 --2- 172.21.15.133:0/1503848895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50cc80 0x5618eb511040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.914+0000 7fc206ffd700 1 -- 172.21.15.133:0/1503848895 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618eb515370 con 0x5618eb50d5d0 2023-11-27T15:24:21.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.914+0000 7fc2077fe700 1 --2- 172.21.15.133:0/1503848895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50cc80 0x5618eb511040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:21.919 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.914+0000 7fc206ffd700 1 --2- 172.21.15.133:0/1503848895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50d5d0 0x5618eb51d850 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc011ab0 tx=0x7fc1fc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b4aabfef9bff94 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:21.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.915+0000 7fc1ee7fc700 1 -- 172.21.15.133:0/1503848895 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc022070 con 0x5618eb50d5d0 2023-11-27T15:24:21.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.915+0000 7fc1ee7fc700 1 -- 172.21.15.133:0/1503848895 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1fc023040 con 0x5618eb50d5d0 2023-11-27T15:24:21.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.915+0000 7fc1ee7fc700 1 -- 172.21.15.133:0/1503848895 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc003970 con 0x5618eb50d5d0 2023-11-27T15:24:21.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.915+0000 7fc221c0ab80 1 -- 172.21.15.133:0/1503848895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50d5d0 msgr2=0x5618eb51d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:21.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.915+0000 7fc221c0ab80 1 --2- 172.21.15.133:0/1503848895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50d5d0 0x5618eb51d850 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc011ab0 tx=0x7fc1fc014000 comp rx=0 tx=0).stop 2023-11-27T15:24:21.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.915+0000 7fc221c0ab80 1 -- 172.21.15.133:0/1503848895 shutdown_connections 2023-11-27T15:24:21.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.915+0000 7fc221c0ab80 1 --2- 172.21.15.133:0/1503848895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50e190 0x5618eb511580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.915+0000 7fc221c0ab80 1 --2- 172.21.15.133:0/1503848895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50cc80 0x5618eb511040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.915+0000 7fc221c0ab80 1 --2- 172.21.15.133:0/1503848895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50d5d0 0x5618eb51d850 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.915+0000 7fc221c0ab80 1 -- 172.21.15.133:0/1503848895 >> 172.21.15.133:0/1503848895 conn(0x5618eb5023e0 msgr2=0x5618eb528900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:21.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.915+0000 7fc221c0ab80 1 -- 172.21.15.133:0/1503848895 shutdown_connections 2023-11-27T15:24:21.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.916+0000 7fc221c0ab80 1 -- 172.21.15.133:0/1503848895 wait complete. 2023-11-27T15:24:21.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.916+0000 7fc221c0ab80 1 -- start start 2023-11-27T15:24:21.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.916+0000 7fc221c0ab80 1 Processor -- start 2023-11-27T15:24:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.917+0000 7fc221c0ab80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50cc80 0x5618eb51d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.917+0000 7fc221c0ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50d5d0 0x5618eb519ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.917+0000 7fc221c0ab80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50e190 0x5618eb51a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.917+0000 7fc221c0ab80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618eb51dc50 con 0x5618eb50cc80 2023-11-27T15:24:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.917+0000 7fc221c0ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618eb51a980 con 0x5618eb50d5d0 2023-11-27T15:24:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.917+0000 7fc221c0ab80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618eb51ab40 con 0x5618eb50e190 2023-11-27T15:24:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.917+0000 7fc221c0ab80 10 client.0 _ll_register_callbacks cb 0x7fc200009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.917+0000 7fc2077fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50cc80 0x5618eb51d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.917+0000 7fc206ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50d5d0 0x5618eb519ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.917+0000 7fc206ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50d5d0 0x5618eb519ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:56758/0 (socket says 172.21.15.133:56758) 2023-11-27T15:24:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.917+0000 7fc206ffd700 1 -- 172.21.15.133:0/1180863548 learned_addr learned my addr 172.21.15.133:0/1180863548 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:21.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.917+0000 7fc2077fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50cc80 0x5618eb51d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:58250/0 (socket says 172.21.15.133:58250) 2023-11-27T15:24:21.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.918+0000 7fc207fff700 1 --2- 172.21.15.133:0/1180863548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50e190 0x5618eb51a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:21.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.918+0000 7fc206ffd700 1 -- 172.21.15.133:0/1180863548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50e190 msgr2=0x5618eb51a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:21.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.918+0000 7fc206ffd700 1 --2- 172.21.15.133:0/1180863548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50e190 0x5618eb51a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.918+0000 7fc206ffd700 1 -- 172.21.15.133:0/1180863548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50cc80 msgr2=0x5618eb51d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:21.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.918+0000 7fc206ffd700 1 --2- 172.21.15.133:0/1180863548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50cc80 0x5618eb51d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.918+0000 7fc206ffd700 1 -- 172.21.15.133:0/1180863548 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618eb424210 con 0x5618eb50d5d0 2023-11-27T15:24:21.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.918+0000 7fc2077fe700 1 --2- 172.21.15.133:0/1180863548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50cc80 0x5618eb51d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:21.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.918+0000 7fc206ffd700 1 --2- 172.21.15.133:0/1180863548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50d5d0 0x5618eb519ed0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc005920 tx=0x7fc1fc02bb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8174704a014e078 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:21.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.918+0000 7fc1e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:24:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.918+0000 7fc1e6ffd700 1 -- 172.21.15.133:0/1180863548 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc022030 con 0x5618eb50d5d0 2023-11-27T15:24:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.919+0000 7fc221c0ab80 1 -- 172.21.15.133:0/1180863548 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1f80a34d0 con 0x5618eb50d5d0 2023-11-27T15:24:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.919+0000 7fc1e6ffd700 1 -- 172.21.15.133:0/1180863548 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc1fc02be30 con 0x5618eb50d5d0 2023-11-27T15:24:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.919+0000 7fc1e6ffd700 1 -- 172.21.15.133:0/1180863548 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1fc02b310 con 0x5618eb50d5d0 2023-11-27T15:24:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.919+0000 7fc221c0ab80 10 client.25802 fetch_fsmap learned FSMap version 23 2023-11-27T15:24:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.919+0000 7fc221c0ab80 1 -- 172.21.15.133:0/1180863548 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fc1fc01daf0 con 0x5618eb50d5d0 2023-11-27T15:24:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.919+0000 7fc1e6ffd700 1 -- 172.21.15.133:0/1180863548 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc1f80a34d0 con 0x5618eb50d5d0 2023-11-27T15:24:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.919+0000 7fc1e6ffd700 1 -- 172.21.15.133:0/1180863548 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc1fc031070 con 0x5618eb50d5d0 2023-11-27T15:24:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.919+0000 7fc221c0ab80 10 client.25802 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:24:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.919+0000 7fc221c0ab80 10 client.25802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:21.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.919+0000 7fc221c0ab80 20 client.25802 populate_metadata read hostname 'smithi133' 2023-11-27T15:24:21.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.920+0000 7fc221c0ab80 1 --2- 172.21.15.133:0/1180863548 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5618eb577330 0x5618eb579740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:21.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.920+0000 7fc221c0ab80 4 client.25802 mds_command: new command op to 24560 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:24:21.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.920+0000 7fc221c0ab80 1 -- 172.21.15.133:0/1180863548 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5618eb42c6e0 con 0x5618eb577330 2023-11-27T15:24:21.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.920+0000 7fc2077fe700 1 --2- 172.21.15.133:0/1180863548 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5618eb577330 0x5618eb579740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:21.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.920+0000 7fc2077fe700 1 --2- 172.21.15.133:0/1180863548 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5618eb577330 0x5618eb579740 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2e9841f36201ae9 server_cookie=890efa3cc863a31e in_seq=0 out_seq=0 2023-11-27T15:24:21.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.920+0000 7fc1e6ffd700 10 client.25802 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:21.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.921+0000 7fc1e6ffd700 1 -- 172.21.15.133:0/1180863548 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5618eb42c6e0 con 0x5618eb577330 2023-11-27T15:24:21.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.921+0000 7fc1e6ffd700 10 client.25802 handle_command_reply: tid=0 2023-11-27T15:24:21.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.921+0000 7fc221c0ab80 1 client.25802 shutdown 2023-11-27T15:24:21.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.922+0000 7fc221c0ab80 1 -- 172.21.15.133:0/1180863548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50d5d0 msgr2=0x5618eb519ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:21.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.922+0000 7fc221c0ab80 1 --2- 172.21.15.133:0/1180863548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50d5d0 0x5618eb519ed0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc005920 tx=0x7fc1fc02bb40 comp rx=0 tx=0).stop 2023-11-27T15:24:21.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.922+0000 7fc221c0ab80 1 -- 172.21.15.133:0/1180863548 shutdown_connections 2023-11-27T15:24:21.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.922+0000 7fc221c0ab80 1 --2- 172.21.15.133:0/1180863548 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5618eb577330 0x5618eb579740 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.922+0000 7fc221c0ab80 1 --2- 172.21.15.133:0/1180863548 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5618eb50e190 0x5618eb51a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.922+0000 7fc221c0ab80 1 --2- 172.21.15.133:0/1180863548 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5618eb50cc80 0x5618eb51d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.922+0000 7fc221c0ab80 1 --2- 172.21.15.133:0/1180863548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5618eb50d5d0 0x5618eb519ed0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.922+0000 7fc221c0ab80 1 -- 172.21.15.133:0/1180863548 >> 172.21.15.133:0/1180863548 conn(0x5618eb5023e0 msgr2=0x5618eb511040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:21.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.922+0000 7fc1e6ffd700 0 client.25802 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:21.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.922+0000 7fc221c0ab80 1 -- 172.21.15.133:0/1180863548 shutdown_connections 2023-11-27T15:24:21.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.922+0000 7fc221c0ab80 1 -- 172.21.15.133:0/1180863548 wait complete. 2023-11-27T15:24:21.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.922+0000 7fc221c0ab80 20 client.25802 trim_cache size 0 max 16384 2023-11-27T15:24:21.929 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:24:21.930 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2023-11-27T15:24:21.930 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "e840364b-dfce-4ae4-9579-41368a7f5d44", 2023-11-27T15:24:21.930 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2023-11-27T15:24:21.930 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:24:21.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.923+0000 7fc20e1ff700 1 -- 172.21.15.133:0/1337573337 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc1f00a11d0 msgr2=0x7fc1f00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:21.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.923+0000 7fc20e1ff700 1 --2- 172.21.15.133:0/1337573337 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc1f00a11d0 0x7fc1f00a3680 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc009c60 tx=0x7fc1fc009260 comp rx=0 tx=0).stop 2023-11-27T15:24:21.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.923+0000 7fc20e1ff700 1 -- 172.21.15.133:0/1337573337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2081002c0 msgr2=0x7fc2080748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:21.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.923+0000 7fc20e1ff700 1 --2- 172.21.15.133:0/1337573337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2081002c0 0x7fc2080748a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8004730 tx=0x7fc1f8004760 comp rx=0 tx=0).stop 2023-11-27T15:24:21.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.923+0000 7fc20e1ff700 1 -- 172.21.15.133:0/1337573337 shutdown_connections 2023-11-27T15:24:21.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.923+0000 7fc20e1ff700 1 --2- 172.21.15.133:0/1337573337 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc1f00a11d0 0x7fc1f00a3680 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.923+0000 7fc20e1ff700 1 --2- 172.21.15.133:0/1337573337 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc208101e40 0x7fc208073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.923+0000 7fc20e1ff700 1 --2- 172.21.15.133:0/1337573337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2081002c0 0x7fc2080748a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.923+0000 7fc20e1ff700 1 --2- 172.21.15.133:0/1337573337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2081014b0 0x7fc208072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:21.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.923+0000 7fc20e1ff700 1 -- 172.21.15.133:0/1337573337 >> 172.21.15.133:0/1337573337 conn(0x7fc2080fb8a0 msgr2=0x7fc2081049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:21.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.923+0000 7fc20e1ff700 1 -- 172.21.15.133:0/1337573337 shutdown_connections 2023-11-27T15:24:21.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:21.923+0000 7fc20e1ff700 1 -- 172.21.15.133:0/1337573337 wait complete. 2023-11-27T15:24:21.976 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:24:22.229 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:22.229 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:22.322 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:22.322 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:22.336 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:22.337 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:22.337 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 47 -lt 100 ']' 2023-11-27T15:24:22.337 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 47 + 1 2023-11-27T15:24:22.371 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=48 2023-11-27T15:24:22.371 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:22.384 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:22.384 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:22.386 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:22.387 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:22.387 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:22.888 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:22.888 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:22.933 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:22.933 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:22.961 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:22.962 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:22.962 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 48 -lt 100 ']' 2023-11-27T15:24:22.962 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 48 + 1 2023-11-27T15:24:23.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:22 smithi133 ceph-mon[97860]: pgmap v392: 129 pgs: 129 active+clean; 1.2 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-11-27T15:24:23.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:22 smithi133 ceph-mon[97860]: scrub queued for path: / 2023-11-27T15:24:23.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:22 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:24:23.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:22 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:24:23.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:22 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:24:23.030 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=49 2023-11-27T15:24:23.030 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:23.068 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:23.068 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:23.071 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:23.071 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:23.072 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:23.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:22 smithi177 ceph-mon[131504]: pgmap v392: 129 pgs: 129 active+clean; 1.2 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-11-27T15:24:23.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:22 smithi177 ceph-mon[131504]: scrub queued for path: / 2023-11-27T15:24:23.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:22 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:24:23.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:22 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:24:23.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:22 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:24:23.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:22 smithi174 ceph-mon[131110]: pgmap v392: 129 pgs: 129 active+clean; 1.2 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 14 op/s 2023-11-27T15:24:23.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:22 smithi174 ceph-mon[131110]: scrub queued for path: / 2023-11-27T15:24:23.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:22 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:24:23.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:22 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:24:23.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:22 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:24:23.554 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:23.554 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:23.740 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:23.740 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:23.762 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:23.763 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:23.763 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 49 -lt 100 ']' 2023-11-27T15:24:23.763 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 49 + 1 2023-11-27T15:24:23.830 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=50 2023-11-27T15:24:23.830 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:23.918 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:23.918 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:23.921 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:23.921 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:23.922 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:24.629 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:24.629 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:24.806 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:24.806 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:24.821 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:24.821 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:24.821 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 50 -lt 100 ']' 2023-11-27T15:24:24.822 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 50 + 1 2023-11-27T15:24:24.846 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=51 2023-11-27T15:24:24.846 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:24.910 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:24.910 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:24.913 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:24.913 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:24.914 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:25.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:24 smithi174 ceph-mon[131110]: pgmap v393: 129 pgs: 129 active+clean; 1.2 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-11-27T15:24:25.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.235+0000 7fe105dff700 1 -- 172.21.15.133:0/718314140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe100101e50 msgr2=0x7fe100108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:25.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.235+0000 7fe105dff700 1 --2- 172.21.15.133:0/718314140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe100101e50 0x7fe100108710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009990 tx=0x7fe0f8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.235+0000 7fe105dff700 1 -- 172.21.15.133:0/718314140 shutdown_connections 2023-11-27T15:24:25.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.235+0000 7fe105dff700 1 --2- 172.21.15.133:0/718314140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe100101e50 0x7fe100108710 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.235+0000 7fe105dff700 1 --2- 172.21.15.133:0/718314140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1001002d0 0x7fe1001006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.235+0000 7fe105dff700 1 --2- 172.21.15.133:0/718314140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1001014c0 0x7fe100101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.235+0000 7fe105dff700 1 -- 172.21.15.133:0/718314140 >> 172.21.15.133:0/718314140 conn(0x7fe1000fb8b0 msgr2=0x7fe1000fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:25.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.235+0000 7fe105dff700 1 -- 172.21.15.133:0/718314140 shutdown_connections 2023-11-27T15:24:25.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.235+0000 7fe105dff700 1 -- 172.21.15.133:0/718314140 wait complete. 2023-11-27T15:24:25.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.237+0000 7fe105dff700 1 Processor -- start 2023-11-27T15:24:25.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.237+0000 7fe105dff700 1 -- start start 2023-11-27T15:24:25.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.237+0000 7fe105dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1001002d0 0x7fe1000748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.237+0000 7fe105dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1001014c0 0x7fe100072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.237+0000 7fe105dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe100101e50 0x7fe100073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.237+0000 7fe105dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe100073bc0 con 0x7fe1001014c0 2023-11-27T15:24:25.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.237+0000 7fe105dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe100073d30 con 0x7fe100101e50 2023-11-27T15:24:25.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.237+0000 7fe105dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1001a0050 con 0x7fe1001002d0 2023-11-27T15:24:25.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.238+0000 7fe0ff7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1001002d0 0x7fe1000748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.238+0000 7fe0f6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1001014c0 0x7fe100072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.238+0000 7fe0ff7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1001002d0 0x7fe1000748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:60790/0 (socket says 172.21.15.133:60790) 2023-11-27T15:24:25.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.238+0000 7fe0ff7fe700 1 -- 172.21.15.133:0/3989741955 learned_addr learned my addr 172.21.15.133:0/3989741955 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:25.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.238+0000 7fe0fffff700 1 --2- 172.21.15.133:0/3989741955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe100101e50 0x7fe100073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.238+0000 7fe0ff7fe700 1 -- 172.21.15.133:0/3989741955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe100101e50 msgr2=0x7fe100073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:25.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.238+0000 7fe0ff7fe700 1 --2- 172.21.15.133:0/3989741955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe100101e50 0x7fe100073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.238+0000 7fe0ff7fe700 1 -- 172.21.15.133:0/3989741955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1001014c0 msgr2=0x7fe100072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:25.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.238+0000 7fe0ff7fe700 1 --2- 172.21.15.133:0/3989741955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1001014c0 0x7fe100072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.238+0000 7fe0ff7fe700 1 -- 172.21.15.133:0/3989741955 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f80095f0 con 0x7fe1001002d0 2023-11-27T15:24:25.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.238+0000 7fe0fffff700 1 --2- 172.21.15.133:0/3989741955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe100101e50 0x7fe100073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:25.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.239+0000 7fe0ff7fe700 1 --2- 172.21.15.133:0/3989741955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1001002d0 0x7fe1000748a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000efc0 tx=0x7fe0f000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:25.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.239+0000 7fe0fd7fa700 1 -- 172.21.15.133:0/3989741955 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0018070 con 0x7fe1001002d0 2023-11-27T15:24:25.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.239+0000 7fe0fd7fa700 1 -- 172.21.15.133:0/3989741955 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0f000f040 con 0x7fe1001002d0 2023-11-27T15:24:25.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.239+0000 7fe0fd7fa700 1 -- 172.21.15.133:0/3989741955 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0004310 con 0x7fe1001002d0 2023-11-27T15:24:25.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.240+0000 7fe105dff700 1 -- 172.21.15.133:0/3989741955 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe10010bca0 con 0x7fe1001002d0 2023-11-27T15:24:25.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.240+0000 7fe105dff700 1 -- 172.21.15.133:0/3989741955 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe10010bb10 con 0x7fe1001002d0 2023-11-27T15:24:25.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.243+0000 7fe0fd7fa700 1 -- 172.21.15.133:0/3989741955 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe0f000ca70 con 0x7fe1001002d0 2023-11-27T15:24:25.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.244+0000 7fe0fd7fa700 1 --2- 172.21.15.133:0/3989741955 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe0ec0a11d0 0x7fe0ec0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.245+0000 7fe0fd7fa700 1 -- 172.21.15.133:0/3989741955 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe0f00ccb30 con 0x7fe1001002d0 2023-11-27T15:24:25.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.245+0000 7fe0f6ffd700 1 --2- 172.21.15.133:0/3989741955 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe0ec0a11d0 0x7fe0ec0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.245+0000 7fe0f6ffd700 1 --2- 172.21.15.133:0/3989741955 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe0ec0a11d0 0x7fe0ec0a3680 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8007be0 tx=0x7fe0e80073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:25.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:24 smithi133 ceph-mon[97860]: pgmap v393: 129 pgs: 129 active+clean; 1.2 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-11-27T15:24:25.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.313+0000 7fe1196b9b80 1 Processor -- start 2023-11-27T15:24:25.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.313+0000 7fe1196b9b80 1 -- start start 2023-11-27T15:24:25.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.313+0000 7fe1196b9b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e44c00 0x55b883e45010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.313+0000 7fe1196b9b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b883e45550 0x55b883e45b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.313+0000 7fe1196b9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e46110 0x55b883e46580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.313+0000 7fe1196b9b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b883d4a060 con 0x55b883e44c00 2023-11-27T15:24:25.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.313+0000 7fe1196b9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b883d9c8a0 con 0x55b883e46110 2023-11-27T15:24:25.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.313+0000 7fe1196b9b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b883d95d30 con 0x55b883e45550 2023-11-27T15:24:25.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.314+0000 7fe0ff7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e44c00 0x55b883e45010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.314+0000 7fe0f6ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b883e45550 0x55b883e45b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.314+0000 7fe0ff7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e44c00 0x55b883e45010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:40514/0 (socket says 172.21.15.133:40514) 2023-11-27T15:24:25.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.314+0000 7fe0ff7fe700 1 -- 172.21.15.133:0/3296886691 learned_addr learned my addr 172.21.15.133:0/3296886691 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:25.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.314+0000 7fe0fffff700 1 --2- 172.21.15.133:0/3296886691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e46110 0x55b883e46580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.314+0000 7fe0ff7fe700 1 -- 172.21.15.133:0/3296886691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b883e45550 msgr2=0x55b883e45b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:25.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.314+0000 7fe0ff7fe700 1 --2- 172.21.15.133:0/3296886691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b883e45550 0x55b883e45b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.314+0000 7fe0ff7fe700 1 -- 172.21.15.133:0/3296886691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e46110 msgr2=0x55b883e46580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:25.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.314+0000 7fe0ff7fe700 1 --2- 172.21.15.133:0/3296886691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e46110 0x55b883e46580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.314+0000 7fe0ff7fe700 1 -- 172.21.15.133:0/3296886691 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b883c17630 con 0x55b883e44c00 2023-11-27T15:24:25.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.314+0000 7fe0ff7fe700 1 --2- 172.21.15.133:0/3296886691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e44c00 0x55b883e45010 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00090a0 tx=0x7fe0f00999f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9adb11b45d64e610 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.315+0000 7fe0f5ffb700 1 -- 172.21.15.133:0/3296886691 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0099580 con 0x55b883e44c00 2023-11-27T15:24:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.315+0000 7fe0f5ffb700 1 -- 172.21.15.133:0/3296886691 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0f0099c20 con 0x55b883e44c00 2023-11-27T15:24:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.315+0000 7fe0f5ffb700 1 -- 172.21.15.133:0/3296886691 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0099e80 con 0x55b883e44c00 2023-11-27T15:24:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.315+0000 7fe1196b9b80 1 -- 172.21.15.133:0/3296886691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e44c00 msgr2=0x55b883e45010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.315+0000 7fe1196b9b80 1 --2- 172.21.15.133:0/3296886691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e44c00 0x55b883e45010 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00090a0 tx=0x7fe0f00999f0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.316+0000 7fe1196b9b80 1 -- 172.21.15.133:0/3296886691 shutdown_connections 2023-11-27T15:24:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.316+0000 7fe1196b9b80 1 --2- 172.21.15.133:0/3296886691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e46110 0x55b883e46580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.316+0000 7fe1196b9b80 1 --2- 172.21.15.133:0/3296886691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e44c00 0x55b883e45010 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.316+0000 7fe1196b9b80 1 --2- 172.21.15.133:0/3296886691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b883e45550 0x55b883e45b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.316+0000 7fe1196b9b80 1 -- 172.21.15.133:0/3296886691 >> 172.21.15.133:0/3296886691 conn(0x55b883e3a360 msgr2=0x55b883e3f670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.316+0000 7fe1196b9b80 1 -- 172.21.15.133:0/3296886691 shutdown_connections 2023-11-27T15:24:25.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.316+0000 7fe1196b9b80 1 -- 172.21.15.133:0/3296886691 wait complete. 2023-11-27T15:24:25.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.316+0000 7fe1196b9b80 1 -- start start 2023-11-27T15:24:25.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.317+0000 7fe1196b9b80 1 Processor -- start 2023-11-27T15:24:25.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.317+0000 7fe1196b9b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e44c00 0x55b883e4e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.317+0000 7fe1196b9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e45550 0x55b883e4eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.317+0000 7fe1196b9b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b883e46110 0x55b883e55bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.317+0000 7fe1196b9b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b883d378e0 con 0x55b883e44c00 2023-11-27T15:24:25.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.317+0000 7fe1196b9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b883d37580 con 0x55b883e45550 2023-11-27T15:24:25.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.317+0000 7fe1196b9b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b883c19920 con 0x55b883e46110 2023-11-27T15:24:25.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.317+0000 7fe1196b9b80 10 client.0 _ll_register_callbacks cb 0x55b883d55780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:25.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.317+0000 7fe0f6ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e45550 0x55b883e4eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.317+0000 7fe0ff7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e44c00 0x55b883e4e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.317+0000 7fe0f6ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e45550 0x55b883e4eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:53908/0 (socket says 172.21.15.133:53908) 2023-11-27T15:24:25.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.317+0000 7fe0f6ffd700 1 -- 172.21.15.133:0/2323391359 learned_addr learned my addr 172.21.15.133:0/2323391359 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:25.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.317+0000 7fe0fffff700 1 --2- 172.21.15.133:0/2323391359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b883e46110 0x55b883e55bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.318+0000 7fe0ff7fe700 1 -- 172.21.15.133:0/2323391359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b883e46110 msgr2=0x55b883e55bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:25.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.318+0000 7fe0ff7fe700 1 --2- 172.21.15.133:0/2323391359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b883e46110 0x55b883e55bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.318+0000 7fe0ff7fe700 1 -- 172.21.15.133:0/2323391359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e45550 msgr2=0x55b883e4eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:25.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.318+0000 7fe0ff7fe700 1 --2- 172.21.15.133:0/2323391359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e45550 0x55b883e4eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.318+0000 7fe0ff7fe700 1 -- 172.21.15.133:0/2323391359 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe10010bb10 con 0x55b883e44c00 2023-11-27T15:24:25.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.318+0000 7fe0f6ffd700 1 --2- 172.21.15.133:0/2323391359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e45550 0x55b883e4eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:25.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.318+0000 7fe0ff7fe700 1 --2- 172.21.15.133:0/2323391359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e44c00 0x55b883e4e640 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00090a0 tx=0x7fe0f0009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7474be90e745089d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:25.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.318+0000 7fe0deffd700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:24:25.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.318+0000 7fe0deffd700 1 -- 172.21.15.133:0/2323391359 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f00a1fe0 con 0x55b883e44c00 2023-11-27T15:24:25.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.318+0000 7fe0deffd700 1 -- 172.21.15.133:0/2323391359 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe0f001be30 con 0x55b883e44c00 2023-11-27T15:24:25.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.318+0000 7fe1196b9b80 1 -- 172.21.15.133:0/2323391359 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b883e37b20 con 0x55b883e44c00 2023-11-27T15:24:25.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.319+0000 7fe0deffd700 1 -- 172.21.15.133:0/2323391359 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0f0099e80 con 0x55b883e44c00 2023-11-27T15:24:25.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.319+0000 7fe1196b9b80 10 client.15774 fetch_fsmap learned FSMap version 23 2023-11-27T15:24:25.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.319+0000 7fe1196b9b80 1 -- 172.21.15.133:0/2323391359 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55b883d5c190 con 0x55b883e44c00 2023-11-27T15:24:25.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.319+0000 7fe0deffd700 1 -- 172.21.15.133:0/2323391359 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b883e37b20 con 0x55b883e44c00 2023-11-27T15:24:25.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.319+0000 7fe0deffd700 1 -- 172.21.15.133:0/2323391359 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe0f0098b80 con 0x55b883e44c00 2023-11-27T15:24:25.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.319+0000 7fe1196b9b80 10 client.15774 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:24:25.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.319+0000 7fe1196b9b80 10 client.15774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:25.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.319+0000 7fe1196b9b80 20 client.15774 populate_metadata read hostname 'smithi133' 2023-11-27T15:24:25.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.319+0000 7fe1196b9b80 1 --2- 172.21.15.133:0/2323391359 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55b883e59e80 0x55b883e5c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.319+0000 7fe1196b9b80 4 client.15774 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:25.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.319+0000 7fe1196b9b80 1 -- 172.21.15.133:0/2323391359 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b883d580a0 con 0x55b883e59e80 2023-11-27T15:24:25.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.320+0000 7fe0f6ffd700 1 --2- 172.21.15.133:0/2323391359 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55b883e59e80 0x55b883e5c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.320+0000 7fe0f6ffd700 1 --2- 172.21.15.133:0/2323391359 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55b883e59e80 0x55b883e5c260 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=459da78e4c51670d server_cookie=c925086898054685 in_seq=0 out_seq=0 2023-11-27T15:24:25.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.320+0000 7fe0deffd700 10 client.15774 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:25.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.321+0000 7fe0deffd700 1 -- 172.21.15.133:0/2323391359 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b883d580a0 con 0x55b883e59e80 2023-11-27T15:24:25.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.321+0000 7fe0deffd700 10 client.15774 handle_command_reply: tid=0 2023-11-27T15:24:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.321+0000 7fe1196b9b80 1 client.15774 shutdown 2023-11-27T15:24:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.321+0000 7fe1196b9b80 1 -- 172.21.15.133:0/2323391359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e44c00 msgr2=0x55b883e4e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.321+0000 7fe1196b9b80 1 --2- 172.21.15.133:0/2323391359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e44c00 0x55b883e4e640 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00090a0 tx=0x7fe0f0009390 comp rx=0 tx=0).stop 2023-11-27T15:24:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.321+0000 7fe1196b9b80 1 -- 172.21.15.133:0/2323391359 shutdown_connections 2023-11-27T15:24:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.321+0000 7fe1196b9b80 1 --2- 172.21.15.133:0/2323391359 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55b883e59e80 0x55b883e5c260 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.321+0000 7fe1196b9b80 1 --2- 172.21.15.133:0/2323391359 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b883e46110 0x55b883e55bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.321+0000 7fe1196b9b80 1 --2- 172.21.15.133:0/2323391359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e44c00 0x55b883e4e640 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.321+0000 7fe1196b9b80 1 --2- 172.21.15.133:0/2323391359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e45550 0x55b883e4eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.321+0000 7fe1196b9b80 1 -- 172.21.15.133:0/2323391359 >> 172.21.15.133:0/2323391359 conn(0x55b883e3a360 msgr2=0x55b883e3ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.321+0000 7fe0deffd700 0 client.15774 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.321+0000 7fe1196b9b80 1 -- 172.21.15.133:0/2323391359 shutdown_connections 2023-11-27T15:24:25.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.321+0000 7fe1196b9b80 1 -- 172.21.15.133:0/2323391359 wait complete. 2023-11-27T15:24:25.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.321+0000 7fe1196b9b80 20 client.15774 trim_cache size 0 max 16384 2023-11-27T15:24:25.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:24 smithi177 ceph-mon[131504]: pgmap v393: 129 pgs: 129 active+clean; 1.2 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-11-27T15:24:25.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.340+0000 7fe1196b9b80 1 Processor -- start 2023-11-27T15:24:25.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.340+0000 7fe1196b9b80 1 -- start start 2023-11-27T15:24:25.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.340+0000 7fe1196b9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e44c00 0x55b883e4d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.340+0000 7fe1196b9b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e45550 0x55b883e55b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.340+0000 7fe1196b9b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b883e46110 0x55b883e564b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.340+0000 7fe1196b9b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b883e4daf0 con 0x55b883e45550 2023-11-27T15:24:25.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.340+0000 7fe1196b9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b883e59c70 con 0x55b883e44c00 2023-11-27T15:24:25.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.340+0000 7fe1196b9b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b883e410f0 con 0x55b883e46110 2023-11-27T15:24:25.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.340+0000 7fe0ff7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e44c00 0x55b883e4d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.340+0000 7fe0f6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e45550 0x55b883e55b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.340+0000 7fe0ff7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e44c00 0x55b883e4d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:53918/0 (socket says 172.21.15.133:53918) 2023-11-27T15:24:25.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.340+0000 7fe0ff7fe700 1 -- 172.21.15.133:0/1865560243 learned_addr learned my addr 172.21.15.133:0/1865560243 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:25.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.340+0000 7fe0fffff700 1 --2- 172.21.15.133:0/1865560243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b883e46110 0x55b883e564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.341+0000 7fe0ff7fe700 1 -- 172.21.15.133:0/1865560243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b883e46110 msgr2=0x55b883e564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:25.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.341+0000 7fe0ff7fe700 1 --2- 172.21.15.133:0/1865560243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b883e46110 0x55b883e564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.341+0000 7fe0ff7fe700 1 -- 172.21.15.133:0/1865560243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e45550 msgr2=0x55b883e55b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:25.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.341+0000 7fe0ff7fe700 1 --2- 172.21.15.133:0/1865560243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e45550 0x55b883e55b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.341+0000 7fe0ff7fe700 1 -- 172.21.15.133:0/1865560243 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b883d5bee0 con 0x55b883e44c00 2023-11-27T15:24:25.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.341+0000 7fe0ff7fe700 1 --2- 172.21.15.133:0/1865560243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e44c00 0x55b883e4d320 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0096120 tx=0x7fe0f00ca240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edef630246e06ac6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:25.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.341+0000 7fe0f67fc700 1 -- 172.21.15.133:0/1865560243 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f00a1fe0 con 0x55b883e44c00 2023-11-27T15:24:25.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.341+0000 7fe0f67fc700 1 -- 172.21.15.133:0/1865560243 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0f001be30 con 0x55b883e44c00 2023-11-27T15:24:25.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.341+0000 7fe0f67fc700 1 -- 172.21.15.133:0/1865560243 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0094950 con 0x55b883e44c00 2023-11-27T15:24:25.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.342+0000 7fe1196b9b80 1 -- 172.21.15.133:0/1865560243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e44c00 msgr2=0x55b883e4d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:25.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.342+0000 7fe1196b9b80 1 --2- 172.21.15.133:0/1865560243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e44c00 0x55b883e4d320 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0096120 tx=0x7fe0f00ca240 comp rx=0 tx=0).stop 2023-11-27T15:24:25.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.342+0000 7fe1196b9b80 1 -- 172.21.15.133:0/1865560243 shutdown_connections 2023-11-27T15:24:25.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.342+0000 7fe1196b9b80 1 --2- 172.21.15.133:0/1865560243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b883e46110 0x55b883e564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.342+0000 7fe1196b9b80 1 --2- 172.21.15.133:0/1865560243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e44c00 0x55b883e4d320 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.342+0000 7fe1196b9b80 1 --2- 172.21.15.133:0/1865560243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e45550 0x55b883e55b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.342+0000 7fe1196b9b80 1 -- 172.21.15.133:0/1865560243 >> 172.21.15.133:0/1865560243 conn(0x55b883e3a360 msgr2=0x55b883e60bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:25.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.342+0000 7fe1196b9b80 1 -- 172.21.15.133:0/1865560243 shutdown_connections 2023-11-27T15:24:25.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.342+0000 7fe1196b9b80 1 -- 172.21.15.133:0/1865560243 wait complete. 2023-11-27T15:24:25.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.342+0000 7fe1196b9b80 1 -- start start 2023-11-27T15:24:25.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.343+0000 7fe1196b9b80 1 Processor -- start 2023-11-27T15:24:25.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.343+0000 7fe1196b9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e44c00 0x55b883e51610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.343+0000 7fe1196b9b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e45550 0x55b883e51b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.343+0000 7fe1196b9b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b883e46110 0x55b883e54b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.343+0000 7fe1196b9b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b883e550b0 con 0x55b883e45550 2023-11-27T15:24:25.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.343+0000 7fe1196b9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b883e55220 con 0x55b883e44c00 2023-11-27T15:24:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.343+0000 7fe1196b9b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b883e553e0 con 0x55b883e46110 2023-11-27T15:24:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.343+0000 7fe1196b9b80 10 client.0 _ll_register_callbacks cb 0x7fe0f8009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.343+0000 7fe0ff7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e44c00 0x55b883e51610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.343+0000 7fe0f6ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e45550 0x55b883e51b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.343+0000 7fe0ff7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e44c00 0x55b883e51610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:53922/0 (socket says 172.21.15.133:53922) 2023-11-27T15:24:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.343+0000 7fe0ff7fe700 1 -- 172.21.15.133:0/374952661 learned_addr learned my addr 172.21.15.133:0/374952661 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.343+0000 7fe0fffff700 1 --2- 172.21.15.133:0/374952661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b883e46110 0x55b883e54b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.344+0000 7fe0ff7fe700 1 -- 172.21.15.133:0/374952661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b883e46110 msgr2=0x55b883e54b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.344+0000 7fe0ff7fe700 1 --2- 172.21.15.133:0/374952661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b883e46110 0x55b883e54b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.344+0000 7fe0ff7fe700 1 -- 172.21.15.133:0/374952661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e45550 msgr2=0x55b883e51b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.344+0000 7fe0ff7fe700 1 --2- 172.21.15.133:0/374952661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e45550 0x55b883e51b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.344+0000 7fe0ff7fe700 1 -- 172.21.15.133:0/374952661 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b883d5c190 con 0x55b883e44c00 2023-11-27T15:24:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.344+0000 7fe0f6ffd700 1 --2- 172.21.15.133:0/374952661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e45550 0x55b883e51b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:25.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.344+0000 7fe0ff7fe700 1 --2- 172.21.15.133:0/374952661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e44c00 0x55b883e51610 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00a1ab0 tx=0x7fe0f00981d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=959fdc698e021804 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:25.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.344+0000 7fe0deffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:24:25.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.344+0000 7fe0deffd700 1 -- 172.21.15.133:0/374952661 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f009a080 con 0x55b883e44c00 2023-11-27T15:24:25.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.344+0000 7fe0deffd700 1 -- 172.21.15.133:0/374952661 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe0f0094930 con 0x55b883e44c00 2023-11-27T15:24:25.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.344+0000 7fe1196b9b80 1 -- 172.21.15.133:0/374952661 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0f0099c20 con 0x55b883e44c00 2023-11-27T15:24:25.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.345+0000 7fe0deffd700 1 -- 172.21.15.133:0/374952661 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0f00a1fe0 con 0x55b883e44c00 2023-11-27T15:24:25.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.345+0000 7fe1196b9b80 10 client.25820 fetch_fsmap learned FSMap version 23 2023-11-27T15:24:25.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.345+0000 7fe1196b9b80 1 -- 172.21.15.133:0/374952661 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55b883bacc60 con 0x55b883e44c00 2023-11-27T15:24:25.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.345+0000 7fe0deffd700 1 -- 172.21.15.133:0/374952661 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe0f0083020 con 0x55b883e44c00 2023-11-27T15:24:25.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.345+0000 7fe0deffd700 1 -- 172.21.15.133:0/374952661 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe0f0095a10 con 0x55b883e44c00 2023-11-27T15:24:25.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.345+0000 7fe1196b9b80 10 client.25820 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:24:25.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.345+0000 7fe1196b9b80 10 client.25820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:25.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.345+0000 7fe1196b9b80 20 client.25820 populate_metadata read hostname 'smithi133' 2023-11-27T15:24:25.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.345+0000 7fe1196b9b80 1 --2- 172.21.15.133:0/374952661 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55b883eae390 0x55b883eb0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:25.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.345+0000 7fe1196b9b80 4 client.25820 mds_command: new command op to 24560 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:24:25.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.345+0000 7fe1196b9b80 1 -- 172.21.15.133:0/374952661 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b883d64660 con 0x55b883eae390 2023-11-27T15:24:25.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.345+0000 7fe0f6ffd700 1 --2- 172.21.15.133:0/374952661 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55b883eae390 0x55b883eb0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:25.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.346+0000 7fe0deffd700 10 client.25820 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:25.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.346+0000 7fe0f6ffd700 1 --2- 172.21.15.133:0/374952661 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55b883eae390 0x55b883eb0770 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47007a4344b4efcb server_cookie=bbf3db1c9d9ffb5e in_seq=0 out_seq=0 2023-11-27T15:24:25.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.346+0000 7fe0deffd700 1 -- 172.21.15.133:0/374952661 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b883d64660 con 0x55b883eae390 2023-11-27T15:24:25.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.346+0000 7fe0deffd700 10 client.25820 handle_command_reply: tid=0 2023-11-27T15:24:25.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.346+0000 7fe1196b9b80 1 client.25820 shutdown 2023-11-27T15:24:25.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.346+0000 7fe1196b9b80 1 -- 172.21.15.133:0/374952661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e44c00 msgr2=0x55b883e51610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:25.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.346+0000 7fe1196b9b80 1 --2- 172.21.15.133:0/374952661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e44c00 0x55b883e51610 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00a1ab0 tx=0x7fe0f00981d0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.346+0000 7fe1196b9b80 1 -- 172.21.15.133:0/374952661 shutdown_connections 2023-11-27T15:24:25.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.346+0000 7fe1196b9b80 1 --2- 172.21.15.133:0/374952661 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55b883eae390 0x55b883eb0770 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.355 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.346+0000 7fe1196b9b80 1 --2- 172.21.15.133:0/374952661 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b883e46110 0x55b883e54b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.355 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.346+0000 7fe1196b9b80 1 --2- 172.21.15.133:0/374952661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b883e44c00 0x55b883e51610 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.355 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.346+0000 7fe1196b9b80 1 --2- 172.21.15.133:0/374952661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b883e45550 0x55b883e51b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.355 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.346+0000 7fe0deffd700 0 client.25820 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:25.355 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.346+0000 7fe1196b9b80 1 -- 172.21.15.133:0/374952661 >> 172.21.15.133:0/374952661 conn(0x55b883e3a360 msgr2=0x55b883e48fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:25.355 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.346+0000 7fe1196b9b80 1 -- 172.21.15.133:0/374952661 shutdown_connections 2023-11-27T15:24:25.355 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.346+0000 7fe1196b9b80 1 -- 172.21.15.133:0/374952661 wait complete. 2023-11-27T15:24:25.355 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.347+0000 7fe1196b9b80 20 client.25820 trim_cache size 0 max 16384 2023-11-27T15:24:25.355 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:24:25.355 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-27T15:24:25.355 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-11-27T15:24:25.355 INFO:teuthology.orchestra.run.smithi133.stdout: "e840364b-dfce-4ae4-9579-41368a7f5d44": { 2023-11-27T15:24:25.356 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-11-27T15:24:25.356 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "e840364b-dfce-4ae4-9579-41368a7f5d44", 2023-11-27T15:24:25.356 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-11-27T15:24:25.356 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:24:25.356 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:24:25.356 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:24:25.356 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.347+0000 7fe105dff700 1 -- 172.21.15.133:0/3989741955 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe0ec0a11d0 msgr2=0x7fe0ec0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:25.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.347+0000 7fe105dff700 1 --2- 172.21.15.133:0/3989741955 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe0ec0a11d0 0x7fe0ec0a3680 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8007be0 tx=0x7fe0e80073d0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.347+0000 7fe105dff700 1 -- 172.21.15.133:0/3989741955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1001002d0 msgr2=0x7fe1000748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:25.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.347+0000 7fe105dff700 1 --2- 172.21.15.133:0/3989741955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1001002d0 0x7fe1000748a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe0f000efc0 tx=0x7fe0f000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.347+0000 7fe105dff700 1 -- 172.21.15.133:0/3989741955 shutdown_connections 2023-11-27T15:24:25.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.347+0000 7fe105dff700 1 --2- 172.21.15.133:0/3989741955 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe0ec0a11d0 0x7fe0ec0a3680 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.347+0000 7fe105dff700 1 --2- 172.21.15.133:0/3989741955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe100101e50 0x7fe100073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.347+0000 7fe105dff700 1 --2- 172.21.15.133:0/3989741955 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe1001002d0 0x7fe1000748a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.347+0000 7fe105dff700 1 --2- 172.21.15.133:0/3989741955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1001014c0 0x7fe100072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:25.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.347+0000 7fe105dff700 1 -- 172.21.15.133:0/3989741955 >> 172.21.15.133:0/3989741955 conn(0x7fe1000fb8b0 msgr2=0x7fe1000fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:25.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.347+0000 7fe105dff700 1 -- 172.21.15.133:0/3989741955 shutdown_connections 2023-11-27T15:24:25.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:25.347+0000 7fe105dff700 1 -- 172.21.15.133:0/3989741955 wait complete. 2023-11-27T15:24:25.409 INFO:tasks.cephfs.filesystem:scrub status for tag:e840364b-dfce-4ae4-9579-41368a7f5d44 - {'path': '/', 'tag': 'e840364b-dfce-4ae4-9579-41368a7f5d44', 'options': 'recursive,force'} 2023-11-27T15:24:25.712 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:25.712 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:25.840 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:25.840 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:25.854 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:25.854 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:25.854 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 51 -lt 100 ']' 2023-11-27T15:24:25.855 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 51 + 1 2023-11-27T15:24:25.874 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=52 2023-11-27T15:24:25.874 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:25.910 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:25.910 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:25.913 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:25.913 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:25.914 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:26.345 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:26.346 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:26.465 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:26.465 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:26.478 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:26.478 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:26.478 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 52 -lt 100 ']' 2023-11-27T15:24:26.478 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 52 + 1 2023-11-27T15:24:26.504 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=53 2023-11-27T15:24:26.504 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:26.532 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:26.533 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:26.535 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:26.536 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:26.536 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:26.970 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:26.971 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:27.106 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:27.106 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:27.129 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:27.129 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:27.129 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 53 -lt 100 ']' 2023-11-27T15:24:27.129 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 53 + 1 2023-11-27T15:24:27.149 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=54 2023-11-27T15:24:27.149 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:27.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:26 smithi174 ceph-mon[131110]: pgmap v394: 129 pgs: 129 active+clean; 1.2 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 15 op/s 2023-11-27T15:24:27.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:26 smithi174 ceph-mon[131110]: scrub summary: idle 2023-11-27T15:24:27.180 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:27.180 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:27.183 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:27.184 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:27.184 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:27.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:26 smithi133 ceph-mon[97860]: pgmap v394: 129 pgs: 129 active+clean; 1.2 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 15 op/s 2023-11-27T15:24:27.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:26 smithi133 ceph-mon[97860]: scrub summary: idle 2023-11-27T15:24:27.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:26 smithi177 ceph-mon[131504]: pgmap v394: 129 pgs: 129 active+clean; 1.2 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 15 op/s 2023-11-27T15:24:27.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:26 smithi177 ceph-mon[131504]: scrub summary: idle 2023-11-27T15:24:27.695 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:27.695 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:27.889 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:27.890 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:27.902 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:27.903 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:27.903 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 54 -lt 100 ']' 2023-11-27T15:24:27.903 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 54 + 1 2023-11-27T15:24:27.923 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=55 2023-11-27T15:24:27.923 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:27.953 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:27.954 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:27.957 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:27.957 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:27.958 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:28.437 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:28.437 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:28.598 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:28.598 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:28.612 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:28.612 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:28.612 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 55 -lt 100 ']' 2023-11-27T15:24:28.612 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 55 + 1 2023-11-27T15:24:28.670 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=56 2023-11-27T15:24:28.671 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:28.734 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:28.735 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:28.738 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:28.738 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:28.738 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:29.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:28 smithi133 ceph-mon[97860]: pgmap v395: 129 pgs: 129 active+clean; 1.2 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2023-11-27T15:24:29.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:28 smithi177 ceph-mon[131504]: pgmap v395: 129 pgs: 129 active+clean; 1.2 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2023-11-27T15:24:29.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:28 smithi174 ceph-mon[131110]: pgmap v395: 129 pgs: 129 active+clean; 1.2 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2023-11-27T15:24:29.195 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:29.195 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:29.323 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:29.323 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:29.336 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:29.336 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:29.336 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 56 -lt 100 ']' 2023-11-27T15:24:29.336 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 56 + 1 2023-11-27T15:24:29.370 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=57 2023-11-27T15:24:29.371 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:29.442 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:29.443 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:29.446 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:29.446 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:29.446 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:29.961 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:29.962 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:30.106 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:30.106 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:30.121 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:30.121 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:30.121 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 57 -lt 100 ']' 2023-11-27T15:24:30.121 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 57 + 1 2023-11-27T15:24:30.140 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=58 2023-11-27T15:24:30.140 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:30.170 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:30.170 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:30.173 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:30.173 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:30.174 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:30.637 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:30.637 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:30.764 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:30.764 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:30.778 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:30.778 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:30.778 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 58 -lt 100 ']' 2023-11-27T15:24:30.779 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 58 + 1 2023-11-27T15:24:30.798 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=59 2023-11-27T15:24:30.798 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:30.828 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:30.829 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:30.831 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:30.832 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:30.832 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:31.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:30 smithi177 ceph-mon[131504]: pgmap v396: 129 pgs: 129 active+clean; 1.2 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-11-27T15:24:31.136 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:30 smithi133 ceph-mon[97860]: pgmap v396: 129 pgs: 129 active+clean; 1.2 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-11-27T15:24:31.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:30 smithi174 ceph-mon[131110]: pgmap v396: 129 pgs: 129 active+clean; 1.2 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-11-27T15:24:31.278 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:31.278 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:31.423 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:31.423 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:31.436 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:31.436 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:31.437 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 59 -lt 100 ']' 2023-11-27T15:24:31.437 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 59 + 1 2023-11-27T15:24:31.456 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=60 2023-11-27T15:24:31.456 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:31.492 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:31.492 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:31.495 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:31.495 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:31.496 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:32.069 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:32.070 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:32.271 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:32.271 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:32.293 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:32.294 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:32.294 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 60 -lt 100 ']' 2023-11-27T15:24:32.294 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 60 + 1 2023-11-27T15:24:32.328 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=61 2023-11-27T15:24:32.328 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:32.400 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:32.401 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:32.403 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:32.404 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:32.404 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:33.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:32 smithi177 ceph-mon[131504]: pgmap v397: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 14 op/s 2023-11-27T15:24:33.144 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:33.144 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:33.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:32 smithi174 ceph-mon[131110]: pgmap v397: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 14 op/s 2023-11-27T15:24:33.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:32 smithi133 ceph-mon[97860]: pgmap v397: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 14 op/s 2023-11-27T15:24:33.488 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:33.488 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:33.527 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:33.527 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:33.527 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 61 -lt 100 ']' 2023-11-27T15:24:33.528 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 61 + 1 2023-11-27T15:24:33.586 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=62 2023-11-27T15:24:33.586 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:33.641 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:33.642 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:33.645 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:33.645 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:33.645 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.090+0000 7f55a2dff700 1 -- 172.21.15.133:0/594837420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559c102460 msgr2=0x7f559c102870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.090+0000 7f55a2dff700 1 --2- 172.21.15.133:0/594837420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559c102460 0x7f559c102870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f55900098f0 tx=0x7f5590009c00 comp rx=0 tx=0).stop 2023-11-27T15:24:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.090+0000 7f55a2dff700 1 -- 172.21.15.133:0/594837420 shutdown_connections 2023-11-27T15:24:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.090+0000 7f55a2dff700 1 --2- 172.21.15.133:0/594837420 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c103fe0 0x7f559c10a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.090+0000 7f55a2dff700 1 --2- 172.21.15.133:0/594837420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559c102460 0x7f559c102870 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.090+0000 7f55a2dff700 1 --2- 172.21.15.133:0/594837420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f559c103650 0x7f559c103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.094 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.090+0000 7f55a2dff700 1 -- 172.21.15.133:0/594837420 >> 172.21.15.133:0/594837420 conn(0x7f559c0fda80 msgr2=0x7f559c0ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.090+0000 7f55a2dff700 1 -- 172.21.15.133:0/594837420 shutdown_connections 2023-11-27T15:24:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.090+0000 7f55a2dff700 1 -- 172.21.15.133:0/594837420 wait complete. 2023-11-27T15:24:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.092+0000 7f55a2dff700 1 Processor -- start 2023-11-27T15:24:34.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.092+0000 7f55a2dff700 1 -- start start 2023-11-27T15:24:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.092+0000 7f55a2dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f559c102460 0x7f559c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.092+0000 7f55a2dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c103650 0x7f559c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.092+0000 7f55a2dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559c103fe0 0x7f559c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.092+0000 7f55a2dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c19fdf0 con 0x7f559c103650 2023-11-27T15:24:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.093+0000 7f55a2dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c19ff30 con 0x7f559c103fe0 2023-11-27T15:24:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.093+0000 7f55a2dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c1a00c0 con 0x7f559c102460 2023-11-27T15:24:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.093+0000 7f55a0b9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f559c102460 0x7f559c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.093+0000 7f559bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c103650 0x7f559c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:34.098 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.093+0000 7f55a0b9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f559c102460 0x7f559c198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:33202/0 (socket says 172.21.15.133:33202) 2023-11-27T15:24:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.093+0000 7f55a0b9b700 1 -- 172.21.15.133:0/3992452076 learned_addr learned my addr 172.21.15.133:0/3992452076 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.093+0000 7f55a139c700 1 --2- 172.21.15.133:0/3992452076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559c103fe0 0x7f559c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.093+0000 7f55a0b9b700 1 -- 172.21.15.133:0/3992452076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559c103fe0 msgr2=0x7f559c19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.093+0000 7f55a0b9b700 1 --2- 172.21.15.133:0/3992452076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559c103fe0 0x7f559c19f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.093+0000 7f55a0b9b700 1 -- 172.21.15.133:0/3992452076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c103650 msgr2=0x7f559c1987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.093+0000 7f55a0b9b700 1 --2- 172.21.15.133:0/3992452076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c103650 0x7f559c1987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.093+0000 7f55a0b9b700 1 -- 172.21.15.133:0/3992452076 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5590009520 con 0x7f559c102460 2023-11-27T15:24:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.093+0000 7f55a139c700 1 --2- 172.21.15.133:0/3992452076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559c103fe0 0x7f559c19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.094+0000 7f55a0b9b700 1 --2- 172.21.15.133:0/3992452076 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f559c102460 0x7f559c198270 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5590005e90 tx=0x7f559000b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:34.099 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.094+0000 7f5599ffb700 1 -- 172.21.15.133:0/3992452076 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559001c070 con 0x7f559c102460 2023-11-27T15:24:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.094+0000 7f5599ffb700 1 -- 172.21.15.133:0/3992452076 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f559000f7f0 con 0x7f559c102460 2023-11-27T15:24:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.094+0000 7f55a2dff700 1 -- 172.21.15.133:0/3992452076 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f559c10de10 con 0x7f559c102460 2023-11-27T15:24:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.094+0000 7f5599ffb700 1 -- 172.21.15.133:0/3992452076 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5590003b00 con 0x7f559c102460 2023-11-27T15:24:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.094+0000 7f55a2dff700 1 -- 172.21.15.133:0/3992452076 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f559c10dc80 con 0x7f559c102460 2023-11-27T15:24:34.100 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.097+0000 7f5599ffb700 1 -- 172.21.15.133:0/3992452076 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5590020030 con 0x7f559c102460 2023-11-27T15:24:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.098+0000 7f5599ffb700 1 --2- 172.21.15.133:0/3992452076 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f558c0a1150 0x7f558c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:34.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.099+0000 7f5599ffb700 1 -- 172.21.15.133:0/3992452076 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f55900cdc90 con 0x7f559c102460 2023-11-27T15:24:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.099+0000 7f559bfff700 1 --2- 172.21.15.133:0/3992452076 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f558c0a1150 0x7f558c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:34.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.099+0000 7f559bfff700 1 --2- 172.21.15.133:0/3992452076 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f558c0a1150 0x7f558c0a3600 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5588009c60 tx=0x7f5588009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.190+0000 7f55b6748b80 1 Processor -- start 2023-11-27T15:24:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.190+0000 7f55b6748b80 1 -- start start 2023-11-27T15:24:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.191+0000 7f55b6748b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556205be6b0 0x555620762870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.191+0000 7f55b6748b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555620762db0 0x5556207635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.191+0000 7f55b6748b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555620763b10 0x555620763f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.191+0000 7f55b6748b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555620668000 con 0x555620763b10 2023-11-27T15:24:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.191+0000 7f55b6748b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556206ba840 con 0x555620762db0 2023-11-27T15:24:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.191+0000 7f55b6748b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556206b3cd0 con 0x5556205be6b0 2023-11-27T15:24:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.191+0000 7f55a0b9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556205be6b0 0x555620762870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.191+0000 7f55a139c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555620763b10 0x555620763f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.191+0000 7f559bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555620762db0 0x5556207635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.191+0000 7f55a0b9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556205be6b0 0x555620762870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:33216/0 (socket says 172.21.15.133:33216) 2023-11-27T15:24:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.191+0000 7f55a0b9b700 1 -- 172.21.15.133:0/3846897409 learned_addr learned my addr 172.21.15.133:0/3846897409 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.192+0000 7f55a0b9b700 1 -- 172.21.15.133:0/3846897409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555620762db0 msgr2=0x5556207635d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.192+0000 7f55a0b9b700 1 --2- 172.21.15.133:0/3846897409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555620762db0 0x5556207635d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.192+0000 7f55a0b9b700 1 -- 172.21.15.133:0/3846897409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555620763b10 msgr2=0x555620763f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.192+0000 7f55a0b9b700 1 --2- 172.21.15.133:0/3846897409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555620763b10 0x555620763f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.192+0000 7f55a0b9b700 1 -- 172.21.15.133:0/3846897409 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556205355d0 con 0x5556205be6b0 2023-11-27T15:24:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.192+0000 7f55a0b9b700 1 --2- 172.21.15.133:0/3846897409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556205be6b0 0x555620762870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f55900a2c10 tx=0x7f559009ab50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a94f3bd47c9c07 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.192+0000 7f5586ffd700 1 -- 172.21.15.133:0/3846897409 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559000bb60 con 0x5556205be6b0 2023-11-27T15:24:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.192+0000 7f5586ffd700 1 -- 172.21.15.133:0/3846897409 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f559009ad80 con 0x5556205be6b0 2023-11-27T15:24:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.192+0000 7f5586ffd700 1 -- 172.21.15.133:0/3846897409 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5590003c60 con 0x5556205be6b0 2023-11-27T15:24:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.193+0000 7f55b6748b80 1 -- 172.21.15.133:0/3846897409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556205be6b0 msgr2=0x555620762870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.193+0000 7f55b6748b80 1 --2- 172.21.15.133:0/3846897409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556205be6b0 0x555620762870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f55900a2c10 tx=0x7f559009ab50 comp rx=0 tx=0).stop 2023-11-27T15:24:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.193+0000 7f55b6748b80 1 -- 172.21.15.133:0/3846897409 shutdown_connections 2023-11-27T15:24:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.193+0000 7f55b6748b80 1 --2- 172.21.15.133:0/3846897409 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555620763b10 0x555620763f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.193+0000 7f55b6748b80 1 --2- 172.21.15.133:0/3846897409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556205be6b0 0x555620762870 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.193+0000 7f55b6748b80 1 --2- 172.21.15.133:0/3846897409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555620762db0 0x5556207635d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.193+0000 7f55b6748b80 1 -- 172.21.15.133:0/3846897409 >> 172.21.15.133:0/3846897409 conn(0x555620757f50 msgr2=0x55562075d100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.193+0000 7f55b6748b80 1 -- 172.21.15.133:0/3846897409 shutdown_connections 2023-11-27T15:24:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.193+0000 7f55b6748b80 1 -- 172.21.15.133:0/3846897409 wait complete. 2023-11-27T15:24:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.194+0000 7f55b6748b80 1 -- start start 2023-11-27T15:24:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.194+0000 7f55b6748b80 1 Processor -- start 2023-11-27T15:24:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.195+0000 7f55b6748b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556205be6b0 0x55562076b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.195+0000 7f55b6748b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555620762db0 0x55562076be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.195+0000 7f55b6748b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555620763b10 0x555620772eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.195+0000 7f55b6748b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555620655880 con 0x555620763b10 2023-11-27T15:24:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.195+0000 7f55b6748b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x555620655520 con 0x5556205be6b0 2023-11-27T15:24:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.195+0000 7f55b6748b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556205378c0 con 0x555620762db0 2023-11-27T15:24:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.195+0000 7f55b6748b80 10 client.0 _ll_register_callbacks cb 0x555620673720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.195+0000 7f559bfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555620762db0 0x55562076be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.196+0000 7f55a0b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556205be6b0 0x55562076b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.196+0000 7f55a139c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555620763b10 0x555620772eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.196+0000 7f559bfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555620762db0 0x55562076be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:33228/0 (socket says 172.21.15.133:33228) 2023-11-27T15:24:34.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.196+0000 7f559bfff700 1 -- 172.21.15.133:0/1861921952 learned_addr learned my addr 172.21.15.133:0/1861921952 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.196+0000 7f55a0b9b700 1 -- 172.21.15.133:0/1861921952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555620762db0 msgr2=0x55562076be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.196+0000 7f55a0b9b700 1 --2- 172.21.15.133:0/1861921952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555620762db0 0x55562076be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.196+0000 7f55a0b9b700 1 -- 172.21.15.133:0/1861921952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555620763b10 msgr2=0x555620772eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.196+0000 7f55a0b9b700 1 --2- 172.21.15.133:0/1861921952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555620763b10 0x555620772eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.196+0000 7f55a0b9b700 1 -- 172.21.15.133:0/1861921952 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f559c10dc80 con 0x5556205be6b0 2023-11-27T15:24:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.196+0000 7f55a0b9b700 1 --2- 172.21.15.133:0/1861921952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556205be6b0 0x55562076b930 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f55900098c0 tx=0x7f559000bbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4e4a9b22cf2e060 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.196+0000 7f557bfff700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:24:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.196+0000 7f557bfff700 1 -- 172.21.15.133:0/1861921952 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559009edb0 con 0x5556205be6b0 2023-11-27T15:24:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.196+0000 7f557bfff700 1 -- 172.21.15.133:0/1861921952 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f5590003c60 con 0x5556205be6b0 2023-11-27T15:24:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.196+0000 7f55b6748b80 1 -- 172.21.15.133:0/1861921952 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555620755710 con 0x5556205be6b0 2023-11-27T15:24:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.197+0000 7f557bfff700 1 -- 172.21.15.133:0/1861921952 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5590003e20 con 0x5556205be6b0 2023-11-27T15:24:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.197+0000 7f55b6748b80 10 client.25832 fetch_fsmap learned FSMap version 23 2023-11-27T15:24:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.197+0000 7f55b6748b80 1 -- 172.21.15.133:0/1861921952 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55562067a130 con 0x5556205be6b0 2023-11-27T15:24:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.197+0000 7f557bfff700 1 -- 172.21.15.133:0/1861921952 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555620755710 con 0x5556205be6b0 2023-11-27T15:24:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.197+0000 7f557bfff700 1 -- 172.21.15.133:0/1861921952 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f559009a270 con 0x5556205be6b0 2023-11-27T15:24:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.197+0000 7f55b6748b80 10 client.25832 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:24:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.197+0000 7f55b6748b80 10 client.25832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.197+0000 7f55b6748b80 20 client.25832 populate_metadata read hostname 'smithi133' 2023-11-27T15:24:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.197+0000 7f55b6748b80 1 --2- 172.21.15.133:0/1861921952 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5556207771e0 0x5556207795f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.197+0000 7f55b6748b80 4 client.25832 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.197+0000 7f55b6748b80 1 -- 172.21.15.133:0/1861921952 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555620676040 con 0x5556207771e0 2023-11-27T15:24:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.198+0000 7f559bfff700 1 --2- 172.21.15.133:0/1861921952 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5556207771e0 0x5556207795f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:34.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.198+0000 7f557bfff700 10 client.25832 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.198+0000 7f559bfff700 1 --2- 172.21.15.133:0/1861921952 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5556207771e0 0x5556207795f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=277d72889be2f64 server_cookie=3b8a83c297779fbe in_seq=0 out_seq=0 2023-11-27T15:24:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.199+0000 7f557bfff700 1 -- 172.21.15.133:0/1861921952 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555620676040 con 0x5556207771e0 2023-11-27T15:24:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.199+0000 7f557bfff700 10 client.25832 handle_command_reply: tid=0 2023-11-27T15:24:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.199+0000 7f55b6748b80 1 client.25832 shutdown 2023-11-27T15:24:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.199+0000 7f55b6748b80 1 -- 172.21.15.133:0/1861921952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556205be6b0 msgr2=0x55562076b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.199+0000 7f55b6748b80 1 --2- 172.21.15.133:0/1861921952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556205be6b0 0x55562076b930 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f55900098c0 tx=0x7f559000bbe0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.199+0000 7f55b6748b80 1 -- 172.21.15.133:0/1861921952 shutdown_connections 2023-11-27T15:24:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.199+0000 7f55b6748b80 1 --2- 172.21.15.133:0/1861921952 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5556207771e0 0x5556207795f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.199+0000 7f55b6748b80 1 --2- 172.21.15.133:0/1861921952 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555620763b10 0x555620772eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.199+0000 7f55b6748b80 1 --2- 172.21.15.133:0/1861921952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556205be6b0 0x55562076b930 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.199+0000 7f557bfff700 0 client.25832 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.199+0000 7f55b6748b80 1 --2- 172.21.15.133:0/1861921952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555620762db0 0x55562076be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.199+0000 7f55b6748b80 1 -- 172.21.15.133:0/1861921952 >> 172.21.15.133:0/1861921952 conn(0x555620757f50 msgr2=0x55562075c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.199+0000 7f55b6748b80 1 -- 172.21.15.133:0/1861921952 shutdown_connections 2023-11-27T15:24:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.199+0000 7f55b6748b80 1 -- 172.21.15.133:0/1861921952 wait complete. 2023-11-27T15:24:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.200+0000 7f55b6748b80 20 client.25832 trim_cache size 0 max 16384 2023-11-27T15:24:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.215+0000 7f55b6748b80 1 Processor -- start 2023-11-27T15:24:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.215+0000 7f55b6748b80 1 -- start start 2023-11-27T15:24:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.215+0000 7f55b6748b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556205be6b0 0x55562076b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.215+0000 7f55b6748b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555620762db0 0x555620772e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.215+0000 7f55b6748b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555620763b10 0x55562076bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.215+0000 7f55b6748b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555620757b70 con 0x5556205be6b0 2023-11-27T15:24:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.216+0000 7f55b6748b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556207673f0 con 0x555620762db0 2023-11-27T15:24:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.216+0000 7f55b6748b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55562075daa0 con 0x555620763b10 2023-11-27T15:24:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.216+0000 7f55a0b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556205be6b0 0x55562076b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.216+0000 7f55a0b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556205be6b0 0x55562076b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33556/0 (socket says 172.21.15.133:33556) 2023-11-27T15:24:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.216+0000 7f55a0b9b700 1 -- 172.21.15.133:0/330128242 learned_addr learned my addr 172.21.15.133:0/330128242 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.216+0000 7f55a139c700 1 --2- 172.21.15.133:0/330128242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555620763b10 0x55562076bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.216+0000 7f559bfff700 1 --2- 172.21.15.133:0/330128242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555620762db0 0x555620772e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.216+0000 7f55a0b9b700 1 -- 172.21.15.133:0/330128242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555620763b10 msgr2=0x55562076bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.216+0000 7f55a0b9b700 1 --2- 172.21.15.133:0/330128242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555620763b10 0x55562076bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.216+0000 7f55a0b9b700 1 -- 172.21.15.133:0/330128242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555620762db0 msgr2=0x555620772e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.216+0000 7f55a0b9b700 1 --2- 172.21.15.133:0/330128242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555620762db0 0x555620772e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.216+0000 7f55a0b9b700 1 -- 172.21.15.133:0/330128242 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555620679e80 con 0x5556205be6b0 2023-11-27T15:24:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.216+0000 7f55a139c700 1 --2- 172.21.15.133:0/330128242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555620763b10 0x55562076bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.216+0000 7f559bfff700 1 --2- 172.21.15.133:0/330128242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555620762db0 0x555620772e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:24:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.216+0000 7f55a0b9b700 1 --2- 172.21.15.133:0/330128242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556205be6b0 0x55562076b6e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5590097280 tx=0x7f55900056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6608c8a1fd60627 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.217+0000 7f55877fe700 1 -- 172.21.15.133:0/330128242 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5590099200 con 0x5556205be6b0 2023-11-27T15:24:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.217+0000 7f55877fe700 1 -- 172.21.15.133:0/330128242 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f559009edb0 con 0x5556205be6b0 2023-11-27T15:24:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.217+0000 7f55877fe700 1 -- 172.21.15.133:0/330128242 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5590097490 con 0x5556205be6b0 2023-11-27T15:24:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.217+0000 7f55b6748b80 1 -- 172.21.15.133:0/330128242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556205be6b0 msgr2=0x55562076b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.217+0000 7f55b6748b80 1 --2- 172.21.15.133:0/330128242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556205be6b0 0x55562076b6e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5590097280 tx=0x7f55900056e0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.217+0000 7f55b6748b80 1 -- 172.21.15.133:0/330128242 shutdown_connections 2023-11-27T15:24:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.217+0000 7f55b6748b80 1 --2- 172.21.15.133:0/330128242 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555620763b10 0x55562076bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.217+0000 7f55b6748b80 1 --2- 172.21.15.133:0/330128242 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556205be6b0 0x55562076b6e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.217+0000 7f55b6748b80 1 --2- 172.21.15.133:0/330128242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555620762db0 0x555620772e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.217+0000 7f55b6748b80 1 -- 172.21.15.133:0/330128242 >> 172.21.15.133:0/330128242 conn(0x555620757f50 msgr2=0x55562077df50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.217+0000 7f55b6748b80 1 -- 172.21.15.133:0/330128242 shutdown_connections 2023-11-27T15:24:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.218+0000 7f55b6748b80 1 -- 172.21.15.133:0/330128242 wait complete. 2023-11-27T15:24:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.218+0000 7f55b6748b80 1 -- start start 2023-11-27T15:24:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.218+0000 7f55b6748b80 1 Processor -- start 2023-11-27T15:24:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.219+0000 7f55b6748b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556205be6b0 0x55562076f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.219+0000 7f55b6748b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555620762db0 0x55562076f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.219+0000 7f55b6748b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555620763b10 0x555620772090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.219+0000 7f55b6748b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556207725d0 con 0x5556205be6b0 2023-11-27T15:24:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.219+0000 7f55b6748b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x555620772740 con 0x555620763b10 2023-11-27T15:24:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.219+0000 7f55b6748b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x555620772900 con 0x555620762db0 2023-11-27T15:24:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.219+0000 7f55b6748b80 10 client.0 _ll_register_callbacks cb 0x7f5594009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.219+0000 7f55a0b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556205be6b0 0x55562076f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.219+0000 7f559bfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555620762db0 0x55562076f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.219+0000 7f55a0b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556205be6b0 0x55562076f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:33564/0 (socket says 172.21.15.133:33564) 2023-11-27T15:24:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.219+0000 7f55a0b9b700 1 -- 172.21.15.133:0/2327046343 learned_addr learned my addr 172.21.15.133:0/2327046343 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.219+0000 7f55a139c700 1 --2- 172.21.15.133:0/2327046343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555620763b10 0x555620772090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.220+0000 7f559bfff700 1 -- 172.21.15.133:0/2327046343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555620763b10 msgr2=0x555620772090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.220+0000 7f559bfff700 1 --2- 172.21.15.133:0/2327046343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555620763b10 0x555620772090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.220+0000 7f559bfff700 1 -- 172.21.15.133:0/2327046343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556205be6b0 msgr2=0x55562076f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.220+0000 7f559bfff700 1 --2- 172.21.15.133:0/2327046343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556205be6b0 0x55562076f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.220+0000 7f559bfff700 1 -- 172.21.15.133:0/2327046343 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55562067a130 con 0x555620762db0 2023-11-27T15:24:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.220+0000 7f55a0b9b700 1 --2- 172.21.15.133:0/2327046343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556205be6b0 0x55562076f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.220+0000 7f559bfff700 1 --2- 172.21.15.133:0/2327046343 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555620762db0 0x55562076f540 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x5556207775b0 tx=0x7f55880047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4faeef7d8d51495 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.220+0000 7f557bfff700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:24:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.220+0000 7f557bfff700 1 -- 172.21.15.133:0/2327046343 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588022040 con 0x555620762db0 2023-11-27T15:24:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.220+0000 7f557bfff700 1 -- 172.21.15.133:0/2327046343 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f558802b5b0 con 0x555620762db0 2023-11-27T15:24:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.220+0000 7f55b6748b80 1 -- 172.21.15.133:0/2327046343 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f559009ad80 con 0x555620762db0 2023-11-27T15:24:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.221+0000 7f557bfff700 1 -- 172.21.15.133:0/2327046343 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f558801d7f0 con 0x555620762db0 2023-11-27T15:24:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.221+0000 7f55b6748b80 10 client.25828 fetch_fsmap learned FSMap version 23 2023-11-27T15:24:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.221+0000 7f55b6748b80 1 -- 172.21.15.133:0/2327046343 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5556204cac00 con 0x555620762db0 2023-11-27T15:24:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.221+0000 7f557bfff700 1 -- 172.21.15.133:0/2327046343 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f559009ad80 con 0x555620762db0 2023-11-27T15:24:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.221+0000 7f557bfff700 1 -- 172.21.15.133:0/2327046343 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f55880348f0 con 0x555620762db0 2023-11-27T15:24:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.221+0000 7f55b6748b80 10 client.25828 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:24:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.221+0000 7f55b6748b80 10 client.25828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.221+0000 7f55b6748b80 20 client.25828 populate_metadata read hostname 'smithi133' 2023-11-27T15:24:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.221+0000 7f55b6748b80 1 --2- 172.21.15.133:0/2327046343 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5556207cbdf0 0x5556207ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.221+0000 7f55b6748b80 4 client.25828 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:24:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.221+0000 7f55b6748b80 1 -- 172.21.15.133:0/2327046343 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555620682600 con 0x5556207cbdf0 2023-11-27T15:24:34.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.222+0000 7f55a0b9b700 1 --2- 172.21.15.133:0/2327046343 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5556207cbdf0 0x5556207ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.222+0000 7f55a0b9b700 1 --2- 172.21.15.133:0/2327046343 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5556207cbdf0 0x5556207ce1d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae619de9dc0923b4 server_cookie=a70ab461c368c137 in_seq=0 out_seq=0 2023-11-27T15:24:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.222+0000 7f557bfff700 10 client.25828 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.244+0000 7f557bfff700 1 -- 172.21.15.133:0/2327046343 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555620682600 con 0x5556207cbdf0 2023-11-27T15:24:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.244+0000 7f557bfff700 10 client.25828 handle_command_reply: tid=0 2023-11-27T15:24:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.244+0000 7f55b6748b80 1 client.25828 shutdown 2023-11-27T15:24:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.244+0000 7f55b6748b80 1 -- 172.21.15.133:0/2327046343 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555620762db0 msgr2=0x55562076f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.244+0000 7f55b6748b80 1 --2- 172.21.15.133:0/2327046343 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555620762db0 0x55562076f540 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x5556207775b0 tx=0x7f55880047c0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.244+0000 7f55b6748b80 1 -- 172.21.15.133:0/2327046343 shutdown_connections 2023-11-27T15:24:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.244+0000 7f55b6748b80 1 --2- 172.21.15.133:0/2327046343 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5556207cbdf0 0x5556207ce1d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.244+0000 7f55b6748b80 1 --2- 172.21.15.133:0/2327046343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555620763b10 0x555620772090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.244+0000 7f557bfff700 0 client.25828 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.244+0000 7f55b6748b80 1 --2- 172.21.15.133:0/2327046343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556205be6b0 0x55562076f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.244+0000 7f55b6748b80 1 --2- 172.21.15.133:0/2327046343 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555620762db0 0x55562076f540 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.244+0000 7f55b6748b80 1 -- 172.21.15.133:0/2327046343 >> 172.21.15.133:0/2327046343 conn(0x555620757f50 msgr2=0x55562076b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.244+0000 7f55b6748b80 1 -- 172.21.15.133:0/2327046343 shutdown_connections 2023-11-27T15:24:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.245+0000 7f55b6748b80 1 -- 172.21.15.133:0/2327046343 wait complete. 2023-11-27T15:24:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.245+0000 7f55b6748b80 20 client.25828 trim_cache size 0 max 16384 2023-11-27T15:24:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:24:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:24:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:24:34.250 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:24:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.245+0000 7f55a2dff700 1 -- 172.21.15.133:0/3992452076 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f558c0a1150 msgr2=0x7f558c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.245+0000 7f55a2dff700 1 --2- 172.21.15.133:0/3992452076 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f558c0a1150 0x7f558c0a3600 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5588009c60 tx=0x7f5588009260 comp rx=0 tx=0).stop 2023-11-27T15:24:34.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.245+0000 7f55a2dff700 1 -- 172.21.15.133:0/3992452076 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f559c102460 msgr2=0x7f559c198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.245+0000 7f55a2dff700 1 --2- 172.21.15.133:0/3992452076 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f559c102460 0x7f559c198270 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5590005e90 tx=0x7f559000b730 comp rx=0 tx=0).stop 2023-11-27T15:24:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.245+0000 7f55a2dff700 1 -- 172.21.15.133:0/3992452076 shutdown_connections 2023-11-27T15:24:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.245+0000 7f55a2dff700 1 --2- 172.21.15.133:0/3992452076 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f558c0a1150 0x7f558c0a3600 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.245+0000 7f55a2dff700 1 --2- 172.21.15.133:0/3992452076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559c103fe0 0x7f559c19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.245+0000 7f55a2dff700 1 --2- 172.21.15.133:0/3992452076 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f559c102460 0x7f559c198270 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.245+0000 7f55a2dff700 1 --2- 172.21.15.133:0/3992452076 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f559c103650 0x7f559c1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.245+0000 7f55a2dff700 1 -- 172.21.15.133:0/3992452076 >> 172.21.15.133:0/3992452076 conn(0x7f559c0fda80 msgr2=0x7f559c106bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.246+0000 7f55a2dff700 1 -- 172.21.15.133:0/3992452076 shutdown_connections 2023-11-27T15:24:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:34.246+0000 7f55a2dff700 1 -- 172.21.15.133:0/3992452076 wait complete. 2023-11-27T15:24:34.468 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:34.469 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:34.563 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:34.563 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:34.586 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:34.586 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:34.586 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 62 -lt 100 ']' 2023-11-27T15:24:34.586 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 62 + 1 2023-11-27T15:24:34.606 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=63 2023-11-27T15:24:34.606 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:34.635 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:34.635 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:34.637 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:34.638 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:34.638 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:35.135 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:35.135 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:35.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:34 smithi174 ceph-mon[131110]: pgmap v398: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 9 op/s 2023-11-27T15:24:35.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:34 smithi133 ceph-mon[97860]: pgmap v398: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 9 op/s 2023-11-27T15:24:35.288 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:35.288 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:35.302 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:35.302 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:35.302 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 63 -lt 100 ']' 2023-11-27T15:24:35.302 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 63 + 1 2023-11-27T15:24:35.322 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=64 2023-11-27T15:24:35.322 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:35.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:34 smithi177 ceph-mon[131504]: pgmap v398: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 9 op/s 2023-11-27T15:24:35.351 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:35.352 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:35.354 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:35.355 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:35.355 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:35.885 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:35.886 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:36.071 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:36.071 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:36.086 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:36.086 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:36.086 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 64 -lt 100 ']' 2023-11-27T15:24:36.086 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 64 + 1 2023-11-27T15:24:36.106 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=65 2023-11-27T15:24:36.106 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:36.135 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:36.135 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:36.137 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:36.138 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:36.138 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:36.619 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:36.619 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:36.780 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:36.780 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:36.801 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:36.801 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:36.801 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 65 -lt 100 ']' 2023-11-27T15:24:36.802 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 65 + 1 2023-11-27T15:24:36.822 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=66 2023-11-27T15:24:36.822 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:36.850 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:36.851 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:36.853 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:36.854 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:36.854 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:37.135 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:36 smithi133 ceph-mon[97860]: pgmap v399: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-11-27T15:24:37.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:36 smithi174 ceph-mon[131110]: pgmap v399: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-11-27T15:24:37.268 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:37.269 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:37.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:36 smithi177 ceph-mon[131504]: pgmap v399: 129 pgs: 129 active+clean; 1.3 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 10 op/s 2023-11-27T15:24:37.404 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:37.404 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:37.426 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:37.426 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:37.426 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 66 -lt 100 ']' 2023-11-27T15:24:37.427 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 66 + 1 2023-11-27T15:24:37.460 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=67 2023-11-27T15:24:37.460 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:37.490 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:37.490 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:37.493 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:37.493 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:37.494 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:38.051 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:38.052 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:38.238 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:38.238 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:38.259 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:38.259 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:38.259 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 67 -lt 100 ']' 2023-11-27T15:24:38.260 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 67 + 1 2023-11-27T15:24:38.280 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=68 2023-11-27T15:24:38.280 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:38.309 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:38.309 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:38.312 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:38.312 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:38.313 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:38.810 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:38.810 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:38.946 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:38.946 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:38.989 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:38.989 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:38.990 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 68 -lt 100 ']' 2023-11-27T15:24:38.990 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 68 + 1 2023-11-27T15:24:39.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:38 smithi133 ceph-mon[97860]: pgmap v400: 129 pgs: 129 active+clean; 1.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-11-27T15:24:39.027 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=69 2023-11-27T15:24:39.027 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:39.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:38 smithi177 ceph-mon[131504]: pgmap v400: 129 pgs: 129 active+clean; 1.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-11-27T15:24:39.090 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:39.091 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:39.093 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:39.094 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:39.094 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:39.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:38 smithi174 ceph-mon[131110]: pgmap v400: 129 pgs: 129 active+clean; 1.4 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-11-27T15:24:39.501 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:39.501 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:39.637 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:39.637 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:39.651 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:39.651 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:39.651 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 69 -lt 100 ']' 2023-11-27T15:24:39.651 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 69 + 1 2023-11-27T15:24:39.671 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=70 2023-11-27T15:24:39.671 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:39.700 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:39.701 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:39.703 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:39.704 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:39.704 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:40.410 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:40.410 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:40.562 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:40.563 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:40.581 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:40.582 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:40.582 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 70 -lt 100 ']' 2023-11-27T15:24:40.582 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 70 + 1 2023-11-27T15:24:40.597 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=71 2023-11-27T15:24:40.597 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:40.627 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:40.628 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:40.631 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:40.632 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:40.632 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:41.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:40 smithi177 ceph-mon[131504]: pgmap v401: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2023-11-27T15:24:41.143 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:41.144 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:41.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:40 smithi174 ceph-mon[131110]: pgmap v401: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2023-11-27T15:24:41.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:40 smithi133 ceph-mon[97860]: pgmap v401: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2023-11-27T15:24:41.287 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:41.287 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:41.300 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:41.301 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:41.301 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 71 -lt 100 ']' 2023-11-27T15:24:41.301 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 71 + 1 2023-11-27T15:24:41.321 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=72 2023-11-27T15:24:41.322 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:41.350 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:41.350 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:41.353 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:41.354 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:41.354 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:41.785 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:41.785 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:42.108 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:42.108 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:42.125 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:42.126 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:42.126 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 72 -lt 100 ']' 2023-11-27T15:24:42.126 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 72 + 1 2023-11-27T15:24:42.146 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=73 2023-11-27T15:24:42.146 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:42.176 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:42.176 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:42.179 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:42.180 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:42.180 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:42.834 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:42.834 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:42.877 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:42.877 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:42.891 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:42.891 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:42.892 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 73 -lt 100 ']' 2023-11-27T15:24:42.892 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 73 + 1 2023-11-27T15:24:42.903 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=74 2023-11-27T15:24:42.903 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:42.931 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:42.931 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:42.934 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:42.935 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:42.935 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:43.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:42 smithi133 ceph-mon[97860]: pgmap v402: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2023-11-27T15:24:43.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:42 smithi174 ceph-mon[131110]: pgmap v402: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2023-11-27T15:24:43.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:42 smithi177 ceph-mon[131504]: pgmap v402: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2023-11-27T15:24:43.417 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:43.418 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:43.545 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:43.545 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:43.559 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:43.559 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:43.559 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 74 -lt 100 ']' 2023-11-27T15:24:43.560 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 74 + 1 2023-11-27T15:24:43.579 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=75 2023-11-27T15:24:43.579 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:43.615 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:43.615 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:43.618 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:43.618 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:43.619 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:44.176 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:44.176 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:44.352 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:44.353 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:44.366 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:44.366 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:44.367 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 75 -lt 100 ']' 2023-11-27T15:24:44.367 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 75 + 1 2023-11-27T15:24:44.401 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=76 2023-11-27T15:24:44.401 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:44.479 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:44.480 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:44.483 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:44.483 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:44.484 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:44.959 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:44.959 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:45.095 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:45.095 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:45.109 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:45.109 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:45.109 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 76 -lt 100 ']' 2023-11-27T15:24:45.110 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 76 + 1 2023-11-27T15:24:45.129 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=77 2023-11-27T15:24:45.129 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:45.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:44 smithi174 ceph-mon[131110]: pgmap v403: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-11-27T15:24:45.158 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:45.158 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:45.161 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:45.162 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:45.162 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:45.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:44 smithi133 ceph-mon[97860]: pgmap v403: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-11-27T15:24:45.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:44 smithi177 ceph-mon[131504]: pgmap v403: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 13 op/s 2023-11-27T15:24:45.642 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:45.642 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:45.786 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:45.786 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:45.800 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:45.800 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:45.800 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 77 -lt 100 ']' 2023-11-27T15:24:45.801 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 77 + 1 2023-11-27T15:24:45.821 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=78 2023-11-27T15:24:45.821 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:45.851 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:45.851 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:45.854 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:45.854 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:45.855 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:46.342 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:46.342 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:46.478 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:46.478 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:46.509 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:46.509 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:46.509 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 78 -lt 100 ']' 2023-11-27T15:24:46.509 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 78 + 1 2023-11-27T15:24:46.551 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=79 2023-11-27T15:24:46.551 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:46.630 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:46.631 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:46.633 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:46.634 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:46.634 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:47.100 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:47.100 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:47.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:46 smithi174 ceph-mon[131110]: pgmap v404: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-11-27T15:24:47.253 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:47.253 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:47.266 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:47.266 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:47.266 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 79 -lt 100 ']' 2023-11-27T15:24:47.267 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 79 + 1 2023-11-27T15:24:47.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:46 smithi133 ceph-mon[97860]: pgmap v404: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-11-27T15:24:47.292 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=80 2023-11-27T15:24:47.292 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:47.316 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:47.316 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:47.319 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:47.320 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:47.320 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:47.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:46 smithi177 ceph-mon[131504]: pgmap v404: 129 pgs: 129 active+clean; 1.4 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-11-27T15:24:47.866 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:47.867 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:48.078 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:48.078 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:48.109 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:48.109 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:48.109 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 80 -lt 100 ']' 2023-11-27T15:24:48.110 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 80 + 1 2023-11-27T15:24:48.142 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=81 2023-11-27T15:24:48.142 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:48.178 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:48.179 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:48.181 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:48.182 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:48.185 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:48.875 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:48.875 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:49.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:48 smithi133 ceph-mon[97860]: pgmap v405: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 14 op/s 2023-11-27T15:24:49.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:48 smithi177 ceph-mon[131504]: pgmap v405: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 14 op/s 2023-11-27T15:24:49.111 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:49.111 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:49.143 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:49.143 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:49.143 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 81 -lt 100 ']' 2023-11-27T15:24:49.143 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 81 + 1 2023-11-27T15:24:49.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:48 smithi174 ceph-mon[131110]: pgmap v405: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 14 op/s 2023-11-27T15:24:49.217 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=82 2023-11-27T15:24:49.217 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:49.249 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:49.250 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:49.253 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:49.253 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:49.254 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:49.850 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:49.850 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.111+0000 7f9917fff700 1 -- 172.21.15.133:0/2474530844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9910100240 msgr2=0x7f9910100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.111+0000 7f9917fff700 1 --2- 172.21.15.133:0/2474530844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9910100240 0x7f9910100650 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f990c0098f0 tx=0x7f990c009c00 comp rx=0 tx=0).stop 2023-11-27T15:24:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.112+0000 7f9917fff700 1 -- 172.21.15.133:0/2474530844 shutdown_connections 2023-11-27T15:24:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.112+0000 7f9917fff700 1 --2- 172.21.15.133:0/2474530844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9910101dc0 0x7f9910108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.112+0000 7f9917fff700 1 --2- 172.21.15.133:0/2474530844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9910100240 0x7f9910100650 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.112+0000 7f9917fff700 1 --2- 172.21.15.133:0/2474530844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9910101430 0x7f9910101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.112+0000 7f9917fff700 1 -- 172.21.15.133:0/2474530844 >> 172.21.15.133:0/2474530844 conn(0x7f99100fb840 msgr2=0x7f99100fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.112+0000 7f9917fff700 1 -- 172.21.15.133:0/2474530844 shutdown_connections 2023-11-27T15:24:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.112+0000 7f9917fff700 1 -- 172.21.15.133:0/2474530844 wait complete. 2023-11-27T15:24:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.113+0000 7f9917fff700 1 Processor -- start 2023-11-27T15:24:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.113+0000 7f9917fff700 1 -- start start 2023-11-27T15:24:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.114+0000 7f9917fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9910100240 0x7f99100748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.114+0000 7f9917fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9910101430 0x7f9910072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.114+0000 7f9917fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9910101dc0 0x7f9910073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.114+0000 7f9917fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9910073bc0 con 0x7f9910101430 2023-11-27T15:24:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.114+0000 7f9917fff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9910073d30 con 0x7f9910101dc0 2023-11-27T15:24:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.114+0000 7f9917fff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99101a0120 con 0x7f9910100240 2023-11-27T15:24:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.114+0000 7f991559a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9910101430 0x7f9910072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.114+0000 7f9915d9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9910100240 0x7f99100748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.114+0000 7f991559a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9910101430 0x7f9910072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46590/0 (socket says 172.21.15.133:46590) 2023-11-27T15:24:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.114+0000 7f991559a700 1 -- 172.21.15.133:0/2471661038 learned_addr learned my addr 172.21.15.133:0/2471661038 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.114+0000 7f991659c700 1 --2- 172.21.15.133:0/2471661038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9910101dc0 0x7f9910073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.114+0000 7f9915d9b700 1 -- 172.21.15.133:0/2471661038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9910101dc0 msgr2=0x7f9910073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.114+0000 7f9915d9b700 1 --2- 172.21.15.133:0/2471661038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9910101dc0 0x7f9910073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.114+0000 7f9915d9b700 1 -- 172.21.15.133:0/2471661038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9910101430 msgr2=0x7f9910072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.114+0000 7f9915d9b700 1 --2- 172.21.15.133:0/2471661038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9910101430 0x7f9910072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.120 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:50.120 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.114+0000 7f9915d9b700 1 -- 172.21.15.133:0/2471661038 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990c009520 con 0x7f9910100240 2023-11-27T15:24:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.115+0000 7f9915d9b700 1 --2- 172.21.15.133:0/2471661038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9910100240 0x7f99100748a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f990c000c00 tx=0x7f990c00b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.115+0000 7f9906ffd700 1 -- 172.21.15.133:0/2471661038 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c01c070 con 0x7f9910100240 2023-11-27T15:24:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.115+0000 7f9906ffd700 1 -- 172.21.15.133:0/2471661038 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f990c00f7d0 con 0x7f9910100240 2023-11-27T15:24:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.115+0000 7f9917fff700 1 -- 172.21.15.133:0/2471661038 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991010bba0 con 0x7f9910100240 2023-11-27T15:24:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.115+0000 7f9906ffd700 1 -- 172.21.15.133:0/2471661038 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c0039b0 con 0x7f9910100240 2023-11-27T15:24:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.115+0000 7f9917fff700 1 -- 172.21.15.133:0/2471661038 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991010ba10 con 0x7f9910100240 2023-11-27T15:24:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.119+0000 7f9906ffd700 1 -- 172.21.15.133:0/2471661038 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f990c020030 con 0x7f9910100240 2023-11-27T15:24:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.120+0000 7f9906ffd700 1 --2- 172.21.15.133:0/2471661038 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f98fc0a11d0 0x7f98fc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.120+0000 7f9906ffd700 1 -- 172.21.15.133:0/2471661038 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f990c0cdb50 con 0x7f9910100240 2023-11-27T15:24:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.120+0000 7f991559a700 1 --2- 172.21.15.133:0/2471661038 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f98fc0a11d0 0x7f98fc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.121+0000 7f991559a700 1 --2- 172.21.15.133:0/2471661038 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f98fc0a11d0 0x7f98fc0a3680 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9900005950 tx=0x7f9900009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:50.140 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:50.141 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:50.141 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 82 -lt 100 ']' 2023-11-27T15:24:50.141 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 82 + 1 2023-11-27T15:24:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.186+0000 7f992b91db80 1 Processor -- start 2023-11-27T15:24:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.187+0000 7f992b91db80 1 -- start start 2023-11-27T15:24:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.187+0000 7f992b91db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dba136b0 0x5592dbbb7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.187+0000 7f992b91db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dbbb7db0 0x5592dbbb85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.187+0000 7f992b91db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dbbb8b10 0x5592dbbb8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.187+0000 7f992b91db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592dbabd000 con 0x5592dba136b0 2023-11-27T15:24:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.187+0000 7f992b91db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592dbb0f840 con 0x5592dbbb7db0 2023-11-27T15:24:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.187+0000 7f992b91db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592dbb08cd0 con 0x5592dbbb8b10 2023-11-27T15:24:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.187+0000 7f9915d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dba136b0 0x5592dbbb7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.188+0000 7f991559a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dbbb7db0 0x5592dbbb85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.188+0000 7f991659c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dbbb8b10 0x5592dbbb8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.188+0000 7f991559a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dbbb7db0 0x5592dbbb85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:54232/0 (socket says 172.21.15.133:54232) 2023-11-27T15:24:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.188+0000 7f991659c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dbbb8b10 0x5592dbbb8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:33842/0 (socket says 172.21.15.133:33842) 2023-11-27T15:24:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.188+0000 7f991559a700 1 -- 172.21.15.133:0/4284629297 learned_addr learned my addr 172.21.15.133:0/4284629297 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.188+0000 7f9915d9b700 1 -- 172.21.15.133:0/4284629297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dbbb8b10 msgr2=0x5592dbbb8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.188+0000 7f9915d9b700 1 --2- 172.21.15.133:0/4284629297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dbbb8b10 0x5592dbbb8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.188+0000 7f9915d9b700 1 -- 172.21.15.133:0/4284629297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dbbb7db0 msgr2=0x5592dbbb85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.188+0000 7f9915d9b700 1 --2- 172.21.15.133:0/4284629297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dbbb7db0 0x5592dbbb85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.188+0000 7f9915d9b700 1 -- 172.21.15.133:0/4284629297 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592db98a5d0 con 0x5592dba136b0 2023-11-27T15:24:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.188+0000 7f991559a700 1 --2- 172.21.15.133:0/4284629297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dbbb7db0 0x5592dbbb85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.188+0000 7f9915d9b700 1 --2- 172.21.15.133:0/4284629297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dba136b0 0x5592dbbb7870 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f990c0cb2a0 tx=0x7f990c00baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4beb28b747d4558 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.189+0000 7f98fbfff700 1 -- 172.21.15.133:0/4284629297 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c0b2070 con 0x5592dba136b0 2023-11-27T15:24:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.189+0000 7f98fbfff700 1 -- 172.21.15.133:0/4284629297 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f990c09e030 con 0x5592dba136b0 2023-11-27T15:24:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.189+0000 7f98fbfff700 1 -- 172.21.15.133:0/4284629297 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c0cb480 con 0x5592dba136b0 2023-11-27T15:24:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.189+0000 7f992b91db80 1 -- 172.21.15.133:0/4284629297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dba136b0 msgr2=0x5592dbbb7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.189+0000 7f992b91db80 1 --2- 172.21.15.133:0/4284629297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dba136b0 0x5592dbbb7870 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f990c0cb2a0 tx=0x7f990c00baf0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.190+0000 7f992b91db80 1 -- 172.21.15.133:0/4284629297 shutdown_connections 2023-11-27T15:24:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.190+0000 7f992b91db80 1 --2- 172.21.15.133:0/4284629297 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dbbb8b10 0x5592dbbb8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.190+0000 7f992b91db80 1 --2- 172.21.15.133:0/4284629297 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dba136b0 0x5592dbbb7870 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.190+0000 7f992b91db80 1 --2- 172.21.15.133:0/4284629297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dbbb7db0 0x5592dbbb85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.190+0000 7f992b91db80 1 -- 172.21.15.133:0/4284629297 >> 172.21.15.133:0/4284629297 conn(0x5592dbbacf50 msgr2=0x5592dbbb2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.190+0000 7f992b91db80 1 -- 172.21.15.133:0/4284629297 shutdown_connections 2023-11-27T15:24:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.190+0000 7f992b91db80 1 -- 172.21.15.133:0/4284629297 wait complete. 2023-11-27T15:24:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.190+0000 7f992b91db80 1 -- start start 2023-11-27T15:24:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.191+0000 7f992b91db80 1 Processor -- start 2023-11-27T15:24:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.192+0000 7f992b91db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dba136b0 0x5592dbbc0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.192+0000 7f992b91db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dbbb7db0 0x5592dbbc0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.192+0000 7f992b91db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dbbb8b10 0x5592dbbc7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.192+0000 7f992b91db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592dbaaa880 con 0x5592dbbb8b10 2023-11-27T15:24:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.192+0000 7f992b91db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592dbaaa520 con 0x5592dba136b0 2023-11-27T15:24:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.192+0000 7f992b91db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592db98c8c0 con 0x5592dbbb7db0 2023-11-27T15:24:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.192+0000 7f992b91db80 10 client.0 _ll_register_callbacks cb 0x5592dbac8720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.192+0000 7f991659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dbbb8b10 0x5592dbbc7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.192+0000 7f991659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dbbb8b10 0x5592dbbc7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46614/0 (socket says 172.21.15.133:46614) 2023-11-27T15:24:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.192+0000 7f9915d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dba136b0 0x5592dbbc0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.192+0000 7f991659c700 1 -- 172.21.15.133:0/1946535328 learned_addr learned my addr 172.21.15.133:0/1946535328 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.192+0000 7f991559a700 1 --2- 172.21.15.133:0/1946535328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dbbb7db0 0x5592dbbc0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.193+0000 7f991659c700 1 -- 172.21.15.133:0/1946535328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dbbb7db0 msgr2=0x5592dbbc0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.193+0000 7f991659c700 1 --2- 172.21.15.133:0/1946535328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dbbb7db0 0x5592dbbc0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.193+0000 7f991659c700 1 -- 172.21.15.133:0/1946535328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dba136b0 msgr2=0x5592dbbc0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.193+0000 7f991659c700 1 --2- 172.21.15.133:0/1946535328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dba136b0 0x5592dbbc0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.193+0000 7f991659c700 1 -- 172.21.15.133:0/1946535328 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991010ba10 con 0x5592dbbb8b10 2023-11-27T15:24:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.193+0000 7f9915d9b700 1 --2- 172.21.15.133:0/1946535328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dba136b0 0x5592dbbc0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.193+0000 7f991559a700 1 --2- 172.21.15.133:0/1946535328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dbbb7db0 0x5592dbbc0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.193+0000 7f991659c700 1 --2- 172.21.15.133:0/1946535328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dbbb8b10 0x5592dbbc7ed0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9908011ef0 tx=0x7f9908014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b7f22532168ee4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.193+0000 7f98f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:24:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.193+0000 7f98f97fa700 1 -- 172.21.15.133:0/1946535328 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990800e040 con 0x5592dbbb8b10 2023-11-27T15:24:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.193+0000 7f98f97fa700 1 -- 172.21.15.133:0/1946535328 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f99080093a0 con 0x5592dbbb8b10 2023-11-27T15:24:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.193+0000 7f992b91db80 1 -- 172.21.15.133:0/1946535328 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592dbbaa710 con 0x5592dbbb8b10 2023-11-27T15:24:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.194+0000 7f98f97fa700 1 -- 172.21.15.133:0/1946535328 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9908014ce0 con 0x5592dbbb8b10 2023-11-27T15:24:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.194+0000 7f992b91db80 10 client.15810 fetch_fsmap learned FSMap version 23 2023-11-27T15:24:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.194+0000 7f992b91db80 1 -- 172.21.15.133:0/1946535328 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5592dbacf130 con 0x5592dbbb8b10 2023-11-27T15:24:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.194+0000 7f98f97fa700 1 -- 172.21.15.133:0/1946535328 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5592dbbaa710 con 0x5592dbbb8b10 2023-11-27T15:24:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.194+0000 7f98f97fa700 1 -- 172.21.15.133:0/1946535328 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f990801c020 con 0x5592dbbb8b10 2023-11-27T15:24:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.194+0000 7f992b91db80 10 client.15810 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:24:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.194+0000 7f992b91db80 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.194+0000 7f992b91db80 20 client.15810 populate_metadata read hostname 'smithi133' 2023-11-27T15:24:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.194+0000 7f992b91db80 1 --2- 172.21.15.133:0/1946535328 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5592dbbcc780 0x5592dbbceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.194+0000 7f992b91db80 4 client.15810 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.194+0000 7f992b91db80 1 -- 172.21.15.133:0/1946535328 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592dbacb040 con 0x5592dbbcc780 2023-11-27T15:24:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.195+0000 7f9915d9b700 1 --2- 172.21.15.133:0/1946535328 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5592dbbcc780 0x5592dbbceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.195+0000 7f9915d9b700 1 --2- 172.21.15.133:0/1946535328 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5592dbbcc780 0x5592dbbceb90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9958ad7634683098 server_cookie=cc0d562fcef89a06 in_seq=0 out_seq=0 2023-11-27T15:24:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.195+0000 7f98f97fa700 10 client.15810 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:50.204 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=83 2023-11-27T15:24:50.204 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.196+0000 7f98f97fa700 1 -- 172.21.15.133:0/1946535328 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5592dbacb040 con 0x5592dbbcc780 2023-11-27T15:24:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.196+0000 7f98f97fa700 10 client.15810 handle_command_reply: tid=0 2023-11-27T15:24:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.196+0000 7f992b91db80 1 client.15810 shutdown 2023-11-27T15:24:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.196+0000 7f992b91db80 1 -- 172.21.15.133:0/1946535328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dbbb8b10 msgr2=0x5592dbbc7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.196+0000 7f992b91db80 1 --2- 172.21.15.133:0/1946535328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dbbb8b10 0x5592dbbc7ed0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9908011ef0 tx=0x7f9908014600 comp rx=0 tx=0).stop 2023-11-27T15:24:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.197+0000 7f992b91db80 1 -- 172.21.15.133:0/1946535328 shutdown_connections 2023-11-27T15:24:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.197+0000 7f992b91db80 1 --2- 172.21.15.133:0/1946535328 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5592dbbcc780 0x5592dbbceb90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.197+0000 7f992b91db80 1 --2- 172.21.15.133:0/1946535328 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dbbb8b10 0x5592dbbc7ed0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.197+0000 7f992b91db80 1 --2- 172.21.15.133:0/1946535328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dba136b0 0x5592dbbc0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.197+0000 7f992b91db80 1 --2- 172.21.15.133:0/1946535328 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dbbb7db0 0x5592dbbc0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.197+0000 7f98f97fa700 0 client.15810 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.197+0000 7f992b91db80 1 -- 172.21.15.133:0/1946535328 >> 172.21.15.133:0/1946535328 conn(0x5592dbbacf50 msgr2=0x5592dbbb17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.197+0000 7f992b91db80 1 -- 172.21.15.133:0/1946535328 shutdown_connections 2023-11-27T15:24:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.197+0000 7f992b91db80 1 -- 172.21.15.133:0/1946535328 wait complete. 2023-11-27T15:24:50.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.197+0000 7f992b91db80 20 client.15810 trim_cache size 0 max 16384 2023-11-27T15:24:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.213+0000 7f992b91db80 1 Processor -- start 2023-11-27T15:24:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.213+0000 7f992b91db80 1 -- start start 2023-11-27T15:24:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.213+0000 7f992b91db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dba136b0 0x5592dbbbed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.213+0000 7f992b91db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dbbb7db0 0x5592dbbc7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.213+0000 7f992b91db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dbbb8b10 0x5592dbbbf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.213+0000 7f992b91db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592dbbc04c0 con 0x5592dbbb8b10 2023-11-27T15:24:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.213+0000 7f992b91db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592dbbc8330 con 0x5592dbbb7db0 2023-11-27T15:24:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.213+0000 7f992b91db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592dbbacb70 con 0x5592dba136b0 2023-11-27T15:24:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.214+0000 7f9915d9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dba136b0 0x5592dbbbed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.214+0000 7f991559a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dbbb7db0 0x5592dbbc7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.214+0000 7f9915d9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dba136b0 0x5592dbbbed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:33848/0 (socket says 172.21.15.133:33848) 2023-11-27T15:24:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.214+0000 7f9915d9b700 1 -- 172.21.15.133:0/881839930 learned_addr learned my addr 172.21.15.133:0/881839930 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.214+0000 7f991659c700 1 --2- 172.21.15.133:0/881839930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dbbb8b10 0x5592dbbbf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.214+0000 7f9915d9b700 1 -- 172.21.15.133:0/881839930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dbbb7db0 msgr2=0x5592dbbc7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.214+0000 7f9915d9b700 1 --2- 172.21.15.133:0/881839930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dbbb7db0 0x5592dbbc7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.214+0000 7f9915d9b700 1 -- 172.21.15.133:0/881839930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dbbb8b10 msgr2=0x5592dbbbf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.214+0000 7f9915d9b700 1 --2- 172.21.15.133:0/881839930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dbbb8b10 0x5592dbbbf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.214+0000 7f9915d9b700 1 -- 172.21.15.133:0/881839930 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592dbacee80 con 0x5592dba136b0 2023-11-27T15:24:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.214+0000 7f991659c700 1 --2- 172.21.15.133:0/881839930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dbbb8b10 0x5592dbbbf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.214+0000 7f9915d9b700 1 --2- 172.21.15.133:0/881839930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dba136b0 0x5592dbbbed30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f990c009810 tx=0x7f990c0a0040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db71077ec585994b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.215+0000 7f9904ff9700 1 -- 172.21.15.133:0/881839930 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c0b2040 con 0x5592dba136b0 2023-11-27T15:24:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.215+0000 7f9904ff9700 1 -- 172.21.15.133:0/881839930 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f990c0ca070 con 0x5592dba136b0 2023-11-27T15:24:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.215+0000 7f9904ff9700 1 -- 172.21.15.133:0/881839930 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c0cb4d0 con 0x5592dba136b0 2023-11-27T15:24:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.215+0000 7f992b91db80 1 -- 172.21.15.133:0/881839930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dba136b0 msgr2=0x5592dbbbed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.215+0000 7f992b91db80 1 --2- 172.21.15.133:0/881839930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dba136b0 0x5592dbbbed30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f990c009810 tx=0x7f990c0a0040 comp rx=0 tx=0).stop 2023-11-27T15:24:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.215+0000 7f992b91db80 1 -- 172.21.15.133:0/881839930 shutdown_connections 2023-11-27T15:24:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.215+0000 7f992b91db80 1 --2- 172.21.15.133:0/881839930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dbbb8b10 0x5592dbbbf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.215+0000 7f992b91db80 1 --2- 172.21.15.133:0/881839930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dba136b0 0x5592dbbbed30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.215+0000 7f992b91db80 1 --2- 172.21.15.133:0/881839930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dbbb7db0 0x5592dbbc7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.216+0000 7f992b91db80 1 -- 172.21.15.133:0/881839930 >> 172.21.15.133:0/881839930 conn(0x5592dbbacf50 msgr2=0x5592dbbc0950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.216+0000 7f992b91db80 1 -- 172.21.15.133:0/881839930 shutdown_connections 2023-11-27T15:24:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.216+0000 7f992b91db80 1 -- 172.21.15.133:0/881839930 wait complete. 2023-11-27T15:24:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.216+0000 7f992b91db80 1 -- start start 2023-11-27T15:24:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.216+0000 7f992b91db80 1 Processor -- start 2023-11-27T15:24:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.217+0000 7f992b91db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dba136b0 0x5592dbbc5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.217+0000 7f992b91db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dbbb7db0 0x5592dbc1c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.217+0000 7f992b91db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dbbb8b10 0x5592dbc1eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.217+0000 7f992b91db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592dbc1f290 con 0x5592dba136b0 2023-11-27T15:24:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.217+0000 7f992b91db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592dbc1f400 con 0x5592dbbb7db0 2023-11-27T15:24:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.217+0000 7f992b91db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592dbc1f5c0 con 0x5592dbbb8b10 2023-11-27T15:24:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.217+0000 7f992b91db80 10 client.0 _ll_register_callbacks cb 0x7f9908011820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.217+0000 7f9915d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dba136b0 0x5592dbbc5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.217+0000 7f9915d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dba136b0 0x5592dbbc5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:46632/0 (socket says 172.21.15.133:46632) 2023-11-27T15:24:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.217+0000 7f9915d9b700 1 -- 172.21.15.133:0/4125068932 learned_addr learned my addr 172.21.15.133:0/4125068932 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.217+0000 7f991559a700 1 --2- 172.21.15.133:0/4125068932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dbbb7db0 0x5592dbc1c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.218+0000 7f9915d9b700 1 -- 172.21.15.133:0/4125068932 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dbbb8b10 msgr2=0x5592dbc1eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.218+0000 7f9915d9b700 1 --2- 172.21.15.133:0/4125068932 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dbbb8b10 0x5592dbc1eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.218+0000 7f9915d9b700 1 -- 172.21.15.133:0/4125068932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dbbb7db0 msgr2=0x5592dbc1c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.218+0000 7f9915d9b700 1 --2- 172.21.15.133:0/4125068932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dbbb7db0 0x5592dbc1c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.218+0000 7f9915d9b700 1 -- 172.21.15.133:0/4125068932 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592db98a5d0 con 0x5592dba136b0 2023-11-27T15:24:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.218+0000 7f9915d9b700 1 --2- 172.21.15.133:0/4125068932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dba136b0 0x5592dbbc5de0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f990c0b8040 tx=0x7f990c00baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd8c11d4b7dc18c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.218+0000 7f98f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:24:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.218+0000 7f98f97fa700 1 -- 172.21.15.133:0/4125068932 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c0b2040 con 0x5592dba136b0 2023-11-27T15:24:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.218+0000 7f98f97fa700 1 -- 172.21.15.133:0/4125068932 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f990c098d80 con 0x5592dba136b0 2023-11-27T15:24:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.218+0000 7f992b91db80 1 -- 172.21.15.133:0/4125068932 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f990c09e030 con 0x5592dba136b0 2023-11-27T15:24:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.218+0000 7f98f97fa700 1 -- 172.21.15.133:0/4125068932 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f990c0cb4b0 con 0x5592dba136b0 2023-11-27T15:24:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.218+0000 7f992b91db80 10 client.15816 fetch_fsmap learned FSMap version 23 2023-11-27T15:24:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.218+0000 7f992b91db80 1 -- 172.21.15.133:0/4125068932 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5592db91fc00 con 0x5592dba136b0 2023-11-27T15:24:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.219+0000 7f98f97fa700 1 -- 172.21.15.133:0/4125068932 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f990c09e030 con 0x5592dba136b0 2023-11-27T15:24:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.219+0000 7f98f97fa700 1 -- 172.21.15.133:0/4125068932 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f990c09aea0 con 0x5592dba136b0 2023-11-27T15:24:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.219+0000 7f992b91db80 10 client.15816 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:24:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.219+0000 7f992b91db80 10 client.15816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.219+0000 7f992b91db80 20 client.15816 populate_metadata read hostname 'smithi133' 2023-11-27T15:24:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.219+0000 7f992b91db80 1 --2- 172.21.15.133:0/4125068932 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5592dbc20b10 0x5592dbc22ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.219+0000 7f992b91db80 4 client.15816 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:24:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.219+0000 7f992b91db80 1 -- 172.21.15.133:0/4125068932 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5592dbad7600 con 0x5592dbc20b10 2023-11-27T15:24:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.219+0000 7f991559a700 1 --2- 172.21.15.133:0/4125068932 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5592dbc20b10 0x5592dbc22ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.220+0000 7f991559a700 1 --2- 172.21.15.133:0/4125068932 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5592dbc20b10 0x5592dbc22ef0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96599f915291fa53 server_cookie=56efa7c34420e8c9 in_seq=0 out_seq=0 2023-11-27T15:24:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.220+0000 7f98f97fa700 10 client.15816 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:50.230 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:50.231 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:50.231 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:50.231 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:50.231 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.247+0000 7f98f97fa700 1 -- 172.21.15.133:0/4125068932 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5592dbad7600 con 0x5592dbc20b10 2023-11-27T15:24:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.247+0000 7f98f97fa700 10 client.15816 handle_command_reply: tid=0 2023-11-27T15:24:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.247+0000 7f992b91db80 1 client.15816 shutdown 2023-11-27T15:24:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.247+0000 7f992b91db80 1 -- 172.21.15.133:0/4125068932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dba136b0 msgr2=0x5592dbbc5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.247+0000 7f992b91db80 1 --2- 172.21.15.133:0/4125068932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dba136b0 0x5592dbbc5de0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f990c0b8040 tx=0x7f990c00baf0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.247+0000 7f992b91db80 1 -- 172.21.15.133:0/4125068932 shutdown_connections 2023-11-27T15:24:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.247+0000 7f992b91db80 1 --2- 172.21.15.133:0/4125068932 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5592dbc20b10 0x5592dbc22ef0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.247+0000 7f992b91db80 1 --2- 172.21.15.133:0/4125068932 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5592dbbb8b10 0x5592dbc1eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.247+0000 7f992b91db80 1 --2- 172.21.15.133:0/4125068932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5592dba136b0 0x5592dbbc5de0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.247+0000 7f98f97fa700 0 client.15816 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.247+0000 7f992b91db80 1 --2- 172.21.15.133:0/4125068932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592dbbb7db0 0x5592dbc1c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.247+0000 7f992b91db80 1 -- 172.21.15.133:0/4125068932 >> 172.21.15.133:0/4125068932 conn(0x5592dbbacf50 msgr2=0x5592dbbbed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.248+0000 7f992b91db80 1 -- 172.21.15.133:0/4125068932 shutdown_connections 2023-11-27T15:24:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.248+0000 7f992b91db80 1 -- 172.21.15.133:0/4125068932 wait complete. 2023-11-27T15:24:50.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.248+0000 7f992b91db80 20 client.15816 trim_cache size 0 max 16384 2023-11-27T15:24:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:24:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:24:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:24:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:24:50.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.248+0000 7f9917fff700 1 -- 172.21.15.133:0/2471661038 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f98fc0a11d0 msgr2=0x7f98fc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.248+0000 7f9917fff700 1 --2- 172.21.15.133:0/2471661038 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f98fc0a11d0 0x7f98fc0a3680 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9900005950 tx=0x7f9900009310 comp rx=0 tx=0).stop 2023-11-27T15:24:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.248+0000 7f9917fff700 1 -- 172.21.15.133:0/2471661038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9910100240 msgr2=0x7f99100748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.248+0000 7f9917fff700 1 --2- 172.21.15.133:0/2471661038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9910100240 0x7f99100748a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f990c000c00 tx=0x7f990c00b790 comp rx=0 tx=0).stop 2023-11-27T15:24:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.249+0000 7f9917fff700 1 -- 172.21.15.133:0/2471661038 shutdown_connections 2023-11-27T15:24:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.249+0000 7f9917fff700 1 --2- 172.21.15.133:0/2471661038 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f98fc0a11d0 0x7f98fc0a3680 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.249+0000 7f9917fff700 1 --2- 172.21.15.133:0/2471661038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9910101dc0 0x7f9910073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.249+0000 7f9917fff700 1 --2- 172.21.15.133:0/2471661038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9910100240 0x7f99100748a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.249+0000 7f9917fff700 1 --2- 172.21.15.133:0/2471661038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9910101430 0x7f9910072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.249+0000 7f9917fff700 1 -- 172.21.15.133:0/2471661038 >> 172.21.15.133:0/2471661038 conn(0x7f99100fb840 msgr2=0x7f9910104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.249+0000 7f9917fff700 1 -- 172.21.15.133:0/2471661038 shutdown_connections 2023-11-27T15:24:50.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:24:50.249+0000 7f9917fff700 1 -- 172.21.15.133:0/2471661038 wait complete. 2023-11-27T15:24:50.841 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:50.841 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:51.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:50 smithi133 ceph-mon[97860]: pgmap v406: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2023-11-27T15:24:51.040 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:51.041 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:51.048 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:51.049 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:51.049 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 83 -lt 100 ']' 2023-11-27T15:24:51.049 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 83 + 1 2023-11-27T15:24:51.078 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=84 2023-11-27T15:24:51.078 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:51.108 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:51.109 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:51.111 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:51.112 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:51.112 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:51.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:50 smithi174 ceph-mon[131110]: pgmap v406: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2023-11-27T15:24:51.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:50 smithi177 ceph-mon[131504]: pgmap v406: 129 pgs: 129 active+clean; 1.5 MiB data, 88 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 10 op/s 2023-11-27T15:24:51.624 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:51.625 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:51.777 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:51.777 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:51.791 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:51.791 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:51.791 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 84 -lt 100 ']' 2023-11-27T15:24:51.791 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 84 + 1 2023-11-27T15:24:51.811 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=85 2023-11-27T15:24:51.811 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:51.841 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:51.841 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:51.844 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:51.845 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:51.845 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:52.307 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:52.308 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:52.441 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:52.441 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:52.457 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:52.457 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:52.457 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 85 -lt 100 ']' 2023-11-27T15:24:52.457 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 85 + 1 2023-11-27T15:24:52.494 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=86 2023-11-27T15:24:52.494 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:52.524 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:52.524 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:52.527 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:52.528 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:52.528 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:53.091 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:53.091 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:53.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:52 smithi174 ceph-mon[131110]: pgmap v407: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-11-27T15:24:53.243 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:53.243 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:53.257 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:53.257 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:53.257 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 86 -lt 100 ']' 2023-11-27T15:24:53.258 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 86 + 1 2023-11-27T15:24:53.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:52 smithi133 ceph-mon[97860]: pgmap v407: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-11-27T15:24:53.277 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=87 2023-11-27T15:24:53.278 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:53.306 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:53.306 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:53.309 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:53.310 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:53.310 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:53.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:52 smithi177 ceph-mon[131504]: pgmap v407: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 12 op/s 2023-11-27T15:24:53.733 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:53.733 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:53.860 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:53.860 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:53.873 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:53.873 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:53.873 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 87 -lt 100 ']' 2023-11-27T15:24:53.873 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 87 + 1 2023-11-27T15:24:53.894 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=88 2023-11-27T15:24:53.894 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:53.922 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:53.923 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:53.925 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:53.926 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:53.926 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:54.490 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:54.490 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:54.651 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:54.652 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:54.674 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:54.674 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:54.674 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 88 -lt 100 ']' 2023-11-27T15:24:54.674 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 88 + 1 2023-11-27T15:24:54.694 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=89 2023-11-27T15:24:54.694 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:54.725 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:54.725 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:54.728 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:54.729 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:54.729 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:55.102 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:54 smithi133 ceph-mon[97860]: pgmap v408: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-11-27T15:24:55.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:54 smithi174 ceph-mon[131110]: pgmap v408: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-11-27T15:24:55.173 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:55.174 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:55.292 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:55.292 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:55.315 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:55.315 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:55.315 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 89 -lt 100 ']' 2023-11-27T15:24:55.315 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 89 + 1 2023-11-27T15:24:55.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:54 smithi177 ceph-mon[131504]: pgmap v408: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 12 op/s 2023-11-27T15:24:55.344 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=90 2023-11-27T15:24:55.344 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:55.379 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:55.379 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:55.382 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:55.382 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:55.383 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:55.410 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:24:55.840 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:55.840 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:55.960 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:55.960 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:55.982 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:55.982 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:55.982 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 90 -lt 100 ']' 2023-11-27T15:24:55.983 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 90 + 1 2023-11-27T15:24:56.016 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=91 2023-11-27T15:24:56.016 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:56.061 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:56.061 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:56.063 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:56.064 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:56.064 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:56.632 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:56.632 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:56.775 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:56.775 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:56.798 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:56.798 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:56.798 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 91 -lt 100 ']' 2023-11-27T15:24:56.799 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 91 + 1 2023-11-27T15:24:56.882 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=92 2023-11-27T15:24:56.882 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:56.950 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:56.950 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:56.953 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:56.953 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:56.954 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:57.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:56 smithi174 ceph-mon[131110]: pgmap v409: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2023-11-27T15:24:57.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:56 smithi133 ceph-mon[97860]: pgmap v409: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2023-11-27T15:24:57.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:56 smithi177 ceph-mon[131504]: pgmap v409: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2023-11-27T15:24:57.665 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:57.665 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:57.868 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:57.868 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:57.908 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:57.908 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:57.908 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 92 -lt 100 ']' 2023-11-27T15:24:57.909 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 92 + 1 2023-11-27T15:24:57.982 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=93 2023-11-27T15:24:57.982 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:58.053 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:58.053 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:58.056 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:58.056 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:58.057 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:58.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.260+0000 7f4b5b9ff700 1 -- 172.21.15.133:0/2323068861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b541014b0 msgr2=0x7f4b54101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:58.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.260+0000 7f4b5b9ff700 1 --2- 172.21.15.133:0/2323068861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b541014b0 0x7f4b54101900 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4b440098f0 tx=0x7f4b44009c00 comp rx=0 tx=0).stop 2023-11-27T15:24:58.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.260+0000 7f4b5b9ff700 1 -- 172.21.15.133:0/2323068861 shutdown_connections 2023-11-27T15:24:58.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.260+0000 7f4b5b9ff700 1 --2- 172.21.15.133:0/2323068861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b54101e40 0x7f4b54108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.260+0000 7f4b5b9ff700 1 --2- 172.21.15.133:0/2323068861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b541002c0 0x7f4b541006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.260+0000 7f4b5b9ff700 1 --2- 172.21.15.133:0/2323068861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b541014b0 0x7f4b54101900 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.260+0000 7f4b5b9ff700 1 -- 172.21.15.133:0/2323068861 >> 172.21.15.133:0/2323068861 conn(0x7f4b540fb8a0 msgr2=0x7f4b540fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:58.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.260+0000 7f4b5b9ff700 1 -- 172.21.15.133:0/2323068861 shutdown_connections 2023-11-27T15:24:58.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.260+0000 7f4b5b9ff700 1 -- 172.21.15.133:0/2323068861 wait complete. 2023-11-27T15:24:58.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.262+0000 7f4b5b9ff700 1 Processor -- start 2023-11-27T15:24:58.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.262+0000 7f4b5b9ff700 1 -- start start 2023-11-27T15:24:58.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.262+0000 7f4b5b9ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b541002c0 0x7f4b541982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.262+0000 7f4b5b9ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b541014b0 0x7f4b541987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.262+0000 7f4b5b9ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b54101e40 0x7f4b5419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.262+0000 7f4b5b9ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5419fe20 con 0x7f4b54101e40 2023-11-27T15:24:58.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.262+0000 7f4b5b9ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5419ff60 con 0x7f4b541002c0 2023-11-27T15:24:58.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.262+0000 7f4b5b9ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b541a00f0 con 0x7f4b541014b0 2023-11-27T15:24:58.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.262+0000 7f4b5979b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b541002c0 0x7f4b541982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.262+0000 7f4b5979b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b541002c0 0x7f4b541982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:59794/0 (socket says 172.21.15.133:59794) 2023-11-27T15:24:58.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.262+0000 7f4b5979b700 1 -- 172.21.15.133:0/3524236784 learned_addr learned my addr 172.21.15.133:0/3524236784 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:58.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.263+0000 7f4b58f9a700 1 --2- 172.21.15.133:0/3524236784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b541014b0 0x7f4b541987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.262+0000 7f4b59f9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b54101e40 0x7f4b5419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.263+0000 7f4b5979b700 1 -- 172.21.15.133:0/3524236784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b541014b0 msgr2=0x7f4b541987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:58.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.263+0000 7f4b5979b700 1 --2- 172.21.15.133:0/3524236784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b541014b0 0x7f4b541987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.263+0000 7f4b5979b700 1 -- 172.21.15.133:0/3524236784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b54101e40 msgr2=0x7f4b5419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:58.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.263+0000 7f4b5979b700 1 --2- 172.21.15.133:0/3524236784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b54101e40 0x7f4b5419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.263+0000 7f4b5979b700 1 -- 172.21.15.133:0/3524236784 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b500095f0 con 0x7f4b541002c0 2023-11-27T15:24:58.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.263+0000 7f4b58f9a700 1 --2- 172.21.15.133:0/3524236784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b541014b0 0x7f4b541987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:58.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.263+0000 7f4b5979b700 1 --2- 172.21.15.133:0/3524236784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b541002c0 0x7f4b541982a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000ebf0 tx=0x7f4b5000ef00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:58.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.263+0000 7f4b4a7fc700 1 -- 172.21.15.133:0/3524236784 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50009a10 con 0x7f4b541002c0 2023-11-27T15:24:58.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.264+0000 7f4b4a7fc700 1 -- 172.21.15.133:0/3524236784 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b5000cc90 con 0x7f4b541002c0 2023-11-27T15:24:58.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.264+0000 7f4b5b9ff700 1 -- 172.21.15.133:0/3524236784 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b44009520 con 0x7f4b541002c0 2023-11-27T15:24:58.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.264+0000 7f4b4a7fc700 1 -- 172.21.15.133:0/3524236784 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50018750 con 0x7f4b541002c0 2023-11-27T15:24:58.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.264+0000 7f4b5b9ff700 1 -- 172.21.15.133:0/3524236784 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5410bc70 con 0x7f4b541002c0 2023-11-27T15:24:58.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.267+0000 7f4b4a7fc700 1 -- 172.21.15.133:0/3524236784 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4b500188b0 con 0x7f4b541002c0 2023-11-27T15:24:58.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.268+0000 7f4b4a7fc700 1 --2- 172.21.15.133:0/3524236784 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4b400a11d0 0x7f4b400a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.268+0000 7f4b4a7fc700 1 -- 172.21.15.133:0/3524236784 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f4b50014070 con 0x7f4b541002c0 2023-11-27T15:24:58.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.268+0000 7f4b58f9a700 1 --2- 172.21.15.133:0/3524236784 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4b400a11d0 0x7f4b400a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.269+0000 7f4b58f9a700 1 --2- 172.21.15.133:0/3524236784 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4b400a11d0 0x7f4b400a3680 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4b44005e90 tx=0x7f4b44019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:58.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.352+0000 7f4b6f2cfb80 1 Processor -- start 2023-11-27T15:24:58.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.352+0000 7f4b6f2cfb80 1 -- start start 2023-11-27T15:24:58.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.352+0000 7f4b6f2cfb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4ac00 0x564d0eb4b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.352+0000 7f4b6f2cfb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4b550 0x564d0eb4bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.352+0000 7f4b6f2cfb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4c110 0x564d0eb4c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.352+0000 7f4b6f2cfb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0ea50060 con 0x564d0eb4ac00 2023-11-27T15:24:58.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.352+0000 7f4b6f2cfb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0eaa28a0 con 0x564d0eb4b550 2023-11-27T15:24:58.357 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.352+0000 7f4b6f2cfb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0ea9bd30 con 0x564d0eb4c110 2023-11-27T15:24:58.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.353+0000 7f4b5979b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4ac00 0x564d0eb4b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.353+0000 7f4b5979b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4ac00 0x564d0eb4b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44014/0 (socket says 172.21.15.133:44014) 2023-11-27T15:24:58.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.353+0000 7f4b5979b700 1 -- 172.21.15.133:0/1449938565 learned_addr learned my addr 172.21.15.133:0/1449938565 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:58.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.353+0000 7f4b58f9a700 1 --2- 172.21.15.133:0/1449938565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4b550 0x564d0eb4bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.353+0000 7f4b59f9c700 1 --2- 172.21.15.133:0/1449938565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4c110 0x564d0eb4c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.353+0000 7f4b5979b700 1 -- 172.21.15.133:0/1449938565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4c110 msgr2=0x564d0eb4c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:58.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.353+0000 7f4b5979b700 1 --2- 172.21.15.133:0/1449938565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4c110 0x564d0eb4c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.353+0000 7f4b5979b700 1 -- 172.21.15.133:0/1449938565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4b550 msgr2=0x564d0eb4bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:58.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.353+0000 7f4b5979b700 1 --2- 172.21.15.133:0/1449938565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4b550 0x564d0eb4bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.353+0000 7f4b5979b700 1 -- 172.21.15.133:0/1449938565 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d0e91d630 con 0x564d0eb4ac00 2023-11-27T15:24:58.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.353+0000 7f4b5979b700 1 --2- 172.21.15.133:0/1449938565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4ac00 0x564d0eb4b010 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4b50099f90 tx=0x7f4b5000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b471340616641d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:58.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.354+0000 7f4b3f7fe700 1 -- 172.21.15.133:0/1449938565 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50094ea0 con 0x564d0eb4ac00 2023-11-27T15:24:58.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.354+0000 7f4b3f7fe700 1 -- 172.21.15.133:0/1449938565 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b500a26e0 con 0x564d0eb4ac00 2023-11-27T15:24:58.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.354+0000 7f4b3f7fe700 1 -- 172.21.15.133:0/1449938565 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b500a24b0 con 0x564d0eb4ac00 2023-11-27T15:24:58.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.354+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/1449938565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4ac00 msgr2=0x564d0eb4b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:58.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.354+0000 7f4b6f2cfb80 1 --2- 172.21.15.133:0/1449938565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4ac00 0x564d0eb4b010 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4b50099f90 tx=0x7f4b5000ced0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.355+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/1449938565 shutdown_connections 2023-11-27T15:24:58.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.355+0000 7f4b6f2cfb80 1 --2- 172.21.15.133:0/1449938565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4c110 0x564d0eb4c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.355+0000 7f4b6f2cfb80 1 --2- 172.21.15.133:0/1449938565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4ac00 0x564d0eb4b010 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.355+0000 7f4b6f2cfb80 1 --2- 172.21.15.133:0/1449938565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4b550 0x564d0eb4bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.355+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/1449938565 >> 172.21.15.133:0/1449938565 conn(0x564d0eb40360 msgr2=0x564d0eb45670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:58.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.355+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/1449938565 shutdown_connections 2023-11-27T15:24:58.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.355+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/1449938565 wait complete. 2023-11-27T15:24:58.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.355+0000 7f4b6f2cfb80 1 -- start start 2023-11-27T15:24:58.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.355+0000 7f4b6f2cfb80 1 Processor -- start 2023-11-27T15:24:58.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.356+0000 7f4b6f2cfb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4ac00 0x564d0eb54540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.356+0000 7f4b6f2cfb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4b550 0x564d0eb54a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.356+0000 7f4b6f2cfb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4c110 0x564d0eb5bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.356+0000 7f4b6f2cfb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0ea3d8e0 con 0x564d0eb4b550 2023-11-27T15:24:58.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.356+0000 7f4b6f2cfb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0ea3d580 con 0x564d0eb4c110 2023-11-27T15:24:58.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.356+0000 7f4b6f2cfb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0e91f920 con 0x564d0eb4ac00 2023-11-27T15:24:58.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.356+0000 7f4b6f2cfb80 10 client.0 _ll_register_callbacks cb 0x564d0ea5b780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:58.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.356+0000 7f4b5979b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4ac00 0x564d0eb54540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.356+0000 7f4b58f9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4b550 0x564d0eb54a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.356+0000 7f4b59f9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4c110 0x564d0eb5bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.356+0000 7f4b5979b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4ac00 0x564d0eb54540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:56948/0 (socket says 172.21.15.133:56948) 2023-11-27T15:24:58.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.356+0000 7f4b5979b700 1 -- 172.21.15.133:0/2798199458 learned_addr learned my addr 172.21.15.133:0/2798199458 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:58.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.357+0000 7f4b5979b700 1 -- 172.21.15.133:0/2798199458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4c110 msgr2=0x564d0eb5bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:58.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.357+0000 7f4b5979b700 1 --2- 172.21.15.133:0/2798199458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4c110 0x564d0eb5bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.357+0000 7f4b5979b700 1 -- 172.21.15.133:0/2798199458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4b550 msgr2=0x564d0eb54a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:58.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.357+0000 7f4b5979b700 1 --2- 172.21.15.133:0/2798199458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4b550 0x564d0eb54a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.357+0000 7f4b5979b700 1 -- 172.21.15.133:0/2798199458 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5410bc70 con 0x564d0eb4ac00 2023-11-27T15:24:58.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.357+0000 7f4b59f9c700 1 --2- 172.21.15.133:0/2798199458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4c110 0x564d0eb5bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:58.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.357+0000 7f4b5979b700 1 --2- 172.21.15.133:0/2798199458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4ac00 0x564d0eb54540 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4b5009a8f0 tx=0x7f4b50010d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=882b19f1eabead2c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:58.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.357+0000 7f4b3cff9700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:24:58.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.357+0000 7f4b3cff9700 1 -- 172.21.15.133:0/2798199458 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50094ea0 con 0x564d0eb4ac00 2023-11-27T15:24:58.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.357+0000 7f4b3cff9700 1 -- 172.21.15.133:0/2798199458 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f4b500cae40 con 0x564d0eb4ac00 2023-11-27T15:24:58.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.357+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/2798199458 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564d0eb3db20 con 0x564d0eb4ac00 2023-11-27T15:24:58.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.358+0000 7f4b3cff9700 1 -- 172.21.15.133:0/2798199458 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b50018d60 con 0x564d0eb4ac00 2023-11-27T15:24:58.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.358+0000 7f4b6f2cfb80 10 client.25855 fetch_fsmap learned FSMap version 23 2023-11-27T15:24:58.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.358+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/2798199458 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564d0ea62190 con 0x564d0eb4ac00 2023-11-27T15:24:58.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.358+0000 7f4b3cff9700 1 -- 172.21.15.133:0/2798199458 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564d0eb3db20 con 0x564d0eb4ac00 2023-11-27T15:24:58.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.358+0000 7f4b3cff9700 1 -- 172.21.15.133:0/2798199458 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4b500991f0 con 0x564d0eb4ac00 2023-11-27T15:24:58.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.358+0000 7f4b6f2cfb80 10 client.25855 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:24:58.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.358+0000 7f4b6f2cfb80 10 client.25855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:58.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.358+0000 7f4b6f2cfb80 20 client.25855 populate_metadata read hostname 'smithi133' 2023-11-27T15:24:58.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.358+0000 7f4b6f2cfb80 1 --2- 172.21.15.133:0/2798199458 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564d0eb5fe30 0x564d0eb62240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.358+0000 7f4b6f2cfb80 4 client.25855 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:24:58.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.358+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/2798199458 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d0ea5e0a0 con 0x564d0eb5fe30 2023-11-27T15:24:58.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.359+0000 7f4b58f9a700 1 --2- 172.21.15.133:0/2798199458 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564d0eb5fe30 0x564d0eb62240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.359+0000 7f4b3cff9700 10 client.25855 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:58.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.359+0000 7f4b58f9a700 1 --2- 172.21.15.133:0/2798199458 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564d0eb5fe30 0x564d0eb62240 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1280613c0ade23f9 server_cookie=35bce643b2701614 in_seq=0 out_seq=0 2023-11-27T15:24:58.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.360+0000 7f4b3cff9700 1 -- 172.21.15.133:0/2798199458 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564d0ea5e0a0 con 0x564d0eb5fe30 2023-11-27T15:24:58.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.360+0000 7f4b3cff9700 10 client.25855 handle_command_reply: tid=0 2023-11-27T15:24:58.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.360+0000 7f4b6f2cfb80 1 client.25855 shutdown 2023-11-27T15:24:58.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.360+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/2798199458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4ac00 msgr2=0x564d0eb54540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:58.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.360+0000 7f4b6f2cfb80 1 --2- 172.21.15.133:0/2798199458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4ac00 0x564d0eb54540 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4b5009a8f0 tx=0x7f4b50010d60 comp rx=0 tx=0).stop 2023-11-27T15:24:58.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.360+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/2798199458 shutdown_connections 2023-11-27T15:24:58.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.360+0000 7f4b6f2cfb80 1 --2- 172.21.15.133:0/2798199458 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564d0eb5fe30 0x564d0eb62240 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.360+0000 7f4b6f2cfb80 1 --2- 172.21.15.133:0/2798199458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4c110 0x564d0eb5bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.360+0000 7f4b3cff9700 0 client.25855 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:58.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.360+0000 7f4b6f2cfb80 1 --2- 172.21.15.133:0/2798199458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4ac00 0x564d0eb54540 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.360+0000 7f4b6f2cfb80 1 --2- 172.21.15.133:0/2798199458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4b550 0x564d0eb54a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.360+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/2798199458 >> 172.21.15.133:0/2798199458 conn(0x564d0eb40360 msgr2=0x564d0eb44de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:58.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.361+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/2798199458 shutdown_connections 2023-11-27T15:24:58.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.361+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/2798199458 wait complete. 2023-11-27T15:24:58.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.361+0000 7f4b6f2cfb80 20 client.25855 trim_cache size 0 max 16384 2023-11-27T15:24:58.384 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.379+0000 7f4b6f2cfb80 1 Processor -- start 2023-11-27T15:24:58.384 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.379+0000 7f4b6f2cfb80 1 -- start start 2023-11-27T15:24:58.384 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.379+0000 7f4b6f2cfb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4ac00 0x564d0eb5c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.379+0000 7f4b6f2cfb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4b550 0x564d0eb5ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.379+0000 7f4b6f2cfb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4c110 0x564d0eb542f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.379+0000 7f4b6f2cfb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0ea5e0a0 con 0x564d0eb4ac00 2023-11-27T15:24:58.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.379+0000 7f4b6f2cfb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0eb470f0 con 0x564d0eb4c110 2023-11-27T15:24:58.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.379+0000 7f4b6f2cfb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0eb3d660 con 0x564d0eb4b550 2023-11-27T15:24:58.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.380+0000 7f4b5979b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4ac00 0x564d0eb5c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.380+0000 7f4b59f9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4c110 0x564d0eb542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.380+0000 7f4b58f9a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4b550 0x564d0eb5ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.380+0000 7f4b59f9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4c110 0x564d0eb542f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:59826/0 (socket says 172.21.15.133:59826) 2023-11-27T15:24:58.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.380+0000 7f4b5979b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4ac00 0x564d0eb5c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44026/0 (socket says 172.21.15.133:44026) 2023-11-27T15:24:58.385 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.380+0000 7f4b59f9c700 1 -- 172.21.15.133:0/3782237070 learned_addr learned my addr 172.21.15.133:0/3782237070 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:58.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.380+0000 7f4b59f9c700 1 -- 172.21.15.133:0/3782237070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4b550 msgr2=0x564d0eb5ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:58.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.380+0000 7f4b59f9c700 1 --2- 172.21.15.133:0/3782237070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4b550 0x564d0eb5ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.380+0000 7f4b59f9c700 1 -- 172.21.15.133:0/3782237070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4ac00 msgr2=0x564d0eb5c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:58.389 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.380+0000 7f4b59f9c700 1 --2- 172.21.15.133:0/3782237070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4ac00 0x564d0eb5c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.380+0000 7f4b59f9c700 1 -- 172.21.15.133:0/3782237070 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d0ea61ee0 con 0x564d0eb4c110 2023-11-27T15:24:58.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.380+0000 7f4b5979b700 1 --2- 172.21.15.133:0/3782237070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4ac00 0x564d0eb5c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:58.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.380+0000 7f4b59f9c700 1 --2- 172.21.15.133:0/3782237070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4c110 0x564d0eb542f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c011bf0 tx=0x7f4b4c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12f8fbf8e4e99617 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:58.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.381+0000 7f4b3ffff700 1 -- 172.21.15.133:0/3782237070 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c00e040 con 0x564d0eb4c110 2023-11-27T15:24:58.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.381+0000 7f4b3ffff700 1 -- 172.21.15.133:0/3782237070 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b4c00f040 con 0x564d0eb4c110 2023-11-27T15:24:58.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.381+0000 7f4b3ffff700 1 -- 172.21.15.133:0/3782237070 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c0042d0 con 0x564d0eb4c110 2023-11-27T15:24:58.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.381+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/3782237070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4c110 msgr2=0x564d0eb542f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:58.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.381+0000 7f4b6f2cfb80 1 --2- 172.21.15.133:0/3782237070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4c110 0x564d0eb542f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c011bf0 tx=0x7f4b4c009920 comp rx=0 tx=0).stop 2023-11-27T15:24:58.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.381+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/3782237070 shutdown_connections 2023-11-27T15:24:58.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.381+0000 7f4b6f2cfb80 1 --2- 172.21.15.133:0/3782237070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4c110 0x564d0eb542f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.390 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.381+0000 7f4b6f2cfb80 1 --2- 172.21.15.133:0/3782237070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4ac00 0x564d0eb5c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.381+0000 7f4b6f2cfb80 1 --2- 172.21.15.133:0/3782237070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4b550 0x564d0eb5ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.381+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/3782237070 >> 172.21.15.133:0/3782237070 conn(0x564d0eb40360 msgr2=0x564d0eb53230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:58.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.382+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/3782237070 shutdown_connections 2023-11-27T15:24:58.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.382+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/3782237070 wait complete. 2023-11-27T15:24:58.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.382+0000 7f4b6f2cfb80 1 -- start start 2023-11-27T15:24:58.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.382+0000 7f4b6f2cfb80 1 Processor -- start 2023-11-27T15:24:58.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.383+0000 7f4b6f2cfb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4ac00 0x564d0eb5b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.383+0000 7f4b6f2cfb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4b550 0x564d0eb5bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.383+0000 7f4b6f2cfb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4c110 0x564d0eb59d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.383+0000 7f4b6f2cfb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0eb5a470 con 0x564d0eb4ac00 2023-11-27T15:24:58.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.383+0000 7f4b6f2cfb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0eb5a5e0 con 0x564d0eb4c110 2023-11-27T15:24:58.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.383+0000 7f4b6f2cfb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d0eb5a7a0 con 0x564d0eb4b550 2023-11-27T15:24:58.391 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.383+0000 7f4b6f2cfb80 10 client.0 _ll_register_callbacks cb 0x7f4b44026040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:24:58.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.383+0000 7f4b5979b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4ac00 0x564d0eb5b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.383+0000 7f4b58f9a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4b550 0x564d0eb5bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.383+0000 7f4b5979b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4ac00 0x564d0eb5b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44034/0 (socket says 172.21.15.133:44034) 2023-11-27T15:24:58.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.383+0000 7f4b5979b700 1 -- 172.21.15.133:0/3973931098 learned_addr learned my addr 172.21.15.133:0/3973931098 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:24:58.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.383+0000 7f4b59f9c700 1 --2- 172.21.15.133:0/3973931098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4c110 0x564d0eb59d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.383+0000 7f4b5979b700 1 -- 172.21.15.133:0/3973931098 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4b550 msgr2=0x564d0eb5bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:58.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.383+0000 7f4b5979b700 1 --2- 172.21.15.133:0/3973931098 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4b550 0x564d0eb5bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.383+0000 7f4b5979b700 1 -- 172.21.15.133:0/3973931098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4c110 msgr2=0x564d0eb59d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:24:58.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.383+0000 7f4b5979b700 1 --2- 172.21.15.133:0/3973931098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4c110 0x564d0eb59d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.383+0000 7f4b5979b700 1 -- 172.21.15.133:0/3973931098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d0ea62190 con 0x564d0eb4ac00 2023-11-27T15:24:58.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.383+0000 7f4b59f9c700 1 --2- 172.21.15.133:0/3973931098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4c110 0x564d0eb59d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:24:58.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.383+0000 7f4b58f9a700 1 --2- 172.21.15.133:0/3973931098 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4b550 0x564d0eb5bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:24:58.392 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.384+0000 7f4b5979b700 1 --2- 172.21.15.133:0/3973931098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4ac00 0x564d0eb5b640 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4b50096790 tx=0x7f4b500b8550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69407462686486a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:24:58.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.384+0000 7f4b3cff9700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:24:58.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.384+0000 7f4b3cff9700 1 -- 172.21.15.133:0/3973931098 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50094ea0 con 0x564d0eb4ac00 2023-11-27T15:24:58.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.384+0000 7f4b3cff9700 1 -- 172.21.15.133:0/3973931098 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f4b500cae40 con 0x564d0eb4ac00 2023-11-27T15:24:58.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.384+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/3973931098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b500a26e0 con 0x564d0eb4ac00 2023-11-27T15:24:58.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.384+0000 7f4b3cff9700 1 -- 172.21.15.133:0/3973931098 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b500b8670 con 0x564d0eb4ac00 2023-11-27T15:24:58.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.384+0000 7f4b6f2cfb80 10 client.15834 fetch_fsmap learned FSMap version 23 2023-11-27T15:24:58.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.384+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/3973931098 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f4b4c011870 con 0x564d0eb4ac00 2023-11-27T15:24:58.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.384+0000 7f4b3cff9700 1 -- 172.21.15.133:0/3973931098 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b500a26e0 con 0x564d0eb4ac00 2023-11-27T15:24:58.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.384+0000 7f4b3cff9700 1 -- 172.21.15.133:0/3973931098 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4b500c4ac0 con 0x564d0eb4ac00 2023-11-27T15:24:58.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.385+0000 7f4b6f2cfb80 10 client.15834 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:24:58.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.385+0000 7f4b6f2cfb80 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:24:58.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.385+0000 7f4b6f2cfb80 20 client.15834 populate_metadata read hostname 'smithi133' 2023-11-27T15:24:58.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.385+0000 7f4b6f2cfb80 1 --2- 172.21.15.133:0/3973931098 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564d0ebb4dc0 0x564d0ebb71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:24:58.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.385+0000 7f4b6f2cfb80 4 client.15834 mds_command: new command op to 24560 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:24:58.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.385+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/3973931098 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564d0ea6a660 con 0x564d0ebb4dc0 2023-11-27T15:24:58.394 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.385+0000 7f4b58f9a700 1 --2- 172.21.15.133:0/3973931098 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564d0ebb4dc0 0x564d0ebb71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:24:58.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.385+0000 7f4b58f9a700 1 --2- 172.21.15.133:0/3973931098 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564d0ebb4dc0 0x564d0ebb71a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cc111413ba4f796 server_cookie=7efcc0588b7eef3b in_seq=0 out_seq=0 2023-11-27T15:24:58.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.385+0000 7f4b3cff9700 10 client.15834 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:58.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.386+0000 7f4b3cff9700 1 -- 172.21.15.133:0/3973931098 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564d0ea6a660 con 0x564d0ebb4dc0 2023-11-27T15:24:58.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.386+0000 7f4b3cff9700 10 client.15834 handle_command_reply: tid=0 2023-11-27T15:24:58.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.386+0000 7f4b6f2cfb80 1 client.15834 shutdown 2023-11-27T15:24:58.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.386+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/3973931098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4ac00 msgr2=0x564d0eb5b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:58.395 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.386+0000 7f4b6f2cfb80 1 --2- 172.21.15.133:0/3973931098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4ac00 0x564d0eb5b640 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4b50096790 tx=0x7f4b500b8550 comp rx=0 tx=0).stop 2023-11-27T15:24:58.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.386+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/3973931098 shutdown_connections 2023-11-27T15:24:58.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.386+0000 7f4b6f2cfb80 1 --2- 172.21.15.133:0/3973931098 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564d0ebb4dc0 0x564d0ebb71a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.386+0000 7f4b6f2cfb80 1 --2- 172.21.15.133:0/3973931098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eb4c110 0x564d0eb59d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.386+0000 7f4b6f2cfb80 1 --2- 172.21.15.133:0/3973931098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564d0eb4ac00 0x564d0eb5b640 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.386+0000 7f4b3cff9700 0 client.15834 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:24:58.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.386+0000 7f4b6f2cfb80 1 --2- 172.21.15.133:0/3973931098 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564d0eb4b550 0x564d0eb5bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.386+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/3973931098 >> 172.21.15.133:0/3973931098 conn(0x564d0eb40360 msgr2=0x564d0eb52580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:58.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.386+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/3973931098 shutdown_connections 2023-11-27T15:24:58.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.386+0000 7f4b6f2cfb80 1 -- 172.21.15.133:0/3973931098 wait complete. 2023-11-27T15:24:58.396 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.386+0000 7f4b6f2cfb80 20 client.15834 trim_cache size 0 max 16384 2023-11-27T15:24:58.397 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:24:58.397 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2023-11-27T15:24:58.397 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2023-11-27T15:24:58.397 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:24:58.397 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.387+0000 7f4b5b9ff700 1 -- 172.21.15.133:0/3524236784 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4b400a11d0 msgr2=0x7f4b400a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:58.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.387+0000 7f4b5b9ff700 1 --2- 172.21.15.133:0/3524236784 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4b400a11d0 0x7f4b400a3680 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4b44005e90 tx=0x7f4b44019040 comp rx=0 tx=0).stop 2023-11-27T15:24:58.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.387+0000 7f4b5b9ff700 1 -- 172.21.15.133:0/3524236784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b541002c0 msgr2=0x7f4b541982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:24:58.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.387+0000 7f4b5b9ff700 1 --2- 172.21.15.133:0/3524236784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b541002c0 0x7f4b541982a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000ebf0 tx=0x7f4b5000ef00 comp rx=0 tx=0).stop 2023-11-27T15:24:58.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.387+0000 7f4b5b9ff700 1 -- 172.21.15.133:0/3524236784 shutdown_connections 2023-11-27T15:24:58.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.387+0000 7f4b5b9ff700 1 --2- 172.21.15.133:0/3524236784 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4b400a11d0 0x7f4b400a3680 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.387+0000 7f4b5b9ff700 1 --2- 172.21.15.133:0/3524236784 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b54101e40 0x7f4b5419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.387+0000 7f4b5b9ff700 1 --2- 172.21.15.133:0/3524236784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b541002c0 0x7f4b541982a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.387+0000 7f4b5b9ff700 1 --2- 172.21.15.133:0/3524236784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b541014b0 0x7f4b541987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:24:58.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.387+0000 7f4b5b9ff700 1 -- 172.21.15.133:0/3524236784 >> 172.21.15.133:0/3524236784 conn(0x7f4b540fb8a0 msgr2=0x7f4b54106de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:24:58.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.387+0000 7f4b5b9ff700 1 -- 172.21.15.133:0/3524236784 shutdown_connections 2023-11-27T15:24:58.398 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:24:58.387+0000 7f4b5b9ff700 1 -- 172.21.15.133:0/3524236784 wait complete. 2023-11-27T15:24:58.470 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:24:58.470 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:24:58.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:24:58 smithi133 ceph-mon[97860]: pgmap v410: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2023-11-27T15:24:58.781 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:58.781 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:58.842 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:58.842 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:58.853 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:58.853 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:58.853 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 93 -lt 100 ']' 2023-11-27T15:24:58.854 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 93 + 1 2023-11-27T15:24:58.898 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=94 2023-11-27T15:24:58.898 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:24:58.929 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:58.929 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:58.931 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:24:58.932 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:24:58.932 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:24:59.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:24:58 smithi177 ceph-mon[131504]: pgmap v410: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2023-11-27T15:24:59.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:24:58 smithi174 ceph-mon[131110]: pgmap v410: 129 pgs: 129 active+clean; 1.6 MiB data, 89 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2023-11-27T15:24:59.856 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:24:59.856 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:24:59.952 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:24:59.952 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:24:59.965 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:59.965 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:24:59.965 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 94 -lt 100 ']' 2023-11-27T15:24:59.965 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 94 + 1 2023-11-27T15:25:00.015 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=95 2023-11-27T15:25:00.015 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:25:00.049 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:25:00.049 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:25:00.051 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:25:00.051 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:25:00.052 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:25:00.631 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:25:00.631 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:25:00.825 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:25:00.826 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:25:00.847 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:25:00.847 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:25:00.847 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 95 -lt 100 ']' 2023-11-27T15:25:00.848 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 95 + 1 2023-11-27T15:25:00.898 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=96 2023-11-27T15:25:00.898 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:25:00.996 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:25:00.996 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:25:00.999 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:25:01.000 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:25:01.000 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:25:01.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:00 smithi174 ceph-mon[131110]: pgmap v411: 129 pgs: 129 active+clean; 1.6 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-11-27T15:25:01.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:00 smithi133 ceph-mon[97860]: pgmap v411: 129 pgs: 129 active+clean; 1.6 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-11-27T15:25:01.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:00 smithi177 ceph-mon[131504]: pgmap v411: 129 pgs: 129 active+clean; 1.6 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 14 op/s 2023-11-27T15:25:01.672 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:25:01.673 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:25:01.858 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:25:01.859 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:25:01.882 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:25:01.882 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:25:01.882 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 96 -lt 100 ']' 2023-11-27T15:25:01.883 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 96 + 1 2023-11-27T15:25:01.915 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=97 2023-11-27T15:25:01.915 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:25:02.008 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:25:02.008 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:25:02.011 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:25:02.012 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:25:02.012 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:25:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.360+0000 7f7be6dff700 1 -- 172.21.15.133:0/987899200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be01002b0 msgr2=0x7f7be01006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.360+0000 7f7be6dff700 1 --2- 172.21.15.133:0/987899200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be01002b0 0x7f7be01006c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009990 tx=0x7f7bd4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.361+0000 7f7be6dff700 1 -- 172.21.15.133:0/987899200 shutdown_connections 2023-11-27T15:25:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.361+0000 7f7be6dff700 1 --2- 172.21.15.133:0/987899200 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be0101e30 0x7f7be01086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.361+0000 7f7be6dff700 1 --2- 172.21.15.133:0/987899200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be01002b0 0x7f7be01006c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.361+0000 7f7be6dff700 1 --2- 172.21.15.133:0/987899200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7be01014a0 0x7f7be01018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.361+0000 7f7be6dff700 1 -- 172.21.15.133:0/987899200 >> 172.21.15.133:0/987899200 conn(0x7f7be00fb890 msgr2=0x7f7be00fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.361+0000 7f7be6dff700 1 -- 172.21.15.133:0/987899200 shutdown_connections 2023-11-27T15:25:02.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.361+0000 7f7be6dff700 1 -- 172.21.15.133:0/987899200 wait complete. 2023-11-27T15:25:02.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.362+0000 7f7be6dff700 1 Processor -- start 2023-11-27T15:25:02.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.362+0000 7f7be6dff700 1 -- start start 2023-11-27T15:25:02.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.363+0000 7f7be6dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be01002b0 0x7f7be00748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:02.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.363+0000 7f7be6dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7be01014a0 0x7f7be0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:02.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.363+0000 7f7be6dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be0101e30 0x7f7be0073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:02.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.363+0000 7f7be6dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be0073bc0 con 0x7f7be0101e30 2023-11-27T15:25:02.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.363+0000 7f7be6dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be0073d30 con 0x7f7be01014a0 2023-11-27T15:25:02.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.363+0000 7f7be6dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be01a0000 con 0x7f7be01002b0 2023-11-27T15:25:02.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.363+0000 7f7be4b9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be01002b0 0x7f7be00748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:02.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.363+0000 7f7be4b9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be01002b0 0x7f7be00748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:56980/0 (socket says 172.21.15.133:56980) 2023-11-27T15:25:02.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.363+0000 7f7be4b9b700 1 -- 172.21.15.133:0/97494913 learned_addr learned my addr 172.21.15.133:0/97494913 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:02.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.363+0000 7f7be539c700 1 --2- 172.21.15.133:0/97494913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be0101e30 0x7f7be0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:02.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.363+0000 7f7bdffff700 1 --2- 172.21.15.133:0/97494913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7be01014a0 0x7f7be0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:02.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.363+0000 7f7be4b9b700 1 -- 172.21.15.133:0/97494913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7be01014a0 msgr2=0x7f7be0072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:02.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.363+0000 7f7be4b9b700 1 --2- 172.21.15.133:0/97494913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7be01014a0 0x7f7be0072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.363+0000 7f7be4b9b700 1 -- 172.21.15.133:0/97494913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be0101e30 msgr2=0x7f7be0073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:02.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.363+0000 7f7be4b9b700 1 --2- 172.21.15.133:0/97494913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be0101e30 0x7f7be0073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.363+0000 7f7be4b9b700 1 -- 172.21.15.133:0/97494913 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd40095f0 con 0x7f7be01002b0 2023-11-27T15:25:02.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.363+0000 7f7bdffff700 1 --2- 172.21.15.133:0/97494913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7be01014a0 0x7f7be0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:02.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.364+0000 7f7be4b9b700 1 --2- 172.21.15.133:0/97494913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be01002b0 0x7f7be00748a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4000c00 tx=0x7f7bd40057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:02.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.364+0000 7f7bddffb700 1 -- 172.21.15.133:0/97494913 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd401d070 con 0x7f7be01002b0 2023-11-27T15:25:02.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.364+0000 7f7be6dff700 1 -- 172.21.15.133:0/97494913 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be010bc10 con 0x7f7be01002b0 2023-11-27T15:25:02.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.364+0000 7f7bddffb700 1 -- 172.21.15.133:0/97494913 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bd400fbc0 con 0x7f7be01002b0 2023-11-27T15:25:02.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.364+0000 7f7bddffb700 1 -- 172.21.15.133:0/97494913 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4017650 con 0x7f7be01002b0 2023-11-27T15:25:02.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.364+0000 7f7be6dff700 1 -- 172.21.15.133:0/97494913 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be010ba80 con 0x7f7be01002b0 2023-11-27T15:25:02.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.366+0000 7f7be6dff700 1 -- 172.21.15.133:0/97494913 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be01078b0 con 0x7f7be01002b0 2023-11-27T15:25:02.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.368+0000 7f7bddffb700 1 -- 172.21.15.133:0/97494913 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7bd400fd30 con 0x7f7be01002b0 2023-11-27T15:25:02.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.369+0000 7f7bddffb700 1 --2- 172.21.15.133:0/97494913 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7bd00a1150 0x7f7bd00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:02.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.369+0000 7f7bddffb700 1 -- 172.21.15.133:0/97494913 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7bd40ce940 con 0x7f7be01002b0 2023-11-27T15:25:02.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.369+0000 7f7bdffff700 1 --2- 172.21.15.133:0/97494913 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7bd00a1150 0x7f7bd00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:02.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.370+0000 7f7bdffff700 1 --2- 172.21.15.133:0/97494913 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7bd00a1150 0x7f7bd00a3600 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009cb0 tx=0x7f7bcc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:02.393 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.389+0000 7f7bddffb700 1 -- 172.21.15.133:0/97494913 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7bd4097d70 con 0x7f7be01002b0 2023-11-27T15:25:02.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.644+0000 7f7be6dff700 1 -- 172.21.15.133:0/97494913 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7be002cfe0 con 0x7f7be01002b0 2023-11-27T15:25:02.648 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.645+0000 7f7bddffb700 1 -- 172.21.15.133:0/97494913 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10829 (secure 0 0 0) 0x7f7bd409b390 con 0x7f7be01002b0 2023-11-27T15:25:02.648 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:25:02.649 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14493,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2381290259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2381290259},{"type":"v1","addr":"172.21.15.133:6839","nonce":2381290259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1704875895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1704875895},{"type":"v1","addr":"172.21.15.133:6841","nonce":1704875895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:20:33.201023+0000","modified":"2023-11-27T15:21:04.794526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24521,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3492215689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3492215689},{"type":"v1","addr":"172.21.15.177:6837","nonce":3492215689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6839/2781994447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2781994447},{"type":"v1","addr":"172.21.15.174:6839","nonce":2781994447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:25:02.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.647+0000 7f7be6dff700 1 -- 172.21.15.133:0/97494913 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7bd00a1150 msgr2=0x7f7bd00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:02.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.647+0000 7f7be6dff700 1 --2- 172.21.15.133:0/97494913 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7bd00a1150 0x7f7bd00a3600 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009cb0 tx=0x7f7bcc009260 comp rx=0 tx=0).stop 2023-11-27T15:25:02.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.647+0000 7f7be6dff700 1 -- 172.21.15.133:0/97494913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be01002b0 msgr2=0x7f7be00748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:02.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.647+0000 7f7be6dff700 1 --2- 172.21.15.133:0/97494913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be01002b0 0x7f7be00748a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4000c00 tx=0x7f7bd40057c0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.647+0000 7f7be6dff700 1 -- 172.21.15.133:0/97494913 shutdown_connections 2023-11-27T15:25:02.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.648+0000 7f7be6dff700 1 --2- 172.21.15.133:0/97494913 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7bd00a1150 0x7f7bd00a3600 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.648+0000 7f7be6dff700 1 --2- 172.21.15.133:0/97494913 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7be0101e30 0x7f7be0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.648+0000 7f7be6dff700 1 --2- 172.21.15.133:0/97494913 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7be01002b0 0x7f7be00748a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.648+0000 7f7be6dff700 1 --2- 172.21.15.133:0/97494913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7be01014a0 0x7f7be0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:02.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.648+0000 7f7be6dff700 1 -- 172.21.15.133:0/97494913 >> 172.21.15.133:0/97494913 conn(0x7f7be00fb890 msgr2=0x7f7be01049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:02.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.648+0000 7f7be6dff700 1 -- 172.21.15.133:0/97494913 shutdown_connections 2023-11-27T15:25:02.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:02.648+0000 7f7be6dff700 1 -- 172.21.15.133:0/97494913 wait complete. 2023-11-27T15:25:02.653 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 23 2023-11-27T15:25:02.744 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:25:02.744 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:25:02.754 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:25:02.923 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:25:02.924 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:25:02.947 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:25:02.947 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:25:02.947 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 97 -lt 100 ']' 2023-11-27T15:25:02.947 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 97 + 1 2023-11-27T15:25:02.973 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=98 2023-11-27T15:25:02.973 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:25:02.983 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:25:02.984 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:25:02.986 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:25:02.987 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:25:02.987 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:25:03.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:02 smithi174 ceph-mon[131110]: pgmap v412: 129 pgs: 129 active+clean; 1.7 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-11-27T15:25:03.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:02 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/97494913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:03.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:02 smithi133 ceph-mon[97860]: pgmap v412: 129 pgs: 129 active+clean; 1.7 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-11-27T15:25:03.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:02 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/97494913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:03.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:02 smithi177 ceph-mon[131504]: pgmap v412: 129 pgs: 129 active+clean; 1.7 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-11-27T15:25:03.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:02 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/97494913' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:03.497 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:25:03.497 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:25:03.598 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:25:03.598 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:25:03.614 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:25:03.614 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:25:03.614 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 98 -lt 100 ']' 2023-11-27T15:25:03.615 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 98 + 1 2023-11-27T15:25:03.647 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=99 2023-11-27T15:25:03.647 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:25:03.712 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:25:03.712 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:25:03.714 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:25:03.715 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:25:03.715 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:25:04.401 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:25:04.401 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:25:04.566 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:25:04.566 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:25:04.596 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:25:04.596 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:25:04.596 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 99 -lt 100 ']' 2023-11-27T15:25:04.596 INFO:tasks.workunit.client.0.smithi133.stderr:++ expr 99 + 1 2023-11-27T15:25:04.631 INFO:tasks.workunit.client.0.smithi133.stderr:+ c=100 2023-11-27T15:25:04.631 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:25:04.700 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:25:04.701 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:25:04.704 INFO:tasks.workunit.client.0.smithi133.stderr:+ c1=2 2023-11-27T15:25:04.705 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo 3 2023-11-27T15:25:04.705 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-11-27T15:25:05.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:04 smithi174 ceph-mon[131110]: pgmap v413: 129 pgs: 129 active+clean; 1.7 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2023-11-27T15:25:05.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:04 smithi133 ceph-mon[97860]: pgmap v413: 129 pgs: 129 active+clean; 1.7 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2023-11-27T15:25:05.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:04 smithi177 ceph-mon[131504]: pgmap v413: 129 pgs: 129 active+clean; 1.7 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.4 KiB/s wr, 11 op/s 2023-11-27T15:25:05.738 INFO:tasks.workunit.client.0.smithi133.stderr:++ getfacl d1 2023-11-27T15:25:05.738 INFO:tasks.workunit.client.0.smithi133.stderr:++ grep -c nobody:rw 2023-11-27T15:25:05.924 INFO:tasks.workunit.client.0.smithi133.stderr:+ c2=2 2023-11-27T15:25:05.925 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:25:05.947 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:25:05.947 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 2 -ne 2 ']' 2023-11-27T15:25:05.947 INFO:tasks.workunit.client.0.smithi133.stderr:+ '[' 100 -lt 100 ']' 2023-11-27T15:25:05.964 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir d1 2023-11-27T15:25:06.160 INFO:tasks.workunit.client.0.smithi133.stderr:+ setfattr -n system.posix_acl_access -v 0x02000000 d1 2023-11-27T15:25:06.181 INFO:tasks.workunit.client.0.smithi133.stderr:+ setfattr -n system.posix_acl_default -v 0x02000000 . 2023-11-27T15:25:06.184 INFO:tasks.workunit.client.0.smithi133.stderr:+ expect_failure getfattr -n system.posix_acl_access d1 2023-11-27T15:25:06.184 INFO:tasks.workunit.client.0.smithi133.stderr:+ getfattr -n system.posix_acl_access d1 2023-11-27T15:25:06.188 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.185+0000 7f471bdff700 1 -- 172.21.15.133:0/2958727998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4714103660 msgr2=0x7f4714105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:06.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.185+0000 7f471bdff700 1 --2- 172.21.15.133:0/2958727998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4714103660 0x7f4714105a40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4704009990 tx=0x7f4704009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.185+0000 7f471bdff700 1 -- 172.21.15.133:0/2958727998 shutdown_connections 2023-11-27T15:25:06.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.185+0000 7f471bdff700 1 --2- 172.21.15.133:0/2958727998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4714105f80 0x7f47141083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.185+0000 7f471bdff700 1 --2- 172.21.15.133:0/2958727998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4714100d40 0x7f4714103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.185+0000 7f471bdff700 1 --2- 172.21.15.133:0/2958727998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4714103660 0x7f4714105a40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.189 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.185+0000 7f471bdff700 1 -- 172.21.15.133:0/2958727998 >> 172.21.15.133:0/2958727998 conn(0x7f47140fa7c0 msgr2=0x7f47140fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.185+0000 7f4b9b3ff700 1 -- 172.21.15.133:0/1355154619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b94073550 msgr2=0x7f4b940739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:06.189 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.185+0000 7f4b9b3ff700 1 --2- 172.21.15.133:0/1355154619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b94073550 0x7f4b940739c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4b840098f0 tx=0x7f4b84009c00 comp rx=0 tx=0).stop 2023-11-27T15:25:06.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.185+0000 7f471bdff700 1 -- 172.21.15.133:0/2958727998 shutdown_connections 2023-11-27T15:25:06.190 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.185+0000 7f471bdff700 1 -- 172.21.15.133:0/2958727998 wait complete. 2023-11-27T15:25:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.185+0000 7f4b9b3ff700 1 -- 172.21.15.133:0/1355154619 shutdown_connections 2023-11-27T15:25:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.185+0000 7f4b9b3ff700 1 --2- 172.21.15.133:0/1355154619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b94108100 0x7f4b9410a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.185+0000 7f4b9b3ff700 1 --2- 172.21.15.133:0/1355154619 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b94074b30 0x7f4b94072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.185+0000 7f4b9b3ff700 1 --2- 172.21.15.133:0/1355154619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b94073550 0x7f4b940739c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.185+0000 7f4b9b3ff700 1 -- 172.21.15.133:0/1355154619 >> 172.21.15.133:0/1355154619 conn(0x7f4b940fb840 msgr2=0x7f4b940fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.185+0000 7f4b9b3ff700 1 -- 172.21.15.133:0/1355154619 shutdown_connections 2023-11-27T15:25:06.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.185+0000 7f4b9b3ff700 1 -- 172.21.15.133:0/1355154619 wait complete. 2023-11-27T15:25:06.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.186+0000 7f471bdff700 1 Processor -- start 2023-11-27T15:25:06.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.186+0000 7f471bdff700 1 -- start start 2023-11-27T15:25:06.192 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.186+0000 7f4b9b3ff700 1 Processor -- start 2023-11-27T15:25:06.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.187+0000 7f471bdff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4714100d40 0x7f47141982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.187+0000 7f471bdff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4714103660 0x7f47141987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.187+0000 7f471bdff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4714105f80 0x7f471419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.187+0000 7f471bdff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471419fe20 con 0x7f4714100d40 2023-11-27T15:25:06.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.187+0000 7f471bdff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471419ff60 con 0x7f4714105f80 2023-11-27T15:25:06.193 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.187+0000 7f471bdff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47141a00f0 con 0x7f4714103660 2023-11-27T15:25:06.193 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.187+0000 7f4b9b3ff700 1 -- start start 2023-11-27T15:25:06.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.187+0000 7f4719b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4714100d40 0x7f47141982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.187+0000 7f4719b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4714100d40 0x7f47141982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59088/0 (socket says 172.21.15.133:59088) 2023-11-27T15:25:06.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.187+0000 7f4719b9b700 1 -- 172.21.15.133:0/4186389805 learned_addr learned my addr 172.21.15.133:0/4186389805 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:06.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.187+0000 7f471a39c700 1 --2- 172.21.15.133:0/4186389805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4714105f80 0x7f471419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.187+0000 7f471939a700 1 --2- 172.21.15.133:0/4186389805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4714103660 0x7f47141987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.187+0000 7f4b9b3ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b94073550 0x7f4b94198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.187+0000 7f4b9b3ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b94074b30 0x7f4b941987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.187+0000 7f4b9b3ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b94108100 0x7f4b9419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.187+0000 7f4b9b3ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9419fe10 con 0x7f4b94073550 2023-11-27T15:25:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.187+0000 7f4b9b3ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9419ff50 con 0x7f4b94108100 2023-11-27T15:25:06.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.187+0000 7f4b9b3ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b941a00e0 con 0x7f4b94074b30 2023-11-27T15:25:06.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f4719b9b700 1 -- 172.21.15.133:0/4186389805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4714103660 msgr2=0x7f47141987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:06.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f4719b9b700 1 --2- 172.21.15.133:0/4186389805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4714103660 0x7f47141987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f4719b9b700 1 -- 172.21.15.133:0/4186389805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4714105f80 msgr2=0x7f471419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:06.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f4719b9b700 1 --2- 172.21.15.133:0/4186389805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4714105f80 0x7f471419f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f4719b9b700 1 -- 172.21.15.133:0/4186389805 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47040095f0 con 0x7f4714100d40 2023-11-27T15:25:06.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f471a39c700 1 --2- 172.21.15.133:0/4186389805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4714105f80 0x7f471419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f4b9899a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b94074b30 0x7f4b941987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f4b9919b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b94073550 0x7f4b94198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f4b9899a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b94074b30 0x7f4b941987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:36568/0 (socket says 172.21.15.133:36568) 2023-11-27T15:25:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f4b9899a700 1 -- 172.21.15.133:0/3678928540 learned_addr learned my addr 172.21.15.133:0/3678928540 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:06.197 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f4b9999c700 1 --2- 172.21.15.133:0/3678928540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b94108100 0x7f4b9419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.197 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f4719b9b700 1 --2- 172.21.15.133:0/4186389805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4714100d40 0x7f47141982a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f471000d980 tx=0x7f471000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f4b9899a700 1 -- 172.21.15.133:0/3678928540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b94108100 msgr2=0x7f4b9419f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f4b9899a700 1 --2- 172.21.15.133:0/3678928540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b94108100 0x7f4b9419f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f4b9899a700 1 -- 172.21.15.133:0/3678928540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b94073550 msgr2=0x7f4b94198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f4b9899a700 1 --2- 172.21.15.133:0/3678928540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b94073550 0x7f4b94198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f4b9899a700 1 -- 172.21.15.133:0/3678928540 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b84009520 con 0x7f4b94074b30 2023-11-27T15:25:06.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f470affd700 1 -- 172.21.15.133:0/4186389805 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710004220 con 0x7f4714100d40 2023-11-27T15:25:06.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f470affd700 1 -- 172.21.15.133:0/4186389805 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4710007640 con 0x7f4714100d40 2023-11-27T15:25:06.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f470affd700 1 -- 172.21.15.133:0/4186389805 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710005690 con 0x7f4714100d40 2023-11-27T15:25:06.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f471bdff700 1 -- 172.21.15.133:0/4186389805 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471410bca0 con 0x7f4714100d40 2023-11-27T15:25:06.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f471bdff700 1 -- 172.21.15.133:0/4186389805 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471410bb10 con 0x7f4714100d40 2023-11-27T15:25:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.188+0000 7f4b9899a700 1 --2- 172.21.15.133:0/3678928540 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b94074b30 0x7f4b941987d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009fd0 tx=0x7f4b8400b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.189+0000 7f4b8a7fc700 1 -- 172.21.15.133:0/3678928540 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8401c070 con 0x7f4b94074b30 2023-11-27T15:25:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.189+0000 7f4b8a7fc700 1 -- 172.21.15.133:0/3678928540 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b8400f7d0 con 0x7f4b94074b30 2023-11-27T15:25:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.189+0000 7f4b8a7fc700 1 -- 172.21.15.133:0/3678928540 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b84003a40 con 0x7f4b94074b30 2023-11-27T15:25:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.189+0000 7f4b9b3ff700 1 -- 172.21.15.133:0/3678928540 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9410de20 con 0x7f4b94074b30 2023-11-27T15:25:06.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.189+0000 7f4b9b3ff700 1 -- 172.21.15.133:0/3678928540 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9410dc90 con 0x7f4b94074b30 2023-11-27T15:25:06.200 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.192+0000 7f470affd700 1 -- 172.21.15.133:0/4186389805 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4710004750 con 0x7f4714100d40 2023-11-27T15:25:06.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.192+0000 7f4b8a7fc700 1 -- 172.21.15.133:0/3678928540 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4b84020030 con 0x7f4b94074b30 2023-11-27T15:25:06.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.193+0000 7f470affd700 1 --2- 172.21.15.133:0/4186389805 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f47000a11d0 0x7f47000a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.193+0000 7f470affd700 1 -- 172.21.15.133:0/4186389805 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f47100cca60 con 0x7f4714100d40 2023-11-27T15:25:06.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.193+0000 7f471939a700 1 --2- 172.21.15.133:0/4186389805 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f47000a11d0 0x7f47000a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.194+0000 7f4b8a7fc700 1 --2- 172.21.15.133:0/3678928540 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4b800a11d0 0x7f4b800a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.194+0000 7f4b8a7fc700 1 -- 172.21.15.133:0/3678928540 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f4b840cdb50 con 0x7f4b94074b30 2023-11-27T15:25:06.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.194+0000 7f4b9919b700 1 --2- 172.21.15.133:0/3678928540 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4b800a11d0 0x7f4b800a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.194+0000 7f471939a700 1 --2- 172.21.15.133:0/4186389805 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f47000a11d0 0x7f47000a3680 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4704000c00 tx=0x7f4704005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:06.203 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.195+0000 7f4b9919b700 1 --2- 172.21.15.133:0/3678928540 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4b800a11d0 0x7f4b800a3680 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4b94074f00 tx=0x7f4b90009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:06.208 INFO:tasks.workunit.client.0.smithi133.stderr:d1: system.posix_acl_access: No such attribute 2023-11-27T15:25:06.208 INFO:tasks.workunit.client.0.smithi133.stderr:+ return 0 2023-11-27T15:25:06.208 INFO:tasks.workunit.client.0.smithi133.stderr:+ expect_failure getfattr -n system.posix_acl_default . 2023-11-27T15:25:06.208 INFO:tasks.workunit.client.0.smithi133.stderr:+ getfattr -n system.posix_acl_default . 2023-11-27T15:25:06.210 INFO:tasks.workunit.client.0.smithi133.stderr:.: system.posix_acl_default: No such attribute 2023-11-27T15:25:06.210 INFO:tasks.workunit.client.0.smithi133.stderr:+ return 0 2023-11-27T15:25:06.210 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir d1 2023-11-27T15:25:06.216 INFO:tasks.workunit.client.0.smithi133.stderr:+ cd .. 2023-11-27T15:25:06.216 INFO:tasks.workunit.client.0.smithi133.stderr:+ rmdir testdir 2023-11-27T15:25:06.231 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo OK 2023-11-27T15:25:06.232 INFO:tasks.workunit.client.0.smithi133.stdout:OK 2023-11-27T15:25:06.232 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-27T15:25:06.232 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-27T15:25:06.279 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.276+0000 7f4baedfdb80 1 Processor -- start 2023-11-27T15:25:06.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.275+0000 7f472f630b80 1 Processor -- start 2023-11-27T15:25:06.279 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.276+0000 7f472f630b80 1 -- start start 2023-11-27T15:25:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.276+0000 7f4baedfdb80 1 -- start start 2023-11-27T15:25:06.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.276+0000 7f472f630b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7ac00 0x55a46fb7b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.276+0000 7f472f630b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a46fb7b550 0x55a46fb7bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.280 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.276+0000 7f4baedfdb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558232f6b6b0 0x55823310f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.276+0000 7f4baedfdb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55823310fdb0 0x5582331105d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.276+0000 7f472f630b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7c110 0x55a46fb7c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.276+0000 7f472f630b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46fa80060 con 0x55a46fb7c110 2023-11-27T15:25:06.281 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.276+0000 7f4baedfdb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558233110b10 0x558233110f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.276+0000 7f4baedfdb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558233015000 con 0x558232f6b6b0 2023-11-27T15:25:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.276+0000 7f4baedfdb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x558233067840 con 0x558233110b10 2023-11-27T15:25:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.276+0000 7f4baedfdb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x558233060cd0 con 0x55823310fdb0 2023-11-27T15:25:06.282 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.276+0000 7f472f630b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46fad28a0 con 0x55a46fb7b550 2023-11-27T15:25:06.282 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.276+0000 7f472f630b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46facbd30 con 0x55a46fb7ac00 2023-11-27T15:25:06.282 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.276+0000 7f4b9919b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558232f6b6b0 0x55823310f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.276+0000 7f4719b9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7ac00 0x55a46fb7b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.276+0000 7f4719b9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7ac00 0x55a46fb7b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:36574/0 (socket says 172.21.15.133:36574) 2023-11-27T15:25:06.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.276+0000 7f4719b9b700 1 -- 172.21.15.133:0/2166272902 learned_addr learned my addr 172.21.15.133:0/2166272902 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:06.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.276+0000 7f471a39c700 1 --2- 172.21.15.133:0/2166272902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7c110 0x55a46fb7c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.277+0000 7f471939a700 1 --2- 172.21.15.133:0/2166272902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a46fb7b550 0x55a46fb7bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.283 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.277+0000 7f4b9999c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558233110b10 0x558233110f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.277+0000 7f4b9919b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558232f6b6b0 0x55823310f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59106/0 (socket says 172.21.15.133:59106) 2023-11-27T15:25:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.277+0000 7f4b9919b700 1 -- 172.21.15.133:0/1268462392 learned_addr learned my addr 172.21.15.133:0/1268462392 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:06.284 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.277+0000 7f4b9899a700 1 --2- 172.21.15.133:0/1268462392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55823310fdb0 0x5582331105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.277+0000 7f4719b9b700 1 -- 172.21.15.133:0/2166272902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a46fb7b550 msgr2=0x55a46fb7bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:06.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.277+0000 7f4719b9b700 1 --2- 172.21.15.133:0/2166272902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a46fb7b550 0x55a46fb7bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.277+0000 7f4719b9b700 1 -- 172.21.15.133:0/2166272902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7c110 msgr2=0x55a46fb7c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:06.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.277+0000 7f4719b9b700 1 --2- 172.21.15.133:0/2166272902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7c110 0x55a46fb7c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.277+0000 7f4719b9b700 1 -- 172.21.15.133:0/2166272902 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a46f94d630 con 0x55a46fb7ac00 2023-11-27T15:25:06.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.277+0000 7f471a39c700 1 --2- 172.21.15.133:0/2166272902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7c110 0x55a46fb7c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.277+0000 7f4b9999c700 1 -- 172.21.15.133:0/1268462392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55823310fdb0 msgr2=0x5582331105d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.277+0000 7f4b9999c700 1 --2- 172.21.15.133:0/1268462392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55823310fdb0 0x5582331105d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.277+0000 7f4b9999c700 1 -- 172.21.15.133:0/1268462392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558232f6b6b0 msgr2=0x55823310f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.277+0000 7f4b9999c700 1 --2- 172.21.15.133:0/1268462392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558232f6b6b0 0x55823310f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.285 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.277+0000 7f4b9999c700 1 -- 172.21.15.133:0/1268462392 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558232ee25d0 con 0x558233110b10 2023-11-27T15:25:06.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.277+0000 7f4719b9b700 1 --2- 172.21.15.133:0/2166272902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7ac00 0x55a46fb7b010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f47100a19e0 tx=0x7f47100224a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6590a5056042f2de server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.277+0000 7f4b9999c700 1 --2- 172.21.15.133:0/1268462392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558233110b10 0x558233110f80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c011580 tx=0x7f4b8c0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bcc25e420675021 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:06.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.277+0000 7f46fffff700 1 -- 172.21.15.133:0/2166272902 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47100a1f10 con 0x55a46fb7ac00 2023-11-27T15:25:06.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.278+0000 7f46fffff700 1 -- 172.21.15.133:0/2166272902 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4710097e60 con 0x55a46fb7ac00 2023-11-27T15:25:06.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.278+0000 7f46fffff700 1 -- 172.21.15.133:0/2166272902 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47100a1ce0 con 0x55a46fb7ac00 2023-11-27T15:25:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.278+0000 7f4b7f7fe700 1 -- 172.21.15.133:0/1268462392 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c00e040 con 0x558233110b10 2023-11-27T15:25:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.278+0000 7f4b7f7fe700 1 -- 172.21.15.133:0/1268462392 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b8c00f040 con 0x558233110b10 2023-11-27T15:25:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.278+0000 7f4b7f7fe700 1 -- 172.21.15.133:0/1268462392 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c004360 con 0x558233110b10 2023-11-27T15:25:06.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.278+0000 7f472f630b80 1 -- 172.21.15.133:0/2166272902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7ac00 msgr2=0x55a46fb7b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:06.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.278+0000 7f472f630b80 1 --2- 172.21.15.133:0/2166272902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7ac00 0x55a46fb7b010 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f47100a19e0 tx=0x7f47100224a0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.278+0000 7f472f630b80 1 -- 172.21.15.133:0/2166272902 shutdown_connections 2023-11-27T15:25:06.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.278+0000 7f472f630b80 1 --2- 172.21.15.133:0/2166272902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7c110 0x55a46fb7c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.278+0000 7f472f630b80 1 --2- 172.21.15.133:0/2166272902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7ac00 0x55a46fb7b010 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.278+0000 7f472f630b80 1 --2- 172.21.15.133:0/2166272902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a46fb7b550 0x55a46fb7bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.278+0000 7f472f630b80 1 -- 172.21.15.133:0/2166272902 >> 172.21.15.133:0/2166272902 conn(0x55a46fb70360 msgr2=0x55a46fb75670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:06.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.279+0000 7f472f630b80 1 -- 172.21.15.133:0/2166272902 shutdown_connections 2023-11-27T15:25:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.279+0000 7f4baedfdb80 1 -- 172.21.15.133:0/1268462392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558233110b10 msgr2=0x558233110f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.279+0000 7f4baedfdb80 1 --2- 172.21.15.133:0/1268462392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558233110b10 0x558233110f80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c011580 tx=0x7f4b8c0099e0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.289 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.279+0000 7f472f630b80 1 -- 172.21.15.133:0/2166272902 wait complete. 2023-11-27T15:25:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.279+0000 7f4baedfdb80 1 -- 172.21.15.133:0/1268462392 shutdown_connections 2023-11-27T15:25:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.279+0000 7f4baedfdb80 1 --2- 172.21.15.133:0/1268462392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558233110b10 0x558233110f80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.279+0000 7f4baedfdb80 1 --2- 172.21.15.133:0/1268462392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558232f6b6b0 0x55823310f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.279+0000 7f4baedfdb80 1 --2- 172.21.15.133:0/1268462392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55823310fdb0 0x5582331105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.279+0000 7f4baedfdb80 1 -- 172.21.15.133:0/1268462392 >> 172.21.15.133:0/1268462392 conn(0x558233104f50 msgr2=0x55823310a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.279+0000 7f4baedfdb80 1 -- 172.21.15.133:0/1268462392 shutdown_connections 2023-11-27T15:25:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.279+0000 7f4baedfdb80 1 -- 172.21.15.133:0/1268462392 wait complete. 2023-11-27T15:25:06.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.279+0000 7f472f630b80 1 -- start start 2023-11-27T15:25:06.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.279+0000 7f472f630b80 1 Processor -- start 2023-11-27T15:25:06.291 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.279+0000 7f4baedfdb80 1 -- start start 2023-11-27T15:25:06.291 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.280+0000 7f4baedfdb80 1 Processor -- start 2023-11-27T15:25:06.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.280+0000 7f472f630b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7ac00 0x55a46fb84670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.280+0000 7f472f630b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7b550 0x55a46fb84bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.280+0000 7f472f630b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a46fb7c110 0x55a46fb8bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.280+0000 7f472f630b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46fa6d8e0 con 0x55a46fb7ac00 2023-11-27T15:25:06.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.280+0000 7f472f630b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46fa6d580 con 0x55a46fb7c110 2023-11-27T15:25:06.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.280+0000 7f472f630b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46f94f920 con 0x55a46fb7b550 2023-11-27T15:25:06.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.280+0000 7f472f630b80 10 client.0 _ll_register_callbacks cb 0x55a46fa8b780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:06.292 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.280+0000 7f4baedfdb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558232f6b6b0 0x558233118c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.292 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.280+0000 7f4baedfdb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55823310fdb0 0x558233119190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.280+0000 7f4719b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7ac00 0x55a46fb84670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.280+0000 7f4719b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7ac00 0x55a46fb84670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59118/0 (socket says 172.21.15.133:59118) 2023-11-27T15:25:06.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.280+0000 7f4719b9b700 1 -- 172.21.15.133:0/3268526845 learned_addr learned my addr 172.21.15.133:0/3268526845 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.280+0000 7f4baedfdb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558233110b10 0x5582331201d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.293 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.280+0000 7f4baedfdb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558233002880 con 0x558232f6b6b0 2023-11-27T15:25:06.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.280+0000 7f471939a700 1 --2- 172.21.15.133:0/3268526845 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7b550 0x55a46fb84bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.280+0000 7f4baedfdb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x558233002520 con 0x558233110b10 2023-11-27T15:25:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.280+0000 7f4baedfdb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x558232ee48c0 con 0x55823310fdb0 2023-11-27T15:25:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.280+0000 7f4baedfdb80 10 client.0 _ll_register_callbacks cb 0x558233020720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:06.294 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.280+0000 7f471a39c700 1 --2- 172.21.15.133:0/3268526845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a46fb7c110 0x55a46fb8bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.281+0000 7f4b9919b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558232f6b6b0 0x558233118c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.281+0000 7f4b9899a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55823310fdb0 0x558233119190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.294 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.281+0000 7f4b9919b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558232f6b6b0 0x558233118c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59132/0 (socket says 172.21.15.133:59132) 2023-11-27T15:25:06.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.281+0000 7f4b9919b700 1 -- 172.21.15.133:0/4006797766 learned_addr learned my addr 172.21.15.133:0/4006797766 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:06.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.281+0000 7f4719b9b700 1 -- 172.21.15.133:0/3268526845 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7b550 msgr2=0x55a46fb84bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:06.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.281+0000 7f4719b9b700 1 --2- 172.21.15.133:0/3268526845 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7b550 0x55a46fb84bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.281+0000 7f4719b9b700 1 -- 172.21.15.133:0/3268526845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a46fb7c110 msgr2=0x55a46fb8bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:06.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.281+0000 7f4719b9b700 1 --2- 172.21.15.133:0/3268526845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a46fb7c110 0x55a46fb8bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.295 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.281+0000 7f4719b9b700 1 -- 172.21.15.133:0/3268526845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471410bb10 con 0x55a46fb7ac00 2023-11-27T15:25:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.281+0000 7f4b9999c700 1 --2- 172.21.15.133:0/4006797766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558233110b10 0x5582331201d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.281+0000 7f4719b9b700 1 --2- 172.21.15.133:0/3268526845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7ac00 0x55a46fb84670 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4710097e10 tx=0x7f471009dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3655601369ac5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.281+0000 7f4b9919b700 1 -- 172.21.15.133:0/4006797766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55823310fdb0 msgr2=0x558233119190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.281+0000 7f4b9919b700 1 --2- 172.21.15.133:0/4006797766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55823310fdb0 0x558233119190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.281+0000 7f4b9919b700 1 -- 172.21.15.133:0/4006797766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558233110b10 msgr2=0x5582331201d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.281+0000 7f4b9919b700 1 --2- 172.21.15.133:0/4006797766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558233110b10 0x5582331201d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.281+0000 7f4b9919b700 1 -- 172.21.15.133:0/4006797766 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8c011230 con 0x558232f6b6b0 2023-11-27T15:25:06.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.281+0000 7f46fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:25:06.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.281+0000 7f46fd7fa700 1 -- 172.21.15.133:0/3268526845 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710098fb0 con 0x55a46fb7ac00 2023-11-27T15:25:06.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.281+0000 7f46fd7fa700 1 -- 172.21.15.133:0/3268526845 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f47100a1f10 con 0x55a46fb7ac00 2023-11-27T15:25:06.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.281+0000 7f472f630b80 1 -- 172.21.15.133:0/3268526845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a46fb6db20 con 0x55a46fb7ac00 2023-11-27T15:25:06.297 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.282+0000 7f4b9919b700 1 --2- 172.21.15.133:0/4006797766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558232f6b6b0 0x558233118c50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4b90004580 tx=0x7f4b900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac04092b49de6510 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:06.297 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.282+0000 7f4b7cff9700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:25:06.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.282+0000 7f46fd7fa700 1 -- 172.21.15.133:0/3268526845 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47100a1a80 con 0x55a46fb7ac00 2023-11-27T15:25:06.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.282+0000 7f472f630b80 10 client.15861 fetch_fsmap learned FSMap version 23 2023-11-27T15:25:06.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.282+0000 7f472f630b80 1 -- 172.21.15.133:0/3268526845 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a46fa92190 con 0x55a46fb7ac00 2023-11-27T15:25:06.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.282+0000 7f4b7cff9700 1 -- 172.21.15.133:0/4006797766 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b90022070 con 0x558232f6b6b0 2023-11-27T15:25:06.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.282+0000 7f4b7cff9700 1 -- 172.21.15.133:0/4006797766 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f4b90003d70 con 0x558232f6b6b0 2023-11-27T15:25:06.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.282+0000 7f4baedfdb80 1 -- 172.21.15.133:0/4006797766 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558233102710 con 0x558232f6b6b0 2023-11-27T15:25:06.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.282+0000 7f46fd7fa700 1 -- 172.21.15.133:0/3268526845 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a46fb6db20 con 0x55a46fb7ac00 2023-11-27T15:25:06.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.282+0000 7f46fd7fa700 1 -- 172.21.15.133:0/3268526845 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4710004a00 con 0x55a46fb7ac00 2023-11-27T15:25:06.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.282+0000 7f4b7cff9700 1 -- 172.21.15.133:0/4006797766 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b90027420 con 0x558232f6b6b0 2023-11-27T15:25:06.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.282+0000 7f4baedfdb80 10 client.15864 fetch_fsmap learned FSMap version 23 2023-11-27T15:25:06.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.282+0000 7f4baedfdb80 1 -- 172.21.15.133:0/4006797766 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558233027130 con 0x558232f6b6b0 2023-11-27T15:25:06.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.282+0000 7f472f630b80 10 client.15861 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:25:06.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.282+0000 7f472f630b80 10 client.15861 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-11-27T15:25:06.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.282+0000 7f472f630b80 20 client.15861 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:06.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.283+0000 7f472f630b80 1 --2- 172.21.15.133:0/3268526845 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55a46fb8ff20 0x55a46fb92330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.283+0000 7f472f630b80 4 client.15861 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:06.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.283+0000 7f472f630b80 1 -- 172.21.15.133:0/3268526845 --> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a46fa8e0a0 con 0x55a46fb8ff20 2023-11-27T15:25:06.301 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.283+0000 7f4b7cff9700 1 -- 172.21.15.133:0/4006797766 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558233102710 con 0x558232f6b6b0 2023-11-27T15:25:06.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.283+0000 7f471939a700 1 --2- 172.21.15.133:0/3268526845 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55a46fb8ff20 0x55a46fb92330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.283+0000 7f4b7cff9700 1 -- 172.21.15.133:0/4006797766 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4b9001d530 con 0x558232f6b6b0 2023-11-27T15:25:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.283+0000 7f4baedfdb80 10 client.15864 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:25:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.283+0000 7f4baedfdb80 10 client.15864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.283+0000 7f4baedfdb80 20 client.15864 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.283+0000 7f4baedfdb80 1 --2- 172.21.15.133:0/4006797766 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x558233124ab0 0x558233126ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.283+0000 7f4baedfdb80 4 client.15864 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.283+0000 7f4baedfdb80 1 -- 172.21.15.133:0/4006797766 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558233023040 con 0x558233124ab0 2023-11-27T15:25:06.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.283+0000 7f4b9899a700 1 --2- 172.21.15.133:0/4006797766 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x558233124ab0 0x558233126ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.284+0000 7f471939a700 1 --2- 172.21.15.133:0/3268526845 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55a46fb8ff20 0x55a46fb92330 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=48d1046b01af5938 server_cookie=9656beb0d8aa17eb in_seq=0 out_seq=0 2023-11-27T15:25:06.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.284+0000 7f46fd7fa700 10 client.15861 ms_handle_connect on v2:172.21.15.177:6838/1463687598 2023-11-27T15:25:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.284+0000 7f4b7cff9700 10 client.15864 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.284+0000 7f4b9899a700 1 --2- 172.21.15.133:0/4006797766 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x558233124ab0 0x558233126ec0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16141954aac629df server_cookie=a6d8d0a5ad73b8ac in_seq=0 out_seq=0 2023-11-27T15:25:06.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.284+0000 7f46fd7fa700 1 -- 172.21.15.133:0/3268526845 <== mds.2 v2:172.21.15.177:6838/1463687598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a46fa8e0a0 con 0x55a46fb8ff20 2023-11-27T15:25:06.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.284+0000 7f46fd7fa700 10 client.15861 handle_command_reply: tid=0 2023-11-27T15:25:06.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.284+0000 7f472f630b80 1 client.15861 shutdown 2023-11-27T15:25:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f4b7cff9700 1 -- 172.21.15.133:0/4006797766 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558233023040 con 0x558233124ab0 2023-11-27T15:25:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f4b7cff9700 10 client.15864 handle_command_reply: tid=0 2023-11-27T15:25:06.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f472f630b80 1 -- 172.21.15.133:0/3268526845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7ac00 msgr2=0x55a46fb84670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:06.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f472f630b80 1 --2- 172.21.15.133:0/3268526845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7ac00 0x55a46fb84670 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4710097e10 tx=0x7f471009dc10 comp rx=0 tx=0).stop 2023-11-27T15:25:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f4baedfdb80 1 client.15864 shutdown 2023-11-27T15:25:06.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f472f630b80 1 -- 172.21.15.133:0/3268526845 shutdown_connections 2023-11-27T15:25:06.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f472f630b80 1 --2- 172.21.15.133:0/3268526845 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55a46fb8ff20 0x55a46fb92330 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f472f630b80 1 --2- 172.21.15.133:0/3268526845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a46fb7c110 0x55a46fb8bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f472f630b80 1 --2- 172.21.15.133:0/3268526845 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7ac00 0x55a46fb84670 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f472f630b80 1 --2- 172.21.15.133:0/3268526845 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7b550 0x55a46fb84bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f472f630b80 1 -- 172.21.15.133:0/3268526845 >> 172.21.15.133:0/3268526845 conn(0x55a46fb70360 msgr2=0x55a46fb74de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:06.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f46fd7fa700 0 client.15861 ms_handle_reset on v2:172.21.15.177:6838/1463687598 2023-11-27T15:25:06.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f472f630b80 1 -- 172.21.15.133:0/3268526845 shutdown_connections 2023-11-27T15:25:06.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f472f630b80 1 -- 172.21.15.133:0/3268526845 wait complete. 2023-11-27T15:25:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f4baedfdb80 1 -- 172.21.15.133:0/4006797766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558232f6b6b0 msgr2=0x558233118c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f4baedfdb80 1 --2- 172.21.15.133:0/4006797766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558232f6b6b0 0x558233118c50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4b90004580 tx=0x7f4b900047c0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f4baedfdb80 1 -- 172.21.15.133:0/4006797766 shutdown_connections 2023-11-27T15:25:06.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f4baedfdb80 1 --2- 172.21.15.133:0/4006797766 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x558233124ab0 0x558233126ec0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f4baedfdb80 1 --2- 172.21.15.133:0/4006797766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558233110b10 0x5582331201d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f4baedfdb80 1 --2- 172.21.15.133:0/4006797766 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558232f6b6b0 0x558233118c50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f4baedfdb80 1 --2- 172.21.15.133:0/4006797766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55823310fdb0 0x558233119190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f4baedfdb80 1 -- 172.21.15.133:0/4006797766 >> 172.21.15.133:0/4006797766 conn(0x558233104f50 msgr2=0x5582331097e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:06.307 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f4b7cff9700 0 client.15864 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:06.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f472f630b80 20 client.15861 trim_cache size 0 max 16384 2023-11-27T15:25:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f4baedfdb80 1 -- 172.21.15.133:0/4006797766 shutdown_connections 2023-11-27T15:25:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.285+0000 7f4baedfdb80 1 -- 172.21.15.133:0/4006797766 wait complete. 2023-11-27T15:25:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.286+0000 7f4baedfdb80 20 client.15864 trim_cache size 0 max 16384 2023-11-27T15:25:06.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.299+0000 7f472f630b80 1 Processor -- start 2023-11-27T15:25:06.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.300+0000 7f472f630b80 1 -- start start 2023-11-27T15:25:06.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.300+0000 7f472f630b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7ac00 0x55a46fb8c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.300+0000 7f472f630b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7b550 0x55a46fb8bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.300+0000 7f472f630b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a46fb7c110 0x55a46fb7f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.300+0000 7f472f630b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46fb83b20 con 0x55a46fb7b550 2023-11-27T15:25:06.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.300+0000 7f472f630b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46fb82da0 con 0x55a46fb7c110 2023-11-27T15:25:06.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.300+0000 7f472f630b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46fb770f0 con 0x55a46fb7ac00 2023-11-27T15:25:06.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.300+0000 7f471939a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7b550 0x55a46fb8bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.300+0000 7f471a39c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a46fb7c110 0x55a46fb7f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.300+0000 7f471939a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7b550 0x55a46fb8bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59148/0 (socket says 172.21.15.133:59148) 2023-11-27T15:25:06.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.300+0000 7f471939a700 1 -- 172.21.15.133:0/3858854479 learned_addr learned my addr 172.21.15.133:0/3858854479 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:06.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.300+0000 7f4719b9b700 1 --2- 172.21.15.133:0/3858854479 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7ac00 0x55a46fb8c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.301+0000 7f471939a700 1 -- 172.21.15.133:0/3858854479 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7ac00 msgr2=0x55a46fb8c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:06.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.301+0000 7f471939a700 1 --2- 172.21.15.133:0/3858854479 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7ac00 0x55a46fb8c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.301+0000 7f471939a700 1 -- 172.21.15.133:0/3858854479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a46fb7c110 msgr2=0x55a46fb7f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:06.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.301+0000 7f471939a700 1 --2- 172.21.15.133:0/3858854479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a46fb7c110 0x55a46fb7f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.301+0000 7f471939a700 1 -- 172.21.15.133:0/3858854479 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a46fa91ee0 con 0x55a46fb7b550 2023-11-27T15:25:06.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.301+0000 7f471a39c700 1 --2- 172.21.15.133:0/3858854479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a46fb7c110 0x55a46fb7f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:06.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.301+0000 7f471939a700 1 --2- 172.21.15.133:0/3858854479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7b550 0x55a46fb8bbb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4704015000 tx=0x7f4704020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d79f3bfecfc1563 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:06.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.301+0000 7f4708bff700 1 -- 172.21.15.133:0/3858854479 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470402b040 con 0x55a46fb7b550 2023-11-27T15:25:06.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.301+0000 7f4708bff700 1 -- 172.21.15.133:0/3858854479 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f470402c040 con 0x55a46fb7b550 2023-11-27T15:25:06.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.301+0000 7f4708bff700 1 -- 172.21.15.133:0/3858854479 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4704004c30 con 0x55a46fb7b550 2023-11-27T15:25:06.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.302+0000 7f472f630b80 1 -- 172.21.15.133:0/3858854479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7b550 msgr2=0x55a46fb8bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:06.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.302+0000 7f472f630b80 1 --2- 172.21.15.133:0/3858854479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7b550 0x55a46fb8bbb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4704015000 tx=0x7f4704020000 comp rx=0 tx=0).stop 2023-11-27T15:25:06.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.302+0000 7f472f630b80 1 -- 172.21.15.133:0/3858854479 shutdown_connections 2023-11-27T15:25:06.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.302+0000 7f472f630b80 1 --2- 172.21.15.133:0/3858854479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a46fb7c110 0x55a46fb7f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.302+0000 7f472f630b80 1 --2- 172.21.15.133:0/3858854479 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7ac00 0x55a46fb8c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.302+0000 7f472f630b80 1 --2- 172.21.15.133:0/3858854479 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7b550 0x55a46fb8bbb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.302+0000 7f472f630b80 1 -- 172.21.15.133:0/3858854479 >> 172.21.15.133:0/3858854479 conn(0x55a46fb70360 msgr2=0x55a46fb832f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:06.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.302+0000 7f472f630b80 1 -- 172.21.15.133:0/3858854479 shutdown_connections 2023-11-27T15:25:06.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.302+0000 7f472f630b80 1 -- 172.21.15.133:0/3858854479 wait complete. 2023-11-27T15:25:06.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.303+0000 7f472f630b80 1 -- start start 2023-11-27T15:25:06.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.303+0000 7f472f630b80 1 Processor -- start 2023-11-27T15:25:06.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.303+0000 7f472f630b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7ac00 0x55a46fb892f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.303+0000 7f472f630b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7b550 0x55a46fb89830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.303+0000 7f472f630b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a46fb7c110 0x55a46fb8a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.303+0000 7f472f630b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46fb8ad80 con 0x55a46fb7ac00 2023-11-27T15:25:06.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.303+0000 7f472f630b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46fb8aef0 con 0x55a46fb7c110 2023-11-27T15:25:06.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.303+0000 7f472f630b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a46fb8b0b0 con 0x55a46fb7b550 2023-11-27T15:25:06.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.303+0000 7f472f630b80 10 client.0 _ll_register_callbacks cb 0x7f4710022350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:06.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f4719b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7ac00 0x55a46fb892f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f4719b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7ac00 0x55a46fb892f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59156/0 (socket says 172.21.15.133:59156) 2023-11-27T15:25:06.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f4719b9b700 1 -- 172.21.15.133:0/1281797636 learned_addr learned my addr 172.21.15.133:0/1281797636 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:06.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f471939a700 1 --2- 172.21.15.133:0/1281797636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7b550 0x55a46fb89830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.314 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f4baedfdb80 1 Processor -- start 2023-11-27T15:25:06.314 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f4baedfdb80 1 -- start start 2023-11-27T15:25:06.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f471a39c700 1 --2- 172.21.15.133:0/1281797636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a46fb7c110 0x55a46fb8a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f4719b9b700 1 -- 172.21.15.133:0/1281797636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7b550 msgr2=0x55a46fb89830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:06.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f4719b9b700 1 --2- 172.21.15.133:0/1281797636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7b550 0x55a46fb89830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f4719b9b700 1 -- 172.21.15.133:0/1281797636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a46fb7c110 msgr2=0x55a46fb8a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:06.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f4719b9b700 1 --2- 172.21.15.133:0/1281797636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a46fb7c110 0x55a46fb8a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f4719b9b700 1 -- 172.21.15.133:0/1281797636 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470c003c90 con 0x55a46fb7ac00 2023-11-27T15:25:06.315 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f471939a700 1 --2- 172.21.15.133:0/1281797636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7b550 0x55a46fb89830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:06.315 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f4baedfdb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558232f6b6b0 0x558233116e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.315 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f4baedfdb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55823310fdb0 0x558233120190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f4baedfdb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558233110b10 0x558233113920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f4baedfdb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x558233110420 con 0x55823310fdb0 2023-11-27T15:25:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f4baedfdb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55823310b930 con 0x558232f6b6b0 2023-11-27T15:25:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f4baedfdb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x558233116bc0 con 0x558233110b10 2023-11-27T15:25:06.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f4719b9b700 1 --2- 172.21.15.133:0/1281797636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7ac00 0x55a46fb892f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4710096530 tx=0x7f4710099550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6533d038c488c9e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:06.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f46fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:25:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f4b9919b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558232f6b6b0 0x558233116e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f4b9919b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558232f6b6b0 0x558233116e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:44506/0 (socket says 172.21.15.133:44506) 2023-11-27T15:25:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.304+0000 7f4b9919b700 1 -- 172.21.15.133:0/1989317322 learned_addr learned my addr 172.21.15.133:0/1989317322 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.305+0000 7f4b9999c700 1 --2- 172.21.15.133:0/1989317322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558233110b10 0x558233113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.305+0000 7f4b9899a700 1 --2- 172.21.15.133:0/1989317322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55823310fdb0 0x558233120190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.305+0000 7f4b9919b700 1 -- 172.21.15.133:0/1989317322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558233110b10 msgr2=0x558233113920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.305+0000 7f4b9919b700 1 --2- 172.21.15.133:0/1989317322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558233110b10 0x558233113920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.305+0000 7f4b9919b700 1 -- 172.21.15.133:0/1989317322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55823310fdb0 msgr2=0x558233120190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:06.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.305+0000 7f4b9919b700 1 --2- 172.21.15.133:0/1989317322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55823310fdb0 0x558233120190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.305+0000 7f4b9919b700 1 -- 172.21.15.133:0/1989317322 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558233026e80 con 0x558232f6b6b0 2023-11-27T15:25:06.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.305+0000 7f4b9899a700 1 --2- 172.21.15.133:0/1989317322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55823310fdb0 0x558233120190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:06.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.305+0000 7f46fd7fa700 1 -- 172.21.15.133:0/1281797636 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47100a1c80 con 0x55a46fb7ac00 2023-11-27T15:25:06.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.305+0000 7f46fd7fa700 1 -- 172.21.15.133:0/1281797636 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f47100a1f10 con 0x55a46fb7ac00 2023-11-27T15:25:06.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.305+0000 7f472f630b80 1 -- 172.21.15.133:0/1281797636 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4710097e60 con 0x55a46fb7ac00 2023-11-27T15:25:06.318 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.305+0000 7f4b9919b700 1 --2- 172.21.15.133:0/1989317322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558232f6b6b0 0x558233116e10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4b90013010 tx=0x7f4b9001d410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53bd4aec120fc8b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:06.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.305+0000 7f46fd7fa700 1 -- 172.21.15.133:0/1281797636 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4710098a20 con 0x55a46fb7ac00 2023-11-27T15:25:06.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.305+0000 7f472f630b80 10 client.15870 fetch_fsmap learned FSMap version 23 2023-11-27T15:25:06.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.305+0000 7f472f630b80 1 -- 172.21.15.133:0/1281797636 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f4714108fd0 con 0x55a46fb7ac00 2023-11-27T15:25:06.319 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.305+0000 7f4b7ffff700 1 -- 172.21.15.133:0/1989317322 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b90022070 con 0x558232f6b6b0 2023-11-27T15:25:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.305+0000 7f4b7ffff700 1 -- 172.21.15.133:0/1989317322 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b900092d0 con 0x558232f6b6b0 2023-11-27T15:25:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.305+0000 7f4b7ffff700 1 -- 172.21.15.133:0/1989317322 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b90034900 con 0x558232f6b6b0 2023-11-27T15:25:06.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.306+0000 7f46fd7fa700 1 -- 172.21.15.133:0/1281797636 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4710097e60 con 0x55a46fb7ac00 2023-11-27T15:25:06.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.306+0000 7f46fd7fa700 1 -- 172.21.15.133:0/1281797636 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4710098c10 con 0x55a46fb7ac00 2023-11-27T15:25:06.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.306+0000 7f472f630b80 10 client.15870 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:25:06.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.306+0000 7f472f630b80 10 client.15870 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-11-27T15:25:06.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.306+0000 7f472f630b80 20 client.15870 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:06.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.306+0000 7f472f630b80 1 --2- 172.21.15.133:0/1281797636 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55a46fbe4230 0x55a46fbe6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.306+0000 7f472f630b80 4 client.15870 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:25:06.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.306+0000 7f472f630b80 1 -- 172.21.15.133:0/1281797636 --> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a46fa9a660 con 0x55a46fbe4230 2023-11-27T15:25:06.321 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.306+0000 7f471939a700 1 --2- 172.21.15.133:0/1281797636 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55a46fbe4230 0x55a46fbe6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.306+0000 7f4baedfdb80 1 -- 172.21.15.133:0/1989317322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558232f6b6b0 msgr2=0x558233116e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.306+0000 7f4baedfdb80 1 --2- 172.21.15.133:0/1989317322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558232f6b6b0 0x558233116e10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4b90013010 tx=0x7f4b9001d410 comp rx=0 tx=0).stop 2023-11-27T15:25:06.322 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.306+0000 7f4baedfdb80 1 -- 172.21.15.133:0/1989317322 shutdown_connections 2023-11-27T15:25:06.322 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.306+0000 7f4baedfdb80 1 --2- 172.21.15.133:0/1989317322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558233110b10 0x558233113920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.322 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.306+0000 7f4baedfdb80 1 --2- 172.21.15.133:0/1989317322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558232f6b6b0 0x558233116e10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.322 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.306+0000 7f4baedfdb80 1 --2- 172.21.15.133:0/1989317322 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55823310fdb0 0x558233120190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.322 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.306+0000 7f4baedfdb80 1 -- 172.21.15.133:0/1989317322 >> 172.21.15.133:0/1989317322 conn(0x558233104f50 msgr2=0x55823312b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:06.322 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.306+0000 7f4baedfdb80 1 -- 172.21.15.133:0/1989317322 shutdown_connections 2023-11-27T15:25:06.322 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.306+0000 7f4baedfdb80 1 -- 172.21.15.133:0/1989317322 wait complete. 2023-11-27T15:25:06.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.307+0000 7f471939a700 1 --2- 172.21.15.133:0/1281797636 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55a46fbe4230 0x55a46fbe6640 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=12c6cbbf27b9f71f server_cookie=f2921f5e644e37c5 in_seq=0 out_seq=0 2023-11-27T15:25:06.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.307+0000 7f46fd7fa700 10 client.15870 ms_handle_connect on v2:172.21.15.177:6838/1463687598 2023-11-27T15:25:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.307+0000 7f4baedfdb80 1 -- start start 2023-11-27T15:25:06.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.307+0000 7f46fd7fa700 1 -- 172.21.15.133:0/1281797636 <== mds.2 v2:172.21.15.177:6838/1463687598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a46fa9a660 con 0x55a46fbe4230 2023-11-27T15:25:06.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.307+0000 7f46fd7fa700 10 client.15870 handle_command_reply: tid=0 2023-11-27T15:25:06.323 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.307+0000 7f472f630b80 1 client.15870 shutdown 2023-11-27T15:25:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.307+0000 7f4baedfdb80 1 Processor -- start 2023-11-27T15:25:06.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.307+0000 7f472f630b80 1 -- 172.21.15.133:0/1281797636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7ac00 msgr2=0x55a46fb892f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:06.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.307+0000 7f472f630b80 1 --2- 172.21.15.133:0/1281797636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7ac00 0x55a46fb892f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4710096530 tx=0x7f4710099550 comp rx=0 tx=0).stop 2023-11-27T15:25:06.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.307+0000 7f472f630b80 1 -- 172.21.15.133:0/1281797636 shutdown_connections 2023-11-27T15:25:06.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.307+0000 7f472f630b80 1 --2- 172.21.15.133:0/1281797636 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55a46fbe4230 0x55a46fbe6640 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.307+0000 7f472f630b80 1 --2- 172.21.15.133:0/1281797636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a46fb7c110 0x55a46fb8a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.307+0000 7f472f630b80 1 --2- 172.21.15.133:0/1281797636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a46fb7ac00 0x55a46fb892f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.307+0000 7f472f630b80 1 --2- 172.21.15.133:0/1281797636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a46fb7b550 0x55a46fb89830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.307+0000 7f46fd7fa700 0 client.15870 ms_handle_reset on v2:172.21.15.177:6838/1463687598 2023-11-27T15:25:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.307+0000 7f472f630b80 1 -- 172.21.15.133:0/1281797636 >> 172.21.15.133:0/1281797636 conn(0x55a46fb70360 msgr2=0x55a46fb8c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.307+0000 7f472f630b80 1 -- 172.21.15.133:0/1281797636 shutdown_connections 2023-11-27T15:25:06.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.307+0000 7f472f630b80 1 -- 172.21.15.133:0/1281797636 wait complete. 2023-11-27T15:25:06.325 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f4baedfdb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558232f6b6b0 0x55823311c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.325 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f4baedfdb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55823310fdb0 0x55823311c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.325 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f4baedfdb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558233110b10 0x55823311f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f4baedfdb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55823311fe30 con 0x55823310fdb0 2023-11-27T15:25:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f4baedfdb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55823311ffa0 con 0x558233110b10 2023-11-27T15:25:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f4baedfdb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x558233120160 con 0x558232f6b6b0 2023-11-27T15:25:06.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f472f630b80 20 client.15870 trim_cache size 0 max 16384 2023-11-27T15:25:06.326 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f4baedfdb80 10 client.0 _ll_register_callbacks cb 0x7f4b900043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:06.326 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2023-11-27T15:25:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f4b9899a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55823310fdb0 0x55823311c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f4b9899a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55823310fdb0 0x55823311c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59178/0 (socket says 172.21.15.133:59178) 2023-11-27T15:25:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f4b9899a700 1 -- 172.21.15.133:0/1607892760 learned_addr learned my addr 172.21.15.133:0/1607892760 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:06.327 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f4b9919b700 1 --2- 172.21.15.133:0/1607892760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558232f6b6b0 0x55823311c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.328 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f4b9999c700 1 --2- 172.21.15.133:0/1607892760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558233110b10 0x55823311f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.328 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f4b9899a700 1 -- 172.21.15.133:0/1607892760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558232f6b6b0 msgr2=0x55823311c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:06.328 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f4b9899a700 1 --2- 172.21.15.133:0/1607892760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558232f6b6b0 0x55823311c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.328 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f4b9899a700 1 -- 172.21.15.133:0/1607892760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558233110b10 msgr2=0x55823311f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:06.328 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f4b9899a700 1 --2- 172.21.15.133:0/1607892760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558233110b10 0x55823311f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.328 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f4b9899a700 1 -- 172.21.15.133:0/1607892760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9002c040 con 0x55823310fdb0 2023-11-27T15:25:06.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f471bdff700 1 -- 172.21.15.133:0/4186389805 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f47000a11d0 msgr2=0x7f47000a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:06.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f471bdff700 1 --2- 172.21.15.133:0/4186389805 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f47000a11d0 0x7f47000a3680 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4704000c00 tx=0x7f4704005fb0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f471bdff700 1 -- 172.21.15.133:0/4186389805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4714100d40 msgr2=0x7f47141982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:06.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.308+0000 7f471bdff700 1 --2- 172.21.15.133:0/4186389805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4714100d40 0x7f47141982a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f471000d980 tx=0x7f471000dc90 comp rx=0 tx=0).stop 2023-11-27T15:25:06.329 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.309+0000 7f4b9899a700 1 --2- 172.21.15.133:0/1607892760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55823310fdb0 0x55823311c8d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4b84026040 tx=0x7f4b84009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41d2dfdbefbaf7d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:06.329 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.309+0000 7f4b7cff9700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:25:06.329 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.309+0000 7f4b7cff9700 1 -- 172.21.15.133:0/1607892760 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b840b3040 con 0x55823310fdb0 2023-11-27T15:25:06.329 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.309+0000 7f4b7cff9700 1 -- 172.21.15.133:0/1607892760 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f4b840a3000 con 0x55823310fdb0 2023-11-27T15:25:06.329 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.309+0000 7f4baedfdb80 1 -- 172.21.15.133:0/1607892760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b8c00f040 con 0x55823310fdb0 2023-11-27T15:25:06.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.309+0000 7f471bdff700 1 -- 172.21.15.133:0/4186389805 shutdown_connections 2023-11-27T15:25:06.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.309+0000 7f471bdff700 1 --2- 172.21.15.133:0/4186389805 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f47000a11d0 0x7f47000a3680 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.309+0000 7f471bdff700 1 --2- 172.21.15.133:0/4186389805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4714105f80 0x7f471419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.309+0000 7f471bdff700 1 --2- 172.21.15.133:0/4186389805 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4714100d40 0x7f47141982a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.309+0000 7f471bdff700 1 --2- 172.21.15.133:0/4186389805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4714103660 0x7f47141987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.309+0000 7f471bdff700 1 -- 172.21.15.133:0/4186389805 >> 172.21.15.133:0/4186389805 conn(0x7f47140fa7c0 msgr2=0x7f47140ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.309+0000 7f4b7cff9700 1 -- 172.21.15.133:0/1607892760 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b840cb3b0 con 0x55823310fdb0 2023-11-27T15:25:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.309+0000 7f4baedfdb80 10 client.15873 fetch_fsmap learned FSMap version 23 2023-11-27T15:25:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.309+0000 7f4baedfdb80 1 -- 172.21.15.133:0/1607892760 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558232e77c00 con 0x55823310fdb0 2023-11-27T15:25:06.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.309+0000 7f471bdff700 1 -- 172.21.15.133:0/4186389805 shutdown_connections 2023-11-27T15:25:06.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:06.309+0000 7f471bdff700 1 -- 172.21.15.133:0/4186389805 wait complete. 2023-11-27T15:25:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.309+0000 7f4b7cff9700 1 -- 172.21.15.133:0/1607892760 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b8c00f040 con 0x55823310fdb0 2023-11-27T15:25:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.310+0000 7f4b7cff9700 1 -- 172.21.15.133:0/1607892760 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4b840b31a0 con 0x55823310fdb0 2023-11-27T15:25:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.310+0000 7f4baedfdb80 10 client.15873 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:25:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.310+0000 7f4baedfdb80 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.310+0000 7f4baedfdb80 20 client.15873 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.310+0000 7f4baedfdb80 1 --2- 172.21.15.133:0/1607892760 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x558233179920 0x55823317bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.310+0000 7f4baedfdb80 4 client.15873 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:25:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.310+0000 7f4baedfdb80 1 -- 172.21.15.133:0/1607892760 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55823302f600 con 0x558233179920 2023-11-27T15:25:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.310+0000 7f4b9919b700 1 --2- 172.21.15.133:0/1607892760 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x558233179920 0x55823317bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.311+0000 7f4b7cff9700 10 client.15873 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.311+0000 7f4b9919b700 1 --2- 172.21.15.133:0/1607892760 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x558233179920 0x55823317bd30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=799b1a51854f150a server_cookie=513749734492afcb in_seq=0 out_seq=0 2023-11-27T15:25:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.328+0000 7f4b7cff9700 1 -- 172.21.15.133:0/1607892760 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55823302f600 con 0x558233179920 2023-11-27T15:25:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.328+0000 7f4b7cff9700 10 client.15873 handle_command_reply: tid=0 2023-11-27T15:25:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.328+0000 7f4baedfdb80 1 client.15873 shutdown 2023-11-27T15:25:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.329+0000 7f4baedfdb80 1 -- 172.21.15.133:0/1607892760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55823310fdb0 msgr2=0x55823311c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.329+0000 7f4baedfdb80 1 --2- 172.21.15.133:0/1607892760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55823310fdb0 0x55823311c8d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4b84026040 tx=0x7f4b84009c00 comp rx=0 tx=0).stop 2023-11-27T15:25:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.329+0000 7f4baedfdb80 1 -- 172.21.15.133:0/1607892760 shutdown_connections 2023-11-27T15:25:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.329+0000 7f4baedfdb80 1 --2- 172.21.15.133:0/1607892760 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x558233179920 0x55823317bd30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.329+0000 7f4baedfdb80 1 --2- 172.21.15.133:0/1607892760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558233110b10 0x55823311f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.329+0000 7f4baedfdb80 1 --2- 172.21.15.133:0/1607892760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x558232f6b6b0 0x55823311c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.329+0000 7f4baedfdb80 1 --2- 172.21.15.133:0/1607892760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55823310fdb0 0x55823311c8d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.329+0000 7f4b7cff9700 0 client.15873 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.329+0000 7f4baedfdb80 1 -- 172.21.15.133:0/1607892760 >> 172.21.15.133:0/1607892760 conn(0x558233104f50 msgr2=0x558233116e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.329+0000 7f4baedfdb80 1 -- 172.21.15.133:0/1607892760 shutdown_connections 2023-11-27T15:25:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.329+0000 7f4baedfdb80 1 -- 172.21.15.133:0/1607892760 wait complete. 2023-11-27T15:25:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.329+0000 7f4baedfdb80 20 client.15873 trim_cache size 0 max 16384 2023-11-27T15:25:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:25:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:25:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:25:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:25:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.330+0000 7f4b9b3ff700 1 -- 172.21.15.133:0/3678928540 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4b800a11d0 msgr2=0x7f4b800a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.330+0000 7f4b9b3ff700 1 --2- 172.21.15.133:0/3678928540 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4b800a11d0 0x7f4b800a3680 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4b94074f00 tx=0x7f4b90009260 comp rx=0 tx=0).stop 2023-11-27T15:25:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.330+0000 7f4b9b3ff700 1 -- 172.21.15.133:0/3678928540 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b94074b30 msgr2=0x7f4b941987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.330+0000 7f4b9b3ff700 1 --2- 172.21.15.133:0/3678928540 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b94074b30 0x7f4b941987d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4b84009fd0 tx=0x7f4b8400b790 comp rx=0 tx=0).stop 2023-11-27T15:25:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.330+0000 7f4b9b3ff700 1 -- 172.21.15.133:0/3678928540 shutdown_connections 2023-11-27T15:25:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.330+0000 7f4b9b3ff700 1 --2- 172.21.15.133:0/3678928540 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4b800a11d0 0x7f4b800a3680 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.330+0000 7f4b9b3ff700 1 --2- 172.21.15.133:0/3678928540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b94108100 0x7f4b9419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.330+0000 7f4b9b3ff700 1 --2- 172.21.15.133:0/3678928540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b94073550 0x7f4b94198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.330+0000 7f4b9b3ff700 1 --2- 172.21.15.133:0/3678928540 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4b94074b30 0x7f4b941987d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.330+0000 7f4b9b3ff700 1 -- 172.21.15.133:0/3678928540 >> 172.21.15.133:0/3678928540 conn(0x7f4b940fb840 msgr2=0x7f4b940fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.330+0000 7f4b9b3ff700 1 -- 172.21.15.133:0/3678928540 shutdown_connections 2023-11-27T15:25:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:06.330+0000 7f4b9b3ff700 1 -- 172.21.15.133:0/3678928540 wait complete. 2023-11-27T15:25:06.376 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:25:06.816 INFO:tasks.workunit:Running workunit fs/misc/chmod.sh... 2023-11-27T15:25:06.816 DEBUG:teuthology.orchestra.run.smithi133:workunit test fs/misc/chmod.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh 2023-11-27T15:25:06.916 INFO:tasks.workunit.client.0.smithi133.stderr:+ set -e 2023-11-27T15:25:06.916 INFO:tasks.workunit.client.0.smithi133.stderr:+ file=test_chmod.152856 2023-11-27T15:25:06.917 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo foo 2023-11-27T15:25:06.922 INFO:tasks.workunit.client.0.smithi133.stderr:+ test 0 '!=' 0 2023-11-27T15:25:06.922 INFO:tasks.workunit.client.0.smithi133.stderr:+ chmod 400 test_chmod.152856 2023-11-27T15:25:06.925 INFO:tasks.workunit.client.0.smithi133.stderr:+ test 0 '!=' 0 2023-11-27T15:25:06.925 INFO:tasks.workunit.client.0.smithi133.stderr:+ check_perms test_chmod.152856 -r-------- 2023-11-27T15:25:06.925 INFO:tasks.workunit.client.0.smithi133.stderr:+ file=test_chmod.152856 2023-11-27T15:25:06.926 INFO:tasks.workunit.client.0.smithi133.stderr:++ ls -la test_chmod.152856 2023-11-27T15:25:06.931 INFO:tasks.workunit.client.0.smithi133.stderr:+ r='-r--------. 1 ubuntu ubuntu 4 Nov 27 15:25 test_chmod.152856' 2023-11-27T15:25:06.931 INFO:tasks.workunit.client.0.smithi133.stderr:+ test 0 '!=' 0 2023-11-27T15:25:06.931 INFO:tasks.workunit.client.0.smithi133.stderr:+ perms=-r-------- 2023-11-27T15:25:06.932 INFO:tasks.workunit.client.0.smithi133.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Nov 27 15:25 test_chmod.152856 2023-11-27T15:25:06.933 INFO:tasks.workunit.client.0.smithi133.stderr:++ awk '{print $1}' 2023-11-27T15:25:07.032 INFO:tasks.workunit.client.0.smithi133.stderr:+ test -r-------- '!=' -r--------. 2023-11-27T15:25:07.033 INFO:tasks.workunit.client.0.smithi133.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Nov 27 15:25 test_chmod.152856 2023-11-27T15:25:07.034 INFO:tasks.workunit.client.0.smithi133.stderr:++ awk '{print $1}' 2023-11-27T15:25:07.038 INFO:tasks.workunit.client.0.smithi133.stderr:+ test -r--------. '!=' -r--------. 2023-11-27T15:25:07.038 INFO:tasks.workunit.client.0.smithi133.stderr:+ set +e 2023-11-27T15:25:07.038 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo bar 2023-11-27T15:25:07.038 INFO:tasks.workunit.client.0.smithi133.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh: line 40: test_chmod.152856: Permission denied 2023-11-27T15:25:07.038 INFO:tasks.workunit.client.0.smithi133.stderr:+ test 1 = 0 2023-11-27T15:25:07.038 INFO:tasks.workunit.client.0.smithi133.stderr:+ set -e 2023-11-27T15:25:07.038 INFO:tasks.workunit.client.0.smithi133.stderr:+ chmod 600 test_chmod.152856 2023-11-27T15:25:07.040 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo bar 2023-11-27T15:25:07.040 INFO:tasks.workunit.client.0.smithi133.stderr:+ test 0 '!=' 0 2023-11-27T15:25:07.040 INFO:tasks.workunit.client.0.smithi133.stderr:+ check_perms test_chmod.152856 -rw------- 2023-11-27T15:25:07.040 INFO:tasks.workunit.client.0.smithi133.stderr:+ file=test_chmod.152856 2023-11-27T15:25:07.041 INFO:tasks.workunit.client.0.smithi133.stderr:++ ls -la test_chmod.152856 2023-11-27T15:25:07.043 INFO:tasks.workunit.client.0.smithi133.stderr:+ r='-rw-------. 1 ubuntu ubuntu 8 Nov 27 15:25 test_chmod.152856' 2023-11-27T15:25:07.044 INFO:tasks.workunit.client.0.smithi133.stderr:+ test 0 '!=' 0 2023-11-27T15:25:07.044 INFO:tasks.workunit.client.0.smithi133.stderr:+ perms=-rw------- 2023-11-27T15:25:07.045 INFO:tasks.workunit.client.0.smithi133.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Nov 27 15:25 test_chmod.152856 2023-11-27T15:25:07.045 INFO:tasks.workunit.client.0.smithi133.stderr:++ awk '{print $1}' 2023-11-27T15:25:07.048 INFO:tasks.workunit.client.0.smithi133.stderr:+ test -rw------- '!=' -rw-------. 2023-11-27T15:25:07.050 INFO:tasks.workunit.client.0.smithi133.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Nov 27 15:25 test_chmod.152856 2023-11-27T15:25:07.050 INFO:tasks.workunit.client.0.smithi133.stderr:++ awk '{print $1}' 2023-11-27T15:25:07.054 INFO:tasks.workunit.client.0.smithi133.stderr:+ test -rw-------. '!=' -rw-------. 2023-11-27T15:25:07.054 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo foo 2023-11-27T15:25:07.054 INFO:tasks.workunit.client.0.smithi133.stderr:+ test 0 '!=' 0 2023-11-27T15:25:07.055 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-27T15:25:07.055 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-27T15:25:07.114 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:06 smithi133 ceph-mon[97860]: pgmap v414: 129 pgs: 129 active+clean; 1.7 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 10 op/s 2023-11-27T15:25:07.145 INFO:tasks.workunit:Running workunit fs/misc/dac_override.sh... 2023-11-27T15:25:07.145 DEBUG:teuthology.orchestra.run.smithi133:workunit test fs/misc/dac_override.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dac_override.sh 2023-11-27T15:25:07.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:06 smithi174 ceph-mon[131110]: pgmap v414: 129 pgs: 129 active+clean; 1.7 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 10 op/s 2023-11-27T15:25:07.201 INFO:tasks.workunit.client.0.smithi133.stderr:+ set -e 2023-11-27T15:25:07.201 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir -p testdir 2023-11-27T15:25:07.205 INFO:tasks.workunit.client.0.smithi133.stderr:+ file=test_chmod.152915 2023-11-27T15:25:07.205 INFO:tasks.workunit.client.0.smithi133.stderr:+ echo foo 2023-11-27T15:25:07.206 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo chmod 600 testdir 2023-11-27T15:25:07.240 INFO:tasks.workunit.client.0.smithi133.stderr:+ expect_failure cat testdir/test_chmod.152915 2023-11-27T15:25:07.240 INFO:tasks.workunit.client.0.smithi133.stderr:+ cat testdir/test_chmod.152915 2023-11-27T15:25:07.247 INFO:tasks.workunit.client.0.smithi133.stderr:cat: testdir/test_chmod.152915: Permission denied 2023-11-27T15:25:07.247 INFO:tasks.workunit.client.0.smithi133.stderr:+ return 0 2023-11-27T15:25:07.247 INFO:tasks.workunit.client.0.smithi133.stderr:+ sudo cat testdir/test_chmod.152915 2023-11-27T15:25:07.272 INFO:tasks.workunit.client.0.smithi133.stdout:foo 2023-11-27T15:25:07.276 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-27T15:25:07.276 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-27T15:25:07.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:06 smithi177 ceph-mon[131504]: pgmap v414: 129 pgs: 129 active+clean; 1.7 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.3 KiB/s wr, 10 op/s 2023-11-27T15:25:07.366 INFO:tasks.workunit:Running workunit fs/misc/direct_io.py... 2023-11-27T15:25:07.367 DEBUG:teuthology.orchestra.run.smithi133:workunit test fs/misc/direct_io.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/direct_io.py 2023-11-27T15:25:07.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.492+0000 7efd71fff700 1 -- 172.21.15.133:0/1499433888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd6c104090 msgr2=0x7efd6c10a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:07.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.492+0000 7efd71fff700 1 --2- 172.21.15.133:0/1499433888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd6c104090 0x7efd6c10a950 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efd6400b0a0 tx=0x7efd6400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.492+0000 7efd71fff700 1 -- 172.21.15.133:0/1499433888 shutdown_connections 2023-11-27T15:25:07.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.492+0000 7efd71fff700 1 --2- 172.21.15.133:0/1499433888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd6c104090 0x7efd6c10a950 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.492+0000 7efd71fff700 1 --2- 172.21.15.133:0/1499433888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd6c102510 0x7efd6c102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.492+0000 7efd71fff700 1 --2- 172.21.15.133:0/1499433888 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6c103700 0x7efd6c103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.492+0000 7efd71fff700 1 -- 172.21.15.133:0/1499433888 >> 172.21.15.133:0/1499433888 conn(0x7efd6c0fdb30 msgr2=0x7efd6c0fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:07.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.492+0000 7efd71fff700 1 -- 172.21.15.133:0/1499433888 shutdown_connections 2023-11-27T15:25:07.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.492+0000 7efd71fff700 1 -- 172.21.15.133:0/1499433888 wait complete. 2023-11-27T15:25:07.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.493+0000 7efd71fff700 1 Processor -- start 2023-11-27T15:25:07.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.493+0000 7efd71fff700 1 -- start start 2023-11-27T15:25:07.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.493+0000 7efd71fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6c102510 0x7efd6c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.493+0000 7efd71fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd6c103700 0x7efd6c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.493+0000 7efd71fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd6c104090 0x7efd6c19f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.493+0000 7efd71fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c19fee0 con 0x7efd6c103700 2023-11-27T15:25:07.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.493+0000 7efd71fff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c1a0020 con 0x7efd6c104090 2023-11-27T15:25:07.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.493+0000 7efd71fff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6c1a01b0 con 0x7efd6c102510 2023-11-27T15:25:07.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.494+0000 7efd6affd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd6c103700 0x7efd6c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.494+0000 7efd6b7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6c102510 0x7efd6c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.494+0000 7efd6affd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd6c103700 0x7efd6c1988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59190/0 (socket says 172.21.15.133:59190) 2023-11-27T15:25:07.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.494+0000 7efd6affd700 1 -- 172.21.15.133:0/3213361697 learned_addr learned my addr 172.21.15.133:0/3213361697 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:07.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.494+0000 7efd6bfff700 1 --2- 172.21.15.133:0/3213361697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd6c104090 0x7efd6c19f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.494+0000 7efd6bfff700 1 -- 172.21.15.133:0/3213361697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6c102510 msgr2=0x7efd6c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:07.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.494+0000 7efd6bfff700 1 --2- 172.21.15.133:0/3213361697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6c102510 0x7efd6c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.494+0000 7efd6bfff700 1 -- 172.21.15.133:0/3213361697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd6c103700 msgr2=0x7efd6c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:07.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.494+0000 7efd6bfff700 1 --2- 172.21.15.133:0/3213361697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd6c103700 0x7efd6c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.494+0000 7efd6bfff700 1 -- 172.21.15.133:0/3213361697 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd64009d00 con 0x7efd6c104090 2023-11-27T15:25:07.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.494+0000 7efd6affd700 1 --2- 172.21.15.133:0/3213361697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd6c103700 0x7efd6c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:07.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.494+0000 7efd6bfff700 1 --2- 172.21.15.133:0/3213361697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd6c104090 0x7efd6c19f910 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efd6400b070 tx=0x7efd6400bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:07.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.494+0000 7efd68ff9700 1 -- 172.21.15.133:0/3213361697 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6400e040 con 0x7efd6c104090 2023-11-27T15:25:07.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.495+0000 7efd71fff700 1 -- 172.21.15.133:0/3213361697 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd6c10ded0 con 0x7efd6c104090 2023-11-27T15:25:07.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.495+0000 7efd68ff9700 1 -- 172.21.15.133:0/3213361697 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd64003e10 con 0x7efd6c104090 2023-11-27T15:25:07.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.495+0000 7efd68ff9700 1 -- 172.21.15.133:0/3213361697 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd640052a0 con 0x7efd6c104090 2023-11-27T15:25:07.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.495+0000 7efd71fff700 1 -- 172.21.15.133:0/3213361697 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6c10dd40 con 0x7efd6c104090 2023-11-27T15:25:07.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.499+0000 7efd68ff9700 1 -- 172.21.15.133:0/3213361697 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7efd6401d480 con 0x7efd6c104090 2023-11-27T15:25:07.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.500+0000 7efd8592db80 1 Processor -- start 2023-11-27T15:25:07.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.500+0000 7efd8592db80 1 -- start start 2023-11-27T15:25:07.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.500+0000 7efd8592db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec7c00 0x56101eec8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.500+0000 7efd8592db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec8550 0x56101eec8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.500+0000 7efd8592db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec9110 0x56101eec9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.500+0000 7efd8592db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56101edcd060 con 0x56101eec8550 2023-11-27T15:25:07.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.501+0000 7efd8592db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56101ee1f8a0 con 0x56101eec9110 2023-11-27T15:25:07.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.501+0000 7efd8592db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56101ee18d30 con 0x56101eec7c00 2023-11-27T15:25:07.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.501+0000 7efd6b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec8550 0x56101eec8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.501+0000 7efd68ff9700 1 --2- 172.21.15.133:0/3213361697 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7efd580a11d0 0x7efd580a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.501+0000 7efd6b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec8550 0x56101eec8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59194/0 (socket says 172.21.15.133:59194) 2023-11-27T15:25:07.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.501+0000 7efd6b7fe700 1 -- 172.21.15.133:0/1730340083 learned_addr learned my addr 172.21.15.133:0/1730340083 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:07.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.501+0000 7efd6affd700 1 --2- 172.21.15.133:0/1730340083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec7c00 0x56101eec8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.501+0000 7efd6affd700 1 --2- 172.21.15.133:0/1730340083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec9110 0x56101eec9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.501+0000 7efd68ff9700 1 -- 172.21.15.133:0/3213361697 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7efd640ce000 con 0x7efd6c104090 2023-11-27T15:25:07.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.501+0000 7efd6b7fe700 1 -- 172.21.15.133:0/1730340083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec7c00 msgr2=0x56101eec8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:07.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.501+0000 7efd6bfff700 1 --2- 172.21.15.133:0/3213361697 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7efd580a11d0 0x7efd580a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.501+0000 7efd6b7fe700 1 --2- 172.21.15.133:0/1730340083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec7c00 0x56101eec8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.501+0000 7efd6b7fe700 1 -- 172.21.15.133:0/1730340083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec9110 msgr2=0x56101eec9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:07.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.501+0000 7efd6b7fe700 1 --2- 172.21.15.133:0/1730340083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec9110 0x56101eec9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.501+0000 7efd6b7fe700 1 -- 172.21.15.133:0/1730340083 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56101ec9a630 con 0x56101eec8550 2023-11-27T15:25:07.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.502+0000 7efd6b7fe700 1 --2- 172.21.15.133:0/1730340083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec8550 0x56101eec8b30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009d30 tx=0x7efd5c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7079935fb99033b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:07.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.502+0000 7efd6bfff700 1 --2- 172.21.15.133:0/3213361697 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7efd580a11d0 0x7efd580a3680 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efd64097bc0 tx=0x7efd64009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:07.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.502+0000 7efd61ffb700 1 -- 172.21.15.133:0/1730340083 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c004220 con 0x56101eec8550 2023-11-27T15:25:07.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.502+0000 7efd61ffb700 1 -- 172.21.15.133:0/1730340083 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd5c004380 con 0x56101eec8550 2023-11-27T15:25:07.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.502+0000 7efd61ffb700 1 -- 172.21.15.133:0/1730340083 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c019870 con 0x56101eec8550 2023-11-27T15:25:07.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.502+0000 7efd8592db80 1 -- 172.21.15.133:0/1730340083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec8550 msgr2=0x56101eec8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:07.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.502+0000 7efd8592db80 1 --2- 172.21.15.133:0/1730340083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec8550 0x56101eec8b30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7efd5c009d30 tx=0x7efd5c011720 comp rx=0 tx=0).stop 2023-11-27T15:25:07.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.503+0000 7efd8592db80 1 -- 172.21.15.133:0/1730340083 shutdown_connections 2023-11-27T15:25:07.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.503+0000 7efd8592db80 1 --2- 172.21.15.133:0/1730340083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec9110 0x56101eec9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.503+0000 7efd8592db80 1 --2- 172.21.15.133:0/1730340083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec7c00 0x56101eec8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.503+0000 7efd8592db80 1 --2- 172.21.15.133:0/1730340083 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec8550 0x56101eec8b30 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.503+0000 7efd8592db80 1 -- 172.21.15.133:0/1730340083 >> 172.21.15.133:0/1730340083 conn(0x56101eebd360 msgr2=0x56101eec2670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.503+0000 7efd8592db80 1 -- 172.21.15.133:0/1730340083 shutdown_connections 2023-11-27T15:25:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.503+0000 7efd8592db80 1 -- 172.21.15.133:0/1730340083 wait complete. 2023-11-27T15:25:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.503+0000 7efd8592db80 1 -- start start 2023-11-27T15:25:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.504+0000 7efd8592db80 1 Processor -- start 2023-11-27T15:25:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.504+0000 7efd8592db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec7c00 0x56101eed1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.504+0000 7efd8592db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec8550 0x56101eed1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.504+0000 7efd8592db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec9110 0x56101eed8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.504+0000 7efd8592db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56101edba8e0 con 0x56101eec7c00 2023-11-27T15:25:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.504+0000 7efd8592db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56101edba580 con 0x56101eec8550 2023-11-27T15:25:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.504+0000 7efd8592db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56101ec9c920 con 0x56101eec9110 2023-11-27T15:25:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.504+0000 7efd8592db80 10 client.0 _ll_register_callbacks cb 0x56101edd8780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.505+0000 7efd6affd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec7c00 0x56101eed1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.505+0000 7efd6b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec8550 0x56101eed1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.505+0000 7efd6b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec8550 0x56101eed1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:44566/0 (socket says 172.21.15.133:44566) 2023-11-27T15:25:07.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.505+0000 7efd6b7fe700 1 -- 172.21.15.133:0/2885618940 learned_addr learned my addr 172.21.15.133:0/2885618940 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:07.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.505+0000 7efd6b7fe700 1 -- 172.21.15.133:0/2885618940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec9110 msgr2=0x56101eed8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:07.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.505+0000 7efd6affd700 1 --2- 172.21.15.133:0/2885618940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec9110 0x56101eed8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.505+0000 7efd6b7fe700 1 --2- 172.21.15.133:0/2885618940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec9110 0x56101eed8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.505+0000 7efd6b7fe700 1 -- 172.21.15.133:0/2885618940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec7c00 msgr2=0x56101eed1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:07.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.505+0000 7efd6b7fe700 1 --2- 172.21.15.133:0/2885618940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec7c00 0x56101eed1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.505+0000 7efd6b7fe700 1 -- 172.21.15.133:0/2885618940 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5c009960 con 0x56101eec8550 2023-11-27T15:25:07.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.505+0000 7efd6affd700 1 --2- 172.21.15.133:0/2885618940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec9110 0x56101eed8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:07.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.505+0000 7efd6b7fe700 1 --2- 172.21.15.133:0/2885618940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec8550 0x56101eed1ad0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efd5c003d10 tx=0x7efd5c003e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dd39e8a894d65d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:07.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.505+0000 7efd4affd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:25:07.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.506+0000 7efd4affd700 1 -- 172.21.15.133:0/2885618940 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c005240 con 0x56101eec8550 2023-11-27T15:25:07.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.506+0000 7efd8592db80 1 -- 172.21.15.133:0/2885618940 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56101eebab20 con 0x56101eec8550 2023-11-27T15:25:07.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.506+0000 7efd4affd700 1 -- 172.21.15.133:0/2885618940 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7efd5c003ac0 con 0x56101eec8550 2023-11-27T15:25:07.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.506+0000 7efd4affd700 1 -- 172.21.15.133:0/2885618940 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd5c0053a0 con 0x56101eec8550 2023-11-27T15:25:07.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.506+0000 7efd8592db80 10 client.25925 fetch_fsmap learned FSMap version 23 2023-11-27T15:25:07.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.506+0000 7efd8592db80 1 -- 172.21.15.133:0/2885618940 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56101eddf190 con 0x56101eec8550 2023-11-27T15:25:07.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.506+0000 7efd4affd700 1 -- 172.21.15.133:0/2885618940 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56101eebab20 con 0x56101eec8550 2023-11-27T15:25:07.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.506+0000 7efd4affd700 1 -- 172.21.15.133:0/2885618940 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efd5c021040 con 0x56101eec8550 2023-11-27T15:25:07.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.506+0000 7efd8592db80 10 client.25925 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:25:07.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.506+0000 7efd8592db80 10 client.25925 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:25:07.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.506+0000 7efd8592db80 20 client.25925 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:07.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.506+0000 7efd8592db80 1 --2- 172.21.15.133:0/2885618940 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56101eedd200 0x56101eedf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.506+0000 7efd8592db80 4 client.25925 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:07.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.506+0000 7efd8592db80 1 -- 172.21.15.133:0/2885618940 --> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56101eddb0a0 con 0x56101eedd200 2023-11-27T15:25:07.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.507+0000 7efd6affd700 1 --2- 172.21.15.133:0/2885618940 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56101eedd200 0x56101eedf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.507+0000 7efd6affd700 1 --2- 172.21.15.133:0/2885618940 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56101eedd200 0x56101eedf610 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4cab0ae5b005ec14 server_cookie=b2f3bbff73aadb1b in_seq=0 out_seq=0 2023-11-27T15:25:07.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.507+0000 7efd4affd700 10 client.25925 ms_handle_connect on v2:172.21.15.174:6836/1955090548 2023-11-27T15:25:07.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.508+0000 7efd4affd700 1 -- 172.21.15.133:0/2885618940 <== mds.1 v2:172.21.15.174:6836/1955090548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56101eddb0a0 con 0x56101eedd200 2023-11-27T15:25:07.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.508+0000 7efd4affd700 10 client.25925 handle_command_reply: tid=0 2023-11-27T15:25:07.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.508+0000 7efd8592db80 1 client.25925 shutdown 2023-11-27T15:25:07.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.508+0000 7efd8592db80 1 -- 172.21.15.133:0/2885618940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec8550 msgr2=0x56101eed1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:07.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.508+0000 7efd8592db80 1 --2- 172.21.15.133:0/2885618940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec8550 0x56101eed1ad0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efd5c003d10 tx=0x7efd5c003e60 comp rx=0 tx=0).stop 2023-11-27T15:25:07.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.508+0000 7efd8592db80 1 -- 172.21.15.133:0/2885618940 shutdown_connections 2023-11-27T15:25:07.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.508+0000 7efd8592db80 1 --2- 172.21.15.133:0/2885618940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec9110 0x56101eed8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.508+0000 7efd8592db80 1 --2- 172.21.15.133:0/2885618940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec7c00 0x56101eed1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.508+0000 7efd8592db80 1 --2- 172.21.15.133:0/2885618940 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56101eedd200 0x56101eedf610 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.508+0000 7efd8592db80 1 --2- 172.21.15.133:0/2885618940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec8550 0x56101eed1ad0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.508+0000 7efd4affd700 0 client.25925 ms_handle_reset on v2:172.21.15.174:6836/1955090548 2023-11-27T15:25:07.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.508+0000 7efd8592db80 1 -- 172.21.15.133:0/2885618940 >> 172.21.15.133:0/2885618940 conn(0x56101eebd360 msgr2=0x56101eec1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:07.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.508+0000 7efd8592db80 1 -- 172.21.15.133:0/2885618940 shutdown_connections 2023-11-27T15:25:07.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.508+0000 7efd8592db80 1 -- 172.21.15.133:0/2885618940 wait complete. 2023-11-27T15:25:07.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.508+0000 7efd8592db80 20 client.25925 trim_cache size 0 max 16384 2023-11-27T15:25:07.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.522+0000 7efd8592db80 1 Processor -- start 2023-11-27T15:25:07.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.522+0000 7efd8592db80 1 -- start start 2023-11-27T15:25:07.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.522+0000 7efd8592db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec7c00 0x56101eed9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.522+0000 7efd8592db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec8550 0x56101eed8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.522+0000 7efd8592db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec9110 0x56101eed1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.522+0000 7efd8592db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56101eed1130 con 0x56101eec8550 2023-11-27T15:25:07.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.522+0000 7efd8592db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56101eedcd70 con 0x56101eec7c00 2023-11-27T15:25:07.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.522+0000 7efd8592db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56101eed25b0 con 0x56101eec9110 2023-11-27T15:25:07.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.523+0000 7efd6b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec8550 0x56101eed8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.523+0000 7efd6b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec8550 0x56101eed8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59220/0 (socket says 172.21.15.133:59220) 2023-11-27T15:25:07.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.523+0000 7efd6b7fe700 1 -- 172.21.15.133:0/1243675886 learned_addr learned my addr 172.21.15.133:0/1243675886 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:07.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.523+0000 7efd6affd700 1 --2- 172.21.15.133:0/1243675886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec7c00 0x56101eed9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.523+0000 7efd6affd700 1 --2- 172.21.15.133:0/1243675886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec9110 0x56101eed1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.523+0000 7efd6b7fe700 1 -- 172.21.15.133:0/1243675886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec9110 msgr2=0x56101eed1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:07.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.523+0000 7efd6b7fe700 1 --2- 172.21.15.133:0/1243675886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec9110 0x56101eed1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.523+0000 7efd6b7fe700 1 -- 172.21.15.133:0/1243675886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec7c00 msgr2=0x56101eed9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:07.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.523+0000 7efd6b7fe700 1 --2- 172.21.15.133:0/1243675886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec7c00 0x56101eed9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.523+0000 7efd6b7fe700 1 -- 172.21.15.133:0/1243675886 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56101eddeee0 con 0x56101eec8550 2023-11-27T15:25:07.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.523+0000 7efd6affd700 1 --2- 172.21.15.133:0/1243675886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec9110 0x56101eed1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:07.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.523+0000 7efd6b7fe700 1 --2- 172.21.15.133:0/1243675886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec8550 0x56101eed8ad0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7efd5c02fe90 tx=0x7efd5c009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a466e9c22f836f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:07.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.524+0000 7efd627fc700 1 -- 172.21.15.133:0/1243675886 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c003730 con 0x56101eec8550 2023-11-27T15:25:07.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.524+0000 7efd627fc700 1 -- 172.21.15.133:0/1243675886 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd5c009260 con 0x56101eec8550 2023-11-27T15:25:07.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.524+0000 7efd627fc700 1 -- 172.21.15.133:0/1243675886 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c022a60 con 0x56101eec8550 2023-11-27T15:25:07.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.524+0000 7efd8592db80 1 -- 172.21.15.133:0/1243675886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec8550 msgr2=0x56101eed8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:07.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.524+0000 7efd8592db80 1 --2- 172.21.15.133:0/1243675886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec8550 0x56101eed8ad0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7efd5c02fe90 tx=0x7efd5c009d00 comp rx=0 tx=0).stop 2023-11-27T15:25:07.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.524+0000 7efd8592db80 1 -- 172.21.15.133:0/1243675886 shutdown_connections 2023-11-27T15:25:07.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.524+0000 7efd8592db80 1 --2- 172.21.15.133:0/1243675886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec9110 0x56101eed1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.524+0000 7efd8592db80 1 --2- 172.21.15.133:0/1243675886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec7c00 0x56101eed9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.524+0000 7efd8592db80 1 --2- 172.21.15.133:0/1243675886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec8550 0x56101eed8ad0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.524+0000 7efd8592db80 1 -- 172.21.15.133:0/1243675886 >> 172.21.15.133:0/1243675886 conn(0x56101eebd360 msgr2=0x56101eee3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:07.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.525+0000 7efd8592db80 1 -- 172.21.15.133:0/1243675886 shutdown_connections 2023-11-27T15:25:07.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.525+0000 7efd8592db80 1 -- 172.21.15.133:0/1243675886 wait complete. 2023-11-27T15:25:07.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.525+0000 7efd8592db80 1 -- start start 2023-11-27T15:25:07.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.525+0000 7efd8592db80 1 Processor -- start 2023-11-27T15:25:07.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.525+0000 7efd8592db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec7c00 0x56101eed46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.526+0000 7efd8592db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec8550 0x56101eed4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.526+0000 7efd8592db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec9110 0x56101eed7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.526+0000 7efd8592db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56101eed8190 con 0x56101eec8550 2023-11-27T15:25:07.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.526+0000 7efd8592db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56101eed8300 con 0x56101eec9110 2023-11-27T15:25:07.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.526+0000 7efd8592db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56101eed84c0 con 0x56101eec7c00 2023-11-27T15:25:07.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.526+0000 7efd8592db80 10 client.0 _ll_register_callbacks cb 0x7efd5400bd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:07.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.526+0000 7efd6b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec8550 0x56101eed4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.526+0000 7efd6affd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec7c00 0x56101eed46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.526+0000 7efd6b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec8550 0x56101eed4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59232/0 (socket says 172.21.15.133:59232) 2023-11-27T15:25:07.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.526+0000 7efd6b7fe700 1 -- 172.21.15.133:0/1603688211 learned_addr learned my addr 172.21.15.133:0/1603688211 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:07.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.526+0000 7efd6affd700 1 --2- 172.21.15.133:0/1603688211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec9110 0x56101eed7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.526+0000 7efd6b7fe700 1 -- 172.21.15.133:0/1603688211 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec7c00 msgr2=0x56101eed46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:07.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.526+0000 7efd6b7fe700 1 --2- 172.21.15.133:0/1603688211 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec7c00 0x56101eed46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.526+0000 7efd6b7fe700 1 -- 172.21.15.133:0/1603688211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec9110 msgr2=0x56101eed7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:07.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.526+0000 7efd6b7fe700 1 --2- 172.21.15.133:0/1603688211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec9110 0x56101eed7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.526+0000 7efd6b7fe700 1 -- 172.21.15.133:0/1603688211 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56101eddf190 con 0x56101eec8550 2023-11-27T15:25:07.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.526+0000 7efd6affd700 1 --2- 172.21.15.133:0/1603688211 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec7c00 0x56101eed46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:07.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.527+0000 7efd6b7fe700 1 --2- 172.21.15.133:0/1603688211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec8550 0x56101eed4c30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7efd5c003f80 tx=0x7efd5c003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe2e3dbb23295ad server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:07.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.527+0000 7efd4affd700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:25:07.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.527+0000 7efd4affd700 1 -- 172.21.15.133:0/1603688211 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c03d070 con 0x56101eec8550 2023-11-27T15:25:07.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.527+0000 7efd8592db80 1 -- 172.21.15.133:0/1603688211 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd5c004380 con 0x56101eec8550 2023-11-27T15:25:07.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.527+0000 7efd4affd700 1 -- 172.21.15.133:0/1603688211 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7efd5c000e40 con 0x56101eec8550 2023-11-27T15:25:07.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.527+0000 7efd4affd700 1 -- 172.21.15.133:0/1603688211 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd5c019640 con 0x56101eec8550 2023-11-27T15:25:07.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.527+0000 7efd8592db80 10 client.15891 fetch_fsmap learned FSMap version 23 2023-11-27T15:25:07.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.527+0000 7efd8592db80 1 -- 172.21.15.133:0/1603688211 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56101ec2fc60 con 0x56101eec8550 2023-11-27T15:25:07.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.527+0000 7efd4affd700 1 -- 172.21.15.133:0/1603688211 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd5c04b080 con 0x56101eec8550 2023-11-27T15:25:07.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.528+0000 7efd4affd700 1 -- 172.21.15.133:0/1603688211 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efd5c021070 con 0x56101eec8550 2023-11-27T15:25:07.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.528+0000 7efd8592db80 10 client.15891 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:25:07.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.528+0000 7efd8592db80 10 client.15891 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:25:07.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.528+0000 7efd8592db80 20 client.15891 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:07.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.528+0000 7efd8592db80 1 --2- 172.21.15.133:0/1603688211 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56101ef31990 0x56101ef33da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.528+0000 7efd8592db80 4 client.15891 mds_command: new command op to 24521 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:25:07.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.528+0000 7efd8592db80 1 -- 172.21.15.133:0/1603688211 --> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56101ede7660 con 0x56101ef31990 2023-11-27T15:25:07.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.528+0000 7efd6affd700 1 --2- 172.21.15.133:0/1603688211 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56101ef31990 0x56101ef33da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.528+0000 7efd4affd700 10 client.15891 ms_handle_connect on v2:172.21.15.174:6836/1955090548 2023-11-27T15:25:07.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.528+0000 7efd6affd700 1 --2- 172.21.15.133:0/1603688211 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56101ef31990 0x56101ef33da0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a976f6d8dcc9143 server_cookie=f0a4d0116620e289 in_seq=0 out_seq=0 2023-11-27T15:25:07.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.529+0000 7efd4affd700 1 -- 172.21.15.133:0/1603688211 <== mds.1 v2:172.21.15.174:6836/1955090548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56101ede7660 con 0x56101ef31990 2023-11-27T15:25:07.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.529+0000 7efd4affd700 10 client.15891 handle_command_reply: tid=0 2023-11-27T15:25:07.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.529+0000 7efd8592db80 1 client.15891 shutdown 2023-11-27T15:25:07.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.529+0000 7efd8592db80 1 -- 172.21.15.133:0/1603688211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec8550 msgr2=0x56101eed4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:07.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.529+0000 7efd8592db80 1 --2- 172.21.15.133:0/1603688211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec8550 0x56101eed4c30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7efd5c003f80 tx=0x7efd5c003cf0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.529+0000 7efd8592db80 1 -- 172.21.15.133:0/1603688211 shutdown_connections 2023-11-27T15:25:07.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.529+0000 7efd8592db80 1 --2- 172.21.15.133:0/1603688211 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56101ef31990 0x56101ef33da0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.529+0000 7efd8592db80 1 --2- 172.21.15.133:0/1603688211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56101eec9110 0x56101eed7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.529+0000 7efd8592db80 1 --2- 172.21.15.133:0/1603688211 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56101eec7c00 0x56101eed46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.529+0000 7efd4affd700 0 client.15891 ms_handle_reset on v2:172.21.15.174:6836/1955090548 2023-11-27T15:25:07.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.529+0000 7efd8592db80 1 --2- 172.21.15.133:0/1603688211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56101eec8550 0x56101eed4c30 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.529+0000 7efd8592db80 1 -- 172.21.15.133:0/1603688211 >> 172.21.15.133:0/1603688211 conn(0x56101eebd360 msgr2=0x56101eecbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:07.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.529+0000 7efd8592db80 1 -- 172.21.15.133:0/1603688211 shutdown_connections 2023-11-27T15:25:07.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.529+0000 7efd8592db80 1 -- 172.21.15.133:0/1603688211 wait complete. 2023-11-27T15:25:07.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.529+0000 7efd8592db80 20 client.15891 trim_cache size 0 max 16384 2023-11-27T15:25:07.538 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2023-11-27T15:25:07.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.530+0000 7efd71fff700 1 -- 172.21.15.133:0/3213361697 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7efd580a11d0 msgr2=0x7efd580a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:07.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.530+0000 7efd71fff700 1 --2- 172.21.15.133:0/3213361697 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7efd580a11d0 0x7efd580a3680 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7efd64097bc0 tx=0x7efd64009040 comp rx=0 tx=0).stop 2023-11-27T15:25:07.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.530+0000 7efd71fff700 1 -- 172.21.15.133:0/3213361697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd6c104090 msgr2=0x7efd6c19f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:07.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.530+0000 7efd71fff700 1 --2- 172.21.15.133:0/3213361697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd6c104090 0x7efd6c19f910 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efd6400b070 tx=0x7efd6400bc80 comp rx=0 tx=0).stop 2023-11-27T15:25:07.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.530+0000 7efd71fff700 1 -- 172.21.15.133:0/3213361697 shutdown_connections 2023-11-27T15:25:07.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.530+0000 7efd71fff700 1 --2- 172.21.15.133:0/3213361697 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7efd580a11d0 0x7efd580a3680 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.530+0000 7efd71fff700 1 --2- 172.21.15.133:0/3213361697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd6c104090 0x7efd6c19f910 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.530+0000 7efd71fff700 1 --2- 172.21.15.133:0/3213361697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd6c102510 0x7efd6c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.530+0000 7efd71fff700 1 --2- 172.21.15.133:0/3213361697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd6c103700 0x7efd6c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.530+0000 7efd71fff700 1 -- 172.21.15.133:0/3213361697 >> 172.21.15.133:0/3213361697 conn(0x7efd6c0fdb30 msgr2=0x7efd6c0ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:07.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.530+0000 7efd71fff700 1 -- 172.21.15.133:0/3213361697 shutdown_connections 2023-11-27T15:25:07.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.530+0000 7efd71fff700 1 -- 172.21.15.133:0/3213361697 wait complete. 2023-11-27T15:25:07.552 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:25:07.653 INFO:tasks.workunit.client.0.smithi133.stdout:ok 2023-11-27T15:25:07.660 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-27T15:25:07.660 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-27T15:25:07.750 INFO:tasks.workunit:Running workunit fs/misc/dirfrag.sh... 2023-11-27T15:25:07.750 DEBUG:teuthology.orchestra.run.smithi133:workunit test fs/misc/dirfrag.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dirfrag.sh 2023-11-27T15:25:07.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.764+0000 7f01767ff700 1 -- 172.21.15.133:0/2123819796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0170074b30 msgr2=0x7f0170072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:07.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.764+0000 7f01767ff700 1 --2- 172.21.15.133:0/2123819796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0170074b30 0x7f0170072f80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0160009990 tx=0x7f0160009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.764+0000 7f01767ff700 1 -- 172.21.15.133:0/2123819796 shutdown_connections 2023-11-27T15:25:07.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.764+0000 7f01767ff700 1 --2- 172.21.15.133:0/2123819796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f017010c8a0 0x7f017010ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.764+0000 7f01767ff700 1 --2- 172.21.15.133:0/2123819796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0170074b30 0x7f0170072f80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.764+0000 7f01767ff700 1 --2- 172.21.15.133:0/2123819796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0170073550 0x7f01700739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.764+0000 7f01767ff700 1 -- 172.21.15.133:0/2123819796 >> 172.21.15.133:0/2123819796 conn(0x7f01700fb8b0 msgr2=0x7f01700fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:07.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.765+0000 7f01767ff700 1 -- 172.21.15.133:0/2123819796 shutdown_connections 2023-11-27T15:25:07.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.765+0000 7f01767ff700 1 -- 172.21.15.133:0/2123819796 wait complete. 2023-11-27T15:25:07.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.766+0000 7f01767ff700 1 Processor -- start 2023-11-27T15:25:07.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.766+0000 7f01767ff700 1 -- start start 2023-11-27T15:25:07.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.766+0000 7f01767ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0170073550 0x7f017019c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.766+0000 7f01767ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0170074b30 0x7f017019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.766+0000 7f01767ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f017010c8a0 0x7f01701a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.766+0000 7f01767ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01701a4260 con 0x7f0170073550 2023-11-27T15:25:07.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.766+0000 7f01767ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01701a43a0 con 0x7f0170074b30 2023-11-27T15:25:07.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.766+0000 7f01767ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01701a4530 con 0x7f017010c8a0 2023-11-27T15:25:07.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.767+0000 7f016ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0170073550 0x7f017019c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.767+0000 7f016ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0170073550 0x7f017019c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59298/0 (socket says 172.21.15.133:59298) 2023-11-27T15:25:07.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.767+0000 7f0167fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0170074b30 0x7f017019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.767+0000 7f016ffff700 1 -- 172.21.15.133:0/3714199884 learned_addr learned my addr 172.21.15.133:0/3714199884 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:07.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.767+0000 7f0174d9c700 1 --2- 172.21.15.133:0/3714199884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f017010c8a0 0x7f01701a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.767+0000 7f016ffff700 1 -- 172.21.15.133:0/3714199884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f017010c8a0 msgr2=0x7f01701a3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:07.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.767+0000 7f016ffff700 1 --2- 172.21.15.133:0/3714199884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f017010c8a0 0x7f01701a3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.767+0000 7f016ffff700 1 -- 172.21.15.133:0/3714199884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0170074b30 msgr2=0x7f017019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:07.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.767+0000 7f016ffff700 1 --2- 172.21.15.133:0/3714199884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0170074b30 0x7f017019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.767+0000 7f016ffff700 1 -- 172.21.15.133:0/3714199884 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01600095f0 con 0x7f0170073550 2023-11-27T15:25:07.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.767+0000 7f0174d9c700 1 --2- 172.21.15.133:0/3714199884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f017010c8a0 0x7f01701a3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:07.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.767+0000 7f0167fff700 1 --2- 172.21.15.133:0/3714199884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0170074b30 0x7f017019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:07.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.767+0000 7f016ffff700 1 --2- 172.21.15.133:0/3714199884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0170073550 0x7f017019c6e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0160004710 tx=0x7f0160009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:07.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.768+0000 7f016dffb700 1 -- 172.21.15.133:0/3714199884 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016001d070 con 0x7f0170073550 2023-11-27T15:25:07.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.768+0000 7f016dffb700 1 -- 172.21.15.133:0/3714199884 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0160021410 con 0x7f0170073550 2023-11-27T15:25:07.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.768+0000 7f01767ff700 1 -- 172.21.15.133:0/3714199884 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0170112290 con 0x7f0170073550 2023-11-27T15:25:07.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.768+0000 7f016dffb700 1 -- 172.21.15.133:0/3714199884 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0160017650 con 0x7f0170073550 2023-11-27T15:25:07.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.768+0000 7f01767ff700 1 -- 172.21.15.133:0/3714199884 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0170112100 con 0x7f0170073550 2023-11-27T15:25:07.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.771+0000 7f016dffb700 1 -- 172.21.15.133:0/3714199884 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0160021580 con 0x7f0170073550 2023-11-27T15:25:07.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.772+0000 7f018a03bb80 1 Processor -- start 2023-11-27T15:25:07.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.773+0000 7f018a03bb80 1 -- start start 2023-11-27T15:25:07.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.773+0000 7f016dffb700 1 --2- 172.21.15.133:0/3714199884 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f015c0a1150 0x7f015c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.773+0000 7f018a03bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f2c00 0x55ac657f3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.773+0000 7f018a03bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f3550 0x55ac657f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.773+0000 7f018a03bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac657f4110 0x55ac657f4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.773+0000 7f016dffb700 1 -- 172.21.15.133:0/3714199884 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f01600ce0c0 con 0x7f0170073550 2023-11-27T15:25:07.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.773+0000 7f018a03bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac656f8060 con 0x55ac657f4110 2023-11-27T15:25:07.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.773+0000 7f018a03bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac6574a8a0 con 0x55ac657f2c00 2023-11-27T15:25:07.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.773+0000 7f018a03bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac65743d30 con 0x55ac657f3550 2023-11-27T15:25:07.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.773+0000 7f0167fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f3550 0x55ac657f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.773+0000 7f016ffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f2c00 0x55ac657f3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.773+0000 7f0167fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f3550 0x55ac657f3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:36776/0 (socket says 172.21.15.133:36776) 2023-11-27T15:25:07.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.773+0000 7f0167fff700 1 -- 172.21.15.133:0/2959768861 learned_addr learned my addr 172.21.15.133:0/2959768861 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:07.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.773+0000 7f0174d9c700 1 --2- 172.21.15.133:0/2959768861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac657f4110 0x55ac657f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.774+0000 7f0174d9c700 1 --2- 172.21.15.133:0/3714199884 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f015c0a1150 0x7f015c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.774+0000 7f016ffff700 1 -- 172.21.15.133:0/2959768861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f3550 msgr2=0x55ac657f3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:07.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.774+0000 7f016ffff700 1 --2- 172.21.15.133:0/2959768861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f3550 0x55ac657f3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.774+0000 7f016ffff700 1 -- 172.21.15.133:0/2959768861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac657f4110 msgr2=0x55ac657f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:07.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.774+0000 7f016ffff700 1 --2- 172.21.15.133:0/2959768861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac657f4110 0x55ac657f4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.774+0000 7f016ffff700 1 -- 172.21.15.133:0/2959768861 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac655c5630 con 0x55ac657f2c00 2023-11-27T15:25:07.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.774+0000 7f016ffff700 1 --2- 172.21.15.133:0/2959768861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f2c00 0x55ac657f3010 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0160021830 tx=0x7f0160009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=355bbfcedfad18de server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:07.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.774+0000 7f0174d9c700 1 --2- 172.21.15.133:0/3714199884 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f015c0a1150 0x7f015c0a3600 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0168009fd0 tx=0x7f016800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:07.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.774+0000 7f01667fc700 1 -- 172.21.15.133:0/2959768861 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016009ab10 con 0x55ac657f2c00 2023-11-27T15:25:07.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.774+0000 7f01667fc700 1 -- 172.21.15.133:0/2959768861 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f016009b1b0 con 0x55ac657f2c00 2023-11-27T15:25:07.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.774+0000 7f01667fc700 1 -- 172.21.15.133:0/2959768861 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016009b410 con 0x55ac657f2c00 2023-11-27T15:25:07.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.775+0000 7f018a03bb80 1 -- 172.21.15.133:0/2959768861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f2c00 msgr2=0x55ac657f3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:07.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.775+0000 7f018a03bb80 1 --2- 172.21.15.133:0/2959768861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f2c00 0x55ac657f3010 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0160021830 tx=0x7f0160009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.775+0000 7f018a03bb80 1 -- 172.21.15.133:0/2959768861 shutdown_connections 2023-11-27T15:25:07.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.775+0000 7f018a03bb80 1 --2- 172.21.15.133:0/2959768861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac657f4110 0x55ac657f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.775+0000 7f018a03bb80 1 --2- 172.21.15.133:0/2959768861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f2c00 0x55ac657f3010 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.775+0000 7f018a03bb80 1 --2- 172.21.15.133:0/2959768861 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f3550 0x55ac657f3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.775+0000 7f018a03bb80 1 -- 172.21.15.133:0/2959768861 >> 172.21.15.133:0/2959768861 conn(0x55ac657e8360 msgr2=0x55ac657ed670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:07.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.775+0000 7f018a03bb80 1 -- 172.21.15.133:0/2959768861 shutdown_connections 2023-11-27T15:25:07.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.776+0000 7f018a03bb80 1 -- 172.21.15.133:0/2959768861 wait complete. 2023-11-27T15:25:07.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.776+0000 7f018a03bb80 1 -- start start 2023-11-27T15:25:07.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.776+0000 7f018a03bb80 1 Processor -- start 2023-11-27T15:25:07.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.777+0000 7f018a03bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f2c00 0x55ac657fc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.777+0000 7f018a03bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac657f3550 0x55ac657fc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.777+0000 7f018a03bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f4110 0x55ac658037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.777+0000 7f018a03bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac656e58e0 con 0x55ac657f3550 2023-11-27T15:25:07.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.777+0000 7f018a03bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac656e5580 con 0x55ac657f2c00 2023-11-27T15:25:07.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.777+0000 7f018a03bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac655c7920 con 0x55ac657f4110 2023-11-27T15:25:07.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.777+0000 7f018a03bb80 10 client.0 _ll_register_callbacks cb 0x55ac65703780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:07.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.777+0000 7f016ffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f2c00 0x55ac657fc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.777+0000 7f016ffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f2c00 0x55ac657fc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:44680/0 (socket says 172.21.15.133:44680) 2023-11-27T15:25:07.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.777+0000 7f016ffff700 1 -- 172.21.15.133:0/2417766614 learned_addr learned my addr 172.21.15.133:0/2417766614 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:07.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.777+0000 7f0167fff700 1 --2- 172.21.15.133:0/2417766614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac657f3550 0x55ac657fc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.778+0000 7f0174d9c700 1 --2- 172.21.15.133:0/2417766614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f4110 0x55ac658037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.778+0000 7f016ffff700 1 -- 172.21.15.133:0/2417766614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f4110 msgr2=0x55ac658037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:07.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.778+0000 7f016ffff700 1 --2- 172.21.15.133:0/2417766614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f4110 0x55ac658037f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.778+0000 7f016ffff700 1 -- 172.21.15.133:0/2417766614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac657f3550 msgr2=0x55ac657fc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:07.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.778+0000 7f016ffff700 1 --2- 172.21.15.133:0/2417766614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac657f3550 0x55ac657fc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.778+0000 7f016ffff700 1 -- 172.21.15.133:0/2417766614 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0170112100 con 0x55ac657f2c00 2023-11-27T15:25:07.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.778+0000 7f0174d9c700 1 --2- 172.21.15.133:0/2417766614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f4110 0x55ac658037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:07.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.778+0000 7f016ffff700 1 --2- 172.21.15.133:0/2417766614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f2c00 0x55ac657fc270 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f01600976b0 tx=0x7f016009b740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b93e7f528740b68 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:07.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.778+0000 7f014f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:25:07.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.778+0000 7f014f7fe700 1 -- 172.21.15.133:0/2417766614 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01600a3570 con 0x55ac657f2c00 2023-11-27T15:25:07.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.778+0000 7f014f7fe700 1 -- 172.21.15.133:0/2417766614 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f01600a32f0 con 0x55ac657f2c00 2023-11-27T15:25:07.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.778+0000 7f018a03bb80 1 -- 172.21.15.133:0/2417766614 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac657e5b20 con 0x55ac657f2c00 2023-11-27T15:25:07.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.779+0000 7f014f7fe700 1 -- 172.21.15.133:0/2417766614 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f016005d4e0 con 0x55ac657f2c00 2023-11-27T15:25:07.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.779+0000 7f018a03bb80 10 client.25961 fetch_fsmap learned FSMap version 23 2023-11-27T15:25:07.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.779+0000 7f018a03bb80 1 -- 172.21.15.133:0/2417766614 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ac6570a190 con 0x55ac657f2c00 2023-11-27T15:25:07.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.779+0000 7f014f7fe700 1 -- 172.21.15.133:0/2417766614 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ac657e5b20 con 0x55ac657f2c00 2023-11-27T15:25:07.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.779+0000 7f014f7fe700 1 -- 172.21.15.133:0/2417766614 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f016005bf30 con 0x55ac657f2c00 2023-11-27T15:25:07.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.780+0000 7f018a03bb80 10 client.25961 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:25:07.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.780+0000 7f018a03bb80 10 client.25961 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:07.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.780+0000 7f018a03bb80 20 client.25961 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:07.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.780+0000 7f018a03bb80 1 --2- 172.21.15.133:0/2417766614 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ac65807be0 0x55ac65809ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.780+0000 7f018a03bb80 4 client.25961 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:07.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.780+0000 7f018a03bb80 1 -- 172.21.15.133:0/2417766614 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac657060a0 con 0x55ac65807be0 2023-11-27T15:25:07.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.780+0000 7f0167fff700 1 --2- 172.21.15.133:0/2417766614 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ac65807be0 0x55ac65809ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.781+0000 7f0167fff700 1 --2- 172.21.15.133:0/2417766614 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ac65807be0 0x55ac65809ff0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b68110734cbca448 server_cookie=d94b868875b174a in_seq=0 out_seq=0 2023-11-27T15:25:07.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.781+0000 7f014f7fe700 10 client.25961 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:07.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.781+0000 7f014f7fe700 1 -- 172.21.15.133:0/2417766614 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ac657060a0 con 0x55ac65807be0 2023-11-27T15:25:07.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.781+0000 7f014f7fe700 10 client.25961 handle_command_reply: tid=0 2023-11-27T15:25:07.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.781+0000 7f018a03bb80 1 client.25961 shutdown 2023-11-27T15:25:07.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.781+0000 7f018a03bb80 1 -- 172.21.15.133:0/2417766614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f2c00 msgr2=0x55ac657fc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:07.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.781+0000 7f018a03bb80 1 --2- 172.21.15.133:0/2417766614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f2c00 0x55ac657fc270 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f01600976b0 tx=0x7f016009b740 comp rx=0 tx=0).stop 2023-11-27T15:25:07.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.782+0000 7f018a03bb80 1 -- 172.21.15.133:0/2417766614 shutdown_connections 2023-11-27T15:25:07.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.782+0000 7f018a03bb80 1 --2- 172.21.15.133:0/2417766614 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ac65807be0 0x55ac65809ff0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.782+0000 7f018a03bb80 1 --2- 172.21.15.133:0/2417766614 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f4110 0x55ac658037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.782+0000 7f014f7fe700 0 client.25961 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:07.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.782+0000 7f018a03bb80 1 --2- 172.21.15.133:0/2417766614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f2c00 0x55ac657fc270 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.782+0000 7f018a03bb80 1 --2- 172.21.15.133:0/2417766614 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac657f3550 0x55ac657fc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.782+0000 7f018a03bb80 1 -- 172.21.15.133:0/2417766614 >> 172.21.15.133:0/2417766614 conn(0x55ac657e8360 msgr2=0x55ac657ecde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:07.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.782+0000 7f018a03bb80 1 -- 172.21.15.133:0/2417766614 shutdown_connections 2023-11-27T15:25:07.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.782+0000 7f018a03bb80 1 -- 172.21.15.133:0/2417766614 wait complete. 2023-11-27T15:25:07.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.782+0000 7f018a03bb80 20 client.25961 trim_cache size 0 max 16384 2023-11-27T15:25:07.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.795+0000 7f018a03bb80 1 Processor -- start 2023-11-27T15:25:07.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.795+0000 7f018a03bb80 1 -- start start 2023-11-27T15:25:07.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.796+0000 7f018a03bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac657f2c00 0x55ac657fc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.796+0000 7f018a03bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f3550 0x55ac658037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.796+0000 7f018a03bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f4110 0x55ac657fc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.796+0000 7f018a03bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac657fd2c0 con 0x55ac657f2c00 2023-11-27T15:25:07.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.796+0000 7f018a03bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac657f3f90 con 0x55ac657f4110 2023-11-27T15:25:07.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.796+0000 7f018a03bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac657ee300 con 0x55ac657f3550 2023-11-27T15:25:07.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.796+0000 7f0174d9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f4110 0x55ac657fc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.796+0000 7f016ffff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac657f2c00 0x55ac657fc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.800 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.796+0000 7f0174d9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f4110 0x55ac657fc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:44682/0 (socket says 172.21.15.133:44682) 2023-11-27T15:25:07.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.796+0000 7f0174d9c700 1 -- 172.21.15.133:0/1410136426 learned_addr learned my addr 172.21.15.133:0/1410136426 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:07.801 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.796+0000 7f0167fff700 1 --2- 172.21.15.133:0/1410136426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f3550 0x55ac658037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.797+0000 7f0174d9c700 1 -- 172.21.15.133:0/1410136426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f3550 msgr2=0x55ac658037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:07.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.797+0000 7f0174d9c700 1 --2- 172.21.15.133:0/1410136426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f3550 0x55ac658037b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.802 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.797+0000 7f0174d9c700 1 -- 172.21.15.133:0/1410136426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac657f2c00 msgr2=0x55ac657fc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:07.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.797+0000 7f0174d9c700 1 --2- 172.21.15.133:0/1410136426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac657f2c00 0x55ac657fc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.797+0000 7f0174d9c700 1 -- 172.21.15.133:0/1410136426 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac65709ee0 con 0x55ac657f4110 2023-11-27T15:25:07.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.797+0000 7f0174d9c700 1 --2- 172.21.15.133:0/1410136426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f4110 0x55ac657fc560 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f016800e880 tx=0x7f0168011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b231c0bd78057340 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:07.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.797+0000 7f0166ffd700 1 -- 172.21.15.133:0/1410136426 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0168007ea0 con 0x55ac657f4110 2023-11-27T15:25:07.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.797+0000 7f0166ffd700 1 -- 172.21.15.133:0/1410136426 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f016800f040 con 0x55ac657f4110 2023-11-27T15:25:07.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.797+0000 7f0166ffd700 1 -- 172.21.15.133:0/1410136426 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0168024880 con 0x55ac657f4110 2023-11-27T15:25:07.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.798+0000 7f018a03bb80 1 -- 172.21.15.133:0/1410136426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f4110 msgr2=0x55ac657fc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:07.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.798+0000 7f018a03bb80 1 --2- 172.21.15.133:0/1410136426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f4110 0x55ac657fc560 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f016800e880 tx=0x7f0168011000 comp rx=0 tx=0).stop 2023-11-27T15:25:07.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.798+0000 7f018a03bb80 1 -- 172.21.15.133:0/1410136426 shutdown_connections 2023-11-27T15:25:07.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.798+0000 7f018a03bb80 1 --2- 172.21.15.133:0/1410136426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f4110 0x55ac657fc560 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.798+0000 7f018a03bb80 1 --2- 172.21.15.133:0/1410136426 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac657f2c00 0x55ac657fc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.803 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.798+0000 7f018a03bb80 1 --2- 172.21.15.133:0/1410136426 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f3550 0x55ac658037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.798+0000 7f018a03bb80 1 -- 172.21.15.133:0/1410136426 >> 172.21.15.133:0/1410136426 conn(0x55ac657e8360 msgr2=0x55ac657f6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:07.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.798+0000 7f018a03bb80 1 -- 172.21.15.133:0/1410136426 shutdown_connections 2023-11-27T15:25:07.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.798+0000 7f018a03bb80 1 -- 172.21.15.133:0/1410136426 wait complete. 2023-11-27T15:25:07.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.799+0000 7f018a03bb80 1 -- start start 2023-11-27T15:25:07.804 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.799+0000 7f018a03bb80 1 Processor -- start 2023-11-27T15:25:07.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.800+0000 7f018a03bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f2c00 0x55ac657ff8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.800+0000 7f018a03bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f3550 0x55ac657ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.800+0000 7f018a03bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac657f4110 0x55ac65802e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.800+0000 7f018a03bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac65803360 con 0x55ac657f4110 2023-11-27T15:25:07.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.800+0000 7f018a03bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac658034d0 con 0x55ac657f3550 2023-11-27T15:25:07.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.800+0000 7f018a03bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac65803690 con 0x55ac657f2c00 2023-11-27T15:25:07.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.800+0000 7f018a03bb80 10 client.0 _ll_register_callbacks cb 0x55ac65703780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:07.807 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.800+0000 7f016ffff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f2c00 0x55ac657ff8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.800+0000 7f0167fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f3550 0x55ac657ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.800+0000 7f0174d9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac657f4110 0x55ac65802e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.800+0000 7f0167fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f3550 0x55ac657ffe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:44684/0 (socket says 172.21.15.133:44684) 2023-11-27T15:25:07.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.800+0000 7f016ffff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f2c00 0x55ac657ff8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:36782/0 (socket says 172.21.15.133:36782) 2023-11-27T15:25:07.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.800+0000 7f016ffff700 1 -- 172.21.15.133:0/2423039532 learned_addr learned my addr 172.21.15.133:0/2423039532 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:07.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.800+0000 7f016ffff700 1 -- 172.21.15.133:0/2423039532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f3550 msgr2=0x55ac657ffe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:07.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.800+0000 7f016ffff700 1 --2- 172.21.15.133:0/2423039532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f3550 0x55ac657ffe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.800+0000 7f016ffff700 1 -- 172.21.15.133:0/2423039532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac657f4110 msgr2=0x55ac65802e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:07.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.800+0000 7f016ffff700 1 --2- 172.21.15.133:0/2423039532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac657f4110 0x55ac65802e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.800+0000 7f016ffff700 1 -- 172.21.15.133:0/2423039532 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0168020040 con 0x55ac657f2c00 2023-11-27T15:25:07.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.800+0000 7f0167fff700 1 --2- 172.21.15.133:0/2423039532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f3550 0x55ac657ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:07.808 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.801+0000 7f016ffff700 1 --2- 172.21.15.133:0/2423039532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f2c00 0x55ac657ff8c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f016000b2e0 tx=0x7f016000ff50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=489a4343ed05956 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:07.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.801+0000 7f014f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:25:07.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.801+0000 7f014f7fe700 1 -- 172.21.15.133:0/2423039532 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016009ab10 con 0x55ac657f2c00 2023-11-27T15:25:07.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.801+0000 7f014f7fe700 1 -- 172.21.15.133:0/2423039532 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f01600a3570 con 0x55ac657f2c00 2023-11-27T15:25:07.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.801+0000 7f018a03bb80 1 -- 172.21.15.133:0/2423039532 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f016009b1b0 con 0x55ac657f2c00 2023-11-27T15:25:07.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.801+0000 7f014f7fe700 1 -- 172.21.15.133:0/2423039532 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01600971b0 con 0x55ac657f2c00 2023-11-27T15:25:07.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.801+0000 7f018a03bb80 10 client.25954 fetch_fsmap learned FSMap version 23 2023-11-27T15:25:07.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.801+0000 7f018a03bb80 1 -- 172.21.15.133:0/2423039532 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ac6555ac60 con 0x55ac657f2c00 2023-11-27T15:25:07.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.801+0000 7f014f7fe700 1 -- 172.21.15.133:0/2423039532 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f016009b1b0 con 0x55ac657f2c00 2023-11-27T15:25:07.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.802+0000 7f014f7fe700 1 -- 172.21.15.133:0/2423039532 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f01600b90d0 con 0x55ac657f2c00 2023-11-27T15:25:07.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.802+0000 7f018a03bb80 10 client.25954 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:25:07.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.802+0000 7f018a03bb80 10 client.25954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:07.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.802+0000 7f018a03bb80 20 client.25954 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:07.809 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.802+0000 7f018a03bb80 1 --2- 172.21.15.133:0/2423039532 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ac6585c710 0x55ac6585eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:07.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.802+0000 7f018a03bb80 4 client.25954 mds_command: new command op to 24560 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:25:07.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.802+0000 7f018a03bb80 1 -- 172.21.15.133:0/2423039532 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ac65712660 con 0x55ac6585c710 2023-11-27T15:25:07.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.802+0000 7f0167fff700 1 --2- 172.21.15.133:0/2423039532 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ac6585c710 0x55ac6585eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:07.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.803+0000 7f0167fff700 1 --2- 172.21.15.133:0/2423039532 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ac6585c710 0x55ac6585eaf0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ffc25dba66c211 server_cookie=acadfcab72998a64 in_seq=0 out_seq=0 2023-11-27T15:25:07.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.803+0000 7f014f7fe700 10 client.25954 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:07.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.803+0000 7f014f7fe700 1 -- 172.21.15.133:0/2423039532 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ac65712660 con 0x55ac6585c710 2023-11-27T15:25:07.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.803+0000 7f014f7fe700 10 client.25954 handle_command_reply: tid=0 2023-11-27T15:25:07.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.803+0000 7f018a03bb80 1 client.25954 shutdown 2023-11-27T15:25:07.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.803+0000 7f018a03bb80 1 -- 172.21.15.133:0/2423039532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f2c00 msgr2=0x55ac657ff8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:07.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.803+0000 7f018a03bb80 1 --2- 172.21.15.133:0/2423039532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f2c00 0x55ac657ff8c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f016000b2e0 tx=0x7f016000ff50 comp rx=0 tx=0).stop 2023-11-27T15:25:07.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.803+0000 7f018a03bb80 1 -- 172.21.15.133:0/2423039532 shutdown_connections 2023-11-27T15:25:07.810 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.803+0000 7f018a03bb80 1 --2- 172.21.15.133:0/2423039532 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55ac6585c710 0x55ac6585eaf0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.803+0000 7f018a03bb80 1 --2- 172.21.15.133:0/2423039532 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ac657f4110 0x55ac65802e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.803+0000 7f018a03bb80 1 --2- 172.21.15.133:0/2423039532 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ac657f2c00 0x55ac657ff8c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.803+0000 7f018a03bb80 1 --2- 172.21.15.133:0/2423039532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac657f3550 0x55ac657ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.803+0000 7f014f7fe700 0 client.25954 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:07.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.803+0000 7f018a03bb80 1 -- 172.21.15.133:0/2423039532 >> 172.21.15.133:0/2423039532 conn(0x55ac657e8360 msgr2=0x55ac657fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:07.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.803+0000 7f018a03bb80 1 -- 172.21.15.133:0/2423039532 shutdown_connections 2023-11-27T15:25:07.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.803+0000 7f018a03bb80 1 -- 172.21.15.133:0/2423039532 wait complete. 2023-11-27T15:25:07.811 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.803+0000 7f018a03bb80 20 client.25954 trim_cache size 0 max 16384 2023-11-27T15:25:07.811 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2023-11-27T15:25:07.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.804+0000 7f01767ff700 1 -- 172.21.15.133:0/3714199884 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f015c0a1150 msgr2=0x7f015c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:07.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.804+0000 7f01767ff700 1 --2- 172.21.15.133:0/3714199884 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f015c0a1150 0x7f015c0a3600 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0168009fd0 tx=0x7f016800b040 comp rx=0 tx=0).stop 2023-11-27T15:25:07.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.804+0000 7f01767ff700 1 -- 172.21.15.133:0/3714199884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0170073550 msgr2=0x7f017019c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:07.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.804+0000 7f01767ff700 1 --2- 172.21.15.133:0/3714199884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0170073550 0x7f017019c6e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0160004710 tx=0x7f0160009580 comp rx=0 tx=0).stop 2023-11-27T15:25:07.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.804+0000 7f01767ff700 1 -- 172.21.15.133:0/3714199884 shutdown_connections 2023-11-27T15:25:07.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.804+0000 7f01767ff700 1 --2- 172.21.15.133:0/3714199884 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f015c0a1150 0x7f015c0a3600 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.812 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.804+0000 7f01767ff700 1 --2- 172.21.15.133:0/3714199884 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f017010c8a0 0x7f01701a3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.804+0000 7f01767ff700 1 --2- 172.21.15.133:0/3714199884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0170073550 0x7f017019c6e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.804+0000 7f01767ff700 1 --2- 172.21.15.133:0/3714199884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0170074b30 0x7f017019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:07.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.804+0000 7f01767ff700 1 -- 172.21.15.133:0/3714199884 >> 172.21.15.133:0/3714199884 conn(0x7f01700fb8b0 msgr2=0x7f01700fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:07.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.804+0000 7f01767ff700 1 -- 172.21.15.133:0/3714199884 shutdown_connections 2023-11-27T15:25:07.813 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:07.804+0000 7f01767ff700 1 -- 172.21.15.133:0/3714199884 wait complete. 2023-11-27T15:25:07.831 INFO:tasks.workunit.client.0.smithi133.stdout:creating folder hierarchy 2023-11-27T15:25:08.827 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:25:08.828 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:25:09.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:08 smithi133 ceph-mon[97860]: pgmap v415: 129 pgs: 129 active+clean; 1.8 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-11-27T15:25:09.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.028+0000 7f9c01dff700 1 -- 172.21.15.133:0/1883709835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bfc1014b0 msgr2=0x7f9bfc101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:09.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.028+0000 7f9c01dff700 1 --2- 172.21.15.133:0/1883709835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bfc1014b0 0x7f9bfc101900 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9be4009990 tx=0x7f9be4009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.028+0000 7f9c01dff700 1 -- 172.21.15.133:0/1883709835 shutdown_connections 2023-11-27T15:25:09.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.028+0000 7f9c01dff700 1 --2- 172.21.15.133:0/1883709835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bfc101e40 0x7f9bfc108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.028+0000 7f9c01dff700 1 --2- 172.21.15.133:0/1883709835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bfc1002c0 0x7f9bfc1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.028+0000 7f9c01dff700 1 --2- 172.21.15.133:0/1883709835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bfc1014b0 0x7f9bfc101900 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.028+0000 7f9c01dff700 1 -- 172.21.15.133:0/1883709835 >> 172.21.15.133:0/1883709835 conn(0x7f9bfc0fb8a0 msgr2=0x7f9bfc0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:09.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.028+0000 7f9c01dff700 1 -- 172.21.15.133:0/1883709835 shutdown_connections 2023-11-27T15:25:09.033 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.028+0000 7f9c01dff700 1 -- 172.21.15.133:0/1883709835 wait complete. 2023-11-27T15:25:09.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.029+0000 7f9c01dff700 1 Processor -- start 2023-11-27T15:25:09.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.029+0000 7f9c01dff700 1 -- start start 2023-11-27T15:25:09.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.029+0000 7f9c01dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bfc1002c0 0x7f9bfc198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.030+0000 7f9c01dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bfc1014b0 0x7f9bfc1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.030+0000 7f9c01dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bfc101e40 0x7f9bfc19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.030+0000 7f9c01dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc19fde0 con 0x7f9bfc101e40 2023-11-27T15:25:09.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.030+0000 7f9c01dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc19ff20 con 0x7f9bfc1014b0 2023-11-27T15:25:09.035 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.030+0000 7f9c01dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bfc1a00b0 con 0x7f9bfc1002c0 2023-11-27T15:25:09.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.030+0000 7f9bfbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bfc101e40 0x7f9bfc19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.030+0000 7f9bfbfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bfc101e40 0x7f9bfc19f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59356/0 (socket says 172.21.15.133:59356) 2023-11-27T15:25:09.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.030+0000 7f9bfbfff700 1 -- 172.21.15.133:0/1138836159 learned_addr learned my addr 172.21.15.133:0/1138836159 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:09.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.030+0000 7f9bfb7fe700 1 --2- 172.21.15.133:0/1138836159 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bfc1002c0 0x7f9bfc198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.030+0000 7f9bf3fff700 1 --2- 172.21.15.133:0/1138836159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bfc1014b0 0x7f9bfc1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.030+0000 7f9bfbfff700 1 -- 172.21.15.133:0/1138836159 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bfc1002c0 msgr2=0x7f9bfc198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:09.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.030+0000 7f9bfbfff700 1 --2- 172.21.15.133:0/1138836159 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bfc1002c0 0x7f9bfc198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.030+0000 7f9bfbfff700 1 -- 172.21.15.133:0/1138836159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bfc1014b0 msgr2=0x7f9bfc1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:09.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.030+0000 7f9bfbfff700 1 --2- 172.21.15.133:0/1138836159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bfc1014b0 0x7f9bfc1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.030+0000 7f9bfbfff700 1 -- 172.21.15.133:0/1138836159 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec009520 con 0x7f9bfc101e40 2023-11-27T15:25:09.036 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.030+0000 7f9bfbfff700 1 --2- 172.21.15.133:0/1138836159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bfc101e40 0x7f9bfc19f810 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400c370 tx=0x7f9bf400c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:09.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.031+0000 7f9bf97fa700 1 -- 172.21.15.133:0/1138836159 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4017070 con 0x7f9bfc101e40 2023-11-27T15:25:09.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.031+0000 7f9bf97fa700 1 -- 172.21.15.133:0/1138836159 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bf400f040 con 0x7f9bfc101e40 2023-11-27T15:25:09.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.031+0000 7f9c01dff700 1 -- 172.21.15.133:0/1138836159 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf4009c40 con 0x7f9bfc101e40 2023-11-27T15:25:09.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.031+0000 7f9bf97fa700 1 -- 172.21.15.133:0/1138836159 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf400e050 con 0x7f9bfc101e40 2023-11-27T15:25:09.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.031+0000 7f9c01dff700 1 -- 172.21.15.133:0/1138836159 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be40095f0 con 0x7f9bfc101e40 2023-11-27T15:25:09.037 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.034+0000 7f9bf97fa700 1 -- 172.21.15.133:0/1138836159 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f9bf400cb60 con 0x7f9bfc101e40 2023-11-27T15:25:09.040 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.035+0000 7f9c15641b80 1 Processor -- start 2023-11-27T15:25:09.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.035+0000 7f9c15641b80 1 -- start start 2023-11-27T15:25:09.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.036+0000 7f9bf97fa700 1 --2- 172.21.15.133:0/1138836159 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9be80a1150 0x7f9be80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.036+0000 7f9c15641b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50ac80 0x55999b50b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.036+0000 7f9c15641b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50b5d0 0x55999b50bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.036+0000 7f9c15641b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50c190 0x55999b50c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.036+0000 7f9c15641b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999b4100e0 con 0x55999b50b5d0 2023-11-27T15:25:09.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.036+0000 7f9c15641b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999b462920 con 0x55999b50c190 2023-11-27T15:25:09.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.036+0000 7f9c15641b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999b45bdb0 con 0x55999b50ac80 2023-11-27T15:25:09.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.036+0000 7f9bf97fa700 1 -- 172.21.15.133:0/1138836159 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f9bf40cc640 con 0x7f9bfc101e40 2023-11-27T15:25:09.041 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.036+0000 7f9bf3fff700 1 --2- 172.21.15.133:0/1138836159 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9be80a1150 0x7f9be80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.036+0000 7f9bfbfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50c190 0x55999b50c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.036+0000 7f9bf3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50b5d0 0x55999b50bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.036+0000 7f9bfb7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50ac80 0x55999b50b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.036+0000 7f9bfbfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50c190 0x55999b50c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:44704/0 (socket says 172.21.15.133:44704) 2023-11-27T15:25:09.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.036+0000 7f9bfbfff700 1 -- 172.21.15.133:0/3881666934 learned_addr learned my addr 172.21.15.133:0/3881666934 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:09.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.036+0000 7f9bf3fff700 1 --2- 172.21.15.133:0/3881666934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50b5d0 0x55999b50bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59362/0 (socket says 172.21.15.133:59362) 2023-11-27T15:25:09.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.036+0000 7f9bfb7fe700 1 -- 172.21.15.133:0/3881666934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50c190 msgr2=0x55999b50c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:09.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.037+0000 7f9bfb7fe700 1 --2- 172.21.15.133:0/3881666934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50c190 0x55999b50c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.037+0000 7f9bfb7fe700 1 -- 172.21.15.133:0/3881666934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50b5d0 msgr2=0x55999b50bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:09.042 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.037+0000 7f9bfb7fe700 1 --2- 172.21.15.133:0/3881666934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50b5d0 0x55999b50bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.037+0000 7f9bfb7fe700 1 -- 172.21.15.133:0/3881666934 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55999b2dd6b0 con 0x55999b50ac80 2023-11-27T15:25:09.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.037+0000 7f9bf3fff700 1 --2- 172.21.15.133:0/1138836159 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9be80a1150 0x7f9be80a3600 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9be400b2e0 tx=0x7f9be4005510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:09.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.037+0000 7f9bfb7fe700 1 --2- 172.21.15.133:0/3881666934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50ac80 0x55999b50b090 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009de0 tx=0x7f9bec011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a573e02f1f39917d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:09.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.037+0000 7f9bf1ffb700 1 -- 172.21.15.133:0/3881666934 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec004250 con 0x55999b50ac80 2023-11-27T15:25:09.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.037+0000 7f9bf1ffb700 1 -- 172.21.15.133:0/3881666934 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bec004cf0 con 0x55999b50ac80 2023-11-27T15:25:09.043 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.037+0000 7f9bf1ffb700 1 -- 172.21.15.133:0/3881666934 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec0055c0 con 0x55999b50ac80 2023-11-27T15:25:09.047 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.038+0000 7f9c15641b80 1 -- 172.21.15.133:0/3881666934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50ac80 msgr2=0x55999b50b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:09.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.038+0000 7f9c15641b80 1 --2- 172.21.15.133:0/3881666934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50ac80 0x55999b50b090 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9bec009de0 tx=0x7f9bec011720 comp rx=0 tx=0).stop 2023-11-27T15:25:09.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.038+0000 7f9c15641b80 1 -- 172.21.15.133:0/3881666934 shutdown_connections 2023-11-27T15:25:09.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.038+0000 7f9c15641b80 1 --2- 172.21.15.133:0/3881666934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50c190 0x55999b50c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.038+0000 7f9c15641b80 1 --2- 172.21.15.133:0/3881666934 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50ac80 0x55999b50b090 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.038+0000 7f9c15641b80 1 --2- 172.21.15.133:0/3881666934 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50b5d0 0x55999b50bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.038+0000 7f9c15641b80 1 -- 172.21.15.133:0/3881666934 >> 172.21.15.133:0/3881666934 conn(0x55999b5003e0 msgr2=0x55999b5056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:09.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.038+0000 7f9c15641b80 1 -- 172.21.15.133:0/3881666934 shutdown_connections 2023-11-27T15:25:09.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.038+0000 7f9c15641b80 1 -- 172.21.15.133:0/3881666934 wait complete. 2023-11-27T15:25:09.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.038+0000 7f9c15641b80 1 -- start start 2023-11-27T15:25:09.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.039+0000 7f9c15641b80 1 Processor -- start 2023-11-27T15:25:09.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.039+0000 7f9c15641b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50ac80 0x55999b5145e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.039+0000 7f9c15641b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50b5d0 0x55999b514b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.039+0000 7f9c15641b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50c190 0x55999b51bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.039+0000 7f9c15641b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999b3fd960 con 0x55999b50ac80 2023-11-27T15:25:09.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.039+0000 7f9c15641b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999b3fd600 con 0x55999b50b5d0 2023-11-27T15:25:09.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.039+0000 7f9c15641b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999b2df9a0 con 0x55999b50c190 2023-11-27T15:25:09.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.039+0000 7f9c15641b80 10 client.0 _ll_register_callbacks cb 0x55999b41b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:09.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.039+0000 7f9bfb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50ac80 0x55999b5145e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.039+0000 7f9bfb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50ac80 0x55999b5145e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59366/0 (socket says 172.21.15.133:59366) 2023-11-27T15:25:09.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.039+0000 7f9bf3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50b5d0 0x55999b514b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.039+0000 7f9bfb7fe700 1 -- 172.21.15.133:0/2211134049 learned_addr learned my addr 172.21.15.133:0/2211134049 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:09.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.039+0000 7f9bfbfff700 1 --2- 172.21.15.133:0/2211134049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50c190 0x55999b51bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.040+0000 7f9bfbfff700 1 -- 172.21.15.133:0/2211134049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50b5d0 msgr2=0x55999b514b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:09.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.040+0000 7f9bfbfff700 1 --2- 172.21.15.133:0/2211134049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50b5d0 0x55999b514b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.040+0000 7f9bfbfff700 1 -- 172.21.15.133:0/2211134049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50ac80 msgr2=0x55999b5145e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:09.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.040+0000 7f9bfbfff700 1 --2- 172.21.15.133:0/2211134049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50ac80 0x55999b5145e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.040+0000 7f9bfbfff700 1 -- 172.21.15.133:0/2211134049 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bec009a40 con 0x55999b50c190 2023-11-27T15:25:09.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.040+0000 7f9bfb7fe700 1 --2- 172.21.15.133:0/2211134049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50ac80 0x55999b5145e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:09.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.040+0000 7f9bfbfff700 1 --2- 172.21.15.133:0/2211134049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50c190 0x55999b51bb60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4095ba0 tx=0x7f9bf40990f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc7b290f219fed92 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:09.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.040+0000 7f9bdaffd700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:25:09.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.040+0000 7f9bdaffd700 1 -- 172.21.15.133:0/2211134049 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf405bea0 con 0x55999b50c190 2023-11-27T15:25:09.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.040+0000 7f9c15641b80 1 -- 172.21.15.133:0/2211134049 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55999b4fdba0 con 0x55999b50c190 2023-11-27T15:25:09.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.040+0000 7f9bdaffd700 1 -- 172.21.15.133:0/2211134049 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f9bf400f1b0 con 0x55999b50c190 2023-11-27T15:25:09.050 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.040+0000 7f9bdaffd700 1 -- 172.21.15.133:0/2211134049 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bf40996a0 con 0x55999b50c190 2023-11-27T15:25:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.040+0000 7f9c15641b80 10 client.25966 fetch_fsmap learned FSMap version 23 2023-11-27T15:25:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.040+0000 7f9c15641b80 1 -- 172.21.15.133:0/2211134049 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55999b422210 con 0x55999b50c190 2023-11-27T15:25:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.041+0000 7f9bdaffd700 1 -- 172.21.15.133:0/2211134049 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55999b4fdba0 con 0x55999b50c190 2023-11-27T15:25:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.041+0000 7f9bdaffd700 1 -- 172.21.15.133:0/2211134049 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9bf405f1a0 con 0x55999b50c190 2023-11-27T15:25:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.041+0000 7f9c15641b80 10 client.25966 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:25:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.041+0000 7f9c15641b80 10 client.25966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.041+0000 7f9c15641b80 20 client.25966 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.041+0000 7f9c15641b80 1 --2- 172.21.15.133:0/2211134049 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55999b520330 0x55999b522710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.041+0000 7f9c15641b80 4 client.25966 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.041+0000 7f9c15641b80 1 -- 172.21.15.133:0/2211134049 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55999b41e120 con 0x55999b520330 2023-11-27T15:25:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.041+0000 7f9bfb7fe700 1 --2- 172.21.15.133:0/2211134049 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55999b520330 0x55999b522710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.051 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.042+0000 7f9bdaffd700 10 client.25966 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:09.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.042+0000 7f9bfb7fe700 1 --2- 172.21.15.133:0/2211134049 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55999b520330 0x55999b522710 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ca9bac443a4005c server_cookie=392fe2348898fea2 in_seq=0 out_seq=0 2023-11-27T15:25:09.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.042+0000 7f9bdaffd700 1 -- 172.21.15.133:0/2211134049 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55999b41e120 con 0x55999b520330 2023-11-27T15:25:09.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.042+0000 7f9bdaffd700 10 client.25966 handle_command_reply: tid=0 2023-11-27T15:25:09.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.043+0000 7f9c15641b80 1 client.25966 shutdown 2023-11-27T15:25:09.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.043+0000 7f9c15641b80 1 -- 172.21.15.133:0/2211134049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50c190 msgr2=0x55999b51bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:09.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.043+0000 7f9c15641b80 1 --2- 172.21.15.133:0/2211134049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50c190 0x55999b51bb60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4095ba0 tx=0x7f9bf40990f0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.043+0000 7f9c15641b80 1 -- 172.21.15.133:0/2211134049 shutdown_connections 2023-11-27T15:25:09.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.043+0000 7f9c15641b80 1 --2- 172.21.15.133:0/2211134049 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55999b520330 0x55999b522710 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.043+0000 7f9c15641b80 1 --2- 172.21.15.133:0/2211134049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50c190 0x55999b51bb60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.043+0000 7f9c15641b80 1 --2- 172.21.15.133:0/2211134049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50ac80 0x55999b5145e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.043+0000 7f9bdaffd700 0 client.25966 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:09.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.043+0000 7f9c15641b80 1 --2- 172.21.15.133:0/2211134049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50b5d0 0x55999b514b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.052 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.043+0000 7f9c15641b80 1 -- 172.21.15.133:0/2211134049 >> 172.21.15.133:0/2211134049 conn(0x55999b5003e0 msgr2=0x55999b504e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:09.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.043+0000 7f9c15641b80 1 -- 172.21.15.133:0/2211134049 shutdown_connections 2023-11-27T15:25:09.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.043+0000 7f9c15641b80 1 -- 172.21.15.133:0/2211134049 wait complete. 2023-11-27T15:25:09.053 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.043+0000 7f9c15641b80 20 client.25966 trim_cache size 0 max 16384 2023-11-27T15:25:09.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.060+0000 7f9c15641b80 1 Processor -- start 2023-11-27T15:25:09.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.060+0000 7f9c15641b80 1 -- start start 2023-11-27T15:25:09.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.061+0000 7f9c15641b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50ac80 0x55999b514390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.061+0000 7f9c15641b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50b5d0 0x55999b51bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.061+0000 7f9c15641b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50c190 0x55999b5148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.061+0000 7f9c15641b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999b514150 con 0x55999b50ac80 2023-11-27T15:25:09.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.061+0000 7f9c15641b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999b506380 con 0x55999b50c190 2023-11-27T15:25:09.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.061+0000 7f9c15641b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999b520110 con 0x55999b50b5d0 2023-11-27T15:25:09.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.061+0000 7f9bfb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50ac80 0x55999b514390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.061+0000 7f9bfb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50ac80 0x55999b514390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59382/0 (socket says 172.21.15.133:59382) 2023-11-27T15:25:09.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.061+0000 7f9bfb7fe700 1 -- 172.21.15.133:0/805247714 learned_addr learned my addr 172.21.15.133:0/805247714 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:09.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.061+0000 7f9bfbfff700 1 --2- 172.21.15.133:0/805247714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50c190 0x55999b5148d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.061+0000 7f9bf3fff700 1 --2- 172.21.15.133:0/805247714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50b5d0 0x55999b51bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.061+0000 7f9bfb7fe700 1 -- 172.21.15.133:0/805247714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50b5d0 msgr2=0x55999b51bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:09.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.061+0000 7f9bfb7fe700 1 --2- 172.21.15.133:0/805247714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50b5d0 0x55999b51bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.061+0000 7f9bfb7fe700 1 -- 172.21.15.133:0/805247714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50c190 msgr2=0x55999b5148d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:09.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.061+0000 7f9bfb7fe700 1 --2- 172.21.15.133:0/805247714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50c190 0x55999b5148d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.061+0000 7f9bfb7fe700 1 -- 172.21.15.133:0/805247714 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55999b501b80 con 0x55999b50ac80 2023-11-27T15:25:09.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.061+0000 7f9bfb7fe700 1 --2- 172.21.15.133:0/805247714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50ac80 0x55999b514390 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9bec011a70 tx=0x7f9bec005630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44d450de8af8a4e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:09.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.062+0000 7f9bf27fc700 1 -- 172.21.15.133:0/805247714 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec029070 con 0x55999b50ac80 2023-11-27T15:25:09.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.062+0000 7f9bf27fc700 1 -- 172.21.15.133:0/805247714 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bec004f40 con 0x55999b50ac80 2023-11-27T15:25:09.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.062+0000 7f9bf27fc700 1 -- 172.21.15.133:0/805247714 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec003680 con 0x55999b50ac80 2023-11-27T15:25:09.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.062+0000 7f9c15641b80 1 -- 172.21.15.133:0/805247714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50ac80 msgr2=0x55999b514390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:09.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.062+0000 7f9c15641b80 1 --2- 172.21.15.133:0/805247714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50ac80 0x55999b514390 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9bec011a70 tx=0x7f9bec005630 comp rx=0 tx=0).stop 2023-11-27T15:25:09.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.062+0000 7f9c15641b80 1 -- 172.21.15.133:0/805247714 shutdown_connections 2023-11-27T15:25:09.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.062+0000 7f9c15641b80 1 --2- 172.21.15.133:0/805247714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50c190 0x55999b5148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.062+0000 7f9c15641b80 1 --2- 172.21.15.133:0/805247714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50ac80 0x55999b514390 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.062+0000 7f9c15641b80 1 --2- 172.21.15.133:0/805247714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50b5d0 0x55999b51bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.062+0000 7f9c15641b80 1 -- 172.21.15.133:0/805247714 >> 172.21.15.133:0/805247714 conn(0x55999b5003e0 msgr2=0x55999b51c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:09.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.063+0000 7f9c15641b80 1 -- 172.21.15.133:0/805247714 shutdown_connections 2023-11-27T15:25:09.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.063+0000 7f9c15641b80 1 -- 172.21.15.133:0/805247714 wait complete. 2023-11-27T15:25:09.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.063+0000 7f9c15641b80 1 -- start start 2023-11-27T15:25:09.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.063+0000 7f9c15641b80 1 Processor -- start 2023-11-27T15:25:09.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.063+0000 7f9c15641b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50ac80 0x55999b51ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.063+0000 7f9c15641b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50b5d0 0x55999b518500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.063+0000 7f9c15641b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50c190 0x55999b518a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.063+0000 7f9c15641b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999b519110 con 0x55999b50b5d0 2023-11-27T15:25:09.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.063+0000 7f9c15641b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999b5741b0 con 0x55999b50ac80 2023-11-27T15:25:09.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.063+0000 7f9c15641b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55999b574340 con 0x55999b50c190 2023-11-27T15:25:09.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.063+0000 7f9c15641b80 10 client.0 _ll_register_callbacks cb 0x55999b41b800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:09.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.064+0000 7f9bfb7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50ac80 0x55999b51ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.064+0000 7f9bf3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50b5d0 0x55999b518500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.064+0000 7f9bfb7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50ac80 0x55999b51ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:44736/0 (socket says 172.21.15.133:44736) 2023-11-27T15:25:09.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.064+0000 7f9bfb7fe700 1 -- 172.21.15.133:0/3496487025 learned_addr learned my addr 172.21.15.133:0/3496487025 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:09.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.064+0000 7f9bfbfff700 1 --2- 172.21.15.133:0/3496487025 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50c190 0x55999b518a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.064+0000 7f9bfb7fe700 1 -- 172.21.15.133:0/3496487025 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50c190 msgr2=0x55999b518a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:09.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.064+0000 7f9bfb7fe700 1 --2- 172.21.15.133:0/3496487025 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50c190 0x55999b518a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.064+0000 7f9bfb7fe700 1 -- 172.21.15.133:0/3496487025 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50b5d0 msgr2=0x55999b518500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:09.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.064+0000 7f9bfb7fe700 1 --2- 172.21.15.133:0/3496487025 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50b5d0 0x55999b518500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.064+0000 7f9bfb7fe700 1 -- 172.21.15.133:0/3496487025 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bec01d040 con 0x55999b50ac80 2023-11-27T15:25:09.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.064+0000 7f9bfbfff700 1 --2- 172.21.15.133:0/3496487025 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50c190 0x55999b518a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:09.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.064+0000 7f9bf3fff700 1 --2- 172.21.15.133:0/3496487025 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50b5d0 0x55999b518500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:09.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.064+0000 7f9bfb7fe700 1 --2- 172.21.15.133:0/3496487025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50ac80 0x55999b51ba60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9bec011760 tx=0x7f9bec009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1e6a727d43a8a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:09.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.064+0000 7f9bdaffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:25:09.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.065+0000 7f9bdaffd700 1 -- 172.21.15.133:0/3496487025 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec029040 con 0x55999b50ac80 2023-11-27T15:25:09.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.065+0000 7f9c15641b80 1 -- 172.21.15.133:0/3496487025 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bec004cf0 con 0x55999b50ac80 2023-11-27T15:25:09.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.065+0000 7f9bdaffd700 1 -- 172.21.15.133:0/3496487025 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f9bec004710 con 0x55999b50ac80 2023-11-27T15:25:09.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.065+0000 7f9bdaffd700 1 -- 172.21.15.133:0/3496487025 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bec0048d0 con 0x55999b50ac80 2023-11-27T15:25:09.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.065+0000 7f9c15641b80 10 client.25985 fetch_fsmap learned FSMap version 23 2023-11-27T15:25:09.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.065+0000 7f9c15641b80 1 -- 172.21.15.133:0/3496487025 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55999b421f60 con 0x55999b50ac80 2023-11-27T15:25:09.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.065+0000 7f9bdaffd700 1 -- 172.21.15.133:0/3496487025 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bec004cf0 con 0x55999b50ac80 2023-11-27T15:25:09.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.066+0000 7f9bdaffd700 1 -- 172.21.15.133:0/3496487025 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9bec037440 con 0x55999b50ac80 2023-11-27T15:25:09.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.066+0000 7f9c15641b80 10 client.25985 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:25:09.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.066+0000 7f9c15641b80 10 client.25985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:09.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.066+0000 7f9c15641b80 20 client.25985 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:09.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.066+0000 7f9c15641b80 1 --2- 172.21.15.133:0/3496487025 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55999b575a60 0x55999b577e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.066+0000 7f9c15641b80 4 client.25985 mds_command: new command op to 24560 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:25:09.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.066+0000 7f9c15641b80 1 -- 172.21.15.133:0/3496487025 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55999b42a6e0 con 0x55999b575a60 2023-11-27T15:25:09.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.066+0000 7f9bf3fff700 1 --2- 172.21.15.133:0/3496487025 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55999b575a60 0x55999b577e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.066+0000 7f9bdaffd700 10 client.25985 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:09.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.066+0000 7f9bf3fff700 1 --2- 172.21.15.133:0/3496487025 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55999b575a60 0x55999b577e70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b57bb60143e82848 server_cookie=d5072aa9beeabe25 in_seq=0 out_seq=0 2023-11-27T15:25:09.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.067+0000 7f9bdaffd700 1 -- 172.21.15.133:0/3496487025 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55999b42a6e0 con 0x55999b575a60 2023-11-27T15:25:09.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.067+0000 7f9bdaffd700 10 client.25985 handle_command_reply: tid=0 2023-11-27T15:25:09.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.067+0000 7f9c15641b80 1 client.25985 shutdown 2023-11-27T15:25:09.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.068+0000 7f9c15641b80 1 -- 172.21.15.133:0/3496487025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50ac80 msgr2=0x55999b51ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:09.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.068+0000 7f9c15641b80 1 --2- 172.21.15.133:0/3496487025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50ac80 0x55999b51ba60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9bec011760 tx=0x7f9bec009260 comp rx=0 tx=0).stop 2023-11-27T15:25:09.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.068+0000 7f9c15641b80 1 -- 172.21.15.133:0/3496487025 shutdown_connections 2023-11-27T15:25:09.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.068+0000 7f9c15641b80 1 --2- 172.21.15.133:0/3496487025 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55999b575a60 0x55999b577e70 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.068+0000 7f9c15641b80 1 --2- 172.21.15.133:0/3496487025 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55999b50c190 0x55999b518a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.068+0000 7f9bdaffd700 0 client.25985 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:09.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.068+0000 7f9c15641b80 1 --2- 172.21.15.133:0/3496487025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55999b50ac80 0x55999b51ba60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.068+0000 7f9c15641b80 1 --2- 172.21.15.133:0/3496487025 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55999b50b5d0 0x55999b518500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.068+0000 7f9c15641b80 1 -- 172.21.15.133:0/3496487025 >> 172.21.15.133:0/3496487025 conn(0x55999b5003e0 msgr2=0x55999b514390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:09.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.068+0000 7f9c15641b80 1 -- 172.21.15.133:0/3496487025 shutdown_connections 2023-11-27T15:25:09.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.068+0000 7f9c15641b80 1 -- 172.21.15.133:0/3496487025 wait complete. 2023-11-27T15:25:09.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.068+0000 7f9c15641b80 20 client.25985 trim_cache size 0 max 16384 2023-11-27T15:25:09.077 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:25:09.077 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2023-11-27T15:25:09.078 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "b65e2907-6261-49f5-a60e-198644c5dc8a", 2023-11-27T15:25:09.078 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2023-11-27T15:25:09.078 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:25:09.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.069+0000 7f9c01dff700 1 -- 172.21.15.133:0/1138836159 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9be80a1150 msgr2=0x7f9be80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:09.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.069+0000 7f9c01dff700 1 --2- 172.21.15.133:0/1138836159 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9be80a1150 0x7f9be80a3600 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9be400b2e0 tx=0x7f9be4005510 comp rx=0 tx=0).stop 2023-11-27T15:25:09.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.069+0000 7f9c01dff700 1 -- 172.21.15.133:0/1138836159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bfc101e40 msgr2=0x7f9bfc19f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:09.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.069+0000 7f9c01dff700 1 --2- 172.21.15.133:0/1138836159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bfc101e40 0x7f9bfc19f810 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9bf400c370 tx=0x7f9bf400c680 comp rx=0 tx=0).stop 2023-11-27T15:25:09.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.069+0000 7f9c01dff700 1 -- 172.21.15.133:0/1138836159 shutdown_connections 2023-11-27T15:25:09.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.069+0000 7f9c01dff700 1 --2- 172.21.15.133:0/1138836159 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f9be80a1150 0x7f9be80a3600 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.069+0000 7f9c01dff700 1 --2- 172.21.15.133:0/1138836159 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9bfc101e40 0x7f9bfc19f810 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.069+0000 7f9c01dff700 1 --2- 172.21.15.133:0/1138836159 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bfc1002c0 0x7f9bfc198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.069+0000 7f9c01dff700 1 --2- 172.21.15.133:0/1138836159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bfc1014b0 0x7f9bfc1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.069+0000 7f9c01dff700 1 -- 172.21.15.133:0/1138836159 >> 172.21.15.133:0/1138836159 conn(0x7f9bfc0fb8a0 msgr2=0x7f9bfc106d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:09.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.069+0000 7f9c01dff700 1 -- 172.21.15.133:0/1138836159 shutdown_connections 2023-11-27T15:25:09.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.069+0000 7f9c01dff700 1 -- 172.21.15.133:0/1138836159 wait complete. 2023-11-27T15:25:09.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:08 smithi177 ceph-mon[131504]: pgmap v415: 129 pgs: 129 active+clean; 1.8 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-11-27T15:25:09.091 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:25:09.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:08 smithi174 ceph-mon[131110]: pgmap v415: 129 pgs: 129 active+clean; 1.8 MiB data, 90 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-11-27T15:25:09.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.296+0000 7f1d67dff700 1 -- 172.21.15.133:0/1408766971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d601003b0 msgr2=0x7f1d60100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:09.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.296+0000 7f1d67dff700 1 --2- 172.21.15.133:0/1408766971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d601003b0 0x7f1d60100800 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009990 tx=0x7f1d50009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.300 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.296+0000 7f1d67dff700 1 -- 172.21.15.133:0/1408766971 shutdown_connections 2023-11-27T15:25:09.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.296+0000 7f1d67dff700 1 --2- 172.21.15.133:0/1408766971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d60100d40 0x7f1d60107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.296+0000 7f1d67dff700 1 --2- 172.21.15.133:0/1408766971 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d600ff1c0 0x7f1d600ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.296+0000 7f1d67dff700 1 --2- 172.21.15.133:0/1408766971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d601003b0 0x7f1d60100800 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.296+0000 7f1d67dff700 1 -- 172.21.15.133:0/1408766971 >> 172.21.15.133:0/1408766971 conn(0x7f1d600fa7c0 msgr2=0x7f1d600fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:09.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.296+0000 7f1d67dff700 1 -- 172.21.15.133:0/1408766971 shutdown_connections 2023-11-27T15:25:09.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.296+0000 7f1d67dff700 1 -- 172.21.15.133:0/1408766971 wait complete. 2023-11-27T15:25:09.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.297+0000 7f1d67dff700 1 Processor -- start 2023-11-27T15:25:09.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.298+0000 7f1d67dff700 1 -- start start 2023-11-27T15:25:09.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.298+0000 7f1d67dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d600ff1c0 0x7f1d60196050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.298+0000 7f1d67dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d601003b0 0x7f1d60196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.303 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.298+0000 7f1d67dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d60100d40 0x7f1d6019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.298+0000 7f1d67dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6019dbd0 con 0x7f1d60100d40 2023-11-27T15:25:09.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.298+0000 7f1d67dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6019dd10 con 0x7f1d600ff1c0 2023-11-27T15:25:09.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.298+0000 7f1d67dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6019dea0 con 0x7f1d601003b0 2023-11-27T15:25:09.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.298+0000 7f1d65b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d600ff1c0 0x7f1d60196050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.298+0000 7f1d65b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d600ff1c0 0x7f1d60196050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:44748/0 (socket says 172.21.15.133:44748) 2023-11-27T15:25:09.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.298+0000 7f1d65b9b700 1 -- 172.21.15.133:0/1500828651 learned_addr learned my addr 172.21.15.133:0/1500828651 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:09.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.298+0000 7f1d6539a700 1 --2- 172.21.15.133:0/1500828651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d601003b0 0x7f1d60196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.298+0000 7f1d6639c700 1 --2- 172.21.15.133:0/1500828651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d60100d40 0x7f1d6019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.298+0000 7f1d65b9b700 1 -- 172.21.15.133:0/1500828651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d601003b0 msgr2=0x7f1d60196590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:09.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.298+0000 7f1d65b9b700 1 --2- 172.21.15.133:0/1500828651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d601003b0 0x7f1d60196590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.304 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.298+0000 7f1d65b9b700 1 -- 172.21.15.133:0/1500828651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d60100d40 msgr2=0x7f1d6019d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:09.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.298+0000 7f1d65b9b700 1 --2- 172.21.15.133:0/1500828651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d60100d40 0x7f1d6019d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.298+0000 7f1d65b9b700 1 -- 172.21.15.133:0/1500828651 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d500095f0 con 0x7f1d600ff1c0 2023-11-27T15:25:09.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.298+0000 7f1d6539a700 1 --2- 172.21.15.133:0/1500828651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d601003b0 0x7f1d60196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:09.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.298+0000 7f1d6639c700 1 --2- 172.21.15.133:0/1500828651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d60100d40 0x7f1d6019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:09.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.299+0000 7f1d65b9b700 1 --2- 172.21.15.133:0/1500828651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d600ff1c0 0x7f1d60196050 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00e970 tx=0x7f1d5c00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:09.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.299+0000 7f1d56ffd700 1 -- 172.21.15.133:0/1500828651 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c009790 con 0x7f1d600ff1c0 2023-11-27T15:25:09.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.299+0000 7f1d56ffd700 1 -- 172.21.15.133:0/1500828651 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d5c0044e0 con 0x7f1d600ff1c0 2023-11-27T15:25:09.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.299+0000 7f1d67dff700 1 -- 172.21.15.133:0/1500828651 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d6010ab70 con 0x7f1d600ff1c0 2023-11-27T15:25:09.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.299+0000 7f1d56ffd700 1 -- 172.21.15.133:0/1500828651 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c003e30 con 0x7f1d600ff1c0 2023-11-27T15:25:09.305 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.299+0000 7f1d67dff700 1 -- 172.21.15.133:0/1500828651 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6010a9e0 con 0x7f1d600ff1c0 2023-11-27T15:25:09.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.302+0000 7f1d56ffd700 1 -- 172.21.15.133:0/1500828651 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1d5c003f90 con 0x7f1d600ff1c0 2023-11-27T15:25:09.306 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.303+0000 7f1d7b717b80 1 Processor -- start 2023-11-27T15:25:09.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.303+0000 7f1d7b717b80 1 -- start start 2023-11-27T15:25:09.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.304+0000 7f1d56ffd700 1 --2- 172.21.15.133:0/1500828651 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1d4c0a11d0 0x7f1d4c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.304+0000 7f1d7b717b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa44c00 0x5638faa45010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.304+0000 7f1d7b717b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa45550 0x5638faa45b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.304+0000 7f1d7b717b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa46110 0x5638faa46580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.309 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.304+0000 7f1d7b717b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638fa94a060 con 0x5638faa45550 2023-11-27T15:25:09.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.304+0000 7f1d7b717b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638fa99c8a0 con 0x5638faa44c00 2023-11-27T15:25:09.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.304+0000 7f1d7b717b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638fa995d30 con 0x5638faa46110 2023-11-27T15:25:09.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.304+0000 7f1d65b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa44c00 0x5638faa45010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.304+0000 7f1d6639c700 1 --2- 172.21.15.133:0/1500828651 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1d4c0a11d0 0x7f1d4c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.304+0000 7f1d65b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa44c00 0x5638faa45010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:44760/0 (socket says 172.21.15.133:44760) 2023-11-27T15:25:09.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.304+0000 7f1d65b9b700 1 -- 172.21.15.133:0/1180808272 learned_addr learned my addr 172.21.15.133:0/1180808272 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:09.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.304+0000 7f1d6539a700 1 --2- 172.21.15.133:0/1180808272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa45550 0x5638faa45b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.304+0000 7f1d56ffd700 1 -- 172.21.15.133:0/1500828651 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1d5c014070 con 0x7f1d600ff1c0 2023-11-27T15:25:09.310 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.304+0000 7f1d6639c700 1 --2- 172.21.15.133:0/1180808272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa46110 0x5638faa46580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.304+0000 7f1d65b9b700 1 -- 172.21.15.133:0/1180808272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa46110 msgr2=0x5638faa46580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:09.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.304+0000 7f1d65b9b700 1 --2- 172.21.15.133:0/1180808272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa46110 0x5638faa46580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.304+0000 7f1d65b9b700 1 -- 172.21.15.133:0/1180808272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa45550 msgr2=0x5638faa45b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:09.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.304+0000 7f1d65b9b700 1 --2- 172.21.15.133:0/1180808272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa45550 0x5638faa45b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.304+0000 7f1d65b9b700 1 -- 172.21.15.133:0/1180808272 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638fa817630 con 0x5638faa44c00 2023-11-27T15:25:09.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.305+0000 7f1d65b9b700 1 --2- 172.21.15.133:0/1180808272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa44c00 0x5638faa45010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0964c0 tx=0x7f1d5c014ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea4ef02c1fbbb332 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:09.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.305+0000 7f1d6639c700 1 --2- 172.21.15.133:0/1500828651 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1d4c0a11d0 0x7f1d4c0a3680 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009fd0 tx=0x7f1d58009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:09.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.305+0000 7f1d4bfff700 1 -- 172.21.15.133:0/1180808272 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c099920 con 0x5638faa44c00 2023-11-27T15:25:09.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.305+0000 7f1d4bfff700 1 -- 172.21.15.133:0/1180808272 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d5c0be5f0 con 0x5638faa44c00 2023-11-27T15:25:09.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.305+0000 7f1d4bfff700 1 -- 172.21.15.133:0/1180808272 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c0a2410 con 0x5638faa44c00 2023-11-27T15:25:09.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.305+0000 7f1d7b717b80 1 -- 172.21.15.133:0/1180808272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa44c00 msgr2=0x5638faa45010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:09.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.305+0000 7f1d7b717b80 1 --2- 172.21.15.133:0/1180808272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa44c00 0x5638faa45010 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0964c0 tx=0x7f1d5c014ad0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.306+0000 7f1d7b717b80 1 -- 172.21.15.133:0/1180808272 shutdown_connections 2023-11-27T15:25:09.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.306+0000 7f1d7b717b80 1 --2- 172.21.15.133:0/1180808272 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa46110 0x5638faa46580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.306+0000 7f1d7b717b80 1 --2- 172.21.15.133:0/1180808272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa44c00 0x5638faa45010 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.306+0000 7f1d7b717b80 1 --2- 172.21.15.133:0/1180808272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa45550 0x5638faa45b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.306+0000 7f1d7b717b80 1 -- 172.21.15.133:0/1180808272 >> 172.21.15.133:0/1180808272 conn(0x5638faa3a360 msgr2=0x5638faa3f670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:09.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.306+0000 7f1d7b717b80 1 -- 172.21.15.133:0/1180808272 shutdown_connections 2023-11-27T15:25:09.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.306+0000 7f1d7b717b80 1 -- 172.21.15.133:0/1180808272 wait complete. 2023-11-27T15:25:09.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.306+0000 7f1d7b717b80 1 -- start start 2023-11-27T15:25:09.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.306+0000 7f1d7b717b80 1 Processor -- start 2023-11-27T15:25:09.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.307+0000 7f1d7b717b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa44c00 0x5638faa4e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.307+0000 7f1d7b717b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa45550 0x5638faa4eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.307+0000 7f1d7b717b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa46110 0x5638faa55b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.307+0000 7f1d7b717b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638fa9378e0 con 0x5638faa45550 2023-11-27T15:25:09.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.307+0000 7f1d7b717b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638fa937580 con 0x5638faa44c00 2023-11-27T15:25:09.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.307+0000 7f1d7b717b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638fa819920 con 0x5638faa46110 2023-11-27T15:25:09.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.307+0000 7f1d7b717b80 10 client.0 _ll_register_callbacks cb 0x5638fa955780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:09.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.307+0000 7f1d65b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa44c00 0x5638faa4e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.316 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.307+0000 7f1d65b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa44c00 0x5638faa4e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:44772/0 (socket says 172.21.15.133:44772) 2023-11-27T15:25:09.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.307+0000 7f1d65b9b700 1 -- 172.21.15.133:0/1081176177 learned_addr learned my addr 172.21.15.133:0/1081176177 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:09.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.307+0000 7f1d6539a700 1 --2- 172.21.15.133:0/1081176177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa45550 0x5638faa4eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.307+0000 7f1d6639c700 1 --2- 172.21.15.133:0/1081176177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa46110 0x5638faa55b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.307+0000 7f1d65b9b700 1 -- 172.21.15.133:0/1081176177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa46110 msgr2=0x5638faa55b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:09.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.307+0000 7f1d65b9b700 1 --2- 172.21.15.133:0/1081176177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa46110 0x5638faa55b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.307+0000 7f1d65b9b700 1 -- 172.21.15.133:0/1081176177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa45550 msgr2=0x5638faa4eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:09.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.307+0000 7f1d65b9b700 1 --2- 172.21.15.133:0/1081176177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa45550 0x5638faa4eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.307+0000 7f1d65b9b700 1 -- 172.21.15.133:0/1081176177 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6010a9e0 con 0x5638faa44c00 2023-11-27T15:25:09.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.308+0000 7f1d65b9b700 1 --2- 172.21.15.133:0/1081176177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa44c00 0x5638faa4e580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c096a30 tx=0x7f1d5c099fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d1bdb7eab61dc69 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:09.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.308+0000 7f1d497fa700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:25:09.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.308+0000 7f1d497fa700 1 -- 172.21.15.133:0/1081176177 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c099920 con 0x5638faa44c00 2023-11-27T15:25:09.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.308+0000 7f1d497fa700 1 -- 172.21.15.133:0/1081176177 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1d5c007e40 con 0x5638faa44c00 2023-11-27T15:25:09.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.308+0000 7f1d7b717b80 1 -- 172.21.15.133:0/1081176177 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638faa37b20 con 0x5638faa44c00 2023-11-27T15:25:09.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.308+0000 7f1d497fa700 1 -- 172.21.15.133:0/1081176177 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d5c0a2410 con 0x5638faa44c00 2023-11-27T15:25:09.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.308+0000 7f1d7b717b80 10 client.25997 fetch_fsmap learned FSMap version 23 2023-11-27T15:25:09.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.308+0000 7f1d7b717b80 1 -- 172.21.15.133:0/1081176177 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5638fa95c190 con 0x5638faa44c00 2023-11-27T15:25:09.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.308+0000 7f1d497fa700 1 -- 172.21.15.133:0/1081176177 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5638faa37b20 con 0x5638faa44c00 2023-11-27T15:25:09.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.309+0000 7f1d497fa700 1 -- 172.21.15.133:0/1081176177 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1d5c0c1030 con 0x5638faa44c00 2023-11-27T15:25:09.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.309+0000 7f1d7b717b80 10 client.25997 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:25:09.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.309+0000 7f1d7b717b80 10 client.25997 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:09.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.309+0000 7f1d7b717b80 20 client.25997 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:09.318 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.309+0000 7f1d7b717b80 1 --2- 172.21.15.133:0/1081176177 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5638faa59ec0 0x5638faa5c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.309+0000 7f1d7b717b80 4 client.25997 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:09.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.309+0000 7f1d7b717b80 1 -- 172.21.15.133:0/1081176177 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638fa9580a0 con 0x5638faa59ec0 2023-11-27T15:25:09.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.309+0000 7f1d6539a700 1 --2- 172.21.15.133:0/1081176177 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5638faa59ec0 0x5638faa5c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.309+0000 7f1d6539a700 1 --2- 172.21.15.133:0/1081176177 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5638faa59ec0 0x5638faa5c2d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee7807105a534304 server_cookie=a7d0d2405df63fd3 in_seq=0 out_seq=0 2023-11-27T15:25:09.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.309+0000 7f1d497fa700 10 client.25997 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:09.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.311+0000 7f1d497fa700 1 -- 172.21.15.133:0/1081176177 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5638fa9580a0 con 0x5638faa59ec0 2023-11-27T15:25:09.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.311+0000 7f1d497fa700 10 client.25997 handle_command_reply: tid=0 2023-11-27T15:25:09.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.311+0000 7f1d7b717b80 1 client.25997 shutdown 2023-11-27T15:25:09.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.311+0000 7f1d7b717b80 1 -- 172.21.15.133:0/1081176177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa44c00 msgr2=0x5638faa4e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:09.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.311+0000 7f1d7b717b80 1 --2- 172.21.15.133:0/1081176177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa44c00 0x5638faa4e580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c096a30 tx=0x7f1d5c099fc0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.319 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.311+0000 7f1d7b717b80 1 -- 172.21.15.133:0/1081176177 shutdown_connections 2023-11-27T15:25:09.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.311+0000 7f1d7b717b80 1 --2- 172.21.15.133:0/1081176177 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5638faa59ec0 0x5638faa5c2d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.311+0000 7f1d7b717b80 1 --2- 172.21.15.133:0/1081176177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa46110 0x5638faa55b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.311+0000 7f1d7b717b80 1 --2- 172.21.15.133:0/1081176177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa44c00 0x5638faa4e580 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.311+0000 7f1d7b717b80 1 --2- 172.21.15.133:0/1081176177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa45550 0x5638faa4eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.311+0000 7f1d497fa700 0 client.25997 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:09.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.311+0000 7f1d7b717b80 1 -- 172.21.15.133:0/1081176177 >> 172.21.15.133:0/1081176177 conn(0x5638faa3a360 msgr2=0x5638faa3ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:09.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.311+0000 7f1d7b717b80 1 -- 172.21.15.133:0/1081176177 shutdown_connections 2023-11-27T15:25:09.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.311+0000 7f1d7b717b80 1 -- 172.21.15.133:0/1081176177 wait complete. 2023-11-27T15:25:09.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.311+0000 7f1d7b717b80 20 client.25997 trim_cache size 0 max 16384 2023-11-27T15:25:09.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.330+0000 7f1d7b717b80 1 Processor -- start 2023-11-27T15:25:09.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.330+0000 7f1d7b717b80 1 -- start start 2023-11-27T15:25:09.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.331+0000 7f1d7b717b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa44c00 0x5638faa48fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.331+0000 7f1d7b717b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa45550 0x5638faa49520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.331+0000 7f1d7b717b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa46110 0x5638faa55ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.331+0000 7f1d7b717b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638faa4da30 con 0x5638faa45550 2023-11-27T15:25:09.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.331+0000 7f1d7b717b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638faa41140 con 0x5638faa46110 2023-11-27T15:25:09.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.331+0000 7f1d7b717b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638faa4cd60 con 0x5638faa44c00 2023-11-27T15:25:09.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.331+0000 7f1d6639c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa46110 0x5638faa55ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.331+0000 7f1d65b9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa44c00 0x5638faa48fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.331+0000 7f1d65b9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa44c00 0x5638faa48fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:36880/0 (socket says 172.21.15.133:36880) 2023-11-27T15:25:09.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.331+0000 7f1d65b9b700 1 -- 172.21.15.133:0/53563883 learned_addr learned my addr 172.21.15.133:0/53563883 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:09.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.331+0000 7f1d6539a700 1 --2- 172.21.15.133:0/53563883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa45550 0x5638faa49520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.332+0000 7f1d6639c700 1 -- 172.21.15.133:0/53563883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa44c00 msgr2=0x5638faa48fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:09.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.332+0000 7f1d6639c700 1 --2- 172.21.15.133:0/53563883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa44c00 0x5638faa48fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.332+0000 7f1d6639c700 1 -- 172.21.15.133:0/53563883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa45550 msgr2=0x5638faa49520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:09.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.332+0000 7f1d6639c700 1 --2- 172.21.15.133:0/53563883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa45550 0x5638faa49520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.332+0000 7f1d6639c700 1 -- 172.21.15.133:0/53563883 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638fa95bee0 con 0x5638faa46110 2023-11-27T15:25:09.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.332+0000 7f1d65b9b700 1 --2- 172.21.15.133:0/53563883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa44c00 0x5638faa48fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:09.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.332+0000 7f1d6539a700 1 --2- 172.21.15.133:0/53563883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa45550 0x5638faa49520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:09.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.333+0000 7f1d6639c700 1 --2- 172.21.15.133:0/53563883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa46110 0x5638faa55ac0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1d580060b0 tx=0x7f1d58012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1201bb7c1772c59 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:09.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.333+0000 7f1d54bff700 1 -- 172.21.15.133:0/53563883 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d580233f0 con 0x5638faa46110 2023-11-27T15:25:09.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.333+0000 7f1d54bff700 1 -- 172.21.15.133:0/53563883 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d58023a10 con 0x5638faa46110 2023-11-27T15:25:09.337 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.333+0000 7f1d54bff700 1 -- 172.21.15.133:0/53563883 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d580168a0 con 0x5638faa46110 2023-11-27T15:25:09.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.334+0000 7f1d7b717b80 1 -- 172.21.15.133:0/53563883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa46110 msgr2=0x5638faa55ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:09.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.334+0000 7f1d7b717b80 1 --2- 172.21.15.133:0/53563883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa46110 0x5638faa55ac0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1d580060b0 tx=0x7f1d58012000 comp rx=0 tx=0).stop 2023-11-27T15:25:09.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.334+0000 7f1d7b717b80 1 -- 172.21.15.133:0/53563883 shutdown_connections 2023-11-27T15:25:09.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.334+0000 7f1d7b717b80 1 --2- 172.21.15.133:0/53563883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa46110 0x5638faa55ac0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.334+0000 7f1d7b717b80 1 --2- 172.21.15.133:0/53563883 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa44c00 0x5638faa48fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.334+0000 7f1d7b717b80 1 --2- 172.21.15.133:0/53563883 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa45550 0x5638faa49520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.334+0000 7f1d7b717b80 1 -- 172.21.15.133:0/53563883 >> 172.21.15.133:0/53563883 conn(0x5638faa3a360 msgr2=0x5638faa4d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:09.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.334+0000 7f1d7b717b80 1 -- 172.21.15.133:0/53563883 shutdown_connections 2023-11-27T15:25:09.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.334+0000 7f1d7b717b80 1 -- 172.21.15.133:0/53563883 wait complete. 2023-11-27T15:25:09.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.334+0000 7f1d7b717b80 1 -- start start 2023-11-27T15:25:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.335+0000 7f1d7b717b80 1 Processor -- start 2023-11-27T15:25:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.335+0000 7f1d7b717b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa44c00 0x5638faa55480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.335+0000 7f1d7b717b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa45550 0x5638faa559c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.335+0000 7f1d7b717b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa46110 0x5638faa51f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.335+0000 7f1d7b717b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638faa525c0 con 0x5638faa44c00 2023-11-27T15:25:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.335+0000 7f1d7b717b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638faa52730 con 0x5638faa46110 2023-11-27T15:25:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.335+0000 7f1d7b717b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638faa528f0 con 0x5638faa45550 2023-11-27T15:25:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.335+0000 7f1d7b717b80 10 client.0 _ll_register_callbacks cb 0x7f1d5c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.335+0000 7f1d6539a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa45550 0x5638faa559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.335+0000 7f1d65b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa44c00 0x5638faa55480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.335+0000 7f1d6639c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa46110 0x5638faa51f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.335+0000 7f1d65b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa44c00 0x5638faa55480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:59452/0 (socket says 172.21.15.133:59452) 2023-11-27T15:25:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.335+0000 7f1d65b9b700 1 -- 172.21.15.133:0/2779285046 learned_addr learned my addr 172.21.15.133:0/2779285046 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.335+0000 7f1d6539a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa45550 0x5638faa559c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:36890/0 (socket says 172.21.15.133:36890) 2023-11-27T15:25:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.336+0000 7f1d65b9b700 1 -- 172.21.15.133:0/2779285046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa45550 msgr2=0x5638faa559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.336+0000 7f1d65b9b700 1 --2- 172.21.15.133:0/2779285046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa45550 0x5638faa559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.336+0000 7f1d65b9b700 1 -- 172.21.15.133:0/2779285046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa46110 msgr2=0x5638faa51f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.336+0000 7f1d65b9b700 1 --2- 172.21.15.133:0/2779285046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa46110 0x5638faa51f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.336+0000 7f1d65b9b700 1 -- 172.21.15.133:0/2779285046 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5638fa95c190 con 0x5638faa44c00 2023-11-27T15:25:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.336+0000 7f1d6639c700 1 --2- 172.21.15.133:0/2779285046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa46110 0x5638faa51f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.336+0000 7f1d65b9b700 1 --2- 172.21.15.133:0/2779285046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa44c00 0x5638faa55480 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c005950 tx=0x7f1d5c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d73798dba9c6b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.336+0000 7f1d497fa700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:25:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.336+0000 7f1d497fa700 1 -- 172.21.15.133:0/2779285046 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c09a030 con 0x5638faa44c00 2023-11-27T15:25:09.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.336+0000 7f1d497fa700 1 -- 172.21.15.133:0/2779285046 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1d5c0a2410 con 0x5638faa44c00 2023-11-27T15:25:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.336+0000 7f1d7b717b80 1 -- 172.21.15.133:0/2779285046 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d5c0be5f0 con 0x5638faa44c00 2023-11-27T15:25:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.337+0000 7f1d497fa700 1 -- 172.21.15.133:0/2779285046 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d5c099920 con 0x5638faa44c00 2023-11-27T15:25:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.337+0000 7f1d7b717b80 10 client.15960 fetch_fsmap learned FSMap version 23 2023-11-27T15:25:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.337+0000 7f1d7b717b80 1 -- 172.21.15.133:0/2779285046 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f1d58007eb0 con 0x5638faa44c00 2023-11-27T15:25:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.337+0000 7f1d497fa700 1 -- 172.21.15.133:0/2779285046 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d5c0be5f0 con 0x5638faa44c00 2023-11-27T15:25:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.337+0000 7f1d497fa700 1 -- 172.21.15.133:0/2779285046 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 23) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1d5c0c1070 con 0x5638faa44c00 2023-11-27T15:25:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.337+0000 7f1d7b717b80 10 client.15960 fetch_fsmap finished waiting for FSMap version 23 2023-11-27T15:25:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.337+0000 7f1d7b717b80 10 client.15960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.337+0000 7f1d7b717b80 20 client.15960 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.337+0000 7f1d7b717b80 1 --2- 172.21.15.133:0/2779285046 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5638faab7720 0x5638faab9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.337+0000 7f1d7b717b80 4 client.15960 mds_command: new command op to 24560 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:25:09.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.337+0000 7f1d7b717b80 1 -- 172.21.15.133:0/2779285046 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5638fa964660 con 0x5638faab7720 2023-11-27T15:25:09.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.337+0000 7f1d6539a700 1 --2- 172.21.15.133:0/2779285046 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5638faab7720 0x5638faab9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:09.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.338+0000 7f1d6539a700 1 --2- 172.21.15.133:0/2779285046 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5638faab7720 0x5638faab9b30 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a894bf44d8f2395 server_cookie=9ab1587cf78df55a in_seq=0 out_seq=0 2023-11-27T15:25:09.345 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.338+0000 7f1d497fa700 10 client.15960 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:09.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.343+0000 7f1d497fa700 1 -- 172.21.15.133:0/2779285046 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5638fa964660 con 0x5638faab7720 2023-11-27T15:25:09.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.343+0000 7f1d497fa700 10 client.15960 handle_command_reply: tid=0 2023-11-27T15:25:09.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.343+0000 7f1d7b717b80 1 client.15960 shutdown 2023-11-27T15:25:09.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.343+0000 7f1d7b717b80 1 -- 172.21.15.133:0/2779285046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa44c00 msgr2=0x5638faa55480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:09.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.343+0000 7f1d7b717b80 1 --2- 172.21.15.133:0/2779285046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa44c00 0x5638faa55480 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c005950 tx=0x7f1d5c004470 comp rx=0 tx=0).stop 2023-11-27T15:25:09.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.343+0000 7f1d7b717b80 1 -- 172.21.15.133:0/2779285046 shutdown_connections 2023-11-27T15:25:09.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.343+0000 7f1d7b717b80 1 --2- 172.21.15.133:0/2779285046 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5638faab7720 0x5638faab9b30 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.343+0000 7f1d7b717b80 1 --2- 172.21.15.133:0/2779285046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638faa46110 0x5638faa51f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.343+0000 7f1d7b717b80 1 --2- 172.21.15.133:0/2779285046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5638faa44c00 0x5638faa55480 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.343+0000 7f1d7b717b80 1 --2- 172.21.15.133:0/2779285046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5638faa45550 0x5638faa559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.343+0000 7f1d497fa700 0 client.15960 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:09.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.343+0000 7f1d7b717b80 1 -- 172.21.15.133:0/2779285046 >> 172.21.15.133:0/2779285046 conn(0x5638faa3a360 msgr2=0x5638faa48fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:09.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.343+0000 7f1d7b717b80 1 -- 172.21.15.133:0/2779285046 shutdown_connections 2023-11-27T15:25:09.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.343+0000 7f1d7b717b80 1 -- 172.21.15.133:0/2779285046 wait complete. 2023-11-27T15:25:09.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.343+0000 7f1d7b717b80 20 client.15960 trim_cache size 0 max 16384 2023-11-27T15:25:09.348 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:25:09.348 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-27T15:25:09.349 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-11-27T15:25:09.349 INFO:teuthology.orchestra.run.smithi133.stdout: "b65e2907-6261-49f5-a60e-198644c5dc8a": { 2023-11-27T15:25:09.349 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-11-27T15:25:09.349 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "b65e2907-6261-49f5-a60e-198644c5dc8a", 2023-11-27T15:25:09.349 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-11-27T15:25:09.349 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:25:09.349 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:25:09.349 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:25:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.344+0000 7f1d67dff700 1 -- 172.21.15.133:0/1500828651 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1d4c0a11d0 msgr2=0x7f1d4c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.344+0000 7f1d67dff700 1 --2- 172.21.15.133:0/1500828651 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1d4c0a11d0 0x7f1d4c0a3680 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009fd0 tx=0x7f1d58009920 comp rx=0 tx=0).stop 2023-11-27T15:25:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.344+0000 7f1d67dff700 1 -- 172.21.15.133:0/1500828651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d600ff1c0 msgr2=0x7f1d60196050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.344+0000 7f1d67dff700 1 --2- 172.21.15.133:0/1500828651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d600ff1c0 0x7f1d60196050 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00e970 tx=0x7f1d5c00ed30 comp rx=0 tx=0).stop 2023-11-27T15:25:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.344+0000 7f1d67dff700 1 -- 172.21.15.133:0/1500828651 shutdown_connections 2023-11-27T15:25:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.344+0000 7f1d67dff700 1 --2- 172.21.15.133:0/1500828651 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1d4c0a11d0 0x7f1d4c0a3680 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.344+0000 7f1d67dff700 1 --2- 172.21.15.133:0/1500828651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d60100d40 0x7f1d6019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.344+0000 7f1d67dff700 1 --2- 172.21.15.133:0/1500828651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d600ff1c0 0x7f1d60196050 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.344+0000 7f1d67dff700 1 --2- 172.21.15.133:0/1500828651 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1d601003b0 0x7f1d60196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:09.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.344+0000 7f1d67dff700 1 -- 172.21.15.133:0/1500828651 >> 172.21.15.133:0/1500828651 conn(0x7f1d600fa7c0 msgr2=0x7f1d600691a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:09.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.344+0000 7f1d67dff700 1 -- 172.21.15.133:0/1500828651 shutdown_connections 2023-11-27T15:25:09.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:09.344+0000 7f1d67dff700 1 -- 172.21.15.133:0/1500828651 wait complete. 2023-11-27T15:25:09.371 INFO:tasks.cephfs.filesystem:scrub status for tag:b65e2907-6261-49f5-a60e-198644c5dc8a - {'path': '/', 'tag': 'b65e2907-6261-49f5-a60e-198644c5dc8a', 'options': 'recursive,force'} 2023-11-27T15:25:10.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:09 smithi133 ceph-mon[97860]: scrub queued for path: / 2023-11-27T15:25:10.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:09 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:10.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:09 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:25:10.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:09 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:10.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:09 smithi177 ceph-mon[131504]: scrub queued for path: / 2023-11-27T15:25:10.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:09 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:10.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:09 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:25:10.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:09 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:10.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:09 smithi174 ceph-mon[131110]: scrub queued for path: / 2023-11-27T15:25:10.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:09 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:10.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:09 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:25:10.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:09 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:11.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:10 smithi133 ceph-mon[97860]: pgmap v416: 129 pgs: 129 active+clean; 1.8 MiB data, 91 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 30 KiB/s wr, 14 op/s 2023-11-27T15:25:11.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:10 smithi177 ceph-mon[131504]: pgmap v416: 129 pgs: 129 active+clean; 1.8 MiB data, 91 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 30 KiB/s wr, 14 op/s 2023-11-27T15:25:11.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:10 smithi174 ceph-mon[131110]: pgmap v416: 129 pgs: 129 active+clean; 1.8 MiB data, 91 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 30 KiB/s wr, 14 op/s 2023-11-27T15:25:12.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:11 smithi133 ceph-mon[97860]: scrub summary: idle 2023-11-27T15:25:12.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:11 smithi177 ceph-mon[131504]: scrub summary: idle 2023-11-27T15:25:12.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:11 smithi174 ceph-mon[131110]: scrub summary: idle 2023-11-27T15:25:13.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:12 smithi133 ceph-mon[97860]: pgmap v417: 129 pgs: 129 active+clean; 4.6 MiB data, 94 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 281 KiB/s wr, 23 op/s 2023-11-27T15:25:13.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:25:13.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:25:13.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:25:13.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:25:13.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:12 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:25:13.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:12 smithi177 ceph-mon[131504]: pgmap v417: 129 pgs: 129 active+clean; 4.6 MiB data, 94 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 281 KiB/s wr, 23 op/s 2023-11-27T15:25:13.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:25:13.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:25:13.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:25:13.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:25:13.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:12 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:25:13.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:12 smithi174 ceph-mon[131110]: pgmap v417: 129 pgs: 129 active+clean; 4.6 MiB data, 94 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 281 KiB/s wr, 23 op/s 2023-11-27T15:25:13.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:25:13.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:25:13.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:25:13.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:25:13.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:12 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:25:15.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:14 smithi174 ceph-mon[131110]: pgmap v418: 129 pgs: 129 active+clean; 7.5 MiB data, 102 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 533 KiB/s wr, 22 op/s 2023-11-27T15:25:15.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:14 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:25:15.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:14 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:25:15.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:14 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:25:15.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:14 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:25:15.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:14 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:25:15.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:14 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:25:15.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:14 smithi174 ceph-mon[131110]: Service mds.all has no related daemon already placed. 2023-11-27T15:25:15.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:14 smithi133 ceph-mon[97860]: pgmap v418: 129 pgs: 129 active+clean; 7.5 MiB data, 102 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 533 KiB/s wr, 22 op/s 2023-11-27T15:25:15.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:25:15.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:25:15.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:14 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:25:15.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:25:15.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:14 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:25:15.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:14 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:25:15.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:14 smithi133 ceph-mon[97860]: Service mds.all has no related daemon already placed. 2023-11-27T15:25:15.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:14 smithi177 ceph-mon[131504]: pgmap v418: 129 pgs: 129 active+clean; 7.5 MiB data, 102 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 533 KiB/s wr, 22 op/s 2023-11-27T15:25:15.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:14 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:25:15.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:14 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:25:15.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:14 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:25:15.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:14 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:25:15.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:14 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:25:15.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:14 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:25:15.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:14 smithi177 ceph-mon[131504]: Service mds.all has no related daemon already placed. 2023-11-27T15:25:17.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:16 smithi174 ceph-mon[131110]: pgmap v419: 129 pgs: 129 active+clean; 13 MiB data, 123 MiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 1.1 MiB/s wr, 44 op/s 2023-11-27T15:25:17.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:16 smithi133 ceph-mon[97860]: pgmap v419: 129 pgs: 129 active+clean; 13 MiB data, 123 MiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 1.1 MiB/s wr, 44 op/s 2023-11-27T15:25:17.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:16 smithi177 ceph-mon[131504]: pgmap v419: 129 pgs: 129 active+clean; 13 MiB data, 123 MiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 1.1 MiB/s wr, 44 op/s 2023-11-27T15:25:18.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:17 smithi174 ceph-mon[131110]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:25:18.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:17 smithi133 ceph-mon[97860]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:25:18.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:17 smithi177 ceph-mon[131504]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:25:19.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:18 smithi174 ceph-mon[131110]: pgmap v420: 129 pgs: 129 active+clean; 18 MiB data, 144 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 60 op/s 2023-11-27T15:25:19.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:18 smithi133 ceph-mon[97860]: pgmap v420: 129 pgs: 129 active+clean; 18 MiB data, 144 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 60 op/s 2023-11-27T15:25:19.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:18 smithi177 ceph-mon[131504]: pgmap v420: 129 pgs: 129 active+clean; 18 MiB data, 144 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 60 op/s 2023-11-27T15:25:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.533+0000 7f6c5d9ff700 1 -- 172.21.15.133:0/906876368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c58100cc0 msgr2=0x7f6c581030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.533+0000 7f6c5d9ff700 1 --2- 172.21.15.133:0/906876368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c58100cc0 0x7f6c581030a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6c480098f0 tx=0x7f6c48009c00 comp rx=0 tx=0).stop 2023-11-27T15:25:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.533+0000 7f6c5d9ff700 1 -- 172.21.15.133:0/906876368 shutdown_connections 2023-11-27T15:25:19.537 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.533+0000 7f6c5d9ff700 1 --2- 172.21.15.133:0/906876368 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c58105f00 0x7f6c58108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.533+0000 7f6c5d9ff700 1 --2- 172.21.15.133:0/906876368 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c58100cc0 0x7f6c581030a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.533+0000 7f6c5d9ff700 1 --2- 172.21.15.133:0/906876368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c581035e0 0x7f6c581059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.533+0000 7f6c5d9ff700 1 -- 172.21.15.133:0/906876368 >> 172.21.15.133:0/906876368 conn(0x7f6c580fa720 msgr2=0x7f6c580fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:19.538 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.533+0000 7f6c5d9ff700 1 -- 172.21.15.133:0/906876368 shutdown_connections 2023-11-27T15:25:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.534+0000 7f6c5d9ff700 1 -- 172.21.15.133:0/906876368 wait complete. 2023-11-27T15:25:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.535+0000 7f6c5d9ff700 1 Processor -- start 2023-11-27T15:25:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.535+0000 7f6c5d9ff700 1 -- start start 2023-11-27T15:25:19.540 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.535+0000 7f6c5d9ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c58100cc0 0x7f6c58198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.536+0000 7f6c5d9ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c581035e0 0x7f6c58198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.536+0000 7f6c5d9ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c58105f00 0x7f6c5819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.536+0000 7f6c5d9ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5819fd80 con 0x7f6c58100cc0 2023-11-27T15:25:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.536+0000 7f6c5d9ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5819fec0 con 0x7f6c58105f00 2023-11-27T15:25:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.536+0000 7f6c5d9ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c581a0050 con 0x7f6c581035e0 2023-11-27T15:25:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.536+0000 7f6c56ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c58100cc0 0x7f6c58198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:19.541 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.536+0000 7f6c56ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c58100cc0 0x7f6c58198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47684/0 (socket says 172.21.15.133:47684) 2023-11-27T15:25:19.542 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.536+0000 7f6c56ffd700 1 -- 172.21.15.133:0/3650871645 learned_addr learned my addr 172.21.15.133:0/3650871645 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:19.542 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.536+0000 7f6c577fe700 1 --2- 172.21.15.133:0/3650871645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c58105f00 0x7f6c5819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:19.542 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.536+0000 7f6c567fc700 1 --2- 172.21.15.133:0/3650871645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c581035e0 0x7f6c58198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:19.542 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.536+0000 7f6c577fe700 1 -- 172.21.15.133:0/3650871645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c581035e0 msgr2=0x7f6c58198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:19.542 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.536+0000 7f6c577fe700 1 --2- 172.21.15.133:0/3650871645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c581035e0 0x7f6c58198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.542 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.536+0000 7f6c577fe700 1 -- 172.21.15.133:0/3650871645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c58100cc0 msgr2=0x7f6c58198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:19.542 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.536+0000 7f6c577fe700 1 --2- 172.21.15.133:0/3650871645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c58100cc0 0x7f6c58198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.536+0000 7f6c577fe700 1 -- 172.21.15.133:0/3650871645 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c48009520 con 0x7f6c58105f00 2023-11-27T15:25:19.543 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.537+0000 7f6c577fe700 1 --2- 172.21.15.133:0/3650871645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c58105f00 0x7f6c5819f7b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000eb80 tx=0x7f6c5000ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.537+0000 7f6c5c9fd700 1 -- 172.21.15.133:0/3650871645 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50018070 con 0x7f6c58105f00 2023-11-27T15:25:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.537+0000 7f6c5c9fd700 1 -- 172.21.15.133:0/3650871645 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c5000f040 con 0x7f6c58105f00 2023-11-27T15:25:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.537+0000 7f6c5d9ff700 1 -- 172.21.15.133:0/3650871645 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c5810bba0 con 0x7f6c58105f00 2023-11-27T15:25:19.544 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.537+0000 7f6c5c9fd700 1 -- 172.21.15.133:0/3650871645 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5000cd40 con 0x7f6c58105f00 2023-11-27T15:25:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.537+0000 7f6c5d9ff700 1 -- 172.21.15.133:0/3650871645 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c5810ba10 con 0x7f6c58105f00 2023-11-27T15:25:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.542+0000 7f6c71280b80 1 Processor -- start 2023-11-27T15:25:19.545 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.542+0000 7f6c5c9fd700 1 -- 172.21.15.133:0/3650871645 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6c50003a50 con 0x7f6c58105f00 2023-11-27T15:25:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.542+0000 7f6c71280b80 1 -- start start 2023-11-27T15:25:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.543+0000 7f6c71280b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba7b46b0 0x55afba958870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.543+0000 7f6c71280b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba958db0 0x55afba9595d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.543+0000 7f6c71280b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba959b10 0x55afba959f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.543+0000 7f6c71280b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afba85e000 con 0x55afba7b46b0 2023-11-27T15:25:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.543+0000 7f6c71280b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afba8b0840 con 0x55afba958db0 2023-11-27T15:25:19.547 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.543+0000 7f6c71280b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afba8a9cd0 con 0x55afba959b10 2023-11-27T15:25:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.543+0000 7f6c567fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba7b46b0 0x55afba958870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.543+0000 7f6c567fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba7b46b0 0x55afba958870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47694/0 (socket says 172.21.15.133:47694) 2023-11-27T15:25:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.543+0000 7f6c567fc700 1 -- 172.21.15.133:0/4249126953 learned_addr learned my addr 172.21.15.133:0/4249126953 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:19.548 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.543+0000 7f6c56ffd700 1 --2- 172.21.15.133:0/4249126953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba958db0 0x55afba9595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.544+0000 7f6c567fc700 1 --2- 172.21.15.133:0/4249126953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba959b10 0x55afba959f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.544+0000 7f6c567fc700 1 -- 172.21.15.133:0/4249126953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba959b10 msgr2=0x55afba959f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.544+0000 7f6c567fc700 1 --2- 172.21.15.133:0/4249126953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba959b10 0x55afba959f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.544+0000 7f6c567fc700 1 -- 172.21.15.133:0/4249126953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba958db0 msgr2=0x55afba9595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:19.551 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.544+0000 7f6c567fc700 1 --2- 172.21.15.133:0/4249126953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba958db0 0x55afba9595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.544+0000 7f6c567fc700 1 -- 172.21.15.133:0/4249126953 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afba72b5d0 con 0x55afba7b46b0 2023-11-27T15:25:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.544+0000 7f6c5c9fd700 1 --2- 172.21.15.133:0/3650871645 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6c400a11d0 0x7f6c400a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.544+0000 7f6c5c9fd700 1 -- 172.21.15.133:0/3650871645 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6c500cc800 con 0x7f6c58105f00 2023-11-27T15:25:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.544+0000 7f6c567fc700 1 --2- 172.21.15.133:0/4249126953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba7b46b0 0x55afba958870 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009540 tx=0x7f6c4c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c066c14032b30d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.545+0000 7f6c3dffb700 1 -- 172.21.15.133:0/4249126953 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c004250 con 0x55afba7b46b0 2023-11-27T15:25:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.545+0000 7f6c3dffb700 1 -- 172.21.15.133:0/4249126953 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c4c0043b0 con 0x55afba7b46b0 2023-11-27T15:25:19.552 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.545+0000 7f6c56ffd700 1 --2- 172.21.15.133:0/3650871645 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6c400a11d0 0x7f6c400a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.545+0000 7f6c3dffb700 1 -- 172.21.15.133:0/4249126953 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c00e790 con 0x55afba7b46b0 2023-11-27T15:25:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.546+0000 7f6c71280b80 1 -- 172.21.15.133:0/4249126953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba7b46b0 msgr2=0x55afba958870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.546+0000 7f6c71280b80 1 --2- 172.21.15.133:0/4249126953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba7b46b0 0x55afba958870 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c009540 tx=0x7f6c4c009a20 comp rx=0 tx=0).stop 2023-11-27T15:25:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.546+0000 7f6c56ffd700 1 --2- 172.21.15.133:0/3650871645 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6c400a11d0 0x7f6c400a3680 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800c010 tx=0x7f6c48019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.546+0000 7f6c71280b80 1 -- 172.21.15.133:0/4249126953 shutdown_connections 2023-11-27T15:25:19.553 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.546+0000 7f6c71280b80 1 --2- 172.21.15.133:0/4249126953 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba959b10 0x55afba959f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.546+0000 7f6c71280b80 1 --2- 172.21.15.133:0/4249126953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba7b46b0 0x55afba958870 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.546+0000 7f6c71280b80 1 --2- 172.21.15.133:0/4249126953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba958db0 0x55afba9595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.546+0000 7f6c71280b80 1 -- 172.21.15.133:0/4249126953 >> 172.21.15.133:0/4249126953 conn(0x55afba94df50 msgr2=0x55afba953100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.546+0000 7f6c71280b80 1 -- 172.21.15.133:0/4249126953 shutdown_connections 2023-11-27T15:25:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.546+0000 7f6c71280b80 1 -- 172.21.15.133:0/4249126953 wait complete. 2023-11-27T15:25:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.547+0000 7f6c71280b80 1 -- start start 2023-11-27T15:25:19.554 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.547+0000 7f6c71280b80 1 Processor -- start 2023-11-27T15:25:19.558 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.548+0000 7f6c71280b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba7b46b0 0x55afba961c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:19.559 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.548+0000 7f6c71280b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba958db0 0x55afba9621c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:19.559 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.548+0000 7f6c71280b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba959b10 0x55afba969200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:19.559 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.548+0000 7f6c71280b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afba84b880 con 0x55afba959b10 2023-11-27T15:25:19.559 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.548+0000 7f6c71280b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afba84b520 con 0x55afba7b46b0 2023-11-27T15:25:19.559 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.548+0000 7f6c71280b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afba72d8c0 con 0x55afba958db0 2023-11-27T15:25:19.559 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.548+0000 7f6c71280b80 10 client.0 _ll_register_callbacks cb 0x55afba869720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:19.559 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.548+0000 7f6c56ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba7b46b0 0x55afba961c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:19.559 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.548+0000 7f6c56ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba7b46b0 0x55afba961c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:56214/0 (socket says 172.21.15.133:56214) 2023-11-27T15:25:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.548+0000 7f6c56ffd700 1 -- 172.21.15.133:0/171643122 learned_addr learned my addr 172.21.15.133:0/171643122 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.548+0000 7f6c567fc700 1 --2- 172.21.15.133:0/171643122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba958db0 0x55afba9621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.549+0000 7f6c577fe700 1 --2- 172.21.15.133:0/171643122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba959b10 0x55afba969200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.549+0000 7f6c56ffd700 1 -- 172.21.15.133:0/171643122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba958db0 msgr2=0x55afba9621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.549+0000 7f6c56ffd700 1 --2- 172.21.15.133:0/171643122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba958db0 0x55afba9621c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.549+0000 7f6c56ffd700 1 -- 172.21.15.133:0/171643122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba959b10 msgr2=0x55afba969200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.549+0000 7f6c56ffd700 1 --2- 172.21.15.133:0/171643122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba959b10 0x55afba969200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.549+0000 7f6c56ffd700 1 -- 172.21.15.133:0/171643122 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4c011440 con 0x55afba7b46b0 2023-11-27T15:25:19.561 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.549+0000 7f6c577fe700 1 --2- 172.21.15.133:0/171643122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba959b10 0x55afba969200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:19.561 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.549+0000 7f6c56ffd700 1 --2- 172.21.15.133:0/171643122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba7b46b0 0x55afba961c80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800b8e0 tx=0x7f6c48009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dee24ddabb45076b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:19.561 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.549+0000 7f6c36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:25:19.561 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.549+0000 7f6c36ffd700 1 -- 172.21.15.133:0/171643122 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4802b070 con 0x55afba7b46b0 2023-11-27T15:25:19.561 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.549+0000 7f6c36ffd700 1 -- 172.21.15.133:0/171643122 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6c48005390 con 0x55afba7b46b0 2023-11-27T15:25:19.561 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.549+0000 7f6c71280b80 1 -- 172.21.15.133:0/171643122 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55afba94b710 con 0x55afba7b46b0 2023-11-27T15:25:19.561 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.550+0000 7f6c36ffd700 1 -- 172.21.15.133:0/171643122 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c48005550 con 0x55afba7b46b0 2023-11-27T15:25:19.561 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.550+0000 7f6c71280b80 10 client.26015 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.550+0000 7f6c71280b80 1 -- 172.21.15.133:0/171643122 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55afba870130 con 0x55afba7b46b0 2023-11-27T15:25:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.550+0000 7f6c36ffd700 1 -- 172.21.15.133:0/171643122 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55afba94b710 con 0x55afba7b46b0 2023-11-27T15:25:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.550+0000 7f6c36ffd700 1 -- 172.21.15.133:0/171643122 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 24) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6c48037070 con 0x55afba7b46b0 2023-11-27T15:25:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.550+0000 7f6c71280b80 10 client.26015 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.550+0000 7f6c71280b80 10 client.26015 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.550+0000 7f6c71280b80 20 client.26015 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.551+0000 7f6c71280b80 1 --2- 172.21.15.133:0/171643122 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55afba96dab0 0x55afba96fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.551+0000 7f6c71280b80 4 client.26015 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.551+0000 7f6c71280b80 1 -- 172.21.15.133:0/171643122 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55afba86c040 con 0x55afba96dab0 2023-11-27T15:25:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.551+0000 7f6c567fc700 1 --2- 172.21.15.133:0/171643122 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55afba96dab0 0x55afba96fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.551+0000 7f6c567fc700 1 --2- 172.21.15.133:0/171643122 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55afba96dab0 0x55afba96fec0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e54378104bf4a1ed server_cookie=f088d7b78df50a68 in_seq=0 out_seq=0 2023-11-27T15:25:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.551+0000 7f6c36ffd700 10 client.26015 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.552+0000 7f6c36ffd700 1 -- 172.21.15.133:0/171643122 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55afba86c040 con 0x55afba96dab0 2023-11-27T15:25:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.552+0000 7f6c36ffd700 10 client.26015 handle_command_reply: tid=0 2023-11-27T15:25:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.552+0000 7f6c71280b80 1 client.26015 shutdown 2023-11-27T15:25:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.552+0000 7f6c71280b80 1 -- 172.21.15.133:0/171643122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba7b46b0 msgr2=0x55afba961c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.552+0000 7f6c71280b80 1 --2- 172.21.15.133:0/171643122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba7b46b0 0x55afba961c80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800b8e0 tx=0x7f6c48009c00 comp rx=0 tx=0).stop 2023-11-27T15:25:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.552+0000 7f6c71280b80 1 -- 172.21.15.133:0/171643122 shutdown_connections 2023-11-27T15:25:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.552+0000 7f6c71280b80 1 --2- 172.21.15.133:0/171643122 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55afba96dab0 0x55afba96fec0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.552+0000 7f6c71280b80 1 --2- 172.21.15.133:0/171643122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba959b10 0x55afba969200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.552+0000 7f6c36ffd700 0 client.26015 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.552+0000 7f6c71280b80 1 --2- 172.21.15.133:0/171643122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba7b46b0 0x55afba961c80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.552+0000 7f6c71280b80 1 --2- 172.21.15.133:0/171643122 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba958db0 0x55afba9621c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.553+0000 7f6c71280b80 1 -- 172.21.15.133:0/171643122 >> 172.21.15.133:0/171643122 conn(0x55afba94df50 msgr2=0x55afba9527e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.553+0000 7f6c71280b80 1 -- 172.21.15.133:0/171643122 shutdown_connections 2023-11-27T15:25:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.553+0000 7f6c71280b80 1 -- 172.21.15.133:0/171643122 wait complete. 2023-11-27T15:25:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.553+0000 7f6c71280b80 20 client.26015 trim_cache size 0 max 16384 2023-11-27T15:25:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.572+0000 7f6c71280b80 1 Processor -- start 2023-11-27T15:25:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.572+0000 7f6c71280b80 1 -- start start 2023-11-27T15:25:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.572+0000 7f6c71280b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba7b46b0 0x55afba95c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.572+0000 7f6c71280b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba958db0 0x55afba9691c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.572+0000 7f6c71280b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba959b10 0x55afba95cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.572+0000 7f6c71280b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afba959180 con 0x55afba959b10 2023-11-27T15:25:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.572+0000 7f6c71280b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afba95fc60 con 0x55afba958db0 2023-11-27T15:25:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.572+0000 7f6c71280b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afba95d550 con 0x55afba7b46b0 2023-11-27T15:25:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.572+0000 7f6c56ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba7b46b0 0x55afba95c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.573+0000 7f6c577fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba959b10 0x55afba95cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.573+0000 7f6c56ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba7b46b0 0x55afba95c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:58992/0 (socket says 172.21.15.133:58992) 2023-11-27T15:25:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.573+0000 7f6c56ffd700 1 -- 172.21.15.133:0/1505221761 learned_addr learned my addr 172.21.15.133:0/1505221761 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.572+0000 7f6c567fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba958db0 0x55afba9691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.573+0000 7f6c577fe700 1 -- 172.21.15.133:0/1505221761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba7b46b0 msgr2=0x55afba95c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.573+0000 7f6c577fe700 1 --2- 172.21.15.133:0/1505221761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba7b46b0 0x55afba95c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.573+0000 7f6c577fe700 1 -- 172.21.15.133:0/1505221761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba958db0 msgr2=0x55afba9691c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.573+0000 7f6c577fe700 1 --2- 172.21.15.133:0/1505221761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba958db0 0x55afba9691c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.573+0000 7f6c577fe700 1 -- 172.21.15.133:0/1505221761 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afba86fe80 con 0x55afba959b10 2023-11-27T15:25:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.573+0000 7f6c567fc700 1 --2- 172.21.15.133:0/1505221761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba958db0 0x55afba9691c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.573+0000 7f6c577fe700 1 --2- 172.21.15.133:0/1505221761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba959b10 0x55afba95cf20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6c5001b000 tx=0x7f6c5000c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958ac677bdaf2ba0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.574+0000 7f6c3e7fc700 1 -- 172.21.15.133:0/1505221761 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50099250 con 0x55afba959b10 2023-11-27T15:25:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.574+0000 7f6c3e7fc700 1 -- 172.21.15.133:0/1505221761 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c500b7e90 con 0x55afba959b10 2023-11-27T15:25:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.574+0000 7f6c3e7fc700 1 -- 172.21.15.133:0/1505221761 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50097d90 con 0x55afba959b10 2023-11-27T15:25:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.574+0000 7f6c71280b80 1 -- 172.21.15.133:0/1505221761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba959b10 msgr2=0x55afba95cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.574+0000 7f6c71280b80 1 --2- 172.21.15.133:0/1505221761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba959b10 0x55afba95cf20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6c5001b000 tx=0x7f6c5000c7a0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.575+0000 7f6c71280b80 1 -- 172.21.15.133:0/1505221761 shutdown_connections 2023-11-27T15:25:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.575+0000 7f6c71280b80 1 --2- 172.21.15.133:0/1505221761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba959b10 0x55afba95cf20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.575+0000 7f6c71280b80 1 --2- 172.21.15.133:0/1505221761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba7b46b0 0x55afba95c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.575+0000 7f6c71280b80 1 --2- 172.21.15.133:0/1505221761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba958db0 0x55afba9691c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.575+0000 7f6c71280b80 1 -- 172.21.15.133:0/1505221761 >> 172.21.15.133:0/1505221761 conn(0x55afba94df50 msgr2=0x55afba974820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.575+0000 7f6c71280b80 1 -- 172.21.15.133:0/1505221761 shutdown_connections 2023-11-27T15:25:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.575+0000 7f6c71280b80 1 -- 172.21.15.133:0/1505221761 wait complete. 2023-11-27T15:25:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.575+0000 7f6c71280b80 1 -- start start 2023-11-27T15:25:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.575+0000 7f6c71280b80 1 Processor -- start 2023-11-27T15:25:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.576+0000 7f6c71280b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba7b46b0 0x55afba964f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.576+0000 7f6c71280b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba958db0 0x55afba9654a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.576+0000 7f6c71280b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba959b10 0x55afba9684c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.576+0000 7f6c71280b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afba968a00 con 0x55afba959b10 2023-11-27T15:25:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.576+0000 7f6c71280b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afba968b70 con 0x55afba7b46b0 2023-11-27T15:25:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.576+0000 7f6c71280b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afba968d30 con 0x55afba958db0 2023-11-27T15:25:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.576+0000 7f6c71280b80 10 client.0 _ll_register_callbacks cb 0x7f6c4800f8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.576+0000 7f6c56ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba7b46b0 0x55afba964f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.576+0000 7f6c56ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba7b46b0 0x55afba964f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:56226/0 (socket says 172.21.15.133:56226) 2023-11-27T15:25:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.576+0000 7f6c56ffd700 1 -- 172.21.15.133:0/775886858 learned_addr learned my addr 172.21.15.133:0/775886858 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.576+0000 7f6c577fe700 1 --2- 172.21.15.133:0/775886858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba959b10 0x55afba9684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.577+0000 7f6c567fc700 1 --2- 172.21.15.133:0/775886858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba958db0 0x55afba9654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.577+0000 7f6c577fe700 1 -- 172.21.15.133:0/775886858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba958db0 msgr2=0x55afba9654a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.577+0000 7f6c577fe700 1 --2- 172.21.15.133:0/775886858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba958db0 0x55afba9654a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.577+0000 7f6c577fe700 1 -- 172.21.15.133:0/775886858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba7b46b0 msgr2=0x55afba964f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.577+0000 7f6c577fe700 1 --2- 172.21.15.133:0/775886858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba7b46b0 0x55afba964f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.577+0000 7f6c577fe700 1 -- 172.21.15.133:0/775886858 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55afba72b5d0 con 0x55afba959b10 2023-11-27T15:25:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.577+0000 7f6c567fc700 1 --2- 172.21.15.133:0/775886858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba958db0 0x55afba9654a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.577+0000 7f6c577fe700 1 --2- 172.21.15.133:0/775886858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba959b10 0x55afba9684c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6c5009dae0 tx=0x7f6c500a1d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13c447bb27a54e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.577+0000 7f6c36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:25:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.577+0000 7f6c36ffd700 1 -- 172.21.15.133:0/775886858 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c500a1a20 con 0x55afba959b10 2023-11-27T15:25:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.578+0000 7f6c36ffd700 1 -- 172.21.15.133:0/775886858 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6c500998f0 con 0x55afba959b10 2023-11-27T15:25:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.578+0000 7f6c71280b80 1 -- 172.21.15.133:0/775886858 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c4c0043b0 con 0x55afba959b10 2023-11-27T15:25:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.578+0000 7f6c36ffd700 1 -- 172.21.15.133:0/775886858 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c5009db80 con 0x55afba959b10 2023-11-27T15:25:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.578+0000 7f6c71280b80 10 client.15978 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.578+0000 7f6c71280b80 1 -- 172.21.15.133:0/775886858 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f6c5810ba10 con 0x55afba959b10 2023-11-27T15:25:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.578+0000 7f6c36ffd700 1 -- 172.21.15.133:0/775886858 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c5007c020 con 0x55afba959b10 2023-11-27T15:25:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.578+0000 7f6c36ffd700 1 -- 172.21.15.133:0/775886858 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 24) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6c500956e0 con 0x55afba959b10 2023-11-27T15:25:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.578+0000 7f6c71280b80 10 client.15978 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.578+0000 7f6c71280b80 10 client.15978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.578+0000 7f6c71280b80 20 client.15978 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.578+0000 7f6c71280b80 1 --2- 172.21.15.133:0/775886858 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55afba9c2630 0x55afba9c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.579+0000 7f6c71280b80 4 client.15978 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:25:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.579+0000 7f6c71280b80 1 -- 172.21.15.133:0/775886858 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55afba878600 con 0x55afba9c2630 2023-11-27T15:25:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.579+0000 7f6c56ffd700 1 --2- 172.21.15.133:0/775886858 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55afba9c2630 0x55afba9c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.579+0000 7f6c56ffd700 1 --2- 172.21.15.133:0/775886858 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55afba9c2630 0x55afba9c4a40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a48e01da3c01ccff server_cookie=f708c8ca674d7991 in_seq=0 out_seq=0 2023-11-27T15:25:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:19.579+0000 7f6c36ffd700 10 client.15978 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:21.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:20 smithi174 ceph-mon[131110]: pgmap v421: 129 pgs: 129 active+clean; 21 MiB data, 166 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 73 op/s 2023-11-27T15:25:21.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:20 smithi133 ceph-mon[97860]: pgmap v421: 129 pgs: 129 active+clean; 21 MiB data, 166 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 73 op/s 2023-11-27T15:25:21.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:20 smithi177 ceph-mon[131504]: pgmap v421: 129 pgs: 129 active+clean; 21 MiB data, 166 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 73 op/s 2023-11-27T15:25:23.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:22 smithi174 ceph-mon[131110]: pgmap v422: 129 pgs: 129 active+clean; 25 MiB data, 179 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 137 op/s 2023-11-27T15:25:23.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:22 smithi133 ceph-mon[97860]: pgmap v422: 129 pgs: 129 active+clean; 25 MiB data, 179 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 137 op/s 2023-11-27T15:25:23.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:22 smithi177 ceph-mon[131504]: pgmap v422: 129 pgs: 129 active+clean; 25 MiB data, 179 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 137 op/s 2023-11-27T15:25:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.840+0000 7f6c36ffd700 1 -- 172.21.15.133:0/775886858 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55afba878600 con 0x55afba9c2630 2023-11-27T15:25:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.840+0000 7f6c36ffd700 10 client.15978 handle_command_reply: tid=0 2023-11-27T15:25:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.840+0000 7f6c71280b80 1 client.15978 shutdown 2023-11-27T15:25:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.840+0000 7f6c71280b80 1 -- 172.21.15.133:0/775886858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba959b10 msgr2=0x55afba9684c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.840+0000 7f6c71280b80 1 --2- 172.21.15.133:0/775886858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba959b10 0x55afba9684c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6c5009dae0 tx=0x7f6c500a1d30 comp rx=0 tx=0).stop 2023-11-27T15:25:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.840+0000 7f6c71280b80 1 -- 172.21.15.133:0/775886858 shutdown_connections 2023-11-27T15:25:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.840+0000 7f6c71280b80 1 --2- 172.21.15.133:0/775886858 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55afba9c2630 0x55afba9c4a40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.840+0000 7f6c71280b80 1 --2- 172.21.15.133:0/775886858 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55afba959b10 0x55afba9684c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.840+0000 7f6c71280b80 1 --2- 172.21.15.133:0/775886858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afba7b46b0 0x55afba964f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.840+0000 7f6c36ffd700 0 client.15978 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.840+0000 7f6c71280b80 1 --2- 172.21.15.133:0/775886858 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55afba958db0 0x55afba9654a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.840+0000 7f6c71280b80 1 -- 172.21.15.133:0/775886858 >> 172.21.15.133:0/775886858 conn(0x55afba94df50 msgr2=0x55afba95feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.840+0000 7f6c71280b80 1 -- 172.21.15.133:0/775886858 shutdown_connections 2023-11-27T15:25:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.840+0000 7f6c71280b80 1 -- 172.21.15.133:0/775886858 wait complete. 2023-11-27T15:25:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.840+0000 7f6c71280b80 20 client.15978 trim_cache size 0 max 16384 2023-11-27T15:25:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:25:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:25:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:25:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:25:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.841+0000 7f6c5d9ff700 1 -- 172.21.15.133:0/3650871645 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6c400a11d0 msgr2=0x7f6c400a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.841+0000 7f6c5d9ff700 1 --2- 172.21.15.133:0/3650871645 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6c400a11d0 0x7f6c400a3680 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800c010 tx=0x7f6c48019040 comp rx=0 tx=0).stop 2023-11-27T15:25:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.841+0000 7f6c5d9ff700 1 -- 172.21.15.133:0/3650871645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c58105f00 msgr2=0x7f6c5819f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.841+0000 7f6c5d9ff700 1 --2- 172.21.15.133:0/3650871645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c58105f00 0x7f6c5819f7b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000eb80 tx=0x7f6c5000ee90 comp rx=0 tx=0).stop 2023-11-27T15:25:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.841+0000 7f6c5d9ff700 1 -- 172.21.15.133:0/3650871645 shutdown_connections 2023-11-27T15:25:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.841+0000 7f6c5d9ff700 1 --2- 172.21.15.133:0/3650871645 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6c400a11d0 0x7f6c400a3680 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.841+0000 7f6c5d9ff700 1 --2- 172.21.15.133:0/3650871645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c58105f00 0x7f6c5819f7b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.841+0000 7f6c5d9ff700 1 --2- 172.21.15.133:0/3650871645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6c58100cc0 0x7f6c58198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.841+0000 7f6c5d9ff700 1 --2- 172.21.15.133:0/3650871645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6c581035e0 0x7f6c58198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.841+0000 7f6c5d9ff700 1 -- 172.21.15.133:0/3650871645 >> 172.21.15.133:0/3650871645 conn(0x7f6c580fa720 msgr2=0x7f6c58101920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.841+0000 7f6c5d9ff700 1 -- 172.21.15.133:0/3650871645 shutdown_connections 2023-11-27T15:25:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:24.841+0000 7f6c5d9ff700 1 -- 172.21.15.133:0/3650871645 wait complete. 2023-11-27T15:25:25.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:24 smithi174 ceph-mon[131110]: pgmap v423: 129 pgs: 129 active+clean; 32 MiB data, 209 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 458 op/s 2023-11-27T15:25:25.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:24 smithi133 ceph-mon[97860]: pgmap v423: 129 pgs: 129 active+clean; 32 MiB data, 209 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 458 op/s 2023-11-27T15:25:25.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:24 smithi177 ceph-mon[131504]: pgmap v423: 129 pgs: 129 active+clean; 32 MiB data, 209 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 458 op/s 2023-11-27T15:25:27.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:26 smithi174 ceph-mon[131110]: pgmap v424: 129 pgs: 129 active+clean; 30 MiB data, 293 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 2.00k op/s 2023-11-27T15:25:27.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:26 smithi133 ceph-mon[97860]: pgmap v424: 129 pgs: 129 active+clean; 30 MiB data, 293 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 2.00k op/s 2023-11-27T15:25:27.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:26 smithi177 ceph-mon[131504]: pgmap v424: 129 pgs: 129 active+clean; 30 MiB data, 293 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 2.00k op/s 2023-11-27T15:25:29.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:28 smithi133 ceph-mon[97860]: pgmap v425: 129 pgs: 129 active+clean; 22 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 2.67k op/s 2023-11-27T15:25:29.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:28 smithi177 ceph-mon[131504]: pgmap v425: 129 pgs: 129 active+clean; 22 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 2.67k op/s 2023-11-27T15:25:29.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:28 smithi174 ceph-mon[131110]: pgmap v425: 129 pgs: 129 active+clean; 22 MiB data, 339 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 2.67k op/s 2023-11-27T15:25:31.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:30 smithi174 ceph-mon[131110]: pgmap v426: 129 pgs: 129 active+clean; 12 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 2.68k op/s 2023-11-27T15:25:31.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:30 smithi133 ceph-mon[97860]: pgmap v426: 129 pgs: 129 active+clean; 12 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 2.68k op/s 2023-11-27T15:25:31.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:30 smithi177 ceph-mon[131504]: pgmap v426: 129 pgs: 129 active+clean; 12 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 2.68k op/s 2023-11-27T15:25:33.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:32 smithi174 ceph-mon[131110]: pgmap v427: 129 pgs: 129 active+clean; 17 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 2.67k op/s 2023-11-27T15:25:33.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:32 smithi133 ceph-mon[97860]: pgmap v427: 129 pgs: 129 active+clean; 17 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 2.67k op/s 2023-11-27T15:25:33.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:32 smithi177 ceph-mon[131504]: pgmap v427: 129 pgs: 129 active+clean; 17 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 2.67k op/s 2023-11-27T15:25:35.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:34 smithi174 ceph-mon[131110]: pgmap v428: 129 pgs: 129 active+clean; 17 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 2.60k op/s 2023-11-27T15:25:35.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:34 smithi133 ceph-mon[97860]: pgmap v428: 129 pgs: 129 active+clean; 17 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 2.60k op/s 2023-11-27T15:25:35.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:34 smithi177 ceph-mon[131504]: pgmap v428: 129 pgs: 129 active+clean; 17 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 2.60k op/s 2023-11-27T15:25:37.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:36 smithi174 ceph-mon[131110]: pgmap v429: 129 pgs: 129 active+clean; 27 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 2.29k op/s 2023-11-27T15:25:37.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:36 smithi133 ceph-mon[97860]: pgmap v429: 129 pgs: 129 active+clean; 27 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 2.29k op/s 2023-11-27T15:25:37.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:36 smithi177 ceph-mon[131504]: pgmap v429: 129 pgs: 129 active+clean; 27 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 2.29k op/s 2023-11-27T15:25:38.034 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.030+0000 7f87fe7ff700 1 -- 172.21.15.133:0/944703017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87f81035e0 msgr2=0x7f87f81059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.030+0000 7f87fe7ff700 1 --2- 172.21.15.133:0/944703017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87f81035e0 0x7f87f81059c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0098f0 tx=0x7f87ec009c00 comp rx=0 tx=0).stop 2023-11-27T15:25:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.031+0000 7f87fe7ff700 1 -- 172.21.15.133:0/944703017 shutdown_connections 2023-11-27T15:25:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.031+0000 7f87fe7ff700 1 --2- 172.21.15.133:0/944703017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87f8105f00 0x7f87f8108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.031+0000 7f87fe7ff700 1 --2- 172.21.15.133:0/944703017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87f8100cc0 0x7f87f81030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.031+0000 7f87fe7ff700 1 --2- 172.21.15.133:0/944703017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87f81035e0 0x7f87f81059c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.031+0000 7f87fe7ff700 1 -- 172.21.15.133:0/944703017 >> 172.21.15.133:0/944703017 conn(0x7f87f80fa720 msgr2=0x7f87f80fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.031+0000 7f87fe7ff700 1 -- 172.21.15.133:0/944703017 shutdown_connections 2023-11-27T15:25:38.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.031+0000 7f87fe7ff700 1 -- 172.21.15.133:0/944703017 wait complete. 2023-11-27T15:25:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.032+0000 7f87fe7ff700 1 Processor -- start 2023-11-27T15:25:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.032+0000 7f87fe7ff700 1 -- start start 2023-11-27T15:25:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.033+0000 7f87fe7ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87f8100cc0 0x7f87f81982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.033+0000 7f87fe7ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87f81035e0 0x7f87f81987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:38.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.033+0000 7f87fe7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87f8105f00 0x7f87f819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.033+0000 7f87fe7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f819fe30 con 0x7f87f8105f00 2023-11-27T15:25:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.033+0000 7f87fe7ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f819ff70 con 0x7f87f81035e0 2023-11-27T15:25:38.037 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.033+0000 7f87fe7ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f81a0100 con 0x7f87f8100cc0 2023-11-27T15:25:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.033+0000 7f87f7fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87f8100cc0 0x7f87f81982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.033+0000 7f87fcd9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87f8105f00 0x7f87f819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.033+0000 7f87f7fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87f8100cc0 0x7f87f81982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48000/0 (socket says 172.21.15.133:48000) 2023-11-27T15:25:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.033+0000 7f87f7fff700 1 -- 172.21.15.133:0/3733308106 learned_addr learned my addr 172.21.15.133:0/3733308106 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.034+0000 7f87f77fe700 1 --2- 172.21.15.133:0/3733308106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87f81035e0 0x7f87f81987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.034+0000 7f87f7fff700 1 -- 172.21.15.133:0/3733308106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87f81035e0 msgr2=0x7f87f81987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.034+0000 7f87f7fff700 1 --2- 172.21.15.133:0/3733308106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87f81035e0 0x7f87f81987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.034+0000 7f87f7fff700 1 -- 172.21.15.133:0/3733308106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87f8105f00 msgr2=0x7f87f819f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.034+0000 7f87f7fff700 1 --2- 172.21.15.133:0/3733308106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87f8105f00 0x7f87f819f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.034+0000 7f87f7fff700 1 -- 172.21.15.133:0/3733308106 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec009520 con 0x7f87f8100cc0 2023-11-27T15:25:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.034+0000 7f87fcd9c700 1 --2- 172.21.15.133:0/3733308106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87f8105f00 0x7f87f819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.034+0000 7f87f77fe700 1 --2- 172.21.15.133:0/3733308106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87f81035e0 0x7f87f81987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.034+0000 7f87f7fff700 1 --2- 172.21.15.133:0/3733308106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87f8100cc0 0x7f87f81982b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f87e800b530 tx=0x7f87e800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.034+0000 7f87f57fa700 1 -- 172.21.15.133:0/3733308106 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8011830 con 0x7f87f8100cc0 2023-11-27T15:25:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.034+0000 7f87f57fa700 1 -- 172.21.15.133:0/3733308106 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87e8011e50 con 0x7f87f8100cc0 2023-11-27T15:25:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.034+0000 7f87fe7ff700 1 -- 172.21.15.133:0/3733308106 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f810bc50 con 0x7f87f8100cc0 2023-11-27T15:25:38.039 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.035+0000 7f87fe7ff700 1 -- 172.21.15.133:0/3733308106 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f810bac0 con 0x7f87f8100cc0 2023-11-27T15:25:38.040 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.035+0000 7f87f57fa700 1 -- 172.21.15.133:0/3733308106 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e800f2e0 con 0x7f87f8100cc0 2023-11-27T15:25:38.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.039+0000 7f87f57fa700 1 -- 172.21.15.133:0/3733308106 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f87e8011990 con 0x7f87f8100cc0 2023-11-27T15:25:38.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.040+0000 7f88121deb80 1 Processor -- start 2023-11-27T15:25:38.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.040+0000 7f88121deb80 1 -- start start 2023-11-27T15:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.041+0000 7f88121deb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af660ab6b0 0x55af6624f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.041+0000 7f88121deb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af6624fdb0 0x55af662505d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.041+0000 7f88121deb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af66250b10 0x55af66250f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.041+0000 7f88121deb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af66155000 con 0x55af660ab6b0 2023-11-27T15:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.041+0000 7f88121deb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af661a7840 con 0x55af66250b10 2023-11-27T15:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.041+0000 7f88121deb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af661a0cd0 con 0x55af6624fdb0 2023-11-27T15:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.041+0000 7f87f77fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af66250b10 0x55af66250f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.041+0000 7f87f77fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af66250b10 0x55af66250f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:58062/0 (socket says 172.21.15.133:58062) 2023-11-27T15:25:38.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.041+0000 7f87f77fe700 1 -- 172.21.15.133:0/1955711828 learned_addr learned my addr 172.21.15.133:0/1955711828 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.041+0000 7f87f57fa700 1 --2- 172.21.15.133:0/3733308106 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f87e00a11d0 0x7f87e00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.041+0000 7f87fcd9c700 1 --2- 172.21.15.133:0/1955711828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af660ab6b0 0x55af6624f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:38.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.041+0000 7f87f57fa700 1 -- 172.21.15.133:0/3733308106 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f87e80cc6a0 con 0x7f87f8100cc0 2023-11-27T15:25:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.042+0000 7f87f77fe700 1 -- 172.21.15.133:0/1955711828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af6624fdb0 msgr2=0x55af662505d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-27T15:25:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.042+0000 7f87f77fe700 1 --2- 172.21.15.133:0/1955711828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af6624fdb0 0x55af662505d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.042+0000 7f87f77fe700 1 -- 172.21.15.133:0/1955711828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af660ab6b0 msgr2=0x55af6624f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.042+0000 7f87f77fe700 1 --2- 172.21.15.133:0/1955711828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af660ab6b0 0x55af6624f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.042+0000 7f87f77fe700 1 -- 172.21.15.133:0/1955711828 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af660225d0 con 0x55af66250b10 2023-11-27T15:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.042+0000 7f87fcd9c700 1 --2- 172.21.15.133:0/1955711828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af660ab6b0 0x55af6624f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.042+0000 7f87fcd9c700 1 --2- 172.21.15.133:0/3733308106 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f87e00a11d0 0x7f87e00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.042+0000 7f87f77fe700 1 --2- 172.21.15.133:0/1955711828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af66250b10 0x55af66250f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0098f0 tx=0x7f87ec005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e500becc13dd4801 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.042+0000 7f87de7fc700 1 -- 172.21.15.133:0/1955711828 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec01d070 con 0x55af66250b10 2023-11-27T15:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.043+0000 7f87fcd9c700 1 --2- 172.21.15.133:0/3733308106 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f87e00a11d0 0x7f87e00a3680 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f87f00106d0 tx=0x7f87f0004b20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.043+0000 7f87de7fc700 1 -- 172.21.15.133:0/1955711828 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87ec00fe10 con 0x55af66250b10 2023-11-27T15:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.043+0000 7f87de7fc700 1 -- 172.21.15.133:0/1955711828 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec00b890 con 0x55af66250b10 2023-11-27T15:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.043+0000 7f88121deb80 1 -- 172.21.15.133:0/1955711828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af66250b10 msgr2=0x55af66250f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.043+0000 7f88121deb80 1 --2- 172.21.15.133:0/1955711828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af66250b10 0x55af66250f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0098f0 tx=0x7f87ec005e40 comp rx=0 tx=0).stop 2023-11-27T15:25:38.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.044+0000 7f88121deb80 1 -- 172.21.15.133:0/1955711828 shutdown_connections 2023-11-27T15:25:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.044+0000 7f88121deb80 1 --2- 172.21.15.133:0/1955711828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af66250b10 0x55af66250f80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.044+0000 7f88121deb80 1 --2- 172.21.15.133:0/1955711828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af660ab6b0 0x55af6624f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.044+0000 7f88121deb80 1 --2- 172.21.15.133:0/1955711828 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af6624fdb0 0x55af662505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.044+0000 7f88121deb80 1 -- 172.21.15.133:0/1955711828 >> 172.21.15.133:0/1955711828 conn(0x55af66244f50 msgr2=0x55af6624a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.044+0000 7f88121deb80 1 -- 172.21.15.133:0/1955711828 shutdown_connections 2023-11-27T15:25:38.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.044+0000 7f88121deb80 1 -- 172.21.15.133:0/1955711828 wait complete. 2023-11-27T15:25:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.044+0000 7f88121deb80 1 -- start start 2023-11-27T15:25:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.045+0000 7f88121deb80 1 Processor -- start 2023-11-27T15:25:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.045+0000 7f88121deb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af660ab6b0 0x55af66258c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.045+0000 7f88121deb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af6624fdb0 0x55af662591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:38.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.045+0000 7f88121deb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af66250b10 0x55af66260200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.045+0000 7f88121deb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af66142880 con 0x55af66250b10 2023-11-27T15:25:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.045+0000 7f88121deb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af66142520 con 0x55af660ab6b0 2023-11-27T15:25:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.045+0000 7f88121deb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af660248c0 con 0x55af6624fdb0 2023-11-27T15:25:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.046+0000 7f88121deb80 10 client.0 _ll_register_callbacks cb 0x55af66160720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.046+0000 7f87f7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af660ab6b0 0x55af66258c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.046+0000 7f87f7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af660ab6b0 0x55af66258c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:58066/0 (socket says 172.21.15.133:58066) 2023-11-27T15:25:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.046+0000 7f87f7fff700 1 -- 172.21.15.133:0/822369153 learned_addr learned my addr 172.21.15.133:0/822369153 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.046+0000 7f87f77fe700 1 --2- 172.21.15.133:0/822369153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af6624fdb0 0x55af662591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.046+0000 7f87f77fe700 1 -- 172.21.15.133:0/822369153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af660ab6b0 msgr2=0x55af66258c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.046+0000 7f87f77fe700 1 --2- 172.21.15.133:0/822369153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af660ab6b0 0x55af66258c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.046+0000 7f87f77fe700 1 -- 172.21.15.133:0/822369153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af66250b10 msgr2=0x55af66260200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:38.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.046+0000 7f87f77fe700 1 --2- 172.21.15.133:0/822369153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af66250b10 0x55af66260200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.046+0000 7f87f77fe700 1 -- 172.21.15.133:0/822369153 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f8108f50 con 0x55af6624fdb0 2023-11-27T15:25:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.047+0000 7f87f77fe700 1 --2- 172.21.15.133:0/822369153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af6624fdb0 0x55af662591c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009810 tx=0x7f87ec00b800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b5076c5b20b1be9 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.047+0000 7f87d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:25:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.047+0000 7f87d77fe700 1 -- 172.21.15.133:0/822369153 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec01d040 con 0x55af6624fdb0 2023-11-27T15:25:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.047+0000 7f87d77fe700 1 -- 172.21.15.133:0/822369153 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f87ec005270 con 0x55af6624fdb0 2023-11-27T15:25:38.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.047+0000 7f88121deb80 1 -- 172.21.15.133:0/822369153 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af66242710 con 0x55af6624fdb0 2023-11-27T15:25:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.047+0000 7f87d77fe700 1 -- 172.21.15.133:0/822369153 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87ec01d040 con 0x55af6624fdb0 2023-11-27T15:25:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.047+0000 7f88121deb80 10 client.26011 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.047+0000 7f88121deb80 1 -- 172.21.15.133:0/822369153 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55af66167130 con 0x55af6624fdb0 2023-11-27T15:25:38.053 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.048+0000 7f87d77fe700 1 -- 172.21.15.133:0/822369153 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55af66242710 con 0x55af6624fdb0 2023-11-27T15:25:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.049+0000 7f87d77fe700 1 -- 172.21.15.133:0/822369153 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 24) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f87ec015020 con 0x55af6624fdb0 2023-11-27T15:25:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.049+0000 7f88121deb80 10 client.26011 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.049+0000 7f88121deb80 10 client.26011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.049+0000 7f88121deb80 20 client.26011 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.049+0000 7f88121deb80 1 --2- 172.21.15.133:0/822369153 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55af66264610 0x55af66266a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.050+0000 7f88121deb80 4 client.26011 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.050+0000 7f88121deb80 1 -- 172.21.15.133:0/822369153 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af66163040 con 0x55af66264610 2023-11-27T15:25:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.050+0000 7f87f7fff700 1 --2- 172.21.15.133:0/822369153 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55af66264610 0x55af66266a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.050+0000 7f87d77fe700 10 client.26011 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:38.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.050+0000 7f87f7fff700 1 --2- 172.21.15.133:0/822369153 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55af66264610 0x55af66266a20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18a18f7a4b1f83d4 server_cookie=14b70b12a79401bf in_seq=0 out_seq=0 2023-11-27T15:25:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.052+0000 7f87d77fe700 1 -- 172.21.15.133:0/822369153 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55af66163040 con 0x55af66264610 2023-11-27T15:25:38.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.052+0000 7f87d77fe700 10 client.26011 handle_command_reply: tid=0 2023-11-27T15:25:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.052+0000 7f88121deb80 1 client.26011 shutdown 2023-11-27T15:25:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.052+0000 7f88121deb80 1 -- 172.21.15.133:0/822369153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af6624fdb0 msgr2=0x55af662591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:38.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.052+0000 7f88121deb80 1 --2- 172.21.15.133:0/822369153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af6624fdb0 0x55af662591c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009810 tx=0x7f87ec00b800 comp rx=0 tx=0).stop 2023-11-27T15:25:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.052+0000 7f88121deb80 1 -- 172.21.15.133:0/822369153 shutdown_connections 2023-11-27T15:25:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.052+0000 7f88121deb80 1 --2- 172.21.15.133:0/822369153 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55af66264610 0x55af66266a20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.053+0000 7f88121deb80 1 --2- 172.21.15.133:0/822369153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af66250b10 0x55af66260200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.053+0000 7f88121deb80 1 --2- 172.21.15.133:0/822369153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af660ab6b0 0x55af66258c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.053+0000 7f88121deb80 1 --2- 172.21.15.133:0/822369153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af6624fdb0 0x55af662591c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.053+0000 7f88121deb80 1 -- 172.21.15.133:0/822369153 >> 172.21.15.133:0/822369153 conn(0x55af66244f50 msgr2=0x55af662497e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.053+0000 7f87d77fe700 0 client.26011 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.054+0000 7f88121deb80 1 -- 172.21.15.133:0/822369153 shutdown_connections 2023-11-27T15:25:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.054+0000 7f88121deb80 1 -- 172.21.15.133:0/822369153 wait complete. 2023-11-27T15:25:38.058 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.054+0000 7f88121deb80 20 client.26011 trim_cache size 0 max 16384 2023-11-27T15:25:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.075+0000 7f88121deb80 1 Processor -- start 2023-11-27T15:25:38.078 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.075+0000 7f88121deb80 1 -- start start 2023-11-27T15:25:38.079 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.075+0000 7f88121deb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af660ab6b0 0x55af66256e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.075+0000 7f88121deb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af6624fdb0 0x55af662573d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.075+0000 7f88121deb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af66250b10 0x55af662601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.075+0000 7f88121deb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af66244b70 con 0x55af66250b10 2023-11-27T15:25:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.075+0000 7f88121deb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af66250180 con 0x55af6624fdb0 2023-11-27T15:25:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.075+0000 7f88121deb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af66259c70 con 0x55af660ab6b0 2023-11-27T15:25:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.076+0000 7f87f77fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af6624fdb0 0x55af662573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.076+0000 7f87f77fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af6624fdb0 0x55af662573d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:58078/0 (socket says 172.21.15.133:58078) 2023-11-27T15:25:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.076+0000 7f87f77fe700 1 -- 172.21.15.133:0/2657182602 learned_addr learned my addr 172.21.15.133:0/2657182602 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.076+0000 7f87f7fff700 1 --2- 172.21.15.133:0/2657182602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af660ab6b0 0x55af66256e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.076+0000 7f87fcd9c700 1 --2- 172.21.15.133:0/2657182602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af66250b10 0x55af662601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.076+0000 7f87f77fe700 1 -- 172.21.15.133:0/2657182602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af660ab6b0 msgr2=0x55af66256e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.076+0000 7f87f77fe700 1 --2- 172.21.15.133:0/2657182602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af660ab6b0 0x55af66256e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.076+0000 7f87f77fe700 1 -- 172.21.15.133:0/2657182602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af66250b10 msgr2=0x55af662601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.076+0000 7f87f77fe700 1 --2- 172.21.15.133:0/2657182602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af66250b10 0x55af662601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.076+0000 7f87f77fe700 1 -- 172.21.15.133:0/2657182602 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af66166e80 con 0x55af6624fdb0 2023-11-27T15:25:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.076+0000 7f87f7fff700 1 --2- 172.21.15.133:0/2657182602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af660ab6b0 0x55af66256e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.076+0000 7f87f77fe700 1 --2- 172.21.15.133:0/2657182602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af6624fdb0 0x55af662573d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f87ec011010 tx=0x7f87ec009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c32756b78ad58be server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.077+0000 7f87deffd700 1 -- 172.21.15.133:0/2657182602 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec01d040 con 0x55af6624fdb0 2023-11-27T15:25:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.077+0000 7f87deffd700 1 -- 172.21.15.133:0/2657182602 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87ec00b920 con 0x55af6624fdb0 2023-11-27T15:25:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.077+0000 7f87deffd700 1 -- 172.21.15.133:0/2657182602 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec003890 con 0x55af6624fdb0 2023-11-27T15:25:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.078+0000 7f88121deb80 1 -- 172.21.15.133:0/2657182602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af6624fdb0 msgr2=0x55af662573d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.078+0000 7f88121deb80 1 --2- 172.21.15.133:0/2657182602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af6624fdb0 0x55af662573d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f87ec011010 tx=0x7f87ec009f50 comp rx=0 tx=0).stop 2023-11-27T15:25:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.078+0000 7f88121deb80 1 -- 172.21.15.133:0/2657182602 shutdown_connections 2023-11-27T15:25:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.078+0000 7f88121deb80 1 --2- 172.21.15.133:0/2657182602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af66250b10 0x55af662601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.078+0000 7f88121deb80 1 --2- 172.21.15.133:0/2657182602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af660ab6b0 0x55af66256e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.078+0000 7f88121deb80 1 --2- 172.21.15.133:0/2657182602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af6624fdb0 0x55af662573d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.078+0000 7f88121deb80 1 -- 172.21.15.133:0/2657182602 >> 172.21.15.133:0/2657182602 conn(0x55af66244f50 msgr2=0x55af66266620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.078+0000 7f88121deb80 1 -- 172.21.15.133:0/2657182602 shutdown_connections 2023-11-27T15:25:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.078+0000 7f88121deb80 1 -- 172.21.15.133:0/2657182602 wait complete. 2023-11-27T15:25:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.079+0000 7f88121deb80 1 -- start start 2023-11-27T15:25:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.079+0000 7f88121deb80 1 Processor -- start 2023-11-27T15:25:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.079+0000 7f88121deb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af660ab6b0 0x55af6625b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.079+0000 7f88121deb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af6624fdb0 0x55af6625b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.079+0000 7f88121deb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af66250b10 0x55af6625e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.079+0000 7f88121deb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af6625ecd0 con 0x55af660ab6b0 2023-11-27T15:25:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.079+0000 7f88121deb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af6625ee40 con 0x55af66250b10 2023-11-27T15:25:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.079+0000 7f88121deb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af6625f000 con 0x55af6624fdb0 2023-11-27T15:25:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.080+0000 7f87fcd9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af66250b10 0x55af6625e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.080+0000 7f88121deb80 10 client.0 _ll_register_callbacks cb 0x7f87f0009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.080+0000 7f87fcd9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af66250b10 0x55af6625e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:58084/0 (socket says 172.21.15.133:58084) 2023-11-27T15:25:38.085 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.080+0000 7f87fcd9c700 1 -- 172.21.15.133:0/1152709591 learned_addr learned my addr 172.21.15.133:0/1152709591 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.080+0000 7f87f7fff700 1 --2- 172.21.15.133:0/1152709591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af660ab6b0 0x55af6625b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.080+0000 7f87f77fe700 1 --2- 172.21.15.133:0/1152709591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af6624fdb0 0x55af6625b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.080+0000 7f87fcd9c700 1 -- 172.21.15.133:0/1152709591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af6624fdb0 msgr2=0x55af6625b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.080+0000 7f87fcd9c700 1 --2- 172.21.15.133:0/1152709591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af6624fdb0 0x55af6625b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.080+0000 7f87fcd9c700 1 -- 172.21.15.133:0/1152709591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af660ab6b0 msgr2=0x55af6625b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.080+0000 7f87fcd9c700 1 --2- 172.21.15.133:0/1152709591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af660ab6b0 0x55af6625b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.080+0000 7f87fcd9c700 1 -- 172.21.15.133:0/1152709591 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f810bac0 con 0x55af66250b10 2023-11-27T15:25:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.081+0000 7f87fcd9c700 1 --2- 172.21.15.133:0/1152709591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af66250b10 0x55af6625e790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f87f0007b40 tx=0x7f87f00099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b4bcc8e6d97d368 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.081+0000 7f87d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:25:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.081+0000 7f87d77fe700 1 -- 172.21.15.133:0/1152709591 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f000e040 con 0x55af66250b10 2023-11-27T15:25:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.081+0000 7f87d77fe700 1 -- 172.21.15.133:0/1152709591 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f87f001b030 con 0x55af66250b10 2023-11-27T15:25:38.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.081+0000 7f88121deb80 1 -- 172.21.15.133:0/1152709591 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af6616f600 con 0x55af66250b10 2023-11-27T15:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.081+0000 7f87d77fe700 1 -- 172.21.15.133:0/1152709591 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87f0015bc0 con 0x55af66250b10 2023-11-27T15:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.081+0000 7f88121deb80 10 client.26036 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.081+0000 7f88121deb80 1 -- 172.21.15.133:0/1152709591 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55af66167130 con 0x55af66250b10 2023-11-27T15:25:38.087 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.082+0000 7f87d77fe700 1 -- 172.21.15.133:0/1152709591 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55af6616f600 con 0x55af66250b10 2023-11-27T15:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.082+0000 7f87d77fe700 1 -- 172.21.15.133:0/1152709591 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 24) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f87f002b3b0 con 0x55af66250b10 2023-11-27T15:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.082+0000 7f88121deb80 10 client.26036 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.082+0000 7f88121deb80 10 client.26036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.082+0000 7f88121deb80 20 client.26036 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.082+0000 7f88121deb80 1 --2- 172.21.15.133:0/1152709591 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55af662b88d0 0x55af662bace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.082+0000 7f88121deb80 4 client.26036 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.082+0000 7f88121deb80 1 -- 172.21.15.133:0/1152709591 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55af65feff10 con 0x55af662b88d0 2023-11-27T15:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.083+0000 7f87f7fff700 1 --2- 172.21.15.133:0/1152709591 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55af662b88d0 0x55af662bace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.083+0000 7f87f7fff700 1 --2- 172.21.15.133:0/1152709591 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55af662b88d0 0x55af662bace0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b01c7a60faddc4e0 server_cookie=e4d5290d39a3419e in_seq=0 out_seq=0 2023-11-27T15:25:38.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:38.083+0000 7f87d77fe700 10 client.26036 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:39.028 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:38 smithi133 ceph-mon[97860]: pgmap v430: 129 pgs: 129 active+clean; 33 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 768 op/s 2023-11-27T15:25:39.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:38 smithi177 ceph-mon[131504]: pgmap v430: 129 pgs: 129 active+clean; 33 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 768 op/s 2023-11-27T15:25:39.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:38 smithi174 ceph-mon[131110]: pgmap v430: 129 pgs: 129 active+clean; 33 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.6 MiB/s wr, 768 op/s 2023-11-27T15:25:39.372 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:25:39.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.921+0000 7fec7719d700 1 -- 172.21.15.133:0/1163568423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec70107850 msgr2=0x7fec70107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:39.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.921+0000 7fec7719d700 1 --2- 172.21.15.133:0/1163568423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec70107850 0x7fec70107ca0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009990 tx=0x7fec6c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:39.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.922+0000 7fec7719d700 1 -- 172.21.15.133:0/1163568423 shutdown_connections 2023-11-27T15:25:39.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.922+0000 7fec7719d700 1 --2- 172.21.15.133:0/1163568423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec701081e0 0x7fec7010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:39.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.922+0000 7fec7719d700 1 --2- 172.21.15.133:0/1163568423 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec70106660 0x7fec70106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:39.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.922+0000 7fec7719d700 1 --2- 172.21.15.133:0/1163568423 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec70107850 0x7fec70107ca0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:39.926 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.922+0000 7fec7719d700 1 -- 172.21.15.133:0/1163568423 >> 172.21.15.133:0/1163568423 conn(0x7fec70075690 msgr2=0x7fec70077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:39.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.923+0000 7fec7719d700 1 -- 172.21.15.133:0/1163568423 shutdown_connections 2023-11-27T15:25:39.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.924+0000 7fec7719d700 1 -- 172.21.15.133:0/1163568423 wait complete. 2023-11-27T15:25:39.928 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.925+0000 7fec7719d700 1 Processor -- start 2023-11-27T15:25:39.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.925+0000 7fec7719d700 1 -- start start 2023-11-27T15:25:39.929 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.926+0000 7fec7719d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec70106660 0x7fec7019c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:39.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.926+0000 7fec7719d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec70107850 0x7fec7019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:39.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.926+0000 7fec7719d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec701081e0 0x7fec701a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:39.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.926+0000 7fec7719d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec701a3f90 con 0x7fec70107850 2023-11-27T15:25:39.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.926+0000 7fec7719d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec701a40d0 con 0x7fec701081e0 2023-11-27T15:25:39.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.926+0000 7fec7719d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec701a4260 con 0x7fec70106660 2023-11-27T15:25:39.930 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.927+0000 7fec7619b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec70106660 0x7fec7019c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:39.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.927+0000 7fec7619b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec70106660 0x7fec7019c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48042/0 (socket says 172.21.15.133:48042) 2023-11-27T15:25:39.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.927+0000 7fec7619b700 1 -- 172.21.15.133:0/1911401645 learned_addr learned my addr 172.21.15.133:0/1911401645 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:39.931 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.927+0000 7fec7599a700 1 --2- 172.21.15.133:0/1911401645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec70107850 0x7fec7019c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:39.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.928+0000 7fec7619b700 1 -- 172.21.15.133:0/1911401645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec701081e0 msgr2=0x7fec701a39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:39.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.928+0000 7fec7699c700 1 --2- 172.21.15.133:0/1911401645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec701081e0 0x7fec701a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:39.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.928+0000 7fec7619b700 1 --2- 172.21.15.133:0/1911401645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec701081e0 0x7fec701a39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:39.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.928+0000 7fec7619b700 1 -- 172.21.15.133:0/1911401645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec70107850 msgr2=0x7fec7019c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:39.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.928+0000 7fec7619b700 1 --2- 172.21.15.133:0/1911401645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec70107850 0x7fec7019c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:39.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.928+0000 7fec7619b700 1 -- 172.21.15.133:0/1911401645 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c0095f0 con 0x7fec70106660 2023-11-27T15:25:39.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.928+0000 7fec7599a700 1 --2- 172.21.15.133:0/1911401645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec70107850 0x7fec7019c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:39.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.928+0000 7fec7699c700 1 --2- 172.21.15.133:0/1911401645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec701081e0 0x7fec701a39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:39.932 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.928+0000 7fec7619b700 1 --2- 172.21.15.133:0/1911401645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec70106660 0x7fec7019c410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fec6800b9c0 tx=0x7fec6800bcd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:39.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.929+0000 7fec677fe700 1 -- 172.21.15.133:0/1911401645 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6800c950 con 0x7fec70106660 2023-11-27T15:25:39.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.929+0000 7fec677fe700 1 -- 172.21.15.133:0/1911401645 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec68014440 con 0x7fec70106660 2023-11-27T15:25:39.933 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.929+0000 7fec677fe700 1 -- 172.21.15.133:0/1911401645 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec680125a0 con 0x7fec70106660 2023-11-27T15:25:39.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.930+0000 7fec7719d700 1 -- 172.21.15.133:0/1911401645 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec701120d0 con 0x7fec70106660 2023-11-27T15:25:39.934 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.931+0000 7fec7719d700 1 -- 172.21.15.133:0/1911401645 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec70111f40 con 0x7fec70106660 2023-11-27T15:25:39.942 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.938+0000 7fec677fe700 1 -- 172.21.15.133:0/1911401645 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fec680148b0 con 0x7fec70106660 2023-11-27T15:25:39.945 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.941+0000 7fec8bddbb80 1 Processor -- start 2023-11-27T15:25:39.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.943+0000 7fec8bddbb80 1 -- start start 2023-11-27T15:25:39.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.944+0000 7fec677fe700 1 --2- 172.21.15.133:0/1911401645 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fec5c0a12a0 0x7fec5c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:39.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.945+0000 7fec8bddbb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a272e90 0x55f36a2732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:39.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.945+0000 7fec8bddbb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2737e0 0x55f36a273d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:39.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.945+0000 7fec8bddbb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a2742b0 0x55f36a274750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:39.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.945+0000 7fec8bddbb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36a178060 con 0x55f36a2742b0 2023-11-27T15:25:39.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.945+0000 7fec8bddbb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36a1ca8a0 con 0x55f36a2737e0 2023-11-27T15:25:39.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.945+0000 7fec8bddbb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36a1c3d30 con 0x55f36a272e90 2023-11-27T15:25:39.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.946+0000 7fec7699c700 1 --2- 172.21.15.133:0/1911401645 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fec5c0a12a0 0x7fec5c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:39.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.946+0000 7fec677fe700 1 -- 172.21.15.133:0/1911401645 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fec68006e40 con 0x7fec70106660 2023-11-27T15:25:39.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.946+0000 7fec7599a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2737e0 0x55f36a273d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:39.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.947+0000 7fec7619b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a272e90 0x55f36a2732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:39.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.947+0000 7fec7619b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a272e90 0x55f36a2732a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48058/0 (socket says 172.21.15.133:48058) 2023-11-27T15:25:39.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.947+0000 7fec7619b700 1 -- 172.21.15.133:0/4146876235 learned_addr learned my addr 172.21.15.133:0/4146876235 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:39.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.947+0000 7fec7599a700 1 -- 172.21.15.133:0/4146876235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a272e90 msgr2=0x55f36a2732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:39.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.947+0000 7fec7599a700 1 --2- 172.21.15.133:0/4146876235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a272e90 0x55f36a2732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:39.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.947+0000 7fec7599a700 1 -- 172.21.15.133:0/4146876235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a2742b0 msgr2=0x55f36a274750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:39.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.947+0000 7fec7599a700 1 --2- 172.21.15.133:0/4146876235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a2742b0 0x55f36a274750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:39.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.947+0000 7fec7599a700 1 -- 172.21.15.133:0/4146876235 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f36a045630 con 0x55f36a2737e0 2023-11-27T15:25:39.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.949+0000 7fec7699c700 1 --2- 172.21.15.133:0/1911401645 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fec5c0a12a0 0x7fec5c0a3750 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fec60006fd0 tx=0x7fec60006ea0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:39.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.949+0000 7fec7599a700 1 --2- 172.21.15.133:0/4146876235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2737e0 0x55f36a273d20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fec6c000c00 tx=0x7fec6c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c524f5d3e67da458 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:39.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.951+0000 7fec64bfe700 1 -- 172.21.15.133:0/4146876235 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c027070 con 0x55f36a2737e0 2023-11-27T15:25:39.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.951+0000 7fec64bfe700 1 -- 172.21.15.133:0/4146876235 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec6c01e040 con 0x55f36a2737e0 2023-11-27T15:25:39.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.951+0000 7fec64bfe700 1 -- 172.21.15.133:0/4146876235 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c01d050 con 0x55f36a2737e0 2023-11-27T15:25:39.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.955+0000 7fec8bddbb80 1 -- 172.21.15.133:0/4146876235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2737e0 msgr2=0x55f36a273d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:39.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.955+0000 7fec8bddbb80 1 --2- 172.21.15.133:0/4146876235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2737e0 0x55f36a273d20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fec6c000c00 tx=0x7fec6c0047c0 comp rx=0 tx=0).stop 2023-11-27T15:25:39.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.956+0000 7fec8bddbb80 1 -- 172.21.15.133:0/4146876235 shutdown_connections 2023-11-27T15:25:39.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.956+0000 7fec8bddbb80 1 --2- 172.21.15.133:0/4146876235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a2742b0 0x55f36a274750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:39.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.956+0000 7fec8bddbb80 1 --2- 172.21.15.133:0/4146876235 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a272e90 0x55f36a2732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:39.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.956+0000 7fec8bddbb80 1 --2- 172.21.15.133:0/4146876235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2737e0 0x55f36a273d20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:39.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.956+0000 7fec8bddbb80 1 -- 172.21.15.133:0/4146876235 >> 172.21.15.133:0/4146876235 conn(0x55f36a2685b0 msgr2=0x55f36a26d850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:39.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.957+0000 7fec8bddbb80 1 -- 172.21.15.133:0/4146876235 shutdown_connections 2023-11-27T15:25:39.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.957+0000 7fec8bddbb80 1 -- 172.21.15.133:0/4146876235 wait complete. 2023-11-27T15:25:39.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.958+0000 7fec8bddbb80 1 -- start start 2023-11-27T15:25:39.962 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.958+0000 7fec8bddbb80 1 Processor -- start 2023-11-27T15:25:39.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.959+0000 7fec8bddbb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a272e90 0x55f36a27c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:39.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.959+0000 7fec8bddbb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2737e0 0x55f36a27cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:39.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.959+0000 7fec8bddbb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a2742b0 0x55f36a283d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:39.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.959+0000 7fec8bddbb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36a1658e0 con 0x55f36a2742b0 2023-11-27T15:25:39.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.959+0000 7fec8bddbb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36a165580 con 0x55f36a2737e0 2023-11-27T15:25:39.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.959+0000 7fec8bddbb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36a047920 con 0x55f36a272e90 2023-11-27T15:25:39.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.959+0000 7fec8bddbb80 10 client.0 _ll_register_callbacks cb 0x55f36a183780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:39.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.960+0000 7fec7619b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a272e90 0x55f36a27c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:39.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.960+0000 7fec7619b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a272e90 0x55f36a27c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48062/0 (socket says 172.21.15.133:48062) 2023-11-27T15:25:39.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.960+0000 7fec7619b700 1 -- 172.21.15.133:0/1299783268 learned_addr learned my addr 172.21.15.133:0/1299783268 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:39.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.960+0000 7fec7599a700 1 --2- 172.21.15.133:0/1299783268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2737e0 0x55f36a27cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:39.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.960+0000 7fec7699c700 1 --2- 172.21.15.133:0/1299783268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a2742b0 0x55f36a283d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:39.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.960+0000 7fec7619b700 1 -- 172.21.15.133:0/1299783268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2737e0 msgr2=0x55f36a27cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:39.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.960+0000 7fec7619b700 1 --2- 172.21.15.133:0/1299783268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2737e0 0x55f36a27cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:39.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.960+0000 7fec7619b700 1 -- 172.21.15.133:0/1299783268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a2742b0 msgr2=0x55f36a283d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:39.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.960+0000 7fec7619b700 1 --2- 172.21.15.133:0/1299783268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a2742b0 0x55f36a283d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:39.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.960+0000 7fec7619b700 1 -- 172.21.15.133:0/1299783268 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7010f3d0 con 0x55f36a272e90 2023-11-27T15:25:39.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.961+0000 7fec7619b700 1 --2- 172.21.15.133:0/1299783268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a272e90 0x55f36a27c7c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fec680c9840 tx=0x7fec68099180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5ae30b26748d6c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:39.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.961+0000 7fec59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:25:39.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.961+0000 7fec59ffb700 1 -- 172.21.15.133:0/1299783268 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68059390 con 0x55f36a272e90 2023-11-27T15:25:39.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.961+0000 7fec59ffb700 1 -- 172.21.15.133:0/1299783268 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fec680594f0 con 0x55f36a272e90 2023-11-27T15:25:39.966 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.962+0000 7fec8bddbb80 1 -- 172.21.15.133:0/1299783268 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f36a265d70 con 0x55f36a272e90 2023-11-27T15:25:39.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.964+0000 7fec59ffb700 1 -- 172.21.15.133:0/1299783268 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec68099360 con 0x55f36a272e90 2023-11-27T15:25:39.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.964+0000 7fec8bddbb80 10 client.26029 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:39.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.964+0000 7fec8bddbb80 1 -- 172.21.15.133:0/1299783268 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f36a18a190 con 0x55f36a272e90 2023-11-27T15:25:39.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.965+0000 7fec59ffb700 1 -- 172.21.15.133:0/1299783268 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f36a265d70 con 0x55f36a272e90 2023-11-27T15:25:39.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.965+0000 7fec59ffb700 1 -- 172.21.15.133:0/1299783268 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 24) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fec680a11d0 con 0x55f36a272e90 2023-11-27T15:25:39.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.965+0000 7fec8bddbb80 10 client.26029 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:39.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.965+0000 7fec8bddbb80 10 client.26029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:39.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.965+0000 7fec8bddbb80 20 client.26029 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:39.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.965+0000 7fec8bddbb80 1 --2- 172.21.15.133:0/1299783268 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55f36a2995b0 0x55f36a29b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:39.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.965+0000 7fec8bddbb80 4 client.26029 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:39.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.965+0000 7fec8bddbb80 1 -- 172.21.15.133:0/1299783268 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f36a1860a0 con 0x55f36a2995b0 2023-11-27T15:25:39.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.966+0000 7fec7599a700 1 --2- 172.21.15.133:0/1299783268 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55f36a2995b0 0x55f36a29b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:39.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.969+0000 7fec59ffb700 10 client.26029 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:39.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:39.969+0000 7fec7599a700 1 --2- 172.21.15.133:0/1299783268 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55f36a2995b0 0x55f36a29b9c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a78129d183c56d5 server_cookie=34c6717d4d70c036 in_seq=0 out_seq=0 2023-11-27T15:25:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.050+0000 7f87d77fe700 1 -- 172.21.15.133:0/1152709591 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55af65feff10 con 0x55af662b88d0 2023-11-27T15:25:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.050+0000 7f87d77fe700 10 client.26036 handle_command_reply: tid=0 2023-11-27T15:25:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.050+0000 7f88121deb80 1 client.26036 shutdown 2023-11-27T15:25:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.050+0000 7f88121deb80 1 -- 172.21.15.133:0/1152709591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af66250b10 msgr2=0x55af6625e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.050+0000 7f88121deb80 1 --2- 172.21.15.133:0/1152709591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af66250b10 0x55af6625e790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f87f0007b40 tx=0x7f87f00099e0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.050+0000 7f88121deb80 1 -- 172.21.15.133:0/1152709591 shutdown_connections 2023-11-27T15:25:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.050+0000 7f88121deb80 1 --2- 172.21.15.133:0/1152709591 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55af662b88d0 0x55af662bace0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.050+0000 7f88121deb80 1 --2- 172.21.15.133:0/1152709591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55af66250b10 0x55af6625e790 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.050+0000 7f88121deb80 1 --2- 172.21.15.133:0/1152709591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55af660ab6b0 0x55af6625b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.050+0000 7f88121deb80 1 --2- 172.21.15.133:0/1152709591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55af6624fdb0 0x55af6625b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.050+0000 7f87d77fe700 0 client.26036 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.050+0000 7f88121deb80 1 -- 172.21.15.133:0/1152709591 >> 172.21.15.133:0/1152709591 conn(0x55af66244f50 msgr2=0x55af66256e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.051+0000 7f88121deb80 1 -- 172.21.15.133:0/1152709591 shutdown_connections 2023-11-27T15:25:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.051+0000 7f88121deb80 1 -- 172.21.15.133:0/1152709591 wait complete. 2023-11-27T15:25:41.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.053+0000 7f88121deb80 20 client.26036 trim_cache size 0 max 16384 2023-11-27T15:25:41.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.051+0000 7fec59ffb700 1 -- 172.21.15.133:0/1299783268 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f36a1860a0 con 0x55f36a2995b0 2023-11-27T15:25:41.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.051+0000 7fec59ffb700 10 client.26029 handle_command_reply: tid=0 2023-11-27T15:25:41.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.051+0000 7fec8bddbb80 1 client.26029 shutdown 2023-11-27T15:25:41.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.052+0000 7fec8bddbb80 1 -- 172.21.15.133:0/1299783268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a272e90 msgr2=0x55f36a27c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:41.057 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.052+0000 7fec8bddbb80 1 --2- 172.21.15.133:0/1299783268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a272e90 0x55f36a27c7c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fec680c9840 tx=0x7fec68099180 comp rx=0 tx=0).stop 2023-11-27T15:25:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:25:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:25:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:25:41.058 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:25:41.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.053+0000 7fec8bddbb80 1 -- 172.21.15.133:0/1299783268 shutdown_connections 2023-11-27T15:25:41.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.053+0000 7fec8bddbb80 1 --2- 172.21.15.133:0/1299783268 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55f36a2995b0 0x55f36a29b9c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.053+0000 7fec8bddbb80 1 --2- 172.21.15.133:0/1299783268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a2742b0 0x55f36a283d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.053+0000 7fec8bddbb80 1 --2- 172.21.15.133:0/1299783268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a272e90 0x55f36a27c7c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.054+0000 7fec8bddbb80 1 --2- 172.21.15.133:0/1299783268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2737e0 0x55f36a27cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.054+0000 7fec8bddbb80 1 -- 172.21.15.133:0/1299783268 >> 172.21.15.133:0/1299783268 conn(0x55f36a2685b0 msgr2=0x55f36a26d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:41.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.054+0000 7fec59ffb700 0 client.26029 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:41.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.055+0000 7fec8bddbb80 1 -- 172.21.15.133:0/1299783268 shutdown_connections 2023-11-27T15:25:41.059 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.055+0000 7fec8bddbb80 1 -- 172.21.15.133:0/1299783268 wait complete. 2023-11-27T15:25:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.055+0000 7f87fe7ff700 1 -- 172.21.15.133:0/3733308106 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f87e00a11d0 msgr2=0x7f87e00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.056+0000 7f87fe7ff700 1 --2- 172.21.15.133:0/3733308106 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f87e00a11d0 0x7f87e00a3680 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f87f00106d0 tx=0x7f87f0004b20 comp rx=0 tx=0).stop 2023-11-27T15:25:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.056+0000 7f87fe7ff700 1 -- 172.21.15.133:0/3733308106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87f8100cc0 msgr2=0x7f87f81982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.056+0000 7f87fe7ff700 1 --2- 172.21.15.133:0/3733308106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87f8100cc0 0x7f87f81982b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f87e800b530 tx=0x7f87e800b840 comp rx=0 tx=0).stop 2023-11-27T15:25:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.056+0000 7f87fe7ff700 1 -- 172.21.15.133:0/3733308106 shutdown_connections 2023-11-27T15:25:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.056+0000 7f87fe7ff700 1 --2- 172.21.15.133:0/3733308106 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f87e00a11d0 0x7f87e00a3680 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.056+0000 7f87fe7ff700 1 --2- 172.21.15.133:0/3733308106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f87f8105f00 0x7f87f819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.056+0000 7f87fe7ff700 1 --2- 172.21.15.133:0/3733308106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87f8100cc0 0x7f87f81982b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.056+0000 7f87fe7ff700 1 --2- 172.21.15.133:0/3733308106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87f81035e0 0x7f87f81987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.056+0000 7f87fe7ff700 1 -- 172.21.15.133:0/3733308106 >> 172.21.15.133:0/3733308106 conn(0x7f87f80fa720 msgr2=0x7f87f8101920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.057+0000 7f87fe7ff700 1 -- 172.21.15.133:0/3733308106 shutdown_connections 2023-11-27T15:25:41.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:41.058+0000 7f87fe7ff700 1 -- 172.21.15.133:0/3733308106 wait complete. 2023-11-27T15:25:41.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.056+0000 7fec8bddbb80 20 client.26029 trim_cache size 0 max 16384 2023-11-27T15:25:41.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.092+0000 7fec8bddbb80 1 Processor -- start 2023-11-27T15:25:41.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.092+0000 7fec8bddbb80 1 -- start start 2023-11-27T15:25:41.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.093+0000 7fec8bddbb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a272e90 0x55f36a29b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:41.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.093+0000 7fec8bddbb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a2737e0 0x55f36a283d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:41.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.093+0000 7fec8bddbb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2742b0 0x55f36a27a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:41.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.093+0000 7fec8bddbb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36a27c390 con 0x55f36a272e90 2023-11-27T15:25:41.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.093+0000 7fec8bddbb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36a26f080 con 0x55f36a2742b0 2023-11-27T15:25:41.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.093+0000 7fec8bddbb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36a27a3f0 con 0x55f36a2737e0 2023-11-27T15:25:41.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.093+0000 7fec7619b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a272e90 0x55f36a29b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:41.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.093+0000 7fec7699c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2742b0 0x55f36a27a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:41.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.093+0000 7fec7699c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2742b0 0x55f36a27a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:58126/0 (socket says 172.21.15.133:58126) 2023-11-27T15:25:41.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.093+0000 7fec7619b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a272e90 0x55f36a29b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39592/0 (socket says 172.21.15.133:39592) 2023-11-27T15:25:41.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.093+0000 7fec7699c700 1 -- 172.21.15.133:0/3150833739 learned_addr learned my addr 172.21.15.133:0/3150833739 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:41.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.093+0000 7fec7599a700 1 --2- 172.21.15.133:0/3150833739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a2737e0 0x55f36a283d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:41.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.093+0000 7fec7619b700 1 -- 172.21.15.133:0/3150833739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a2737e0 msgr2=0x55f36a283d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:41.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.093+0000 7fec7619b700 1 --2- 172.21.15.133:0/3150833739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a2737e0 0x55f36a283d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.094+0000 7fec7619b700 1 -- 172.21.15.133:0/3150833739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2742b0 msgr2=0x55f36a27a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:41.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.094+0000 7fec7619b700 1 --2- 172.21.15.133:0/3150833739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2742b0 0x55f36a27a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.094+0000 7fec7619b700 1 -- 172.21.15.133:0/3150833739 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f36a189ee0 con 0x55f36a272e90 2023-11-27T15:25:41.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.094+0000 7fec7599a700 1 --2- 172.21.15.133:0/3150833739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a2737e0 0x55f36a283d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:41.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.094+0000 7fec7619b700 1 --2- 172.21.15.133:0/3150833739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a272e90 0x55f36a29b5c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fec68095b70 tx=0x7fec680a16d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa8de683ea0311e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:41.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.094+0000 7fec653ff700 1 -- 172.21.15.133:0/3150833739 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68098ed0 con 0x55f36a272e90 2023-11-27T15:25:41.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.094+0000 7fec653ff700 1 -- 172.21.15.133:0/3150833739 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec680983f0 con 0x55f36a272e90 2023-11-27T15:25:41.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.094+0000 7fec653ff700 1 -- 172.21.15.133:0/3150833739 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec680985f0 con 0x55f36a272e90 2023-11-27T15:25:41.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.095+0000 7fec8bddbb80 1 -- 172.21.15.133:0/3150833739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a272e90 msgr2=0x55f36a29b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:41.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.095+0000 7fec8bddbb80 1 --2- 172.21.15.133:0/3150833739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a272e90 0x55f36a29b5c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fec68095b70 tx=0x7fec680a16d0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.095+0000 7fec8bddbb80 1 -- 172.21.15.133:0/3150833739 shutdown_connections 2023-11-27T15:25:41.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.095+0000 7fec8bddbb80 1 --2- 172.21.15.133:0/3150833739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2742b0 0x55f36a27a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.095+0000 7fec8bddbb80 1 --2- 172.21.15.133:0/3150833739 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a272e90 0x55f36a29b5c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.095+0000 7fec8bddbb80 1 --2- 172.21.15.133:0/3150833739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a2737e0 0x55f36a283d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.095+0000 7fec8bddbb80 1 -- 172.21.15.133:0/3150833739 >> 172.21.15.133:0/3150833739 conn(0x55f36a2685b0 msgr2=0x55f36a2a0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:41.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.095+0000 7fec8bddbb80 1 -- 172.21.15.133:0/3150833739 shutdown_connections 2023-11-27T15:25:41.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.095+0000 7fec8bddbb80 1 -- 172.21.15.133:0/3150833739 wait complete. 2023-11-27T15:25:41.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.096+0000 7fec8bddbb80 1 -- start start 2023-11-27T15:25:41.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.096+0000 7fec8bddbb80 1 Processor -- start 2023-11-27T15:25:41.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.097+0000 7fec8bddbb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a272e90 0x55f36a27f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:41.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.097+0000 7fec8bddbb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a2737e0 0x55f36a27fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:41.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.097+0000 7fec8bddbb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2742b0 0x55f36a282d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:41.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.097+0000 7fec8bddbb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36a283290 con 0x55f36a272e90 2023-11-27T15:25:41.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.097+0000 7fec8bddbb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36a283400 con 0x55f36a2742b0 2023-11-27T15:25:41.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.097+0000 7fec8bddbb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f36a2835c0 con 0x55f36a2737e0 2023-11-27T15:25:41.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.097+0000 7fec8bddbb80 10 client.0 _ll_register_callbacks cb 0x55f36a183780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:41.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.097+0000 7fec7599a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a2737e0 0x55f36a27fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:41.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.097+0000 7fec7699c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2742b0 0x55f36a282d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:41.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.097+0000 7fec7619b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a272e90 0x55f36a27f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:41.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.097+0000 7fec7699c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2742b0 0x55f36a282d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:58132/0 (socket says 172.21.15.133:58132) 2023-11-27T15:25:41.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.097+0000 7fec7699c700 1 -- 172.21.15.133:0/2464903282 learned_addr learned my addr 172.21.15.133:0/2464903282 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:41.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.097+0000 7fec7599a700 1 -- 172.21.15.133:0/2464903282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2742b0 msgr2=0x55f36a282d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:41.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.097+0000 7fec7599a700 1 --2- 172.21.15.133:0/2464903282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2742b0 0x55f36a282d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.097+0000 7fec7599a700 1 -- 172.21.15.133:0/2464903282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a272e90 msgr2=0x55f36a27f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:41.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.097+0000 7fec7599a700 1 --2- 172.21.15.133:0/2464903282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a272e90 0x55f36a27f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.097+0000 7fec7599a700 1 -- 172.21.15.133:0/2464903282 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f36a18a190 con 0x55f36a2737e0 2023-11-27T15:25:41.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.097+0000 7fec7619b700 1 --2- 172.21.15.133:0/2464903282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a272e90 0x55f36a27f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:41.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.098+0000 7fec7599a700 1 --2- 172.21.15.133:0/2464903282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a2737e0 0x55f36a27fd30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009ce0 tx=0x7fec6c00bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab28c65bd74d275a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:41.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.098+0000 7fec59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:25:41.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.098+0000 7fec59ffb700 1 -- 172.21.15.133:0/2464903282 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c032070 con 0x55f36a2737e0 2023-11-27T15:25:41.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.098+0000 7fec8bddbb80 1 -- 172.21.15.133:0/2464903282 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec6c01e040 con 0x55f36a2737e0 2023-11-27T15:25:41.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.098+0000 7fec59ffb700 1 -- 172.21.15.133:0/2464903282 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fec6c0055d0 con 0x55f36a2737e0 2023-11-27T15:25:41.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.098+0000 7fec59ffb700 1 -- 172.21.15.133:0/2464903282 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec6c027040 con 0x55f36a2737e0 2023-11-27T15:25:41.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.098+0000 7fec8bddbb80 10 client.26035 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:41.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.098+0000 7fec8bddbb80 1 -- 172.21.15.133:0/2464903282 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f369fdac60 con 0x55f36a2737e0 2023-11-27T15:25:41.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.099+0000 7fec59ffb700 1 -- 172.21.15.133:0/2464903282 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec6c01e040 con 0x55f36a2737e0 2023-11-27T15:25:41.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.099+0000 7fec59ffb700 1 -- 172.21.15.133:0/2464903282 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 24) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fec6c023830 con 0x55f36a2737e0 2023-11-27T15:25:41.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.099+0000 7fec8bddbb80 10 client.26035 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:41.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.099+0000 7fec8bddbb80 10 client.26035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:41.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.099+0000 7fec8bddbb80 20 client.26035 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:41.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.099+0000 7fec8bddbb80 1 --2- 172.21.15.133:0/2464903282 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55f36a2edc60 0x55f36a2f0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:41.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.099+0000 7fec8bddbb80 4 client.26035 mds_command: new command op to 24560 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:25:41.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.099+0000 7fec8bddbb80 1 -- 172.21.15.133:0/2464903282 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f36a192660 con 0x55f36a2edc60 2023-11-27T15:25:41.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.099+0000 7fec7619b700 1 --2- 172.21.15.133:0/2464903282 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55f36a2edc60 0x55f36a2f0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:41.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.100+0000 7fec59ffb700 10 client.26035 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:41.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.100+0000 7fec7619b700 1 --2- 172.21.15.133:0/2464903282 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55f36a2edc60 0x55f36a2f0070 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd20657cff4fc071 server_cookie=6969fb3db434cca6 in_seq=0 out_seq=0 2023-11-27T15:25:41.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.100+0000 7fec59ffb700 1 -- 172.21.15.133:0/2464903282 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f36a192660 con 0x55f36a2edc60 2023-11-27T15:25:41.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.100+0000 7fec59ffb700 10 client.26035 handle_command_reply: tid=0 2023-11-27T15:25:41.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.100+0000 7fec8bddbb80 1 client.26035 shutdown 2023-11-27T15:25:41.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.100+0000 7fec8bddbb80 1 -- 172.21.15.133:0/2464903282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a2737e0 msgr2=0x55f36a27fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:41.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.100+0000 7fec8bddbb80 1 --2- 172.21.15.133:0/2464903282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a2737e0 0x55f36a27fd30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009ce0 tx=0x7fec6c00bf20 comp rx=0 tx=0).stop 2023-11-27T15:25:41.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.100+0000 7fec8bddbb80 1 -- 172.21.15.133:0/2464903282 shutdown_connections 2023-11-27T15:25:41.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.100+0000 7fec8bddbb80 1 --2- 172.21.15.133:0/2464903282 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55f36a2edc60 0x55f36a2f0070 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.100+0000 7fec8bddbb80 1 --2- 172.21.15.133:0/2464903282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f36a2742b0 0x55f36a282d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.100+0000 7fec8bddbb80 1 --2- 172.21.15.133:0/2464903282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f36a272e90 0x55f36a27f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.100+0000 7fec8bddbb80 1 --2- 172.21.15.133:0/2464903282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f36a2737e0 0x55f36a27fd30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.100+0000 7fec8bddbb80 1 -- 172.21.15.133:0/2464903282 >> 172.21.15.133:0/2464903282 conn(0x55f36a2685b0 msgr2=0x55f36a27b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:41.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.100+0000 7fec59ffb700 0 client.26035 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:41.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.100+0000 7fec8bddbb80 1 -- 172.21.15.133:0/2464903282 shutdown_connections 2023-11-27T15:25:41.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.100+0000 7fec8bddbb80 1 -- 172.21.15.133:0/2464903282 wait complete. 2023-11-27T15:25:41.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.101+0000 7fec8bddbb80 20 client.26035 trim_cache size 0 max 16384 2023-11-27T15:25:41.108 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:25:41.108 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2023-11-27T15:25:41.108 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2023-11-27T15:25:41.108 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:25:41.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.101+0000 7fec7719d700 1 -- 172.21.15.133:0/1911401645 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fec5c0a12a0 msgr2=0x7fec5c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:41.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.101+0000 7fec7719d700 1 --2- 172.21.15.133:0/1911401645 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fec5c0a12a0 0x7fec5c0a3750 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fec60006fd0 tx=0x7fec60006ea0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.101+0000 7fec7719d700 1 -- 172.21.15.133:0/1911401645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec70106660 msgr2=0x7fec7019c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:41.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.101+0000 7fec7719d700 1 --2- 172.21.15.133:0/1911401645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec70106660 0x7fec7019c410 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fec6800b9c0 tx=0x7fec6800bcd0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.102+0000 7fec7719d700 1 -- 172.21.15.133:0/1911401645 shutdown_connections 2023-11-27T15:25:41.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.102+0000 7fec7719d700 1 --2- 172.21.15.133:0/1911401645 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fec5c0a12a0 0x7fec5c0a3750 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.102+0000 7fec7719d700 1 --2- 172.21.15.133:0/1911401645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec701081e0 0x7fec701a39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.102+0000 7fec7719d700 1 --2- 172.21.15.133:0/1911401645 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec70106660 0x7fec7019c410 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.102+0000 7fec7719d700 1 --2- 172.21.15.133:0/1911401645 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec70107850 0x7fec7019c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.102+0000 7fec7719d700 1 -- 172.21.15.133:0/1911401645 >> 172.21.15.133:0/1911401645 conn(0x7fec70075690 msgr2=0x7fec700778a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:41.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.102+0000 7fec7719d700 1 -- 172.21.15.133:0/1911401645 shutdown_connections 2023-11-27T15:25:41.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.103+0000 7fec7719d700 1 -- 172.21.15.133:0/1911401645 wait complete. 2023-11-27T15:25:41.126 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:40 smithi133 ceph-mon[97860]: pgmap v431: 129 pgs: 129 active+clean; 34 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 453 op/s 2023-11-27T15:25:41.129 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:25:41.129 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:25:41.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:40 smithi174 ceph-mon[131110]: pgmap v431: 129 pgs: 129 active+clean; 34 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 453 op/s 2023-11-27T15:25:41.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:40 smithi177 ceph-mon[131504]: pgmap v431: 129 pgs: 129 active+clean; 34 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 453 op/s 2023-11-27T15:25:41.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.405+0000 7eff721ff700 1 -- 172.21.15.133:0/676217107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff6c0feb80 msgr2=0x7eff6c100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:41.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.405+0000 7eff721ff700 1 --2- 172.21.15.133:0/676217107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff6c0feb80 0x7eff6c100fa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7eff54009990 tx=0x7eff54009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.405+0000 7eff721ff700 1 -- 172.21.15.133:0/676217107 shutdown_connections 2023-11-27T15:25:41.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.405+0000 7eff721ff700 1 --2- 172.21.15.133:0/676217107 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff6c103ea0 0x7eff6c104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.405+0000 7eff721ff700 1 --2- 172.21.15.133:0/676217107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff6c0feb80 0x7eff6c100fa0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.405+0000 7eff721ff700 1 --2- 172.21.15.133:0/676217107 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff6c1014e0 0x7eff6c103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.405+0000 7eff721ff700 1 -- 172.21.15.133:0/676217107 >> 172.21.15.133:0/676217107 conn(0x7eff6c0fa750 msgr2=0x7eff6c0fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:41.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.405+0000 7eff721ff700 1 -- 172.21.15.133:0/676217107 shutdown_connections 2023-11-27T15:25:41.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.406+0000 7eff721ff700 1 -- 172.21.15.133:0/676217107 wait complete. 2023-11-27T15:25:41.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.407+0000 7eff721ff700 1 Processor -- start 2023-11-27T15:25:41.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.407+0000 7eff721ff700 1 -- start start 2023-11-27T15:25:41.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.408+0000 7eff721ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff6c0feb80 0x7eff6c19c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:41.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.408+0000 7eff721ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff6c1014e0 0x7eff6c19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:41.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.408+0000 7eff721ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff6c103ea0 0x7eff6c1a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:41.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.408+0000 7eff721ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c1a4150 con 0x7eff6c103ea0 2023-11-27T15:25:41.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.408+0000 7eff721ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c1a4290 con 0x7eff6c0feb80 2023-11-27T15:25:41.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.408+0000 7eff721ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c1a4420 con 0x7eff6c1014e0 2023-11-27T15:25:41.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.408+0000 7eff6b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff6c0feb80 0x7eff6c19c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:41.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.408+0000 7eff6b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff6c0feb80 0x7eff6c19c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:58142/0 (socket says 172.21.15.133:58142) 2023-11-27T15:25:41.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.408+0000 7eff6b7fe700 1 -- 172.21.15.133:0/2138315354 learned_addr learned my addr 172.21.15.133:0/2138315354 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:41.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.408+0000 7eff6bfff700 1 --2- 172.21.15.133:0/2138315354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff6c103ea0 0x7eff6c1a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:41.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.409+0000 7eff6bfff700 1 -- 172.21.15.133:0/2138315354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff6c1014e0 msgr2=0x7eff6c19cb10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:25:41.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.409+0000 7eff6bfff700 1 --2- 172.21.15.133:0/2138315354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff6c1014e0 0x7eff6c19cb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.409+0000 7eff6bfff700 1 -- 172.21.15.133:0/2138315354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff6c0feb80 msgr2=0x7eff6c19c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:41.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.409+0000 7eff6bfff700 1 --2- 172.21.15.133:0/2138315354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff6c0feb80 0x7eff6c19c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.409+0000 7eff6bfff700 1 -- 172.21.15.133:0/2138315354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff540095f0 con 0x7eff6c103ea0 2023-11-27T15:25:41.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.410+0000 7eff6bfff700 1 --2- 172.21.15.133:0/2138315354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff6c103ea0 0x7eff6c1a3b80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7eff6400efc0 tx=0x7eff6400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:41.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.410+0000 7eff68ff9700 1 -- 172.21.15.133:0/2138315354 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff64018070 con 0x7eff6c103ea0 2023-11-27T15:25:41.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.410+0000 7eff68ff9700 1 -- 172.21.15.133:0/2138315354 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff6400f040 con 0x7eff6c103ea0 2023-11-27T15:25:41.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.410+0000 7eff68ff9700 1 -- 172.21.15.133:0/2138315354 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff64004020 con 0x7eff6c103ea0 2023-11-27T15:25:41.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.410+0000 7eff721ff700 1 -- 172.21.15.133:0/2138315354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff6c112230 con 0x7eff6c103ea0 2023-11-27T15:25:41.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.410+0000 7eff721ff700 1 -- 172.21.15.133:0/2138315354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff6c1120a0 con 0x7eff6c103ea0 2023-11-27T15:25:41.415 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.412+0000 7eff721ff700 1 -- 172.21.15.133:0/2138315354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6c1a4a60 con 0x7eff6c103ea0 2023-11-27T15:25:41.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.414+0000 7eff68ff9700 1 -- 172.21.15.133:0/2138315354 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7eff64004ad0 con 0x7eff6c103ea0 2023-11-27T15:25:41.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.417+0000 7eff68ff9700 1 --2- 172.21.15.133:0/2138315354 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7eff580a11d0 0x7eff580a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:41.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.417+0000 7eff68ff9700 1 -- 172.21.15.133:0/2138315354 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7eff640cc800 con 0x7eff6c103ea0 2023-11-27T15:25:41.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.417+0000 7eff6b7fe700 1 --2- 172.21.15.133:0/2138315354 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7eff580a11d0 0x7eff580a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:41.422 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.418+0000 7eff6b7fe700 1 --2- 172.21.15.133:0/2138315354 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7eff580a11d0 0x7eff580a3680 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7eff5400b2e0 tx=0x7eff54005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:41.426 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.423+0000 7eff68ff9700 1 -- 172.21.15.133:0/2138315354 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7eff64095c30 con 0x7eff6c103ea0 2023-11-27T15:25:41.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.790+0000 7eff721ff700 1 -- 172.21.15.133:0/2138315354 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff6c1a5100 con 0x7eff6c103ea0 2023-11-27T15:25:41.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.791+0000 7eff68ff9700 1 -- 172.21.15.133:0/2138315354 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10832 (secure 0 0 0) 0x7eff64099250 con 0x7eff6c103ea0 2023-11-27T15:25:41.795 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:25:41.796 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14493,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2381290259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2381290259},{"type":"v1","addr":"172.21.15.133:6839","nonce":2381290259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1704875895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1704875895},{"type":"v1","addr":"172.21.15.133:6841","nonce":1704875895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:20:33.201023+0000","modified":"2023-11-27T15:25:16.846884+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24521,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3492215689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3492215689},{"type":"v1","addr":"172.21.15.177:6837","nonce":3492215689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6839/2781994447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2781994447},{"type":"v1","addr":"172.21.15.174:6839","nonce":2781994447}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:25:41.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.795+0000 7eff721ff700 1 -- 172.21.15.133:0/2138315354 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7eff580a11d0 msgr2=0x7eff580a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:41.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.795+0000 7eff721ff700 1 --2- 172.21.15.133:0/2138315354 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7eff580a11d0 0x7eff580a3680 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7eff5400b2e0 tx=0x7eff54005fb0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.795+0000 7eff721ff700 1 -- 172.21.15.133:0/2138315354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff6c103ea0 msgr2=0x7eff6c1a3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:41.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.795+0000 7eff721ff700 1 --2- 172.21.15.133:0/2138315354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff6c103ea0 0x7eff6c1a3b80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7eff6400efc0 tx=0x7eff6400c3a0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.795+0000 7eff721ff700 1 -- 172.21.15.133:0/2138315354 shutdown_connections 2023-11-27T15:25:41.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.795+0000 7eff721ff700 1 --2- 172.21.15.133:0/2138315354 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7eff580a11d0 0x7eff580a3680 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.795+0000 7eff721ff700 1 --2- 172.21.15.133:0/2138315354 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7eff6c103ea0 0x7eff6c1a3b80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.795+0000 7eff721ff700 1 --2- 172.21.15.133:0/2138315354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff6c0feb80 0x7eff6c19c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.795+0000 7eff721ff700 1 --2- 172.21.15.133:0/2138315354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff6c1014e0 0x7eff6c19cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:41.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.795+0000 7eff721ff700 1 -- 172.21.15.133:0/2138315354 >> 172.21.15.133:0/2138315354 conn(0x7eff6c0fa750 msgr2=0x7eff6c1029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:41.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.795+0000 7eff721ff700 1 -- 172.21.15.133:0/2138315354 shutdown_connections 2023-11-27T15:25:41.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:41.795+0000 7eff721ff700 1 -- 172.21.15.133:0/2138315354 wait complete. 2023-11-27T15:25:41.800 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 24 2023-11-27T15:25:41.825 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:25:42.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.070+0000 7f22037ff700 1 -- 172.21.15.133:0/957146565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21fc074ab0 msgr2=0x7f21fc072f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.070+0000 7f22037ff700 1 --2- 172.21.15.133:0/957146565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21fc074ab0 0x7f21fc072f00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009990 tx=0x7f21ec009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.070+0000 7f22037ff700 1 -- 172.21.15.133:0/957146565 shutdown_connections 2023-11-27T15:25:42.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.070+0000 7f22037ff700 1 --2- 172.21.15.133:0/957146565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21fc108180 0x7f21fc10a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.070+0000 7f22037ff700 1 --2- 172.21.15.133:0/957146565 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21fc074ab0 0x7f21fc072f00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.070+0000 7f22037ff700 1 --2- 172.21.15.133:0/957146565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21fc0734d0 0x7f21fc073940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.070+0000 7f22037ff700 1 -- 172.21.15.133:0/957146565 >> 172.21.15.133:0/957146565 conn(0x7f21fc0fb8b0 msgr2=0x7f21fc0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:42.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.070+0000 7f22037ff700 1 -- 172.21.15.133:0/957146565 shutdown_connections 2023-11-27T15:25:42.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.070+0000 7f22037ff700 1 -- 172.21.15.133:0/957146565 wait complete. 2023-11-27T15:25:42.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.071+0000 7f22037ff700 1 Processor -- start 2023-11-27T15:25:42.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.071+0000 7f22037ff700 1 -- start start 2023-11-27T15:25:42.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.072+0000 7f22037ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21fc0734d0 0x7f21fc198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.072+0000 7f22037ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21fc074ab0 0x7f21fc198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.072+0000 7f22037ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21fc108180 0x7f21fc19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.072+0000 7f22037ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc19fda0 con 0x7f21fc108180 2023-11-27T15:25:42.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.072+0000 7f22037ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc19fee0 con 0x7f21fc0734d0 2023-11-27T15:25:42.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.072+0000 7f22037ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21fc1a0070 con 0x7f21fc074ab0 2023-11-27T15:25:42.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.072+0000 7f220159b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21fc0734d0 0x7f21fc198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.072+0000 7f2200d9a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21fc074ab0 0x7f21fc198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.072+0000 7f220159b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21fc0734d0 0x7f21fc198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:58162/0 (socket says 172.21.15.133:58162) 2023-11-27T15:25:42.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.072+0000 7f220159b700 1 -- 172.21.15.133:0/660615734 learned_addr learned my addr 172.21.15.133:0/660615734 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:42.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.072+0000 7f2201d9c700 1 --2- 172.21.15.133:0/660615734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21fc108180 0x7f21fc19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.072+0000 7f2200d9a700 1 -- 172.21.15.133:0/660615734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21fc0734d0 msgr2=0x7f21fc198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.072+0000 7f2200d9a700 1 --2- 172.21.15.133:0/660615734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21fc0734d0 0x7f21fc198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.072+0000 7f2200d9a700 1 -- 172.21.15.133:0/660615734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21fc108180 msgr2=0x7f21fc19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.072+0000 7f2200d9a700 1 --2- 172.21.15.133:0/660615734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21fc108180 0x7f21fc19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.072+0000 7f2200d9a700 1 -- 172.21.15.133:0/660615734 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ec0095f0 con 0x7f21fc074ab0 2023-11-27T15:25:42.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.072+0000 7f2200d9a700 1 --2- 172.21.15.133:0/660615734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21fc074ab0 0x7f21fc198760 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f21f800e9f0 tx=0x7f21f800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:42.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.073+0000 7f21f27fc700 1 -- 172.21.15.133:0/660615734 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f800eee0 con 0x7f21fc074ab0 2023-11-27T15:25:42.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.073+0000 7f21f27fc700 1 -- 172.21.15.133:0/660615734 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21f800cbd0 con 0x7f21fc074ab0 2023-11-27T15:25:42.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.073+0000 7f22037ff700 1 -- 172.21.15.133:0/660615734 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21fc10de60 con 0x7f21fc074ab0 2023-11-27T15:25:42.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.073+0000 7f21f27fc700 1 -- 172.21.15.133:0/660615734 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8018680 con 0x7f21fc074ab0 2023-11-27T15:25:42.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.073+0000 7f22037ff700 1 -- 172.21.15.133:0/660615734 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21fc10dcd0 con 0x7f21fc074ab0 2023-11-27T15:25:42.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.076+0000 7f21f27fc700 1 -- 172.21.15.133:0/660615734 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f21f800cd40 con 0x7f21fc074ab0 2023-11-27T15:25:42.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.077+0000 7f21f27fc700 1 --2- 172.21.15.133:0/660615734 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f21e80a1150 0x7f21e80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.077+0000 7f21f27fc700 1 -- 172.21.15.133:0/660615734 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f21f8014070 con 0x7f21fc074ab0 2023-11-27T15:25:42.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.077+0000 7f220159b700 1 --2- 172.21.15.133:0/660615734 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f21e80a1150 0x7f21e80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.077+0000 7f2217103b80 1 Processor -- start 2023-11-27T15:25:42.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.078+0000 7f2217103b80 1 -- start start 2023-11-27T15:25:42.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.078+0000 7f2217103b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8adc1cc00 0x55b8adc1d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.078+0000 7f220159b700 1 --2- 172.21.15.133:0/660615734 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f21e80a1150 0x7f21e80a3600 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00b2e0 tx=0x7f21ec005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:42.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.078+0000 7f2217103b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1d550 0x55b8adc1db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.078+0000 7f2217103b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1e110 0x55b8adc1e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.078+0000 7f2217103b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8adb22060 con 0x55b8adc1cc00 2023-11-27T15:25:42.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.078+0000 7f2217103b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8adb748a0 con 0x55b8adc1e110 2023-11-27T15:25:42.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.078+0000 7f2217103b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8adb6dd30 con 0x55b8adc1d550 2023-11-27T15:25:42.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.078+0000 7f2201d9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1e110 0x55b8adc1e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.078+0000 7f2200d9a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1d550 0x55b8adc1db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.078+0000 7f2201d9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1e110 0x55b8adc1e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:58164/0 (socket says 172.21.15.133:58164) 2023-11-27T15:25:42.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.078+0000 7f2201d9c700 1 -- 172.21.15.133:0/2153774539 learned_addr learned my addr 172.21.15.133:0/2153774539 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:42.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.078+0000 7f220159b700 1 --2- 172.21.15.133:0/2153774539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8adc1cc00 0x55b8adc1d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.079+0000 7f2201d9c700 1 -- 172.21.15.133:0/2153774539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1d550 msgr2=0x55b8adc1db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.079+0000 7f2201d9c700 1 --2- 172.21.15.133:0/2153774539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1d550 0x55b8adc1db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.079+0000 7f2201d9c700 1 -- 172.21.15.133:0/2153774539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8adc1cc00 msgr2=0x55b8adc1d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.079+0000 7f2201d9c700 1 --2- 172.21.15.133:0/2153774539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8adc1cc00 0x55b8adc1d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.079+0000 7f2201d9c700 1 -- 172.21.15.133:0/2153774539 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8ad9ef630 con 0x55b8adc1e110 2023-11-27T15:25:42.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.079+0000 7f2201d9c700 1 --2- 172.21.15.133:0/2153774539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1e110 0x55b8adc1e580 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009440 tx=0x7f21f4009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9fc05b899cad2fa server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:42.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.079+0000 7f21e77fe700 1 -- 172.21.15.133:0/2153774539 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f400e070 con 0x55b8adc1e110 2023-11-27T15:25:42.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.079+0000 7f21e77fe700 1 -- 172.21.15.133:0/2153774539 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21f400f040 con 0x55b8adc1e110 2023-11-27T15:25:42.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.079+0000 7f21e77fe700 1 -- 172.21.15.133:0/2153774539 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f4015ac0 con 0x55b8adc1e110 2023-11-27T15:25:42.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.080+0000 7f2217103b80 1 -- 172.21.15.133:0/2153774539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1e110 msgr2=0x55b8adc1e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.080+0000 7f2217103b80 1 --2- 172.21.15.133:0/2153774539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1e110 0x55b8adc1e580 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009440 tx=0x7f21f4009920 comp rx=0 tx=0).stop 2023-11-27T15:25:42.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.080+0000 7f2217103b80 1 -- 172.21.15.133:0/2153774539 shutdown_connections 2023-11-27T15:25:42.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.080+0000 7f2217103b80 1 --2- 172.21.15.133:0/2153774539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1e110 0x55b8adc1e580 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.080+0000 7f2217103b80 1 --2- 172.21.15.133:0/2153774539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8adc1cc00 0x55b8adc1d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.080+0000 7f2217103b80 1 --2- 172.21.15.133:0/2153774539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1d550 0x55b8adc1db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.080+0000 7f2217103b80 1 -- 172.21.15.133:0/2153774539 >> 172.21.15.133:0/2153774539 conn(0x55b8adc12360 msgr2=0x55b8adc17670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:42.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.080+0000 7f2217103b80 1 -- 172.21.15.133:0/2153774539 shutdown_connections 2023-11-27T15:25:42.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.080+0000 7f2217103b80 1 -- 172.21.15.133:0/2153774539 wait complete. 2023-11-27T15:25:42.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.081+0000 7f2217103b80 1 -- start start 2023-11-27T15:25:42.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.081+0000 7f2217103b80 1 Processor -- start 2023-11-27T15:25:42.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.081+0000 7f2217103b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1cc00 0x55b8adc26560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.081+0000 7f2217103b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8adc1d550 0x55b8adc26aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.081+0000 7f2217103b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1e110 0x55b8adc2dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.081+0000 7f2217103b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8adb0f8e0 con 0x55b8adc1d550 2023-11-27T15:25:42.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.081+0000 7f2217103b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8adb0f580 con 0x55b8adc1cc00 2023-11-27T15:25:42.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.081+0000 7f2217103b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8ad9f1920 con 0x55b8adc1e110 2023-11-27T15:25:42.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.081+0000 7f2217103b80 10 client.0 _ll_register_callbacks cb 0x55b8adb2d780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:42.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.081+0000 7f220159b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1cc00 0x55b8adc26560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.081+0000 7f220159b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1cc00 0x55b8adc26560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:58180/0 (socket says 172.21.15.133:58180) 2023-11-27T15:25:42.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.081+0000 7f220159b700 1 -- 172.21.15.133:0/4076095012 learned_addr learned my addr 172.21.15.133:0/4076095012 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:42.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.081+0000 7f2200d9a700 1 --2- 172.21.15.133:0/4076095012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8adc1d550 0x55b8adc26aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.082+0000 7f2201d9c700 1 --2- 172.21.15.133:0/4076095012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1e110 0x55b8adc2dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.082+0000 7f220159b700 1 -- 172.21.15.133:0/4076095012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1e110 msgr2=0x55b8adc2dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.082+0000 7f220159b700 1 --2- 172.21.15.133:0/4076095012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1e110 0x55b8adc2dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.082+0000 7f220159b700 1 -- 172.21.15.133:0/4076095012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8adc1d550 msgr2=0x55b8adc26aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.082+0000 7f220159b700 1 --2- 172.21.15.133:0/4076095012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8adc1d550 0x55b8adc26aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.082+0000 7f220159b700 1 -- 172.21.15.133:0/4076095012 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f40090f0 con 0x55b8adc1cc00 2023-11-27T15:25:42.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.082+0000 7f220159b700 1 --2- 172.21.15.133:0/4076095012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1cc00 0x55b8adc26560 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f21ec005800 tx=0x7f21ec01a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2db915d1edab76e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:42.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.082+0000 7f21e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:25:42.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.082+0000 7f21e4ff9700 1 -- 172.21.15.133:0/4076095012 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec02b070 con 0x55b8adc1cc00 2023-11-27T15:25:42.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.082+0000 7f21e4ff9700 1 -- 172.21.15.133:0/4076095012 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f21ec031660 con 0x55b8adc1cc00 2023-11-27T15:25:42.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.082+0000 7f2217103b80 1 -- 172.21.15.133:0/4076095012 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8adc0fb20 con 0x55b8adc1cc00 2023-11-27T15:25:42.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.083+0000 7f21e4ff9700 1 -- 172.21.15.133:0/4076095012 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21ec031820 con 0x55b8adc1cc00 2023-11-27T15:25:42.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.083+0000 7f2217103b80 10 client.26066 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:42.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.083+0000 7f2217103b80 1 -- 172.21.15.133:0/4076095012 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b8adb34190 con 0x55b8adc1cc00 2023-11-27T15:25:42.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.083+0000 7f21e4ff9700 1 -- 172.21.15.133:0/4076095012 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b8adc0fb20 con 0x55b8adc1cc00 2023-11-27T15:25:42.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.083+0000 7f21e4ff9700 1 -- 172.21.15.133:0/4076095012 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 24) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f21ec00f960 con 0x55b8adc1cc00 2023-11-27T15:25:42.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.083+0000 7f2217103b80 10 client.26066 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:42.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.083+0000 7f2217103b80 10 client.26066 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-11-27T15:25:42.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.083+0000 7f2217103b80 20 client.26066 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:42.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.083+0000 7f2217103b80 1 --2- 172.21.15.133:0/4076095012 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55b8adc32390 0x55b8adc347a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.083+0000 7f2217103b80 4 client.26066 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:42.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.083+0000 7f2217103b80 1 -- 172.21.15.133:0/4076095012 --> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8adb300a0 con 0x55b8adc32390 2023-11-27T15:25:42.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.084+0000 7f2200d9a700 1 --2- 172.21.15.133:0/4076095012 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55b8adc32390 0x55b8adc347a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.084+0000 7f2200d9a700 1 --2- 172.21.15.133:0/4076095012 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55b8adc32390 0x55b8adc347a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc6685702befdee server_cookie=bb30136eb5da3849 in_seq=0 out_seq=0 2023-11-27T15:25:42.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.084+0000 7f21e4ff9700 10 client.26066 ms_handle_connect on v2:172.21.15.177:6838/1463687598 2023-11-27T15:25:42.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.085+0000 7f21e4ff9700 1 -- 172.21.15.133:0/4076095012 <== mds.2 v2:172.21.15.177:6838/1463687598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b8adb300a0 con 0x55b8adc32390 2023-11-27T15:25:42.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.085+0000 7f21e4ff9700 10 client.26066 handle_command_reply: tid=0 2023-11-27T15:25:42.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.085+0000 7f2217103b80 1 client.26066 shutdown 2023-11-27T15:25:42.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.085+0000 7f2217103b80 1 -- 172.21.15.133:0/4076095012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1cc00 msgr2=0x55b8adc26560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.085+0000 7f2217103b80 1 --2- 172.21.15.133:0/4076095012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1cc00 0x55b8adc26560 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f21ec005800 tx=0x7f21ec01a050 comp rx=0 tx=0).stop 2023-11-27T15:25:42.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.085+0000 7f2217103b80 1 -- 172.21.15.133:0/4076095012 shutdown_connections 2023-11-27T15:25:42.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.085+0000 7f2217103b80 1 --2- 172.21.15.133:0/4076095012 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55b8adc32390 0x55b8adc347a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.085+0000 7f2217103b80 1 --2- 172.21.15.133:0/4076095012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1e110 0x55b8adc2dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.085+0000 7f2217103b80 1 --2- 172.21.15.133:0/4076095012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1cc00 0x55b8adc26560 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.085+0000 7f21e4ff9700 0 client.26066 ms_handle_reset on v2:172.21.15.177:6838/1463687598 2023-11-27T15:25:42.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.085+0000 7f2217103b80 1 --2- 172.21.15.133:0/4076095012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8adc1d550 0x55b8adc26aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.085+0000 7f2217103b80 1 -- 172.21.15.133:0/4076095012 >> 172.21.15.133:0/4076095012 conn(0x55b8adc12360 msgr2=0x55b8adc16de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:42.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.085+0000 7f2217103b80 1 -- 172.21.15.133:0/4076095012 shutdown_connections 2023-11-27T15:25:42.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.085+0000 7f2217103b80 1 -- 172.21.15.133:0/4076095012 wait complete. 2023-11-27T15:25:42.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.085+0000 7f2217103b80 20 client.26066 trim_cache size 0 max 16384 2023-11-27T15:25:42.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.099+0000 7f2217103b80 1 Processor -- start 2023-11-27T15:25:42.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.099+0000 7f2217103b80 1 -- start start 2023-11-27T15:25:42.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.099+0000 7f2217103b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1cc00 0x55b8adc25c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.099+0000 7f2217103b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8adc1d550 0x55b8adc2daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.099+0000 7f2217103b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1e110 0x55b8adc17640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.099+0000 7f2217103b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8adc25a60 con 0x55b8adc1d550 2023-11-27T15:25:42.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.099+0000 7f2217103b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8adc18570 con 0x55b8adc1e110 2023-11-27T15:25:42.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.099+0000 7f2217103b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8adc17bd0 con 0x55b8adc1cc00 2023-11-27T15:25:42.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.100+0000 7f2200d9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8adc1d550 0x55b8adc2daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.100+0000 7f2201d9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1e110 0x55b8adc17640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.100+0000 7f2201d9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1e110 0x55b8adc17640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:58192/0 (socket says 172.21.15.133:58192) 2023-11-27T15:25:42.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.100+0000 7f2201d9c700 1 -- 172.21.15.133:0/4243063570 learned_addr learned my addr 172.21.15.133:0/4243063570 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:42.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.100+0000 7f220159b700 1 --2- 172.21.15.133:0/4243063570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1cc00 0x55b8adc25c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.100+0000 7f220159b700 1 -- 172.21.15.133:0/4243063570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1e110 msgr2=0x55b8adc17640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.100+0000 7f220159b700 1 --2- 172.21.15.133:0/4243063570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1e110 0x55b8adc17640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.100+0000 7f220159b700 1 -- 172.21.15.133:0/4243063570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8adc1d550 msgr2=0x55b8adc2daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.100+0000 7f220159b700 1 --2- 172.21.15.133:0/4243063570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8adc1d550 0x55b8adc2daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.100+0000 7f220159b700 1 -- 172.21.15.133:0/4243063570 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8adb33ee0 con 0x55b8adc1cc00 2023-11-27T15:25:42.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.100+0000 7f2200d9a700 1 --2- 172.21.15.133:0/4243063570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8adc1d550 0x55b8adc2daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:42.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.100+0000 7f220159b700 1 --2- 172.21.15.133:0/4243063570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1cc00 0x55b8adc25c10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00fcf0 tx=0x7f21ec00ff10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7623aa6e668790de server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:42.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.100+0000 7f21e7fff700 1 -- 172.21.15.133:0/4243063570 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec02b070 con 0x55b8adc1cc00 2023-11-27T15:25:42.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.100+0000 7f21e7fff700 1 -- 172.21.15.133:0/4243063570 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21ec02c040 con 0x55b8adc1cc00 2023-11-27T15:25:42.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.100+0000 7f21e7fff700 1 -- 172.21.15.133:0/4243063570 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec024860 con 0x55b8adc1cc00 2023-11-27T15:25:42.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.101+0000 7f2217103b80 1 -- 172.21.15.133:0/4243063570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1cc00 msgr2=0x55b8adc25c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.101+0000 7f2217103b80 1 --2- 172.21.15.133:0/4243063570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1cc00 0x55b8adc25c10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00fcf0 tx=0x7f21ec00ff10 comp rx=0 tx=0).stop 2023-11-27T15:25:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.101+0000 7f2217103b80 1 -- 172.21.15.133:0/4243063570 shutdown_connections 2023-11-27T15:25:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.101+0000 7f2217103b80 1 --2- 172.21.15.133:0/4243063570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1e110 0x55b8adc17640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.101+0000 7f2217103b80 1 --2- 172.21.15.133:0/4243063570 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1cc00 0x55b8adc25c10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.101+0000 7f2217103b80 1 --2- 172.21.15.133:0/4243063570 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8adc1d550 0x55b8adc2daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.101+0000 7f2217103b80 1 -- 172.21.15.133:0/4243063570 >> 172.21.15.133:0/4243063570 conn(0x55b8adc12360 msgr2=0x55b8adc13830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.101+0000 7f2217103b80 1 -- 172.21.15.133:0/4243063570 shutdown_connections 2023-11-27T15:25:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.101+0000 7f2217103b80 1 -- 172.21.15.133:0/4243063570 wait complete. 2023-11-27T15:25:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.102+0000 7f2217103b80 1 -- start start 2023-11-27T15:25:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.102+0000 7f2217103b80 1 Processor -- start 2023-11-27T15:25:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.102+0000 7f2217103b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1cc00 0x55b8adc29280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.102+0000 7f2217103b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8adc1d550 0x55b8adc297c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.102+0000 7f2217103b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1e110 0x55b8adc2c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.102+0000 7f2217103b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8adc2cd20 con 0x55b8adc1d550 2023-11-27T15:25:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.102+0000 7f2217103b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8adc2ce90 con 0x55b8adc1cc00 2023-11-27T15:25:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.102+0000 7f2217103b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8adc2d050 con 0x55b8adc1e110 2023-11-27T15:25:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.102+0000 7f2217103b80 10 client.0 _ll_register_callbacks cb 0x55b8adb2d780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.102+0000 7f220159b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1cc00 0x55b8adc29280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.102+0000 7f2200d9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8adc1d550 0x55b8adc297c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.102+0000 7f2201d9c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1e110 0x55b8adc2c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.102+0000 7f220159b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1cc00 0x55b8adc29280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:58194/0 (socket says 172.21.15.133:58194) 2023-11-27T15:25:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.102+0000 7f220159b700 1 -- 172.21.15.133:0/1319887843 learned_addr learned my addr 172.21.15.133:0/1319887843 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.103+0000 7f220159b700 1 -- 172.21.15.133:0/1319887843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1e110 msgr2=0x55b8adc2c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.103+0000 7f220159b700 1 --2- 172.21.15.133:0/1319887843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1e110 0x55b8adc2c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.103+0000 7f220159b700 1 -- 172.21.15.133:0/1319887843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8adc1d550 msgr2=0x55b8adc297c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.103+0000 7f220159b700 1 --2- 172.21.15.133:0/1319887843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8adc1d550 0x55b8adc297c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.103+0000 7f220159b700 1 -- 172.21.15.133:0/1319887843 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21fc10dcd0 con 0x55b8adc1cc00 2023-11-27T15:25:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.103+0000 7f2201d9c700 1 --2- 172.21.15.133:0/1319887843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1e110 0x55b8adc2c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.103+0000 7f220159b700 1 --2- 172.21.15.133:0/1319887843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1cc00 0x55b8adc29280 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f21ec005850 tx=0x7f21ec005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f8583256eb68883 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.103+0000 7f21e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:25:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.103+0000 7f21e4ff9700 1 -- 172.21.15.133:0/1319887843 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec02b070 con 0x55b8adc1cc00 2023-11-27T15:25:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.103+0000 7f2217103b80 1 -- 172.21.15.133:0/1319887843 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f21f400f040 con 0x55b8adc1cc00 2023-11-27T15:25:42.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.103+0000 7f21e4ff9700 1 -- 172.21.15.133:0/1319887843 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f21ec00f870 con 0x55b8adc1cc00 2023-11-27T15:25:42.112 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.104+0000 7f21e4ff9700 1 -- 172.21.15.133:0/1319887843 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21ec04b080 con 0x55b8adc1cc00 2023-11-27T15:25:42.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.104+0000 7f2217103b80 10 client.26072 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:42.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.104+0000 7f2217103b80 1 -- 172.21.15.133:0/1319887843 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b8adb34190 con 0x55b8adc1cc00 2023-11-27T15:25:42.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.104+0000 7f21e4ff9700 1 -- 172.21.15.133:0/1319887843 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21f400f040 con 0x55b8adc1cc00 2023-11-27T15:25:42.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.104+0000 7f21e4ff9700 1 -- 172.21.15.133:0/1319887843 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 24) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f21ec004500 con 0x55b8adc1cc00 2023-11-27T15:25:42.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.104+0000 7f2217103b80 10 client.26072 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:42.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.104+0000 7f2217103b80 10 client.26072 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-11-27T15:25:42.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.104+0000 7f2217103b80 20 client.26072 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:42.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.104+0000 7f2217103b80 1 --2- 172.21.15.133:0/1319887843 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55b8adc86770 0x55b8adc88b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.104+0000 7f2217103b80 4 client.26072 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:25:42.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.104+0000 7f2217103b80 1 -- 172.21.15.133:0/1319887843 --> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b8adb3c660 con 0x55b8adc86770 2023-11-27T15:25:42.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.104+0000 7f2200d9a700 1 --2- 172.21.15.133:0/1319887843 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55b8adc86770 0x55b8adc88b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.105+0000 7f2200d9a700 1 --2- 172.21.15.133:0/1319887843 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55b8adc86770 0x55b8adc88b80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c17f593b197c5306 server_cookie=cd776a8d800e98a6 in_seq=0 out_seq=0 2023-11-27T15:25:42.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.105+0000 7f21e4ff9700 10 client.26072 ms_handle_connect on v2:172.21.15.177:6838/1463687598 2023-11-27T15:25:42.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.105+0000 7f21e4ff9700 1 -- 172.21.15.133:0/1319887843 <== mds.2 v2:172.21.15.177:6838/1463687598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b8adb3c660 con 0x55b8adc86770 2023-11-27T15:25:42.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.105+0000 7f21e4ff9700 10 client.26072 handle_command_reply: tid=0 2023-11-27T15:25:42.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.105+0000 7f2217103b80 1 client.26072 shutdown 2023-11-27T15:25:42.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.105+0000 7f2217103b80 1 -- 172.21.15.133:0/1319887843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1cc00 msgr2=0x55b8adc29280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.105+0000 7f2217103b80 1 --2- 172.21.15.133:0/1319887843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1cc00 0x55b8adc29280 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f21ec005850 tx=0x7f21ec005d50 comp rx=0 tx=0).stop 2023-11-27T15:25:42.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.105+0000 7f2217103b80 1 -- 172.21.15.133:0/1319887843 shutdown_connections 2023-11-27T15:25:42.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.105+0000 7f2217103b80 1 --2- 172.21.15.133:0/1319887843 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55b8adc86770 0x55b8adc88b80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.105+0000 7f2217103b80 1 --2- 172.21.15.133:0/1319887843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b8adc1e110 0x55b8adc2c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.105+0000 7f21e4ff9700 0 client.26072 ms_handle_reset on v2:172.21.15.177:6838/1463687598 2023-11-27T15:25:42.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.105+0000 7f2217103b80 1 --2- 172.21.15.133:0/1319887843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b8adc1cc00 0x55b8adc29280 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.105+0000 7f2217103b80 1 --2- 172.21.15.133:0/1319887843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b8adc1d550 0x55b8adc297c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.105+0000 7f2217103b80 1 -- 172.21.15.133:0/1319887843 >> 172.21.15.133:0/1319887843 conn(0x55b8adc12360 msgr2=0x55b8adc25a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:42.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.105+0000 7f2217103b80 1 -- 172.21.15.133:0/1319887843 shutdown_connections 2023-11-27T15:25:42.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.105+0000 7f2217103b80 1 -- 172.21.15.133:0/1319887843 wait complete. 2023-11-27T15:25:42.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.106+0000 7f2217103b80 20 client.26072 trim_cache size 0 max 16384 2023-11-27T15:25:42.115 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2023-11-27T15:25:42.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.106+0000 7f22037ff700 1 -- 172.21.15.133:0/660615734 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f21e80a1150 msgr2=0x7f21e80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.106+0000 7f22037ff700 1 --2- 172.21.15.133:0/660615734 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f21e80a1150 0x7f21e80a3600 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00b2e0 tx=0x7f21ec005fb0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.106+0000 7f22037ff700 1 -- 172.21.15.133:0/660615734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21fc074ab0 msgr2=0x7f21fc198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.106+0000 7f22037ff700 1 --2- 172.21.15.133:0/660615734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21fc074ab0 0x7f21fc198760 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f21f800e9f0 tx=0x7f21f800ed00 comp rx=0 tx=0).stop 2023-11-27T15:25:42.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.106+0000 7f22037ff700 1 -- 172.21.15.133:0/660615734 shutdown_connections 2023-11-27T15:25:42.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.106+0000 7f22037ff700 1 --2- 172.21.15.133:0/660615734 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f21e80a1150 0x7f21e80a3600 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.106+0000 7f22037ff700 1 --2- 172.21.15.133:0/660615734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21fc108180 0x7f21fc19f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.106+0000 7f22037ff700 1 --2- 172.21.15.133:0/660615734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21fc0734d0 0x7f21fc198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.106+0000 7f22037ff700 1 --2- 172.21.15.133:0/660615734 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21fc074ab0 0x7f21fc198760 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.106+0000 7f22037ff700 1 -- 172.21.15.133:0/660615734 >> 172.21.15.133:0/660615734 conn(0x7f21fc0fb8b0 msgr2=0x7f21fc0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:42.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.107+0000 7f22037ff700 1 -- 172.21.15.133:0/660615734 shutdown_connections 2023-11-27T15:25:42.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.107+0000 7f22037ff700 1 -- 172.21.15.133:0/660615734 wait complete. 2023-11-27T15:25:42.127 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:41 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2138315354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:42.130 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:25:42.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:41 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2138315354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:42.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:41 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2138315354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:25:42.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.354+0000 7fdb5dfff700 1 -- 172.21.15.133:0/3967833591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb581002d0 msgr2=0x7fdb581006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.358 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.354+0000 7fdb5dfff700 1 --2- 172.21.15.133:0/3967833591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb581002d0 0x7fdb581006e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdb480098f0 tx=0x7fdb48009c00 comp rx=0 tx=0).stop 2023-11-27T15:25:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.354+0000 7fdb5dfff700 1 -- 172.21.15.133:0/3967833591 shutdown_connections 2023-11-27T15:25:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.354+0000 7fdb5dfff700 1 --2- 172.21.15.133:0/3967833591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb58101e50 0x7fdb58108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.354+0000 7fdb5dfff700 1 --2- 172.21.15.133:0/3967833591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb581002d0 0x7fdb581006e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.354+0000 7fdb5dfff700 1 --2- 172.21.15.133:0/3967833591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb581014c0 0x7fdb58101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.354+0000 7fdb5dfff700 1 -- 172.21.15.133:0/3967833591 >> 172.21.15.133:0/3967833591 conn(0x7fdb580fb8d0 msgr2=0x7fdb580fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.354+0000 7fdb5dfff700 1 -- 172.21.15.133:0/3967833591 shutdown_connections 2023-11-27T15:25:42.359 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.354+0000 7fdb5dfff700 1 -- 172.21.15.133:0/3967833591 wait complete. 2023-11-27T15:25:42.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.356+0000 7fdb5dfff700 1 Processor -- start 2023-11-27T15:25:42.360 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.356+0000 7fdb5dfff700 1 -- start start 2023-11-27T15:25:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.356+0000 7fdb5dfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb581002d0 0x7fdb5819c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.361 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.356+0000 7fdb5dfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb581014c0 0x7fdb5819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.356+0000 7fdb5dfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb58101e50 0x7fdb581a3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.356+0000 7fdb5dfff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb581a4200 con 0x7fdb581014c0 2023-11-27T15:25:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.356+0000 7fdb5dfff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb581a4340 con 0x7fdb58101e50 2023-11-27T15:25:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.356+0000 7fdb5dfff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb581a4500 con 0x7fdb581002d0 2023-11-27T15:25:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.357+0000 7fdb56ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb581014c0 0x7fdb5819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.357+0000 7fdb577fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb581002d0 0x7fdb5819c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.357+0000 7fdb56ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb581014c0 0x7fdb5819cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:39712/0 (socket says 172.21.15.133:39712) 2023-11-27T15:25:42.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.357+0000 7fdb56ffd700 1 -- 172.21.15.133:0/2714421804 learned_addr learned my addr 172.21.15.133:0/2714421804 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:42.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.357+0000 7fdb577fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb581002d0 0x7fdb5819c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48158/0 (socket says 172.21.15.133:48158) 2023-11-27T15:25:42.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.357+0000 7fdb57fff700 1 --2- 172.21.15.133:0/2714421804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb58101e50 0x7fdb581a3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.357+0000 7fdb577fe700 1 -- 172.21.15.133:0/2714421804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb58101e50 msgr2=0x7fdb581a3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.357+0000 7fdb577fe700 1 --2- 172.21.15.133:0/2714421804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb58101e50 0x7fdb581a3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.357+0000 7fdb577fe700 1 -- 172.21.15.133:0/2714421804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb581014c0 msgr2=0x7fdb5819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.357+0000 7fdb577fe700 1 --2- 172.21.15.133:0/2714421804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb581014c0 0x7fdb5819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.357+0000 7fdb577fe700 1 -- 172.21.15.133:0/2714421804 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4c0095f0 con 0x7fdb581002d0 2023-11-27T15:25:42.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.357+0000 7fdb56ffd700 1 --2- 172.21.15.133:0/2714421804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb581014c0 0x7fdb5819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:42.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.358+0000 7fdb577fe700 1 --2- 172.21.15.133:0/2714421804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb581002d0 0x7fdb5819c6d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009810 tx=0x7fdb4800b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:42.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.358+0000 7fdb54ff9700 1 -- 172.21.15.133:0/2714421804 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4801c070 con 0x7fdb581002d0 2023-11-27T15:25:42.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.358+0000 7fdb54ff9700 1 -- 172.21.15.133:0/2714421804 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb4800f7d0 con 0x7fdb581002d0 2023-11-27T15:25:42.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.358+0000 7fdb54ff9700 1 -- 172.21.15.133:0/2714421804 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48003c40 con 0x7fdb581002d0 2023-11-27T15:25:42.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.358+0000 7fdb5dfff700 1 -- 172.21.15.133:0/2714421804 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb48009520 con 0x7fdb581002d0 2023-11-27T15:25:42.364 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.358+0000 7fdb5dfff700 1 -- 172.21.15.133:0/2714421804 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb58073f10 con 0x7fdb581002d0 2023-11-27T15:25:42.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.363+0000 7fdb54ff9700 1 -- 172.21.15.133:0/2714421804 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fdb48005040 con 0x7fdb581002d0 2023-11-27T15:25:42.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.364+0000 7fdb71874b80 1 Processor -- start 2023-11-27T15:25:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.364+0000 7fdb71874b80 1 -- start start 2023-11-27T15:25:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.365+0000 7fdb71874b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40cc00 0x56474b40d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.365+0000 7fdb71874b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40d550 0x56474b40db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.365+0000 7fdb71874b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56474b40e110 0x56474b40e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.365+0000 7fdb71874b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474b312060 con 0x56474b40e110 2023-11-27T15:25:42.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.365+0000 7fdb71874b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474b3648a0 con 0x56474b40d550 2023-11-27T15:25:42.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.365+0000 7fdb71874b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474b35dd30 con 0x56474b40cc00 2023-11-27T15:25:42.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.365+0000 7fdb57fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40cc00 0x56474b40d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.365+0000 7fdb57fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40cc00 0x56474b40d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48174/0 (socket says 172.21.15.133:48174) 2023-11-27T15:25:42.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.365+0000 7fdb57fff700 1 -- 172.21.15.133:0/881774094 learned_addr learned my addr 172.21.15.133:0/881774094 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:42.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.366+0000 7fdb54ff9700 1 --2- 172.21.15.133:0/2714421804 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fdb400a11d0 0x7fdb400a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.366+0000 7fdb54ff9700 1 -- 172.21.15.133:0/2714421804 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fdb480cd8b0 con 0x7fdb581002d0 2023-11-27T15:25:42.373 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.366+0000 7fdb56ffd700 1 --2- 172.21.15.133:0/881774094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56474b40e110 0x56474b40e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.366+0000 7fdb577fe700 1 --2- 172.21.15.133:0/881774094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40d550 0x56474b40db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.366+0000 7fdb56ffd700 1 -- 172.21.15.133:0/881774094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40cc00 msgr2=0x56474b40d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.366+0000 7fdb56ffd700 1 --2- 172.21.15.133:0/881774094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40cc00 0x56474b40d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.374 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.366+0000 7fdb56ffd700 1 -- 172.21.15.133:0/881774094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40d550 msgr2=0x56474b40db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.366+0000 7fdb56ffd700 1 --2- 172.21.15.133:0/881774094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40d550 0x56474b40db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.366+0000 7fdb56ffd700 1 -- 172.21.15.133:0/881774094 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56474b1df630 con 0x56474b40e110 2023-11-27T15:25:42.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.367+0000 7fdb57fff700 1 --2- 172.21.15.133:0/2714421804 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fdb400a11d0 0x7fdb400a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.367+0000 7fdb56ffd700 1 --2- 172.21.15.133:0/881774094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56474b40e110 0x56474b40e580 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c011460 tx=0x7fdb4c011770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924c509dd7528948 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:42.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.367+0000 7fdb3dffb700 1 -- 172.21.15.133:0/881774094 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c004830 con 0x56474b40e110 2023-11-27T15:25:42.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.367+0000 7fdb3dffb700 1 -- 172.21.15.133:0/881774094 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb4c004e50 con 0x56474b40e110 2023-11-27T15:25:42.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.367+0000 7fdb3dffb700 1 -- 172.21.15.133:0/881774094 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c0126d0 con 0x56474b40e110 2023-11-27T15:25:42.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.368+0000 7fdb71874b80 1 -- 172.21.15.133:0/881774094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56474b40e110 msgr2=0x56474b40e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.368+0000 7fdb71874b80 1 --2- 172.21.15.133:0/881774094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56474b40e110 0x56474b40e580 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c011460 tx=0x7fdb4c011770 comp rx=0 tx=0).stop 2023-11-27T15:25:42.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.368+0000 7fdb57fff700 1 --2- 172.21.15.133:0/2714421804 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fdb400a11d0 0x7fdb400a3680 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdb500075c0 tx=0x7fdb50009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:42.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.368+0000 7fdb71874b80 1 -- 172.21.15.133:0/881774094 shutdown_connections 2023-11-27T15:25:42.375 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.368+0000 7fdb71874b80 1 --2- 172.21.15.133:0/881774094 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56474b40e110 0x56474b40e580 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.368+0000 7fdb71874b80 1 --2- 172.21.15.133:0/881774094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40cc00 0x56474b40d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.368+0000 7fdb71874b80 1 --2- 172.21.15.133:0/881774094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40d550 0x56474b40db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.368+0000 7fdb71874b80 1 -- 172.21.15.133:0/881774094 >> 172.21.15.133:0/881774094 conn(0x56474b402360 msgr2=0x56474b407670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:42.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.369+0000 7fdb71874b80 1 -- 172.21.15.133:0/881774094 shutdown_connections 2023-11-27T15:25:42.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.369+0000 7fdb71874b80 1 -- 172.21.15.133:0/881774094 wait complete. 2023-11-27T15:25:42.376 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.370+0000 7fdb71874b80 1 -- start start 2023-11-27T15:25:42.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.370+0000 7fdb71874b80 1 Processor -- start 2023-11-27T15:25:42.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.371+0000 7fdb71874b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40cc00 0x56474b416620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.371+0000 7fdb71874b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40d550 0x56474b416b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.371+0000 7fdb71874b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56474b40e110 0x56474b41dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.371+0000 7fdb71874b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474b2ff8e0 con 0x56474b40e110 2023-11-27T15:25:42.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.371+0000 7fdb71874b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474b2ff580 con 0x56474b40d550 2023-11-27T15:25:42.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.371+0000 7fdb71874b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474b1e1920 con 0x56474b40cc00 2023-11-27T15:25:42.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.371+0000 7fdb577fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40cc00 0x56474b416620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.371+0000 7fdb577fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40cc00 0x56474b416620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48176/0 (socket says 172.21.15.133:48176) 2023-11-27T15:25:42.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.371+0000 7fdb577fe700 1 -- 172.21.15.133:0/2385567139 learned_addr learned my addr 172.21.15.133:0/2385567139 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:42.378 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.371+0000 7fdb56ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40d550 0x56474b416b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.372+0000 7fdb71874b80 10 client.0 _ll_register_callbacks cb 0x56474b31d780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:42.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.372+0000 7fdb577fe700 1 -- 172.21.15.133:0/2385567139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40d550 msgr2=0x56474b416b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.372+0000 7fdb577fe700 1 --2- 172.21.15.133:0/2385567139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40d550 0x56474b416b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.372+0000 7fdb577fe700 1 -- 172.21.15.133:0/2385567139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56474b40e110 msgr2=0x56474b41dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.372+0000 7fdb577fe700 1 --2- 172.21.15.133:0/2385567139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56474b40e110 0x56474b41dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.372+0000 7fdb577fe700 1 -- 172.21.15.133:0/2385567139 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4c009c30 con 0x56474b40cc00 2023-11-27T15:25:42.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.372+0000 7fdb577fe700 1 --2- 172.21.15.133:0/2385567139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40cc00 0x56474b416620 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800c010 tx=0x7fdb48000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f57d20c1fdf8f54 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:42.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.372+0000 7fdb71874b80 1 -- 172.21.15.133:0/2385567139 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56474b3ffb20 con 0x56474b40cc00 2023-11-27T15:25:42.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.372+0000 7fdb36ffd700 10 client.26068 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:25:42.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.372+0000 7fdb36ffd700 1 -- 172.21.15.133:0/2385567139 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb480a2d60 con 0x56474b40cc00 2023-11-27T15:25:42.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.372+0000 7fdb36ffd700 1 -- 172.21.15.133:0/2385567139 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fdb480cb110 con 0x56474b40cc00 2023-11-27T15:25:42.379 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.373+0000 7fdb36ffd700 1 -- 172.21.15.133:0/2385567139 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb48099e90 con 0x56474b40cc00 2023-11-27T15:25:42.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.373+0000 7fdb71874b80 10 client.26068 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:42.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.373+0000 7fdb71874b80 1 -- 172.21.15.133:0/2385567139 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({fsmap=24}) v3 -- 0x56474b324190 con 0x56474b40cc00 2023-11-27T15:25:42.380 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.373+0000 7fdb36ffd700 1 -- 172.21.15.133:0/2385567139 <== mon.1 v2:172.21.15.177:3300/0 4 ==== fsmap(e 24) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdb480a28d0 con 0x56474b40cc00 2023-11-27T15:25:42.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.373+0000 7fdb71874b80 10 client.26068 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:42.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.373+0000 7fdb71874b80 10 client.26068 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:25:42.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.374+0000 7fdb71874b80 20 client.26068 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:42.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.374+0000 7fdb71874b80 1 --2- 172.21.15.133:0/2385567139 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56474b421ef0 0x56474b4242d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.381 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.374+0000 7fdb71874b80 4 client.26068 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:42.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.374+0000 7fdb71874b80 1 -- 172.21.15.133:0/2385567139 --> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56474b3200a0 con 0x56474b421ef0 2023-11-27T15:25:42.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.374+0000 7fdb56ffd700 1 --2- 172.21.15.133:0/2385567139 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56474b421ef0 0x56474b4242d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.374+0000 7fdb56ffd700 1 --2- 172.21.15.133:0/2385567139 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56474b421ef0 0x56474b4242d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db563bd787900444 server_cookie=9d1fd41d34c0d5be in_seq=0 out_seq=0 2023-11-27T15:25:42.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.375+0000 7fdb36ffd700 10 client.26068 ms_handle_connect on v2:172.21.15.174:6836/1955090548 2023-11-27T15:25:42.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.376+0000 7fdb36ffd700 1 -- 172.21.15.133:0/2385567139 <== mds.1 v2:172.21.15.174:6836/1955090548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56474b3200a0 con 0x56474b421ef0 2023-11-27T15:25:42.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.376+0000 7fdb36ffd700 10 client.26068 handle_command_reply: tid=0 2023-11-27T15:25:42.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.376+0000 7fdb71874b80 1 client.26068 shutdown 2023-11-27T15:25:42.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.376+0000 7fdb71874b80 1 -- 172.21.15.133:0/2385567139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40cc00 msgr2=0x56474b416620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.376+0000 7fdb71874b80 1 --2- 172.21.15.133:0/2385567139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40cc00 0x56474b416620 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800c010 tx=0x7fdb48000bc0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.376+0000 7fdb71874b80 1 -- 172.21.15.133:0/2385567139 shutdown_connections 2023-11-27T15:25:42.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.376+0000 7fdb71874b80 1 --2- 172.21.15.133:0/2385567139 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56474b40e110 0x56474b41dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.376+0000 7fdb71874b80 1 --2- 172.21.15.133:0/2385567139 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40cc00 0x56474b416620 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.382 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.376+0000 7fdb71874b80 1 --2- 172.21.15.133:0/2385567139 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56474b421ef0 0x56474b4242d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.376+0000 7fdb71874b80 1 --2- 172.21.15.133:0/2385567139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40d550 0x56474b416b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.376+0000 7fdb36ffd700 0 client.26068 ms_handle_reset on v2:172.21.15.174:6836/1955090548 2023-11-27T15:25:42.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.376+0000 7fdb71874b80 1 -- 172.21.15.133:0/2385567139 >> 172.21.15.133:0/2385567139 conn(0x56474b402360 msgr2=0x56474b406de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:42.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.376+0000 7fdb71874b80 1 -- 172.21.15.133:0/2385567139 shutdown_connections 2023-11-27T15:25:42.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.376+0000 7fdb71874b80 1 -- 172.21.15.133:0/2385567139 wait complete. 2023-11-27T15:25:42.383 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.377+0000 7fdb71874b80 20 client.26068 trim_cache size 0 max 16384 2023-11-27T15:25:42.400 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.396+0000 7fdb71874b80 1 Processor -- start 2023-11-27T15:25:42.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.396+0000 7fdb71874b80 1 -- start start 2023-11-27T15:25:42.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.397+0000 7fdb71874b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40cc00 0x56474b41e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.397+0000 7fdb71874b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56474b40d550 0x56474b41db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.397+0000 7fdb71874b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40e110 0x56474b4163d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.397+0000 7fdb71874b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474b415ad0 con 0x56474b40d550 2023-11-27T15:25:42.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.397+0000 7fdb71874b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474b4091a0 con 0x56474b40cc00 2023-11-27T15:25:42.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.397+0000 7fdb71874b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474b408aa0 con 0x56474b40e110 2023-11-27T15:25:42.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.397+0000 7fdb577fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40cc00 0x56474b41e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.397+0000 7fdb577fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40cc00 0x56474b41e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:58238/0 (socket says 172.21.15.133:58238) 2023-11-27T15:25:42.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.397+0000 7fdb577fe700 1 -- 172.21.15.133:0/3467573530 learned_addr learned my addr 172.21.15.133:0/3467573530 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:42.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.397+0000 7fdb56ffd700 1 --2- 172.21.15.133:0/3467573530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56474b40d550 0x56474b41db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.397+0000 7fdb57fff700 1 --2- 172.21.15.133:0/3467573530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40e110 0x56474b4163d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.398+0000 7fdb577fe700 1 -- 172.21.15.133:0/3467573530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40e110 msgr2=0x56474b4163d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.398+0000 7fdb577fe700 1 --2- 172.21.15.133:0/3467573530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40e110 0x56474b4163d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.398+0000 7fdb577fe700 1 -- 172.21.15.133:0/3467573530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56474b40d550 msgr2=0x56474b41db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.398+0000 7fdb577fe700 1 --2- 172.21.15.133:0/3467573530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56474b40d550 0x56474b41db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.398+0000 7fdb577fe700 1 -- 172.21.15.133:0/3467573530 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56474b323ee0 con 0x56474b40cc00 2023-11-27T15:25:42.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.398+0000 7fdb57fff700 1 --2- 172.21.15.133:0/3467573530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40e110 0x56474b4163d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:42.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.398+0000 7fdb577fe700 1 --2- 172.21.15.133:0/3467573530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40cc00 0x56474b41e490 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800fa40 tx=0x7fdb48025000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6826eb2105938d9e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:42.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.398+0000 7fdb3e7fc700 1 -- 172.21.15.133:0/3467573530 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb480a2d60 con 0x56474b40cc00 2023-11-27T15:25:42.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.398+0000 7fdb3e7fc700 1 -- 172.21.15.133:0/3467573530 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56474b3ffb20 con 0x56474b40cc00 2023-11-27T15:25:42.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.398+0000 7fdb3e7fc700 1 -- 172.21.15.133:0/3467573530 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4809b780 con 0x56474b40cc00 2023-11-27T15:25:42.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.399+0000 7fdb71874b80 1 -- 172.21.15.133:0/3467573530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40cc00 msgr2=0x56474b41e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.399+0000 7fdb71874b80 1 --2- 172.21.15.133:0/3467573530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40cc00 0x56474b41e490 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800fa40 tx=0x7fdb48025000 comp rx=0 tx=0).stop 2023-11-27T15:25:42.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.399+0000 7fdb71874b80 1 -- 172.21.15.133:0/3467573530 shutdown_connections 2023-11-27T15:25:42.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.399+0000 7fdb71874b80 1 --2- 172.21.15.133:0/3467573530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40e110 0x56474b4163d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.399+0000 7fdb71874b80 1 --2- 172.21.15.133:0/3467573530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40cc00 0x56474b41e490 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.399+0000 7fdb71874b80 1 --2- 172.21.15.133:0/3467573530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56474b40d550 0x56474b41db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.399+0000 7fdb71874b80 1 -- 172.21.15.133:0/3467573530 >> 172.21.15.133:0/3467573530 conn(0x56474b402360 msgr2=0x56474b4152f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.399+0000 7fdb71874b80 1 -- 172.21.15.133:0/3467573530 shutdown_connections 2023-11-27T15:25:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.399+0000 7fdb71874b80 1 -- 172.21.15.133:0/3467573530 wait complete. 2023-11-27T15:25:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.400+0000 7fdb71874b80 1 -- start start 2023-11-27T15:25:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.400+0000 7fdb71874b80 1 Processor -- start 2023-11-27T15:25:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.401+0000 7fdb71874b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40cc00 0x56474b41d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.401+0000 7fdb71874b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56474b40d550 0x56474b41d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.401+0000 7fdb71874b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40e110 0x56474b419d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.401+0000 7fdb71874b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474b41a290 con 0x56474b40d550 2023-11-27T15:25:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.401+0000 7fdb71874b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474b41a400 con 0x56474b40cc00 2023-11-27T15:25:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.401+0000 7fdb71874b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56474b41a5c0 con 0x56474b40e110 2023-11-27T15:25:42.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.401+0000 7fdb71874b80 10 client.0 _ll_register_callbacks cb 0x7fdb4809a300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:42.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.401+0000 7fdb577fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40cc00 0x56474b41d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.401+0000 7fdb577fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40cc00 0x56474b41d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:58250/0 (socket says 172.21.15.133:58250) 2023-11-27T15:25:42.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.401+0000 7fdb577fe700 1 -- 172.21.15.133:0/126616011 learned_addr learned my addr 172.21.15.133:0/126616011 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:42.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.401+0000 7fdb56ffd700 1 --2- 172.21.15.133:0/126616011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56474b40d550 0x56474b41d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.401+0000 7fdb57fff700 1 --2- 172.21.15.133:0/126616011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40e110 0x56474b419d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.401+0000 7fdb577fe700 1 -- 172.21.15.133:0/126616011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40e110 msgr2=0x56474b419d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.401+0000 7fdb577fe700 1 --2- 172.21.15.133:0/126616011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40e110 0x56474b419d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.401+0000 7fdb577fe700 1 -- 172.21.15.133:0/126616011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56474b40d550 msgr2=0x56474b41d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.401+0000 7fdb577fe700 1 --2- 172.21.15.133:0/126616011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56474b40d550 0x56474b41d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.401+0000 7fdb577fe700 1 -- 172.21.15.133:0/126616011 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56474b324190 con 0x56474b40cc00 2023-11-27T15:25:42.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.401+0000 7fdb577fe700 1 --2- 172.21.15.133:0/126616011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40cc00 0x56474b41d2b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdb480973c0 tx=0x7fdb48009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff57906f9e3ce86e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:42.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.401+0000 7fdb36ffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:25:42.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.402+0000 7fdb36ffd700 1 -- 172.21.15.133:0/126616011 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb480a28d0 con 0x56474b40cc00 2023-11-27T15:25:42.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.402+0000 7fdb36ffd700 1 -- 172.21.15.133:0/126616011 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fdb480ba2c0 con 0x56474b40cc00 2023-11-27T15:25:42.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.402+0000 7fdb71874b80 1 -- 172.21.15.133:0/126616011 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb4c004e50 con 0x56474b40cc00 2023-11-27T15:25:42.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.402+0000 7fdb36ffd700 1 -- 172.21.15.133:0/126616011 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb48096790 con 0x56474b40cc00 2023-11-27T15:25:42.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.402+0000 7fdb71874b80 10 client.26090 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:42.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.402+0000 7fdb71874b80 1 -- 172.21.15.133:0/126616011 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56474b174c60 con 0x56474b40cc00 2023-11-27T15:25:42.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.402+0000 7fdb36ffd700 1 -- 172.21.15.133:0/126616011 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb480c6020 con 0x56474b40cc00 2023-11-27T15:25:42.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.402+0000 7fdb36ffd700 1 -- 172.21.15.133:0/126616011 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 24) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdb480052f0 con 0x56474b40cc00 2023-11-27T15:25:42.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.403+0000 7fdb71874b80 10 client.26090 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:42.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.403+0000 7fdb71874b80 10 client.26090 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:25:42.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.403+0000 7fdb71874b80 20 client.26090 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:42.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.403+0000 7fdb71874b80 1 --2- 172.21.15.133:0/126616011 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56474b476ba0 0x56474b478f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.403+0000 7fdb71874b80 4 client.26090 mds_command: new command op to 24521 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:25:42.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.403+0000 7fdb71874b80 1 -- 172.21.15.133:0/126616011 --> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56474b32c660 con 0x56474b476ba0 2023-11-27T15:25:42.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.403+0000 7fdb56ffd700 1 --2- 172.21.15.133:0/126616011 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56474b476ba0 0x56474b478f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.403+0000 7fdb36ffd700 10 client.26090 ms_handle_connect on v2:172.21.15.174:6836/1955090548 2023-11-27T15:25:42.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.403+0000 7fdb56ffd700 1 --2- 172.21.15.133:0/126616011 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56474b476ba0 0x56474b478f80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9364e7fc3de3a739 server_cookie=4cd152be51dabbf6 in_seq=0 out_seq=0 2023-11-27T15:25:42.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.404+0000 7fdb36ffd700 1 -- 172.21.15.133:0/126616011 <== mds.1 v2:172.21.15.174:6836/1955090548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56474b32c660 con 0x56474b476ba0 2023-11-27T15:25:42.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.404+0000 7fdb36ffd700 10 client.26090 handle_command_reply: tid=0 2023-11-27T15:25:42.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.404+0000 7fdb71874b80 1 client.26090 shutdown 2023-11-27T15:25:42.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.404+0000 7fdb71874b80 1 -- 172.21.15.133:0/126616011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40cc00 msgr2=0x56474b41d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.404+0000 7fdb71874b80 1 --2- 172.21.15.133:0/126616011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40cc00 0x56474b41d2b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdb480973c0 tx=0x7fdb48009260 comp rx=0 tx=0).stop 2023-11-27T15:25:42.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.404+0000 7fdb71874b80 1 -- 172.21.15.133:0/126616011 shutdown_connections 2023-11-27T15:25:42.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.404+0000 7fdb71874b80 1 --2- 172.21.15.133:0/126616011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x56474b40e110 0x56474b419d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.404+0000 7fdb71874b80 1 --2- 172.21.15.133:0/126616011 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x56474b476ba0 0x56474b478f80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.404+0000 7fdb71874b80 1 --2- 172.21.15.133:0/126616011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474b40cc00 0x56474b41d2b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.404+0000 7fdb71874b80 1 --2- 172.21.15.133:0/126616011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56474b40d550 0x56474b41d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.404+0000 7fdb71874b80 1 -- 172.21.15.133:0/126616011 >> 172.21.15.133:0/126616011 conn(0x56474b402360 msgr2=0x56474b410fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:42.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.404+0000 7fdb36ffd700 0 client.26090 ms_handle_reset on v2:172.21.15.174:6836/1955090548 2023-11-27T15:25:42.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.404+0000 7fdb71874b80 1 -- 172.21.15.133:0/126616011 shutdown_connections 2023-11-27T15:25:42.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.404+0000 7fdb71874b80 1 -- 172.21.15.133:0/126616011 wait complete. 2023-11-27T15:25:42.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.405+0000 7fdb71874b80 20 client.26090 trim_cache size 0 max 16384 2023-11-27T15:25:42.413 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2023-11-27T15:25:42.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.405+0000 7fdb5dfff700 1 -- 172.21.15.133:0/2714421804 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fdb400a11d0 msgr2=0x7fdb400a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.405+0000 7fdb5dfff700 1 --2- 172.21.15.133:0/2714421804 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fdb400a11d0 0x7fdb400a3680 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdb500075c0 tx=0x7fdb50009040 comp rx=0 tx=0).stop 2023-11-27T15:25:42.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.405+0000 7fdb5dfff700 1 -- 172.21.15.133:0/2714421804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb581002d0 msgr2=0x7fdb5819c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.405+0000 7fdb5dfff700 1 --2- 172.21.15.133:0/2714421804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb581002d0 0x7fdb5819c6d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009810 tx=0x7fdb4800b790 comp rx=0 tx=0).stop 2023-11-27T15:25:42.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.405+0000 7fdb5dfff700 1 -- 172.21.15.133:0/2714421804 shutdown_connections 2023-11-27T15:25:42.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.405+0000 7fdb5dfff700 1 --2- 172.21.15.133:0/2714421804 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fdb400a11d0 0x7fdb400a3680 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.405+0000 7fdb5dfff700 1 --2- 172.21.15.133:0/2714421804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb58101e50 0x7fdb581a3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.405+0000 7fdb5dfff700 1 --2- 172.21.15.133:0/2714421804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdb581002d0 0x7fdb5819c6d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.405+0000 7fdb5dfff700 1 --2- 172.21.15.133:0/2714421804 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdb581014c0 0x7fdb5819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.405+0000 7fdb5dfff700 1 -- 172.21.15.133:0/2714421804 >> 172.21.15.133:0/2714421804 conn(0x7fdb580fb8d0 msgr2=0x7fdb580fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:42.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.406+0000 7fdb5dfff700 1 -- 172.21.15.133:0/2714421804 shutdown_connections 2023-11-27T15:25:42.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.406+0000 7fdb5dfff700 1 -- 172.21.15.133:0/2714421804 wait complete. 2023-11-27T15:25:42.434 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:25:42.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.698+0000 7f92d93ff700 1 -- 172.21.15.133:0/1172511493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92d4105f80 msgr2=0x7f92d41083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.698+0000 7f92d93ff700 1 --2- 172.21.15.133:0/1172511493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92d4105f80 0x7f92d41083e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009f90 tx=0x7f92cc00b330 comp rx=0 tx=0).stop 2023-11-27T15:25:42.703 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.699+0000 7f92d93ff700 1 -- 172.21.15.133:0/1172511493 shutdown_connections 2023-11-27T15:25:42.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.699+0000 7f92d93ff700 1 --2- 172.21.15.133:0/1172511493 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92d4105f80 0x7f92d41083e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.699+0000 7f92d93ff700 1 --2- 172.21.15.133:0/1172511493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92d4100d40 0x7f92d4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.699+0000 7f92d93ff700 1 --2- 172.21.15.133:0/1172511493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92d4103660 0x7f92d4105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.699+0000 7f92d93ff700 1 -- 172.21.15.133:0/1172511493 >> 172.21.15.133:0/1172511493 conn(0x7f92d40fa7c0 msgr2=0x7f92d40fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:42.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.699+0000 7f92d93ff700 1 -- 172.21.15.133:0/1172511493 shutdown_connections 2023-11-27T15:25:42.704 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.699+0000 7f92d93ff700 1 -- 172.21.15.133:0/1172511493 wait complete. 2023-11-27T15:25:42.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.701+0000 7f92d93ff700 1 Processor -- start 2023-11-27T15:25:42.705 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.701+0000 7f92d93ff700 1 -- start start 2023-11-27T15:25:42.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.701+0000 7f92d93ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92d4100d40 0x7f92d4193ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.701+0000 7f92d93ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92d4103660 0x7f92d4194420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.701+0000 7f92d93ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92d4105f80 0x7f92d41a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.701+0000 7f92d93ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d41a43a0 con 0x7f92d4105f80 2023-11-27T15:25:42.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.701+0000 7f92d93ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d41a44e0 con 0x7f92d4103660 2023-11-27T15:25:42.706 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.701+0000 7f92d93ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d41a4620 con 0x7f92d4100d40 2023-11-27T15:25:42.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.702+0000 7f92d2ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92d4100d40 0x7f92d4193ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.702+0000 7f92d2ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92d4100d40 0x7f92d4193ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48214/0 (socket says 172.21.15.133:48214) 2023-11-27T15:25:42.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.702+0000 7f92d2ffd700 1 -- 172.21.15.133:0/4154164959 learned_addr learned my addr 172.21.15.133:0/4154164959 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:42.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.702+0000 7f92d37fe700 1 --2- 172.21.15.133:0/4154164959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92d4105f80 0x7f92d41a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.707 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.702+0000 7f92d27fc700 1 --2- 172.21.15.133:0/4154164959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92d4103660 0x7f92d4194420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.702+0000 7f92d37fe700 1 -- 172.21.15.133:0/4154164959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92d4100d40 msgr2=0x7f92d4193ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.702+0000 7f92d37fe700 1 --2- 172.21.15.133:0/4154164959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92d4100d40 0x7f92d4193ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.702+0000 7f92d37fe700 1 -- 172.21.15.133:0/4154164959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92d4103660 msgr2=0x7f92d4194420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.702+0000 7f92d37fe700 1 --2- 172.21.15.133:0/4154164959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92d4103660 0x7f92d4194420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.702+0000 7f92d37fe700 1 -- 172.21.15.133:0/4154164959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92cc009c40 con 0x7f92d4105f80 2023-11-27T15:25:42.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.703+0000 7f92d37fe700 1 --2- 172.21.15.133:0/4154164959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92d4105f80 0x7f92d41a3e60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00b6c0 tx=0x7f92cc004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:42.708 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.703+0000 7f92bbfff700 1 -- 172.21.15.133:0/4154164959 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc00e040 con 0x7f92d4105f80 2023-11-27T15:25:42.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.703+0000 7f92bbfff700 1 -- 172.21.15.133:0/4154164959 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92cc008ec0 con 0x7f92d4105f80 2023-11-27T15:25:42.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.703+0000 7f92bbfff700 1 -- 172.21.15.133:0/4154164959 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc0052a0 con 0x7f92d4105f80 2023-11-27T15:25:42.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.703+0000 7f92d93ff700 1 -- 172.21.15.133:0/4154164959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d410bc70 con 0x7f92d4105f80 2023-11-27T15:25:42.709 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.703+0000 7f92d93ff700 1 -- 172.21.15.133:0/4154164959 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d410bae0 con 0x7f92d4105f80 2023-11-27T15:25:42.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.708+0000 7f92bbfff700 1 -- 172.21.15.133:0/4154164959 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f92cc01d470 con 0x7f92d4105f80 2023-11-27T15:25:42.713 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.709+0000 7f92bbfff700 1 --2- 172.21.15.133:0/4154164959 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92bc0a11d0 0x7f92bc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.709+0000 7f92bbfff700 1 -- 172.21.15.133:0/4154164959 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f92cc0ce210 con 0x7f92d4105f80 2023-11-27T15:25:42.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.710+0000 7f92d2ffd700 1 --2- 172.21.15.133:0/4154164959 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92bc0a11d0 0x7f92bc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.714 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.710+0000 7f92ecde1b80 1 Processor -- start 2023-11-27T15:25:42.715 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.710+0000 7f92ecde1b80 1 -- start start 2023-11-27T15:25:42.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.710+0000 7f92ecde1b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d76e6c3c00 0x55d76e6c4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.710+0000 7f92ecde1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c4550 0x55d76e6c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.710+0000 7f92ecde1b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d76e6c5110 0x55d76e6c5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.710+0000 7f92ecde1b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d76e5c9060 con 0x55d76e6c3c00 2023-11-27T15:25:42.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.710+0000 7f92ecde1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d76e61b8a0 con 0x55d76e6c4550 2023-11-27T15:25:42.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.710+0000 7f92ecde1b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d76e614d30 con 0x55d76e6c5110 2023-11-27T15:25:42.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.711+0000 7f92d2ffd700 1 --2- 172.21.15.133:0/4154164959 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92bc0a11d0 0x7f92bc0a3680 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f92c400ad20 tx=0x7f92c4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:42.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.711+0000 7f92d27fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c4550 0x55d76e6c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.711+0000 7f92d27fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c4550 0x55d76e6c4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:58264/0 (socket says 172.21.15.133:58264) 2023-11-27T15:25:42.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.711+0000 7f92d27fc700 1 -- 172.21.15.133:0/1281669112 learned_addr learned my addr 172.21.15.133:0/1281669112 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:42.716 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.711+0000 7f92d2ffd700 1 --2- 172.21.15.133:0/1281669112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d76e6c3c00 0x55d76e6c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.711+0000 7f92d27fc700 1 -- 172.21.15.133:0/1281669112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d76e6c5110 msgr2=0x55d76e6c5580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-27T15:25:42.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.711+0000 7f92d27fc700 1 --2- 172.21.15.133:0/1281669112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d76e6c5110 0x55d76e6c5580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.711+0000 7f92d27fc700 1 -- 172.21.15.133:0/1281669112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d76e6c3c00 msgr2=0x55d76e6c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.711+0000 7f92d27fc700 1 --2- 172.21.15.133:0/1281669112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d76e6c3c00 0x55d76e6c4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.711+0000 7f92d27fc700 1 -- 172.21.15.133:0/1281669112 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d76e496630 con 0x55d76e6c4550 2023-11-27T15:25:42.717 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.711+0000 7f92d2ffd700 1 --2- 172.21.15.133:0/1281669112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d76e6c3c00 0x55d76e6c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:42.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.712+0000 7f92d27fc700 1 --2- 172.21.15.133:0/1281669112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c4550 0x55d76e6c4b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009d60 tx=0x7f92c8011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adbc217b7d869f99 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:42.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.712+0000 7f92b97fa700 1 -- 172.21.15.133:0/1281669112 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c8019630 con 0x55d76e6c4550 2023-11-27T15:25:42.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.712+0000 7f92b97fa700 1 -- 172.21.15.133:0/1281669112 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92c8019c50 con 0x55d76e6c4550 2023-11-27T15:25:42.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.712+0000 7f92b97fa700 1 -- 172.21.15.133:0/1281669112 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c800ebb0 con 0x55d76e6c4550 2023-11-27T15:25:42.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.713+0000 7f92ecde1b80 1 -- 172.21.15.133:0/1281669112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c4550 msgr2=0x55d76e6c4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.713+0000 7f92ecde1b80 1 --2- 172.21.15.133:0/1281669112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c4550 0x55d76e6c4b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f92c8009d60 tx=0x7f92c8011720 comp rx=0 tx=0).stop 2023-11-27T15:25:42.718 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.713+0000 7f92ecde1b80 1 -- 172.21.15.133:0/1281669112 shutdown_connections 2023-11-27T15:25:42.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.713+0000 7f92ecde1b80 1 --2- 172.21.15.133:0/1281669112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d76e6c5110 0x55d76e6c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.713+0000 7f92ecde1b80 1 --2- 172.21.15.133:0/1281669112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d76e6c3c00 0x55d76e6c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.713+0000 7f92ecde1b80 1 --2- 172.21.15.133:0/1281669112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c4550 0x55d76e6c4b30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.713+0000 7f92ecde1b80 1 -- 172.21.15.133:0/1281669112 >> 172.21.15.133:0/1281669112 conn(0x55d76e6b9360 msgr2=0x55d76e6be670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:42.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.713+0000 7f92ecde1b80 1 -- 172.21.15.133:0/1281669112 shutdown_connections 2023-11-27T15:25:42.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.713+0000 7f92ecde1b80 1 -- 172.21.15.133:0/1281669112 wait complete. 2023-11-27T15:25:42.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.714+0000 7f92ecde1b80 1 -- start start 2023-11-27T15:25:42.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.714+0000 7f92ecde1b80 1 Processor -- start 2023-11-27T15:25:42.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.715+0000 7f92ecde1b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d76e6c3c00 0x55d76e6cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.715+0000 7f92ecde1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c4550 0x55d76e6cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.715+0000 7f92ecde1b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d76e6c5110 0x55d76e6d4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.715+0000 7f92ecde1b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d76e5b68e0 con 0x55d76e6c3c00 2023-11-27T15:25:42.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.715+0000 7f92ecde1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d76e5b6580 con 0x55d76e6c4550 2023-11-27T15:25:42.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.715+0000 7f92ecde1b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d76e498920 con 0x55d76e6c5110 2023-11-27T15:25:42.720 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.715+0000 7f92ecde1b80 10 client.0 _ll_register_callbacks cb 0x55d76e5d4780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:42.721 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.716+0000 7f92d27fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c4550 0x55d76e6cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.716+0000 7f92d27fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c4550 0x55d76e6cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:58272/0 (socket says 172.21.15.133:58272) 2023-11-27T15:25:42.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.716+0000 7f92d27fc700 1 -- 172.21.15.133:0/3263604416 learned_addr learned my addr 172.21.15.133:0/3263604416 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:42.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.716+0000 7f92d37fe700 1 --2- 172.21.15.133:0/3263604416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d76e6c5110 0x55d76e6d4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.716+0000 7f92d27fc700 1 -- 172.21.15.133:0/3263604416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d76e6c5110 msgr2=0x55d76e6d4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.716+0000 7f92d27fc700 1 --2- 172.21.15.133:0/3263604416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d76e6c5110 0x55d76e6d4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.716+0000 7f92d27fc700 1 -- 172.21.15.133:0/3263604416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d76e6c3c00 msgr2=0x55d76e6cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.716+0000 7f92d27fc700 1 --2- 172.21.15.133:0/3263604416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d76e6c3c00 0x55d76e6cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.716+0000 7f92d27fc700 1 -- 172.21.15.133:0/3263604416 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c80099c0 con 0x55d76e6c4550 2023-11-27T15:25:42.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.716+0000 7f92d37fe700 1 --2- 172.21.15.133:0/3263604416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d76e6c5110 0x55d76e6d4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:42.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.717+0000 7f92d27fc700 1 --2- 172.21.15.133:0/3263604416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c4550 0x55d76e6cdbb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f92c8005920 tx=0x7f92c800ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ead975e12a43279 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:42.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.717+0000 7f92b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:25:42.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.717+0000 7f92b6ff5700 1 -- 172.21.15.133:0/3263604416 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c8019840 con 0x55d76e6c4550 2023-11-27T15:25:42.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.717+0000 7f92ecde1b80 1 -- 172.21.15.133:0/3263604416 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d76e6b6b20 con 0x55d76e6c4550 2023-11-27T15:25:42.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.717+0000 7f92b6ff5700 1 -- 172.21.15.133:0/3263604416 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f92c80199a0 con 0x55d76e6c4550 2023-11-27T15:25:42.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.718+0000 7f92b6ff5700 1 -- 172.21.15.133:0/3263604416 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92c801ae60 con 0x55d76e6c4550 2023-11-27T15:25:42.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.719+0000 7f92ecde1b80 10 client.26102 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:42.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.719+0000 7f92ecde1b80 1 -- 172.21.15.133:0/3263604416 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d76e5db190 con 0x55d76e6c4550 2023-11-27T15:25:42.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.719+0000 7f92b6ff5700 1 -- 172.21.15.133:0/3263604416 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d76e6b6b20 con 0x55d76e6c4550 2023-11-27T15:25:42.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.719+0000 7f92b6ff5700 1 -- 172.21.15.133:0/3263604416 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 24) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f92c8004b20 con 0x55d76e6c4550 2023-11-27T15:25:42.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.719+0000 7f92ecde1b80 10 client.26102 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:42.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.719+0000 7f92ecde1b80 10 client.26102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:42.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.719+0000 7f92ecde1b80 20 client.26102 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:42.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.719+0000 7f92ecde1b80 1 --2- 172.21.15.133:0/3263604416 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d76e6d9270 0x55d76e6db680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.720+0000 7f92ecde1b80 4 client.26102 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:42.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.720+0000 7f92ecde1b80 1 -- 172.21.15.133:0/3263604416 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d76e5d70a0 con 0x55d76e6d9270 2023-11-27T15:25:42.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.720+0000 7f92d2ffd700 1 --2- 172.21.15.133:0/3263604416 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d76e6d9270 0x55d76e6db680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.725 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.720+0000 7f92d2ffd700 1 --2- 172.21.15.133:0/3263604416 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d76e6d9270 0x55d76e6db680 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b45e58026a57646 server_cookie=7a0bb98b4757ca42 in_seq=0 out_seq=0 2023-11-27T15:25:42.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.720+0000 7f92b6ff5700 10 client.26102 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:42.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.721+0000 7f92b6ff5700 1 -- 172.21.15.133:0/3263604416 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d76e5d70a0 con 0x55d76e6d9270 2023-11-27T15:25:42.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.721+0000 7f92b6ff5700 10 client.26102 handle_command_reply: tid=0 2023-11-27T15:25:42.726 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.721+0000 7f92ecde1b80 1 client.26102 shutdown 2023-11-27T15:25:42.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.721+0000 7f92ecde1b80 1 -- 172.21.15.133:0/3263604416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c4550 msgr2=0x55d76e6cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.721+0000 7f92ecde1b80 1 --2- 172.21.15.133:0/3263604416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c4550 0x55d76e6cdbb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f92c8005920 tx=0x7f92c800ec30 comp rx=0 tx=0).stop 2023-11-27T15:25:42.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.722+0000 7f92ecde1b80 1 -- 172.21.15.133:0/3263604416 shutdown_connections 2023-11-27T15:25:42.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.722+0000 7f92ecde1b80 1 --2- 172.21.15.133:0/3263604416 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d76e6d9270 0x55d76e6db680 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.722+0000 7f92b6ff5700 0 client.26102 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:42.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.722+0000 7f92ecde1b80 1 --2- 172.21.15.133:0/3263604416 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d76e6c5110 0x55d76e6d4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.722+0000 7f92ecde1b80 1 --2- 172.21.15.133:0/3263604416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d76e6c3c00 0x55d76e6cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.722+0000 7f92ecde1b80 1 --2- 172.21.15.133:0/3263604416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c4550 0x55d76e6cdbb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.722+0000 7f92ecde1b80 1 -- 172.21.15.133:0/3263604416 >> 172.21.15.133:0/3263604416 conn(0x55d76e6b9360 msgr2=0x55d76e6bdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:42.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.722+0000 7f92ecde1b80 1 -- 172.21.15.133:0/3263604416 shutdown_connections 2023-11-27T15:25:42.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.723+0000 7f92ecde1b80 1 -- 172.21.15.133:0/3263604416 wait complete. 2023-11-27T15:25:42.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.723+0000 7f92ecde1b80 20 client.26102 trim_cache size 0 max 16384 2023-11-27T15:25:42.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.742+0000 7f92ecde1b80 1 Processor -- start 2023-11-27T15:25:42.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.742+0000 7f92ecde1b80 1 -- start start 2023-11-27T15:25:42.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.742+0000 7f92ecde1b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d76e6c3c00 0x55d76e6c8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.742+0000 7f92ecde1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c4550 0x55d76e6d4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.742+0000 7f92ecde1b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d76e6c5110 0x55d76e6c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.742+0000 7f92ecde1b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d76e6ce860 con 0x55d76e6c5110 2023-11-27T15:25:42.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.742+0000 7f92ecde1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d76e6c4f90 con 0x55d76e6c4550 2023-11-27T15:25:42.747 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.742+0000 7f92ecde1b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d76e6ccb20 con 0x55d76e6c3c00 2023-11-27T15:25:42.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.743+0000 7f92d2ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d76e6c3c00 0x55d76e6c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.743+0000 7f92d27fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c4550 0x55d76e6d4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.743+0000 7f92d2ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d76e6c3c00 0x55d76e6c8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48228/0 (socket says 172.21.15.133:48228) 2023-11-27T15:25:42.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.743+0000 7f92d2ffd700 1 -- 172.21.15.133:0/2007324745 learned_addr learned my addr 172.21.15.133:0/2007324745 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:42.748 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.743+0000 7f92d37fe700 1 --2- 172.21.15.133:0/2007324745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d76e6c5110 0x55d76e6c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.743+0000 7f92d2ffd700 1 -- 172.21.15.133:0/2007324745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c4550 msgr2=0x55d76e6d4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.743+0000 7f92d2ffd700 1 --2- 172.21.15.133:0/2007324745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c4550 0x55d76e6d4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.743+0000 7f92d2ffd700 1 -- 172.21.15.133:0/2007324745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d76e6c5110 msgr2=0x55d76e6c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.743+0000 7f92d2ffd700 1 --2- 172.21.15.133:0/2007324745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d76e6c5110 0x55d76e6c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.743+0000 7f92d2ffd700 1 -- 172.21.15.133:0/2007324745 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d76e5daee0 con 0x55d76e6c3c00 2023-11-27T15:25:42.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.743+0000 7f92d37fe700 1 --2- 172.21.15.133:0/2007324745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d76e6c5110 0x55d76e6c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:42.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.743+0000 7f92d2ffd700 1 --2- 172.21.15.133:0/2007324745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d76e6c3c00 0x55d76e6c8060 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f92c400fa30 tx=0x7f92c4012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5709e6ddba01b60d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:42.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.744+0000 7f92b9ffb700 1 -- 172.21.15.133:0/2007324745 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c4021070 con 0x55d76e6c3c00 2023-11-27T15:25:42.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.744+0000 7f92b9ffb700 1 -- 172.21.15.133:0/2007324745 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92c4022040 con 0x55d76e6c3c00 2023-11-27T15:25:42.749 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.744+0000 7f92b9ffb700 1 -- 172.21.15.133:0/2007324745 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c401b410 con 0x55d76e6c3c00 2023-11-27T15:25:42.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.745+0000 7f92ecde1b80 1 -- 172.21.15.133:0/2007324745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d76e6c3c00 msgr2=0x55d76e6c8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.745+0000 7f92ecde1b80 1 --2- 172.21.15.133:0/2007324745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d76e6c3c00 0x55d76e6c8060 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f92c400fa30 tx=0x7f92c4012000 comp rx=0 tx=0).stop 2023-11-27T15:25:42.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.745+0000 7f92ecde1b80 1 -- 172.21.15.133:0/2007324745 shutdown_connections 2023-11-27T15:25:42.750 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.745+0000 7f92ecde1b80 1 --2- 172.21.15.133:0/2007324745 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d76e6c5110 0x55d76e6c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.745+0000 7f92ecde1b80 1 --2- 172.21.15.133:0/2007324745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d76e6c3c00 0x55d76e6c8060 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.745+0000 7f92ecde1b80 1 --2- 172.21.15.133:0/2007324745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c4550 0x55d76e6d4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.745+0000 7f92ecde1b80 1 -- 172.21.15.133:0/2007324745 >> 172.21.15.133:0/2007324745 conn(0x55d76e6b9360 msgr2=0x55d76e6cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.745+0000 7f92ecde1b80 1 -- 172.21.15.133:0/2007324745 shutdown_connections 2023-11-27T15:25:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.745+0000 7f92ecde1b80 1 -- 172.21.15.133:0/2007324745 wait complete. 2023-11-27T15:25:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.745+0000 7f92ecde1b80 1 -- start start 2023-11-27T15:25:42.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.746+0000 7f92ecde1b80 1 Processor -- start 2023-11-27T15:25:42.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.746+0000 7f92ecde1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c3c00 0x55d76e6d2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.746+0000 7f92ecde1b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d76e6c4550 0x55d76e6d2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.746+0000 7f92ecde1b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d76e6c5110 0x55d76e6d3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.746+0000 7f92ecde1b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d76e6d4110 con 0x55d76e6c4550 2023-11-27T15:25:42.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.746+0000 7f92ecde1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d76e6d4280 con 0x55d76e6c3c00 2023-11-27T15:25:42.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.746+0000 7f92ecde1b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d76e6d43f0 con 0x55d76e6c5110 2023-11-27T15:25:42.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.746+0000 7f92ecde1b80 10 client.0 _ll_register_callbacks cb 0x7f92c8009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:42.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.746+0000 7f92d2ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c3c00 0x55d76e6d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.747+0000 7f92d2ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c3c00 0x55d76e6d2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:58302/0 (socket says 172.21.15.133:58302) 2023-11-27T15:25:42.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.747+0000 7f92d2ffd700 1 -- 172.21.15.133:0/3554224317 learned_addr learned my addr 172.21.15.133:0/3554224317 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:42.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.747+0000 7f92d2ffd700 1 -- 172.21.15.133:0/3554224317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d76e6c5110 msgr2=0x55d76e6d3bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-27T15:25:42.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.747+0000 7f92d2ffd700 1 --2- 172.21.15.133:0/3554224317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d76e6c5110 0x55d76e6d3bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.747+0000 7f92d2ffd700 1 -- 172.21.15.133:0/3554224317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d76e6c4550 msgr2=0x55d76e6d2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:42.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.747+0000 7f92d2ffd700 1 --2- 172.21.15.133:0/3554224317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d76e6c4550 0x55d76e6d2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.747+0000 7f92d2ffd700 1 -- 172.21.15.133:0/3554224317 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c400fc40 con 0x55d76e6c3c00 2023-11-27T15:25:42.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.748+0000 7f92d2ffd700 1 --2- 172.21.15.133:0/3554224317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c3c00 0x55d76e6d2680 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f92c4010010 tx=0x7f92c4010b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa1bbe027e4406e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:42.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.748+0000 7f92b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:25:42.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.748+0000 7f92b6ff5700 1 -- 172.21.15.133:0/3554224317 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c4021070 con 0x55d76e6c3c00 2023-11-27T15:25:42.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.748+0000 7f92b6ff5700 1 -- 172.21.15.133:0/3554224317 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f92c401b410 con 0x55d76e6c3c00 2023-11-27T15:25:42.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.748+0000 7f92ecde1b80 1 -- 172.21.15.133:0/3554224317 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92c8019c50 con 0x55d76e6c3c00 2023-11-27T15:25:42.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.749+0000 7f92b6ff5700 1 -- 172.21.15.133:0/3554224317 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92c401b5d0 con 0x55d76e6c3c00 2023-11-27T15:25:42.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.749+0000 7f92ecde1b80 10 client.26108 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:42.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.749+0000 7f92ecde1b80 1 -- 172.21.15.133:0/3554224317 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d76e42bc60 con 0x55d76e6c3c00 2023-11-27T15:25:42.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.749+0000 7f92b6ff5700 1 -- 172.21.15.133:0/3554224317 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f92c8019c50 con 0x55d76e6c3c00 2023-11-27T15:25:42.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.750+0000 7f92b6ff5700 1 -- 172.21.15.133:0/3554224317 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 24) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f92c4029020 con 0x55d76e6c3c00 2023-11-27T15:25:42.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.750+0000 7f92ecde1b80 10 client.26108 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:42.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.750+0000 7f92ecde1b80 10 client.26108 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:42.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.750+0000 7f92ecde1b80 20 client.26108 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:42.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.750+0000 7f92ecde1b80 1 --2- 172.21.15.133:0/3554224317 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d76e6d9270 0x55d76e740ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:42.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.750+0000 7f92ecde1b80 4 client.26108 mds_command: new command op to 24560 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:25:42.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.750+0000 7f92ecde1b80 1 -- 172.21.15.133:0/3554224317 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d76e5e3660 con 0x55d76e6d9270 2023-11-27T15:25:42.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.750+0000 7f92d27fc700 1 --2- 172.21.15.133:0/3554224317 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d76e6d9270 0x55d76e740ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:42.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.750+0000 7f92d27fc700 1 --2- 172.21.15.133:0/3554224317 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d76e6d9270 0x55d76e740ca0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e50286a67da5dcac server_cookie=95559e167aa79aba in_seq=0 out_seq=0 2023-11-27T15:25:42.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.750+0000 7f92b6ff5700 10 client.26108 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:42.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.751+0000 7f92b6ff5700 1 -- 172.21.15.133:0/3554224317 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d76e5e3660 con 0x55d76e6d9270 2023-11-27T15:25:42.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.751+0000 7f92b6ff5700 10 client.26108 handle_command_reply: tid=0 2023-11-27T15:25:42.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.751+0000 7f92ecde1b80 1 client.26108 shutdown 2023-11-27T15:25:42.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.751+0000 7f92ecde1b80 1 -- 172.21.15.133:0/3554224317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c3c00 msgr2=0x55d76e6d2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.751+0000 7f92ecde1b80 1 --2- 172.21.15.133:0/3554224317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c3c00 0x55d76e6d2680 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f92c4010010 tx=0x7f92c4010b90 comp rx=0 tx=0).stop 2023-11-27T15:25:42.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.752+0000 7f92ecde1b80 1 -- 172.21.15.133:0/3554224317 shutdown_connections 2023-11-27T15:25:42.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.752+0000 7f92ecde1b80 1 --2- 172.21.15.133:0/3554224317 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d76e6d9270 0x55d76e740ca0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.752+0000 7f92ecde1b80 1 --2- 172.21.15.133:0/3554224317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d76e6c5110 0x55d76e6d3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.752+0000 7f92ecde1b80 1 --2- 172.21.15.133:0/3554224317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d76e6c3c00 0x55d76e6d2680 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.752+0000 7f92ecde1b80 1 --2- 172.21.15.133:0/3554224317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d76e6c4550 0x55d76e6d2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.752+0000 7f92ecde1b80 1 -- 172.21.15.133:0/3554224317 >> 172.21.15.133:0/3554224317 conn(0x55d76e6b9360 msgr2=0x55d76e6c8060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:42.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.752+0000 7f92b6ff5700 0 client.26108 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:42.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.752+0000 7f92ecde1b80 1 -- 172.21.15.133:0/3554224317 shutdown_connections 2023-11-27T15:25:42.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.752+0000 7f92ecde1b80 1 -- 172.21.15.133:0/3554224317 wait complete. 2023-11-27T15:25:42.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.753+0000 7f92ecde1b80 20 client.26108 trim_cache size 0 max 16384 2023-11-27T15:25:42.758 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2023-11-27T15:25:42.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.753+0000 7f92d93ff700 1 -- 172.21.15.133:0/4154164959 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92bc0a11d0 msgr2=0x7f92bc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.753+0000 7f92d93ff700 1 --2- 172.21.15.133:0/4154164959 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92bc0a11d0 0x7f92bc0a3680 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f92c400ad20 tx=0x7f92c4005f90 comp rx=0 tx=0).stop 2023-11-27T15:25:42.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.753+0000 7f92d93ff700 1 -- 172.21.15.133:0/4154164959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92d4105f80 msgr2=0x7f92d41a3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:42.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.753+0000 7f92d93ff700 1 --2- 172.21.15.133:0/4154164959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92d4105f80 0x7f92d41a3e60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00b6c0 tx=0x7f92cc004c60 comp rx=0 tx=0).stop 2023-11-27T15:25:42.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.754+0000 7f92d93ff700 1 -- 172.21.15.133:0/4154164959 shutdown_connections 2023-11-27T15:25:42.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.754+0000 7f92d93ff700 1 --2- 172.21.15.133:0/4154164959 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f92bc0a11d0 0x7f92bc0a3680 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.754+0000 7f92d93ff700 1 --2- 172.21.15.133:0/4154164959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f92d4105f80 0x7f92d41a3e60 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.754+0000 7f92d93ff700 1 --2- 172.21.15.133:0/4154164959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92d4100d40 0x7f92d4193ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.754+0000 7f92d93ff700 1 --2- 172.21.15.133:0/4154164959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92d4103660 0x7f92d4194420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:42.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.754+0000 7f92d93ff700 1 -- 172.21.15.133:0/4154164959 >> 172.21.15.133:0/4154164959 conn(0x7f92d40fa7c0 msgr2=0x7f92d40ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:42.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.754+0000 7f92d93ff700 1 -- 172.21.15.133:0/4154164959 shutdown_connections 2023-11-27T15:25:42.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:42.754+0000 7f92d93ff700 1 -- 172.21.15.133:0/4154164959 wait complete. 2023-11-27T15:25:43.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:42 smithi174 ceph-mon[131110]: pgmap v432: 129 pgs: 129 active+clean; 38 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 952 op/s 2023-11-27T15:25:43.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:42 smithi133 ceph-mon[97860]: pgmap v432: 129 pgs: 129 active+clean; 38 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 952 op/s 2023-11-27T15:25:43.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:42 smithi177 ceph-mon[131504]: pgmap v432: 129 pgs: 129 active+clean; 38 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 952 op/s 2023-11-27T15:25:43.783 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:25:43.783 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:25:44.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.063+0000 7f1063bff700 1 -- 172.21.15.133:0/2742172869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f105c103700 msgr2=0x7f105c103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:44.068 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.063+0000 7f1063bff700 1 --2- 172.21.15.133:0/2742172869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f105c103700 0x7f105c103b50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f104c009990 tx=0x7f104c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.064+0000 7f1063bff700 1 -- 172.21.15.133:0/2742172869 shutdown_connections 2023-11-27T15:25:44.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.064+0000 7f1063bff700 1 --2- 172.21.15.133:0/2742172869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f105c104090 0x7f105c10a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.064+0000 7f1063bff700 1 --2- 172.21.15.133:0/2742172869 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f105c102510 0x7f105c102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.064+0000 7f1063bff700 1 --2- 172.21.15.133:0/2742172869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f105c103700 0x7f105c103b50 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.064+0000 7f1063bff700 1 -- 172.21.15.133:0/2742172869 >> 172.21.15.133:0/2742172869 conn(0x7f105c0fdb30 msgr2=0x7f105c0fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:44.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.064+0000 7f1063bff700 1 -- 172.21.15.133:0/2742172869 shutdown_connections 2023-11-27T15:25:44.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.064+0000 7f1063bff700 1 -- 172.21.15.133:0/2742172869 wait complete. 2023-11-27T15:25:44.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.065+0000 7f1063bff700 1 Processor -- start 2023-11-27T15:25:44.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.065+0000 7f1063bff700 1 -- start start 2023-11-27T15:25:44.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.066+0000 7f1063bff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f105c102510 0x7f105c193e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.066+0000 7f1063bff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f105c103700 0x7f105c194360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.066+0000 7f1063bff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f105c104090 0x7f105c19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.066+0000 7f1063bff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c19b9a0 con 0x7f105c104090 2023-11-27T15:25:44.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.066+0000 7f1063bff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c19bae0 con 0x7f105c102510 2023-11-27T15:25:44.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.066+0000 7f1063bff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c19bc70 con 0x7f105c103700 2023-11-27T15:25:44.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.066+0000 7f106199b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f105c102510 0x7f105c193e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.066+0000 7f106199b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f105c102510 0x7f105c193e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:54540/0 (socket says 172.21.15.133:54540) 2023-11-27T15:25:44.071 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.066+0000 7f106199b700 1 -- 172.21.15.133:0/3471942331 learned_addr learned my addr 172.21.15.133:0/3471942331 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:44.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.067+0000 7f106199b700 1 -- 172.21.15.133:0/3471942331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f105c103700 msgr2=0x7f105c194360 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-27T15:25:44.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.067+0000 7f106199b700 1 --2- 172.21.15.133:0/3471942331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f105c103700 0x7f105c194360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.067+0000 7f106219c700 1 --2- 172.21.15.133:0/3471942331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f105c104090 0x7f105c19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.067+0000 7f106199b700 1 -- 172.21.15.133:0/3471942331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f105c104090 msgr2=0x7f105c19b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:44.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.067+0000 7f106199b700 1 --2- 172.21.15.133:0/3471942331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f105c104090 0x7f105c19b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.067+0000 7f106199b700 1 -- 172.21.15.133:0/3471942331 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104c0095f0 con 0x7f105c102510 2023-11-27T15:25:44.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.067+0000 7f106219c700 1 --2- 172.21.15.133:0/3471942331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f105c104090 0x7f105c19b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:44.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.067+0000 7f106199b700 1 --2- 172.21.15.133:0/3471942331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f105c102510 0x7f105c193e20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f105800e970 tx=0x7f105800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:44.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.068+0000 7f1052ffd700 1 -- 172.21.15.133:0/3471942331 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058009790 con 0x7f105c102510 2023-11-27T15:25:44.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.068+0000 7f1052ffd700 1 -- 172.21.15.133:0/3471942331 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10580044e0 con 0x7f105c102510 2023-11-27T15:25:44.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.068+0000 7f1052ffd700 1 -- 172.21.15.133:0/3471942331 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058003e30 con 0x7f105c102510 2023-11-27T15:25:44.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.068+0000 7f1063bff700 1 -- 172.21.15.133:0/3471942331 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105c077990 con 0x7f105c102510 2023-11-27T15:25:44.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.068+0000 7f1063bff700 1 -- 172.21.15.133:0/3471942331 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105c077800 con 0x7f105c102510 2023-11-27T15:25:44.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.073+0000 7f1052ffd700 1 -- 172.21.15.133:0/3471942331 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1058003f90 con 0x7f105c102510 2023-11-27T15:25:44.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.074+0000 7f107749db80 1 Processor -- start 2023-11-27T15:25:44.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.074+0000 7f107749db80 1 -- start start 2023-11-27T15:25:44.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.075+0000 7f107749db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cd9c80 0x55c5b5cda090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.075+0000 7f107749db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5b5cda5d0 0x55c5b5cdabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.075+0000 7f107749db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cdb190 0x55c5b5cdb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.075+0000 7f106219c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cd9c80 0x55c5b5cda090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.075+0000 7f106219c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cd9c80 0x55c5b5cda090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48212/0 (socket says 172.21.15.133:48212) 2023-11-27T15:25:44.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.075+0000 7f107749db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b5bdf0e0 con 0x55c5b5cda5d0 2023-11-27T15:25:44.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.075+0000 7f107749db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b5c31920 con 0x55c5b5cdb190 2023-11-27T15:25:44.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.075+0000 7f107749db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b5c2adb0 con 0x55c5b5cd9c80 2023-11-27T15:25:44.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.075+0000 7f106219c700 1 -- 172.21.15.133:0/1808016697 learned_addr learned my addr 172.21.15.133:0/1808016697 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:44.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.076+0000 7f106219c700 1 -- 172.21.15.133:0/1808016697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cdb190 msgr2=0x55c5b5cdb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:44.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.076+0000 7f106219c700 1 --2- 172.21.15.133:0/1808016697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cdb190 0x55c5b5cdb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.076+0000 7f106219c700 1 -- 172.21.15.133:0/1808016697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5b5cda5d0 msgr2=0x55c5b5cdabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:44.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.076+0000 7f106219c700 1 --2- 172.21.15.133:0/1808016697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5b5cda5d0 0x55c5b5cdabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.076+0000 7f106219c700 1 -- 172.21.15.133:0/1808016697 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5b5aac6b0 con 0x55c5b5cd9c80 2023-11-27T15:25:44.081 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.076+0000 7f106219c700 1 --2- 172.21.15.133:0/1808016697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cd9c80 0x55c5b5cda090 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1054009400 tx=0x7f1054012b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e88fc716cc874e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:44.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.077+0000 7f1052ffd700 1 --2- 172.21.15.133:0/3471942331 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f10480a1320 0x7f10480a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.077+0000 7f1052ffd700 1 -- 172.21.15.133:0/3471942331 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1058014070 con 0x7f105c102510 2023-11-27T15:25:44.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.077+0000 7f1047fff700 1 -- 172.21.15.133:0/1808016697 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10540079d0 con 0x55c5b5cd9c80 2023-11-27T15:25:44.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.077+0000 7f106199b700 1 --2- 172.21.15.133:0/3471942331 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f10480a1320 0x7f10480a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.077+0000 7f1047fff700 1 -- 172.21.15.133:0/1808016697 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1054019040 con 0x55c5b5cd9c80 2023-11-27T15:25:44.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.077+0000 7f1047fff700 1 -- 172.21.15.133:0/1808016697 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105401c390 con 0x55c5b5cd9c80 2023-11-27T15:25:44.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.078+0000 7f106199b700 1 --2- 172.21.15.133:0/3471942331 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f10480a1320 0x7f10480a37d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f10580a1e40 tx=0x7f1058095b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:44.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.079+0000 7f107749db80 1 -- 172.21.15.133:0/1808016697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cd9c80 msgr2=0x55c5b5cda090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:44.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.079+0000 7f107749db80 1 --2- 172.21.15.133:0/1808016697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cd9c80 0x55c5b5cda090 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1054009400 tx=0x7f1054012b30 comp rx=0 tx=0).stop 2023-11-27T15:25:44.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.079+0000 7f107749db80 1 -- 172.21.15.133:0/1808016697 shutdown_connections 2023-11-27T15:25:44.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.079+0000 7f107749db80 1 --2- 172.21.15.133:0/1808016697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cdb190 0x55c5b5cdb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.079+0000 7f107749db80 1 --2- 172.21.15.133:0/1808016697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cd9c80 0x55c5b5cda090 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.079+0000 7f107749db80 1 --2- 172.21.15.133:0/1808016697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5b5cda5d0 0x55c5b5cdabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.079+0000 7f107749db80 1 -- 172.21.15.133:0/1808016697 >> 172.21.15.133:0/1808016697 conn(0x55c5b5ccf3e0 msgr2=0x55c5b5cd46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:44.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.079+0000 7f107749db80 1 -- 172.21.15.133:0/1808016697 shutdown_connections 2023-11-27T15:25:44.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.079+0000 7f107749db80 1 -- 172.21.15.133:0/1808016697 wait complete. 2023-11-27T15:25:44.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.080+0000 7f107749db80 1 -- start start 2023-11-27T15:25:44.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.080+0000 7f107749db80 1 Processor -- start 2023-11-27T15:25:44.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.081+0000 7f107749db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5b5cd9c80 0x55c5b5ce3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.081+0000 7f107749db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cda5d0 0x55c5b5ce3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.081+0000 7f107749db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cdb190 0x55c5b5ceacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.081+0000 7f107749db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b5bcc960 con 0x55c5b5cd9c80 2023-11-27T15:25:44.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.081+0000 7f107749db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b5bcc600 con 0x55c5b5cdb190 2023-11-27T15:25:44.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.081+0000 7f107749db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b5aae9a0 con 0x55c5b5cda5d0 2023-11-27T15:25:44.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.081+0000 7f107749db80 10 client.0 _ll_register_callbacks cb 0x55c5b5bea800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:44.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.081+0000 7f106119a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cda5d0 0x55c5b5ce3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.081+0000 7f106119a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cda5d0 0x55c5b5ce3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48216/0 (socket says 172.21.15.133:48216) 2023-11-27T15:25:44.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.081+0000 7f106119a700 1 -- 172.21.15.133:0/2775000401 learned_addr learned my addr 172.21.15.133:0/2775000401 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:44.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.082+0000 7f106219c700 1 --2- 172.21.15.133:0/2775000401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5b5cd9c80 0x55c5b5ce3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.082+0000 7f106119a700 1 -- 172.21.15.133:0/2775000401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cdb190 msgr2=0x55c5b5ceacb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-27T15:25:44.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.082+0000 7f106119a700 1 --2- 172.21.15.133:0/2775000401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cdb190 0x55c5b5ceacb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.082+0000 7f106119a700 1 -- 172.21.15.133:0/2775000401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5b5cd9c80 msgr2=0x55c5b5ce3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:44.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.082+0000 7f106119a700 1 --2- 172.21.15.133:0/2775000401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5b5cd9c80 0x55c5b5ce3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.082+0000 7f106119a700 1 -- 172.21.15.133:0/2775000401 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10540090b0 con 0x55c5b5cda5d0 2023-11-27T15:25:44.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.082+0000 7f106119a700 1 --2- 172.21.15.133:0/2775000401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cda5d0 0x55c5b5ce3c70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f104c00b620 tx=0x7f104c00bad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc9c85f39dce69c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:44.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.082+0000 7f10457fa700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:25:44.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.083+0000 7f10457fa700 1 -- 172.21.15.133:0/2775000401 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c01d070 con 0x55c5b5cda5d0 2023-11-27T15:25:44.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.083+0000 7f10457fa700 1 -- 172.21.15.133:0/2775000401 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f104c00f670 con 0x55c5b5cda5d0 2023-11-27T15:25:44.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.083+0000 7f107749db80 1 -- 172.21.15.133:0/2775000401 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5b5cccba0 con 0x55c5b5cda5d0 2023-11-27T15:25:44.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.083+0000 7f10457fa700 1 -- 172.21.15.133:0/2775000401 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f104c00f830 con 0x55c5b5cda5d0 2023-11-27T15:25:44.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.083+0000 7f107749db80 10 client.26095 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:44.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.083+0000 7f107749db80 1 -- 172.21.15.133:0/2775000401 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c5b5bf1210 con 0x55c5b5cda5d0 2023-11-27T15:25:44.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.084+0000 7f10457fa700 1 -- 172.21.15.133:0/2775000401 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c5b5cccba0 con 0x55c5b5cda5d0 2023-11-27T15:25:44.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.084+0000 7f10457fa700 1 -- 172.21.15.133:0/2775000401 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 24) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f104c017b60 con 0x55c5b5cda5d0 2023-11-27T15:25:44.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.085+0000 7f107749db80 10 client.26095 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:44.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.085+0000 7f107749db80 10 client.26095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:44.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.085+0000 7f107749db80 20 client.26095 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:44.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.085+0000 7f107749db80 1 --2- 172.21.15.133:0/2775000401 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c5b5cef530 0x55c5b5cf1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.085+0000 7f107749db80 4 client.26095 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:44.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.085+0000 7f107749db80 1 -- 172.21.15.133:0/2775000401 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5b5bed120 con 0x55c5b5cef530 2023-11-27T15:25:44.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.085+0000 7f106219c700 1 --2- 172.21.15.133:0/2775000401 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c5b5cef530 0x55c5b5cf1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.086+0000 7f106219c700 1 --2- 172.21.15.133:0/2775000401 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c5b5cef530 0x55c5b5cf1940 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79ba7cbc1dbe4919 server_cookie=5138fcac23c667a1 in_seq=0 out_seq=0 2023-11-27T15:25:44.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.086+0000 7f10457fa700 10 client.26095 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:44.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.089+0000 7f10457fa700 1 -- 172.21.15.133:0/2775000401 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c5b5bed120 con 0x55c5b5cef530 2023-11-27T15:25:44.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.089+0000 7f10457fa700 10 client.26095 handle_command_reply: tid=0 2023-11-27T15:25:44.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.089+0000 7f107749db80 1 client.26095 shutdown 2023-11-27T15:25:44.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.090+0000 7f107749db80 1 -- 172.21.15.133:0/2775000401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cda5d0 msgr2=0x55c5b5ce3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:44.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.090+0000 7f107749db80 1 --2- 172.21.15.133:0/2775000401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cda5d0 0x55c5b5ce3c70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f104c00b620 tx=0x7f104c00bad0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.090+0000 7f107749db80 1 -- 172.21.15.133:0/2775000401 shutdown_connections 2023-11-27T15:25:44.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.090+0000 7f107749db80 1 --2- 172.21.15.133:0/2775000401 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c5b5cef530 0x55c5b5cf1940 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.090+0000 7f107749db80 1 --2- 172.21.15.133:0/2775000401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cdb190 0x55c5b5ceacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.090+0000 7f10457fa700 0 client.26095 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:44.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.090+0000 7f107749db80 1 --2- 172.21.15.133:0/2775000401 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5b5cd9c80 0x55c5b5ce3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.090+0000 7f107749db80 1 --2- 172.21.15.133:0/2775000401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cda5d0 0x55c5b5ce3c70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.090+0000 7f107749db80 1 -- 172.21.15.133:0/2775000401 >> 172.21.15.133:0/2775000401 conn(0x55c5b5ccf3e0 msgr2=0x55c5b5cd3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:44.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.090+0000 7f107749db80 1 -- 172.21.15.133:0/2775000401 shutdown_connections 2023-11-27T15:25:44.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.090+0000 7f107749db80 1 -- 172.21.15.133:0/2775000401 wait complete. 2023-11-27T15:25:44.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.091+0000 7f107749db80 20 client.26095 trim_cache size 0 max 16384 2023-11-27T15:25:44.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.116+0000 7f107749db80 1 Processor -- start 2023-11-27T15:25:44.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.116+0000 7f107749db80 1 -- start start 2023-11-27T15:25:44.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.116+0000 7f107749db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cd9c80 0x55c5b5cd35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.116+0000 7f107749db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5b5cda5d0 0x55c5b5ceac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.117+0000 7f107749db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cdb190 0x55c5b5cd3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.117+0000 7f107749db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b5cdba80 con 0x55c5b5cda5d0 2023-11-27T15:25:44.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.117+0000 7f107749db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b5ce4920 con 0x55c5b5cd9c80 2023-11-27T15:25:44.121 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.117+0000 7f107749db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b5ce2be0 con 0x55c5b5cdb190 2023-11-27T15:25:44.122 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.117+0000 7f106219c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cd9c80 0x55c5b5cd35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.122 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.117+0000 7f106219c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cd9c80 0x55c5b5cd35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:54566/0 (socket says 172.21.15.133:54566) 2023-11-27T15:25:44.122 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.117+0000 7f106219c700 1 -- 172.21.15.133:0/3816707372 learned_addr learned my addr 172.21.15.133:0/3816707372 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:44.122 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.117+0000 7f106219c700 1 --2- 172.21.15.133:0/3816707372 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cdb190 0x55c5b5cd3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.122 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.117+0000 7f106119a700 1 --2- 172.21.15.133:0/3816707372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5b5cda5d0 0x55c5b5ceac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.122 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.117+0000 7f106219c700 1 -- 172.21.15.133:0/3816707372 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cdb190 msgr2=0x55c5b5cd3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:44.122 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.117+0000 7f106219c700 1 --2- 172.21.15.133:0/3816707372 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cdb190 0x55c5b5cd3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.122 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.117+0000 7f106219c700 1 -- 172.21.15.133:0/3816707372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5b5cda5d0 msgr2=0x55c5b5ceac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:44.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.117+0000 7f106219c700 1 --2- 172.21.15.133:0/3816707372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5b5cda5d0 0x55c5b5ceac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.117+0000 7f106219c700 1 -- 172.21.15.133:0/3816707372 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5b5bf0f60 con 0x55c5b5cd9c80 2023-11-27T15:25:44.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.117+0000 7f106119a700 1 --2- 172.21.15.133:0/3816707372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5b5cda5d0 0x55c5b5ceac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:44.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.117+0000 7f106219c700 1 --2- 172.21.15.133:0/3816707372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cd9c80 0x55c5b5cd35d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f10540093a0 tx=0x7f1054012b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6e45a4d383b924a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:44.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.118+0000 7f1050bff700 1 -- 172.21.15.133:0/3816707372 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105401d430 con 0x55c5b5cd9c80 2023-11-27T15:25:44.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.118+0000 7f1050bff700 1 -- 172.21.15.133:0/3816707372 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1054012270 con 0x55c5b5cd9c80 2023-11-27T15:25:44.123 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.118+0000 7f1050bff700 1 -- 172.21.15.133:0/3816707372 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105402b2f0 con 0x55c5b5cd9c80 2023-11-27T15:25:44.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.119+0000 7f107749db80 1 -- 172.21.15.133:0/3816707372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cd9c80 msgr2=0x55c5b5cd35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:44.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.119+0000 7f107749db80 1 --2- 172.21.15.133:0/3816707372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cd9c80 0x55c5b5cd35d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f10540093a0 tx=0x7f1054012b30 comp rx=0 tx=0).stop 2023-11-27T15:25:44.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.120+0000 7f107749db80 1 -- 172.21.15.133:0/3816707372 shutdown_connections 2023-11-27T15:25:44.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.120+0000 7f107749db80 1 --2- 172.21.15.133:0/3816707372 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cdb190 0x55c5b5cd3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.124 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.120+0000 7f107749db80 1 --2- 172.21.15.133:0/3816707372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cd9c80 0x55c5b5cd35d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.120+0000 7f107749db80 1 --2- 172.21.15.133:0/3816707372 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5b5cda5d0 0x55c5b5ceac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.120+0000 7f107749db80 1 -- 172.21.15.133:0/3816707372 >> 172.21.15.133:0/3816707372 conn(0x55c5b5ccf3e0 msgr2=0x55c5b5ce36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:44.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.120+0000 7f107749db80 1 -- 172.21.15.133:0/3816707372 shutdown_connections 2023-11-27T15:25:44.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.120+0000 7f107749db80 1 -- 172.21.15.133:0/3816707372 wait complete. 2023-11-27T15:25:44.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.121+0000 7f107749db80 1 -- start start 2023-11-27T15:25:44.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.121+0000 7f107749db80 1 Processor -- start 2023-11-27T15:25:44.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.122+0000 7f107749db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cd9c80 0x55c5b5ceaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.122+0000 7f107749db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5b5cda5d0 0x55c5b5ce90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.122+0000 7f107749db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cdb190 0x55c5b5ce9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.122+0000 7f107749db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b5ce9ba0 con 0x55c5b5cda5d0 2023-11-27T15:25:44.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.122+0000 7f107749db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b5ce9d10 con 0x55c5b5cd9c80 2023-11-27T15:25:44.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.122+0000 7f107749db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5b5ce9ed0 con 0x55c5b5cdb190 2023-11-27T15:25:44.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.122+0000 7f107749db80 10 client.0 _ll_register_callbacks cb 0x7f10540097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:44.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.122+0000 7f106219c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cd9c80 0x55c5b5ceaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.122+0000 7f106219c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cd9c80 0x55c5b5ceaad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:54574/0 (socket says 172.21.15.133:54574) 2023-11-27T15:25:44.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.122+0000 7f106219c700 1 -- 172.21.15.133:0/976103017 learned_addr learned my addr 172.21.15.133:0/976103017 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:44.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.122+0000 7f106219c700 1 --2- 172.21.15.133:0/976103017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cdb190 0x55c5b5ce9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.122+0000 7f106119a700 1 --2- 172.21.15.133:0/976103017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5b5cda5d0 0x55c5b5ce90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.123+0000 7f106219c700 1 -- 172.21.15.133:0/976103017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cdb190 msgr2=0x55c5b5ce9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:44.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.123+0000 7f106219c700 1 --2- 172.21.15.133:0/976103017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cdb190 0x55c5b5ce9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.128 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.123+0000 7f106219c700 1 -- 172.21.15.133:0/976103017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5b5cda5d0 msgr2=0x55c5b5ce90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:44.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.123+0000 7f106219c700 1 --2- 172.21.15.133:0/976103017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5b5cda5d0 0x55c5b5ce90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.123+0000 7f106219c700 1 -- 172.21.15.133:0/976103017 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5b5aac6b0 con 0x55c5b5cd9c80 2023-11-27T15:25:44.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.123+0000 7f106219c700 1 --2- 172.21.15.133:0/976103017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cd9c80 0x55c5b5ceaad0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1054015040 tx=0x7f1054009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d091fce2feb53a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:44.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.123+0000 7f10457fa700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:25:44.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.123+0000 7f10457fa700 1 -- 172.21.15.133:0/976103017 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105402bb20 con 0x55c5b5cd9c80 2023-11-27T15:25:44.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.123+0000 7f10457fa700 1 -- 172.21.15.133:0/976103017 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f105402bc80 con 0x55c5b5cd9c80 2023-11-27T15:25:44.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.123+0000 7f107749db80 1 -- 172.21.15.133:0/976103017 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1054019040 con 0x55c5b5cd9c80 2023-11-27T15:25:44.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.124+0000 7f10457fa700 1 -- 172.21.15.133:0/976103017 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f105402be40 con 0x55c5b5cd9c80 2023-11-27T15:25:44.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.124+0000 7f107749db80 10 client.26120 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:44.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.124+0000 7f107749db80 1 -- 172.21.15.133:0/976103017 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c5b5a41ce0 con 0x55c5b5cd9c80 2023-11-27T15:25:44.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.124+0000 7f10457fa700 1 -- 172.21.15.133:0/976103017 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1054019040 con 0x55c5b5cd9c80 2023-11-27T15:25:44.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.124+0000 7f10457fa700 1 -- 172.21.15.133:0/976103017 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 24) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f10540078f0 con 0x55c5b5cd9c80 2023-11-27T15:25:44.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.124+0000 7f107749db80 10 client.26120 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:44.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.124+0000 7f107749db80 10 client.26120 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:44.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.124+0000 7f107749db80 20 client.26120 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:44.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.124+0000 7f107749db80 1 --2- 172.21.15.133:0/976103017 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c5b5d556e0 0x55c5b5d57af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.124+0000 7f107749db80 4 client.26120 mds_command: new command op to 24560 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:25:44.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.124+0000 7f107749db80 1 -- 172.21.15.133:0/976103017 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c5b5bf96e0 con 0x55c5b5d556e0 2023-11-27T15:25:44.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.125+0000 7f106119a700 1 --2- 172.21.15.133:0/976103017 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c5b5d556e0 0x55c5b5d57af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.125+0000 7f106119a700 1 --2- 172.21.15.133:0/976103017 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c5b5d556e0 0x55c5b5d57af0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39233dcf8cd1ca63 server_cookie=d462286b9f1a10 in_seq=0 out_seq=0 2023-11-27T15:25:44.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.125+0000 7f10457fa700 10 client.26120 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:44.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.126+0000 7f10457fa700 1 -- 172.21.15.133:0/976103017 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c5b5bf96e0 con 0x55c5b5d556e0 2023-11-27T15:25:44.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.126+0000 7f10457fa700 10 client.26120 handle_command_reply: tid=0 2023-11-27T15:25:44.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.126+0000 7f107749db80 1 client.26120 shutdown 2023-11-27T15:25:44.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.127+0000 7f107749db80 1 -- 172.21.15.133:0/976103017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cd9c80 msgr2=0x55c5b5ceaad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:44.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.127+0000 7f107749db80 1 --2- 172.21.15.133:0/976103017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cd9c80 0x55c5b5ceaad0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1054015040 tx=0x7f1054009920 comp rx=0 tx=0).stop 2023-11-27T15:25:44.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.127+0000 7f107749db80 1 -- 172.21.15.133:0/976103017 shutdown_connections 2023-11-27T15:25:44.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.127+0000 7f107749db80 1 --2- 172.21.15.133:0/976103017 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c5b5d556e0 0x55c5b5d57af0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.127+0000 7f107749db80 1 --2- 172.21.15.133:0/976103017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c5b5cdb190 0x55c5b5ce9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.127+0000 7f107749db80 1 --2- 172.21.15.133:0/976103017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5b5cd9c80 0x55c5b5ceaad0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.127+0000 7f10457fa700 0 client.26120 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:44.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.127+0000 7f107749db80 1 --2- 172.21.15.133:0/976103017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5b5cda5d0 0x55c5b5ce90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.127+0000 7f107749db80 1 -- 172.21.15.133:0/976103017 >> 172.21.15.133:0/976103017 conn(0x55c5b5ccf3e0 msgr2=0x55c5b5cd89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:44.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.127+0000 7f107749db80 1 -- 172.21.15.133:0/976103017 shutdown_connections 2023-11-27T15:25:44.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.127+0000 7f107749db80 1 -- 172.21.15.133:0/976103017 wait complete. 2023-11-27T15:25:44.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.127+0000 7f107749db80 20 client.26120 trim_cache size 0 max 16384 2023-11-27T15:25:44.133 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:25:44.133 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2023-11-27T15:25:44.133 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "b6638f49-a408-46a9-83f0-3995bb2029f5", 2023-11-27T15:25:44.133 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2023-11-27T15:25:44.133 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:25:44.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.128+0000 7f1063bff700 1 -- 172.21.15.133:0/3471942331 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f10480a1320 msgr2=0x7f10480a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:44.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.128+0000 7f1063bff700 1 --2- 172.21.15.133:0/3471942331 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f10480a1320 0x7f10480a37d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f10580a1e40 tx=0x7f1058095b40 comp rx=0 tx=0).stop 2023-11-27T15:25:44.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.128+0000 7f1063bff700 1 -- 172.21.15.133:0/3471942331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f105c102510 msgr2=0x7f105c193e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:44.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.128+0000 7f1063bff700 1 --2- 172.21.15.133:0/3471942331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f105c102510 0x7f105c193e20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f105800e970 tx=0x7f105800ed30 comp rx=0 tx=0).stop 2023-11-27T15:25:44.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.128+0000 7f1063bff700 1 -- 172.21.15.133:0/3471942331 shutdown_connections 2023-11-27T15:25:44.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.128+0000 7f1063bff700 1 --2- 172.21.15.133:0/3471942331 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f10480a1320 0x7f10480a37d0 secure :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f10580a1e40 tx=0x7f1058095b40 comp rx=0 tx=0).stop 2023-11-27T15:25:44.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.128+0000 7f1063bff700 1 --2- 172.21.15.133:0/3471942331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f105c104090 0x7f105c19b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.128+0000 7f1063bff700 1 --2- 172.21.15.133:0/3471942331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f105c102510 0x7f105c193e20 secure :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f105800e970 tx=0x7f105800ed30 comp rx=0 tx=0).stop 2023-11-27T15:25:44.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.128+0000 7f1063bff700 1 --2- 172.21.15.133:0/3471942331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f105c103700 0x7f105c194360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.128+0000 7f1063bff700 1 -- 172.21.15.133:0/3471942331 >> 172.21.15.133:0/3471942331 conn(0x7f105c0fdb30 msgr2=0x7f105c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:44.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.129+0000 7f1063bff700 1 -- 172.21.15.133:0/3471942331 shutdown_connections 2023-11-27T15:25:44.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.129+0000 7f1063bff700 1 -- 172.21.15.133:0/3471942331 wait complete. 2023-11-27T15:25:44.156 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:25:44.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.403+0000 7f16a7dff700 1 -- 172.21.15.133:0/795327567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16a0101e50 msgr2=0x7f16a0108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:44.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.403+0000 7f16a7dff700 1 --2- 172.21.15.133:0/795327567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16a0101e50 0x7f16a0108710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f169800b0a0 tx=0x7f169800b3b0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.403+0000 7f16a7dff700 1 -- 172.21.15.133:0/795327567 shutdown_connections 2023-11-27T15:25:44.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.403+0000 7f16a7dff700 1 --2- 172.21.15.133:0/795327567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16a0101e50 0x7f16a0108710 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.403+0000 7f16a7dff700 1 --2- 172.21.15.133:0/795327567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a01002d0 0x7f16a01006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.403+0000 7f16a7dff700 1 --2- 172.21.15.133:0/795327567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16a01014c0 0x7f16a0101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.408 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.403+0000 7f16a7dff700 1 -- 172.21.15.133:0/795327567 >> 172.21.15.133:0/795327567 conn(0x7f16a00fb8b0 msgr2=0x7f16a00fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:44.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.404+0000 7f16a7dff700 1 -- 172.21.15.133:0/795327567 shutdown_connections 2023-11-27T15:25:44.409 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.404+0000 7f16a7dff700 1 -- 172.21.15.133:0/795327567 wait complete. 2023-11-27T15:25:44.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.406+0000 7f16a7dff700 1 Processor -- start 2023-11-27T15:25:44.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.406+0000 7f16a7dff700 1 -- start start 2023-11-27T15:25:44.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.407+0000 7f16a7dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a01002d0 0x7f16a0198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.407+0000 7f16a7dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16a01014c0 0x7f16a01985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.407+0000 7f16a7dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16a0101e50 0x7f16a019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.407+0000 7f16a7dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a019fc10 con 0x7f16a0101e50 2023-11-27T15:25:44.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.407+0000 7f16a7dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a019fd50 con 0x7f16a01002d0 2023-11-27T15:25:44.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.407+0000 7f16a7dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a019fee0 con 0x7f16a01014c0 2023-11-27T15:25:44.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.407+0000 7f16a5b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a01002d0 0x7f16a0198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.407+0000 7f16a5b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a01002d0 0x7f16a0198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:54596/0 (socket says 172.21.15.133:54596) 2023-11-27T15:25:44.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.407+0000 7f16a5b9b700 1 -- 172.21.15.133:0/385574340 learned_addr learned my addr 172.21.15.133:0/385574340 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:44.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.407+0000 7f16a539a700 1 --2- 172.21.15.133:0/385574340 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16a01014c0 0x7f16a01985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.408+0000 7f16a5b9b700 1 -- 172.21.15.133:0/385574340 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16a01014c0 msgr2=0x7f16a01985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:44.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.408+0000 7f16a5b9b700 1 --2- 172.21.15.133:0/385574340 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16a01014c0 0x7f16a01985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.408+0000 7f16a5b9b700 1 -- 172.21.15.133:0/385574340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16a0101e50 msgr2=0x7f16a019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:44.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.408+0000 7f16a5b9b700 1 --2- 172.21.15.133:0/385574340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16a0101e50 0x7f16a019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.408+0000 7f16a5b9b700 1 -- 172.21.15.133:0/385574340 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1698009d00 con 0x7f16a01002d0 2023-11-27T15:25:44.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.408+0000 7f16a5b9b700 1 --2- 172.21.15.133:0/385574340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a01002d0 0x7f16a0198090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f169c00e970 tx=0x7f169c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:44.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.408+0000 7f1696ffd700 1 -- 172.21.15.133:0/385574340 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c009790 con 0x7f16a01002d0 2023-11-27T15:25:44.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.409+0000 7f1696ffd700 1 -- 172.21.15.133:0/385574340 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f169c00eec0 con 0x7f16a01002d0 2023-11-27T15:25:44.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.409+0000 7f1696ffd700 1 -- 172.21.15.133:0/385574340 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c003d60 con 0x7f16a01002d0 2023-11-27T15:25:44.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.409+0000 7f16a7dff700 1 -- 172.21.15.133:0/385574340 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a010bb80 con 0x7f16a01002d0 2023-11-27T15:25:44.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.409+0000 7f16a7dff700 1 -- 172.21.15.133:0/385574340 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a010b9f0 con 0x7f16a01002d0 2023-11-27T15:25:44.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.413+0000 7f1696ffd700 1 -- 172.21.15.133:0/385574340 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f169c003ec0 con 0x7f16a01002d0 2023-11-27T15:25:44.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.415+0000 7f16bb6ecb80 1 Processor -- start 2023-11-27T15:25:44.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.415+0000 7f16bb6ecb80 1 -- start start 2023-11-27T15:25:44.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.415+0000 7f16bb6ecb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09bc00 0x5632dc09c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.422 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.416+0000 7f16bb6ecb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632dc09c550 0x5632dc09cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.422 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.416+0000 7f16bb6ecb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09d110 0x5632dc09d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.422 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.416+0000 7f16bb6ecb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632dbfa1060 con 0x5632dc09bc00 2023-11-27T15:25:44.422 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.416+0000 7f16bb6ecb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632dbff38a0 con 0x5632dc09d110 2023-11-27T15:25:44.422 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.416+0000 7f16bb6ecb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632dbfecd30 con 0x5632dc09c550 2023-11-27T15:25:44.422 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.416+0000 7f16a539a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09d110 0x5632dc09d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.416+0000 7f16a539a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09d110 0x5632dc09d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:54602/0 (socket says 172.21.15.133:54602) 2023-11-27T15:25:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.416+0000 7f16a539a700 1 -- 172.21.15.133:0/4231022045 learned_addr learned my addr 172.21.15.133:0/4231022045 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.416+0000 7f16a5b9b700 1 --2- 172.21.15.133:0/4231022045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632dc09c550 0x5632dc09cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.416+0000 7f16a639c700 1 --2- 172.21.15.133:0/4231022045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09bc00 0x5632dc09c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.416+0000 7f16a539a700 1 -- 172.21.15.133:0/4231022045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632dc09c550 msgr2=0x5632dc09cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.416+0000 7f16a539a700 1 --2- 172.21.15.133:0/4231022045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632dc09c550 0x5632dc09cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.416+0000 7f16a539a700 1 -- 172.21.15.133:0/4231022045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09bc00 msgr2=0x5632dc09c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.416+0000 7f16a539a700 1 --2- 172.21.15.133:0/4231022045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09bc00 0x5632dc09c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.416+0000 7f16a539a700 1 -- 172.21.15.133:0/4231022045 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632dbe6e630 con 0x5632dc09d110 2023-11-27T15:25:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.416+0000 7f1696ffd700 1 --2- 172.21.15.133:0/385574340 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f168c0a11d0 0x7f168c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.417+0000 7f16a639c700 1 --2- 172.21.15.133:0/4231022045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09bc00 0x5632dc09c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:25:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.417+0000 7f1696ffd700 1 -- 172.21.15.133:0/385574340 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f169c014070 con 0x7f16a01002d0 2023-11-27T15:25:44.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.417+0000 7f16a639c700 1 --2- 172.21.15.133:0/385574340 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f168c0a11d0 0x7f168c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.417+0000 7f16a539a700 1 --2- 172.21.15.133:0/4231022045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09d110 0x5632dc09d580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f169000ad70 tx=0x7f169000f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46dddf18854d55df server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:44.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.418+0000 7f168bfff700 1 -- 172.21.15.133:0/4231022045 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169000a870 con 0x5632dc09d110 2023-11-27T15:25:44.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.418+0000 7f16a639c700 1 --2- 172.21.15.133:0/385574340 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f168c0a11d0 0x7f168c0a3680 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1698009610 tx=0x7f1698006210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:44.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.418+0000 7f168bfff700 1 -- 172.21.15.133:0/4231022045 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f169000fec0 con 0x5632dc09d110 2023-11-27T15:25:44.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.418+0000 7f168bfff700 1 -- 172.21.15.133:0/4231022045 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169000a870 con 0x5632dc09d110 2023-11-27T15:25:44.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.419+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/4231022045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09d110 msgr2=0x5632dc09d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:44.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.419+0000 7f16bb6ecb80 1 --2- 172.21.15.133:0/4231022045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09d110 0x5632dc09d580 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f169000ad70 tx=0x7f169000f6a0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.419+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/4231022045 shutdown_connections 2023-11-27T15:25:44.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.419+0000 7f16bb6ecb80 1 --2- 172.21.15.133:0/4231022045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09d110 0x5632dc09d580 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.420+0000 7f16bb6ecb80 1 --2- 172.21.15.133:0/4231022045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09bc00 0x5632dc09c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.420+0000 7f16bb6ecb80 1 --2- 172.21.15.133:0/4231022045 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632dc09c550 0x5632dc09cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.420+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/4231022045 >> 172.21.15.133:0/4231022045 conn(0x5632dc091360 msgr2=0x5632dc096670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:44.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.420+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/4231022045 shutdown_connections 2023-11-27T15:25:44.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.420+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/4231022045 wait complete. 2023-11-27T15:25:44.426 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.421+0000 7f16bb6ecb80 1 -- start start 2023-11-27T15:25:44.426 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.421+0000 7f16bb6ecb80 1 Processor -- start 2023-11-27T15:25:44.426 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.422+0000 7f16bb6ecb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09bc00 0x5632dc0ade00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.427 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.422+0000 7f16bb6ecb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632dc09c550 0x5632dc0ae340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.427 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.422+0000 7f16bb6ecb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09d110 0x5632dc0b5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.427 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.422+0000 7f16bb6ecb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632dbf8e8e0 con 0x5632dc09bc00 2023-11-27T15:25:44.427 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.422+0000 7f16bb6ecb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632dbf8e580 con 0x5632dc09d110 2023-11-27T15:25:44.427 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.422+0000 7f16bb6ecb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632dbe70920 con 0x5632dc09c550 2023-11-27T15:25:44.427 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.422+0000 7f16bb6ecb80 10 client.0 _ll_register_callbacks cb 0x5632dbfac780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:44.427 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.422+0000 7f16a5b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09bc00 0x5632dc0ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.427 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.422+0000 7f16a5b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09bc00 0x5632dc0ade00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60646/0 (socket says 172.21.15.133:60646) 2023-11-27T15:25:44.427 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.422+0000 7f16a5b9b700 1 -- 172.21.15.133:0/3181863808 learned_addr learned my addr 172.21.15.133:0/3181863808 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:44.428 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.422+0000 7f16a539a700 1 --2- 172.21.15.133:0/3181863808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632dc09c550 0x5632dc0ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.422+0000 7f16a639c700 1 --2- 172.21.15.133:0/3181863808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09d110 0x5632dc0b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.423+0000 7f16a639c700 1 -- 172.21.15.133:0/3181863808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632dc09c550 msgr2=0x5632dc0ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.423+0000 7f16a639c700 1 --2- 172.21.15.133:0/3181863808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632dc09c550 0x5632dc0ae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.423+0000 7f16a639c700 1 -- 172.21.15.133:0/3181863808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09bc00 msgr2=0x5632dc0ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.423+0000 7f16a639c700 1 --2- 172.21.15.133:0/3181863808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09bc00 0x5632dc0ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.423+0000 7f16a639c700 1 -- 172.21.15.133:0/3181863808 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169000a9d0 con 0x5632dc09d110 2023-11-27T15:25:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.423+0000 7f16a5b9b700 1 --2- 172.21.15.133:0/3181863808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09bc00 0x5632dc0ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:25:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.423+0000 7f16a639c700 1 --2- 172.21.15.133:0/3181863808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09d110 0x5632dc0b5380 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f16980062a0 tx=0x7f1698006000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea6d321ca438d456 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.423+0000 7f16897fa700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:25:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.423+0000 7f16897fa700 1 -- 172.21.15.133:0/3181863808 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169800e040 con 0x5632dc09d110 2023-11-27T15:25:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.423+0000 7f16897fa700 1 -- 172.21.15.133:0/3181863808 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1698022040 con 0x5632dc09d110 2023-11-27T15:25:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.424+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/3181863808 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632dbfaf0a0 con 0x5632dc09d110 2023-11-27T15:25:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.424+0000 7f16897fa700 1 -- 172.21.15.133:0/3181863808 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f169801d860 con 0x5632dc09d110 2023-11-27T15:25:44.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.424+0000 7f16bb6ecb80 10 client.26132 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:44.430 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.424+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/3181863808 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5632dbfb3190 con 0x5632dc09d110 2023-11-27T15:25:44.430 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.425+0000 7f16897fa700 1 -- 172.21.15.133:0/3181863808 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5632dbfaf0a0 con 0x5632dc09d110 2023-11-27T15:25:44.430 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.425+0000 7f16897fa700 1 -- 172.21.15.133:0/3181863808 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 24) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f169801ba20 con 0x5632dc09d110 2023-11-27T15:25:44.430 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.425+0000 7f16bb6ecb80 10 client.26132 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:44.430 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.426+0000 7f16bb6ecb80 10 client.26132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:44.431 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.426+0000 7f16bb6ecb80 20 client.26132 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:44.431 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.426+0000 7f16bb6ecb80 1 --2- 172.21.15.133:0/3181863808 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5632dc0b9ad0 0x5632dc0bbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.431 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.426+0000 7f16bb6ecb80 4 client.26132 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:44.431 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.426+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/3181863808 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632dbfbb660 con 0x5632dc0b9ad0 2023-11-27T15:25:44.431 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.426+0000 7f16a5b9b700 1 --2- 172.21.15.133:0/3181863808 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5632dc0b9ad0 0x5632dc0bbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.432 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.426+0000 7f16a5b9b700 1 --2- 172.21.15.133:0/3181863808 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5632dc0b9ad0 0x5632dc0bbee0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7363ced16db0bbaa server_cookie=ca3672ae506f9034 in_seq=0 out_seq=0 2023-11-27T15:25:44.432 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.427+0000 7f16897fa700 10 client.26132 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:44.432 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.428+0000 7f16897fa700 1 -- 172.21.15.133:0/3181863808 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5632dbfbb660 con 0x5632dc0b9ad0 2023-11-27T15:25:44.432 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.428+0000 7f16897fa700 10 client.26132 handle_command_reply: tid=0 2023-11-27T15:25:44.432 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.428+0000 7f16bb6ecb80 1 client.26132 shutdown 2023-11-27T15:25:44.433 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.428+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/3181863808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09d110 msgr2=0x5632dc0b5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:44.433 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.428+0000 7f16bb6ecb80 1 --2- 172.21.15.133:0/3181863808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09d110 0x5632dc0b5380 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f16980062a0 tx=0x7f1698006000 comp rx=0 tx=0).stop 2023-11-27T15:25:44.433 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.428+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/3181863808 shutdown_connections 2023-11-27T15:25:44.433 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.428+0000 7f16bb6ecb80 1 --2- 172.21.15.133:0/3181863808 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5632dc0b9ad0 0x5632dc0bbee0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.433 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.428+0000 7f16bb6ecb80 1 --2- 172.21.15.133:0/3181863808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09d110 0x5632dc0b5380 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.433 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.428+0000 7f16bb6ecb80 1 --2- 172.21.15.133:0/3181863808 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09bc00 0x5632dc0ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.428+0000 7f16897fa700 0 client.26132 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:44.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.428+0000 7f16bb6ecb80 1 --2- 172.21.15.133:0/3181863808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632dc09c550 0x5632dc0ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.429+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/3181863808 >> 172.21.15.133:0/3181863808 conn(0x5632dc091360 msgr2=0x5632dc095de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:44.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.429+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/3181863808 shutdown_connections 2023-11-27T15:25:44.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.429+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/3181863808 wait complete. 2023-11-27T15:25:44.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.429+0000 7f16bb6ecb80 20 client.26132 trim_cache size 0 max 16384 2023-11-27T15:25:44.463 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.459+0000 7f16bb6ecb80 1 Processor -- start 2023-11-27T15:25:44.463 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.459+0000 7f16bb6ecb80 1 -- start start 2023-11-27T15:25:44.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.459+0000 7f16bb6ecb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09bc00 0x5632dc0c0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.459+0000 7f16bb6ecb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09c550 0x5632dc0b5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.459+0000 7f16bb6ecb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632dc09d110 0x5632dc0b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.459+0000 7f16bb6ecb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632dc0a4ae0 con 0x5632dc09c550 2023-11-27T15:25:44.464 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.459+0000 7f16bb6ecb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632dc0aee40 con 0x5632dc09bc00 2023-11-27T15:25:44.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.459+0000 7f16bb6ecb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632dc097ed0 con 0x5632dc09d110 2023-11-27T15:25:44.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.460+0000 7f16a5b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09bc00 0x5632dc0c0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.460+0000 7f16a639c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632dc09d110 0x5632dc0b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.460+0000 7f16a5b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09bc00 0x5632dc0c0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:54622/0 (socket says 172.21.15.133:54622) 2023-11-27T15:25:44.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.460+0000 7f16a5b9b700 1 -- 172.21.15.133:0/1197736196 learned_addr learned my addr 172.21.15.133:0/1197736196 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:44.465 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.460+0000 7f16a639c700 1 -- 172.21.15.133:0/1197736196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09bc00 msgr2=0x5632dc0c0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:44.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.460+0000 7f16a639c700 1 --2- 172.21.15.133:0/1197736196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09bc00 0x5632dc0c0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.460+0000 7f16a639c700 1 -- 172.21.15.133:0/1197736196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09c550 msgr2=0x5632dc0b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:44.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.460+0000 7f16a639c700 1 --2- 172.21.15.133:0/1197736196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09c550 0x5632dc0b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.460+0000 7f16a639c700 1 -- 172.21.15.133:0/1197736196 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632dbfb2ee0 con 0x5632dc09d110 2023-11-27T15:25:44.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.460+0000 7f16a639c700 1 --2- 172.21.15.133:0/1197736196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632dc09d110 0x5632dc0b5c70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1698007ab0 tx=0x7f169801cfb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2a9c70dc5f31ca0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:44.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.461+0000 7f1694bff700 1 -- 172.21.15.133:0/1197736196 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169800e040 con 0x5632dc09d110 2023-11-27T15:25:44.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.461+0000 7f1694bff700 1 -- 172.21.15.133:0/1197736196 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1698027540 con 0x5632dc09d110 2023-11-27T15:25:44.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.461+0000 7f1694bff700 1 -- 172.21.15.133:0/1197736196 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169801bad0 con 0x5632dc09d110 2023-11-27T15:25:44.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.462+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/1197736196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632dc09d110 msgr2=0x5632dc0b5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:44.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.462+0000 7f16bb6ecb80 1 --2- 172.21.15.133:0/1197736196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632dc09d110 0x5632dc0b5c70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1698007ab0 tx=0x7f169801cfb0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.462+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/1197736196 shutdown_connections 2023-11-27T15:25:44.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.462+0000 7f16bb6ecb80 1 --2- 172.21.15.133:0/1197736196 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632dc09d110 0x5632dc0b5c70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.462+0000 7f16bb6ecb80 1 --2- 172.21.15.133:0/1197736196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09bc00 0x5632dc0c0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.462+0000 7f16bb6ecb80 1 --2- 172.21.15.133:0/1197736196 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09c550 0x5632dc0b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.462+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/1197736196 >> 172.21.15.133:0/1197736196 conn(0x5632dc091360 msgr2=0x5632dc096640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:44.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.462+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/1197736196 shutdown_connections 2023-11-27T15:25:44.467 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.462+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/1197736196 wait complete. 2023-11-27T15:25:44.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.463+0000 7f16bb6ecb80 1 -- start start 2023-11-27T15:25:44.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.463+0000 7f16bb6ecb80 1 Processor -- start 2023-11-27T15:25:44.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.464+0000 7f16bb6ecb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09bc00 0x5632dc0b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.464+0000 7f16bb6ecb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632dc09c550 0x5632dc0b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.464+0000 7f16bb6ecb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09d110 0x5632dc0b4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.464+0000 7f16bb6ecb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632dc0b4780 con 0x5632dc09d110 2023-11-27T15:25:44.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.464+0000 7f16bb6ecb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632dc0b48f0 con 0x5632dc09bc00 2023-11-27T15:25:44.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.464+0000 7f16bb6ecb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632dc0b4ab0 con 0x5632dc09c550 2023-11-27T15:25:44.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.464+0000 7f16bb6ecb80 10 client.0 _ll_register_callbacks cb 0x7f16980077f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:44.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.464+0000 7f16a5b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09bc00 0x5632dc0b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.464+0000 7f16a5b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09bc00 0x5632dc0b0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:54628/0 (socket says 172.21.15.133:54628) 2023-11-27T15:25:44.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.464+0000 7f16a5b9b700 1 -- 172.21.15.133:0/1609757051 learned_addr learned my addr 172.21.15.133:0/1609757051 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:44.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.464+0000 7f16a639c700 1 --2- 172.21.15.133:0/1609757051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09d110 0x5632dc0b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.470 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.465+0000 7f16a539a700 1 --2- 172.21.15.133:0/1609757051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632dc09c550 0x5632dc0b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.465+0000 7f16a5b9b700 1 -- 172.21.15.133:0/1609757051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632dc09c550 msgr2=0x5632dc0b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:44.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.465+0000 7f16a5b9b700 1 --2- 172.21.15.133:0/1609757051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632dc09c550 0x5632dc0b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.465+0000 7f16a5b9b700 1 -- 172.21.15.133:0/1609757051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09d110 msgr2=0x5632dc0b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:44.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.465+0000 7f16a5b9b700 1 --2- 172.21.15.133:0/1609757051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09d110 0x5632dc0b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.465+0000 7f16a5b9b700 1 -- 172.21.15.133:0/1609757051 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632dbfb3190 con 0x5632dc09bc00 2023-11-27T15:25:44.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.465+0000 7f16a639c700 1 --2- 172.21.15.133:0/1609757051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09d110 0x5632dc0b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:25:44.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.465+0000 7f16a5b9b700 1 --2- 172.21.15.133:0/1609757051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09bc00 0x5632dc0b0ce0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f169c096960 tx=0x7f169c099fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fb65687f8fcfd7e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:44.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.465+0000 7f16897fa700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:25:44.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.465+0000 7f16897fa700 1 -- 172.21.15.133:0/1609757051 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c0a2340 con 0x5632dc09bc00 2023-11-27T15:25:44.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.465+0000 7f16897fa700 1 -- 172.21.15.133:0/1609757051 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f169c0ca6f0 con 0x5632dc09bc00 2023-11-27T15:25:44.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.466+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/1609757051 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632dc08eb20 con 0x5632dc09bc00 2023-11-27T15:25:44.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.466+0000 7f16897fa700 1 -- 172.21.15.133:0/1609757051 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f169c00ccd0 con 0x5632dc09bc00 2023-11-27T15:25:44.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.467+0000 7f16bb6ecb80 10 client.26138 fetch_fsmap learned FSMap version 24 2023-11-27T15:25:44.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.467+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/1609757051 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5632dbe03c60 con 0x5632dc09bc00 2023-11-27T15:25:44.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.467+0000 7f16897fa700 1 -- 172.21.15.133:0/1609757051 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5632dc08eb20 con 0x5632dc09bc00 2023-11-27T15:25:44.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.467+0000 7f16897fa700 1 -- 172.21.15.133:0/1609757051 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 24) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f169c0a1eb0 con 0x5632dc09bc00 2023-11-27T15:25:44.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.468+0000 7f16bb6ecb80 10 client.26138 fetch_fsmap finished waiting for FSMap version 24 2023-11-27T15:25:44.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.468+0000 7f16bb6ecb80 10 client.26138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:44.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.468+0000 7f16bb6ecb80 20 client.26138 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:44.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.468+0000 7f16bb6ecb80 1 --2- 172.21.15.133:0/1609757051 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5632dc10e6c0 0x5632dc110ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:44.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.468+0000 7f16bb6ecb80 4 client.26138 mds_command: new command op to 24560 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:25:44.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.468+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/1609757051 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5632dbe3bf70 con 0x5632dc10e6c0 2023-11-27T15:25:44.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.468+0000 7f16a539a700 1 --2- 172.21.15.133:0/1609757051 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5632dc10e6c0 0x5632dc110ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:44.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.469+0000 7f16897fa700 10 client.26138 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:44.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.469+0000 7f16a539a700 1 --2- 172.21.15.133:0/1609757051 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5632dc10e6c0 0x5632dc110ad0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86cbab262f7640cb server_cookie=b3f77ea52b5bd07c in_seq=0 out_seq=0 2023-11-27T15:25:44.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.470+0000 7f16897fa700 1 -- 172.21.15.133:0/1609757051 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x5632dbe3bf70 con 0x5632dc10e6c0 2023-11-27T15:25:44.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.470+0000 7f16897fa700 10 client.26138 handle_command_reply: tid=0 2023-11-27T15:25:44.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.470+0000 7f16bb6ecb80 1 client.26138 shutdown 2023-11-27T15:25:44.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.470+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/1609757051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09bc00 msgr2=0x5632dc0b0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:44.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.470+0000 7f16bb6ecb80 1 --2- 172.21.15.133:0/1609757051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09bc00 0x5632dc0b0ce0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f169c096960 tx=0x7f169c099fa0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.470+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/1609757051 shutdown_connections 2023-11-27T15:25:44.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.470+0000 7f16bb6ecb80 1 --2- 172.21.15.133:0/1609757051 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5632dc10e6c0 0x5632dc110ad0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.470+0000 7f16bb6ecb80 1 --2- 172.21.15.133:0/1609757051 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5632dc09d110 0x5632dc0b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.470+0000 7f16bb6ecb80 1 --2- 172.21.15.133:0/1609757051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632dc09bc00 0x5632dc0b0ce0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.470+0000 7f16bb6ecb80 1 --2- 172.21.15.133:0/1609757051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5632dc09c550 0x5632dc0b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.470+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/1609757051 >> 172.21.15.133:0/1609757051 conn(0x5632dc091360 msgr2=0x5632dc09ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:44.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.470+0000 7f16897fa700 0 client.26138 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:44.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.471+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/1609757051 shutdown_connections 2023-11-27T15:25:44.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.471+0000 7f16bb6ecb80 1 -- 172.21.15.133:0/1609757051 wait complete. 2023-11-27T15:25:44.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.471+0000 7f16bb6ecb80 20 client.26138 trim_cache size 0 max 16384 2023-11-27T15:25:44.476 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:25:44.477 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (7360 inodes in the stack)", 2023-11-27T15:25:44.477 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-11-27T15:25:44.477 INFO:teuthology.orchestra.run.smithi133.stdout: "b6638f49-a408-46a9-83f0-3995bb2029f5": { 2023-11-27T15:25:44.477 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-11-27T15:25:44.477 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "b6638f49-a408-46a9-83f0-3995bb2029f5", 2023-11-27T15:25:44.477 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-11-27T15:25:44.477 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:25:44.477 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:25:44.477 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:25:44.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.473+0000 7f16a7dff700 1 -- 172.21.15.133:0/385574340 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f168c0a11d0 msgr2=0x7f168c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:44.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.473+0000 7f16a7dff700 1 --2- 172.21.15.133:0/385574340 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f168c0a11d0 0x7f168c0a3680 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1698009610 tx=0x7f1698006210 comp rx=0 tx=0).stop 2023-11-27T15:25:44.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.473+0000 7f16a7dff700 1 -- 172.21.15.133:0/385574340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a01002d0 msgr2=0x7f16a0198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:44.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.473+0000 7f16a7dff700 1 --2- 172.21.15.133:0/385574340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a01002d0 0x7f16a0198090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f169c00e970 tx=0x7f169c00ec80 comp rx=0 tx=0).stop 2023-11-27T15:25:44.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.473+0000 7f16a7dff700 1 -- 172.21.15.133:0/385574340 shutdown_connections 2023-11-27T15:25:44.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.473+0000 7f16a7dff700 1 --2- 172.21.15.133:0/385574340 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f168c0a11d0 0x7f168c0a3680 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.473+0000 7f16a7dff700 1 --2- 172.21.15.133:0/385574340 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f16a0101e50 0x7f16a019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.473+0000 7f16a7dff700 1 --2- 172.21.15.133:0/385574340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a01002d0 0x7f16a0198090 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.473+0000 7f16a7dff700 1 --2- 172.21.15.133:0/385574340 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f16a01014c0 0x7f16a01985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:44.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.473+0000 7f16a7dff700 1 -- 172.21.15.133:0/385574340 >> 172.21.15.133:0/385574340 conn(0x7f16a00fb8b0 msgr2=0x7f16a00fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:44.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.474+0000 7f16a7dff700 1 -- 172.21.15.133:0/385574340 shutdown_connections 2023-11-27T15:25:44.479 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:25:44.474+0000 7f16a7dff700 1 -- 172.21.15.133:0/385574340 wait complete. 2023-11-27T15:25:44.507 INFO:tasks.cephfs.filesystem:scrub status for tag:b6638f49-a408-46a9-83f0-3995bb2029f5 - {'path': '/', 'tag': 'b6638f49-a408-46a9-83f0-3995bb2029f5', 'options': 'recursive,force'} 2023-11-27T15:25:45.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:44 smithi174 ceph-mon[131110]: pgmap v433: 129 pgs: 129 active+clean; 34 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 1.28k op/s 2023-11-27T15:25:45.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:44 smithi174 ceph-mon[131110]: scrub queued for path: / 2023-11-27T15:25:45.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:44 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:45.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:44 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:25:45.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:44 smithi133 ceph-mon[97860]: pgmap v433: 129 pgs: 129 active+clean; 34 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 1.28k op/s 2023-11-27T15:25:45.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:44 smithi133 ceph-mon[97860]: scrub queued for path: / 2023-11-27T15:25:45.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:44 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:45.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:44 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:25:45.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:44 smithi177 ceph-mon[131504]: pgmap v433: 129 pgs: 129 active+clean; 34 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 1.28k op/s 2023-11-27T15:25:45.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:44 smithi177 ceph-mon[131504]: scrub queued for path: / 2023-11-27T15:25:45.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:44 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:45.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:44 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:25:47.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:46 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:25:47.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:46 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:25:47.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:46 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:47.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:46 smithi174 ceph-mon[131110]: pgmap v434: 129 pgs: 129 active+clean; 32 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 1.63k op/s 2023-11-27T15:25:47.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:46 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:25:47.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:46 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:25:47.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:46 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:47.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:46 smithi133 ceph-mon[97860]: pgmap v434: 129 pgs: 129 active+clean; 32 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 1.63k op/s 2023-11-27T15:25:47.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:46 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:25:47.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:46 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:25:47.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:46 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:25:47.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:46 smithi177 ceph-mon[131504]: pgmap v434: 129 pgs: 129 active+clean; 32 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 1.63k op/s 2023-11-27T15:25:49.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:48 smithi133 ceph-mon[97860]: pgmap v435: 129 pgs: 129 active+clean; 37 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 1.74k op/s 2023-11-27T15:25:49.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:48 smithi177 ceph-mon[131504]: pgmap v435: 129 pgs: 129 active+clean; 37 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 1.74k op/s 2023-11-27T15:25:49.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:48 smithi174 ceph-mon[131110]: pgmap v435: 129 pgs: 129 active+clean; 37 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 1.74k op/s 2023-11-27T15:25:51.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:50 smithi174 ceph-mon[131110]: pgmap v436: 129 pgs: 129 active+clean; 42 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 2.17k op/s 2023-11-27T15:25:51.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:50 smithi133 ceph-mon[97860]: pgmap v436: 129 pgs: 129 active+clean; 42 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 2.17k op/s 2023-11-27T15:25:51.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:50 smithi177 ceph-mon[131504]: pgmap v436: 129 pgs: 129 active+clean; 42 MiB data, 554 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.3 MiB/s wr, 2.17k op/s 2023-11-27T15:25:53.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:52 smithi174 ceph-mon[131110]: pgmap v437: 129 pgs: 129 active+clean; 49 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 2.13k op/s 2023-11-27T15:25:53.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:52 smithi174 ceph-mon[131110]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:25:53.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:52 smithi133 ceph-mon[97860]: pgmap v437: 129 pgs: 129 active+clean; 49 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 2.13k op/s 2023-11-27T15:25:53.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:52 smithi133 ceph-mon[97860]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:25:53.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:52 smithi177 ceph-mon[131504]: pgmap v437: 129 pgs: 129 active+clean; 49 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 2.13k op/s 2023-11-27T15:25:53.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:52 smithi177 ceph-mon[131504]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:25:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.223+0000 7fe4799ff700 1 -- 172.21.15.133:0/1639587270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4741014b0 msgr2=0x7fe474101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.223+0000 7fe4799ff700 1 --2- 172.21.15.133:0/1639587270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4741014b0 0x7fe474101900 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe4680098f0 tx=0x7fe468009c00 comp rx=0 tx=0).stop 2023-11-27T15:25:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.223+0000 7fe4799ff700 1 -- 172.21.15.133:0/1639587270 shutdown_connections 2023-11-27T15:25:54.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.223+0000 7fe4799ff700 1 --2- 172.21.15.133:0/1639587270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe474101e40 0x7fe474108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.223+0000 7fe4799ff700 1 --2- 172.21.15.133:0/1639587270 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe4741002c0 0x7fe4741006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.223+0000 7fe4799ff700 1 --2- 172.21.15.133:0/1639587270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4741014b0 0x7fe474101900 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.223+0000 7fe4799ff700 1 -- 172.21.15.133:0/1639587270 >> 172.21.15.133:0/1639587270 conn(0x7fe4740fb8e0 msgr2=0x7fe4740fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.223+0000 7fe4799ff700 1 -- 172.21.15.133:0/1639587270 shutdown_connections 2023-11-27T15:25:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.224+0000 7fe4799ff700 1 -- 172.21.15.133:0/1639587270 wait complete. 2023-11-27T15:25:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.224+0000 7fe4799ff700 1 Processor -- start 2023-11-27T15:25:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.225+0000 7fe4799ff700 1 -- start start 2023-11-27T15:25:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.225+0000 7fe4799ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4741002c0 0x7fe474198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.225+0000 7fe4799ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4741014b0 0x7fe4741987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.225+0000 7fe4799ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe474101e40 0x7fe47419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.225+0000 7fe4799ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47419fde0 con 0x7fe4741002c0 2023-11-27T15:25:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.225+0000 7fe4799ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47419ff20 con 0x7fe4741014b0 2023-11-27T15:25:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.225+0000 7fe4799ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4741a00b0 con 0x7fe474101e40 2023-11-27T15:25:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.225+0000 7fe472ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4741002c0 0x7fe474198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.225+0000 7fe4727fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4741014b0 0x7fe4741987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.225+0000 7fe472ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4741002c0 0x7fe474198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:44866/0 (socket says 172.21.15.133:44866) 2023-11-27T15:25:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.225+0000 7fe472ffd700 1 -- 172.21.15.133:0/2466189489 learned_addr learned my addr 172.21.15.133:0/2466189489 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.225+0000 7fe4737fe700 1 --2- 172.21.15.133:0/2466189489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe474101e40 0x7fe47419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.226+0000 7fe472ffd700 1 -- 172.21.15.133:0/2466189489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe474101e40 msgr2=0x7fe47419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.226+0000 7fe472ffd700 1 --2- 172.21.15.133:0/2466189489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe474101e40 0x7fe47419f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.226+0000 7fe472ffd700 1 -- 172.21.15.133:0/2466189489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4741014b0 msgr2=0x7fe4741987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.226+0000 7fe472ffd700 1 --2- 172.21.15.133:0/2466189489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4741014b0 0x7fe4741987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.226+0000 7fe472ffd700 1 -- 172.21.15.133:0/2466189489 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c009d00 con 0x7fe4741002c0 2023-11-27T15:25:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.226+0000 7fe472ffd700 1 --2- 172.21.15.133:0/2466189489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4741002c0 0x7fe474198260 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe46400e970 tx=0x7fe46400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.226+0000 7fe4789fd700 1 -- 172.21.15.133:0/2466189489 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe464009790 con 0x7fe4741002c0 2023-11-27T15:25:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.226+0000 7fe4789fd700 1 -- 172.21.15.133:0/2466189489 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe46400c7e0 con 0x7fe4741002c0 2023-11-27T15:25:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.226+0000 7fe4789fd700 1 -- 172.21.15.133:0/2466189489 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe464003c70 con 0x7fe4741002c0 2023-11-27T15:25:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.226+0000 7fe4799ff700 1 -- 172.21.15.133:0/2466189489 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe468009520 con 0x7fe4741002c0 2023-11-27T15:25:54.233 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.226+0000 7fe4799ff700 1 -- 172.21.15.133:0/2466189489 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe47410bc10 con 0x7fe4741002c0 2023-11-27T15:25:54.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.229+0000 7fe4789fd700 1 -- 172.21.15.133:0/2466189489 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe46400cc50 con 0x7fe4741002c0 2023-11-27T15:25:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.231+0000 7fe48d259b80 1 Processor -- start 2023-11-27T15:25:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.231+0000 7fe48d259b80 1 -- start start 2023-11-27T15:25:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.231+0000 7fe4789fd700 1 --2- 172.21.15.133:0/2466189489 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe45c0a1150 0x7fe45c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.231+0000 7fe4789fd700 1 -- 172.21.15.133:0/2466189489 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe464014070 con 0x7fe4741002c0 2023-11-27T15:25:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.231+0000 7fe48d259b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6cecb6b0 0x561e6d06f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.231+0000 7fe48d259b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6d06fdb0 0x561e6d0705d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.231+0000 7fe48d259b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e6d070b10 0x561e6d070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.231+0000 7fe48d259b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6cf75000 con 0x561e6d070b10 2023-11-27T15:25:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.231+0000 7fe48d259b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6cfc7840 con 0x561e6d06fdb0 2023-11-27T15:25:54.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.231+0000 7fe48d259b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6cfc0cd0 con 0x561e6cecb6b0 2023-11-27T15:25:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.232+0000 7fe4727fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6d06fdb0 0x561e6d0705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.232+0000 7fe472ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6cecb6b0 0x561e6d06f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.232+0000 7fe4727fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6d06fdb0 0x561e6d0705d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:52684/0 (socket says 172.21.15.133:52684) 2023-11-27T15:25:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.232+0000 7fe4737fe700 1 --2- 172.21.15.133:0/2466189489 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe45c0a1150 0x7fe45c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.232+0000 7fe4727fc700 1 -- 172.21.15.133:0/810378589 learned_addr learned my addr 172.21.15.133:0/810378589 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.232+0000 7fe4737fe700 1 --2- 172.21.15.133:0/810378589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e6d070b10 0x561e6d070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.232+0000 7fe4727fc700 1 -- 172.21.15.133:0/810378589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6cecb6b0 msgr2=0x561e6d06f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.232+0000 7fe4727fc700 1 --2- 172.21.15.133:0/810378589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6cecb6b0 0x561e6d06f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.232+0000 7fe4727fc700 1 -- 172.21.15.133:0/810378589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e6d070b10 msgr2=0x561e6d070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.232+0000 7fe4727fc700 1 --2- 172.21.15.133:0/810378589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e6d070b10 0x561e6d070f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.232+0000 7fe4727fc700 1 -- 172.21.15.133:0/810378589 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e6ce425d0 con 0x561e6d06fdb0 2023-11-27T15:25:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.232+0000 7fe4727fc700 1 --2- 172.21.15.133:0/810378589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6d06fdb0 0x561e6d0705d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe46800f9e0 tx=0x7fe468005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=220aea8fed2fe232 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.232+0000 7fe4737fe700 1 --2- 172.21.15.133:0/2466189489 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe45c0a1150 0x7fe45c0a3600 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009fd0 tx=0x7fe46c0099e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.232+0000 7fe459ffb700 1 -- 172.21.15.133:0/810378589 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46800fe70 con 0x561e6d06fdb0 2023-11-27T15:25:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.233+0000 7fe459ffb700 1 -- 172.21.15.133:0/810378589 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe46801d040 con 0x561e6d06fdb0 2023-11-27T15:25:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.233+0000 7fe459ffb700 1 -- 172.21.15.133:0/810378589 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46800bbc0 con 0x561e6d06fdb0 2023-11-27T15:25:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.233+0000 7fe48d259b80 1 -- 172.21.15.133:0/810378589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6d06fdb0 msgr2=0x561e6d0705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.233+0000 7fe48d259b80 1 --2- 172.21.15.133:0/810378589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6d06fdb0 0x561e6d0705d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe46800f9e0 tx=0x7fe468005dc0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.233+0000 7fe48d259b80 1 -- 172.21.15.133:0/810378589 shutdown_connections 2023-11-27T15:25:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.233+0000 7fe48d259b80 1 --2- 172.21.15.133:0/810378589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e6d070b10 0x561e6d070f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.233+0000 7fe48d259b80 1 --2- 172.21.15.133:0/810378589 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6cecb6b0 0x561e6d06f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.233+0000 7fe48d259b80 1 --2- 172.21.15.133:0/810378589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6d06fdb0 0x561e6d0705d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.233+0000 7fe48d259b80 1 -- 172.21.15.133:0/810378589 >> 172.21.15.133:0/810378589 conn(0x561e6d064f50 msgr2=0x561e6d06a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.233+0000 7fe48d259b80 1 -- 172.21.15.133:0/810378589 shutdown_connections 2023-11-27T15:25:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.233+0000 7fe48d259b80 1 -- 172.21.15.133:0/810378589 wait complete. 2023-11-27T15:25:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.234+0000 7fe48d259b80 1 -- start start 2023-11-27T15:25:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.234+0000 7fe48d259b80 1 Processor -- start 2023-11-27T15:25:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.235+0000 7fe48d259b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6cecb6b0 0x561e6d078bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.235+0000 7fe48d259b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6d06fdb0 0x561e6d079100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.235+0000 7fe48d259b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e6d070b10 0x561e6d080140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.235+0000 7fe48d259b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6cf62880 con 0x561e6d070b10 2023-11-27T15:25:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.235+0000 7fe48d259b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6cf62520 con 0x561e6d06fdb0 2023-11-27T15:25:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.235+0000 7fe48d259b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6ce448c0 con 0x561e6cecb6b0 2023-11-27T15:25:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.235+0000 7fe48d259b80 10 client.0 _ll_register_callbacks cb 0x561e6cf80720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.235+0000 7fe472ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6cecb6b0 0x561e6d078bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.235+0000 7fe472ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6cecb6b0 0x561e6d078bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:46852/0 (socket says 172.21.15.133:46852) 2023-11-27T15:25:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.235+0000 7fe4727fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6d06fdb0 0x561e6d079100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.235+0000 7fe472ffd700 1 -- 172.21.15.133:0/184030947 learned_addr learned my addr 172.21.15.133:0/184030947 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.235+0000 7fe4737fe700 1 --2- 172.21.15.133:0/184030947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e6d070b10 0x561e6d080140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.235+0000 7fe472ffd700 1 -- 172.21.15.133:0/184030947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6d06fdb0 msgr2=0x561e6d079100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.235+0000 7fe472ffd700 1 --2- 172.21.15.133:0/184030947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6d06fdb0 0x561e6d079100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.235+0000 7fe472ffd700 1 -- 172.21.15.133:0/184030947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e6d070b10 msgr2=0x561e6d080140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.235+0000 7fe472ffd700 1 --2- 172.21.15.133:0/184030947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e6d070b10 0x561e6d080140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.235+0000 7fe472ffd700 1 -- 172.21.15.133:0/184030947 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe474109030 con 0x561e6cecb6b0 2023-11-27T15:25:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.235+0000 7fe472ffd700 1 --2- 172.21.15.133:0/184030947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6cecb6b0 0x561e6d078bc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe4640ca540 tx=0x7fe464003e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5be5450e629dc68d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.235+0000 7fe452ffd700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:25:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.236+0000 7fe452ffd700 1 -- 172.21.15.133:0/184030947 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46409a310 con 0x561e6cecb6b0 2023-11-27T15:25:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.236+0000 7fe452ffd700 1 -- 172.21.15.133:0/184030947 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe4640a2300 con 0x561e6cecb6b0 2023-11-27T15:25:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.236+0000 7fe48d259b80 1 -- 172.21.15.133:0/184030947 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e6d062710 con 0x561e6cecb6b0 2023-11-27T15:25:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.236+0000 7fe452ffd700 1 -- 172.21.15.133:0/184030947 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4640b8250 con 0x561e6cecb6b0 2023-11-27T15:25:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.236+0000 7fe48d259b80 10 client.26131 fetch_fsmap learned FSMap version 25 2023-11-27T15:25:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.236+0000 7fe48d259b80 1 -- 172.21.15.133:0/184030947 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561e6cf87130 con 0x561e6cecb6b0 2023-11-27T15:25:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.236+0000 7fe452ffd700 1 -- 172.21.15.133:0/184030947 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561e6d062710 con 0x561e6cecb6b0 2023-11-27T15:25:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.236+0000 7fe452ffd700 1 -- 172.21.15.133:0/184030947 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 25) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fe4640a1e70 con 0x561e6cecb6b0 2023-11-27T15:25:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.236+0000 7fe48d259b80 10 client.26131 fetch_fsmap finished waiting for FSMap version 25 2023-11-27T15:25:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.236+0000 7fe48d259b80 10 client.26131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.236+0000 7fe48d259b80 20 client.26131 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.236+0000 7fe48d259b80 1 --2- 172.21.15.133:0/184030947 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x561e6d084980 0x561e6d086d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.236+0000 7fe48d259b80 4 client.26131 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:25:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.236+0000 7fe48d259b80 1 -- 172.21.15.133:0/184030947 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e6cf83040 con 0x561e6d084980 2023-11-27T15:25:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.237+0000 7fe4727fc700 1 --2- 172.21.15.133:0/184030947 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x561e6d084980 0x561e6d086d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.237+0000 7fe4727fc700 1 --2- 172.21.15.133:0/184030947 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x561e6d084980 0x561e6d086d90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f91d83de9d070d21 server_cookie=8aec71cf269aff1 in_seq=0 out_seq=0 2023-11-27T15:25:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.237+0000 7fe452ffd700 10 client.26131 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.239+0000 7fe452ffd700 1 -- 172.21.15.133:0/184030947 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561e6cf83040 con 0x561e6d084980 2023-11-27T15:25:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.239+0000 7fe452ffd700 10 client.26131 handle_command_reply: tid=0 2023-11-27T15:25:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.240+0000 7fe48d259b80 1 client.26131 shutdown 2023-11-27T15:25:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.240+0000 7fe48d259b80 1 -- 172.21.15.133:0/184030947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6cecb6b0 msgr2=0x561e6d078bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.240+0000 7fe48d259b80 1 --2- 172.21.15.133:0/184030947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6cecb6b0 0x561e6d078bc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe4640ca540 tx=0x7fe464003e50 comp rx=0 tx=0).stop 2023-11-27T15:25:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.240+0000 7fe48d259b80 1 -- 172.21.15.133:0/184030947 shutdown_connections 2023-11-27T15:25:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.240+0000 7fe48d259b80 1 --2- 172.21.15.133:0/184030947 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x561e6d084980 0x561e6d086d90 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.240+0000 7fe48d259b80 1 --2- 172.21.15.133:0/184030947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e6d070b10 0x561e6d080140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.240+0000 7fe48d259b80 1 --2- 172.21.15.133:0/184030947 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6cecb6b0 0x561e6d078bc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.240+0000 7fe48d259b80 1 --2- 172.21.15.133:0/184030947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6d06fdb0 0x561e6d079100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.240+0000 7fe452ffd700 0 client.26131 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:54.248 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.240+0000 7fe48d259b80 1 -- 172.21.15.133:0/184030947 >> 172.21.15.133:0/184030947 conn(0x561e6d064f50 msgr2=0x561e6d0697e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:54.248 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.240+0000 7fe48d259b80 1 -- 172.21.15.133:0/184030947 shutdown_connections 2023-11-27T15:25:54.248 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.240+0000 7fe48d259b80 1 -- 172.21.15.133:0/184030947 wait complete. 2023-11-27T15:25:54.248 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.240+0000 7fe48d259b80 20 client.26131 trim_cache size 0 max 16384 2023-11-27T15:25:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.258+0000 7fe48d259b80 1 Processor -- start 2023-11-27T15:25:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.258+0000 7fe48d259b80 1 -- start start 2023-11-27T15:25:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.258+0000 7fe48d259b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6cecb6b0 0x561e6d06b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.258+0000 7fe48d259b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6d06fdb0 0x561e6d080100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.258+0000 7fe48d259b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e6d070b10 0x561e6d077990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.258+0000 7fe48d259b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6d078730 con 0x561e6d070b10 2023-11-27T15:25:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.258+0000 7fe48d259b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6d081dd0 con 0x561e6cecb6b0 2023-11-27T15:25:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.258+0000 7fe48d259b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6d076b80 con 0x561e6d06fdb0 2023-11-27T15:25:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.258+0000 7fe4727fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6d06fdb0 0x561e6d080100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.258+0000 7fe472ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6cecb6b0 0x561e6d06b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.258+0000 7fe4727fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6d06fdb0 0x561e6d080100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:46862/0 (socket says 172.21.15.133:46862) 2023-11-27T15:25:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.258+0000 7fe4727fc700 1 -- 172.21.15.133:0/1500593060 learned_addr learned my addr 172.21.15.133:0/1500593060 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.258+0000 7fe4737fe700 1 --2- 172.21.15.133:0/1500593060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e6d070b10 0x561e6d077990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.259+0000 7fe4727fc700 1 -- 172.21.15.133:0/1500593060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6cecb6b0 msgr2=0x561e6d06b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.259+0000 7fe4727fc700 1 --2- 172.21.15.133:0/1500593060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6cecb6b0 0x561e6d06b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.259+0000 7fe4727fc700 1 -- 172.21.15.133:0/1500593060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e6d070b10 msgr2=0x561e6d077990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.259+0000 7fe4727fc700 1 --2- 172.21.15.133:0/1500593060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e6d070b10 0x561e6d077990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.259+0000 7fe4727fc700 1 -- 172.21.15.133:0/1500593060 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e6cf86e80 con 0x561e6d06fdb0 2023-11-27T15:25:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.259+0000 7fe4727fc700 1 --2- 172.21.15.133:0/1500593060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6d06fdb0 0x561e6d080100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe468009fd0 tx=0x7fe468000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d466b895f63c81c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.259+0000 7fe45a7fc700 1 -- 172.21.15.133:0/1500593060 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe468032490 con 0x561e6d06fdb0 2023-11-27T15:25:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.259+0000 7fe45a7fc700 1 -- 172.21.15.133:0/1500593060 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe468030070 con 0x561e6d06fdb0 2023-11-27T15:25:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.259+0000 7fe45a7fc700 1 -- 172.21.15.133:0/1500593060 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe468032a50 con 0x561e6d06fdb0 2023-11-27T15:25:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.260+0000 7fe48d259b80 1 -- 172.21.15.133:0/1500593060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6d06fdb0 msgr2=0x561e6d080100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.260+0000 7fe48d259b80 1 --2- 172.21.15.133:0/1500593060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6d06fdb0 0x561e6d080100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe468009fd0 tx=0x7fe468000bc0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.260+0000 7fe48d259b80 1 -- 172.21.15.133:0/1500593060 shutdown_connections 2023-11-27T15:25:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.260+0000 7fe48d259b80 1 --2- 172.21.15.133:0/1500593060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e6d070b10 0x561e6d077990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.260+0000 7fe48d259b80 1 --2- 172.21.15.133:0/1500593060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6cecb6b0 0x561e6d06b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.260+0000 7fe48d259b80 1 --2- 172.21.15.133:0/1500593060 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6d06fdb0 0x561e6d080100 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.260+0000 7fe48d259b80 1 -- 172.21.15.133:0/1500593060 >> 172.21.15.133:0/1500593060 conn(0x561e6d064f50 msgr2=0x561e6d06a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.260+0000 7fe48d259b80 1 -- 172.21.15.133:0/1500593060 shutdown_connections 2023-11-27T15:25:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.260+0000 7fe48d259b80 1 -- 172.21.15.133:0/1500593060 wait complete. 2023-11-27T15:25:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.260+0000 7fe48d259b80 1 -- start start 2023-11-27T15:25:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.261+0000 7fe48d259b80 1 Processor -- start 2023-11-27T15:25:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.261+0000 7fe48d259b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e6cecb6b0 0x561e6d0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.261+0000 7fe48d259b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6d06fdb0 0x561e6d075f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.261+0000 7fe48d259b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6d070b10 0x561e6d07e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.261+0000 7fe48d259b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6d07ec30 con 0x561e6cecb6b0 2023-11-27T15:25:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.261+0000 7fe48d259b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6d07eda0 con 0x561e6d070b10 2023-11-27T15:25:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.261+0000 7fe48d259b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e6d07ef60 con 0x561e6d06fdb0 2023-11-27T15:25:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.261+0000 7fe48d259b80 10 client.0 _ll_register_callbacks cb 0x561e6cf80720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:25:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.261+0000 7fe4727fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6d06fdb0 0x561e6d075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.261+0000 7fe4737fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6d070b10 0x561e6d07e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.261+0000 7fe4727fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6d06fdb0 0x561e6d075f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:46864/0 (socket says 172.21.15.133:46864) 2023-11-27T15:25:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.261+0000 7fe4727fc700 1 -- 172.21.15.133:0/2257647190 learned_addr learned my addr 172.21.15.133:0/2257647190 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:25:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.261+0000 7fe472ffd700 1 --2- 172.21.15.133:0/2257647190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e6cecb6b0 0x561e6d0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.262+0000 7fe4727fc700 1 -- 172.21.15.133:0/2257647190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6d070b10 msgr2=0x561e6d07e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.262+0000 7fe4727fc700 1 --2- 172.21.15.133:0/2257647190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6d070b10 0x561e6d07e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.262+0000 7fe4727fc700 1 -- 172.21.15.133:0/2257647190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e6cecb6b0 msgr2=0x561e6d0759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:25:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.262+0000 7fe4727fc700 1 --2- 172.21.15.133:0/2257647190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e6cecb6b0 0x561e6d0759f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.262+0000 7fe4727fc700 1 -- 172.21.15.133:0/2257647190 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e6ce425d0 con 0x561e6d06fdb0 2023-11-27T15:25:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.262+0000 7fe4727fc700 1 --2- 172.21.15.133:0/2257647190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6d06fdb0 0x561e6d075f30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe46800b3b0 tx=0x7fe46800f900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aec95adf0fb721f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:25:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.262+0000 7fe452ffd700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:25:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.262+0000 7fe452ffd700 1 -- 172.21.15.133:0/2257647190 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46800b700 con 0x561e6d06fdb0 2023-11-27T15:25:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.262+0000 7fe452ffd700 1 -- 172.21.15.133:0/2257647190 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe4680050a0 con 0x561e6d06fdb0 2023-11-27T15:25:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.262+0000 7fe48d259b80 1 -- 172.21.15.133:0/2257647190 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe46801d040 con 0x561e6d06fdb0 2023-11-27T15:25:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.262+0000 7fe452ffd700 1 -- 172.21.15.133:0/2257647190 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe468005920 con 0x561e6d06fdb0 2023-11-27T15:25:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.262+0000 7fe48d259b80 10 client.26137 fetch_fsmap learned FSMap version 25 2023-11-27T15:25:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.262+0000 7fe48d259b80 1 -- 172.21.15.133:0/2257647190 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561e6cdd7c00 con 0x561e6d06fdb0 2023-11-27T15:25:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.263+0000 7fe452ffd700 1 -- 172.21.15.133:0/2257647190 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe46801d040 con 0x561e6d06fdb0 2023-11-27T15:25:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.263+0000 7fe452ffd700 1 -- 172.21.15.133:0/2257647190 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 25) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fe468009c70 con 0x561e6d06fdb0 2023-11-27T15:25:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.263+0000 7fe48d259b80 10 client.26137 fetch_fsmap finished waiting for FSMap version 25 2023-11-27T15:25:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.263+0000 7fe48d259b80 10 client.26137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:25:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.263+0000 7fe48d259b80 20 client.26137 populate_metadata read hostname 'smithi133' 2023-11-27T15:25:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.263+0000 7fe48d259b80 1 --2- 172.21.15.133:0/2257647190 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x561e6d0d8830 0x561e6d0dac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:25:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.263+0000 7fe48d259b80 4 client.26137 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:25:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.263+0000 7fe48d259b80 1 -- 172.21.15.133:0/2257647190 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561e6cf8f600 con 0x561e6d0d8830 2023-11-27T15:25:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.263+0000 7fe472ffd700 1 --2- 172.21.15.133:0/2257647190 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x561e6d0d8830 0x561e6d0dac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:25:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.264+0000 7fe452ffd700 10 client.26137 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:54.264+0000 7fe472ffd700 1 --2- 172.21.15.133:0/2257647190 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x561e6d0d8830 0x561e6d0dac10 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cc53c64feecec6c server_cookie=7694a4ee575f7a52 in_seq=0 out_seq=0 2023-11-27T15:25:55.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:54 smithi133 ceph-mon[97860]: pgmap v438: 129 pgs: 129 active+clean; 49 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 1.71k op/s 2023-11-27T15:25:55.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:54 smithi177 ceph-mon[131504]: pgmap v438: 129 pgs: 129 active+clean; 49 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 1.71k op/s 2023-11-27T15:25:55.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:54 smithi174 ceph-mon[131110]: pgmap v438: 129 pgs: 129 active+clean; 49 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 1.71k op/s 2023-11-27T15:25:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.476+0000 7fe452ffd700 1 -- 172.21.15.133:0/2257647190 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561e6cf8f600 con 0x561e6d0d8830 2023-11-27T15:25:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.476+0000 7fe452ffd700 10 client.26137 handle_command_reply: tid=0 2023-11-27T15:25:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.476+0000 7fe48d259b80 1 client.26137 shutdown 2023-11-27T15:25:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.477+0000 7fe48d259b80 1 -- 172.21.15.133:0/2257647190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6d06fdb0 msgr2=0x561e6d075f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.477+0000 7fe48d259b80 1 --2- 172.21.15.133:0/2257647190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6d06fdb0 0x561e6d075f30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe46800b3b0 tx=0x7fe46800f900 comp rx=0 tx=0).stop 2023-11-27T15:25:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.477+0000 7fe48d259b80 1 -- 172.21.15.133:0/2257647190 shutdown_connections 2023-11-27T15:25:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.477+0000 7fe48d259b80 1 --2- 172.21.15.133:0/2257647190 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x561e6d0d8830 0x561e6d0dac10 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.477+0000 7fe48d259b80 1 --2- 172.21.15.133:0/2257647190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e6d070b10 0x561e6d07e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.482 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.477+0000 7fe452ffd700 0 client.26137 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:25:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.477+0000 7fe48d259b80 1 --2- 172.21.15.133:0/2257647190 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561e6cecb6b0 0x561e6d0759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.477+0000 7fe48d259b80 1 --2- 172.21.15.133:0/2257647190 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561e6d06fdb0 0x561e6d075f30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.477+0000 7fe48d259b80 1 -- 172.21.15.133:0/2257647190 >> 172.21.15.133:0/2257647190 conn(0x561e6d064f50 msgr2=0x561e6d08b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.477+0000 7fe48d259b80 1 -- 172.21.15.133:0/2257647190 shutdown_connections 2023-11-27T15:25:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.477+0000 7fe48d259b80 1 -- 172.21.15.133:0/2257647190 wait complete. 2023-11-27T15:25:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.477+0000 7fe48d259b80 20 client.26137 trim_cache size 0 max 16384 2023-11-27T15:25:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:25:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:25:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:25:56.483 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:25:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.478+0000 7fe4799ff700 1 -- 172.21.15.133:0/2466189489 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe45c0a1150 msgr2=0x7fe45c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.478+0000 7fe4799ff700 1 --2- 172.21.15.133:0/2466189489 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe45c0a1150 0x7fe45c0a3600 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009fd0 tx=0x7fe46c0099e0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.478+0000 7fe4799ff700 1 -- 172.21.15.133:0/2466189489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4741002c0 msgr2=0x7fe474198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:25:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.478+0000 7fe4799ff700 1 --2- 172.21.15.133:0/2466189489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4741002c0 0x7fe474198260 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe46400e970 tx=0x7fe46400ec80 comp rx=0 tx=0).stop 2023-11-27T15:25:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.478+0000 7fe4799ff700 1 -- 172.21.15.133:0/2466189489 shutdown_connections 2023-11-27T15:25:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.478+0000 7fe4799ff700 1 --2- 172.21.15.133:0/2466189489 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe45c0a1150 0x7fe45c0a3600 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.484 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.478+0000 7fe4799ff700 1 --2- 172.21.15.133:0/2466189489 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe474101e40 0x7fe47419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.478+0000 7fe4799ff700 1 --2- 172.21.15.133:0/2466189489 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4741002c0 0x7fe474198260 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.478+0000 7fe4799ff700 1 --2- 172.21.15.133:0/2466189489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4741014b0 0x7fe4741987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:25:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.478+0000 7fe4799ff700 1 -- 172.21.15.133:0/2466189489 >> 172.21.15.133:0/2466189489 conn(0x7fe4740fb8e0 msgr2=0x7fe4740fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:25:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.478+0000 7fe4799ff700 1 -- 172.21.15.133:0/2466189489 shutdown_connections 2023-11-27T15:25:56.485 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:25:56.478+0000 7fe4799ff700 1 -- 172.21.15.133:0/2466189489 wait complete. 2023-11-27T15:25:57.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:56 smithi174 ceph-mon[131110]: pgmap v439: 129 pgs: 129 active+clean; 57 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 1.66k op/s 2023-11-27T15:25:57.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:56 smithi133 ceph-mon[97860]: pgmap v439: 129 pgs: 129 active+clean; 57 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 1.66k op/s 2023-11-27T15:25:57.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:56 smithi177 ceph-mon[131504]: pgmap v439: 129 pgs: 129 active+clean; 57 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 1.66k op/s 2023-11-27T15:25:58.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:57 smithi133 ceph-mon[97860]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:25:58.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:57 smithi177 ceph-mon[131504]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:25:58.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:57 smithi174 ceph-mon[131110]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:25:59.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:25:58 smithi133 ceph-mon[97860]: pgmap v440: 129 pgs: 129 active+clean; 55 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 1.66k op/s 2023-11-27T15:25:59.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:25:58 smithi177 ceph-mon[131504]: pgmap v440: 129 pgs: 129 active+clean; 55 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 1.66k op/s 2023-11-27T15:25:59.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:25:58 smithi174 ceph-mon[131110]: pgmap v440: 129 pgs: 129 active+clean; 55 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.4 MiB/s wr, 1.66k op/s 2023-11-27T15:26:01.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:00 smithi133 ceph-mon[97860]: pgmap v441: 129 pgs: 129 active+clean; 61 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 2.02k op/s 2023-11-27T15:26:01.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:00 smithi177 ceph-mon[131504]: pgmap v441: 129 pgs: 129 active+clean; 61 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 2.02k op/s 2023-11-27T15:26:01.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:00 smithi174 ceph-mon[131110]: pgmap v441: 129 pgs: 129 active+clean; 61 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 2.02k op/s 2023-11-27T15:26:02.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:01 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:26:02.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:01 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:26:02.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:01 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:26:03.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:02 smithi133 ceph-mon[97860]: pgmap v442: 129 pgs: 129 active+clean; 65 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 1.76k op/s 2023-11-27T15:26:03.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:02 smithi177 ceph-mon[131504]: pgmap v442: 129 pgs: 129 active+clean; 65 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 1.76k op/s 2023-11-27T15:26:03.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:02 smithi174 ceph-mon[131110]: pgmap v442: 129 pgs: 129 active+clean; 65 MiB data, 673 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 1.76k op/s 2023-11-27T15:26:05.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:04 smithi133 ceph-mon[97860]: pgmap v443: 129 pgs: 129 active+clean; 72 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 1.46k op/s 2023-11-27T15:26:05.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:04 smithi177 ceph-mon[131504]: pgmap v443: 129 pgs: 129 active+clean; 72 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 1.46k op/s 2023-11-27T15:26:05.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:04 smithi174 ceph-mon[131110]: pgmap v443: 129 pgs: 129 active+clean; 72 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.1 MiB/s wr, 1.46k op/s 2023-11-27T15:26:07.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:06 smithi133 ceph-mon[97860]: pgmap v444: 129 pgs: 129 active+clean; 78 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 1.42k op/s 2023-11-27T15:26:07.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:06 smithi177 ceph-mon[131504]: pgmap v444: 129 pgs: 129 active+clean; 78 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 1.42k op/s 2023-11-27T15:26:07.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:06 smithi174 ceph-mon[131110]: pgmap v444: 129 pgs: 129 active+clean; 78 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.6 MiB/s wr, 1.42k op/s 2023-11-27T15:26:09.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:08 smithi133 ceph-mon[97860]: pgmap v445: 129 pgs: 129 active+clean; 83 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 1.16k op/s 2023-11-27T15:26:09.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:08 smithi177 ceph-mon[131504]: pgmap v445: 129 pgs: 129 active+clean; 83 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 1.16k op/s 2023-11-27T15:26:09.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:08 smithi174 ceph-mon[131110]: pgmap v445: 129 pgs: 129 active+clean; 83 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 1.16k op/s 2023-11-27T15:26:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.626+0000 7fd6529ff700 1 -- 172.21.15.133:0/1283594250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd64c101430 msgr2=0x7fd64c101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.626+0000 7fd6529ff700 1 --2- 172.21.15.133:0/1283594250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd64c101430 0x7fd64c101880 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd640009990 tx=0x7fd640009ca0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.626+0000 7fd6529ff700 1 -- 172.21.15.133:0/1283594250 shutdown_connections 2023-11-27T15:26:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.626+0000 7fd6529ff700 1 --2- 172.21.15.133:0/1283594250 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd64c101dc0 0x7fd64c108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.626+0000 7fd6529ff700 1 --2- 172.21.15.133:0/1283594250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd64c100240 0x7fd64c100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.626+0000 7fd6529ff700 1 --2- 172.21.15.133:0/1283594250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd64c101430 0x7fd64c101880 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.626+0000 7fd6529ff700 1 -- 172.21.15.133:0/1283594250 >> 172.21.15.133:0/1283594250 conn(0x7fd64c0fb840 msgr2=0x7fd64c0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.626+0000 7fd6529ff700 1 -- 172.21.15.133:0/1283594250 shutdown_connections 2023-11-27T15:26:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.626+0000 7fd6529ff700 1 -- 172.21.15.133:0/1283594250 wait complete. 2023-11-27T15:26:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.627+0000 7fd6529ff700 1 Processor -- start 2023-11-27T15:26:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.627+0000 7fd6529ff700 1 -- start start 2023-11-27T15:26:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.628+0000 7fd6529ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd64c100240 0x7fd64c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.628+0000 7fd6529ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd64c101430 0x7fd64c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.628+0000 7fd6529ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd64c101dc0 0x7fd64c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.628+0000 7fd6529ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c19fd10 con 0x7fd64c101dc0 2023-11-27T15:26:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.628+0000 7fd6529ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c19fe50 con 0x7fd64c101430 2023-11-27T15:26:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.628+0000 7fd6529ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c19ffe0 con 0x7fd64c100240 2023-11-27T15:26:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.628+0000 7fd64b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd64c101430 0x7fd64c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.628+0000 7fd64bfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd64c100240 0x7fd64c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.628+0000 7fd64b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd64c101430 0x7fd64c1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:57556/0 (socket says 172.21.15.133:57556) 2023-11-27T15:26:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.628+0000 7fd64b7fe700 1 -- 172.21.15.133:0/3146301530 learned_addr learned my addr 172.21.15.133:0/3146301530 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.628+0000 7fd650f9c700 1 --2- 172.21.15.133:0/3146301530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd64c101dc0 0x7fd64c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.628+0000 7fd64bfff700 1 -- 172.21.15.133:0/3146301530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd64c101430 msgr2=0x7fd64c1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.628+0000 7fd64bfff700 1 --2- 172.21.15.133:0/3146301530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd64c101430 0x7fd64c1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.628+0000 7fd64bfff700 1 -- 172.21.15.133:0/3146301530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd64c101dc0 msgr2=0x7fd64c19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.628+0000 7fd64bfff700 1 --2- 172.21.15.133:0/3146301530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd64c101dc0 0x7fd64c19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.628+0000 7fd64bfff700 1 -- 172.21.15.133:0/3146301530 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6400095f0 con 0x7fd64c100240 2023-11-27T15:26:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.628+0000 7fd64bfff700 1 --2- 172.21.15.133:0/3146301530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd64c100240 0x7fd64c198190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00e9c0 tx=0x7fd63c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.629+0000 7fd6497fa700 1 -- 172.21.15.133:0/3146301530 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c004220 con 0x7fd64c100240 2023-11-27T15:26:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.629+0000 7fd6497fa700 1 -- 172.21.15.133:0/3146301530 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd63c004380 con 0x7fd64c100240 2023-11-27T15:26:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.629+0000 7fd6497fa700 1 -- 172.21.15.133:0/3146301530 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63c003be0 con 0x7fd64c100240 2023-11-27T15:26:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.629+0000 7fd6529ff700 1 -- 172.21.15.133:0/3146301530 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64c10bb90 con 0x7fd64c100240 2023-11-27T15:26:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.629+0000 7fd6529ff700 1 -- 172.21.15.133:0/3146301530 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64c10ba00 con 0x7fd64c100240 2023-11-27T15:26:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.633+0000 7fd6497fa700 1 -- 172.21.15.133:0/3146301530 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd63c003d40 con 0x7fd64c100240 2023-11-27T15:26:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.634+0000 7fd66636db80 1 Processor -- start 2023-11-27T15:26:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.634+0000 7fd66636db80 1 -- start start 2023-11-27T15:26:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.634+0000 7fd66636db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795baa6b0 0x561795d4e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.635+0000 7fd66636db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 0x561795d4f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.635+0000 7fd6497fa700 1 --2- 172.21.15.133:0/3146301530 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd6340a11d0 0x7fd6340a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.635+0000 7fd66636db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795d4fb10 0x561795d4ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.635+0000 7fd66636db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561795c54000 con 0x561795d4edb0 2023-11-27T15:26:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.635+0000 7fd66636db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x561795ca6840 con 0x561795baa6b0 2023-11-27T15:26:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.635+0000 7fd66636db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x561795c9fcd0 con 0x561795d4fb10 2023-11-27T15:26:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.635+0000 7fd6497fa700 1 -- 172.21.15.133:0/3146301530 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd63c014070 con 0x7fd64c100240 2023-11-27T15:26:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.635+0000 7fd650f9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795baa6b0 0x561795d4e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.635+0000 7fd64b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 0x561795d4f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.635+0000 7fd650f9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795baa6b0 0x561795d4e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:57572/0 (socket says 172.21.15.133:57572) 2023-11-27T15:26:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.635+0000 7fd650f9c700 1 -- 172.21.15.133:0/1167317442 learned_addr learned my addr 172.21.15.133:0/1167317442 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.635+0000 7fd64bfff700 1 --2- 172.21.15.133:0/3146301530 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd6340a11d0 0x7fd6340a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.635+0000 7fd650f9c700 1 --2- 172.21.15.133:0/1167317442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795d4fb10 0x561795d4ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.635+0000 7fd650f9c700 1 -- 172.21.15.133:0/1167317442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795d4fb10 msgr2=0x561795d4ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.635+0000 7fd650f9c700 1 --2- 172.21.15.133:0/1167317442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795d4fb10 0x561795d4ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.635+0000 7fd650f9c700 1 -- 172.21.15.133:0/1167317442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 msgr2=0x561795d4f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.635+0000 7fd650f9c700 1 --2- 172.21.15.133:0/1167317442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 0x561795d4f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.635+0000 7fd650f9c700 1 -- 172.21.15.133:0/1167317442 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561795b215d0 con 0x561795baa6b0 2023-11-27T15:26:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.635+0000 7fd64b7fe700 1 --2- 172.21.15.133:0/1167317442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 0x561795d4f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:26:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.636+0000 7fd650f9c700 1 --2- 172.21.15.133:0/1167317442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795baa6b0 0x561795d4e870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd644009550 tx=0x7fd644009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6599c0178bbf3dc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.636+0000 7fd64bfff700 1 --2- 172.21.15.133:0/3146301530 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd6340a11d0 0x7fd6340a3680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd63c099ec0 tx=0x7fd63c00cf10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.636+0000 7fd6327fc700 1 -- 172.21.15.133:0/1167317442 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64400e040 con 0x561795baa6b0 2023-11-27T15:26:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.636+0000 7fd6327fc700 1 -- 172.21.15.133:0/1167317442 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd644024070 con 0x561795baa6b0 2023-11-27T15:26:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.636+0000 7fd6327fc700 1 -- 172.21.15.133:0/1167317442 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd644004020 con 0x561795baa6b0 2023-11-27T15:26:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.637+0000 7fd66636db80 1 -- 172.21.15.133:0/1167317442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795baa6b0 msgr2=0x561795d4e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.637+0000 7fd66636db80 1 --2- 172.21.15.133:0/1167317442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795baa6b0 0x561795d4e870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd644009550 tx=0x7fd644009920 comp rx=0 tx=0).stop 2023-11-27T15:26:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.637+0000 7fd66636db80 1 -- 172.21.15.133:0/1167317442 shutdown_connections 2023-11-27T15:26:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.637+0000 7fd66636db80 1 --2- 172.21.15.133:0/1167317442 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795d4fb10 0x561795d4ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.637+0000 7fd66636db80 1 --2- 172.21.15.133:0/1167317442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795baa6b0 0x561795d4e870 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.637+0000 7fd66636db80 1 --2- 172.21.15.133:0/1167317442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 0x561795d4f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.637+0000 7fd66636db80 1 -- 172.21.15.133:0/1167317442 >> 172.21.15.133:0/1167317442 conn(0x561795d43f50 msgr2=0x561795d49100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.637+0000 7fd66636db80 1 -- 172.21.15.133:0/1167317442 shutdown_connections 2023-11-27T15:26:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.637+0000 7fd66636db80 1 -- 172.21.15.133:0/1167317442 wait complete. 2023-11-27T15:26:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.637+0000 7fd66636db80 1 -- start start 2023-11-27T15:26:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.638+0000 7fd66636db80 1 Processor -- start 2023-11-27T15:26:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.638+0000 7fd66636db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795baa6b0 0x561795d57c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.638+0000 7fd66636db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 0x561795d581b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.638+0000 7fd66636db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795d4fb10 0x561795d5f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.638+0000 7fd66636db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561795c41880 con 0x561795d4edb0 2023-11-27T15:26:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.638+0000 7fd66636db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x561795c41520 con 0x561795baa6b0 2023-11-27T15:26:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.638+0000 7fd66636db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x561795b238c0 con 0x561795d4fb10 2023-11-27T15:26:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.638+0000 7fd66636db80 10 client.0 _ll_register_callbacks cb 0x561795c5f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.638+0000 7fd650f9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795baa6b0 0x561795d57c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.638+0000 7fd650f9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795baa6b0 0x561795d57c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:57580/0 (socket says 172.21.15.133:57580) 2023-11-27T15:26:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.638+0000 7fd650f9c700 1 -- 172.21.15.133:0/666232729 learned_addr learned my addr 172.21.15.133:0/666232729 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.638+0000 7fd64b7fe700 1 --2- 172.21.15.133:0/666232729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 0x561795d581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.638+0000 7fd650f9c700 1 --2- 172.21.15.133:0/666232729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795d4fb10 0x561795d5f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.639+0000 7fd650f9c700 1 -- 172.21.15.133:0/666232729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795d4fb10 msgr2=0x561795d5f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.639+0000 7fd650f9c700 1 --2- 172.21.15.133:0/666232729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795d4fb10 0x561795d5f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.639+0000 7fd650f9c700 1 -- 172.21.15.133:0/666232729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 msgr2=0x561795d581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.639+0000 7fd650f9c700 1 --2- 172.21.15.133:0/666232729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 0x561795d581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.639+0000 7fd650f9c700 1 -- 172.21.15.133:0/666232729 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd644013640 con 0x561795baa6b0 2023-11-27T15:26:09.646 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.639+0000 7fd650f9c700 1 --2- 172.21.15.133:0/666232729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795baa6b0 0x561795d57c70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd644013fd0 tx=0x7fd644013f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4db603dc9acf6c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.639+0000 7fd62b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:26:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.639+0000 7fd62b7fe700 1 -- 172.21.15.133:0/666232729 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64400e070 con 0x561795baa6b0 2023-11-27T15:26:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.639+0000 7fd66636db80 1 -- 172.21.15.133:0/666232729 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561795d41710 con 0x561795baa6b0 2023-11-27T15:26:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.639+0000 7fd62b7fe700 1 -- 172.21.15.133:0/666232729 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd644004ad0 con 0x561795baa6b0 2023-11-27T15:26:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.639+0000 7fd62b7fe700 1 -- 172.21.15.133:0/666232729 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd644004c90 con 0x561795baa6b0 2023-11-27T15:26:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.639+0000 7fd66636db80 10 client.26168 fetch_fsmap learned FSMap version 26 2023-11-27T15:26:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.639+0000 7fd66636db80 1 -- 172.21.15.133:0/666232729 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561795c66130 con 0x561795baa6b0 2023-11-27T15:26:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.640+0000 7fd62b7fe700 1 -- 172.21.15.133:0/666232729 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd644037080 con 0x561795baa6b0 2023-11-27T15:26:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.640+0000 7fd62b7fe700 1 -- 172.21.15.133:0/666232729 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 26) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd644013960 con 0x561795baa6b0 2023-11-27T15:26:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.640+0000 7fd66636db80 10 client.26168 fetch_fsmap finished waiting for FSMap version 26 2023-11-27T15:26:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.640+0000 7fd66636db80 10 client.26168 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:26:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.640+0000 7fd66636db80 20 client.26168 populate_metadata read hostname 'smithi133' 2023-11-27T15:26:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.640+0000 7fd66636db80 1 --2- 172.21.15.133:0/666232729 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x561795d63920 0x561795d65d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:09.647 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.640+0000 7fd66636db80 4 client.26168 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:26:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.640+0000 7fd66636db80 1 -- 172.21.15.133:0/666232729 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561795c62040 con 0x561795d63920 2023-11-27T15:26:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.640+0000 7fd64b7fe700 1 --2- 172.21.15.133:0/666232729 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x561795d63920 0x561795d65d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.641+0000 7fd64b7fe700 1 --2- 172.21.15.133:0/666232729 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x561795d63920 0x561795d65d30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=256dabdb70a5a9a4 server_cookie=44b260a0a3035cf5 in_seq=0 out_seq=0 2023-11-27T15:26:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.641+0000 7fd62b7fe700 10 client.26168 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.642+0000 7fd62b7fe700 1 -- 172.21.15.133:0/666232729 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561795c62040 con 0x561795d63920 2023-11-27T15:26:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.642+0000 7fd62b7fe700 10 client.26168 handle_command_reply: tid=0 2023-11-27T15:26:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.643+0000 7fd66636db80 1 client.26168 shutdown 2023-11-27T15:26:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.643+0000 7fd66636db80 1 -- 172.21.15.133:0/666232729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795baa6b0 msgr2=0x561795d57c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.643+0000 7fd66636db80 1 --2- 172.21.15.133:0/666232729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795baa6b0 0x561795d57c70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd644013fd0 tx=0x7fd644013f20 comp rx=0 tx=0).stop 2023-11-27T15:26:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.643+0000 7fd66636db80 1 -- 172.21.15.133:0/666232729 shutdown_connections 2023-11-27T15:26:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.643+0000 7fd66636db80 1 --2- 172.21.15.133:0/666232729 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x561795d63920 0x561795d65d30 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.643+0000 7fd66636db80 1 --2- 172.21.15.133:0/666232729 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795d4fb10 0x561795d5f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.643+0000 7fd62b7fe700 0 client.26168 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.643+0000 7fd66636db80 1 --2- 172.21.15.133:0/666232729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795baa6b0 0x561795d57c70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.643+0000 7fd66636db80 1 --2- 172.21.15.133:0/666232729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 0x561795d581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.643+0000 7fd66636db80 1 -- 172.21.15.133:0/666232729 >> 172.21.15.133:0/666232729 conn(0x561795d43f50 msgr2=0x561795d487e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.643+0000 7fd66636db80 1 -- 172.21.15.133:0/666232729 shutdown_connections 2023-11-27T15:26:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.643+0000 7fd66636db80 1 -- 172.21.15.133:0/666232729 wait complete. 2023-11-27T15:26:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.643+0000 7fd66636db80 20 client.26168 trim_cache size 0 max 16384 2023-11-27T15:26:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.658+0000 7fd66636db80 1 Processor -- start 2023-11-27T15:26:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.658+0000 7fd66636db80 1 -- start start 2023-11-27T15:26:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.658+0000 7fd66636db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795baa6b0 0x561795d56990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.658+0000 7fd66636db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 0x561795d5f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.658+0000 7fd66636db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795d4fb10 0x561795d52a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.658+0000 7fd66636db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561795d56750 con 0x561795d4edb0 2023-11-27T15:26:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.658+0000 7fd66636db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x561795d49ec0 con 0x561795baa6b0 2023-11-27T15:26:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.658+0000 7fd66636db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x561795d49ce0 con 0x561795d4fb10 2023-11-27T15:26:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.658+0000 7fd64b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 0x561795d5f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.659+0000 7fd650f9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795baa6b0 0x561795d56990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.659+0000 7fd64b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 0x561795d5f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:43624/0 (socket says 172.21.15.133:43624) 2023-11-27T15:26:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.659+0000 7fd64b7fe700 1 -- 172.21.15.133:0/208984393 learned_addr learned my addr 172.21.15.133:0/208984393 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.659+0000 7fd650f9c700 1 --2- 172.21.15.133:0/208984393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795d4fb10 0x561795d52a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.659+0000 7fd64b7fe700 1 -- 172.21.15.133:0/208984393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795d4fb10 msgr2=0x561795d52a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.659+0000 7fd64b7fe700 1 --2- 172.21.15.133:0/208984393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795d4fb10 0x561795d52a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.659+0000 7fd64b7fe700 1 -- 172.21.15.133:0/208984393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795baa6b0 msgr2=0x561795d56990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.659+0000 7fd64b7fe700 1 --2- 172.21.15.133:0/208984393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795baa6b0 0x561795d56990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.659+0000 7fd64b7fe700 1 -- 172.21.15.133:0/208984393 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561795c65e80 con 0x561795d4edb0 2023-11-27T15:26:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.659+0000 7fd64b7fe700 1 --2- 172.21.15.133:0/208984393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 0x561795d5f1b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd640005910 tx=0x7fd64000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6507f326490a77 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.659+0000 7fd632ffd700 1 -- 172.21.15.133:0/208984393 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64001d040 con 0x561795d4edb0 2023-11-27T15:26:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.659+0000 7fd632ffd700 1 -- 172.21.15.133:0/208984393 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd64001e040 con 0x561795d4edb0 2023-11-27T15:26:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.659+0000 7fd632ffd700 1 -- 172.21.15.133:0/208984393 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6400054e0 con 0x561795d4edb0 2023-11-27T15:26:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.660+0000 7fd66636db80 1 -- 172.21.15.133:0/208984393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 msgr2=0x561795d5f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.660+0000 7fd66636db80 1 --2- 172.21.15.133:0/208984393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 0x561795d5f1b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd640005910 tx=0x7fd64000b260 comp rx=0 tx=0).stop 2023-11-27T15:26:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.660+0000 7fd66636db80 1 -- 172.21.15.133:0/208984393 shutdown_connections 2023-11-27T15:26:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.660+0000 7fd66636db80 1 --2- 172.21.15.133:0/208984393 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795d4fb10 0x561795d52a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.660+0000 7fd66636db80 1 --2- 172.21.15.133:0/208984393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795baa6b0 0x561795d56990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.660+0000 7fd66636db80 1 --2- 172.21.15.133:0/208984393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 0x561795d5f1b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.660+0000 7fd66636db80 1 -- 172.21.15.133:0/208984393 >> 172.21.15.133:0/208984393 conn(0x561795d43f50 msgr2=0x561795d6a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.660+0000 7fd66636db80 1 -- 172.21.15.133:0/208984393 shutdown_connections 2023-11-27T15:26:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.660+0000 7fd66636db80 1 -- 172.21.15.133:0/208984393 wait complete. 2023-11-27T15:26:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.660+0000 7fd66636db80 1 -- start start 2023-11-27T15:26:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.661+0000 7fd66636db80 1 Processor -- start 2023-11-27T15:26:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.661+0000 7fd66636db80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795baa6b0 0x561795d5b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.661+0000 7fd66636db80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 0x561795d5b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.661+0000 7fd66636db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795d4fb10 0x561795d5e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.661+0000 7fd66636db80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x561795d5ec40 con 0x561795d4edb0 2023-11-27T15:26:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.661+0000 7fd66636db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x561795d5edb0 con 0x561795d4fb10 2023-11-27T15:26:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.661+0000 7fd66636db80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x561795d5ef70 con 0x561795baa6b0 2023-11-27T15:26:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.661+0000 7fd66636db80 10 client.0 _ll_register_callbacks cb 0x7fd644013860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.661+0000 7fd650f9c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795baa6b0 0x561795d5b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.661+0000 7fd650f9c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795baa6b0 0x561795d5b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:43212/0 (socket says 172.21.15.133:43212) 2023-11-27T15:26:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.661+0000 7fd650f9c700 1 -- 172.21.15.133:0/2207664603 learned_addr learned my addr 172.21.15.133:0/2207664603 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.661+0000 7fd650f9c700 1 --2- 172.21.15.133:0/2207664603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795d4fb10 0x561795d5e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.661+0000 7fd64b7fe700 1 --2- 172.21.15.133:0/2207664603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 0x561795d5b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.662+0000 7fd650f9c700 1 -- 172.21.15.133:0/2207664603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795d4fb10 msgr2=0x561795d5e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.662+0000 7fd650f9c700 1 --2- 172.21.15.133:0/2207664603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795d4fb10 0x561795d5e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.662+0000 7fd650f9c700 1 -- 172.21.15.133:0/2207664603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 msgr2=0x561795d5b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.662+0000 7fd650f9c700 1 --2- 172.21.15.133:0/2207664603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 0x561795d5b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.662+0000 7fd650f9c700 1 -- 172.21.15.133:0/2207664603 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64000bbd0 con 0x561795baa6b0 2023-11-27T15:26:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.662+0000 7fd650f9c700 1 --2- 172.21.15.133:0/2207664603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795baa6b0 0x561795d5b1a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd644029ee0 tx=0x7fd644009dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36c1fd8b98d2684a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.662+0000 7fd62b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:26:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.662+0000 7fd62b7fe700 1 -- 172.21.15.133:0/2207664603 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64400e020 con 0x561795baa6b0 2023-11-27T15:26:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.662+0000 7fd66636db80 1 -- 172.21.15.133:0/2207664603 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd644024070 con 0x561795baa6b0 2023-11-27T15:26:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.662+0000 7fd62b7fe700 1 -- 172.21.15.133:0/2207664603 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd64403d040 con 0x561795baa6b0 2023-11-27T15:26:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.662+0000 7fd62b7fe700 1 -- 172.21.15.133:0/2207664603 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd644006000 con 0x561795baa6b0 2023-11-27T15:26:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.662+0000 7fd66636db80 10 client.26155 fetch_fsmap learned FSMap version 26 2023-11-27T15:26:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.662+0000 7fd66636db80 1 -- 172.21.15.133:0/2207664603 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561795ab6c00 con 0x561795baa6b0 2023-11-27T15:26:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.663+0000 7fd62b7fe700 1 -- 172.21.15.133:0/2207664603 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd644024070 con 0x561795baa6b0 2023-11-27T15:26:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.663+0000 7fd62b7fe700 1 -- 172.21.15.133:0/2207664603 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 26) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd6440078a0 con 0x561795baa6b0 2023-11-27T15:26:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.663+0000 7fd66636db80 10 client.26155 fetch_fsmap finished waiting for FSMap version 26 2023-11-27T15:26:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.663+0000 7fd66636db80 10 client.26155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:26:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.663+0000 7fd66636db80 20 client.26155 populate_metadata read hostname 'smithi133' 2023-11-27T15:26:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.663+0000 7fd66636db80 1 --2- 172.21.15.133:0/2207664603 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x561795db8610 0x561795dbaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.663+0000 7fd66636db80 4 client.26155 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:26:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.663+0000 7fd66636db80 1 -- 172.21.15.133:0/2207664603 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561795c6e600 con 0x561795db8610 2023-11-27T15:26:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.663+0000 7fd64b7fe700 1 --2- 172.21.15.133:0/2207664603 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x561795db8610 0x561795dbaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.664+0000 7fd64b7fe700 1 --2- 172.21.15.133:0/2207664603 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x561795db8610 0x561795dbaa20 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cf09eb25ec59336 server_cookie=4c32f40099981e67 in_seq=0 out_seq=0 2023-11-27T15:26:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:09.664+0000 7fd62b7fe700 10 client.26155 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:11.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:10 smithi174 ceph-mon[131110]: pgmap v446: 129 pgs: 129 active+clean; 83 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 835 op/s 2023-11-27T15:26:11.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:10 smithi133 ceph-mon[97860]: pgmap v446: 129 pgs: 129 active+clean; 83 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 835 op/s 2023-11-27T15:26:11.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:10 smithi177 ceph-mon[131504]: pgmap v446: 129 pgs: 129 active+clean; 83 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 835 op/s 2023-11-27T15:26:12.089 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.084+0000 7fd62b7fe700 1 -- 172.21.15.133:0/2207664603 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561795c6e600 con 0x561795db8610 2023-11-27T15:26:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.084+0000 7fd62b7fe700 10 client.26155 handle_command_reply: tid=0 2023-11-27T15:26:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.085+0000 7fd66636db80 1 client.26155 shutdown 2023-11-27T15:26:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.085+0000 7fd66636db80 1 -- 172.21.15.133:0/2207664603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795baa6b0 msgr2=0x561795d5b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.085+0000 7fd66636db80 1 --2- 172.21.15.133:0/2207664603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795baa6b0 0x561795d5b1a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd644029ee0 tx=0x7fd644009dd0 comp rx=0 tx=0).stop 2023-11-27T15:26:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.085+0000 7fd66636db80 1 -- 172.21.15.133:0/2207664603 shutdown_connections 2023-11-27T15:26:12.090 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.085+0000 7fd66636db80 1 --2- 172.21.15.133:0/2207664603 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x561795db8610 0x561795dbaa20 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.085+0000 7fd66636db80 1 --2- 172.21.15.133:0/2207664603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561795d4fb10 0x561795d5e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.085+0000 7fd66636db80 1 --2- 172.21.15.133:0/2207664603 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x561795baa6b0 0x561795d5b1a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.085+0000 7fd66636db80 1 --2- 172.21.15.133:0/2207664603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561795d4edb0 0x561795d5b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.085+0000 7fd62b7fe700 0 client.26155 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.085+0000 7fd66636db80 1 -- 172.21.15.133:0/2207664603 >> 172.21.15.133:0/2207664603 conn(0x561795d43f50 msgr2=0x561795d56990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.085+0000 7fd66636db80 1 -- 172.21.15.133:0/2207664603 shutdown_connections 2023-11-27T15:26:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.085+0000 7fd66636db80 1 -- 172.21.15.133:0/2207664603 wait complete. 2023-11-27T15:26:12.091 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.085+0000 7fd66636db80 20 client.26155 trim_cache size 0 max 16384 2023-11-27T15:26:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:26:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:26:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:26:12.092 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:26:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.086+0000 7fd6529ff700 1 -- 172.21.15.133:0/3146301530 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd6340a11d0 msgr2=0x7fd6340a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:12.093 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.086+0000 7fd6529ff700 1 --2- 172.21.15.133:0/3146301530 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd6340a11d0 0x7fd6340a3680 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd63c099ec0 tx=0x7fd63c00cf10 comp rx=0 tx=0).stop 2023-11-27T15:26:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.086+0000 7fd6529ff700 1 -- 172.21.15.133:0/3146301530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd64c100240 msgr2=0x7fd64c198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.086+0000 7fd6529ff700 1 --2- 172.21.15.133:0/3146301530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd64c100240 0x7fd64c198190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00e9c0 tx=0x7fd63c00ecd0 comp rx=0 tx=0).stop 2023-11-27T15:26:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.086+0000 7fd6529ff700 1 -- 172.21.15.133:0/3146301530 shutdown_connections 2023-11-27T15:26:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.086+0000 7fd6529ff700 1 --2- 172.21.15.133:0/3146301530 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd6340a11d0 0x7fd6340a3680 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.086+0000 7fd6529ff700 1 --2- 172.21.15.133:0/3146301530 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd64c101dc0 0x7fd64c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.086+0000 7fd6529ff700 1 --2- 172.21.15.133:0/3146301530 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd64c100240 0x7fd64c198190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.086+0000 7fd6529ff700 1 --2- 172.21.15.133:0/3146301530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd64c101430 0x7fd64c1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.086+0000 7fd6529ff700 1 -- 172.21.15.133:0/3146301530 >> 172.21.15.133:0/3146301530 conn(0x7fd64c0fb840 msgr2=0x7fd64c0fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.086+0000 7fd6529ff700 1 -- 172.21.15.133:0/3146301530 shutdown_connections 2023-11-27T15:26:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:12.086+0000 7fd6529ff700 1 -- 172.21.15.133:0/3146301530 wait complete. 2023-11-27T15:26:13.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:12 smithi174 ceph-mon[131110]: pgmap v447: 129 pgs: 129 active+clean; 96 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2023-11-27T15:26:13.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:12 smithi133 ceph-mon[97860]: pgmap v447: 129 pgs: 129 active+clean; 96 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2023-11-27T15:26:13.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:12 smithi177 ceph-mon[131504]: pgmap v447: 129 pgs: 129 active+clean; 96 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2023-11-27T15:26:14.508 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:26:14.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.769+0000 7f76fdbff700 1 -- 172.21.15.133:0/1106234153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76f8100d40 msgr2=0x7f76f8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:14.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.769+0000 7f76fdbff700 1 --2- 172.21.15.133:0/1106234153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76f8100d40 0x7f76f8103120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f76e8009990 tx=0x7f76e8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.769+0000 7f76fdbff700 1 -- 172.21.15.133:0/1106234153 shutdown_connections 2023-11-27T15:26:14.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.769+0000 7f76fdbff700 1 --2- 172.21.15.133:0/1106234153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f8105f80 0x7f76f81083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.769+0000 7f76fdbff700 1 --2- 172.21.15.133:0/1106234153 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76f8100d40 0x7f76f8103120 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.769+0000 7f76fdbff700 1 --2- 172.21.15.133:0/1106234153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76f8103660 0x7f76f8105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.769+0000 7f76fdbff700 1 -- 172.21.15.133:0/1106234153 >> 172.21.15.133:0/1106234153 conn(0x7f76f80fa7c0 msgr2=0x7f76f80fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:14.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.770+0000 7f76fdbff700 1 -- 172.21.15.133:0/1106234153 shutdown_connections 2023-11-27T15:26:14.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.770+0000 7f76fdbff700 1 -- 172.21.15.133:0/1106234153 wait complete. 2023-11-27T15:26:14.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.771+0000 7f76fdbff700 1 Processor -- start 2023-11-27T15:26:14.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.772+0000 7f76fdbff700 1 -- start start 2023-11-27T15:26:14.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.772+0000 7f76fdbff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76f8100d40 0x7f76f81982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:14.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.772+0000 7f76fdbff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76f8103660 0x7f76f8198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:14.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.772+0000 7f76fdbff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f8105f80 0x7f76f819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:14.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.772+0000 7f76fdbff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f819fe70 con 0x7f76f8103660 2023-11-27T15:26:14.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.772+0000 7f76fdbff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f806a510 con 0x7f76f8105f80 2023-11-27T15:26:14.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.772+0000 7f76fdbff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f806a6a0 con 0x7f76f8100d40 2023-11-27T15:26:14.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.772+0000 7f76f77fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76f8100d40 0x7f76f81982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:14.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.772+0000 7f76f7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f8105f80 0x7f76f819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:14.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.772+0000 7f76f77fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76f8100d40 0x7f76f81982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:39544/0 (socket says 172.21.15.133:39544) 2023-11-27T15:26:14.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.772+0000 7f76f77fe700 1 -- 172.21.15.133:0/144059121 learned_addr learned my addr 172.21.15.133:0/144059121 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:14.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.772+0000 7f76f6ffd700 1 --2- 172.21.15.133:0/144059121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76f8103660 0x7f76f8198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:14.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.773+0000 7f76f77fe700 1 -- 172.21.15.133:0/144059121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f8105f80 msgr2=0x7f76f819f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:14.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.773+0000 7f76f77fe700 1 --2- 172.21.15.133:0/144059121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f8105f80 0x7f76f819f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.773+0000 7f76f77fe700 1 -- 172.21.15.133:0/144059121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76f8103660 msgr2=0x7f76f8198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:14.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.773+0000 7f76f77fe700 1 --2- 172.21.15.133:0/144059121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76f8103660 0x7f76f8198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.773+0000 7f76f77fe700 1 -- 172.21.15.133:0/144059121 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e80095f0 con 0x7f76f8100d40 2023-11-27T15:26:14.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.773+0000 7f76f77fe700 1 --2- 172.21.15.133:0/144059121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76f8100d40 0x7f76f81982f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f76e800b5c0 tx=0x7f76e8003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:14.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.773+0000 7f76f4ff9700 1 -- 172.21.15.133:0/144059121 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e801d070 con 0x7f76f8100d40 2023-11-27T15:26:14.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.773+0000 7f76f4ff9700 1 -- 172.21.15.133:0/144059121 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76e800f460 con 0x7f76f8100d40 2023-11-27T15:26:14.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.773+0000 7f76f4ff9700 1 -- 172.21.15.133:0/144059121 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e8003ea0 con 0x7f76f8100d40 2023-11-27T15:26:14.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.773+0000 7f76fdbff700 1 -- 172.21.15.133:0/144059121 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f810bcf0 con 0x7f76f8100d40 2023-11-27T15:26:14.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.774+0000 7f76fdbff700 1 -- 172.21.15.133:0/144059121 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f810bb60 con 0x7f76f8100d40 2023-11-27T15:26:14.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.777+0000 7f76f4ff9700 1 -- 172.21.15.133:0/144059121 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f76e800f5d0 con 0x7f76f8100d40 2023-11-27T15:26:14.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.780+0000 7f771159cb80 1 Processor -- start 2023-11-27T15:26:14.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.780+0000 7f771159cb80 1 -- start start 2023-11-27T15:26:14.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.780+0000 7f76f4ff9700 1 --2- 172.21.15.133:0/144059121 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f76e00a11d0 0x7f76e00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:14.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.780+0000 7f76f4ff9700 1 -- 172.21.15.133:0/144059121 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f76e80cded0 con 0x7f76f8100d40 2023-11-27T15:26:14.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.781+0000 7f771159cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd7466fc00 0x55fd74670010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.781+0000 7f771159cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74670550 0x55fd74670b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.781+0000 7f771159cb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd74671110 0x55fd74671580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.781+0000 7f771159cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd74575060 con 0x55fd7466fc00 2023-11-27T15:26:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.781+0000 7f771159cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd745c78a0 con 0x55fd74670550 2023-11-27T15:26:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.781+0000 7f771159cb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd745c0d30 con 0x55fd74671110 2023-11-27T15:26:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.781+0000 7f76f7fff700 1 --2- 172.21.15.133:0/144059121 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f76e00a11d0 0x7f76e00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:14.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.781+0000 7f76f7fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd74671110 0x55fd74671580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:14.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.781+0000 7f76f6ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74670550 0x55fd74670b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:14.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.781+0000 7f76f7fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd74671110 0x55fd74671580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:39558/0 (socket says 172.21.15.133:39558) 2023-11-27T15:26:14.786 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.781+0000 7f76f7fff700 1 -- 172.21.15.133:0/2167389783 learned_addr learned my addr 172.21.15.133:0/2167389783 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:14.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.782+0000 7f76f77fe700 1 --2- 172.21.15.133:0/2167389783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd7466fc00 0x55fd74670010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:14.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.782+0000 7f76f7fff700 1 -- 172.21.15.133:0/2167389783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74670550 msgr2=0x55fd74670b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:14.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.782+0000 7f76f7fff700 1 --2- 172.21.15.133:0/2167389783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74670550 0x55fd74670b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.782+0000 7f76f7fff700 1 -- 172.21.15.133:0/2167389783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd7466fc00 msgr2=0x55fd74670010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:14.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.782+0000 7f76f7fff700 1 --2- 172.21.15.133:0/2167389783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd7466fc00 0x55fd74670010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.787 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.782+0000 7f76f7fff700 1 -- 172.21.15.133:0/2167389783 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd74442630 con 0x55fd74671110 2023-11-27T15:26:14.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.782+0000 7f76f77fe700 1 --2- 172.21.15.133:0/2167389783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd7466fc00 0x55fd74670010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:26:14.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.782+0000 7f76f6ffd700 1 --2- 172.21.15.133:0/2167389783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74670550 0x55fd74670b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:14.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.782+0000 7f76f7fff700 1 --2- 172.21.15.133:0/144059121 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f76e00a11d0 0x7f76e00a3680 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f76f000adc0 tx=0x7f76f000a400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:14.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.782+0000 7f76f7fff700 1 --2- 172.21.15.133:0/2167389783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd74671110 0x55fd74671580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f76f0007170 tx=0x7f76f001a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0cc3957fc641283 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:14.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.783+0000 7f76ddffb700 1 -- 172.21.15.133:0/2167389783 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f00099d0 con 0x55fd74671110 2023-11-27T15:26:14.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.783+0000 7f76ddffb700 1 -- 172.21.15.133:0/2167389783 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76f0019660 con 0x55fd74671110 2023-11-27T15:26:14.788 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.783+0000 7f76ddffb700 1 -- 172.21.15.133:0/2167389783 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f001a650 con 0x55fd74671110 2023-11-27T15:26:14.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.784+0000 7f771159cb80 1 -- 172.21.15.133:0/2167389783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd74671110 msgr2=0x55fd74671580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:14.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.784+0000 7f771159cb80 1 --2- 172.21.15.133:0/2167389783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd74671110 0x55fd74671580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f76f0007170 tx=0x7f76f001a000 comp rx=0 tx=0).stop 2023-11-27T15:26:14.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.785+0000 7f771159cb80 1 -- 172.21.15.133:0/2167389783 shutdown_connections 2023-11-27T15:26:14.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.785+0000 7f771159cb80 1 --2- 172.21.15.133:0/2167389783 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd74671110 0x55fd74671580 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.785+0000 7f771159cb80 1 --2- 172.21.15.133:0/2167389783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd7466fc00 0x55fd74670010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.785+0000 7f771159cb80 1 --2- 172.21.15.133:0/2167389783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74670550 0x55fd74670b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.785+0000 7f771159cb80 1 -- 172.21.15.133:0/2167389783 >> 172.21.15.133:0/2167389783 conn(0x55fd74665360 msgr2=0x55fd7466a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:14.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.785+0000 7f771159cb80 1 -- 172.21.15.133:0/2167389783 shutdown_connections 2023-11-27T15:26:14.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.785+0000 7f771159cb80 1 -- 172.21.15.133:0/2167389783 wait complete. 2023-11-27T15:26:14.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.786+0000 7f771159cb80 1 -- start start 2023-11-27T15:26:14.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.786+0000 7f771159cb80 1 Processor -- start 2023-11-27T15:26:14.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.787+0000 7f771159cb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd7466fc00 0x55fd746795d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:14.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.787+0000 7f771159cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74670550 0x55fd74679b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:14.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.787+0000 7f771159cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd74671110 0x55fd74680b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:14.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.787+0000 7f771159cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd745628e0 con 0x55fd74671110 2023-11-27T15:26:14.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.787+0000 7f771159cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd74562580 con 0x55fd74670550 2023-11-27T15:26:14.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.787+0000 7f771159cb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd74444920 con 0x55fd7466fc00 2023-11-27T15:26:14.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.787+0000 7f771159cb80 10 client.0 _ll_register_callbacks cb 0x55fd74580780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:14.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.787+0000 7f76f6ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74670550 0x55fd74679b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:14.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.787+0000 7f76f6ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74670550 0x55fd74679b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:36534/0 (socket says 172.21.15.133:36534) 2023-11-27T15:26:14.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.787+0000 7f76f6ffd700 1 -- 172.21.15.133:0/35864912 learned_addr learned my addr 172.21.15.133:0/35864912 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:14.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.788+0000 7f76f6ffd700 1 -- 172.21.15.133:0/35864912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd7466fc00 msgr2=0x55fd746795d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-27T15:26:14.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.788+0000 7f76f6ffd700 1 --2- 172.21.15.133:0/35864912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd7466fc00 0x55fd746795d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.788+0000 7f76f6ffd700 1 -- 172.21.15.133:0/35864912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd74671110 msgr2=0x55fd74680b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-27T15:26:14.794 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.788+0000 7f76f7fff700 1 --2- 172.21.15.133:0/35864912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd74671110 0x55fd74680b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:14.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.788+0000 7f76f6ffd700 1 --2- 172.21.15.133:0/35864912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd74671110 0x55fd74680b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.788+0000 7f76f6ffd700 1 -- 172.21.15.133:0/35864912 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f0017a70 con 0x55fd74670550 2023-11-27T15:26:14.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.788+0000 7f76f7fff700 1 --2- 172.21.15.133:0/35864912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd74671110 0x55fd74680b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:26:14.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.788+0000 7f76f6ffd700 1 --2- 172.21.15.133:0/35864912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74670550 0x55fd74679b10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f76ec010810 tx=0x7f76ec010bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cb2237b93a82bf0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:14.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.788+0000 7f76d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:26:14.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.789+0000 7f76d6ffd700 1 -- 172.21.15.133:0/35864912 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec004a30 con 0x55fd74670550 2023-11-27T15:26:14.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.789+0000 7f76d6ffd700 1 -- 172.21.15.133:0/35864912 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f76ec017070 con 0x55fd74670550 2023-11-27T15:26:14.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.789+0000 7f771159cb80 1 -- 172.21.15.133:0/35864912 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd74662b20 con 0x55fd74670550 2023-11-27T15:26:14.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.789+0000 7f76d6ffd700 1 -- 172.21.15.133:0/35864912 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76ec004b90 con 0x55fd74670550 2023-11-27T15:26:14.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.789+0000 7f771159cb80 10 client.26183 fetch_fsmap learned FSMap version 26 2023-11-27T15:26:14.795 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.789+0000 7f771159cb80 1 -- 172.21.15.133:0/35864912 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fd74587190 con 0x55fd74670550 2023-11-27T15:26:14.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.790+0000 7f76d6ffd700 1 -- 172.21.15.133:0/35864912 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fd74662b20 con 0x55fd74670550 2023-11-27T15:26:14.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.790+0000 7f76d6ffd700 1 -- 172.21.15.133:0/35864912 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 26) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f76ec005390 con 0x55fd74670550 2023-11-27T15:26:14.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.790+0000 7f771159cb80 10 client.26183 fetch_fsmap finished waiting for FSMap version 26 2023-11-27T15:26:14.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.790+0000 7f771159cb80 10 client.26183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:26:14.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.790+0000 7f771159cb80 20 client.26183 populate_metadata read hostname 'smithi133' 2023-11-27T15:26:14.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.790+0000 7f771159cb80 1 --2- 172.21.15.133:0/35864912 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55fd746854b0 0x55fd746878c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:14.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.790+0000 7f771159cb80 4 client.26183 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:26:14.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.790+0000 7f771159cb80 1 -- 172.21.15.133:0/35864912 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd745830a0 con 0x55fd746854b0 2023-11-27T15:26:14.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.790+0000 7f76f77fe700 1 --2- 172.21.15.133:0/35864912 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55fd746854b0 0x55fd746878c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:14.796 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.791+0000 7f76f77fe700 1 --2- 172.21.15.133:0/35864912 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55fd746854b0 0x55fd746878c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72e47b0c21047390 server_cookie=a560dd215f2a855d in_seq=0 out_seq=0 2023-11-27T15:26:14.797 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.791+0000 7f76d6ffd700 10 client.26183 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:14.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.793+0000 7f76d6ffd700 1 -- 172.21.15.133:0/35864912 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fd745830a0 con 0x55fd746854b0 2023-11-27T15:26:14.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.793+0000 7f76d6ffd700 10 client.26183 handle_command_reply: tid=0 2023-11-27T15:26:14.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.794+0000 7f771159cb80 1 client.26183 shutdown 2023-11-27T15:26:14.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.794+0000 7f771159cb80 1 -- 172.21.15.133:0/35864912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74670550 msgr2=0x55fd74679b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:14.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.794+0000 7f771159cb80 1 --2- 172.21.15.133:0/35864912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74670550 0x55fd74679b10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f76ec010810 tx=0x7f76ec010bd0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.798 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.794+0000 7f771159cb80 1 -- 172.21.15.133:0/35864912 shutdown_connections 2023-11-27T15:26:14.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.794+0000 7f771159cb80 1 --2- 172.21.15.133:0/35864912 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55fd746854b0 0x55fd746878c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.794+0000 7f771159cb80 1 --2- 172.21.15.133:0/35864912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd74671110 0x55fd74680b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.794+0000 7f771159cb80 1 --2- 172.21.15.133:0/35864912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd7466fc00 0x55fd746795d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.794+0000 7f771159cb80 1 --2- 172.21.15.133:0/35864912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74670550 0x55fd74679b10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.794+0000 7f771159cb80 1 -- 172.21.15.133:0/35864912 >> 172.21.15.133:0/35864912 conn(0x55fd74665360 msgr2=0x55fd74669de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:14.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.794+0000 7f76d6ffd700 0 client.26183 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:14.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.794+0000 7f771159cb80 1 -- 172.21.15.133:0/35864912 shutdown_connections 2023-11-27T15:26:14.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.794+0000 7f771159cb80 1 -- 172.21.15.133:0/35864912 wait complete. 2023-11-27T15:26:14.799 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.795+0000 7f771159cb80 20 client.26183 trim_cache size 0 max 16384 2023-11-27T15:26:14.821 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.817+0000 7f771159cb80 1 Processor -- start 2023-11-27T15:26:14.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.817+0000 7f771159cb80 1 -- start start 2023-11-27T15:26:14.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.818+0000 7f771159cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd7466fc00 0x55fd74677670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:14.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.818+0000 7f771159cb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd74670550 0x55fd74680a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:14.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.818+0000 7f771159cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74671110 0x55fd746874c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:14.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.818+0000 7f771159cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd746791a0 con 0x55fd7466fc00 2023-11-27T15:26:14.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.818+0000 7f771159cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd74662660 con 0x55fd74671110 2023-11-27T15:26:14.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.818+0000 7f771159cb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd745830a0 con 0x55fd74670550 2023-11-27T15:26:14.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.818+0000 7f76f6ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd74670550 0x55fd74680a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:14.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.818+0000 7f76f7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74671110 0x55fd746874c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:14.822 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.818+0000 7f76f77fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd7466fc00 0x55fd74677670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:14.823 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.818+0000 7f76f6ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd74670550 0x55fd74680a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:39570/0 (socket says 172.21.15.133:39570) 2023-11-27T15:26:14.823 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.818+0000 7f76f6ffd700 1 -- 172.21.15.133:0/1736846128 learned_addr learned my addr 172.21.15.133:0/1736846128 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:14.823 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.818+0000 7f76f7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74671110 0x55fd746874c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:36546/0 (socket says 172.21.15.133:36546) 2023-11-27T15:26:14.824 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.818+0000 7f76f7fff700 1 -- 172.21.15.133:0/1736846128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd74670550 msgr2=0x55fd74680a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:14.824 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.818+0000 7f76f7fff700 1 --2- 172.21.15.133:0/1736846128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd74670550 0x55fd74680a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.824 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.819+0000 7f76f7fff700 1 -- 172.21.15.133:0/1736846128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd7466fc00 msgr2=0x55fd74677670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:14.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.819+0000 7f76f7fff700 1 --2- 172.21.15.133:0/1736846128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd7466fc00 0x55fd74677670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.819+0000 7f76f7fff700 1 -- 172.21.15.133:0/1736846128 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd74586ee0 con 0x55fd74671110 2023-11-27T15:26:14.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.819+0000 7f76f6ffd700 1 --2- 172.21.15.133:0/1736846128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd74670550 0x55fd74680a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:26:14.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.819+0000 7f76f7fff700 1 --2- 172.21.15.133:0/1736846128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74671110 0x55fd746874c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009720 tx=0x7f76f00178e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dccb9009fd9bffb server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:14.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.819+0000 7f76de7fc700 1 -- 172.21.15.133:0/1736846128 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f001a3f0 con 0x55fd74671110 2023-11-27T15:26:14.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.819+0000 7f76de7fc700 1 -- 172.21.15.133:0/1736846128 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76f0032070 con 0x55fd74671110 2023-11-27T15:26:14.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.819+0000 7f76de7fc700 1 -- 172.21.15.133:0/1736846128 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f00197d0 con 0x55fd74671110 2023-11-27T15:26:14.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.820+0000 7f771159cb80 1 -- 172.21.15.133:0/1736846128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74671110 msgr2=0x55fd746874c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:14.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.820+0000 7f771159cb80 1 --2- 172.21.15.133:0/1736846128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74671110 0x55fd746874c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009720 tx=0x7f76f00178e0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.820+0000 7f771159cb80 1 -- 172.21.15.133:0/1736846128 shutdown_connections 2023-11-27T15:26:14.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.820+0000 7f771159cb80 1 --2- 172.21.15.133:0/1736846128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74671110 0x55fd746874c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.820+0000 7f771159cb80 1 --2- 172.21.15.133:0/1736846128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd7466fc00 0x55fd74677670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.820+0000 7f771159cb80 1 --2- 172.21.15.133:0/1736846128 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd74670550 0x55fd74680a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.820+0000 7f771159cb80 1 -- 172.21.15.133:0/1736846128 >> 172.21.15.133:0/1736846128 conn(0x55fd74665360 msgr2=0x55fd746782e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:14.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.820+0000 7f771159cb80 1 -- 172.21.15.133:0/1736846128 shutdown_connections 2023-11-27T15:26:14.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.820+0000 7f771159cb80 1 -- 172.21.15.133:0/1736846128 wait complete. 2023-11-27T15:26:14.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.821+0000 7f771159cb80 1 -- start start 2023-11-27T15:26:14.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.821+0000 7f771159cb80 1 Processor -- start 2023-11-27T15:26:14.829 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.821+0000 7f771159cb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd7466fc00 0x55fd7467c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:14.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.821+0000 7f771159cb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd74670550 0x55fd7467c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:14.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.821+0000 7f771159cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74671110 0x55fd7467f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:14.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.821+0000 7f771159cb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7467fde0 con 0x55fd7466fc00 2023-11-27T15:26:14.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.821+0000 7f771159cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd7467ff50 con 0x55fd74671110 2023-11-27T15:26:14.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.821+0000 7f771159cb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd74680110 con 0x55fd74670550 2023-11-27T15:26:14.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.821+0000 7f771159cb80 10 client.0 _ll_register_callbacks cb 0x7f76ec008c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:14.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.822+0000 7f76f6ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd74670550 0x55fd7467c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:14.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.822+0000 7f76f6ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd74670550 0x55fd7467c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:39576/0 (socket says 172.21.15.133:39576) 2023-11-27T15:26:14.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.822+0000 7f76f7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74671110 0x55fd7467f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:14.830 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.822+0000 7f76f6ffd700 1 -- 172.21.15.133:0/1897988905 learned_addr learned my addr 172.21.15.133:0/1897988905 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:14.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.822+0000 7f76f77fe700 1 --2- 172.21.15.133:0/1897988905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd7466fc00 0x55fd7467c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:14.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.822+0000 7f76f7fff700 1 -- 172.21.15.133:0/1897988905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd74670550 msgr2=0x55fd7467c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:14.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.822+0000 7f76f7fff700 1 --2- 172.21.15.133:0/1897988905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd74670550 0x55fd7467c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.822+0000 7f76f7fff700 1 -- 172.21.15.133:0/1897988905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd7466fc00 msgr2=0x55fd7467c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:14.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.822+0000 7f76f7fff700 1 --2- 172.21.15.133:0/1897988905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd7466fc00 0x55fd7467c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.822+0000 7f76f7fff700 1 -- 172.21.15.133:0/1897988905 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd74587190 con 0x55fd74671110 2023-11-27T15:26:14.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.822+0000 7f76f6ffd700 1 --2- 172.21.15.133:0/1897988905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd74670550 0x55fd7467c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:26:14.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.822+0000 7f76f77fe700 1 --2- 172.21.15.133:0/1897988905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd7466fc00 0x55fd7467c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:26:14.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.822+0000 7f76f7fff700 1 --2- 172.21.15.133:0/1897988905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74671110 0x55fd7467f8a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f76f0000f80 tx=0x7f76f0026ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39105540da59a59f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:14.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.822+0000 7f76d6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:26:14.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.823+0000 7f76d6ffd700 1 -- 172.21.15.133:0/1897988905 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f001a5f0 con 0x55fd74671110 2023-11-27T15:26:14.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.823+0000 7f771159cb80 1 -- 172.21.15.133:0/1897988905 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76f0019660 con 0x55fd74671110 2023-11-27T15:26:14.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.823+0000 7f76d6ffd700 1 -- 172.21.15.133:0/1897988905 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f76f0034070 con 0x55fd74671110 2023-11-27T15:26:14.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.823+0000 7f76d6ffd700 1 -- 172.21.15.133:0/1897988905 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76f0003aa0 con 0x55fd74671110 2023-11-27T15:26:14.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.823+0000 7f771159cb80 10 client.26189 fetch_fsmap learned FSMap version 26 2023-11-27T15:26:14.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.823+0000 7f771159cb80 1 -- 172.21.15.133:0/1897988905 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fd74442630 con 0x55fd74671110 2023-11-27T15:26:14.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.823+0000 7f76d6ffd700 1 -- 172.21.15.133:0/1897988905 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76f0019660 con 0x55fd74671110 2023-11-27T15:26:14.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.823+0000 7f76d6ffd700 1 -- 172.21.15.133:0/1897988905 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 26) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f76f0019a30 con 0x55fd74671110 2023-11-27T15:26:14.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.823+0000 7f771159cb80 10 client.26189 fetch_fsmap finished waiting for FSMap version 26 2023-11-27T15:26:14.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.823+0000 7f771159cb80 10 client.26189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:26:14.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.823+0000 7f771159cb80 20 client.26189 populate_metadata read hostname 'smithi133' 2023-11-27T15:26:14.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.823+0000 7f771159cb80 1 --2- 172.21.15.133:0/1897988905 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55fd746d9ad0 0x55fd746dbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:14.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.824+0000 7f771159cb80 4 client.26189 mds_command: new command op to 24560 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:26:14.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.824+0000 7f771159cb80 1 -- 172.21.15.133:0/1897988905 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fd7458f660 con 0x55fd746d9ad0 2023-11-27T15:26:14.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.824+0000 7f76f77fe700 1 --2- 172.21.15.133:0/1897988905 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55fd746d9ad0 0x55fd746dbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:14.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.824+0000 7f76f77fe700 1 --2- 172.21.15.133:0/1897988905 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55fd746d9ad0 0x55fd746dbee0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3329273cd91c21 server_cookie=b619460312248dd8 in_seq=0 out_seq=0 2023-11-27T15:26:14.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.824+0000 7f76d6ffd700 10 client.26189 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:14.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.825+0000 7f76d6ffd700 1 -- 172.21.15.133:0/1897988905 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55fd7458f660 con 0x55fd746d9ad0 2023-11-27T15:26:14.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.825+0000 7f76d6ffd700 10 client.26189 handle_command_reply: tid=0 2023-11-27T15:26:14.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.825+0000 7f771159cb80 1 client.26189 shutdown 2023-11-27T15:26:14.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.825+0000 7f771159cb80 1 -- 172.21.15.133:0/1897988905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74671110 msgr2=0x55fd7467f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:14.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.825+0000 7f771159cb80 1 --2- 172.21.15.133:0/1897988905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74671110 0x55fd7467f8a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f76f0000f80 tx=0x7f76f0026ab0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.826+0000 7f771159cb80 1 -- 172.21.15.133:0/1897988905 shutdown_connections 2023-11-27T15:26:14.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.826+0000 7f771159cb80 1 --2- 172.21.15.133:0/1897988905 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55fd746d9ad0 0x55fd746dbee0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.826+0000 7f771159cb80 1 --2- 172.21.15.133:0/1897988905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fd74671110 0x55fd7467f8a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.826+0000 7f771159cb80 1 --2- 172.21.15.133:0/1897988905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd7466fc00 0x55fd7467c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.826+0000 7f76d6ffd700 0 client.26189 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:14.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.826+0000 7f771159cb80 1 --2- 172.21.15.133:0/1897988905 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55fd74670550 0x55fd7467c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.826+0000 7f771159cb80 1 -- 172.21.15.133:0/1897988905 >> 172.21.15.133:0/1897988905 conn(0x55fd74665360 msgr2=0x55fd74677670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:14.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.826+0000 7f771159cb80 1 -- 172.21.15.133:0/1897988905 shutdown_connections 2023-11-27T15:26:14.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.826+0000 7f771159cb80 1 -- 172.21.15.133:0/1897988905 wait complete. 2023-11-27T15:26:14.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.826+0000 7f771159cb80 20 client.26189 trim_cache size 0 max 16384 2023-11-27T15:26:14.835 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:26:14.835 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (2 inodes in the stack)", 2023-11-27T15:26:14.835 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-11-27T15:26:14.835 INFO:teuthology.orchestra.run.smithi133.stdout: "b6638f49-a408-46a9-83f0-3995bb2029f5": { 2023-11-27T15:26:14.835 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-11-27T15:26:14.835 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "b6638f49-a408-46a9-83f0-3995bb2029f5", 2023-11-27T15:26:14.835 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-11-27T15:26:14.835 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:26:14.835 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:26:14.835 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:26:14.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.827+0000 7f76fdbff700 1 -- 172.21.15.133:0/144059121 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f76e00a11d0 msgr2=0x7f76e00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:14.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.827+0000 7f76fdbff700 1 --2- 172.21.15.133:0/144059121 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f76e00a11d0 0x7f76e00a3680 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f76f000adc0 tx=0x7f76f000a400 comp rx=0 tx=0).stop 2023-11-27T15:26:14.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.827+0000 7f76fdbff700 1 -- 172.21.15.133:0/144059121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76f8100d40 msgr2=0x7f76f81982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:14.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.827+0000 7f76fdbff700 1 --2- 172.21.15.133:0/144059121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76f8100d40 0x7f76f81982f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f76e800b5c0 tx=0x7f76e8003c00 comp rx=0 tx=0).stop 2023-11-27T15:26:14.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.827+0000 7f76fdbff700 1 -- 172.21.15.133:0/144059121 shutdown_connections 2023-11-27T15:26:14.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.827+0000 7f76fdbff700 1 --2- 172.21.15.133:0/144059121 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f76e00a11d0 0x7f76e00a3680 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.827+0000 7f76fdbff700 1 --2- 172.21.15.133:0/144059121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f8105f80 0x7f76f819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.827+0000 7f76fdbff700 1 --2- 172.21.15.133:0/144059121 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f76f8100d40 0x7f76f81982f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.827+0000 7f76fdbff700 1 --2- 172.21.15.133:0/144059121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f76f8103660 0x7f76f8198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:14.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.827+0000 7f76fdbff700 1 -- 172.21.15.133:0/144059121 >> 172.21.15.133:0/144059121 conn(0x7f76f80fa7c0 msgr2=0x7f76f80ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:14.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.827+0000 7f76fdbff700 1 -- 172.21.15.133:0/144059121 shutdown_connections 2023-11-27T15:26:14.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:14.827+0000 7f76fdbff700 1 -- 172.21.15.133:0/144059121 wait complete. 2023-11-27T15:26:14.852 INFO:tasks.cephfs.filesystem:scrub status for tag:b6638f49-a408-46a9-83f0-3995bb2029f5 - {'path': '/', 'tag': 'b6638f49-a408-46a9-83f0-3995bb2029f5', 'options': 'recursive,force'} 2023-11-27T15:26:15.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:14 smithi174 ceph-mon[131110]: pgmap v448: 129 pgs: 129 active+clean; 97 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 271 op/s 2023-11-27T15:26:15.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:14 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:26:15.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:14 smithi133 ceph-mon[97860]: pgmap v448: 129 pgs: 129 active+clean; 97 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 271 op/s 2023-11-27T15:26:15.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:14 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:26:15.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:14 smithi177 ceph-mon[131504]: pgmap v448: 129 pgs: 129 active+clean; 97 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 271 op/s 2023-11-27T15:26:15.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:14 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:26:16.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:15 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:26:16.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:15 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:26:16.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:15 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:26:16.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:15 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:26:16.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:15 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:26:16.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:15 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:26:16.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:15 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:26:16.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:15 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:26:16.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:15 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:26:17.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:16 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:26:17.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:16 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:26:17.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:16 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:26:17.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:16 smithi174 ceph-mon[131110]: Service mds.all has no related daemon already placed. 2023-11-27T15:26:17.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:16 smithi174 ceph-mon[131110]: pgmap v449: 129 pgs: 129 active+clean; 92 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 412 op/s 2023-11-27T15:26:17.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:16 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:26:17.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:16 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:26:17.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:16 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:26:17.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:16 smithi133 ceph-mon[97860]: Service mds.all has no related daemon already placed. 2023-11-27T15:26:17.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:16 smithi133 ceph-mon[97860]: pgmap v449: 129 pgs: 129 active+clean; 92 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 412 op/s 2023-11-27T15:26:17.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:16 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:26:17.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:16 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:26:17.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:16 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:26:17.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:16 smithi177 ceph-mon[131504]: Service mds.all has no related daemon already placed. 2023-11-27T15:26:17.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:16 smithi177 ceph-mon[131504]: pgmap v449: 129 pgs: 129 active+clean; 92 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.1 MiB/s wr, 412 op/s 2023-11-27T15:26:19.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:18 smithi133 ceph-mon[97860]: pgmap v450: 129 pgs: 129 active+clean; 97 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 409 op/s 2023-11-27T15:26:19.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:18 smithi177 ceph-mon[131504]: pgmap v450: 129 pgs: 129 active+clean; 97 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 409 op/s 2023-11-27T15:26:19.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:18 smithi174 ceph-mon[131110]: pgmap v450: 129 pgs: 129 active+clean; 97 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 409 op/s 2023-11-27T15:26:21.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:20 smithi174 ceph-mon[131110]: pgmap v451: 129 pgs: 129 active+clean; 106 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 390 op/s 2023-11-27T15:26:21.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:20 smithi133 ceph-mon[97860]: pgmap v451: 129 pgs: 129 active+clean; 106 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 390 op/s 2023-11-27T15:26:21.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:20 smithi177 ceph-mon[131504]: pgmap v451: 129 pgs: 129 active+clean; 106 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 390 op/s 2023-11-27T15:26:23.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:22 smithi174 ceph-mon[131110]: pgmap v452: 129 pgs: 129 active+clean; 110 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.5 MiB/s wr, 390 op/s 2023-11-27T15:26:23.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:22 smithi133 ceph-mon[97860]: pgmap v452: 129 pgs: 129 active+clean; 110 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.5 MiB/s wr, 390 op/s 2023-11-27T15:26:23.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:22 smithi177 ceph-mon[131504]: pgmap v452: 129 pgs: 129 active+clean; 110 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.5 MiB/s wr, 390 op/s 2023-11-27T15:26:25.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:24 smithi174 ceph-mon[131110]: pgmap v453: 129 pgs: 129 active+clean; 126 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.6 MiB/s wr, 409 op/s 2023-11-27T15:26:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.252+0000 7fda51dff700 1 -- 172.21.15.133:0/511511671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda4c0feab0 msgr2=0x7fda4c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.252+0000 7fda51dff700 1 --2- 172.21.15.133:0/511511671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda4c0feab0 0x7fda4c100ed0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009990 tx=0x7fda3c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.252+0000 7fda51dff700 1 -- 172.21.15.133:0/511511671 shutdown_connections 2023-11-27T15:26:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.252+0000 7fda51dff700 1 --2- 172.21.15.133:0/511511671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda4c103dd0 0x7fda4c104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.252+0000 7fda51dff700 1 --2- 172.21.15.133:0/511511671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda4c0feab0 0x7fda4c100ed0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.252+0000 7fda51dff700 1 --2- 172.21.15.133:0/511511671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda4c101410 0x7fda4c103890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.252+0000 7fda51dff700 1 -- 172.21.15.133:0/511511671 >> 172.21.15.133:0/511511671 conn(0x7fda4c0fa720 msgr2=0x7fda4c0fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:25.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.252+0000 7fda51dff700 1 -- 172.21.15.133:0/511511671 shutdown_connections 2023-11-27T15:26:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.252+0000 7fda51dff700 1 -- 172.21.15.133:0/511511671 wait complete. 2023-11-27T15:26:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.253+0000 7fda51dff700 1 Processor -- start 2023-11-27T15:26:25.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.254+0000 7fda51dff700 1 -- start start 2023-11-27T15:26:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.254+0000 7fda51dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda4c0feab0 0x7fda4c19c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.254+0000 7fda51dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda4c101410 0x7fda4c19caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.254+0000 7fda51dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda4c103dd0 0x7fda4c1a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.254+0000 7fda51dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c1a4130 con 0x7fda4c103dd0 2023-11-27T15:26:25.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.254+0000 7fda51dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c1a4270 con 0x7fda4c0feab0 2023-11-27T15:26:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.254+0000 7fda51dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c1a4400 con 0x7fda4c101410 2023-11-27T15:26:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.254+0000 7fda4b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda4c0feab0 0x7fda4c19c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.255+0000 7fda4b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda4c0feab0 0x7fda4c19c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:46390/0 (socket says 172.21.15.133:46390) 2023-11-27T15:26:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.255+0000 7fda4b7fe700 1 -- 172.21.15.133:0/867607167 learned_addr learned my addr 172.21.15.133:0/867607167 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.255+0000 7fda4b7fe700 1 -- 172.21.15.133:0/867607167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda4c101410 msgr2=0x7fda4c19caf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-27T15:26:25.259 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.255+0000 7fda4affd700 1 --2- 172.21.15.133:0/867607167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda4c101410 0x7fda4c19caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.255+0000 7fda4b7fe700 1 --2- 172.21.15.133:0/867607167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda4c101410 0x7fda4c19caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.255+0000 7fda4b7fe700 1 -- 172.21.15.133:0/867607167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda4c103dd0 msgr2=0x7fda4c1a3b60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-27T15:26:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.255+0000 7fda4b7fe700 1 --2- 172.21.15.133:0/867607167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda4c103dd0 0x7fda4c1a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.255+0000 7fda4b7fe700 1 -- 172.21.15.133:0/867607167 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c0095f0 con 0x7fda4c0feab0 2023-11-27T15:26:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.256+0000 7fda4b7fe700 1 --2- 172.21.15.133:0/867607167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda4c0feab0 0x7fda4c19c5b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00b680 tx=0x7fda3c00f460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.256+0000 7fda48ff9700 1 -- 172.21.15.133:0/867607167 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c01d070 con 0x7fda4c0feab0 2023-11-27T15:26:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.256+0000 7fda48ff9700 1 -- 172.21.15.133:0/867607167 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda3c00fc30 con 0x7fda4c0feab0 2023-11-27T15:26:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.256+0000 7fda48ff9700 1 -- 172.21.15.133:0/867607167 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c017650 con 0x7fda4c0feab0 2023-11-27T15:26:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.256+0000 7fda51dff700 1 -- 172.21.15.133:0/867607167 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda4c112230 con 0x7fda4c0feab0 2023-11-27T15:26:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.257+0000 7fda51dff700 1 -- 172.21.15.133:0/867607167 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda4c1120a0 con 0x7fda4c0feab0 2023-11-27T15:26:25.263 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.260+0000 7fda48ff9700 1 -- 172.21.15.133:0/867607167 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fda3c021410 con 0x7fda4c0feab0 2023-11-27T15:26:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.262+0000 7fda48ff9700 1 --2- 172.21.15.133:0/867607167 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fda340a1220 0x7fda340a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:25.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.262+0000 7fda48ff9700 1 -- 172.21.15.133:0/867607167 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fda3c0ce0f0 con 0x7fda4c0feab0 2023-11-27T15:26:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.262+0000 7fda4affd700 1 --2- 172.21.15.133:0/867607167 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fda340a1220 0x7fda340a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.263+0000 7fda6570fb80 1 Processor -- start 2023-11-27T15:26:25.266 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.263+0000 7fda6570fb80 1 -- start start 2023-11-27T15:26:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.263+0000 7fda4affd700 1 --2- 172.21.15.133:0/867607167 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fda340a1220 0x7fda340a36d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fda40009b30 tx=0x7fda40009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.263+0000 7fda6570fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c955f6b0 0x55d0c9703870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.263+0000 7fda6570fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c9703db0 0x55d0c97045d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.263+0000 7fda6570fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9704b10 0x55d0c9704f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.263+0000 7fda6570fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c9609000 con 0x55d0c955f6b0 2023-11-27T15:26:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.263+0000 7fda6570fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c965b840 con 0x55d0c9704b10 2023-11-27T15:26:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.263+0000 7fda6570fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c9654cd0 con 0x55d0c9703db0 2023-11-27T15:26:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.264+0000 7fda4affd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c9703db0 0x55d0c97045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.264+0000 7fda4bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9704b10 0x55d0c9704f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.264+0000 7fda4affd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c9703db0 0x55d0c97045d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:49406/0 (socket says 172.21.15.133:49406) 2023-11-27T15:26:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.264+0000 7fda4affd700 1 -- 172.21.15.133:0/4251619617 learned_addr learned my addr 172.21.15.133:0/4251619617 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.264+0000 7fda4b7fe700 1 --2- 172.21.15.133:0/4251619617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c955f6b0 0x55d0c9703870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.264+0000 7fda4affd700 1 -- 172.21.15.133:0/4251619617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9704b10 msgr2=0x55d0c9704f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.264+0000 7fda4affd700 1 --2- 172.21.15.133:0/4251619617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9704b10 0x55d0c9704f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.264+0000 7fda4affd700 1 -- 172.21.15.133:0/4251619617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c955f6b0 msgr2=0x55d0c9703870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.264+0000 7fda4affd700 1 --2- 172.21.15.133:0/4251619617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c955f6b0 0x55d0c9703870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.264+0000 7fda4affd700 1 -- 172.21.15.133:0/4251619617 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0c94d65d0 con 0x55d0c9703db0 2023-11-27T15:26:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.264+0000 7fda4affd700 1 --2- 172.21.15.133:0/4251619617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c9703db0 0x55d0c97045d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fda40010c20 tx=0x7fda40013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3afbbe38df1ce747 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.265+0000 7fda31ffb700 1 -- 172.21.15.133:0/4251619617 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda40012ae0 con 0x55d0c9703db0 2023-11-27T15:26:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.265+0000 7fda31ffb700 1 -- 172.21.15.133:0/4251619617 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda40004cf0 con 0x55d0c9703db0 2023-11-27T15:26:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.265+0000 7fda31ffb700 1 -- 172.21.15.133:0/4251619617 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda40005520 con 0x55d0c9703db0 2023-11-27T15:26:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.266+0000 7fda6570fb80 1 -- 172.21.15.133:0/4251619617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c9703db0 msgr2=0x55d0c97045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.266+0000 7fda6570fb80 1 --2- 172.21.15.133:0/4251619617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c9703db0 0x55d0c97045d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fda40010c20 tx=0x7fda40013000 comp rx=0 tx=0).stop 2023-11-27T15:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.266+0000 7fda6570fb80 1 -- 172.21.15.133:0/4251619617 shutdown_connections 2023-11-27T15:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.266+0000 7fda6570fb80 1 --2- 172.21.15.133:0/4251619617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9704b10 0x55d0c9704f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.266+0000 7fda6570fb80 1 --2- 172.21.15.133:0/4251619617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c955f6b0 0x55d0c9703870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.266+0000 7fda6570fb80 1 --2- 172.21.15.133:0/4251619617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c9703db0 0x55d0c97045d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.266+0000 7fda6570fb80 1 -- 172.21.15.133:0/4251619617 >> 172.21.15.133:0/4251619617 conn(0x55d0c96f8f50 msgr2=0x55d0c96fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.266+0000 7fda6570fb80 1 -- 172.21.15.133:0/4251619617 shutdown_connections 2023-11-27T15:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.266+0000 7fda6570fb80 1 -- 172.21.15.133:0/4251619617 wait complete. 2023-11-27T15:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.267+0000 7fda6570fb80 1 -- start start 2023-11-27T15:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.267+0000 7fda6570fb80 1 Processor -- start 2023-11-27T15:26:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.268+0000 7fda6570fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c955f6b0 0x55d0c970cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.268+0000 7fda6570fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9703db0 0x55d0c970d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.268+0000 7fda6570fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c9704b10 0x55d0c9714260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.268+0000 7fda6570fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c95f6880 con 0x55d0c9704b10 2023-11-27T15:26:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.268+0000 7fda6570fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c95f6520 con 0x55d0c9703db0 2023-11-27T15:26:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.268+0000 7fda6570fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c94d88c0 con 0x55d0c955f6b0 2023-11-27T15:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.268+0000 7fda6570fb80 10 client.0 _ll_register_callbacks cb 0x55d0c9614720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.268+0000 7fda4b7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c955f6b0 0x55d0c970cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.268+0000 7fda4b7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c955f6b0 0x55d0c970cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:49412/0 (socket says 172.21.15.133:49412) 2023-11-27T15:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.268+0000 7fda4b7fe700 1 -- 172.21.15.133:0/2903825628 learned_addr learned my addr 172.21.15.133:0/2903825628 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.268+0000 7fda4affd700 1 --2- 172.21.15.133:0/2903825628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9703db0 0x55d0c970d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.268+0000 7fda4bfff700 1 --2- 172.21.15.133:0/2903825628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c9704b10 0x55d0c9714260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.268+0000 7fda4b7fe700 1 -- 172.21.15.133:0/2903825628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9703db0 msgr2=0x55d0c970d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.268+0000 7fda4b7fe700 1 --2- 172.21.15.133:0/2903825628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9703db0 0x55d0c970d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.268+0000 7fda4b7fe700 1 -- 172.21.15.133:0/2903825628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c9704b10 msgr2=0x55d0c9714260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.268+0000 7fda4b7fe700 1 --2- 172.21.15.133:0/2903825628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c9704b10 0x55d0c9714260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.268+0000 7fda4b7fe700 1 -- 172.21.15.133:0/2903825628 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda40010900 con 0x55d0c955f6b0 2023-11-27T15:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.268+0000 7fda4b7fe700 1 --2- 172.21.15.133:0/2903825628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c955f6b0 0x55d0c970cce0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda3c0095c0 tx=0x7fda3c005910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de37e4dbd6483006 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.268+0000 7fda2affd700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.269+0000 7fda2affd700 1 -- 172.21.15.133:0/2903825628 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0a36c0 con 0x55d0c955f6b0 2023-11-27T15:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.269+0000 7fda2affd700 1 -- 172.21.15.133:0/2903825628 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fda3c0a10b0 con 0x55d0c955f6b0 2023-11-27T15:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.269+0000 7fda6570fb80 1 -- 172.21.15.133:0/2903825628 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0c96f6710 con 0x55d0c955f6b0 2023-11-27T15:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.269+0000 7fda2affd700 1 -- 172.21.15.133:0/2903825628 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda3c0a3230 con 0x55d0c955f6b0 2023-11-27T15:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.269+0000 7fda6570fb80 10 client.26179 fetch_fsmap learned FSMap version 26 2023-11-27T15:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.269+0000 7fda6570fb80 1 -- 172.21.15.133:0/2903825628 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d0c961b130 con 0x55d0c955f6b0 2023-11-27T15:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.269+0000 7fda2affd700 1 -- 172.21.15.133:0/2903825628 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda3c0c9020 con 0x55d0c955f6b0 2023-11-27T15:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.269+0000 7fda2affd700 1 -- 172.21.15.133:0/2903825628 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 26) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fda3c0216c0 con 0x55d0c955f6b0 2023-11-27T15:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.269+0000 7fda6570fb80 10 client.26179 fetch_fsmap finished waiting for FSMap version 26 2023-11-27T15:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.270+0000 7fda6570fb80 10 client.26179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.270+0000 7fda6570fb80 20 client.26179 populate_metadata read hostname 'smithi133' 2023-11-27T15:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.270+0000 7fda6570fb80 1 --2- 172.21.15.133:0/2903825628 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d0c9718780 0x55d0c971ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.270+0000 7fda6570fb80 4 client.26179 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.270+0000 7fda6570fb80 1 -- 172.21.15.133:0/2903825628 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0c9617040 con 0x55d0c9718780 2023-11-27T15:26:25.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.270+0000 7fda4affd700 1 --2- 172.21.15.133:0/2903825628 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d0c9718780 0x55d0c971ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:25.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.270+0000 7fda4affd700 1 --2- 172.21.15.133:0/2903825628 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d0c9718780 0x55d0c971ab90 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dd5097f6372c6ac server_cookie=7adf0f60a95adb52 in_seq=0 out_seq=0 2023-11-27T15:26:25.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.270+0000 7fda2affd700 10 client.26179 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:25.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.272+0000 7fda2affd700 1 -- 172.21.15.133:0/2903825628 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d0c9617040 con 0x55d0c9718780 2023-11-27T15:26:25.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.272+0000 7fda2affd700 10 client.26179 handle_command_reply: tid=0 2023-11-27T15:26:25.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.272+0000 7fda6570fb80 1 client.26179 shutdown 2023-11-27T15:26:25.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.272+0000 7fda6570fb80 1 -- 172.21.15.133:0/2903825628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c955f6b0 msgr2=0x55d0c970cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:25.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.272+0000 7fda6570fb80 1 --2- 172.21.15.133:0/2903825628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c955f6b0 0x55d0c970cce0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda3c0095c0 tx=0x7fda3c005910 comp rx=0 tx=0).stop 2023-11-27T15:26:25.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.272+0000 7fda6570fb80 1 -- 172.21.15.133:0/2903825628 shutdown_connections 2023-11-27T15:26:25.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.272+0000 7fda6570fb80 1 --2- 172.21.15.133:0/2903825628 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d0c9718780 0x55d0c971ab90 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.272+0000 7fda6570fb80 1 --2- 172.21.15.133:0/2903825628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c9704b10 0x55d0c9714260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.272+0000 7fda6570fb80 1 --2- 172.21.15.133:0/2903825628 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c955f6b0 0x55d0c970cce0 secure :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda3c0095c0 tx=0x7fda3c005910 comp rx=0 tx=0).stop 2023-11-27T15:26:25.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.272+0000 7fda6570fb80 1 --2- 172.21.15.133:0/2903825628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9703db0 0x55d0c970d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.272+0000 7fda6570fb80 1 -- 172.21.15.133:0/2903825628 >> 172.21.15.133:0/2903825628 conn(0x55d0c96f8f50 msgr2=0x55d0c96fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:25.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.272+0000 7fda2affd700 0 client.26179 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:25.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.273+0000 7fda6570fb80 1 -- 172.21.15.133:0/2903825628 shutdown_connections 2023-11-27T15:26:25.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.273+0000 7fda6570fb80 1 -- 172.21.15.133:0/2903825628 wait complete. 2023-11-27T15:26:25.278 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.273+0000 7fda6570fb80 20 client.26179 trim_cache size 0 max 16384 2023-11-27T15:26:25.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:24 smithi133 ceph-mon[97860]: pgmap v453: 129 pgs: 129 active+clean; 126 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.6 MiB/s wr, 409 op/s 2023-11-27T15:26:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.293+0000 7fda6570fb80 1 Processor -- start 2023-11-27T15:26:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.293+0000 7fda6570fb80 1 -- start start 2023-11-27T15:26:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.293+0000 7fda6570fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c9703db0 0x55d0c970ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.293+0000 7fda6570fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c9704b10 0x55d0c9714220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.293+0000 7fda6570fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9707a20 0x55d0c9707e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.293+0000 7fda6570fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c9617040 con 0x55d0c9704b10 2023-11-27T15:26:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.293+0000 7fda6570fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c96fea20 con 0x55d0c9707a20 2023-11-27T15:26:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.293+0000 7fda6570fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c96f6350 con 0x55d0c9703db0 2023-11-27T15:26:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.293+0000 7fda4b7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c9703db0 0x55d0c970ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.293+0000 7fda4b7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c9703db0 0x55d0c970ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:49424/0 (socket says 172.21.15.133:49424) 2023-11-27T15:26:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.293+0000 7fda4b7fe700 1 -- 172.21.15.133:0/3574830541 learned_addr learned my addr 172.21.15.133:0/3574830541 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.294+0000 7fda4bfff700 1 --2- 172.21.15.133:0/3574830541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9707a20 0x55d0c9707e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.294+0000 7fda4affd700 1 --2- 172.21.15.133:0/3574830541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c9704b10 0x55d0c9714220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.294+0000 7fda4b7fe700 1 -- 172.21.15.133:0/3574830541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9707a20 msgr2=0x55d0c9707e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.294+0000 7fda4b7fe700 1 --2- 172.21.15.133:0/3574830541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9707a20 0x55d0c9707e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.294+0000 7fda4b7fe700 1 -- 172.21.15.133:0/3574830541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c9704b10 msgr2=0x55d0c9714220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.294+0000 7fda4b7fe700 1 --2- 172.21.15.133:0/3574830541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c9704b10 0x55d0c9714220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.294+0000 7fda4b7fe700 1 -- 172.21.15.133:0/3574830541 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0c961ae80 con 0x55d0c9703db0 2023-11-27T15:26:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.294+0000 7fda4bfff700 1 --2- 172.21.15.133:0/3574830541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9707a20 0x55d0c9707e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.294+0000 7fda4b7fe700 1 --2- 172.21.15.133:0/3574830541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c9703db0 0x55d0c970ba00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fda3c0a3190 tx=0x7fda3c021c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e808a4797ec3859 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.294+0000 7fda327fc700 1 -- 172.21.15.133:0/3574830541 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0a36c0 con 0x55d0c9703db0 2023-11-27T15:26:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.294+0000 7fda327fc700 1 -- 172.21.15.133:0/3574830541 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda3c0c9070 con 0x55d0c9703db0 2023-11-27T15:26:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.294+0000 7fda327fc700 1 -- 172.21.15.133:0/3574830541 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0216c0 con 0x55d0c9703db0 2023-11-27T15:26:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.295+0000 7fda6570fb80 1 -- 172.21.15.133:0/3574830541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c9703db0 msgr2=0x55d0c970ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.295+0000 7fda6570fb80 1 --2- 172.21.15.133:0/3574830541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c9703db0 0x55d0c970ba00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fda3c0a3190 tx=0x7fda3c021c70 comp rx=0 tx=0).stop 2023-11-27T15:26:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.295+0000 7fda6570fb80 1 -- 172.21.15.133:0/3574830541 shutdown_connections 2023-11-27T15:26:25.301 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.295+0000 7fda6570fb80 1 --2- 172.21.15.133:0/3574830541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9707a20 0x55d0c9707e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.295+0000 7fda6570fb80 1 --2- 172.21.15.133:0/3574830541 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c9703db0 0x55d0c970ba00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.295+0000 7fda6570fb80 1 --2- 172.21.15.133:0/3574830541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c9704b10 0x55d0c9714220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.295+0000 7fda6570fb80 1 -- 172.21.15.133:0/3574830541 >> 172.21.15.133:0/3574830541 conn(0x55d0c96f8f50 msgr2=0x55d0c971f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.295+0000 7fda6570fb80 1 -- 172.21.15.133:0/3574830541 shutdown_connections 2023-11-27T15:26:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.295+0000 7fda6570fb80 1 -- 172.21.15.133:0/3574830541 wait complete. 2023-11-27T15:26:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.296+0000 7fda6570fb80 1 -- start start 2023-11-27T15:26:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.296+0000 7fda6570fb80 1 Processor -- start 2023-11-27T15:26:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.297+0000 7fda6570fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9703db0 0x55d0c97103b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.297+0000 7fda6570fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c9704b10 0x55d0c97108f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.297+0000 7fda6570fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c9707a20 0x55d0c97137d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.297+0000 7fda6570fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c9713d10 con 0x55d0c9707a20 2023-11-27T15:26:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.297+0000 7fda6570fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c9713e80 con 0x55d0c9703db0 2023-11-27T15:26:25.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.297+0000 7fda6570fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0c9714040 con 0x55d0c9704b10 2023-11-27T15:26:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.297+0000 7fda6570fb80 10 client.0 _ll_register_callbacks cb 0x55d0c9614720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.297+0000 7fda4b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9703db0 0x55d0c97103b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.297+0000 7fda4b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9703db0 0x55d0c97103b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:46438/0 (socket says 172.21.15.133:46438) 2023-11-27T15:26:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.297+0000 7fda4b7fe700 1 -- 172.21.15.133:0/1460519452 learned_addr learned my addr 172.21.15.133:0/1460519452 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.297+0000 7fda4affd700 1 --2- 172.21.15.133:0/1460519452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c9704b10 0x55d0c97108f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.297+0000 7fda4bfff700 1 --2- 172.21.15.133:0/1460519452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c9707a20 0x55d0c97137d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.297+0000 7fda4b7fe700 1 -- 172.21.15.133:0/1460519452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c9704b10 msgr2=0x55d0c97108f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.297+0000 7fda4b7fe700 1 --2- 172.21.15.133:0/1460519452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c9704b10 0x55d0c97108f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.297+0000 7fda4b7fe700 1 -- 172.21.15.133:0/1460519452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c9707a20 msgr2=0x55d0c97137d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.297+0000 7fda4b7fe700 1 --2- 172.21.15.133:0/1460519452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c9707a20 0x55d0c97137d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.297+0000 7fda4b7fe700 1 -- 172.21.15.133:0/1460519452 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0c961b130 con 0x55d0c9703db0 2023-11-27T15:26:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.297+0000 7fda4bfff700 1 --2- 172.21.15.133:0/1460519452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c9707a20 0x55d0c97137d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.298+0000 7fda4b7fe700 1 --2- 172.21.15.133:0/1460519452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9703db0 0x55d0c97103b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fda3c09b110 tx=0x7fda3c0ce020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f49c3b9627aac867 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:25.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.298+0000 7fda2affd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:26:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.298+0000 7fda2affd700 1 -- 172.21.15.133:0/1460519452 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0a3430 con 0x55d0c9703db0 2023-11-27T15:26:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.298+0000 7fda2affd700 1 -- 172.21.15.133:0/1460519452 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fda3c0ca080 con 0x55d0c9703db0 2023-11-27T15:26:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.298+0000 7fda6570fb80 1 -- 172.21.15.133:0/1460519452 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda40004cf0 con 0x55d0c9703db0 2023-11-27T15:26:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.298+0000 7fda2affd700 1 -- 172.21.15.133:0/1460519452 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda3c0a36c0 con 0x55d0c9703db0 2023-11-27T15:26:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.298+0000 7fda6570fb80 10 client.26207 fetch_fsmap learned FSMap version 26 2023-11-27T15:26:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.298+0000 7fda6570fb80 1 -- 172.21.15.133:0/1460519452 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d0c946bc00 con 0x55d0c9703db0 2023-11-27T15:26:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.299+0000 7fda2affd700 1 -- 172.21.15.133:0/1460519452 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda40004cf0 con 0x55d0c9703db0 2023-11-27T15:26:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.299+0000 7fda2affd700 1 -- 172.21.15.133:0/1460519452 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 26) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fda3c09a490 con 0x55d0c9703db0 2023-11-27T15:26:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.299+0000 7fda6570fb80 10 client.26207 fetch_fsmap finished waiting for FSMap version 26 2023-11-27T15:26:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.299+0000 7fda6570fb80 10 client.26207 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:26:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.299+0000 7fda6570fb80 20 client.26207 populate_metadata read hostname 'smithi133' 2023-11-27T15:26:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.299+0000 7fda6570fb80 1 --2- 172.21.15.133:0/1460519452 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d0c976d310 0x55d0c976f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.299+0000 7fda6570fb80 4 client.26207 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:26:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.299+0000 7fda6570fb80 1 -- 172.21.15.133:0/1460519452 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d0c9623600 con 0x55d0c976d310 2023-11-27T15:26:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.299+0000 7fda4affd700 1 --2- 172.21.15.133:0/1460519452 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d0c976d310 0x55d0c976f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.300+0000 7fda4affd700 1 --2- 172.21.15.133:0/1460519452 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d0c976d310 0x55d0c976f720 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53608e4d82bc2706 server_cookie=b26301b13a7f15fb in_seq=0 out_seq=0 2023-11-27T15:26:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:25.300+0000 7fda2affd700 10 client.26207 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:25.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:24 smithi177 ceph-mon[131504]: pgmap v453: 129 pgs: 129 active+clean; 126 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.6 MiB/s wr, 409 op/s 2023-11-27T15:26:27.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:26 smithi174 ceph-mon[131110]: pgmap v454: 129 pgs: 129 active+clean; 139 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.2 MiB/s wr, 362 op/s 2023-11-27T15:26:27.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:26 smithi133 ceph-mon[97860]: pgmap v454: 129 pgs: 129 active+clean; 139 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.2 MiB/s wr, 362 op/s 2023-11-27T15:26:27.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:26 smithi177 ceph-mon[131504]: pgmap v454: 129 pgs: 129 active+clean; 139 MiB data, 938 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.2 MiB/s wr, 362 op/s 2023-11-27T15:26:27.954 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.950+0000 7fda2affd700 1 -- 172.21.15.133:0/1460519452 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d0c9623600 con 0x55d0c976d310 2023-11-27T15:26:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.950+0000 7fda2affd700 10 client.26207 handle_command_reply: tid=0 2023-11-27T15:26:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.950+0000 7fda6570fb80 1 client.26207 shutdown 2023-11-27T15:26:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.951+0000 7fda6570fb80 1 -- 172.21.15.133:0/1460519452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9703db0 msgr2=0x55d0c97103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.951+0000 7fda6570fb80 1 --2- 172.21.15.133:0/1460519452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9703db0 0x55d0c97103b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fda3c09b110 tx=0x7fda3c0ce020 comp rx=0 tx=0).stop 2023-11-27T15:26:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.951+0000 7fda6570fb80 1 -- 172.21.15.133:0/1460519452 shutdown_connections 2023-11-27T15:26:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.951+0000 7fda6570fb80 1 --2- 172.21.15.133:0/1460519452 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d0c976d310 0x55d0c976f720 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.951+0000 7fda6570fb80 1 --2- 172.21.15.133:0/1460519452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d0c9707a20 0x55d0c97137d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.951+0000 7fda6570fb80 1 --2- 172.21.15.133:0/1460519452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0c9703db0 0x55d0c97103b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.951+0000 7fda2affd700 0 client.26207 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.951+0000 7fda6570fb80 1 --2- 172.21.15.133:0/1460519452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d0c9704b10 0x55d0c97108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.951+0000 7fda6570fb80 1 -- 172.21.15.133:0/1460519452 >> 172.21.15.133:0/1460519452 conn(0x55d0c96f8f50 msgr2=0x55d0c971f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:27.955 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.951+0000 7fda6570fb80 1 -- 172.21.15.133:0/1460519452 shutdown_connections 2023-11-27T15:26:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.951+0000 7fda6570fb80 1 -- 172.21.15.133:0/1460519452 wait complete. 2023-11-27T15:26:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.951+0000 7fda6570fb80 20 client.26207 trim_cache size 0 max 16384 2023-11-27T15:26:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:26:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:26:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:26:27.956 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:26:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.952+0000 7fda51dff700 1 -- 172.21.15.133:0/867607167 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fda340a1220 msgr2=0x7fda340a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.952+0000 7fda51dff700 1 --2- 172.21.15.133:0/867607167 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fda340a1220 0x7fda340a36d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fda40009b30 tx=0x7fda40009260 comp rx=0 tx=0).stop 2023-11-27T15:26:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.952+0000 7fda51dff700 1 -- 172.21.15.133:0/867607167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda4c0feab0 msgr2=0x7fda4c19c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.952+0000 7fda51dff700 1 --2- 172.21.15.133:0/867607167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda4c0feab0 0x7fda4c19c5b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00b680 tx=0x7fda3c00f460 comp rx=0 tx=0).stop 2023-11-27T15:26:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.952+0000 7fda51dff700 1 -- 172.21.15.133:0/867607167 shutdown_connections 2023-11-27T15:26:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.952+0000 7fda51dff700 1 --2- 172.21.15.133:0/867607167 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fda340a1220 0x7fda340a36d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.952+0000 7fda51dff700 1 --2- 172.21.15.133:0/867607167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda4c103dd0 0x7fda4c1a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.952+0000 7fda51dff700 1 --2- 172.21.15.133:0/867607167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda4c0feab0 0x7fda4c19c5b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.952+0000 7fda51dff700 1 --2- 172.21.15.133:0/867607167 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fda4c101410 0x7fda4c19caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.952+0000 7fda51dff700 1 -- 172.21.15.133:0/867607167 >> 172.21.15.133:0/867607167 conn(0x7fda4c0fa720 msgr2=0x7fda4c102110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.952+0000 7fda51dff700 1 -- 172.21.15.133:0/867607167 shutdown_connections 2023-11-27T15:26:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:27.952+0000 7fda51dff700 1 -- 172.21.15.133:0/867607167 wait complete. 2023-11-27T15:26:29.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:28 smithi133 ceph-mon[97860]: pgmap v455: 129 pgs: 129 active+clean; 143 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 368 op/s 2023-11-27T15:26:29.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:28 smithi177 ceph-mon[131504]: pgmap v455: 129 pgs: 129 active+clean; 143 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 368 op/s 2023-11-27T15:26:29.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:28 smithi174 ceph-mon[131110]: pgmap v455: 129 pgs: 129 active+clean; 143 MiB data, 967 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.6 MiB/s wr, 368 op/s 2023-11-27T15:26:31.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:30 smithi174 ceph-mon[131110]: pgmap v456: 129 pgs: 129 active+clean; 144 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 676 op/s 2023-11-27T15:26:31.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:30 smithi133 ceph-mon[97860]: pgmap v456: 129 pgs: 129 active+clean; 144 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 676 op/s 2023-11-27T15:26:31.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:30 smithi177 ceph-mon[131504]: pgmap v456: 129 pgs: 129 active+clean; 144 MiB data, 994 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 676 op/s 2023-11-27T15:26:32.807 INFO:tasks.workunit.client.0.smithi133.stdout:created hierarchy, now cleaning up 2023-11-27T15:26:33.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:32 smithi174 ceph-mon[131110]: pgmap v457: 129 pgs: 129 active+clean; 154 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.5 MiB/s wr, 972 op/s 2023-11-27T15:26:33.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:32 smithi133 ceph-mon[97860]: pgmap v457: 129 pgs: 129 active+clean; 154 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.5 MiB/s wr, 972 op/s 2023-11-27T15:26:33.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:32 smithi177 ceph-mon[131504]: pgmap v457: 129 pgs: 129 active+clean; 154 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.5 MiB/s wr, 972 op/s 2023-11-27T15:26:35.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:34 smithi174 ceph-mon[131110]: pgmap v458: 129 pgs: 129 active+clean; 163 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.8 MiB/s wr, 982 op/s 2023-11-27T15:26:35.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:34 smithi133 ceph-mon[97860]: pgmap v458: 129 pgs: 129 active+clean; 163 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.8 MiB/s wr, 982 op/s 2023-11-27T15:26:35.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:34 smithi177 ceph-mon[131504]: pgmap v458: 129 pgs: 129 active+clean; 163 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.8 MiB/s wr, 982 op/s 2023-11-27T15:26:37.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:36 smithi174 ceph-mon[131110]: pgmap v459: 129 pgs: 129 active+clean; 167 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 948 op/s 2023-11-27T15:26:37.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:36 smithi133 ceph-mon[97860]: pgmap v459: 129 pgs: 129 active+clean; 167 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 948 op/s 2023-11-27T15:26:37.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:36 smithi177 ceph-mon[131504]: pgmap v459: 129 pgs: 129 active+clean; 167 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.8 MiB/s wr, 948 op/s 2023-11-27T15:26:39.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:38 smithi133 ceph-mon[97860]: pgmap v460: 129 pgs: 129 active+clean; 169 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 947 op/s 2023-11-27T15:26:39.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:38 smithi177 ceph-mon[131504]: pgmap v460: 129 pgs: 129 active+clean; 169 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 947 op/s 2023-11-27T15:26:39.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:38 smithi174 ceph-mon[131110]: pgmap v460: 129 pgs: 129 active+clean; 169 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 947 op/s 2023-11-27T15:26:40.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:39 smithi133 ceph-mon[97860]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:26:40.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:39 smithi177 ceph-mon[131504]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:26:40.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:39 smithi174 ceph-mon[131110]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:26:41.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:40 smithi133 ceph-mon[97860]: pgmap v461: 129 pgs: 129 active+clean; 176 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 841 op/s 2023-11-27T15:26:41.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:40 smithi177 ceph-mon[131504]: pgmap v461: 129 pgs: 129 active+clean; 176 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 841 op/s 2023-11-27T15:26:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.111+0000 7f078f7ff700 1 -- 172.21.15.133:0/3237114317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0788101e40 msgr2=0x7f0788108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.111+0000 7f078f7ff700 1 --2- 172.21.15.133:0/3237114317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0788101e40 0x7f0788108700 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0780009f90 tx=0x7f078000b330 comp rx=0 tx=0).stop 2023-11-27T15:26:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.111+0000 7f078f7ff700 1 -- 172.21.15.133:0/3237114317 shutdown_connections 2023-11-27T15:26:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.111+0000 7f078f7ff700 1 --2- 172.21.15.133:0/3237114317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0788101e40 0x7f0788108700 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.111+0000 7f078f7ff700 1 --2- 172.21.15.133:0/3237114317 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f07881002c0 0x7f07881006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.111+0000 7f078f7ff700 1 --2- 172.21.15.133:0/3237114317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07881014b0 0x7f0788101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.111+0000 7f078f7ff700 1 -- 172.21.15.133:0/3237114317 >> 172.21.15.133:0/3237114317 conn(0x7f07880fb8e0 msgr2=0x7f07880fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.111+0000 7f078f7ff700 1 -- 172.21.15.133:0/3237114317 shutdown_connections 2023-11-27T15:26:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.111+0000 7f078f7ff700 1 -- 172.21.15.133:0/3237114317 wait complete. 2023-11-27T15:26:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.112+0000 7f078f7ff700 1 Processor -- start 2023-11-27T15:26:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.112+0000 7f078f7ff700 1 -- start start 2023-11-27T15:26:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.113+0000 7f078f7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07881002c0 0x7f07881982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.113+0000 7f078f7ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07881014b0 0x7f0788198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.113+0000 7f078f7ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0788101e40 0x7f078819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.113+0000 7f078f7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078819fe60 con 0x7f07881002c0 2023-11-27T15:26:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.113+0000 7f078f7ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078819ffa0 con 0x7f07881014b0 2023-11-27T15:26:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.113+0000 7f078f7ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07881a0130 con 0x7f0788101e40 2023-11-27T15:26:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.113+0000 7f078d59b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07881002c0 0x7f07881982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.113+0000 7f078cd9a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07881014b0 0x7f0788198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.113+0000 7f078d59b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07881002c0 0x7f07881982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57432/0 (socket says 172.21.15.133:57432) 2023-11-27T15:26:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.113+0000 7f078dd9c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0788101e40 0x7f078819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.113+0000 7f078d59b700 1 -- 172.21.15.133:0/311084695 learned_addr learned my addr 172.21.15.133:0/311084695 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.113+0000 7f078cd9a700 1 -- 172.21.15.133:0/311084695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0788101e40 msgr2=0x7f078819f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.113+0000 7f078cd9a700 1 --2- 172.21.15.133:0/311084695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0788101e40 0x7f078819f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.113+0000 7f078cd9a700 1 -- 172.21.15.133:0/311084695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07881002c0 msgr2=0x7f07881982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.113+0000 7f078cd9a700 1 --2- 172.21.15.133:0/311084695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07881002c0 0x7f07881982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.113+0000 7f078cd9a700 1 -- 172.21.15.133:0/311084695 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07780095f0 con 0x7f07881014b0 2023-11-27T15:26:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.113+0000 7f078dd9c700 1 --2- 172.21.15.133:0/311084695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0788101e40 0x7f078819f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:26:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.113+0000 7f078d59b700 1 --2- 172.21.15.133:0/311084695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07881002c0 0x7f07881982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:26:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.114+0000 7f078cd9a700 1 --2- 172.21.15.133:0/311084695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07881014b0 0x7f0788198820 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f077800eb90 tx=0x7f077800eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.114+0000 7f077e7fc700 1 -- 172.21.15.133:0/311084695 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778009970 con 0x7f07881014b0 2023-11-27T15:26:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.114+0000 7f078f7ff700 1 -- 172.21.15.133:0/311084695 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0780009c40 con 0x7f07881014b0 2023-11-27T15:26:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.114+0000 7f077e7fc700 1 -- 172.21.15.133:0/311084695 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07780044e0 con 0x7f07881014b0 2023-11-27T15:26:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.114+0000 7f077e7fc700 1 -- 172.21.15.133:0/311084695 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778003e40 con 0x7f07881014b0 2023-11-27T15:26:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.114+0000 7f078f7ff700 1 -- 172.21.15.133:0/311084695 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078810bc30 con 0x7f07881014b0 2023-11-27T15:26:41.121 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.117+0000 7f077e7fc700 1 -- 172.21.15.133:0/311084695 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0778003fa0 con 0x7f07881014b0 2023-11-27T15:26:41.122 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.118+0000 7f07a3179b80 1 Processor -- start 2023-11-27T15:26:41.123 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.119+0000 7f07a3179b80 1 -- start start 2023-11-27T15:26:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.119+0000 7f07a3179b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7a706b0 0x55a7a7c14870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.119+0000 7f07a3179b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c14db0 0x55a7a7c155d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.119+0000 7f077e7fc700 1 --2- 172.21.15.133:0/311084695 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f07740a1150 0x7f07740a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.119+0000 7f07a3179b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7c15b10 0x55a7a7c15f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.119+0000 7f07a3179b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a7b1a000 con 0x55a7a7c15b10 2023-11-27T15:26:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.119+0000 7f07a3179b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a7b6c840 con 0x55a7a7a706b0 2023-11-27T15:26:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.119+0000 7f07a3179b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a7b65cd0 con 0x55a7a7c14db0 2023-11-27T15:26:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.119+0000 7f077e7fc700 1 -- 172.21.15.133:0/311084695 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f0778014070 con 0x7f07881014b0 2023-11-27T15:26:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.119+0000 7f078dd9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7a706b0 0x55a7a7c14870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.120+0000 7f078cd9a700 1 --2- 172.21.15.133:0/311084695 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f07740a1150 0x7f07740a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.120+0000 7f078d59b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c14db0 0x55a7a7c155d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.120+0000 7f078d59b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c14db0 0x55a7a7c155d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:54540/0 (socket says 172.21.15.133:54540) 2023-11-27T15:26:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.120+0000 7f078d59b700 1 -- 172.21.15.133:0/1160320797 learned_addr learned my addr 172.21.15.133:0/1160320797 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.120+0000 7f078d59b700 1 -- 172.21.15.133:0/1160320797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7a706b0 msgr2=0x55a7a7c14870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.120+0000 7f078d59b700 1 --2- 172.21.15.133:0/1160320797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7a706b0 0x55a7a7c14870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.120+0000 7f078d59b700 1 -- 172.21.15.133:0/1160320797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7c15b10 msgr2=0x55a7a7c15f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.120+0000 7f078d59b700 1 --2- 172.21.15.133:0/1160320797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7c15b10 0x55a7a7c15f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.120+0000 7f078d59b700 1 -- 172.21.15.133:0/1160320797 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7a79e75d0 con 0x55a7a7c14db0 2023-11-27T15:26:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.120+0000 7f078cd9a700 1 --2- 172.21.15.133:0/311084695 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f07740a1150 0x7f07740a3600 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0778009890 tx=0x7f0778009820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.121+0000 7f078d59b700 1 --2- 172.21.15.133:0/1160320797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c14db0 0x55a7a7c155d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0784011650 tx=0x7f0784011960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cd6dd10c93bf926 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.121+0000 7f07737fe700 1 -- 172.21.15.133:0/1160320797 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784004a30 con 0x55a7a7c14db0 2023-11-27T15:26:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.121+0000 7f07737fe700 1 -- 172.21.15.133:0/1160320797 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07840054e0 con 0x55a7a7c14db0 2023-11-27T15:26:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.121+0000 7f07737fe700 1 -- 172.21.15.133:0/1160320797 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078400fa80 con 0x55a7a7c14db0 2023-11-27T15:26:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.123+0000 7f07a3179b80 1 -- 172.21.15.133:0/1160320797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c14db0 msgr2=0x55a7a7c155d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.123+0000 7f07a3179b80 1 --2- 172.21.15.133:0/1160320797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c14db0 0x55a7a7c155d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0784011650 tx=0x7f0784011960 comp rx=0 tx=0).stop 2023-11-27T15:26:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.123+0000 7f07a3179b80 1 -- 172.21.15.133:0/1160320797 shutdown_connections 2023-11-27T15:26:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.123+0000 7f07a3179b80 1 --2- 172.21.15.133:0/1160320797 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7c15b10 0x55a7a7c15f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.123+0000 7f07a3179b80 1 --2- 172.21.15.133:0/1160320797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7a706b0 0x55a7a7c14870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.123+0000 7f07a3179b80 1 --2- 172.21.15.133:0/1160320797 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c14db0 0x55a7a7c155d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.123+0000 7f07a3179b80 1 -- 172.21.15.133:0/1160320797 >> 172.21.15.133:0/1160320797 conn(0x55a7a7c09f50 msgr2=0x55a7a7c0f100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.123+0000 7f07a3179b80 1 -- 172.21.15.133:0/1160320797 shutdown_connections 2023-11-27T15:26:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.123+0000 7f07a3179b80 1 -- 172.21.15.133:0/1160320797 wait complete. 2023-11-27T15:26:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.123+0000 7f07a3179b80 1 -- start start 2023-11-27T15:26:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.124+0000 7f07a3179b80 1 Processor -- start 2023-11-27T15:26:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.124+0000 7f07a3179b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7a706b0 0x55a7a7c1dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.124+0000 7f07a3179b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c14db0 0x55a7a7c1e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.124+0000 7f07a3179b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7c15b10 0x55a7a7c25260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.124+0000 7f07a3179b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a7b07880 con 0x55a7a7a706b0 2023-11-27T15:26:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.124+0000 7f07a3179b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a7b07520 con 0x55a7a7c15b10 2023-11-27T15:26:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.124+0000 7f07a3179b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a79e98c0 con 0x55a7a7c14db0 2023-11-27T15:26:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.124+0000 7f07a3179b80 10 client.0 _ll_register_callbacks cb 0x55a7a7b25720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.124+0000 7f078dd9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7a706b0 0x55a7a7c1dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.125+0000 7f078dd9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7a706b0 0x55a7a7c1dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57442/0 (socket says 172.21.15.133:57442) 2023-11-27T15:26:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.125+0000 7f078d59b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c14db0 0x55a7a7c1e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.125+0000 7f078dd9c700 1 -- 172.21.15.133:0/2861957714 learned_addr learned my addr 172.21.15.133:0/2861957714 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.125+0000 7f078dd9c700 1 --2- 172.21.15.133:0/2861957714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7c15b10 0x55a7a7c25260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.125+0000 7f078dd9c700 1 -- 172.21.15.133:0/2861957714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c14db0 msgr2=0x55a7a7c1e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.125+0000 7f078dd9c700 1 --2- 172.21.15.133:0/2861957714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c14db0 0x55a7a7c1e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.125+0000 7f078dd9c700 1 -- 172.21.15.133:0/2861957714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7c15b10 msgr2=0x55a7a7c25260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.125+0000 7f078dd9c700 1 --2- 172.21.15.133:0/2861957714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7c15b10 0x55a7a7c25260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.125+0000 7f078dd9c700 1 -- 172.21.15.133:0/2861957714 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0784009e10 con 0x55a7a7a706b0 2023-11-27T15:26:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.125+0000 7f078dd9c700 1 --2- 172.21.15.133:0/2861957714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7a706b0 0x55a7a7c1dce0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0780007f00 tx=0x7f0780007490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd597a72058ffa3 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.126+0000 7f0770ff9700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:26:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.126+0000 7f0770ff9700 1 -- 172.21.15.133:0/2861957714 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078000e040 con 0x55a7a7a706b0 2023-11-27T15:26:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.126+0000 7f0770ff9700 1 -- 172.21.15.133:0/2861957714 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f078000bda0 con 0x55a7a7a706b0 2023-11-27T15:26:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.126+0000 7f07a3179b80 1 -- 172.21.15.133:0/2861957714 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7a7c07710 con 0x55a7a7a706b0 2023-11-27T15:26:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.126+0000 7f0770ff9700 1 -- 172.21.15.133:0/2861957714 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0780009ac0 con 0x55a7a7a706b0 2023-11-27T15:26:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.126+0000 7f07a3179b80 10 client.16131 fetch_fsmap learned FSMap version 27 2023-11-27T15:26:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.126+0000 7f07a3179b80 1 -- 172.21.15.133:0/2861957714 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a7a7b2c130 con 0x55a7a7a706b0 2023-11-27T15:26:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.126+0000 7f0770ff9700 1 -- 172.21.15.133:0/2861957714 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a7a7c07710 con 0x55a7a7a706b0 2023-11-27T15:26:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.126+0000 7f0770ff9700 1 -- 172.21.15.133:0/2861957714 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f0780019070 con 0x55a7a7a706b0 2023-11-27T15:26:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.127+0000 7f07a3179b80 10 client.16131 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:26:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.127+0000 7f07a3179b80 10 client.16131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:26:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.127+0000 7f07a3179b80 20 client.16131 populate_metadata read hostname 'smithi133' 2023-11-27T15:26:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.127+0000 7f07a3179b80 1 --2- 172.21.15.133:0/2861957714 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55a7a7c29c10 0x55a7a7c2c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.127+0000 7f07a3179b80 4 client.16131 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:26:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.127+0000 7f07a3179b80 1 -- 172.21.15.133:0/2861957714 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7a7b28040 con 0x55a7a7c29c10 2023-11-27T15:26:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.127+0000 7f078d59b700 1 --2- 172.21.15.133:0/2861957714 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55a7a7c29c10 0x55a7a7c2c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.127+0000 7f078d59b700 1 --2- 172.21.15.133:0/2861957714 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55a7a7c29c10 0x55a7a7c2c020 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24bf7a0c1ace1476 server_cookie=4546ff375b20388e in_seq=0 out_seq=0 2023-11-27T15:26:41.133 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.127+0000 7f0770ff9700 10 client.16131 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.134+0000 7f0770ff9700 1 -- 172.21.15.133:0/2861957714 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a7a7b28040 con 0x55a7a7c29c10 2023-11-27T15:26:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.134+0000 7f0770ff9700 10 client.16131 handle_command_reply: tid=0 2023-11-27T15:26:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.134+0000 7f07a3179b80 1 client.16131 shutdown 2023-11-27T15:26:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.135+0000 7f07a3179b80 1 -- 172.21.15.133:0/2861957714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7a706b0 msgr2=0x55a7a7c1dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.135+0000 7f07a3179b80 1 --2- 172.21.15.133:0/2861957714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7a706b0 0x55a7a7c1dce0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0780007f00 tx=0x7f0780007490 comp rx=0 tx=0).stop 2023-11-27T15:26:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.135+0000 7f07a3179b80 1 -- 172.21.15.133:0/2861957714 shutdown_connections 2023-11-27T15:26:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.135+0000 7f07a3179b80 1 --2- 172.21.15.133:0/2861957714 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55a7a7c29c10 0x55a7a7c2c020 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.135+0000 7f07a3179b80 1 --2- 172.21.15.133:0/2861957714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7c15b10 0x55a7a7c25260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.135+0000 7f07a3179b80 1 --2- 172.21.15.133:0/2861957714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7a706b0 0x55a7a7c1dce0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.135+0000 7f0770ff9700 0 client.16131 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.135+0000 7f07a3179b80 1 --2- 172.21.15.133:0/2861957714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c14db0 0x55a7a7c1e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.135+0000 7f07a3179b80 1 -- 172.21.15.133:0/2861957714 >> 172.21.15.133:0/2861957714 conn(0x55a7a7c09f50 msgr2=0x55a7a7c0e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.135+0000 7f07a3179b80 1 -- 172.21.15.133:0/2861957714 shutdown_connections 2023-11-27T15:26:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.135+0000 7f07a3179b80 1 -- 172.21.15.133:0/2861957714 wait complete. 2023-11-27T15:26:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.135+0000 7f07a3179b80 20 client.16131 trim_cache size 0 max 16384 2023-11-27T15:26:41.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:40 smithi174 ceph-mon[131110]: pgmap v461: 129 pgs: 129 active+clean; 176 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 841 op/s 2023-11-27T15:26:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.152+0000 7f07a3179b80 1 Processor -- start 2023-11-27T15:26:41.156 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.152+0000 7f07a3179b80 1 -- start start 2023-11-27T15:26:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.152+0000 7f07a3179b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7a706b0 0x55a7a7c1be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.152+0000 7f07a3179b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7c14db0 0x55a7a7c1c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.152+0000 7f07a3179b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c15b10 0x55a7a7c25220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.152+0000 7f07a3179b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a7c26f20 con 0x55a7a7c14db0 2023-11-27T15:26:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.152+0000 7f07a3179b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a7c0c000 con 0x55a7a7a706b0 2023-11-27T15:26:41.157 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.152+0000 7f07a3179b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a7c09b90 con 0x55a7a7c15b10 2023-11-27T15:26:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.152+0000 7f078dd9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7a706b0 0x55a7a7c1be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.152+0000 7f078dd9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7a706b0 0x55a7a7c1be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:34898/0 (socket says 172.21.15.133:34898) 2023-11-27T15:26:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.152+0000 7f078dd9c700 1 -- 172.21.15.133:0/2805243939 learned_addr learned my addr 172.21.15.133:0/2805243939 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.152+0000 7f078d59b700 1 --2- 172.21.15.133:0/2805243939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7c14db0 0x55a7a7c1c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.152+0000 7f078dd9c700 1 --2- 172.21.15.133:0/2805243939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c15b10 0x55a7a7c25220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.153+0000 7f078dd9c700 1 -- 172.21.15.133:0/2805243939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c15b10 msgr2=0x55a7a7c25220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:41.158 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.153+0000 7f078dd9c700 1 --2- 172.21.15.133:0/2805243939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c15b10 0x55a7a7c25220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.153+0000 7f078dd9c700 1 -- 172.21.15.133:0/2805243939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7c14db0 msgr2=0x55a7a7c1c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.153+0000 7f078dd9c700 1 --2- 172.21.15.133:0/2805243939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7c14db0 0x55a7a7c1c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.153+0000 7f078dd9c700 1 -- 172.21.15.133:0/2805243939 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7a7b2be80 con 0x55a7a7a706b0 2023-11-27T15:26:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.153+0000 7f078d59b700 1 --2- 172.21.15.133:0/2805243939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7c14db0 0x55a7a7c1c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.153+0000 7f078dd9c700 1 --2- 172.21.15.133:0/2805243939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7a706b0 0x55a7a7c1be50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0780002230 tx=0x7f0780006160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5262a94a4d5b78d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.153+0000 7f0773fff700 1 -- 172.21.15.133:0/2805243939 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078000e040 con 0x55a7a7a706b0 2023-11-27T15:26:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.153+0000 7f0773fff700 1 -- 172.21.15.133:0/2805243939 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f078000f040 con 0x55a7a7a706b0 2023-11-27T15:26:41.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.153+0000 7f0773fff700 1 -- 172.21.15.133:0/2805243939 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780031a70 con 0x55a7a7a706b0 2023-11-27T15:26:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.154+0000 7f07a3179b80 1 -- 172.21.15.133:0/2805243939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7a706b0 msgr2=0x55a7a7c1be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.154+0000 7f07a3179b80 1 --2- 172.21.15.133:0/2805243939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7a706b0 0x55a7a7c1be50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0780002230 tx=0x7f0780006160 comp rx=0 tx=0).stop 2023-11-27T15:26:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.154+0000 7f07a3179b80 1 -- 172.21.15.133:0/2805243939 shutdown_connections 2023-11-27T15:26:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.154+0000 7f07a3179b80 1 --2- 172.21.15.133:0/2805243939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c15b10 0x55a7a7c25220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.154+0000 7f07a3179b80 1 --2- 172.21.15.133:0/2805243939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7a706b0 0x55a7a7c1be50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.154+0000 7f07a3179b80 1 --2- 172.21.15.133:0/2805243939 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7c14db0 0x55a7a7c1c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.154+0000 7f07a3179b80 1 -- 172.21.15.133:0/2805243939 >> 172.21.15.133:0/2805243939 conn(0x55a7a7c09f50 msgr2=0x55a7a7c30940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.155+0000 7f07a3179b80 1 -- 172.21.15.133:0/2805243939 shutdown_connections 2023-11-27T15:26:41.160 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.155+0000 7f07a3179b80 1 -- 172.21.15.133:0/2805243939 wait complete. 2023-11-27T15:26:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.155+0000 7f07a3179b80 1 -- start start 2023-11-27T15:26:41.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.155+0000 7f07a3179b80 1 Processor -- start 2023-11-27T15:26:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.156+0000 7f07a3179b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7a706b0 0x55a7a7c20e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.156+0000 7f07a3179b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7c14db0 0x55a7a7c21370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.156+0000 7f07a3179b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c15b10 0x55a7a7c24390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.156+0000 7f07a3179b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a7c248d0 con 0x55a7a7c14db0 2023-11-27T15:26:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.156+0000 7f07a3179b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a7c24a40 con 0x55a7a7a706b0 2023-11-27T15:26:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.156+0000 7f07a3179b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7a7c24c00 con 0x55a7a7c15b10 2023-11-27T15:26:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.156+0000 7f07a3179b80 10 client.0 _ll_register_callbacks cb 0x7f0780007c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.156+0000 7f078dd9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7a706b0 0x55a7a7c20e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:41.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.156+0000 7f078dd9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7a706b0 0x55a7a7c20e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:34910/0 (socket says 172.21.15.133:34910) 2023-11-27T15:26:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.156+0000 7f078dd9c700 1 -- 172.21.15.133:0/1782684608 learned_addr learned my addr 172.21.15.133:0/1782684608 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.156+0000 7f078d59b700 1 --2- 172.21.15.133:0/1782684608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7c14db0 0x55a7a7c21370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.156+0000 7f078dd9c700 1 --2- 172.21.15.133:0/1782684608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c15b10 0x55a7a7c24390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.157+0000 7f078dd9c700 1 -- 172.21.15.133:0/1782684608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c15b10 msgr2=0x55a7a7c24390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.157+0000 7f078dd9c700 1 --2- 172.21.15.133:0/1782684608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c15b10 0x55a7a7c24390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.157+0000 7f078dd9c700 1 -- 172.21.15.133:0/1782684608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7c14db0 msgr2=0x55a7a7c21370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.157+0000 7f078dd9c700 1 --2- 172.21.15.133:0/1782684608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7c14db0 0x55a7a7c21370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.157+0000 7f078dd9c700 1 -- 172.21.15.133:0/1782684608 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7a7b2c130 con 0x55a7a7a706b0 2023-11-27T15:26:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.157+0000 7f078dd9c700 1 --2- 172.21.15.133:0/1782684608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7a706b0 0x55a7a7c20e30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f078001ba10 tx=0x7f078000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c4e6e725131e9e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.157+0000 7f0770ff9700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:26:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.157+0000 7f0770ff9700 1 -- 172.21.15.133:0/1782684608 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078000e040 con 0x55a7a7a706b0 2023-11-27T15:26:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.157+0000 7f07a3179b80 1 -- 172.21.15.133:0/1782684608 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07840054e0 con 0x55a7a7a706b0 2023-11-27T15:26:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.157+0000 7f0770ff9700 1 -- 172.21.15.133:0/1782684608 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f07800317f0 con 0x55a7a7a706b0 2023-11-27T15:26:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.157+0000 7f0770ff9700 1 -- 172.21.15.133:0/1782684608 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f078001bed0 con 0x55a7a7a706b0 2023-11-27T15:26:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.157+0000 7f07a3179b80 10 client.26222 fetch_fsmap learned FSMap version 27 2023-11-27T15:26:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.157+0000 7f07a3179b80 1 -- 172.21.15.133:0/1782684608 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a7a797cc00 con 0x55a7a7a706b0 2023-11-27T15:26:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.158+0000 7f0770ff9700 1 -- 172.21.15.133:0/1782684608 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07840054e0 con 0x55a7a7a706b0 2023-11-27T15:26:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.158+0000 7f0770ff9700 1 -- 172.21.15.133:0/1782684608 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f0780019070 con 0x55a7a7a706b0 2023-11-27T15:26:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.158+0000 7f07a3179b80 10 client.26222 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:26:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.158+0000 7f07a3179b80 10 client.26222 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:26:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.158+0000 7f07a3179b80 20 client.26222 populate_metadata read hostname 'smithi133' 2023-11-27T15:26:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.158+0000 7f07a3179b80 1 --2- 172.21.15.133:0/1782684608 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55a7a7c7e3d0 0x55a7a7c807e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.158+0000 7f07a3179b80 4 client.26222 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:26:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.158+0000 7f07a3179b80 1 -- 172.21.15.133:0/1782684608 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a7a7b34600 con 0x55a7a7c7e3d0 2023-11-27T15:26:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.158+0000 7f078d59b700 1 --2- 172.21.15.133:0/1782684608 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55a7a7c7e3d0 0x55a7a7c807e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.159+0000 7f0770ff9700 10 client.26222 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.159+0000 7f078d59b700 1 --2- 172.21.15.133:0/1782684608 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55a7a7c7e3d0 0x55a7a7c807e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3e0469d82fa6bd9 server_cookie=32c04bc8925e38d0 in_seq=0 out_seq=0 2023-11-27T15:26:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.898+0000 7f0770ff9700 1 -- 172.21.15.133:0/1782684608 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a7a7b34600 con 0x55a7a7c7e3d0 2023-11-27T15:26:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.898+0000 7f0770ff9700 10 client.26222 handle_command_reply: tid=0 2023-11-27T15:26:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.898+0000 7f07a3179b80 1 client.26222 shutdown 2023-11-27T15:26:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.899+0000 7f07a3179b80 1 -- 172.21.15.133:0/1782684608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7a706b0 msgr2=0x55a7a7c20e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.899+0000 7f07a3179b80 1 --2- 172.21.15.133:0/1782684608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7a706b0 0x55a7a7c20e30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f078001ba10 tx=0x7f078000b330 comp rx=0 tx=0).stop 2023-11-27T15:26:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.899+0000 7f07a3179b80 1 -- 172.21.15.133:0/1782684608 shutdown_connections 2023-11-27T15:26:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.899+0000 7f07a3179b80 1 --2- 172.21.15.133:0/1782684608 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55a7a7c7e3d0 0x55a7a7c807e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.900+0000 7f07a3179b80 1 --2- 172.21.15.133:0/1782684608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a7a7c15b10 0x55a7a7c24390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.900+0000 7f07a3179b80 1 --2- 172.21.15.133:0/1782684608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7a7a706b0 0x55a7a7c20e30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.900+0000 7f07a3179b80 1 --2- 172.21.15.133:0/1782684608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a7a7c14db0 0x55a7a7c21370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.900+0000 7f07a3179b80 1 -- 172.21.15.133:0/1782684608 >> 172.21.15.133:0/1782684608 conn(0x55a7a7c09f50 msgr2=0x55a7a7c1be50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.900+0000 7f0770ff9700 0 client.26222 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:41.905 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.900+0000 7f07a3179b80 1 -- 172.21.15.133:0/1782684608 shutdown_connections 2023-11-27T15:26:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.901+0000 7f07a3179b80 1 -- 172.21.15.133:0/1782684608 wait complete. 2023-11-27T15:26:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.906+0000 7f07a3179b80 20 client.26222 trim_cache size 0 max 16384 2023-11-27T15:26:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:26:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:26:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:26:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:26:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.908+0000 7f078f7ff700 1 -- 172.21.15.133:0/311084695 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f07740a1150 msgr2=0x7f07740a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.908+0000 7f078f7ff700 1 --2- 172.21.15.133:0/311084695 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f07740a1150 0x7f07740a3600 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0778009890 tx=0x7f0778009820 comp rx=0 tx=0).stop 2023-11-27T15:26:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.908+0000 7f078f7ff700 1 -- 172.21.15.133:0/311084695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07881014b0 msgr2=0x7f0788198820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.908+0000 7f078f7ff700 1 --2- 172.21.15.133:0/311084695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07881014b0 0x7f0788198820 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f077800eb90 tx=0x7f077800eea0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.909+0000 7f078f7ff700 1 -- 172.21.15.133:0/311084695 shutdown_connections 2023-11-27T15:26:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.909+0000 7f078f7ff700 1 --2- 172.21.15.133:0/311084695 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f07740a1150 0x7f07740a3600 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.909+0000 7f078f7ff700 1 --2- 172.21.15.133:0/311084695 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0788101e40 0x7f078819f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.909+0000 7f078f7ff700 1 --2- 172.21.15.133:0/311084695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07881002c0 0x7f07881982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.909+0000 7f078f7ff700 1 --2- 172.21.15.133:0/311084695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07881014b0 0x7f0788198820 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.909+0000 7f078f7ff700 1 -- 172.21.15.133:0/311084695 >> 172.21.15.133:0/311084695 conn(0x7f07880fb8e0 msgr2=0x7f0788106d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.909+0000 7f078f7ff700 1 -- 172.21.15.133:0/311084695 shutdown_connections 2023-11-27T15:26:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:41.909+0000 7f078f7ff700 1 -- 172.21.15.133:0/311084695 wait complete. 2023-11-27T15:26:43.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:42 smithi174 ceph-mon[131110]: pgmap v462: 129 pgs: 129 active+clean; 182 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 564 op/s 2023-11-27T15:26:46.290 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:26:46.292 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:44 smithi174 ceph-mon[131110]: pgmap v463: 129 pgs: 129 active+clean; 186 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 405 op/s 2023-11-27T15:26:46.292 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:42 smithi177 ceph-mon[131504]: pgmap v462: 129 pgs: 129 active+clean; 182 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 564 op/s 2023-11-27T15:26:46.292 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:44 smithi177 ceph-mon[131504]: pgmap v463: 129 pgs: 129 active+clean; 186 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 405 op/s 2023-11-27T15:26:46.292 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:42 smithi133 ceph-mon[97860]: pgmap v462: 129 pgs: 129 active+clean; 182 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.7 MiB/s wr, 564 op/s 2023-11-27T15:26:46.292 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:44 smithi133 ceph-mon[97860]: pgmap v463: 129 pgs: 129 active+clean; 186 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.1 MiB/s wr, 405 op/s 2023-11-27T15:26:46.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.624+0000 7fe2237ff700 1 -- 172.21.15.133:0/1490202851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe21c107af0 msgr2=0x7fe21c107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:46.629 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.624+0000 7fe2237ff700 1 --2- 172.21.15.133:0/1490202851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe21c107af0 0x7fe21c107f40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009990 tx=0x7fe20c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.625+0000 7fe2237ff700 1 -- 172.21.15.133:0/1490202851 shutdown_connections 2023-11-27T15:26:46.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.625+0000 7fe2237ff700 1 --2- 172.21.15.133:0/1490202851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe21c108480 0x7fe21c10ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.625+0000 7fe2237ff700 1 --2- 172.21.15.133:0/1490202851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe21c106d10 0x7fe21c107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.625+0000 7fe2237ff700 1 --2- 172.21.15.133:0/1490202851 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe21c107af0 0x7fe21c107f40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.625+0000 7fe2237ff700 1 -- 172.21.15.133:0/1490202851 >> 172.21.15.133:0/1490202851 conn(0x7fe21c075930 msgr2=0x7fe21c077d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:46.630 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.626+0000 7fe2237ff700 1 -- 172.21.15.133:0/1490202851 shutdown_connections 2023-11-27T15:26:46.631 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.626+0000 7fe2237ff700 1 -- 172.21.15.133:0/1490202851 wait complete. 2023-11-27T15:26:46.635 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.631+0000 7fe2237ff700 1 Processor -- start 2023-11-27T15:26:46.635 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.631+0000 7fe2237ff700 1 -- start start 2023-11-27T15:26:46.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.632+0000 7fe2237ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe21c106d10 0x7fe21c06f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:46.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.632+0000 7fe2237ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe21c107af0 0x7fe21c06f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:46.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.633+0000 7fe22159b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe21c106d10 0x7fe21c06f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:46.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.633+0000 7fe2237ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe21c108480 0x7fe21c1ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:46.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.633+0000 7fe22159b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe21c106d10 0x7fe21c06f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:38158/0 (socket says 172.21.15.133:38158) 2023-11-27T15:26:46.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.633+0000 7fe22159b700 1 -- 172.21.15.133:0/256624038 learned_addr learned my addr 172.21.15.133:0/256624038 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:46.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.633+0000 7fe2237ff700 1 -- 172.21.15.133:0/256624038 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c0728d0 con 0x7fe21c107af0 2023-11-27T15:26:46.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.633+0000 7fe2237ff700 1 -- 172.21.15.133:0/256624038 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c072a40 con 0x7fe21c108480 2023-11-27T15:26:46.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.633+0000 7fe2237ff700 1 -- 172.21.15.133:0/256624038 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c1acdd0 con 0x7fe21c106d10 2023-11-27T15:26:46.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.633+0000 7fe220d9a700 1 --2- 172.21.15.133:0/256624038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe21c107af0 0x7fe21c06f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:46.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.633+0000 7fe221d9c700 1 --2- 172.21.15.133:0/256624038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe21c108480 0x7fe21c1ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:46.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.634+0000 7fe22159b700 1 -- 172.21.15.133:0/256624038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe21c108480 msgr2=0x7fe21c1ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:46.639 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.634+0000 7fe22159b700 1 --2- 172.21.15.133:0/256624038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe21c108480 0x7fe21c1ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.634+0000 7fe22159b700 1 -- 172.21.15.133:0/256624038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe21c107af0 msgr2=0x7fe21c06f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:46.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.634+0000 7fe22159b700 1 --2- 172.21.15.133:0/256624038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe21c107af0 0x7fe21c06f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.635+0000 7fe22159b700 1 -- 172.21.15.133:0/256624038 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20c0095f0 con 0x7fe21c106d10 2023-11-27T15:26:46.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.635+0000 7fe220d9a700 1 --2- 172.21.15.133:0/256624038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe21c107af0 0x7fe21c06f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:46.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.635+0000 7fe22159b700 1 --2- 172.21.15.133:0/256624038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe21c106d10 0x7fe21c06f2b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe21400ada0 tx=0x7fe21400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:46.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.635+0000 7fe2127fc700 1 -- 172.21.15.133:0/256624038 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21400f070 con 0x7fe21c106d10 2023-11-27T15:26:46.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.635+0000 7fe2127fc700 1 -- 172.21.15.133:0/256624038 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe21400c980 con 0x7fe21c106d10 2023-11-27T15:26:46.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.635+0000 7fe2127fc700 1 -- 172.21.15.133:0/256624038 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2140043a0 con 0x7fe21c106d10 2023-11-27T15:26:46.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.636+0000 7fe2237ff700 1 -- 172.21.15.133:0/256624038 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21c1122c0 con 0x7fe21c106d10 2023-11-27T15:26:46.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.636+0000 7fe2237ff700 1 -- 172.21.15.133:0/256624038 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21c112130 con 0x7fe21c106d10 2023-11-27T15:26:46.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.650+0000 7fe2127fc700 1 -- 172.21.15.133:0/256624038 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fe21400caf0 con 0x7fe21c106d10 2023-11-27T15:26:46.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.655+0000 7fe2127fc700 1 --2- 172.21.15.133:0/256624038 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe2080a1350 0x7fe2080a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:46.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.655+0000 7fe237089b80 1 Processor -- start 2023-11-27T15:26:46.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.655+0000 7fe2127fc700 1 -- 172.21.15.133:0/256624038 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe2140cc330 con 0x7fe21c106d10 2023-11-27T15:26:46.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.656+0000 7fe220d9a700 1 --2- 172.21.15.133:0/256624038 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe2080a1350 0x7fe2080a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:46.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.656+0000 7fe237089b80 1 -- start start 2023-11-27T15:26:46.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.657+0000 7fe237089b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a40c00 0x557a56a41010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:46.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.657+0000 7fe237089b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a41550 0x557a56a41b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:46.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.657+0000 7fe237089b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a42110 0x557a56a42580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:46.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.657+0000 7fe237089b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a56946060 con 0x557a56a40c00 2023-11-27T15:26:46.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.657+0000 7fe237089b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a569988a0 con 0x557a56a42110 2023-11-27T15:26:46.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.657+0000 7fe237089b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a56991d30 con 0x557a56a41550 2023-11-27T15:26:46.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.657+0000 7fe22159b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a40c00 0x557a56a41010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:46.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.657+0000 7fe22159b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a40c00 0x557a56a41010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60934/0 (socket says 172.21.15.133:60934) 2023-11-27T15:26:46.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.657+0000 7fe22159b700 1 -- 172.21.15.133:0/3948983069 learned_addr learned my addr 172.21.15.133:0/3948983069 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:46.663 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.658+0000 7fe221d9c700 1 --2- 172.21.15.133:0/3948983069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a42110 0x557a56a42580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:46.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.658+0000 7fe220d9a700 1 --2- 172.21.15.133:0/3948983069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a41550 0x557a56a41b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:46.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.659+0000 7fe221d9c700 1 -- 172.21.15.133:0/3948983069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a41550 msgr2=0x557a56a41b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:46.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.659+0000 7fe221d9c700 1 --2- 172.21.15.133:0/3948983069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a41550 0x557a56a41b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.659+0000 7fe221d9c700 1 -- 172.21.15.133:0/3948983069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a40c00 msgr2=0x557a56a41010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:46.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.659+0000 7fe221d9c700 1 --2- 172.21.15.133:0/3948983069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a40c00 0x557a56a41010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.664 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.659+0000 7fe221d9c700 1 -- 172.21.15.133:0/3948983069 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a56813630 con 0x557a56a42110 2023-11-27T15:26:46.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.660+0000 7fe22159b700 1 --2- 172.21.15.133:0/3948983069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a40c00 0x557a56a41010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:26:46.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.660+0000 7fe220d9a700 1 --2- 172.21.15.133:0/3948983069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a41550 0x557a56a41b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:26:46.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.660+0000 7fe221d9c700 1 --2- 172.21.15.133:0/3948983069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a42110 0x557a56a42580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe21800bbf0 tx=0x7fe21800bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df3b381554d4ac2f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:46.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.660+0000 7fe220d9a700 1 --2- 172.21.15.133:0/256624038 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe2080a1350 0x7fe2080a3800 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0036f0 tx=0x7fe20c003680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:46.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.660+0000 7fe2077fe700 1 -- 172.21.15.133:0/3948983069 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21800fb10 con 0x557a56a42110 2023-11-27T15:26:46.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.661+0000 7fe2077fe700 1 -- 172.21.15.133:0/3948983069 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe218004cf0 con 0x557a56a42110 2023-11-27T15:26:46.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.661+0000 7fe2077fe700 1 -- 172.21.15.133:0/3948983069 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe218005650 con 0x557a56a42110 2023-11-27T15:26:46.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.662+0000 7fe237089b80 1 -- 172.21.15.133:0/3948983069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a42110 msgr2=0x557a56a42580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:46.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.662+0000 7fe237089b80 1 --2- 172.21.15.133:0/3948983069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a42110 0x557a56a42580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe21800bbf0 tx=0x7fe21800bf00 comp rx=0 tx=0).stop 2023-11-27T15:26:46.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.664+0000 7fe237089b80 1 -- 172.21.15.133:0/3948983069 shutdown_connections 2023-11-27T15:26:46.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.664+0000 7fe237089b80 1 --2- 172.21.15.133:0/3948983069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a42110 0x557a56a42580 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.664+0000 7fe237089b80 1 --2- 172.21.15.133:0/3948983069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a40c00 0x557a56a41010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.664+0000 7fe237089b80 1 --2- 172.21.15.133:0/3948983069 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a41550 0x557a56a41b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.664+0000 7fe237089b80 1 -- 172.21.15.133:0/3948983069 >> 172.21.15.133:0/3948983069 conn(0x557a56a36360 msgr2=0x557a56a3b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:46.670 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.666+0000 7fe237089b80 1 -- 172.21.15.133:0/3948983069 shutdown_connections 2023-11-27T15:26:46.671 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.666+0000 7fe237089b80 1 -- 172.21.15.133:0/3948983069 wait complete. 2023-11-27T15:26:46.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.668+0000 7fe237089b80 1 -- start start 2023-11-27T15:26:46.672 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.668+0000 7fe237089b80 1 Processor -- start 2023-11-27T15:26:46.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.670+0000 7fe237089b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a40c00 0x557a56a4a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:46.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.670+0000 7fe237089b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a41550 0x557a56a4ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:46.674 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.670+0000 7fe237089b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a42110 0x557a56a51b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:46.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.670+0000 7fe237089b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a569338e0 con 0x557a56a41550 2023-11-27T15:26:46.675 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.670+0000 7fe221d9c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a42110 0x557a56a51b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:46.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.671+0000 7fe221d9c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a42110 0x557a56a51b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:38162/0 (socket says 172.21.15.133:38162) 2023-11-27T15:26:46.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.671+0000 7fe221d9c700 1 -- 172.21.15.133:0/1847686993 learned_addr learned my addr 172.21.15.133:0/1847686993 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:46.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.671+0000 7fe237089b80 1 -- 172.21.15.133:0/1847686993 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a56933580 con 0x557a56a40c00 2023-11-27T15:26:46.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.671+0000 7fe220d9a700 1 --2- 172.21.15.133:0/1847686993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a41550 0x557a56a4ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:46.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.671+0000 7fe237089b80 1 -- 172.21.15.133:0/1847686993 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a56815920 con 0x557a56a42110 2023-11-27T15:26:46.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.671+0000 7fe237089b80 10 client.0 _ll_register_callbacks cb 0x557a56951780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:46.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.671+0000 7fe22159b700 1 --2- 172.21.15.133:0/1847686993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a40c00 0x557a56a4a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:46.676 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.672+0000 7fe221d9c700 1 -- 172.21.15.133:0/1847686993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a40c00 msgr2=0x557a56a4a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:46.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.672+0000 7fe221d9c700 1 --2- 172.21.15.133:0/1847686993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a40c00 0x557a56a4a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.677 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.672+0000 7fe221d9c700 1 -- 172.21.15.133:0/1847686993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a41550 msgr2=0x557a56a4ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:46.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.673+0000 7fe221d9c700 1 --2- 172.21.15.133:0/1847686993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a41550 0x557a56a4ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.673+0000 7fe221d9c700 1 -- 172.21.15.133:0/1847686993 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21800b820 con 0x557a56a42110 2023-11-27T15:26:46.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.673+0000 7fe22159b700 1 --2- 172.21.15.133:0/1847686993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a40c00 0x557a56a4a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:46.678 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.673+0000 7fe220d9a700 1 --2- 172.21.15.133:0/1847686993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a41550 0x557a56a4ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:46.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.674+0000 7fe221d9c700 1 --2- 172.21.15.133:0/1847686993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a42110 0x557a56a51b60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe218005600 tx=0x7fe2180056e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=582be613f36e1ed2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:46.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.674+0000 7fe204ff9700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:26:46.679 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.675+0000 7fe204ff9700 1 -- 172.21.15.133:0/1847686993 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe218020070 con 0x557a56a42110 2023-11-27T15:26:46.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.675+0000 7fe204ff9700 1 -- 172.21.15.133:0/1847686993 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe218004a60 con 0x557a56a42110 2023-11-27T15:26:46.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.676+0000 7fe237089b80 1 -- 172.21.15.133:0/1847686993 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a56a33b20 con 0x557a56a42110 2023-11-27T15:26:46.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.676+0000 7fe204ff9700 1 -- 172.21.15.133:0/1847686993 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe21800f910 con 0x557a56a42110 2023-11-27T15:26:46.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.676+0000 7fe237089b80 10 client.26212 fetch_fsmap learned FSMap version 27 2023-11-27T15:26:46.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.676+0000 7fe237089b80 1 -- 172.21.15.133:0/1847686993 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557a56958190 con 0x557a56a42110 2023-11-27T15:26:46.681 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.677+0000 7fe204ff9700 1 -- 172.21.15.133:0/1847686993 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557a56a33b20 con 0x557a56a42110 2023-11-27T15:26:46.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.677+0000 7fe204ff9700 1 -- 172.21.15.133:0/1847686993 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fe218011bf0 con 0x557a56a42110 2023-11-27T15:26:46.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.678+0000 7fe237089b80 10 client.26212 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:26:46.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.678+0000 7fe237089b80 10 client.26212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:26:46.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.678+0000 7fe237089b80 20 client.26212 populate_metadata read hostname 'smithi133' 2023-11-27T15:26:46.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.678+0000 7fe237089b80 1 --2- 172.21.15.133:0/1847686993 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x557a56a561f0 0x557a56a58600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:46.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.678+0000 7fe237089b80 4 client.26212 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:26:46.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.678+0000 7fe237089b80 1 -- 172.21.15.133:0/1847686993 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a569540a0 con 0x557a56a561f0 2023-11-27T15:26:46.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.679+0000 7fe22159b700 1 --2- 172.21.15.133:0/1847686993 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x557a56a561f0 0x557a56a58600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:46.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.680+0000 7fe22159b700 1 --2- 172.21.15.133:0/1847686993 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x557a56a561f0 0x557a56a58600 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3239276f43b42b8c server_cookie=5b4f2c16e873e7e8 in_seq=0 out_seq=0 2023-11-27T15:26:46.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.680+0000 7fe204ff9700 10 client.26212 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:46.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.683+0000 7fe204ff9700 1 -- 172.21.15.133:0/1847686993 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557a569540a0 con 0x557a56a561f0 2023-11-27T15:26:46.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.683+0000 7fe204ff9700 10 client.26212 handle_command_reply: tid=0 2023-11-27T15:26:46.688 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.684+0000 7fe237089b80 1 client.26212 shutdown 2023-11-27T15:26:46.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.684+0000 7fe237089b80 1 -- 172.21.15.133:0/1847686993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a42110 msgr2=0x557a56a51b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:46.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.684+0000 7fe237089b80 1 --2- 172.21.15.133:0/1847686993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a42110 0x557a56a51b60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe218005600 tx=0x7fe2180056e0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.685+0000 7fe237089b80 1 -- 172.21.15.133:0/1847686993 shutdown_connections 2023-11-27T15:26:46.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.685+0000 7fe237089b80 1 --2- 172.21.15.133:0/1847686993 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x557a56a561f0 0x557a56a58600 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.689 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.685+0000 7fe237089b80 1 --2- 172.21.15.133:0/1847686993 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a42110 0x557a56a51b60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.685+0000 7fe237089b80 1 --2- 172.21.15.133:0/1847686993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a40c00 0x557a56a4a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.685+0000 7fe237089b80 1 --2- 172.21.15.133:0/1847686993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a41550 0x557a56a4ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.685+0000 7fe237089b80 1 -- 172.21.15.133:0/1847686993 >> 172.21.15.133:0/1847686993 conn(0x557a56a36360 msgr2=0x557a56a3ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:46.690 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.685+0000 7fe204ff9700 0 client.26212 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:46.691 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.687+0000 7fe237089b80 1 -- 172.21.15.133:0/1847686993 shutdown_connections 2023-11-27T15:26:46.692 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.688+0000 7fe237089b80 1 -- 172.21.15.133:0/1847686993 wait complete. 2023-11-27T15:26:46.693 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.689+0000 7fe237089b80 20 client.26212 trim_cache size 0 max 16384 2023-11-27T15:26:46.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.748+0000 7fe237089b80 1 Processor -- start 2023-11-27T15:26:46.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.748+0000 7fe237089b80 1 -- start start 2023-11-27T15:26:46.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.749+0000 7fe237089b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a40c00 0x557a56a4a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:46.754 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.749+0000 7fe237089b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a41550 0x557a56a51b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:46.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.749+0000 7fe237089b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a42110 0x557a56a4ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:46.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.749+0000 7fe22159b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a40c00 0x557a56a4a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:46.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.750+0000 7fe22159b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a40c00 0x557a56a4a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:38168/0 (socket says 172.21.15.133:38168) 2023-11-27T15:26:46.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.750+0000 7fe220d9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a41550 0x557a56a51b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:46.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.750+0000 7fe220d9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a41550 0x557a56a51b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60950/0 (socket says 172.21.15.133:60950) 2023-11-27T15:26:46.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.750+0000 7fe22159b700 1 -- 172.21.15.133:0/2850263291 learned_addr learned my addr 172.21.15.133:0/2850263291 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:46.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.750+0000 7fe237089b80 1 -- 172.21.15.133:0/2850263291 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a56a52b60 con 0x557a56a41550 2023-11-27T15:26:46.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.750+0000 7fe237089b80 1 -- 172.21.15.133:0/2850263291 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a569540a0 con 0x557a56a42110 2023-11-27T15:26:46.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.750+0000 7fe237089b80 1 -- 172.21.15.133:0/2850263291 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a56a41f90 con 0x557a56a40c00 2023-11-27T15:26:46.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.750+0000 7fe221d9c700 1 --2- 172.21.15.133:0/2850263291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a42110 0x557a56a4ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:46.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.750+0000 7fe220d9a700 1 -- 172.21.15.133:0/2850263291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a40c00 msgr2=0x557a56a4a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:46.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.750+0000 7fe220d9a700 1 --2- 172.21.15.133:0/2850263291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a40c00 0x557a56a4a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.750+0000 7fe220d9a700 1 -- 172.21.15.133:0/2850263291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a42110 msgr2=0x557a56a4ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:46.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.750+0000 7fe220d9a700 1 --2- 172.21.15.133:0/2850263291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a42110 0x557a56a4ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.750+0000 7fe220d9a700 1 -- 172.21.15.133:0/2850263291 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a56957ee0 con 0x557a56a41550 2023-11-27T15:26:46.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.750+0000 7fe221d9c700 1 --2- 172.21.15.133:0/2850263291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a42110 0x557a56a4ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:46.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.751+0000 7fe220d9a700 1 --2- 172.21.15.133:0/2850263291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a41550 0x557a56a51b20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe20c003790 tx=0x7fe20c01a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40add5f5a6bdc268 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:46.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.753+0000 7fe207fff700 1 -- 172.21.15.133:0/2850263291 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c00bab0 con 0x557a56a41550 2023-11-27T15:26:46.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.753+0000 7fe207fff700 1 -- 172.21.15.133:0/2850263291 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe20c029040 con 0x557a56a41550 2023-11-27T15:26:46.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.754+0000 7fe237089b80 1 -- 172.21.15.133:0/2850263291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a41550 msgr2=0x557a56a51b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:46.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.754+0000 7fe207fff700 1 -- 172.21.15.133:0/2850263291 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c02fc80 con 0x557a56a41550 2023-11-27T15:26:46.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.754+0000 7fe237089b80 1 --2- 172.21.15.133:0/2850263291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a41550 0x557a56a51b20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe20c003790 tx=0x7fe20c01a070 comp rx=0 tx=0).stop 2023-11-27T15:26:46.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.754+0000 7fe237089b80 1 -- 172.21.15.133:0/2850263291 shutdown_connections 2023-11-27T15:26:46.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.754+0000 7fe237089b80 1 --2- 172.21.15.133:0/2850263291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a42110 0x557a56a4ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.754+0000 7fe237089b80 1 --2- 172.21.15.133:0/2850263291 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a40c00 0x557a56a4a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.754+0000 7fe237089b80 1 --2- 172.21.15.133:0/2850263291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a41550 0x557a56a51b20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.754+0000 7fe237089b80 1 -- 172.21.15.133:0/2850263291 >> 172.21.15.133:0/2850263291 conn(0x557a56a36360 msgr2=0x557a56a52450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:46.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.754+0000 7fe237089b80 1 -- 172.21.15.133:0/2850263291 shutdown_connections 2023-11-27T15:26:46.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.754+0000 7fe237089b80 1 -- 172.21.15.133:0/2850263291 wait complete. 2023-11-27T15:26:46.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.755+0000 7fe237089b80 1 -- start start 2023-11-27T15:26:46.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.755+0000 7fe237089b80 1 Processor -- start 2023-11-27T15:26:46.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.756+0000 7fe237089b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a40c00 0x557a56a4dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:46.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.756+0000 7fe237089b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a41550 0x557a56a4e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:46.761 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.756+0000 7fe237089b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a42110 0x557a56a51210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:46.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.756+0000 7fe237089b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a56a51750 con 0x557a56a40c00 2023-11-27T15:26:46.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.756+0000 7fe237089b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a56a518c0 con 0x557a56a42110 2023-11-27T15:26:46.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.756+0000 7fe237089b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a56a51a80 con 0x557a56a41550 2023-11-27T15:26:46.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.756+0000 7fe237089b80 10 client.0 _ll_register_callbacks cb 0x7fe214005ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:46.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.757+0000 7fe22159b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a40c00 0x557a56a4dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:46.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.757+0000 7fe221d9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a42110 0x557a56a51210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:46.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.757+0000 7fe22159b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a40c00 0x557a56a4dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:60954/0 (socket says 172.21.15.133:60954) 2023-11-27T15:26:46.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.757+0000 7fe22159b700 1 -- 172.21.15.133:0/2774308735 learned_addr learned my addr 172.21.15.133:0/2774308735 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:46.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.757+0000 7fe22159b700 1 -- 172.21.15.133:0/2774308735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a41550 msgr2=0x557a56a4e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:46.763 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.757+0000 7fe22159b700 1 --2- 172.21.15.133:0/2774308735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a41550 0x557a56a4e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.763 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.757+0000 7fe22159b700 1 -- 172.21.15.133:0/2774308735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a42110 msgr2=0x557a56a51210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:46.763 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.757+0000 7fe22159b700 1 --2- 172.21.15.133:0/2774308735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a42110 0x557a56a51210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.763 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.757+0000 7fe22159b700 1 -- 172.21.15.133:0/2774308735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21c10f670 con 0x557a56a40c00 2023-11-27T15:26:46.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.758+0000 7fe22159b700 1 --2- 172.21.15.133:0/2774308735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a40c00 0x557a56a4dcb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe214095920 tx=0x7fe214095760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4b7d7c3b0ae3 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:46.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.758+0000 7fe204ff9700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:26:46.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.759+0000 7fe204ff9700 1 -- 172.21.15.133:0/2774308735 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214099420 con 0x557a56a40c00 2023-11-27T15:26:46.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.759+0000 7fe204ff9700 1 -- 172.21.15.133:0/2774308735 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fe21409d450 con 0x557a56a40c00 2023-11-27T15:26:46.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.759+0000 7fe237089b80 1 -- 172.21.15.133:0/2774308735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe218004cf0 con 0x557a56a40c00 2023-11-27T15:26:46.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.759+0000 7fe204ff9700 1 -- 172.21.15.133:0/2774308735 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2140a17e0 con 0x557a56a40c00 2023-11-27T15:26:46.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.759+0000 7fe237089b80 10 client.16152 fetch_fsmap learned FSMap version 27 2023-11-27T15:26:46.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.759+0000 7fe237089b80 1 -- 172.21.15.133:0/2774308735 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557a567a8c60 con 0x557a56a40c00 2023-11-27T15:26:46.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.760+0000 7fe204ff9700 1 -- 172.21.15.133:0/2774308735 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe2140ba070 con 0x557a56a40c00 2023-11-27T15:26:46.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.760+0000 7fe204ff9700 1 -- 172.21.15.133:0/2774308735 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fe214099680 con 0x557a56a40c00 2023-11-27T15:26:46.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.760+0000 7fe237089b80 10 client.16152 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:26:46.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.760+0000 7fe237089b80 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:26:46.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.760+0000 7fe237089b80 20 client.16152 populate_metadata read hostname 'smithi133' 2023-11-27T15:26:46.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.760+0000 7fe237089b80 1 --2- 172.21.15.133:0/2774308735 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x557a56aab2d0 0x557a56aad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:46.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.760+0000 7fe237089b80 4 client.16152 mds_command: new command op to 24560 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:26:46.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.760+0000 7fe237089b80 1 -- 172.21.15.133:0/2774308735 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557a56960660 con 0x557a56aab2d0 2023-11-27T15:26:46.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.761+0000 7fe220d9a700 1 --2- 172.21.15.133:0/2774308735 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x557a56aab2d0 0x557a56aad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:46.767 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.762+0000 7fe220d9a700 1 --2- 172.21.15.133:0/2774308735 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x557a56aab2d0 0x557a56aad6e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e8592026c78682 server_cookie=829a94be5ace8a6e in_seq=0 out_seq=0 2023-11-27T15:26:46.767 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.762+0000 7fe204ff9700 10 client.16152 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:46.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.769+0000 7fe204ff9700 1 -- 172.21.15.133:0/2774308735 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x557a56960660 con 0x557a56aab2d0 2023-11-27T15:26:46.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.769+0000 7fe204ff9700 10 client.16152 handle_command_reply: tid=0 2023-11-27T15:26:46.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.769+0000 7fe237089b80 1 client.16152 shutdown 2023-11-27T15:26:46.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.770+0000 7fe237089b80 1 -- 172.21.15.133:0/2774308735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a40c00 msgr2=0x557a56a4dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:46.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.770+0000 7fe237089b80 1 --2- 172.21.15.133:0/2774308735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a40c00 0x557a56a4dcb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe214095920 tx=0x7fe214095760 comp rx=0 tx=0).stop 2023-11-27T15:26:46.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.770+0000 7fe237089b80 1 -- 172.21.15.133:0/2774308735 shutdown_connections 2023-11-27T15:26:46.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.770+0000 7fe237089b80 1 --2- 172.21.15.133:0/2774308735 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x557a56aab2d0 0x557a56aad6e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.770+0000 7fe237089b80 1 --2- 172.21.15.133:0/2774308735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a56a42110 0x557a56a51210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.770+0000 7fe237089b80 1 --2- 172.21.15.133:0/2774308735 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557a56a40c00 0x557a56a4dcb0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.770+0000 7fe237089b80 1 --2- 172.21.15.133:0/2774308735 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x557a56a41550 0x557a56a4e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.770+0000 7fe237089b80 1 -- 172.21.15.133:0/2774308735 >> 172.21.15.133:0/2774308735 conn(0x557a56a36360 msgr2=0x557a56a45050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:46.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.770+0000 7fe204ff9700 0 client.16152 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:46.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.771+0000 7fe237089b80 1 -- 172.21.15.133:0/2774308735 shutdown_connections 2023-11-27T15:26:46.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.771+0000 7fe237089b80 1 -- 172.21.15.133:0/2774308735 wait complete. 2023-11-27T15:26:46.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.772+0000 7fe237089b80 20 client.16152 trim_cache size 0 max 16384 2023-11-27T15:26:46.776 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:26:46.777 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (2 inodes in the stack)", 2023-11-27T15:26:46.777 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-11-27T15:26:46.777 INFO:teuthology.orchestra.run.smithi133.stdout: "b6638f49-a408-46a9-83f0-3995bb2029f5": { 2023-11-27T15:26:46.777 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-11-27T15:26:46.777 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "b6638f49-a408-46a9-83f0-3995bb2029f5", 2023-11-27T15:26:46.777 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-11-27T15:26:46.777 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:26:46.777 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:26:46.777 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:26:46.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.773+0000 7fe2237ff700 1 -- 172.21.15.133:0/256624038 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe2080a1350 msgr2=0x7fe2080a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:46.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.773+0000 7fe2237ff700 1 --2- 172.21.15.133:0/256624038 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe2080a1350 0x7fe2080a3800 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0036f0 tx=0x7fe20c003680 comp rx=0 tx=0).stop 2023-11-27T15:26:46.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.773+0000 7fe2237ff700 1 -- 172.21.15.133:0/256624038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe21c106d10 msgr2=0x7fe21c06f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:46.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.773+0000 7fe2237ff700 1 --2- 172.21.15.133:0/256624038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe21c106d10 0x7fe21c06f2b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe21400ada0 tx=0x7fe21400c330 comp rx=0 tx=0).stop 2023-11-27T15:26:46.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.773+0000 7fe2237ff700 1 -- 172.21.15.133:0/256624038 shutdown_connections 2023-11-27T15:26:46.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.773+0000 7fe2237ff700 1 --2- 172.21.15.133:0/256624038 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fe2080a1350 0x7fe2080a3800 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.773+0000 7fe2237ff700 1 --2- 172.21.15.133:0/256624038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe21c108480 0x7fe21c1ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.773+0000 7fe2237ff700 1 --2- 172.21.15.133:0/256624038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe21c106d10 0x7fe21c06f2b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.773+0000 7fe2237ff700 1 --2- 172.21.15.133:0/256624038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe21c107af0 0x7fe21c06f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:46.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.774+0000 7fe2237ff700 1 -- 172.21.15.133:0/256624038 >> 172.21.15.133:0/256624038 conn(0x7fe21c075930 msgr2=0x7fe21c077ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:46.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.775+0000 7fe2237ff700 1 -- 172.21.15.133:0/256624038 shutdown_connections 2023-11-27T15:26:46.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:26:46.775+0000 7fe2237ff700 1 -- 172.21.15.133:0/256624038 wait complete. 2023-11-27T15:26:46.824 INFO:tasks.cephfs.filesystem:scrub status for tag:b6638f49-a408-46a9-83f0-3995bb2029f5 - {'path': '/', 'tag': 'b6638f49-a408-46a9-83f0-3995bb2029f5', 'options': 'recursive,force'} 2023-11-27T15:26:47.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:46 smithi174 ceph-mon[131110]: pgmap v464: 129 pgs: 129 active+clean; 191 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 609 op/s 2023-11-27T15:26:47.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:46 smithi133 ceph-mon[97860]: pgmap v464: 129 pgs: 129 active+clean; 191 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 609 op/s 2023-11-27T15:26:47.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:46 smithi177 ceph-mon[131504]: pgmap v464: 129 pgs: 129 active+clean; 191 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.2 MiB/s wr, 609 op/s 2023-11-27T15:26:49.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:48 smithi133 ceph-mon[97860]: pgmap v465: 129 pgs: 129 active+clean; 201 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 697 op/s 2023-11-27T15:26:49.091 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:48 smithi177 ceph-mon[131504]: pgmap v465: 129 pgs: 129 active+clean; 201 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 697 op/s 2023-11-27T15:26:49.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:48 smithi174 ceph-mon[131110]: pgmap v465: 129 pgs: 129 active+clean; 201 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.0 MiB/s wr, 697 op/s 2023-11-27T15:26:51.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:50 smithi174 ceph-mon[131110]: pgmap v466: 129 pgs: 129 active+clean; 203 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 731 op/s 2023-11-27T15:26:51.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:50 smithi133 ceph-mon[97860]: pgmap v466: 129 pgs: 129 active+clean; 203 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 731 op/s 2023-11-27T15:26:51.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:50 smithi177 ceph-mon[131504]: pgmap v466: 129 pgs: 129 active+clean; 203 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.0 MiB/s wr, 731 op/s 2023-11-27T15:26:53.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:52 smithi174 ceph-mon[131110]: pgmap v467: 129 pgs: 129 active+clean; 213 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.3 MiB/s wr, 732 op/s 2023-11-27T15:26:53.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:52 smithi133 ceph-mon[97860]: pgmap v467: 129 pgs: 129 active+clean; 213 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.3 MiB/s wr, 732 op/s 2023-11-27T15:26:53.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:52 smithi177 ceph-mon[131504]: pgmap v467: 129 pgs: 129 active+clean; 213 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.3 MiB/s wr, 732 op/s 2023-11-27T15:26:54.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:53 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:26:54.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:53 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:26:54.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:53 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:26:54.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:53 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:26:54.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:53 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:26:54.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:53 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:26:55.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:54 smithi174 ceph-mon[131110]: pgmap v468: 129 pgs: 129 active+clean; 224 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 716 op/s 2023-11-27T15:26:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.247+0000 7fc32679d700 1 -- 172.21.15.133:0/3829793591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc31809b070 msgr2=0x7fc31809d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.247+0000 7fc32679d700 1 --2- 172.21.15.133:0/3829793591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc31809b070 0x7fc31809d4d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc320065e20 tx=0x7fc320067260 comp rx=0 tx=0).stop 2023-11-27T15:26:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.247+0000 7fc32679d700 1 -- 172.21.15.133:0/3829793591 shutdown_connections 2023-11-27T15:26:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.247+0000 7fc32679d700 1 --2- 172.21.15.133:0/3829793591 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc31809b070 0x7fc31809d4d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.247+0000 7fc32679d700 1 --2- 172.21.15.133:0/3829793591 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc318095e30 0x7fc318098210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.247+0000 7fc32679d700 1 --2- 172.21.15.133:0/3829793591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc318098750 0x7fc31809ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.247+0000 7fc32679d700 1 -- 172.21.15.133:0/3829793591 >> 172.21.15.133:0/3829793591 conn(0x7fc31808f8b0 msgr2=0x7fc318091d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.248+0000 7fc32679d700 1 -- 172.21.15.133:0/3829793591 shutdown_connections 2023-11-27T15:26:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.248+0000 7fc32679d700 1 -- 172.21.15.133:0/3829793591 wait complete. 2023-11-27T15:26:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.249+0000 7fc32679d700 1 Processor -- start 2023-11-27T15:26:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.249+0000 7fc32679d700 1 -- start start 2023-11-27T15:26:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.250+0000 7fc32679d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc318095e30 0x7fc31813c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.250+0000 7fc32679d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc318098750 0x7fc31813c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.250+0000 7fc32679d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc31809b070 0x7fc3181435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.250+0000 7fc32679d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc318143b90 con 0x7fc318098750 2023-11-27T15:26:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.250+0000 7fc32579b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc318095e30 0x7fc31813c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.250+0000 7fc32579b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc318095e30 0x7fc31813c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:47508/0 (socket says 172.21.15.133:47508) 2023-11-27T15:26:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.250+0000 7fc32579b700 1 -- 172.21.15.133:0/479287714 learned_addr learned my addr 172.21.15.133:0/479287714 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.250+0000 7fc32679d700 1 -- 172.21.15.133:0/479287714 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc318143cd0 con 0x7fc31809b070 2023-11-27T15:26:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.250+0000 7fc32679d700 1 -- 172.21.15.133:0/479287714 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc318143e60 con 0x7fc318095e30 2023-11-27T15:26:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.250+0000 7fc324f9a700 1 --2- 172.21.15.133:0/479287714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc318098750 0x7fc31813c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.250+0000 7fc325f9c700 1 --2- 172.21.15.133:0/479287714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc31809b070 0x7fc3181435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.251+0000 7fc325f9c700 1 -- 172.21.15.133:0/479287714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc318095e30 msgr2=0x7fc31813c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.251+0000 7fc325f9c700 1 --2- 172.21.15.133:0/479287714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc318095e30 0x7fc31813c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.251+0000 7fc325f9c700 1 -- 172.21.15.133:0/479287714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc318098750 msgr2=0x7fc31813c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.251+0000 7fc325f9c700 1 --2- 172.21.15.133:0/479287714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc318098750 0x7fc31813c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.251+0000 7fc325f9c700 1 -- 172.21.15.133:0/479287714 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3200670d0 con 0x7fc31809b070 2023-11-27T15:26:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.251+0000 7fc325f9c700 1 --2- 172.21.15.133:0/479287714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc31809b070 0x7fc3181435c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc32002d1e0 tx=0x7fc320066da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.251+0000 7fc3167fc700 1 -- 172.21.15.133:0/479287714 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc320073ea0 con 0x7fc31809b070 2023-11-27T15:26:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.251+0000 7fc3167fc700 1 -- 172.21.15.133:0/479287714 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc32007ec40 con 0x7fc31809b070 2023-11-27T15:26:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.251+0000 7fc32679d700 1 -- 172.21.15.133:0/479287714 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3180a0ce0 con 0x7fc31809b070 2023-11-27T15:26:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.251+0000 7fc3167fc700 1 -- 172.21.15.133:0/479287714 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32007a040 con 0x7fc31809b070 2023-11-27T15:26:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.251+0000 7fc32679d700 1 -- 172.21.15.133:0/479287714 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3180a0b50 con 0x7fc31809b070 2023-11-27T15:26:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.255+0000 7fc3167fc700 1 -- 172.21.15.133:0/479287714 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fc320072540 con 0x7fc31809b070 2023-11-27T15:26:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.257+0000 7fc3167fc700 1 --2- 172.21.15.133:0/479287714 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc30c0a11d0 0x7fc30c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.257+0000 7fc3167fc700 1 -- 172.21.15.133:0/479287714 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc32012f550 con 0x7fc31809b070 2023-11-27T15:26:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.258+0000 7fc32579b700 1 --2- 172.21.15.133:0/479287714 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc30c0a11d0 0x7fc30c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.258+0000 7fc33b358b80 1 Processor -- start 2023-11-27T15:26:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.258+0000 7fc33b358b80 1 -- start start 2023-11-27T15:26:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.258+0000 7fc32579b700 1 --2- 172.21.15.133:0/479287714 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc30c0a11d0 0x7fc30c0a3680 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00ad20 tx=0x7fc31c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.259+0000 7fc33b358b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e5a24c6b0 0x563e5a3f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.259+0000 7fc33b358b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a3f53e0 0x563e5a3f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.259+0000 7fc33b358b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f61b0 0x563e5a3f6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.259+0000 7fc33b358b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5a2f6000 con 0x563e5a3f61b0 2023-11-27T15:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.259+0000 7fc33b358b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5a348840 con 0x563e5a3f53e0 2023-11-27T15:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.259+0000 7fc33b358b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5a341cd0 con 0x563e5a24c6b0 2023-11-27T15:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.259+0000 7fc324f9a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a3f53e0 0x563e5a3f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.259+0000 7fc324f9a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a3f53e0 0x563e5a3f5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:57656/0 (socket says 172.21.15.133:57656) 2023-11-27T15:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.259+0000 7fc324f9a700 1 -- 172.21.15.133:0/3253065226 learned_addr learned my addr 172.21.15.133:0/3253065226 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.259+0000 7fc32579b700 1 --2- 172.21.15.133:0/3253065226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e5a24c6b0 0x563e5a3f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.259+0000 7fc324f9a700 1 -- 172.21.15.133:0/3253065226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e5a24c6b0 msgr2=0x563e5a3f4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.259+0000 7fc324f9a700 1 --2- 172.21.15.133:0/3253065226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e5a24c6b0 0x563e5a3f4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.259+0000 7fc325f9c700 1 --2- 172.21.15.133:0/3253065226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f61b0 0x563e5a3f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.259+0000 7fc324f9a700 1 -- 172.21.15.133:0/3253065226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f61b0 msgr2=0x563e5a3f6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.259+0000 7fc324f9a700 1 --2- 172.21.15.133:0/3253065226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f61b0 0x563e5a3f6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.259+0000 7fc324f9a700 1 -- 172.21.15.133:0/3253065226 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e5a1c35d0 con 0x563e5a3f53e0 2023-11-27T15:26:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.259+0000 7fc325f9c700 1 --2- 172.21.15.133:0/3253065226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f61b0 0x563e5a3f6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:26:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.260+0000 7fc324f9a700 1 --2- 172.21.15.133:0/3253065226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a3f53e0 0x563e5a3f5c70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc310009d80 tx=0x7fc310011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb903e6c87a7650a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.260+0000 7fc30b7fe700 1 -- 172.21.15.133:0/3253065226 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc310004030 con 0x563e5a3f53e0 2023-11-27T15:26:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.260+0000 7fc30b7fe700 1 -- 172.21.15.133:0/3253065226 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc31001a850 con 0x563e5a3f53e0 2023-11-27T15:26:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.260+0000 7fc30b7fe700 1 -- 172.21.15.133:0/3253065226 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc310003970 con 0x563e5a3f53e0 2023-11-27T15:26:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.261+0000 7fc33b358b80 1 -- 172.21.15.133:0/3253065226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a3f53e0 msgr2=0x563e5a3f5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.261+0000 7fc33b358b80 1 --2- 172.21.15.133:0/3253065226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a3f53e0 0x563e5a3f5c70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc310009d80 tx=0x7fc310011720 comp rx=0 tx=0).stop 2023-11-27T15:26:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.261+0000 7fc33b358b80 1 -- 172.21.15.133:0/3253065226 shutdown_connections 2023-11-27T15:26:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.261+0000 7fc33b358b80 1 --2- 172.21.15.133:0/3253065226 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f61b0 0x563e5a3f6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.261+0000 7fc33b358b80 1 --2- 172.21.15.133:0/3253065226 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e5a24c6b0 0x563e5a3f4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.261+0000 7fc33b358b80 1 --2- 172.21.15.133:0/3253065226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a3f53e0 0x563e5a3f5c70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.261+0000 7fc33b358b80 1 -- 172.21.15.133:0/3253065226 >> 172.21.15.133:0/3253065226 conn(0x563e5a3e61a0 msgr2=0x563e5a3eb320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.262+0000 7fc33b358b80 1 -- 172.21.15.133:0/3253065226 shutdown_connections 2023-11-27T15:26:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.262+0000 7fc33b358b80 1 -- 172.21.15.133:0/3253065226 wait complete. 2023-11-27T15:26:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.263+0000 7fc33b358b80 1 -- start start 2023-11-27T15:26:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.263+0000 7fc33b358b80 1 Processor -- start 2023-11-27T15:26:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.264+0000 7fc33b358b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e5a24c6b0 0x563e5a3f9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.264+0000 7fc33b358b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f53e0 0x563e5a3fa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.264+0000 7fc33b358b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a3f61b0 0x563e5a401410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.264+0000 7fc33b358b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5a2e3880 con 0x563e5a3f53e0 2023-11-27T15:26:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.264+0000 7fc33b358b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5a2e3520 con 0x563e5a3f61b0 2023-11-27T15:26:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.264+0000 7fc33b358b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5a1c58c0 con 0x563e5a24c6b0 2023-11-27T15:26:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.264+0000 7fc32579b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e5a24c6b0 0x563e5a3f9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:55.270 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.264+0000 7fc325f9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a3f61b0 0x563e5a401410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.264+0000 7fc32579b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e5a24c6b0 0x563e5a3f9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:47522/0 (socket says 172.21.15.133:47522) 2023-11-27T15:26:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.264+0000 7fc32579b700 1 -- 172.21.15.133:0/3606820448 learned_addr learned my addr 172.21.15.133:0/3606820448 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.264+0000 7fc324f9a700 1 --2- 172.21.15.133:0/3606820448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f53e0 0x563e5a3fa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.264+0000 7fc33b358b80 10 client.0 _ll_register_callbacks cb 0x563e5a301720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.265+0000 7fc325f9c700 1 -- 172.21.15.133:0/3606820448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e5a24c6b0 msgr2=0x563e5a3f9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.265+0000 7fc325f9c700 1 --2- 172.21.15.133:0/3606820448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e5a24c6b0 0x563e5a3f9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.265+0000 7fc325f9c700 1 -- 172.21.15.133:0/3606820448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f53e0 msgr2=0x563e5a3fa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.265+0000 7fc325f9c700 1 --2- 172.21.15.133:0/3606820448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f53e0 0x563e5a3fa3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.265+0000 7fc325f9c700 1 -- 172.21.15.133:0/3606820448 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3100099e0 con 0x563e5a3f61b0 2023-11-27T15:26:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.265+0000 7fc325f9c700 1 --2- 172.21.15.133:0/3606820448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a3f61b0 0x563e5a401410 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x563e5a3ec160 tx=0x7fc320072c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8066381d6c455ea1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.266+0000 7fc308ff9700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:26:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.266+0000 7fc308ff9700 1 -- 172.21.15.133:0/3606820448 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32011c0d0 con 0x563e5a3f61b0 2023-11-27T15:26:55.273 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.266+0000 7fc308ff9700 1 -- 172.21.15.133:0/3606820448 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc320104a60 con 0x563e5a3f61b0 2023-11-27T15:26:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.266+0000 7fc33b358b80 1 -- 172.21.15.133:0/3606820448 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e5a3e3960 con 0x563e5a3f61b0 2023-11-27T15:26:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.266+0000 7fc308ff9700 1 -- 172.21.15.133:0/3606820448 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3200fc900 con 0x563e5a3f61b0 2023-11-27T15:26:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.267+0000 7fc33b358b80 10 client.26249 fetch_fsmap learned FSMap version 27 2023-11-27T15:26:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.267+0000 7fc33b358b80 1 -- 172.21.15.133:0/3606820448 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563e5a308130 con 0x563e5a3f61b0 2023-11-27T15:26:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.267+0000 7fc308ff9700 1 -- 172.21.15.133:0/3606820448 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563e5a3e3960 con 0x563e5a3f61b0 2023-11-27T15:26:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.267+0000 7fc308ff9700 1 -- 172.21.15.133:0/3606820448 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fc3200fbcb0 con 0x563e5a3f61b0 2023-11-27T15:26:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.267+0000 7fc33b358b80 10 client.26249 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:26:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.267+0000 7fc33b358b80 10 client.26249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:26:55.274 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.267+0000 7fc33b358b80 20 client.26249 populate_metadata read hostname 'smithi133' 2023-11-27T15:26:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.267+0000 7fc33b358b80 1 --2- 172.21.15.133:0/3606820448 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x563e5a4058c0 0x563e5a407cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.267+0000 7fc33b358b80 4 client.26249 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:26:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.267+0000 7fc33b358b80 1 -- 172.21.15.133:0/3606820448 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e5a304040 con 0x563e5a4058c0 2023-11-27T15:26:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.268+0000 7fc32579b700 1 --2- 172.21.15.133:0/3606820448 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x563e5a4058c0 0x563e5a407cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.268+0000 7fc308ff9700 10 client.26249 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.269+0000 7fc32579b700 1 --2- 172.21.15.133:0/3606820448 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x563e5a4058c0 0x563e5a407cd0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=800326361d78fc76 server_cookie=f774362d7f54c19d in_seq=0 out_seq=0 2023-11-27T15:26:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.269+0000 7fc308ff9700 1 -- 172.21.15.133:0/3606820448 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563e5a304040 con 0x563e5a4058c0 2023-11-27T15:26:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.269+0000 7fc308ff9700 10 client.26249 handle_command_reply: tid=0 2023-11-27T15:26:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.270+0000 7fc33b358b80 1 client.26249 shutdown 2023-11-27T15:26:55.275 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.270+0000 7fc33b358b80 1 -- 172.21.15.133:0/3606820448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a3f61b0 msgr2=0x563e5a401410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.270+0000 7fc33b358b80 1 --2- 172.21.15.133:0/3606820448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a3f61b0 0x563e5a401410 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x563e5a3ec160 tx=0x7fc320072c70 comp rx=0 tx=0).stop 2023-11-27T15:26:55.276 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.270+0000 7fc33b358b80 1 -- 172.21.15.133:0/3606820448 shutdown_connections 2023-11-27T15:26:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.270+0000 7fc33b358b80 1 --2- 172.21.15.133:0/3606820448 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x563e5a4058c0 0x563e5a407cd0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.270+0000 7fc33b358b80 1 --2- 172.21.15.133:0/3606820448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a3f61b0 0x563e5a401410 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.270+0000 7fc33b358b80 1 --2- 172.21.15.133:0/3606820448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e5a24c6b0 0x563e5a3f9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.270+0000 7fc33b358b80 1 --2- 172.21.15.133:0/3606820448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f53e0 0x563e5a3fa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.270+0000 7fc33b358b80 1 -- 172.21.15.133:0/3606820448 >> 172.21.15.133:0/3606820448 conn(0x563e5a3e61a0 msgr2=0x563e5a3eaa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.270+0000 7fc308ff9700 0 client.26249 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.271+0000 7fc33b358b80 1 -- 172.21.15.133:0/3606820448 shutdown_connections 2023-11-27T15:26:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.271+0000 7fc33b358b80 1 -- 172.21.15.133:0/3606820448 wait complete. 2023-11-27T15:26:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.271+0000 7fc33b358b80 20 client.26249 trim_cache size 0 max 16384 2023-11-27T15:26:55.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:54 smithi133 ceph-mon[97860]: pgmap v468: 129 pgs: 129 active+clean; 224 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 716 op/s 2023-11-27T15:26:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.290+0000 7fc33b358b80 1 Processor -- start 2023-11-27T15:26:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.290+0000 7fc33b358b80 1 -- start start 2023-11-27T15:26:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.291+0000 7fc33b358b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a24c6b0 0x563e5a3f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.291+0000 7fc33b358b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e5a3f53e0 0x563e5a3f3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.291+0000 7fc33b358b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f61b0 0x563e5a401330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.291+0000 7fc33b358b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5a304070 con 0x563e5a3f61b0 2023-11-27T15:26:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.291+0000 7fc33b358b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5a3f9a30 con 0x563e5a24c6b0 2023-11-27T15:26:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.291+0000 7fc33b358b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5a3ecba0 con 0x563e5a3f53e0 2023-11-27T15:26:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.291+0000 7fc325f9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f61b0 0x563e5a401330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.291+0000 7fc325f9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f61b0 0x563e5a401330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47394/0 (socket says 172.21.15.133:47394) 2023-11-27T15:26:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.291+0000 7fc325f9c700 1 -- 172.21.15.133:0/1381712944 learned_addr learned my addr 172.21.15.133:0/1381712944 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.291+0000 7fc324f9a700 1 --2- 172.21.15.133:0/1381712944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e5a3f53e0 0x563e5a3f3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.291+0000 7fc32579b700 1 --2- 172.21.15.133:0/1381712944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a24c6b0 0x563e5a3f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.292+0000 7fc324f9a700 1 -- 172.21.15.133:0/1381712944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a24c6b0 msgr2=0x563e5a3f2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.292+0000 7fc324f9a700 1 --2- 172.21.15.133:0/1381712944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a24c6b0 0x563e5a3f2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.292+0000 7fc324f9a700 1 -- 172.21.15.133:0/1381712944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f61b0 msgr2=0x563e5a401330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.292+0000 7fc324f9a700 1 --2- 172.21.15.133:0/1381712944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f61b0 0x563e5a401330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.292+0000 7fc324f9a700 1 -- 172.21.15.133:0/1381712944 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e5a307e80 con 0x563e5a3f53e0 2023-11-27T15:26:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.292+0000 7fc32579b700 1 --2- 172.21.15.133:0/1381712944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a24c6b0 0x563e5a3f2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:26:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.292+0000 7fc325f9c700 1 --2- 172.21.15.133:0/1381712944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f61b0 0x563e5a401330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.292+0000 7fc324f9a700 1 --2- 172.21.15.133:0/1381712944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e5a3f53e0 0x563e5a3f3000 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc310011b20 tx=0x7fc310009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84c302eeaae1382a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.292+0000 7fc30bfff700 1 -- 172.21.15.133:0/1381712944 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31001abc0 con 0x563e5a3f53e0 2023-11-27T15:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.292+0000 7fc30bfff700 1 -- 172.21.15.133:0/1381712944 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc310011eb0 con 0x563e5a3f53e0 2023-11-27T15:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.292+0000 7fc30bfff700 1 -- 172.21.15.133:0/1381712944 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc310003b80 con 0x563e5a3f53e0 2023-11-27T15:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.293+0000 7fc33b358b80 1 -- 172.21.15.133:0/1381712944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e5a3f53e0 msgr2=0x563e5a3f3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.293+0000 7fc33b358b80 1 --2- 172.21.15.133:0/1381712944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e5a3f53e0 0x563e5a3f3000 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc310011b20 tx=0x7fc310009f80 comp rx=0 tx=0).stop 2023-11-27T15:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.293+0000 7fc33b358b80 1 -- 172.21.15.133:0/1381712944 shutdown_connections 2023-11-27T15:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.293+0000 7fc33b358b80 1 --2- 172.21.15.133:0/1381712944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f61b0 0x563e5a401330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.293+0000 7fc33b358b80 1 --2- 172.21.15.133:0/1381712944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a24c6b0 0x563e5a3f2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.293+0000 7fc33b358b80 1 --2- 172.21.15.133:0/1381712944 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e5a3f53e0 0x563e5a3f3000 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.293+0000 7fc33b358b80 1 -- 172.21.15.133:0/1381712944 >> 172.21.15.133:0/1381712944 conn(0x563e5a3e61a0 msgr2=0x563e5a3f5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.293+0000 7fc33b358b80 1 -- 172.21.15.133:0/1381712944 shutdown_connections 2023-11-27T15:26:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.293+0000 7fc33b358b80 1 -- 172.21.15.133:0/1381712944 wait complete. 2023-11-27T15:26:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.294+0000 7fc33b358b80 1 -- start start 2023-11-27T15:26:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.294+0000 7fc33b358b80 1 Processor -- start 2023-11-27T15:26:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.295+0000 7fc33b358b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a24c6b0 0x563e5a3fd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.295+0000 7fc33b358b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f53e0 0x563e5a3fdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.295+0000 7fc33b358b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e5a3f61b0 0x563e5a400930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.295+0000 7fc33b358b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5a400e70 con 0x563e5a3f53e0 2023-11-27T15:26:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.295+0000 7fc33b358b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5a400fe0 con 0x563e5a24c6b0 2023-11-27T15:26:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.295+0000 7fc33b358b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e5a4011a0 con 0x563e5a3f61b0 2023-11-27T15:26:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.295+0000 7fc33b358b80 10 client.0 _ll_register_callbacks cb 0x7fc3200fc000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:26:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.296+0000 7fc324f9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f53e0 0x563e5a3fdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.296+0000 7fc324f9a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f53e0 0x563e5a3fdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47406/0 (socket says 172.21.15.133:47406) 2023-11-27T15:26:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.296+0000 7fc324f9a700 1 -- 172.21.15.133:0/4076113102 learned_addr learned my addr 172.21.15.133:0/4076113102 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:26:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.296+0000 7fc32579b700 1 --2- 172.21.15.133:0/4076113102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a24c6b0 0x563e5a3fd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.296+0000 7fc324f9a700 1 -- 172.21.15.133:0/4076113102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e5a3f61b0 msgr2=0x563e5a400930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-27T15:26:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.296+0000 7fc324f9a700 1 --2- 172.21.15.133:0/4076113102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e5a3f61b0 0x563e5a400930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.296+0000 7fc324f9a700 1 -- 172.21.15.133:0/4076113102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a24c6b0 msgr2=0x563e5a3fd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:26:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.296+0000 7fc324f9a700 1 --2- 172.21.15.133:0/4076113102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a24c6b0 0x563e5a3fd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.296+0000 7fc324f9a700 1 -- 172.21.15.133:0/4076113102 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e5a308130 con 0x563e5a3f53e0 2023-11-27T15:26:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.296+0000 7fc32579b700 1 --2- 172.21.15.133:0/4076113102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a24c6b0 0x563e5a3fd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:26:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.297+0000 7fc324f9a700 1 --2- 172.21.15.133:0/4076113102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f53e0 0x563e5a3fdaa0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc31000a010 tx=0x7fc31002d7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9a25bda3e2a92f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:26:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.297+0000 7fc308ff9700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:26:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.297+0000 7fc308ff9700 1 -- 172.21.15.133:0/4076113102 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31002de50 con 0x563e5a3f53e0 2023-11-27T15:26:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.297+0000 7fc33b358b80 1 -- 172.21.15.133:0/4076113102 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc31001a850 con 0x563e5a3f53e0 2023-11-27T15:26:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.297+0000 7fc308ff9700 1 -- 172.21.15.133:0/4076113102 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc31001be40 con 0x563e5a3f53e0 2023-11-27T15:26:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.297+0000 7fc308ff9700 1 -- 172.21.15.133:0/4076113102 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc31002de50 con 0x563e5a3f53e0 2023-11-27T15:26:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.298+0000 7fc33b358b80 10 client.16164 fetch_fsmap learned FSMap version 27 2023-11-27T15:26:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.298+0000 7fc33b358b80 1 -- 172.21.15.133:0/4076113102 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563e5a1c35d0 con 0x563e5a3f53e0 2023-11-27T15:26:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.298+0000 7fc308ff9700 1 -- 172.21.15.133:0/4076113102 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc31001a850 con 0x563e5a3f53e0 2023-11-27T15:26:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.298+0000 7fc308ff9700 1 -- 172.21.15.133:0/4076113102 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fc31002a050 con 0x563e5a3f53e0 2023-11-27T15:26:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.298+0000 7fc33b358b80 10 client.16164 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:26:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.298+0000 7fc33b358b80 10 client.16164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:26:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.298+0000 7fc33b358b80 20 client.16164 populate_metadata read hostname 'smithi133' 2023-11-27T15:26:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.298+0000 7fc33b358b80 1 --2- 172.21.15.133:0/4076113102 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x563e5a45a410 0x563e5a45c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:26:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.298+0000 7fc33b358b80 4 client.16164 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:26:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.298+0000 7fc33b358b80 1 -- 172.21.15.133:0/4076113102 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563e5a310600 con 0x563e5a45a410 2023-11-27T15:26:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.298+0000 7fc32579b700 1 --2- 172.21.15.133:0/4076113102 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x563e5a45a410 0x563e5a45c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:26:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.299+0000 7fc32579b700 1 --2- 172.21.15.133:0/4076113102 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x563e5a45a410 0x563e5a45c820 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=642e96c44c9f8fa9 server_cookie=ef5b74287ce414dd in_seq=0 out_seq=0 2023-11-27T15:26:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:55.300+0000 7fc308ff9700 10 client.16164 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:55.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:54 smithi177 ceph-mon[131504]: pgmap v468: 129 pgs: 129 active+clean; 224 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 716 op/s 2023-11-27T15:26:56.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:55 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:26:56.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:55 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:26:56.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:55 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:26:56.280 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:55 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:26:56.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:55 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:26:56.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:55 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:26:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.872+0000 7fc308ff9700 1 -- 172.21.15.133:0/4076113102 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563e5a310600 con 0x563e5a45a410 2023-11-27T15:26:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.872+0000 7fc308ff9700 10 client.16164 handle_command_reply: tid=0 2023-11-27T15:26:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.872+0000 7fc33b358b80 1 client.16164 shutdown 2023-11-27T15:26:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.872+0000 7fc33b358b80 1 -- 172.21.15.133:0/4076113102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f53e0 msgr2=0x563e5a3fdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:56.877 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.872+0000 7fc33b358b80 1 --2- 172.21.15.133:0/4076113102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f53e0 0x563e5a3fdaa0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc31000a010 tx=0x7fc31002d7a0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.873+0000 7fc33b358b80 1 -- 172.21.15.133:0/4076113102 shutdown_connections 2023-11-27T15:26:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.873+0000 7fc33b358b80 1 --2- 172.21.15.133:0/4076113102 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x563e5a45a410 0x563e5a45c820 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.873+0000 7fc33b358b80 1 --2- 172.21.15.133:0/4076113102 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563e5a3f61b0 0x563e5a400930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.873+0000 7fc33b358b80 1 --2- 172.21.15.133:0/4076113102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e5a24c6b0 0x563e5a3fd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.873+0000 7fc33b358b80 1 --2- 172.21.15.133:0/4076113102 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563e5a3f53e0 0x563e5a3fdaa0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.873+0000 7fc308ff9700 0 client.16164 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:26:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.873+0000 7fc33b358b80 1 -- 172.21.15.133:0/4076113102 >> 172.21.15.133:0/4076113102 conn(0x563e5a3e61a0 msgr2=0x563e5a3f9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:56.878 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.873+0000 7fc33b358b80 1 -- 172.21.15.133:0/4076113102 shutdown_connections 2023-11-27T15:26:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.873+0000 7fc33b358b80 1 -- 172.21.15.133:0/4076113102 wait complete. 2023-11-27T15:26:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.873+0000 7fc33b358b80 20 client.16164 trim_cache size 0 max 16384 2023-11-27T15:26:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:26:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:26:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:26:56.879 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:26:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.874+0000 7fc32679d700 1 -- 172.21.15.133:0/479287714 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc30c0a11d0 msgr2=0x7fc30c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.874+0000 7fc32679d700 1 --2- 172.21.15.133:0/479287714 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc30c0a11d0 0x7fc30c0a3680 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00ad20 tx=0x7fc31c005f90 comp rx=0 tx=0).stop 2023-11-27T15:26:56.880 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.874+0000 7fc32679d700 1 -- 172.21.15.133:0/479287714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc31809b070 msgr2=0x7fc3181435c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:26:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.874+0000 7fc32679d700 1 --2- 172.21.15.133:0/479287714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc31809b070 0x7fc3181435c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc32002d1e0 tx=0x7fc320066da0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.874+0000 7fc32679d700 1 -- 172.21.15.133:0/479287714 shutdown_connections 2023-11-27T15:26:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.874+0000 7fc32679d700 1 --2- 172.21.15.133:0/479287714 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc30c0a11d0 0x7fc30c0a3680 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.874+0000 7fc32679d700 1 --2- 172.21.15.133:0/479287714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc31809b070 0x7fc3181435c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.874+0000 7fc32679d700 1 --2- 172.21.15.133:0/479287714 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc318095e30 0x7fc31813c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.874+0000 7fc32679d700 1 --2- 172.21.15.133:0/479287714 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc318098750 0x7fc31813c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:26:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.874+0000 7fc32679d700 1 -- 172.21.15.133:0/479287714 >> 172.21.15.133:0/479287714 conn(0x7fc31808f8b0 msgr2=0x7fc318096a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:26:56.881 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.875+0000 7fc32679d700 1 -- 172.21.15.133:0/479287714 shutdown_connections 2023-11-27T15:26:56.882 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:26:56.875+0000 7fc32679d700 1 -- 172.21.15.133:0/479287714 wait complete. 2023-11-27T15:26:57.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:56 smithi133 ceph-mon[97860]: pgmap v469: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.7 MiB/s wr, 712 op/s 2023-11-27T15:26:57.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:56 smithi177 ceph-mon[131504]: pgmap v469: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.7 MiB/s wr, 712 op/s 2023-11-27T15:26:57.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:56 smithi174 ceph-mon[131110]: pgmap v469: 129 pgs: 129 active+clean; 233 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.7 MiB/s wr, 712 op/s 2023-11-27T15:26:59.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:26:58 smithi133 ceph-mon[97860]: pgmap v470: 129 pgs: 129 active+clean; 237 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.6 MiB/s wr, 599 op/s 2023-11-27T15:26:59.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:26:58 smithi177 ceph-mon[131504]: pgmap v470: 129 pgs: 129 active+clean; 237 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.6 MiB/s wr, 599 op/s 2023-11-27T15:26:59.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:26:58 smithi174 ceph-mon[131110]: pgmap v470: 129 pgs: 129 active+clean; 237 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.6 MiB/s wr, 599 op/s 2023-11-27T15:27:01.156 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:00 smithi174 ceph-mon[131110]: pgmap v471: 129 pgs: 129 active+clean; 239 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 607 op/s 2023-11-27T15:27:01.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:00 smithi133 ceph-mon[97860]: pgmap v471: 129 pgs: 129 active+clean; 239 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 607 op/s 2023-11-27T15:27:01.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:00 smithi177 ceph-mon[131504]: pgmap v471: 129 pgs: 129 active+clean; 239 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 607 op/s 2023-11-27T15:27:03.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:02 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:27:03.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:02 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:03.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:02 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:27:03.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:02 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:03.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:02 smithi174 ceph-mon[131110]: pgmap v472: 129 pgs: 129 active+clean; 250 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.3 MiB/s wr, 675 op/s 2023-11-27T15:27:03.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:02 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:27:03.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:02 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:03.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:02 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:27:03.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:02 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:03.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:02 smithi133 ceph-mon[97860]: pgmap v472: 129 pgs: 129 active+clean; 250 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.3 MiB/s wr, 675 op/s 2023-11-27T15:27:03.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:02 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:27:03.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:02 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:03.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:02 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:27:03.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:02 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:03.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:02 smithi177 ceph-mon[131504]: pgmap v472: 129 pgs: 129 active+clean; 250 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.3 MiB/s wr, 675 op/s 2023-11-27T15:27:04.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:03 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:27:04.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:03 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:04.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:03 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:27:04.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:03 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:04.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:03 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:27:04.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:03 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:04.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:03 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:27:04.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:03 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:04.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:03 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:27:04.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:03 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:04.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:03 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:27:04.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:03 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:04.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:03 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:27:04.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:03 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:04.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:03 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:27:04.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:03 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:04.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:03 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:27:04.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:03 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:05.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:04 smithi174 ceph-mon[131110]: pgmap v473: 129 pgs: 129 active+clean; 250 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.5 MiB/s wr, 661 op/s 2023-11-27T15:27:05.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:04 smithi133 ceph-mon[97860]: pgmap v473: 129 pgs: 129 active+clean; 250 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.5 MiB/s wr, 661 op/s 2023-11-27T15:27:05.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:04 smithi177 ceph-mon[131504]: pgmap v473: 129 pgs: 129 active+clean; 250 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.5 MiB/s wr, 661 op/s 2023-11-27T15:27:06.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:05 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:27:06.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:05 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:06.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:05 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:27:06.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:05 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:06.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:05 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:27:06.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:05 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:07.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:06 smithi133 ceph-mon[97860]: pgmap v474: 129 pgs: 129 active+clean; 260 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 681 op/s 2023-11-27T15:27:07.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:06 smithi177 ceph-mon[131504]: pgmap v474: 129 pgs: 129 active+clean; 260 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 681 op/s 2023-11-27T15:27:07.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:06 smithi174 ceph-mon[131110]: pgmap v474: 129 pgs: 129 active+clean; 260 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 681 op/s 2023-11-27T15:27:08.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:07 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:27:08.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:07 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:08.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:07 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:27:08.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:07 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:08.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:07 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:27:08.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:07 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:09.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:08 smithi133 ceph-mon[97860]: pgmap v475: 129 pgs: 129 active+clean; 267 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.3 MiB/s wr, 616 op/s 2023-11-27T15:27:09.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:08 smithi177 ceph-mon[131504]: pgmap v475: 129 pgs: 129 active+clean; 267 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.3 MiB/s wr, 616 op/s 2023-11-27T15:27:09.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:08 smithi174 ceph-mon[131110]: pgmap v475: 129 pgs: 129 active+clean; 267 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.3 MiB/s wr, 616 op/s 2023-11-27T15:27:10.126 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.122+0000 7f4afefff700 1 -- 172.21.15.133:0/3553666435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4af80721d0 msgr2=0x7f4af80725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:10.127 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.122+0000 7f4afefff700 1 --2- 172.21.15.133:0/3553666435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4af80721d0 0x7f4af80725c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009f90 tx=0x7f4ae800b330 comp rx=0 tx=0).stop 2023-11-27T15:27:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.123+0000 7f4afefff700 1 -- 172.21.15.133:0/3553666435 shutdown_connections 2023-11-27T15:27:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.123+0000 7f4afefff700 1 --2- 172.21.15.133:0/3553666435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4af81084a0 0x7f4af8110e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.128 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.123+0000 7f4afefff700 1 --2- 172.21.15.133:0/3553666435 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4af80721d0 0x7f4af80725c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.124+0000 7f4afefff700 1 --2- 172.21.15.133:0/3553666435 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4af8107af0 0x7f4af8107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.124+0000 7f4afefff700 1 -- 172.21.15.133:0/3553666435 >> 172.21.15.133:0/3553666435 conn(0x7f4af806db10 msgr2=0x7f4af806ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.125+0000 7f4afefff700 1 -- 172.21.15.133:0/3553666435 shutdown_connections 2023-11-27T15:27:10.129 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.126+0000 7f4afefff700 1 -- 172.21.15.133:0/3553666435 wait complete. 2023-11-27T15:27:10.131 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.127+0000 7f4afefff700 1 Processor -- start 2023-11-27T15:27:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.129+0000 7f4afefff700 1 -- start start 2023-11-27T15:27:10.133 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.130+0000 7f4afefff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4af80721d0 0x7f4af81a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.130+0000 7f4afefff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4af8107af0 0x7f4af81a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.130+0000 7f4afefff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4af81084a0 0x7f4af81a8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.130+0000 7f4afefff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af81a86f0 con 0x7f4af81084a0 2023-11-27T15:27:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.130+0000 7f4afefff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af81a8830 con 0x7f4af80721d0 2023-11-27T15:27:10.134 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.130+0000 7f4afefff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af81a89c0 con 0x7f4af8107af0 2023-11-27T15:27:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.130+0000 7f4aeffff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4af8107af0 0x7f4af81a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.130+0000 7f4aeffff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4af8107af0 0x7f4af81a10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:37830/0 (socket says 172.21.15.133:37830) 2023-11-27T15:27:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.130+0000 7f4aeffff700 1 -- 172.21.15.133:0/2749190441 learned_addr learned my addr 172.21.15.133:0/2749190441 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.131+0000 7f4afcd9b700 1 --2- 172.21.15.133:0/2749190441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4af80721d0 0x7f4af81a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:10.135 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.131+0000 7f4afd59c700 1 --2- 172.21.15.133:0/2749190441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4af81084a0 0x7f4af81a8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.132+0000 7f4aeffff700 1 -- 172.21.15.133:0/2749190441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4af80721d0 msgr2=0x7f4af81a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.132+0000 7f4aeffff700 1 --2- 172.21.15.133:0/2749190441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4af80721d0 0x7f4af81a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.132+0000 7f4aeffff700 1 -- 172.21.15.133:0/2749190441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4af81084a0 msgr2=0x7f4af81a8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.132+0000 7f4aeffff700 1 --2- 172.21.15.133:0/2749190441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4af81084a0 0x7f4af81a8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.132+0000 7f4aeffff700 1 -- 172.21.15.133:0/2749190441 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae8009c40 con 0x7f4af8107af0 2023-11-27T15:27:10.136 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.132+0000 7f4aeffff700 1 --2- 172.21.15.133:0/2749190441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4af8107af0 0x7f4af81a10b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000d950 tx=0x7f4ae000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.132+0000 7f4afd59c700 1 --2- 172.21.15.133:0/2749190441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4af81084a0 0x7f4af81a8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:10.137 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.132+0000 7f4af67fc700 1 -- 172.21.15.133:0/2749190441 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0009940 con 0x7f4af8107af0 2023-11-27T15:27:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.132+0000 7f4af67fc700 1 -- 172.21.15.133:0/2749190441 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ae0010460 con 0x7f4af8107af0 2023-11-27T15:27:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.132+0000 7f4af67fc700 1 -- 172.21.15.133:0/2749190441 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae000f620 con 0x7f4af8107af0 2023-11-27T15:27:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.133+0000 7f4afefff700 1 -- 172.21.15.133:0/2749190441 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af81143d0 con 0x7f4af8107af0 2023-11-27T15:27:10.138 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.133+0000 7f4afefff700 1 -- 172.21.15.133:0/2749190441 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af8114240 con 0x7f4af8107af0 2023-11-27T15:27:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.141+0000 7f4b12904b80 1 Processor -- start 2023-11-27T15:27:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.142+0000 7f4b12904b80 1 -- start start 2023-11-27T15:27:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.143+0000 7f4b12904b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621605db6b0 0x562160788080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.143+0000 7f4b12904b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621607885c0 0x562160788de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.143+0000 7f4b12904b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562160789320 0x562160789790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.143+0000 7f4b12904b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562160685000 con 0x562160789320 2023-11-27T15:27:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.143+0000 7f4b12904b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621606d7840 con 0x5621605db6b0 2023-11-27T15:27:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.143+0000 7f4b12904b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621606d0cd0 con 0x5621607885c0 2023-11-27T15:27:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.143+0000 7f4af67fc700 1 -- 172.21.15.133:0/2749190441 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f4ae0009aa0 con 0x7f4af8107af0 2023-11-27T15:27:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.144+0000 7f4afd59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621605db6b0 0x562160788080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.144+0000 7f4afd59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621605db6b0 0x562160788080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:49194/0 (socket says 172.21.15.133:49194) 2023-11-27T15:27:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.144+0000 7f4afd59c700 1 -- 172.21.15.133:0/400890129 learned_addr learned my addr 172.21.15.133:0/400890129 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.145+0000 7f4afcd9b700 1 --2- 172.21.15.133:0/400890129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621607885c0 0x562160788de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.146+0000 7f4afd59c700 1 -- 172.21.15.133:0/400890129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621607885c0 msgr2=0x562160788de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:10.150 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.146+0000 7f4afd59c700 1 --2- 172.21.15.133:0/400890129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621607885c0 0x562160788de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.146+0000 7f4aeffff700 1 --2- 172.21.15.133:0/400890129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562160789320 0x562160789790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.146+0000 7f4afd59c700 1 -- 172.21.15.133:0/400890129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562160789320 msgr2=0x562160789790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.146+0000 7f4afd59c700 1 --2- 172.21.15.133:0/400890129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562160789320 0x562160789790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.147+0000 7f4afd59c700 1 -- 172.21.15.133:0/400890129 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621605525d0 con 0x5621605db6b0 2023-11-27T15:27:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.147+0000 7f4aeffff700 1 --2- 172.21.15.133:0/400890129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562160789320 0x562160789790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:10.151 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.147+0000 7f4afd59c700 1 --2- 172.21.15.133:0/400890129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621605db6b0 0x562160788080 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4af0009ca0 tx=0x7f4af000e2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41581b65f97e2576 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.147+0000 7f4af67fc700 1 --2- 172.21.15.133:0/2749190441 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4ae40a1250 0x7f4ae40a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.148+0000 7f4af67fc700 1 -- 172.21.15.133:0/2749190441 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f4ae00cce50 con 0x7f4af8107af0 2023-11-27T15:27:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.148+0000 7f4afcd9b700 1 --2- 172.21.15.133:0/2749190441 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4ae40a1250 0x7f4ae40a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.150+0000 7f4afcd9b700 1 --2- 172.21.15.133:0/2749190441 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4ae40a1250 0x7f4ae40a3700 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8007a40 tx=0x7f4ae80096c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.150+0000 7f4aeeffd700 1 -- 172.21.15.133:0/400890129 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af000ec20 con 0x5621605db6b0 2023-11-27T15:27:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.150+0000 7f4aeeffd700 1 -- 172.21.15.133:0/400890129 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4af000ed80 con 0x5621605db6b0 2023-11-27T15:27:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.150+0000 7f4aeeffd700 1 -- 172.21.15.133:0/400890129 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af0010740 con 0x5621605db6b0 2023-11-27T15:27:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.151+0000 7f4b12904b80 1 -- 172.21.15.133:0/400890129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621605db6b0 msgr2=0x562160788080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.151+0000 7f4b12904b80 1 --2- 172.21.15.133:0/400890129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621605db6b0 0x562160788080 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4af0009ca0 tx=0x7f4af000e2f0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.153+0000 7f4b12904b80 1 -- 172.21.15.133:0/400890129 shutdown_connections 2023-11-27T15:27:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.153+0000 7f4b12904b80 1 --2- 172.21.15.133:0/400890129 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562160789320 0x562160789790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.153+0000 7f4b12904b80 1 --2- 172.21.15.133:0/400890129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621605db6b0 0x562160788080 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.157 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.153+0000 7f4b12904b80 1 --2- 172.21.15.133:0/400890129 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621607885c0 0x562160788de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.153+0000 7f4b12904b80 1 -- 172.21.15.133:0/400890129 >> 172.21.15.133:0/400890129 conn(0x562160774f50 msgr2=0x56216077a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.154+0000 7f4b12904b80 1 -- 172.21.15.133:0/400890129 shutdown_connections 2023-11-27T15:27:10.158 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.154+0000 7f4b12904b80 1 -- 172.21.15.133:0/400890129 wait complete. 2023-11-27T15:27:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.156+0000 7f4b12904b80 1 -- start start 2023-11-27T15:27:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.156+0000 7f4b12904b80 1 Processor -- start 2023-11-27T15:27:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.158+0000 7f4b12904b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621605db6b0 0x562160791520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.158+0000 7f4b12904b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621607885c0 0x562160791a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.158+0000 7f4b12904b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562160789320 0x562160798aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.158+0000 7f4b12904b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562160672880 con 0x5621607885c0 2023-11-27T15:27:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.158+0000 7f4b12904b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x562160672520 con 0x562160789320 2023-11-27T15:27:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.158+0000 7f4b12904b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621605548c0 con 0x5621605db6b0 2023-11-27T15:27:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.158+0000 7f4b12904b80 10 client.0 _ll_register_callbacks cb 0x562160690720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.158+0000 7f4afcd9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621605db6b0 0x562160791520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.158+0000 7f4afcd9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621605db6b0 0x562160791520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:37848/0 (socket says 172.21.15.133:37848) 2023-11-27T15:27:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.158+0000 7f4afcd9b700 1 -- 172.21.15.133:0/1715015192 learned_addr learned my addr 172.21.15.133:0/1715015192 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.159+0000 7f4aeffff700 1 --2- 172.21.15.133:0/1715015192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621607885c0 0x562160791a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.159+0000 7f4afd59c700 1 --2- 172.21.15.133:0/1715015192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562160789320 0x562160798aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.159+0000 7f4afcd9b700 1 -- 172.21.15.133:0/1715015192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562160789320 msgr2=0x562160798aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.159+0000 7f4afcd9b700 1 --2- 172.21.15.133:0/1715015192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562160789320 0x562160798aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.159+0000 7f4afcd9b700 1 -- 172.21.15.133:0/1715015192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621607885c0 msgr2=0x562160791a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.159+0000 7f4afcd9b700 1 --2- 172.21.15.133:0/1715015192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621607885c0 0x562160791a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.159+0000 7f4afcd9b700 1 -- 172.21.15.133:0/1715015192 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af0009900 con 0x5621605db6b0 2023-11-27T15:27:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.160+0000 7f4afcd9b700 1 --2- 172.21.15.133:0/1715015192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621605db6b0 0x562160791520 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8007920 tx=0x7f4ae800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d275f38431b85886 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:10.164 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.160+0000 7f4ad7fff700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:27:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.161+0000 7f4ad7fff700 1 -- 172.21.15.133:0/1715015192 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae800e040 con 0x5621605db6b0 2023-11-27T15:27:10.165 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.161+0000 7f4ad7fff700 1 -- 172.21.15.133:0/1715015192 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f4ae802a680 con 0x5621605db6b0 2023-11-27T15:27:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.161+0000 7f4b12904b80 1 -- 172.21.15.133:0/1715015192 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562160772710 con 0x5621605db6b0 2023-11-27T15:27:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.162+0000 7f4ad7fff700 1 -- 172.21.15.133:0/1715015192 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ae8009ac0 con 0x5621605db6b0 2023-11-27T15:27:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.162+0000 7f4b12904b80 10 client.26242 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:10.166 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.162+0000 7f4b12904b80 1 -- 172.21.15.133:0/1715015192 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562160697130 con 0x5621605db6b0 2023-11-27T15:27:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.162+0000 7f4ad7fff700 1 -- 172.21.15.133:0/1715015192 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562160772710 con 0x5621605db6b0 2023-11-27T15:27:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.163+0000 7f4ad7fff700 1 -- 172.21.15.133:0/1715015192 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f4ae80057c0 con 0x5621605db6b0 2023-11-27T15:27:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.163+0000 7f4b12904b80 10 client.26242 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.163+0000 7f4b12904b80 10 client.26242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.163+0000 7f4b12904b80 20 client.26242 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.163+0000 7f4b12904b80 1 --2- 172.21.15.133:0/1715015192 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56216079d340 0x56216079f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:10.167 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.163+0000 7f4b12904b80 4 client.26242 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:10.168 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.163+0000 7f4b12904b80 1 -- 172.21.15.133:0/1715015192 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562160693040 con 0x56216079d340 2023-11-27T15:27:10.171 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.167+0000 7f4aeffff700 1 --2- 172.21.15.133:0/1715015192 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56216079d340 0x56216079f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.170+0000 7f4aeffff700 1 --2- 172.21.15.133:0/1715015192 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56216079d340 0x56216079f750 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d395eabee091a00 server_cookie=d8527213b838a42a in_seq=0 out_seq=0 2023-11-27T15:27:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.170+0000 7f4ad7fff700 10 client.26242 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:10.176 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.173+0000 7f4ad7fff700 1 -- 172.21.15.133:0/1715015192 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562160693040 con 0x56216079d340 2023-11-27T15:27:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.173+0000 7f4ad7fff700 10 client.26242 handle_command_reply: tid=0 2023-11-27T15:27:10.177 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.173+0000 7f4b12904b80 1 client.26242 shutdown 2023-11-27T15:27:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.175+0000 7f4b12904b80 1 -- 172.21.15.133:0/1715015192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621605db6b0 msgr2=0x562160791520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.175+0000 7f4b12904b80 1 --2- 172.21.15.133:0/1715015192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621605db6b0 0x562160791520 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8007920 tx=0x7f4ae800b330 comp rx=0 tx=0).stop 2023-11-27T15:27:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.176+0000 7f4b12904b80 1 -- 172.21.15.133:0/1715015192 shutdown_connections 2023-11-27T15:27:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.176+0000 7f4b12904b80 1 --2- 172.21.15.133:0/1715015192 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56216079d340 0x56216079f750 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.176+0000 7f4ad7fff700 0 client.26242 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.176+0000 7f4b12904b80 1 --2- 172.21.15.133:0/1715015192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562160789320 0x562160798aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.176+0000 7f4b12904b80 1 --2- 172.21.15.133:0/1715015192 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621605db6b0 0x562160791520 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.176+0000 7f4b12904b80 1 --2- 172.21.15.133:0/1715015192 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621607885c0 0x562160791a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.176+0000 7f4b12904b80 1 -- 172.21.15.133:0/1715015192 >> 172.21.15.133:0/1715015192 conn(0x562160774f50 msgr2=0x5621607797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.176+0000 7f4b12904b80 1 -- 172.21.15.133:0/1715015192 shutdown_connections 2023-11-27T15:27:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.176+0000 7f4b12904b80 1 -- 172.21.15.133:0/1715015192 wait complete. 2023-11-27T15:27:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.177+0000 7f4b12904b80 20 client.26242 trim_cache size 0 max 16384 2023-11-27T15:27:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.211+0000 7f4b12904b80 1 Processor -- start 2023-11-27T15:27:10.215 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.212+0000 7f4b12904b80 1 -- start start 2023-11-27T15:27:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.212+0000 7f4b12904b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621605db6b0 0x56216078c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.212+0000 7f4b12904b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621607885c0 0x56216078c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:10.216 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.212+0000 7f4b12904b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562160789320 0x562160798a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.213+0000 7f4afcd9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621605db6b0 0x56216078c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.213+0000 7f4aeffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621607885c0 0x56216078c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.213+0000 7f4afcd9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621605db6b0 0x56216078c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52452/0 (socket says 172.21.15.133:52452) 2023-11-27T15:27:10.217 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.213+0000 7f4aeffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621607885c0 0x56216078c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:49212/0 (socket says 172.21.15.133:49212) 2023-11-27T15:27:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.213+0000 7f4afd59c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562160789320 0x562160798a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.213+0000 7f4b12904b80 1 -- 172.21.15.133:0/4123359460 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562160792710 con 0x5621605db6b0 2023-11-27T15:27:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.213+0000 7f4afcd9b700 1 -- 172.21.15.133:0/4123359460 learned_addr learned my addr 172.21.15.133:0/4123359460 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.213+0000 7f4b12904b80 1 -- 172.21.15.133:0/4123359460 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621607909d0 con 0x5621607885c0 2023-11-27T15:27:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.213+0000 7f4b12904b80 1 -- 172.21.15.133:0/4123359460 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56216079a680 con 0x562160789320 2023-11-27T15:27:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.214+0000 7f4aeffff700 1 -- 172.21.15.133:0/4123359460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562160789320 msgr2=0x562160798a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.214+0000 7f4aeffff700 1 --2- 172.21.15.133:0/4123359460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562160789320 0x562160798a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.214+0000 7f4aeffff700 1 -- 172.21.15.133:0/4123359460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621605db6b0 msgr2=0x56216078c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.214+0000 7f4aeffff700 1 --2- 172.21.15.133:0/4123359460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621605db6b0 0x56216078c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.214+0000 7f4aeffff700 1 -- 172.21.15.133:0/4123359460 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562160696e80 con 0x5621607885c0 2023-11-27T15:27:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.214+0000 7f4afcd9b700 1 --2- 172.21.15.133:0/4123359460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621605db6b0 0x56216078c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.214+0000 7f4afd59c700 1 --2- 172.21.15.133:0/4123359460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562160789320 0x562160798a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:27:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.215+0000 7f4aeffff700 1 --2- 172.21.15.133:0/4123359460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621607885c0 0x56216078c750 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0000c00 tx=0x7f4ae0010cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d573f7648b215bfd server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.215+0000 7f4aef7fe700 1 -- 172.21.15.133:0/4123359460 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0097ff0 con 0x5621607885c0 2023-11-27T15:27:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.215+0000 7f4aef7fe700 1 -- 172.21.15.133:0/4123359460 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ae0098250 con 0x5621607885c0 2023-11-27T15:27:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.215+0000 7f4aef7fe700 1 -- 172.21.15.133:0/4123359460 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0097ff0 con 0x5621607885c0 2023-11-27T15:27:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.216+0000 7f4b12904b80 1 -- 172.21.15.133:0/4123359460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621607885c0 msgr2=0x56216078c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.216+0000 7f4b12904b80 1 --2- 172.21.15.133:0/4123359460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621607885c0 0x56216078c750 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0000c00 tx=0x7f4ae0010cb0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.217+0000 7f4b12904b80 1 -- 172.21.15.133:0/4123359460 shutdown_connections 2023-11-27T15:27:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.217+0000 7f4b12904b80 1 --2- 172.21.15.133:0/4123359460 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562160789320 0x562160798a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.217+0000 7f4b12904b80 1 --2- 172.21.15.133:0/4123359460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5621605db6b0 0x56216078c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.217+0000 7f4b12904b80 1 --2- 172.21.15.133:0/4123359460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621607885c0 0x56216078c750 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.217+0000 7f4b12904b80 1 -- 172.21.15.133:0/4123359460 >> 172.21.15.133:0/4123359460 conn(0x562160774f50 msgr2=0x562160790210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.217+0000 7f4b12904b80 1 -- 172.21.15.133:0/4123359460 shutdown_connections 2023-11-27T15:27:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.217+0000 7f4b12904b80 1 -- 172.21.15.133:0/4123359460 wait complete. 2023-11-27T15:27:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.218+0000 7f4b12904b80 1 -- start start 2023-11-27T15:27:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.218+0000 7f4b12904b80 1 Processor -- start 2023-11-27T15:27:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.219+0000 7f4b12904b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621605db6b0 0x562160794c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.219+0000 7f4b12904b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621607885c0 0x5621607951d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.219+0000 7f4b12904b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562160789320 0x562160797dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:10.223 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.219+0000 7f4b12904b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562160798310 con 0x562160789320 2023-11-27T15:27:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.219+0000 7f4b12904b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x562160798450 con 0x5621605db6b0 2023-11-27T15:27:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.219+0000 7f4b12904b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x562160798610 con 0x5621607885c0 2023-11-27T15:27:10.224 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.219+0000 7f4b12904b80 10 client.0 _ll_register_callbacks cb 0x7f4ae8007730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.220+0000 7f4aeffff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621607885c0 0x5621607951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.220+0000 7f4aeffff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621607885c0 0x5621607951d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:37870/0 (socket says 172.21.15.133:37870) 2023-11-27T15:27:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.220+0000 7f4aeffff700 1 -- 172.21.15.133:0/1357560163 learned_addr learned my addr 172.21.15.133:0/1357560163 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.221+0000 7f4afcd9b700 1 --2- 172.21.15.133:0/1357560163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621605db6b0 0x562160794c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.221+0000 7f4aeffff700 1 -- 172.21.15.133:0/1357560163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621605db6b0 msgr2=0x562160794c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.221+0000 7f4aeffff700 1 --2- 172.21.15.133:0/1357560163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621605db6b0 0x562160794c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.221+0000 7f4aeffff700 1 -- 172.21.15.133:0/1357560163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562160789320 msgr2=0x562160797dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:10.225 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.221+0000 7f4afd59c700 1 --2- 172.21.15.133:0/1357560163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562160789320 0x562160797dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.222+0000 7f4aeffff700 1 --2- 172.21.15.133:0/1357560163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562160789320 0x562160797dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.222+0000 7f4aeffff700 1 -- 172.21.15.133:0/1357560163 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621605525d0 con 0x5621607885c0 2023-11-27T15:27:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.222+0000 7f4afd59c700 1 --2- 172.21.15.133:0/1357560163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562160789320 0x562160797dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.222+0000 7f4aeffff700 1 --2- 172.21.15.133:0/1357560163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621607885c0 0x5621607951d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00963b0 tx=0x7f4ae00ca540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30098c89c5adf277 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.222+0000 7f4ad7fff700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:27:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.223+0000 7f4ad7fff700 1 -- 172.21.15.133:0/1357560163 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0097ff0 con 0x5621607885c0 2023-11-27T15:27:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.223+0000 7f4ad7fff700 1 -- 172.21.15.133:0/1357560163 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f4ae009a320 con 0x5621607885c0 2023-11-27T15:27:10.227 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.223+0000 7f4b12904b80 1 -- 172.21.15.133:0/1357560163 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4af000ed80 con 0x5621607885c0 2023-11-27T15:27:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.223+0000 7f4ad7fff700 1 -- 172.21.15.133:0/1357560163 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ae0009e90 con 0x5621607885c0 2023-11-27T15:27:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.223+0000 7f4b12904b80 10 client.26248 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.223+0000 7f4b12904b80 1 -- 172.21.15.133:0/1357560163 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f4af8114240 con 0x5621607885c0 2023-11-27T15:27:10.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.224+0000 7f4ad7fff700 1 -- 172.21.15.133:0/1357560163 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4af000ed80 con 0x5621607885c0 2023-11-27T15:27:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.224+0000 7f4ad7fff700 1 -- 172.21.15.133:0/1357560163 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f4ae00a1e70 con 0x5621607885c0 2023-11-27T15:27:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.225+0000 7f4b12904b80 10 client.26248 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.225+0000 7f4b12904b80 10 client.26248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.225+0000 7f4b12904b80 20 client.26248 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.225+0000 7f4b12904b80 1 --2- 172.21.15.133:0/1357560163 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56216079d340 0x5621607f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.225+0000 7f4b12904b80 4 client.26248 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:27:10.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.225+0000 7f4b12904b80 1 -- 172.21.15.133:0/1357560163 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56216069f600 con 0x56216079d340 2023-11-27T15:27:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.225+0000 7f4afcd9b700 1 --2- 172.21.15.133:0/1357560163 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56216079d340 0x5621607f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.226+0000 7f4afcd9b700 1 --2- 172.21.15.133:0/1357560163 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56216079d340 0x5621607f3e30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcd3ff89910e4ac6 server_cookie=95dec2bcd00d66e8 in_seq=0 out_seq=0 2023-11-27T15:27:10.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:10.226+0000 7f4ad7fff700 10 client.26248 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:11.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:10 smithi133 ceph-mon[97860]: pgmap v476: 129 pgs: 129 active+clean; 278 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.4 MiB/s wr, 576 op/s 2023-11-27T15:27:11.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:10 smithi177 ceph-mon[131504]: pgmap v476: 129 pgs: 129 active+clean; 278 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.4 MiB/s wr, 576 op/s 2023-11-27T15:27:11.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:10 smithi174 ceph-mon[131110]: pgmap v476: 129 pgs: 129 active+clean; 278 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.4 MiB/s wr, 576 op/s 2023-11-27T15:27:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.815+0000 7f4ad7fff700 1 -- 172.21.15.133:0/1357560163 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56216069f600 con 0x56216079d340 2023-11-27T15:27:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.815+0000 7f4ad7fff700 10 client.26248 handle_command_reply: tid=0 2023-11-27T15:27:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.816+0000 7f4b12904b80 1 client.26248 shutdown 2023-11-27T15:27:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.816+0000 7f4b12904b80 1 -- 172.21.15.133:0/1357560163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621607885c0 msgr2=0x5621607951d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.816+0000 7f4b12904b80 1 --2- 172.21.15.133:0/1357560163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621607885c0 0x5621607951d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00963b0 tx=0x7f4ae00ca540 comp rx=0 tx=0).stop 2023-11-27T15:27:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.816+0000 7f4b12904b80 1 -- 172.21.15.133:0/1357560163 shutdown_connections 2023-11-27T15:27:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.816+0000 7f4b12904b80 1 --2- 172.21.15.133:0/1357560163 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56216079d340 0x5621607f3e30 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.816+0000 7f4b12904b80 1 --2- 172.21.15.133:0/1357560163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562160789320 0x562160797dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.816+0000 7f4ad7fff700 0 client.26248 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.816+0000 7f4b12904b80 1 --2- 172.21.15.133:0/1357560163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621605db6b0 0x562160794c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.816+0000 7f4b12904b80 1 --2- 172.21.15.133:0/1357560163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5621607885c0 0x5621607951d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.816+0000 7f4b12904b80 1 -- 172.21.15.133:0/1357560163 >> 172.21.15.133:0/1357560163 conn(0x562160774f50 msgr2=0x56216078f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.816+0000 7f4b12904b80 1 -- 172.21.15.133:0/1357560163 shutdown_connections 2023-11-27T15:27:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.816+0000 7f4b12904b80 1 -- 172.21.15.133:0/1357560163 wait complete. 2023-11-27T15:27:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.816+0000 7f4b12904b80 20 client.26248 trim_cache size 0 max 16384 2023-11-27T15:27:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:27:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:27:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:27:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:27:11.822 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.817+0000 7f4afefff700 1 -- 172.21.15.133:0/2749190441 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4ae40a1250 msgr2=0x7f4ae40a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:11.822 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.817+0000 7f4afefff700 1 --2- 172.21.15.133:0/2749190441 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4ae40a1250 0x7f4ae40a3700 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8007a40 tx=0x7f4ae80096c0 comp rx=0 tx=0).stop 2023-11-27T15:27:11.822 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.817+0000 7f4afefff700 1 -- 172.21.15.133:0/2749190441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4af8107af0 msgr2=0x7f4af81a10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:11.822 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.817+0000 7f4afefff700 1 --2- 172.21.15.133:0/2749190441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4af8107af0 0x7f4af81a10b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4ae000d950 tx=0x7f4ae000dc60 comp rx=0 tx=0).stop 2023-11-27T15:27:11.822 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.817+0000 7f4afefff700 1 -- 172.21.15.133:0/2749190441 shutdown_connections 2023-11-27T15:27:11.822 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.817+0000 7f4afefff700 1 --2- 172.21.15.133:0/2749190441 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f4ae40a1250 0x7f4ae40a3700 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:11.822 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.817+0000 7f4afefff700 1 --2- 172.21.15.133:0/2749190441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4af81084a0 0x7f4af81a8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.817+0000 7f4afefff700 1 --2- 172.21.15.133:0/2749190441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4af80721d0 0x7f4af81a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.817+0000 7f4afefff700 1 --2- 172.21.15.133:0/2749190441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4af8107af0 0x7f4af81a10b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.817+0000 7f4afefff700 1 -- 172.21.15.133:0/2749190441 >> 172.21.15.133:0/2749190441 conn(0x7f4af806db10 msgr2=0x7f4af810f520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.817+0000 7f4afefff700 1 -- 172.21.15.133:0/2749190441 shutdown_connections 2023-11-27T15:27:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:11.817+0000 7f4afefff700 1 -- 172.21.15.133:0/2749190441 wait complete. 2023-11-27T15:27:13.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:12 smithi133 ceph-mon[97860]: pgmap v477: 129 pgs: 129 active+clean; 274 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 559 op/s 2023-11-27T15:27:13.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:12 smithi177 ceph-mon[131504]: pgmap v477: 129 pgs: 129 active+clean; 274 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 559 op/s 2023-11-27T15:27:13.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:12 smithi174 ceph-mon[131110]: pgmap v477: 129 pgs: 129 active+clean; 274 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 559 op/s 2023-11-27T15:27:15.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:14 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:27:15.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:14 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:15.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:14 smithi133 ceph-mon[97860]: pgmap v478: 129 pgs: 129 active+clean; 277 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 548 op/s 2023-11-27T15:27:15.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:14 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:27:15.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:14 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:15.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:14 smithi177 ceph-mon[131504]: pgmap v478: 129 pgs: 129 active+clean; 277 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 548 op/s 2023-11-27T15:27:15.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:14 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:27:15.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:14 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:15.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:14 smithi174 ceph-mon[131110]: pgmap v478: 129 pgs: 129 active+clean; 277 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 548 op/s 2023-11-27T15:27:16.825 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:27:17.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.219+0000 7f6d05b9d700 1 -- 172.21.15.133:0/273492154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d0006d490 msgr2=0x7f6d0006d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:17.223 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.219+0000 7f6d05b9d700 1 --2- 172.21.15.133:0/273492154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d0006d490 0x7f6d0006d900 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6cec0098f0 tx=0x7f6cec009c00 comp rx=0 tx=0).stop 2023-11-27T15:27:17.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.219+0000 7f6d05b9d700 1 -- 172.21.15.133:0/273492154 shutdown_connections 2023-11-27T15:27:17.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.219+0000 7f6d05b9d700 1 --2- 172.21.15.133:0/273492154 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d0006de40 0x7f6d00117340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.219+0000 7f6d05b9d700 1 --2- 172.21.15.133:0/273492154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d0006c1b0 0x7f6d0006c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.219+0000 7f6d05b9d700 1 --2- 172.21.15.133:0/273492154 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d0006d490 0x7f6d0006d900 secure :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6cec0098f0 tx=0x7f6cec009c00 comp rx=0 tx=0).stop 2023-11-27T15:27:17.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.219+0000 7f6d05b9d700 1 -- 172.21.15.133:0/273492154 >> 172.21.15.133:0/273492154 conn(0x7f6d0006aa70 msgr2=0x7f6d0006ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:17.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.220+0000 7f6d05b9d700 1 -- 172.21.15.133:0/273492154 shutdown_connections 2023-11-27T15:27:17.224 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.220+0000 7f6d05b9d700 1 -- 172.21.15.133:0/273492154 wait complete. 2023-11-27T15:27:17.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.223+0000 7f6d05b9d700 1 Processor -- start 2023-11-27T15:27:17.226 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.223+0000 7f6d05b9d700 1 -- start start 2023-11-27T15:27:17.227 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.223+0000 7f6d05b9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d0006c1b0 0x7f6d0019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:17.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.224+0000 7f6d05b9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d0006de40 0x7f6d0019d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:17.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.225+0000 7f6d05b9d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d001a1c10 0x7f6d001a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:17.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.225+0000 7f6d05b9d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d001a45c0 con 0x7f6d0006c1b0 2023-11-27T15:27:17.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.225+0000 7f6d05b9d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d001a4700 con 0x7f6d0006de40 2023-11-27T15:27:17.228 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.225+0000 7f6d05b9d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d001a48c0 con 0x7f6d001a1c10 2023-11-27T15:27:17.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.225+0000 7f6d04b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d0006c1b0 0x7f6d0019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:17.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.225+0000 7f6d0539c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d001a1c10 0x7f6d001a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:17.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.225+0000 7f6d0539c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d001a1c10 0x7f6d001a3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:50536/0 (socket says 172.21.15.133:50536) 2023-11-27T15:27:17.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.225+0000 7f6d0539c700 1 -- 172.21.15.133:0/3319104374 learned_addr learned my addr 172.21.15.133:0/3319104374 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:17.229 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.225+0000 7f6cfffff700 1 --2- 172.21.15.133:0/3319104374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d0006de40 0x7f6d0019d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:17.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.226+0000 7f6d04b9b700 1 -- 172.21.15.133:0/3319104374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d001a1c10 msgr2=0x7f6d001a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:17.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.226+0000 7f6d04b9b700 1 --2- 172.21.15.133:0/3319104374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d001a1c10 0x7f6d001a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.226+0000 7f6d04b9b700 1 -- 172.21.15.133:0/3319104374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d0006de40 msgr2=0x7f6d0019d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:17.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.226+0000 7f6d04b9b700 1 --2- 172.21.15.133:0/3319104374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d0006de40 0x7f6d0019d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.230 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.226+0000 7f6d04b9b700 1 -- 172.21.15.133:0/3319104374 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec009520 con 0x7f6d0006c1b0 2023-11-27T15:27:17.232 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.228+0000 7f6d04b9b700 1 --2- 172.21.15.133:0/3319104374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d0006c1b0 0x7f6d0019ccd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4005fd0 tx=0x7f6cf401fd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:17.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.229+0000 7f6cfdffb700 1 -- 172.21.15.133:0/3319104374 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf401feb0 con 0x7f6d0006c1b0 2023-11-27T15:27:17.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.229+0000 7f6d05b9d700 1 -- 172.21.15.133:0/3319104374 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d0010d5c0 con 0x7f6d0006c1b0 2023-11-27T15:27:17.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.229+0000 7f6d05b9d700 1 -- 172.21.15.133:0/3319104374 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d0010d430 con 0x7f6d0006c1b0 2023-11-27T15:27:17.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.230+0000 7f6cfdffb700 1 -- 172.21.15.133:0/3319104374 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cf4004020 con 0x7f6d0006c1b0 2023-11-27T15:27:17.233 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.230+0000 7f6cfdffb700 1 -- 172.21.15.133:0/3319104374 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf401feb0 con 0x7f6d0006c1b0 2023-11-27T15:27:17.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.238+0000 7f6cfdffb700 1 -- 172.21.15.133:0/3319104374 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f6cf401d9e0 con 0x7f6d0006c1b0 2023-11-27T15:27:17.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.243+0000 7f6cfdffb700 1 --2- 172.21.15.133:0/3319104374 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6cf00a12b0 0x7f6cf00a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:17.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.243+0000 7f6cfdffb700 1 -- 172.21.15.133:0/3319104374 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6cf4025070 con 0x7f6d0006c1b0 2023-11-27T15:27:17.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.244+0000 7f6d1a765b80 1 Processor -- start 2023-11-27T15:27:17.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.244+0000 7f6d1a765b80 1 -- start start 2023-11-27T15:27:17.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.244+0000 7f6d1a765b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e4e90 0x55f8934e52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:17.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.244+0000 7f6d1a765b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8934e57e0 0x55f8934e5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:17.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.244+0000 7f6d1a765b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e62b0 0x55f8934e6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:17.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.244+0000 7f6d1a765b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8933ea060 con 0x55f8934e57e0 2023-11-27T15:27:17.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.244+0000 7f6d1a765b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f89343c8a0 con 0x55f8934e4e90 2023-11-27T15:27:17.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.244+0000 7f6d1a765b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f893435d30 con 0x55f8934e62b0 2023-11-27T15:27:17.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.245+0000 7f6d04b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e4e90 0x55f8934e52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:17.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.245+0000 7f6d04b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e4e90 0x55f8934e52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:34020/0 (socket says 172.21.15.133:34020) 2023-11-27T15:27:17.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.245+0000 7f6d04b9b700 1 -- 172.21.15.133:0/62328142 learned_addr learned my addr 172.21.15.133:0/62328142 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:17.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.245+0000 7f6d0539c700 1 --2- 172.21.15.133:0/62328142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e62b0 0x55f8934e6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:17.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.245+0000 7f6d04b9b700 1 -- 172.21.15.133:0/62328142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e62b0 msgr2=0x55f8934e6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:17.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.246+0000 7f6d04b9b700 1 --2- 172.21.15.133:0/62328142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e62b0 0x55f8934e6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.246+0000 7f6d04b9b700 1 -- 172.21.15.133:0/62328142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8934e57e0 msgr2=0x55f8934e5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:17.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.246+0000 7f6d04b9b700 1 --2- 172.21.15.133:0/62328142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8934e57e0 0x55f8934e5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.246+0000 7f6cfffff700 1 --2- 172.21.15.133:0/3319104374 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6cf00a12b0 0x7f6cf00a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:17.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.246+0000 7f6d04b9b700 1 -- 172.21.15.133:0/62328142 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8932b7630 con 0x55f8934e4e90 2023-11-27T15:27:17.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.247+0000 7f6d04b9b700 1 --2- 172.21.15.133:0/62328142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e4e90 0x55f8934e52a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40a7360 tx=0x7f6cf40aa560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=901a1b1e036e9007 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.248+0000 7f6ceaffd700 1 -- 172.21.15.133:0/62328142 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf40b2d40 con 0x55f8934e4e90 2023-11-27T15:27:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.248+0000 7f6ceaffd700 1 -- 172.21.15.133:0/62328142 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cf4095040 con 0x55f8934e4e90 2023-11-27T15:27:17.252 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.248+0000 7f6ceaffd700 1 -- 172.21.15.133:0/62328142 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf40b2d40 con 0x55f8934e4e90 2023-11-27T15:27:17.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.250+0000 7f6cfffff700 1 --2- 172.21.15.133:0/3319104374 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6cf00a12b0 0x7f6cf00a3760 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6cec000c00 tx=0x7f6cec019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:17.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.250+0000 7f6d1a765b80 1 -- 172.21.15.133:0/62328142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e4e90 msgr2=0x55f8934e52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:17.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.250+0000 7f6d1a765b80 1 --2- 172.21.15.133:0/62328142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e4e90 0x55f8934e52a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40a7360 tx=0x7f6cf40aa560 comp rx=0 tx=0).stop 2023-11-27T15:27:17.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.250+0000 7f6d1a765b80 1 -- 172.21.15.133:0/62328142 shutdown_connections 2023-11-27T15:27:17.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.250+0000 7f6d1a765b80 1 --2- 172.21.15.133:0/62328142 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e62b0 0x55f8934e6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.250+0000 7f6d1a765b80 1 --2- 172.21.15.133:0/62328142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e4e90 0x55f8934e52a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.250+0000 7f6d1a765b80 1 --2- 172.21.15.133:0/62328142 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8934e57e0 0x55f8934e5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.250+0000 7f6d1a765b80 1 -- 172.21.15.133:0/62328142 >> 172.21.15.133:0/62328142 conn(0x55f8934da5b0 msgr2=0x55f8934df850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:17.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.250+0000 7f6d1a765b80 1 -- 172.21.15.133:0/62328142 shutdown_connections 2023-11-27T15:27:17.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.251+0000 7f6d1a765b80 1 -- 172.21.15.133:0/62328142 wait complete. 2023-11-27T15:27:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.252+0000 7f6d1a765b80 1 -- start start 2023-11-27T15:27:17.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.252+0000 7f6d1a765b80 1 Processor -- start 2023-11-27T15:27:17.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.254+0000 7f6d1a765b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e4e90 0x55f8934ee630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:17.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.254+0000 7f6d1a765b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8934e57e0 0x55f8934eeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.255+0000 7f6d04b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e4e90 0x55f8934ee630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.255+0000 7f6d04b9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e4e90 0x55f8934ee630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:34034/0 (socket says 172.21.15.133:34034) 2023-11-27T15:27:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.255+0000 7f6d1a765b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e62b0 0x55f8934f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.255+0000 7f6d1a765b80 1 -- 172.21.15.133:0/2280890629 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8933d78e0 con 0x55f8934e57e0 2023-11-27T15:27:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.255+0000 7f6d1a765b80 1 -- 172.21.15.133:0/2280890629 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8933d7580 con 0x55f8934e4e90 2023-11-27T15:27:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.255+0000 7f6d1a765b80 1 -- 172.21.15.133:0/2280890629 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8932b9920 con 0x55f8934e62b0 2023-11-27T15:27:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.255+0000 7f6d04b9b700 1 -- 172.21.15.133:0/2280890629 learned_addr learned my addr 172.21.15.133:0/2280890629 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.255+0000 7f6d1a765b80 10 client.0 _ll_register_callbacks cb 0x55f8933f5780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:17.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.255+0000 7f6d0539c700 1 --2- 172.21.15.133:0/2280890629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e62b0 0x55f8934f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.256+0000 7f6cfffff700 1 --2- 172.21.15.133:0/2280890629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8934e57e0 0x55f8934eeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.256+0000 7f6d04b9b700 1 -- 172.21.15.133:0/2280890629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e62b0 msgr2=0x55f8934f5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.256+0000 7f6d04b9b700 1 --2- 172.21.15.133:0/2280890629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e62b0 0x55f8934f5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.256+0000 7f6d04b9b700 1 -- 172.21.15.133:0/2280890629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8934e57e0 msgr2=0x55f8934eeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.256+0000 7f6d04b9b700 1 --2- 172.21.15.133:0/2280890629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8934e57e0 0x55f8934eeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.256+0000 7f6d04b9b700 1 -- 172.21.15.133:0/2280890629 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d0010d430 con 0x55f8934e4e90 2023-11-27T15:27:17.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.257+0000 7f6d04b9b700 1 --2- 172.21.15.133:0/2280890629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e4e90 0x55f8934ee630 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40b2810 tx=0x7f6cf40aad80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cca5fce618682ee4 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:17.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.257+0000 7f6cdffff700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:27:17.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.258+0000 7f6cdffff700 1 -- 172.21.15.133:0/2280890629 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf40b2b00 con 0x55f8934e4e90 2023-11-27T15:27:17.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.258+0000 7f6cdffff700 1 -- 172.21.15.133:0/2280890629 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6cf40b2d40 con 0x55f8934e4e90 2023-11-27T15:27:17.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.259+0000 7f6d1a765b80 1 -- 172.21.15.133:0/2280890629 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8934d7d70 con 0x55f8934e4e90 2023-11-27T15:27:17.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.262+0000 7f6cdffff700 1 -- 172.21.15.133:0/2280890629 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cf40a9ff0 con 0x55f8934e4e90 2023-11-27T15:27:17.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.262+0000 7f6d1a765b80 10 client.26282 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:17.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.262+0000 7f6d1a765b80 1 -- 172.21.15.133:0/2280890629 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f8933fc190 con 0x55f8934e4e90 2023-11-27T15:27:17.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.262+0000 7f6cdffff700 1 -- 172.21.15.133:0/2280890629 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cf408d030 con 0x55f8934e4e90 2023-11-27T15:27:17.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.262+0000 7f6cdffff700 1 -- 172.21.15.133:0/2280890629 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f6cf40a9850 con 0x55f8934e4e90 2023-11-27T15:27:17.266 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.263+0000 7f6d1a765b80 10 client.26282 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:17.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.263+0000 7f6d1a765b80 10 client.26282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:17.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.263+0000 7f6d1a765b80 20 client.26282 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:17.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.263+0000 7f6d1a765b80 1 --2- 172.21.15.133:0/2280890629 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55f8934f9f20 0x55f8934fc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:17.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.263+0000 7f6d1a765b80 4 client.26282 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:17.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.263+0000 7f6d1a765b80 1 -- 172.21.15.133:0/2280890629 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8933f80a0 con 0x55f8934f9f20 2023-11-27T15:27:17.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.263+0000 7f6cfffff700 1 --2- 172.21.15.133:0/2280890629 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55f8934f9f20 0x55f8934fc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:17.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.267+0000 7f6cdffff700 10 client.26282 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:17.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.267+0000 7f6cfffff700 1 --2- 172.21.15.133:0/2280890629 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55f8934f9f20 0x55f8934fc330 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aed9a195689af642 server_cookie=91078ed64a496dd0 in_seq=0 out_seq=0 2023-11-27T15:27:17.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.268+0000 7f6cdffff700 1 -- 172.21.15.133:0/2280890629 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f8933f80a0 con 0x55f8934f9f20 2023-11-27T15:27:17.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.268+0000 7f6cdffff700 10 client.26282 handle_command_reply: tid=0 2023-11-27T15:27:17.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.269+0000 7f6d1a765b80 1 client.26282 shutdown 2023-11-27T15:27:17.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.269+0000 7f6d1a765b80 1 -- 172.21.15.133:0/2280890629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e4e90 msgr2=0x55f8934ee630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:17.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.269+0000 7f6d1a765b80 1 --2- 172.21.15.133:0/2280890629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e4e90 0x55f8934ee630 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40b2810 tx=0x7f6cf40aad80 comp rx=0 tx=0).stop 2023-11-27T15:27:17.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.273+0000 7f6d1a765b80 1 -- 172.21.15.133:0/2280890629 shutdown_connections 2023-11-27T15:27:17.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.273+0000 7f6d1a765b80 1 --2- 172.21.15.133:0/2280890629 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55f8934f9f20 0x55f8934fc330 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.273+0000 7f6d1a765b80 1 --2- 172.21.15.133:0/2280890629 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e62b0 0x55f8934f5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.273+0000 7f6d1a765b80 1 --2- 172.21.15.133:0/2280890629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e4e90 0x55f8934ee630 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.273+0000 7f6d1a765b80 1 --2- 172.21.15.133:0/2280890629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8934e57e0 0x55f8934eeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.273+0000 7f6cdffff700 0 client.26282 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:17.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.273+0000 7f6d1a765b80 1 -- 172.21.15.133:0/2280890629 >> 172.21.15.133:0/2280890629 conn(0x55f8934da5b0 msgr2=0x55f8934df030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:17.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:16 smithi133 ceph-mon[97860]: pgmap v479: 129 pgs: 129 active+clean; 290 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.4 MiB/s wr, 643 op/s 2023-11-27T15:27:17.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:16 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:27:17.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.275+0000 7f6d1a765b80 1 -- 172.21.15.133:0/2280890629 shutdown_connections 2023-11-27T15:27:17.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.275+0000 7f6d1a765b80 1 -- 172.21.15.133:0/2280890629 wait complete. 2023-11-27T15:27:17.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.277+0000 7f6d1a765b80 20 client.26282 trim_cache size 0 max 16384 2023-11-27T15:27:17.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.317+0000 7f6d1a765b80 1 Processor -- start 2023-11-27T15:27:17.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.318+0000 7f6d1a765b80 1 -- start start 2023-11-27T15:27:17.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.319+0000 7f6d1a765b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e4e90 0x55f893500c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:17.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.319+0000 7f6d1a765b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8934e57e0 0x55f8934f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:17.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.319+0000 7f6d1a765b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e62b0 0x55f8934f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:17.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.319+0000 7f6d1a765b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8934e0370 con 0x55f8934e57e0 2023-11-27T15:27:17.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.319+0000 7f6d1a765b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8934e12d0 con 0x55f8934e62b0 2023-11-27T15:27:17.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.319+0000 7f6d1a765b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8934ec140 con 0x55f8934e4e90 2023-11-27T15:27:17.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.319+0000 7f6d04b9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e4e90 0x55f893500c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:17.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.319+0000 7f6d04b9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e4e90 0x55f893500c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:50570/0 (socket says 172.21.15.133:50570) 2023-11-27T15:27:17.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.319+0000 7f6d04b9b700 1 -- 172.21.15.133:0/2230596609 learned_addr learned my addr 172.21.15.133:0/2230596609 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:17.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.320+0000 7f6cfffff700 1 --2- 172.21.15.133:0/2230596609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8934e57e0 0x55f8934f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:17.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.321+0000 7f6d0539c700 1 --2- 172.21.15.133:0/2230596609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e62b0 0x55f8934f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:17.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.321+0000 7f6d04b9b700 1 -- 172.21.15.133:0/2230596609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e62b0 msgr2=0x55f8934f64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:17.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.321+0000 7f6d04b9b700 1 --2- 172.21.15.133:0/2230596609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e62b0 0x55f8934f64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.321+0000 7f6d04b9b700 1 -- 172.21.15.133:0/2230596609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8934e57e0 msgr2=0x55f8934f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:17.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.321+0000 7f6d04b9b700 1 --2- 172.21.15.133:0/2230596609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8934e57e0 0x55f8934f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.321+0000 7f6d04b9b700 1 -- 172.21.15.133:0/2230596609 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8933fbee0 con 0x55f8934e4e90 2023-11-27T15:27:17.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.322+0000 7f6d04b9b700 1 --2- 172.21.15.133:0/2230596609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e4e90 0x55f893500c50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4000be0 tx=0x7f6cf40a6d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68b54b5c09f9903e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:17.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.324+0000 7f6ceb7fe700 1 -- 172.21.15.133:0/2230596609 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4004190 con 0x55f8934e4e90 2023-11-27T15:27:17.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.324+0000 7f6ceb7fe700 1 -- 172.21.15.133:0/2230596609 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cf408d070 con 0x55f8934e4e90 2023-11-27T15:27:17.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.324+0000 7f6ceb7fe700 1 -- 172.21.15.133:0/2230596609 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4004350 con 0x55f8934e4e90 2023-11-27T15:27:17.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.325+0000 7f6d1a765b80 1 -- 172.21.15.133:0/2230596609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e4e90 msgr2=0x55f893500c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:17.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.325+0000 7f6d1a765b80 1 --2- 172.21.15.133:0/2230596609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e4e90 0x55f893500c50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4000be0 tx=0x7f6cf40a6d60 comp rx=0 tx=0).stop 2023-11-27T15:27:17.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.325+0000 7f6d1a765b80 1 -- 172.21.15.133:0/2230596609 shutdown_connections 2023-11-27T15:27:17.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.325+0000 7f6d1a765b80 1 --2- 172.21.15.133:0/2230596609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e62b0 0x55f8934f64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.325+0000 7f6d1a765b80 1 --2- 172.21.15.133:0/2230596609 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e4e90 0x55f893500c50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.325+0000 7f6d1a765b80 1 --2- 172.21.15.133:0/2230596609 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8934e57e0 0x55f8934f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.329 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.325+0000 7f6d1a765b80 1 -- 172.21.15.133:0/2230596609 >> 172.21.15.133:0/2230596609 conn(0x55f8934da5b0 msgr2=0x55f8934dba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:17.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.328+0000 7f6d1a765b80 1 -- 172.21.15.133:0/2230596609 shutdown_connections 2023-11-27T15:27:17.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.328+0000 7f6d1a765b80 1 -- 172.21.15.133:0/2230596609 wait complete. 2023-11-27T15:27:17.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.328+0000 7f6d1a765b80 1 -- start start 2023-11-27T15:27:17.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.329+0000 7f6d1a765b80 1 Processor -- start 2023-11-27T15:27:17.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.330+0000 7f6d1a765b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e4e90 0x55f8934f1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:17.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.330+0000 7f6d1a765b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8934e57e0 0x55f8934f1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:17.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.330+0000 7f6d1a765b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e62b0 0x55f8934f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:17.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.330+0000 7f6d1a765b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8934f5400 con 0x55f8934e57e0 2023-11-27T15:27:17.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.330+0000 7f6d1a765b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8934f5570 con 0x55f8934e62b0 2023-11-27T15:27:17.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.330+0000 7f6d1a765b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8934f5730 con 0x55f8934e4e90 2023-11-27T15:27:17.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.330+0000 7f6d1a765b80 10 client.0 _ll_register_callbacks cb 0x7f6cf40aa2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:17.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.330+0000 7f6d04b9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e4e90 0x55f8934f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:17.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.330+0000 7f6d04b9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e4e90 0x55f8934f1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:50576/0 (socket says 172.21.15.133:50576) 2023-11-27T15:27:17.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.330+0000 7f6d04b9b700 1 -- 172.21.15.133:0/3349174456 learned_addr learned my addr 172.21.15.133:0/3349174456 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:17.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.331+0000 7f6d04b9b700 1 -- 172.21.15.133:0/3349174456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e62b0 msgr2=0x55f8934f4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:17.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.331+0000 7f6d04b9b700 1 --2- 172.21.15.133:0/3349174456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e62b0 0x55f8934f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.331+0000 7f6d04b9b700 1 -- 172.21.15.133:0/3349174456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8934e57e0 msgr2=0x55f8934f1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:17.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.331+0000 7f6d04b9b700 1 --2- 172.21.15.133:0/3349174456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8934e57e0 0x55f8934f1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.331+0000 7f6d04b9b700 1 -- 172.21.15.133:0/3349174456 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8933fc190 con 0x55f8934e4e90 2023-11-27T15:27:17.335 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.332+0000 7f6d04b9b700 1 --2- 172.21.15.133:0/3349174456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e4e90 0x55f8934f1960 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4020010 tx=0x7f6cf40aa530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20f59ec7b740a146 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:17.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.332+0000 7f6cdffff700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:27:17.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.333+0000 7f6cdffff700 1 -- 172.21.15.133:0/3349174456 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4004190 con 0x55f8934e4e90 2023-11-27T15:27:17.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.333+0000 7f6cdffff700 1 -- 172.21.15.133:0/3349174456 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f6cf40b2d40 con 0x55f8934e4e90 2023-11-27T15:27:17.336 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.333+0000 7f6d1a765b80 1 -- 172.21.15.133:0/3349174456 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6cf4095040 con 0x55f8934e4e90 2023-11-27T15:27:17.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:16 smithi177 ceph-mon[131504]: pgmap v479: 129 pgs: 129 active+clean; 290 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.4 MiB/s wr, 643 op/s 2023-11-27T15:27:17.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:16 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:27:17.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.333+0000 7f6cdffff700 1 -- 172.21.15.133:0/3349174456 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cf40aa0a0 con 0x55f8934e4e90 2023-11-27T15:27:17.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.333+0000 7f6d1a765b80 10 client.26266 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:17.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.333+0000 7f6d1a765b80 1 -- 172.21.15.133:0/3349174456 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f89324cc60 con 0x55f8934e4e90 2023-11-27T15:27:17.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.334+0000 7f6cdffff700 1 -- 172.21.15.133:0/3349174456 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cf40d1020 con 0x55f8934e4e90 2023-11-27T15:27:17.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.334+0000 7f6cdffff700 1 -- 172.21.15.133:0/3349174456 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f6cf40951b0 con 0x55f8934e4e90 2023-11-27T15:27:17.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.335+0000 7f6d1a765b80 10 client.26266 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:17.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.335+0000 7f6d1a765b80 10 client.26266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:17.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.335+0000 7f6d1a765b80 20 client.26266 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:17.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.335+0000 7f6d1a765b80 1 --2- 172.21.15.133:0/3349174456 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55f89354e800 0x55f893550be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:17.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.335+0000 7f6d1a765b80 4 client.26266 mds_command: new command op to 24560 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:27:17.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.335+0000 7f6d1a765b80 1 -- 172.21.15.133:0/3349174456 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f893404660 con 0x55f89354e800 2023-11-27T15:27:17.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.336+0000 7f6cfffff700 1 --2- 172.21.15.133:0/3349174456 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55f89354e800 0x55f893550be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:17.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.337+0000 7f6cfffff700 1 --2- 172.21.15.133:0/3349174456 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55f89354e800 0x55f893550be0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e784ef4df2e5d62a server_cookie=8ece568fddf313a7 in_seq=0 out_seq=0 2023-11-27T15:27:17.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.337+0000 7f6cdffff700 10 client.26266 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:17.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.338+0000 7f6cdffff700 1 -- 172.21.15.133:0/3349174456 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55f893404660 con 0x55f89354e800 2023-11-27T15:27:17.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.339+0000 7f6cdffff700 10 client.26266 handle_command_reply: tid=0 2023-11-27T15:27:17.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.339+0000 7f6d1a765b80 1 client.26266 shutdown 2023-11-27T15:27:17.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.339+0000 7f6d1a765b80 1 -- 172.21.15.133:0/3349174456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e4e90 msgr2=0x55f8934f1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:17.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.339+0000 7f6d1a765b80 1 --2- 172.21.15.133:0/3349174456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e4e90 0x55f8934f1960 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4020010 tx=0x7f6cf40aa530 comp rx=0 tx=0).stop 2023-11-27T15:27:17.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.339+0000 7f6d1a765b80 1 -- 172.21.15.133:0/3349174456 shutdown_connections 2023-11-27T15:27:17.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.339+0000 7f6d1a765b80 1 --2- 172.21.15.133:0/3349174456 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55f89354e800 0x55f893550be0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.339+0000 7f6d1a765b80 1 --2- 172.21.15.133:0/3349174456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8934e62b0 0x55f8934f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.339+0000 7f6cdffff700 0 client.26266 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:17.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.339+0000 7f6d1a765b80 1 --2- 172.21.15.133:0/3349174456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f8934e4e90 0x55f8934f1960 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.339+0000 7f6d1a765b80 1 --2- 172.21.15.133:0/3349174456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8934e57e0 0x55f8934f1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.340+0000 7f6d1a765b80 1 -- 172.21.15.133:0/3349174456 >> 172.21.15.133:0/3349174456 conn(0x55f8934da5b0 msgr2=0x55f893500c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:17.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.340+0000 7f6d1a765b80 1 -- 172.21.15.133:0/3349174456 shutdown_connections 2023-11-27T15:27:17.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.340+0000 7f6d1a765b80 1 -- 172.21.15.133:0/3349174456 wait complete. 2023-11-27T15:27:17.345 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:27:17.345 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (1 inodes in the stack)", 2023-11-27T15:27:17.345 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-11-27T15:27:17.346 INFO:teuthology.orchestra.run.smithi133.stdout: "b6638f49-a408-46a9-83f0-3995bb2029f5": { 2023-11-27T15:27:17.346 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-11-27T15:27:17.346 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "b6638f49-a408-46a9-83f0-3995bb2029f5", 2023-11-27T15:27:17.346 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-11-27T15:27:17.346 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:27:17.346 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:27:17.346 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:27:17.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.340+0000 7f6d1a765b80 20 client.26266 trim_cache size 0 max 16384 2023-11-27T15:27:17.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.342+0000 7f6d05b9d700 1 -- 172.21.15.133:0/3319104374 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6cf00a12b0 msgr2=0x7f6cf00a3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:17.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.342+0000 7f6d05b9d700 1 --2- 172.21.15.133:0/3319104374 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6cf00a12b0 0x7f6cf00a3760 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6cec000c00 tx=0x7f6cec019040 comp rx=0 tx=0).stop 2023-11-27T15:27:17.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.342+0000 7f6d05b9d700 1 -- 172.21.15.133:0/3319104374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d0006c1b0 msgr2=0x7f6d0019ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:17.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.342+0000 7f6d05b9d700 1 --2- 172.21.15.133:0/3319104374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d0006c1b0 0x7f6d0019ccd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4005fd0 tx=0x7f6cf401fd20 comp rx=0 tx=0).stop 2023-11-27T15:27:17.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.342+0000 7f6d05b9d700 1 -- 172.21.15.133:0/3319104374 shutdown_connections 2023-11-27T15:27:17.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.342+0000 7f6d05b9d700 1 --2- 172.21.15.133:0/3319104374 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f6cf00a12b0 0x7f6cf00a3760 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.342+0000 7f6d05b9d700 1 --2- 172.21.15.133:0/3319104374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d001a1c10 0x7f6d001a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.342+0000 7f6d05b9d700 1 --2- 172.21.15.133:0/3319104374 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d0006c1b0 0x7f6d0019ccd0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.342+0000 7f6d05b9d700 1 --2- 172.21.15.133:0/3319104374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d0006de40 0x7f6d0019d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:17.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.342+0000 7f6d05b9d700 1 -- 172.21.15.133:0/3319104374 >> 172.21.15.133:0/3319104374 conn(0x7f6d0006aa70 msgr2=0x7f6d00108380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:17.347 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.343+0000 7f6d05b9d700 1 -- 172.21.15.133:0/3319104374 shutdown_connections 2023-11-27T15:27:17.348 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:17.343+0000 7f6d05b9d700 1 -- 172.21.15.133:0/3319104374 wait complete. 2023-11-27T15:27:17.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:16 smithi174 ceph-mon[131110]: pgmap v479: 129 pgs: 129 active+clean; 290 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.4 MiB/s wr, 643 op/s 2023-11-27T15:27:17.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:16 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:27:17.406 INFO:tasks.cephfs.filesystem:scrub status for tag:b6638f49-a408-46a9-83f0-3995bb2029f5 - {'path': '/', 'tag': 'b6638f49-a408-46a9-83f0-3995bb2029f5', 'options': 'recursive,force'} 2023-11-27T15:27:18.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:17 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:27:18.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:17 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:27:18.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:17 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:27:18.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:17 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:27:18.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:17 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:27:18.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:17 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:27:18.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:17 smithi133 ceph-mon[97860]: Service mds.all has no related daemon already placed. 2023-11-27T15:27:18.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:17 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:27:18.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:17 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:27:18.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:17 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:27:18.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:17 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:27:18.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:17 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:27:18.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:17 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:27:18.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:17 smithi177 ceph-mon[131504]: Service mds.all has no related daemon already placed. 2023-11-27T15:27:18.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:17 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:27:18.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:17 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:27:18.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:17 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:27:18.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:17 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:27:18.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:17 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:27:18.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:17 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:27:18.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:17 smithi174 ceph-mon[131110]: Service mds.all has no related daemon already placed. 2023-11-27T15:27:19.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:18 smithi133 ceph-mon[97860]: pgmap v480: 129 pgs: 129 active+clean; 290 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 605 op/s 2023-11-27T15:27:19.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:18 smithi177 ceph-mon[131504]: pgmap v480: 129 pgs: 129 active+clean; 290 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 605 op/s 2023-11-27T15:27:19.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:18 smithi174 ceph-mon[131110]: pgmap v480: 129 pgs: 129 active+clean; 290 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 605 op/s 2023-11-27T15:27:21.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:20 smithi133 ceph-mon[97860]: pgmap v481: 129 pgs: 129 active+clean; 296 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 640 op/s 2023-11-27T15:27:21.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:20 smithi177 ceph-mon[131504]: pgmap v481: 129 pgs: 129 active+clean; 296 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 640 op/s 2023-11-27T15:27:21.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:20 smithi174 ceph-mon[131110]: pgmap v481: 129 pgs: 129 active+clean; 296 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 640 op/s 2023-11-27T15:27:23.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:22 smithi133 ceph-mon[97860]: pgmap v482: 129 pgs: 129 active+clean; 309 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 641 op/s 2023-11-27T15:27:23.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:22 smithi177 ceph-mon[131504]: pgmap v482: 129 pgs: 129 active+clean; 309 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 641 op/s 2023-11-27T15:27:23.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:22 smithi174 ceph-mon[131110]: pgmap v482: 129 pgs: 129 active+clean; 309 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 641 op/s 2023-11-27T15:27:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.991+0000 7f5fb63ff700 1 -- 172.21.15.133:0/2698113595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fb00754b0 msgr2=0x7f5fb00758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.991+0000 7f5fb63ff700 1 --2- 172.21.15.133:0/2698113595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fb00754b0 0x7f5fb00758c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5fa801c0d0 tx=0x7f5fa801c3e0 comp rx=0 tx=0).stop 2023-11-27T15:27:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.991+0000 7f5fb63ff700 1 -- 172.21.15.133:0/2698113595 shutdown_connections 2023-11-27T15:27:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.991+0000 7f5fb63ff700 1 --2- 172.21.15.133:0/2698113595 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fb00770a0 0x7f5fb0077930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.991+0000 7f5fb63ff700 1 --2- 172.21.15.133:0/2698113595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fb00754b0 0x7f5fb00758c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.991+0000 7f5fb63ff700 1 --2- 172.21.15.133:0/2698113595 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb00766f0 0x7f5fb0076b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.991+0000 7f5fb63ff700 1 -- 172.21.15.133:0/2698113595 >> 172.21.15.133:0/2698113595 conn(0x7f5fb00fdac0 msgr2=0x7f5fb00ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.991+0000 7f5fb63ff700 1 -- 172.21.15.133:0/2698113595 shutdown_connections 2023-11-27T15:27:24.995 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.991+0000 7f5fb63ff700 1 -- 172.21.15.133:0/2698113595 wait complete. 2023-11-27T15:27:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.993+0000 7f5fb63ff700 1 Processor -- start 2023-11-27T15:27:24.996 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.993+0000 7f5fb63ff700 1 -- start start 2023-11-27T15:27:24.997 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.993+0000 7f5fb63ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb00754b0 0x7f5fb0193db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.993+0000 7f5fb63ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fb00766f0 0x7f5fb01942f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.993+0000 7f5fb63ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fb00770a0 0x7f5fb019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.993+0000 7f5fb63ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb019b8f0 con 0x7f5fb00754b0 2023-11-27T15:27:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.993+0000 7f5fb63ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb019ba60 con 0x7f5fb00766f0 2023-11-27T15:27:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.993+0000 7f5fb63ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb019bc20 con 0x7f5fb00770a0 2023-11-27T15:27:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.994+0000 7f5faf7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fb00766f0 0x7f5fb01942f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.994+0000 7f5faf7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fb00766f0 0x7f5fb01942f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:45730/0 (socket says 172.21.15.133:45730) 2023-11-27T15:27:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.994+0000 7f5faf7fe700 1 -- 172.21.15.133:0/2103539989 learned_addr learned my addr 172.21.15.133:0/2103539989 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.994+0000 7f5faffff700 1 --2- 172.21.15.133:0/2103539989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb00754b0 0x7f5fb0193db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.994+0000 7f5fb499c700 1 --2- 172.21.15.133:0/2103539989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fb00770a0 0x7f5fb019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.994+0000 7f5faf7fe700 1 -- 172.21.15.133:0/2103539989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fb00770a0 msgr2=0x7f5fb019b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:24.998 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.994+0000 7f5faf7fe700 1 --2- 172.21.15.133:0/2103539989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fb00770a0 0x7f5fb019b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.994+0000 7f5faf7fe700 1 -- 172.21.15.133:0/2103539989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb00754b0 msgr2=0x7f5fb0193db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.994+0000 7f5faf7fe700 1 --2- 172.21.15.133:0/2103539989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb00754b0 0x7f5fb0193db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.994+0000 7f5faf7fe700 1 -- 172.21.15.133:0/2103539989 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa801ad10 con 0x7f5fb00766f0 2023-11-27T15:27:24.999 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.994+0000 7f5fb499c700 1 --2- 172.21.15.133:0/2103539989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fb00770a0 0x7f5fb019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.994+0000 7f5faf7fe700 1 --2- 172.21.15.133:0/2103539989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fb00766f0 0x7f5fb01942f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400e9a0 tx=0x7f5fa400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.995+0000 7f5fad7fa700 1 -- 172.21.15.133:0/2103539989 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4009790 con 0x7f5fb00766f0 2023-11-27T15:27:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.995+0000 7f5fad7fa700 1 -- 172.21.15.133:0/2103539989 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fa400cbd0 con 0x7f5fb00766f0 2023-11-27T15:27:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.995+0000 7f5fb63ff700 1 -- 172.21.15.133:0/2103539989 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb0103a90 con 0x7f5fb00766f0 2023-11-27T15:27:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.995+0000 7f5fad7fa700 1 -- 172.21.15.133:0/2103539989 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4018680 con 0x7f5fb00766f0 2023-11-27T15:27:25.000 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:24.995+0000 7f5fb63ff700 1 -- 172.21.15.133:0/2103539989 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb0103900 con 0x7f5fb00766f0 2023-11-27T15:27:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.000+0000 7f5fad7fa700 1 -- 172.21.15.133:0/2103539989 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5fa400cd40 con 0x7f5fb00766f0 2023-11-27T15:27:25.003 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.000+0000 7f5fc9da2b80 1 Processor -- start 2023-11-27T15:27:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.000+0000 7f5fc9da2b80 1 -- start start 2023-11-27T15:27:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.000+0000 7f5fc9da2b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557612326b0 0x5557613d6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.001+0000 7f5fc9da2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d6db0 0x5557613d75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:25.004 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.001+0000 7f5fc9da2b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557613d7b10 0x5557613d7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:25.005 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.001+0000 7f5fc9da2b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557612dc000 con 0x5557613d7b10 2023-11-27T15:27:25.005 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.001+0000 7f5fc9da2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55576132e840 con 0x5557613d6db0 2023-11-27T15:27:25.005 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.001+0000 7f5fc9da2b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x555761327cd0 con 0x5557612326b0 2023-11-27T15:27:25.005 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.001+0000 7f5fb499c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557612326b0 0x5557613d6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:25.005 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.001+0000 7f5fb499c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557612326b0 0x5557613d6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:36886/0 (socket says 172.21.15.133:36886) 2023-11-27T15:27:25.005 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.001+0000 7f5fb499c700 1 -- 172.21.15.133:0/4050596827 learned_addr learned my addr 172.21.15.133:0/4050596827 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:25.005 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.001+0000 7f5faffff700 1 --2- 172.21.15.133:0/4050596827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d6db0 0x5557613d75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.001+0000 7f5faf7fe700 1 --2- 172.21.15.133:0/4050596827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557613d7b10 0x5557613d7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.001+0000 7f5fb499c700 1 -- 172.21.15.133:0/4050596827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d6db0 msgr2=0x5557613d75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.001+0000 7f5fb499c700 1 --2- 172.21.15.133:0/4050596827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d6db0 0x5557613d75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.001+0000 7f5fb499c700 1 -- 172.21.15.133:0/4050596827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557613d7b10 msgr2=0x5557613d7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.001+0000 7f5fb499c700 1 --2- 172.21.15.133:0/4050596827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557613d7b10 0x5557613d7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.001+0000 7f5fb499c700 1 -- 172.21.15.133:0/4050596827 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557611a95d0 con 0x5557612326b0 2023-11-27T15:27:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.002+0000 7f5fb499c700 1 --2- 172.21.15.133:0/4050596827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557612326b0 0x5557613d6870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0009d50 tx=0x7f5fa00116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6c3e15d70f5bc5f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.002+0000 7f5fad7fa700 1 --2- 172.21.15.133:0/2103539989 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5f980a1150 0x7f5f980a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.002+0000 7f5f967fc700 1 -- 172.21.15.133:0/4050596827 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0009ea0 con 0x5557612326b0 2023-11-27T15:27:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.002+0000 7f5fad7fa700 1 -- 172.21.15.133:0/2103539989 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f5fa4014070 con 0x7f5fb00766f0 2023-11-27T15:27:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.002+0000 7f5faffff700 1 --2- 172.21.15.133:0/2103539989 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5f980a1150 0x7f5f980a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.002+0000 7f5f967fc700 1 -- 172.21.15.133:0/4050596827 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fa0011e20 con 0x5557612326b0 2023-11-27T15:27:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.002+0000 7f5f967fc700 1 -- 172.21.15.133:0/4050596827 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0009820 con 0x5557612326b0 2023-11-27T15:27:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.003+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/4050596827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557612326b0 msgr2=0x5557613d6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.003+0000 7f5faffff700 1 --2- 172.21.15.133:0/2103539989 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5f980a1150 0x7f5f980a3600 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8018b50 tx=0x7f5fa801c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.003+0000 7f5fc9da2b80 1 --2- 172.21.15.133:0/4050596827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557612326b0 0x5557613d6870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0009d50 tx=0x7f5fa00116c0 comp rx=0 tx=0).stop 2023-11-27T15:27:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.003+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/4050596827 shutdown_connections 2023-11-27T15:27:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.003+0000 7f5fc9da2b80 1 --2- 172.21.15.133:0/4050596827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557613d7b10 0x5557613d7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.003+0000 7f5fc9da2b80 1 --2- 172.21.15.133:0/4050596827 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557612326b0 0x5557613d6870 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.003+0000 7f5fc9da2b80 1 --2- 172.21.15.133:0/4050596827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d6db0 0x5557613d75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.003+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/4050596827 >> 172.21.15.133:0/4050596827 conn(0x5557613cbf50 msgr2=0x5557613d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.003+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/4050596827 shutdown_connections 2023-11-27T15:27:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.004+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/4050596827 wait complete. 2023-11-27T15:27:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.004+0000 7f5fc9da2b80 1 -- start start 2023-11-27T15:27:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.004+0000 7f5fc9da2b80 1 Processor -- start 2023-11-27T15:27:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.005+0000 7f5fc9da2b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557612326b0 0x5557613dfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.005+0000 7f5fc9da2b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557613d6db0 0x5557613e0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.005+0000 7f5fc9da2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d7b10 0x5557613e71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.005+0000 7f5fc9da2b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557612c9880 con 0x5557612326b0 2023-11-27T15:27:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.005+0000 7f5fc9da2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557612c9520 con 0x5557613d7b10 2023-11-27T15:27:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.005+0000 7f5fc9da2b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557611ab8c0 con 0x5557613d6db0 2023-11-27T15:27:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.005+0000 7f5fc9da2b80 10 client.0 _ll_register_callbacks cb 0x5557612e7720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.005+0000 7f5fb499c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d7b10 0x5557613e71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.005+0000 7f5fb499c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d7b10 0x5557613e71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:45758/0 (socket says 172.21.15.133:45758) 2023-11-27T15:27:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.005+0000 7f5fb499c700 1 -- 172.21.15.133:0/1861895256 learned_addr learned my addr 172.21.15.133:0/1861895256 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:25.010 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.005+0000 7f5faf7fe700 1 --2- 172.21.15.133:0/1861895256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557613d6db0 0x5557613e0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.006+0000 7f5fb499c700 1 -- 172.21.15.133:0/1861895256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557613d6db0 msgr2=0x5557613e0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.006+0000 7f5fb499c700 1 --2- 172.21.15.133:0/1861895256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557613d6db0 0x5557613e0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.006+0000 7f5fb499c700 1 -- 172.21.15.133:0/1861895256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557612326b0 msgr2=0x5557613dfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.006+0000 7f5fb499c700 1 --2- 172.21.15.133:0/1861895256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557612326b0 0x5557613dfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.006+0000 7f5fb499c700 1 -- 172.21.15.133:0/1861895256 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa0009980 con 0x5557613d7b10 2023-11-27T15:27:25.012 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.006+0000 7f5fb499c700 1 --2- 172.21.15.133:0/1861895256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d7b10 0x5557613e71d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0012010 tx=0x7f5fa0004f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5cf983da8ec318c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.006+0000 7f5f8f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:27:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.006+0000 7f5f8f7fe700 1 -- 172.21.15.133:0/1861895256 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0012660 con 0x5557613d7b10 2023-11-27T15:27:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.006+0000 7f5f8f7fe700 1 -- 172.21.15.133:0/1861895256 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f5fa00127c0 con 0x5557613d7b10 2023-11-27T15:27:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.006+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/1861895256 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557613c9710 con 0x5557613d7b10 2023-11-27T15:27:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.007+0000 7f5f8f7fe700 1 -- 172.21.15.133:0/1861895256 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fa0012980 con 0x5557613d7b10 2023-11-27T15:27:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.007+0000 7f5fc9da2b80 10 client.26297 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.007+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/1861895256 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5557612ee130 con 0x5557613d7b10 2023-11-27T15:27:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.007+0000 7f5f8f7fe700 1 -- 172.21.15.133:0/1861895256 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5557613c9710 con 0x5557613d7b10 2023-11-27T15:27:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.007+0000 7f5f8f7fe700 1 -- 172.21.15.133:0/1861895256 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5fa000f870 con 0x5557613d7b10 2023-11-27T15:27:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.008+0000 7f5fc9da2b80 10 client.26297 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.008+0000 7f5fc9da2b80 10 client.26297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.008+0000 7f5fc9da2b80 20 client.26297 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:25.013 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.008+0000 7f5fc9da2b80 1 --2- 172.21.15.133:0/1861895256 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5557613eb940 0x5557613edd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.008+0000 7f5fc9da2b80 4 client.26297 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.008+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/1861895256 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557612ea040 con 0x5557613eb940 2023-11-27T15:27:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.008+0000 7f5faffff700 1 --2- 172.21.15.133:0/1861895256 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5557613eb940 0x5557613edd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.008+0000 7f5faffff700 1 --2- 172.21.15.133:0/1861895256 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5557613eb940 0x5557613edd50 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=869aa3668a5304be server_cookie=8e6aa9c9bfb98e5f in_seq=0 out_seq=0 2023-11-27T15:27:25.014 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.008+0000 7f5f8f7fe700 10 client.26297 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.011+0000 7f5f8f7fe700 1 -- 172.21.15.133:0/1861895256 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5557612ea040 con 0x5557613eb940 2023-11-27T15:27:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.011+0000 7f5f8f7fe700 10 client.26297 handle_command_reply: tid=0 2023-11-27T15:27:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.011+0000 7f5fc9da2b80 1 client.26297 shutdown 2023-11-27T15:27:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.011+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/1861895256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d7b10 msgr2=0x5557613e71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:25.015 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.011+0000 7f5fc9da2b80 1 --2- 172.21.15.133:0/1861895256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d7b10 0x5557613e71d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0012010 tx=0x7f5fa0004f20 comp rx=0 tx=0).stop 2023-11-27T15:27:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.011+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/1861895256 shutdown_connections 2023-11-27T15:27:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.011+0000 7f5fc9da2b80 1 --2- 172.21.15.133:0/1861895256 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5557613eb940 0x5557613edd50 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.011+0000 7f5fc9da2b80 1 --2- 172.21.15.133:0/1861895256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d7b10 0x5557613e71d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.011+0000 7f5fc9da2b80 1 --2- 172.21.15.133:0/1861895256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557612326b0 0x5557613dfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.011+0000 7f5fc9da2b80 1 --2- 172.21.15.133:0/1861895256 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557613d6db0 0x5557613e0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.011+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/1861895256 >> 172.21.15.133:0/1861895256 conn(0x5557613cbf50 msgr2=0x5557613d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.011+0000 7f5f8f7fe700 0 client.26297 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.012+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/1861895256 shutdown_connections 2023-11-27T15:27:25.016 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.012+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/1861895256 wait complete. 2023-11-27T15:27:25.017 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.012+0000 7f5fc9da2b80 20 client.26297 trim_cache size 0 max 16384 2023-11-27T15:27:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.030+0000 7f5fc9da2b80 1 Processor -- start 2023-11-27T15:27:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.030+0000 7f5fc9da2b80 1 -- start start 2023-11-27T15:27:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.031+0000 7f5fc9da2b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557612326b0 0x5557613df300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.031+0000 7f5fc9da2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d6db0 0x5557613e7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:25.034 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.031+0000 7f5fc9da2b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557613d7b10 0x5557613dea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.031+0000 7f5fc9da2b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557613def40 con 0x5557613d7b10 2023-11-27T15:27:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.031+0000 7f5fc9da2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557613d7420 con 0x5557613d6db0 2023-11-27T15:27:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.031+0000 7f5fc9da2b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557613db4d0 con 0x5557612326b0 2023-11-27T15:27:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.031+0000 7f5faf7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d6db0 0x5557613e7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.031+0000 7f5faffff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557612326b0 0x5557613df300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.031+0000 7f5faf7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d6db0 0x5557613e7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:45766/0 (socket says 172.21.15.133:45766) 2023-11-27T15:27:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.031+0000 7f5faf7fe700 1 -- 172.21.15.133:0/3354951896 learned_addr learned my addr 172.21.15.133:0/3354951896 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.031+0000 7f5fb499c700 1 --2- 172.21.15.133:0/3354951896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557613d7b10 0x5557613dea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.032+0000 7f5faffff700 1 -- 172.21.15.133:0/3354951896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d6db0 msgr2=0x5557613e7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.032+0000 7f5faffff700 1 --2- 172.21.15.133:0/3354951896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d6db0 0x5557613e7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.032+0000 7f5faffff700 1 -- 172.21.15.133:0/3354951896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557613d7b10 msgr2=0x5557613dea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:25.036 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.032+0000 7f5faffff700 1 --2- 172.21.15.133:0/3354951896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557613d7b10 0x5557613dea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.032+0000 7f5faffff700 1 -- 172.21.15.133:0/3354951896 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557612ede80 con 0x5557612326b0 2023-11-27T15:27:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.032+0000 7f5fb499c700 1 --2- 172.21.15.133:0/3354951896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557613d7b10 0x5557613dea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.032+0000 7f5faffff700 1 --2- 172.21.15.133:0/3354951896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557612326b0 0x5557613df300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5fa801c7b0 tx=0x7f5fa802e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1ecf7db9157ab3c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.032+0000 7f5f96ffd700 1 -- 172.21.15.133:0/3354951896 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa803f070 con 0x5557612326b0 2023-11-27T15:27:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.032+0000 7f5f96ffd700 1 -- 172.21.15.133:0/3354951896 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fa8018950 con 0x5557612326b0 2023-11-27T15:27:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.032+0000 7f5f96ffd700 1 -- 172.21.15.133:0/3354951896 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa801f040 con 0x5557612326b0 2023-11-27T15:27:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.033+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/3354951896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557612326b0 msgr2=0x5557613df300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.033+0000 7f5fc9da2b80 1 --2- 172.21.15.133:0/3354951896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557612326b0 0x5557613df300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5fa801c7b0 tx=0x7f5fa802e000 comp rx=0 tx=0).stop 2023-11-27T15:27:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.033+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/3354951896 shutdown_connections 2023-11-27T15:27:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.033+0000 7f5fc9da2b80 1 --2- 172.21.15.133:0/3354951896 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557613d7b10 0x5557613dea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.033+0000 7f5fc9da2b80 1 --2- 172.21.15.133:0/3354951896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557612326b0 0x5557613df300 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.033+0000 7f5fc9da2b80 1 --2- 172.21.15.133:0/3354951896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d6db0 0x5557613e7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.033+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/3354951896 >> 172.21.15.133:0/3354951896 conn(0x5557613cbf50 msgr2=0x5557613f26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.033+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/3354951896 shutdown_connections 2023-11-27T15:27:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.033+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/3354951896 wait complete. 2023-11-27T15:27:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.034+0000 7f5fc9da2b80 1 -- start start 2023-11-27T15:27:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.034+0000 7f5fc9da2b80 1 Processor -- start 2023-11-27T15:27:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.035+0000 7f5fc9da2b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557612326b0 0x5557613e2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.035+0000 7f5fc9da2b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557613d6db0 0x5557613e2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:25.039 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.035+0000 7f5fc9da2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d7b10 0x5557613e5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.035+0000 7f5fc9da2b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557613e63a0 con 0x5557613d6db0 2023-11-27T15:27:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.035+0000 7f5fc9da2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557613e6510 con 0x5557613d7b10 2023-11-27T15:27:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.035+0000 7f5fc9da2b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557613e66d0 con 0x5557612326b0 2023-11-27T15:27:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.035+0000 7f5fc9da2b80 10 client.0 _ll_register_callbacks cb 0x7f5fa4005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.035+0000 7f5faffff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557612326b0 0x5557613e2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.035+0000 7f5faffff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557612326b0 0x5557613e2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:36908/0 (socket says 172.21.15.133:36908) 2023-11-27T15:27:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.035+0000 7f5faffff700 1 -- 172.21.15.133:0/2178104755 learned_addr learned my addr 172.21.15.133:0/2178104755 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.035+0000 7f5faf7fe700 1 --2- 172.21.15.133:0/2178104755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557613d6db0 0x5557613e2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.035+0000 7f5faffff700 1 -- 172.21.15.133:0/2178104755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d7b10 msgr2=0x5557613e5e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-27T15:27:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.035+0000 7f5faffff700 1 --2- 172.21.15.133:0/2178104755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d7b10 0x5557613e5e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.035+0000 7f5faffff700 1 -- 172.21.15.133:0/2178104755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557613d6db0 msgr2=0x5557613e2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.036+0000 7f5faffff700 1 --2- 172.21.15.133:0/2178104755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557613d6db0 0x5557613e2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:25.040 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.036+0000 7f5faffff700 1 -- 172.21.15.133:0/2178104755 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb0078220 con 0x5557612326b0 2023-11-27T15:27:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.036+0000 7f5faffff700 1 --2- 172.21.15.133:0/2178104755 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557612326b0 0x5557613e2900 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8000f80 tx=0x7f5fa801a820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30e77bb41737937e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.036+0000 7f5f8f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:27:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.037+0000 7f5f8f7fe700 1 -- 172.21.15.133:0/2178104755 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8051070 con 0x5557612326b0 2023-11-27T15:27:25.041 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.037+0000 7f5f8f7fe700 1 -- 172.21.15.133:0/2178104755 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f5fa803f1a0 con 0x5557612326b0 2023-11-27T15:27:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.037+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/2178104755 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fa0011e20 con 0x5557612326b0 2023-11-27T15:27:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.037+0000 7f5f8f7fe700 1 -- 172.21.15.133:0/2178104755 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fa803d070 con 0x5557612326b0 2023-11-27T15:27:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.037+0000 7f5fc9da2b80 10 client.26284 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.037+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/2178104755 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55576113ec00 con 0x5557612326b0 2023-11-27T15:27:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.037+0000 7f5f8f7fe700 1 -- 172.21.15.133:0/2178104755 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5fa0011e20 con 0x5557612326b0 2023-11-27T15:27:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.040+0000 7f5f8f7fe700 1 -- 172.21.15.133:0/2178104755 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5fa8054030 con 0x5557612326b0 2023-11-27T15:27:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.040+0000 7f5fc9da2b80 10 client.26284 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.040+0000 7f5fc9da2b80 10 client.26284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.040+0000 7f5fc9da2b80 20 client.26284 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.040+0000 7f5fc9da2b80 1 --2- 172.21.15.133:0/2178104755 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55576143fd50 0x555761442160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.040+0000 7f5fc9da2b80 4 client.26284 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:27:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.040+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/2178104755 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5557612f6600 con 0x55576143fd50 2023-11-27T15:27:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.040+0000 7f5faf7fe700 1 --2- 172.21.15.133:0/2178104755 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55576143fd50 0x555761442160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.041+0000 7f5faf7fe700 1 --2- 172.21.15.133:0/2178104755 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55576143fd50 0x555761442160 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b633206ad4668c5 server_cookie=4fa706ec7ed45e2a in_seq=0 out_seq=0 2023-11-27T15:27:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:25.041+0000 7f5f8f7fe700 10 client.26284 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:25.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:24 smithi133 ceph-mon[97860]: pgmap v483: 129 pgs: 129 active+clean; 309 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 596 op/s 2023-11-27T15:27:25.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:24 smithi177 ceph-mon[131504]: pgmap v483: 129 pgs: 129 active+clean; 309 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 596 op/s 2023-11-27T15:27:25.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:24 smithi174 ceph-mon[131110]: pgmap v483: 129 pgs: 129 active+clean; 309 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.3 MiB/s wr, 596 op/s 2023-11-27T15:27:27.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:26 smithi133 ceph-mon[97860]: pgmap v484: 129 pgs: 129 active+clean; 321 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 625 op/s 2023-11-27T15:27:27.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:26 smithi177 ceph-mon[131504]: pgmap v484: 129 pgs: 129 active+clean; 321 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 625 op/s 2023-11-27T15:27:27.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:26 smithi174 ceph-mon[131110]: pgmap v484: 129 pgs: 129 active+clean; 321 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 625 op/s 2023-11-27T15:27:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.495+0000 7f5f8f7fe700 1 -- 172.21.15.133:0/2178104755 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5557612f6600 con 0x55576143fd50 2023-11-27T15:27:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.495+0000 7f5f8f7fe700 10 client.26284 handle_command_reply: tid=0 2023-11-27T15:27:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.495+0000 7f5fc9da2b80 1 client.26284 shutdown 2023-11-27T15:27:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.496+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/2178104755 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557612326b0 msgr2=0x5557613e2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.496+0000 7f5fc9da2b80 1 --2- 172.21.15.133:0/2178104755 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557612326b0 0x5557613e2900 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8000f80 tx=0x7f5fa801a820 comp rx=0 tx=0).stop 2023-11-27T15:27:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.497+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/2178104755 shutdown_connections 2023-11-27T15:27:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.497+0000 7f5fc9da2b80 1 --2- 172.21.15.133:0/2178104755 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55576143fd50 0x555761442160 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.497+0000 7f5fc9da2b80 1 --2- 172.21.15.133:0/2178104755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5557613d7b10 0x5557613e5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.501 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.497+0000 7f5fc9da2b80 1 --2- 172.21.15.133:0/2178104755 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5557612326b0 0x5557613e2900 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.501 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.497+0000 7f5fc9da2b80 1 --2- 172.21.15.133:0/2178104755 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5557613d6db0 0x5557613e2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.501 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.497+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/2178104755 >> 172.21.15.133:0/2178104755 conn(0x5557613cbf50 msgr2=0x5557613da960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:27.501 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.497+0000 7f5f8f7fe700 0 client.26284 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.500+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/2178104755 shutdown_connections 2023-11-27T15:27:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.500+0000 7f5fc9da2b80 1 -- 172.21.15.133:0/2178104755 wait complete. 2023-11-27T15:27:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.501+0000 7f5fc9da2b80 20 client.26284 trim_cache size 0 max 16384 2023-11-27T15:27:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:27:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:27:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:27:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:27:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.502+0000 7f5fb63ff700 1 -- 172.21.15.133:0/2103539989 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5f980a1150 msgr2=0x7f5f980a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.502+0000 7f5fb63ff700 1 --2- 172.21.15.133:0/2103539989 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5f980a1150 0x7f5f980a3600 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8018b50 tx=0x7f5fa801c040 comp rx=0 tx=0).stop 2023-11-27T15:27:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.502+0000 7f5fb63ff700 1 -- 172.21.15.133:0/2103539989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fb00766f0 msgr2=0x7f5fb01942f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.502+0000 7f5fb63ff700 1 --2- 172.21.15.133:0/2103539989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fb00766f0 0x7f5fb01942f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5fa400e9a0 tx=0x7f5fa400ecb0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.502+0000 7f5fb63ff700 1 -- 172.21.15.133:0/2103539989 shutdown_connections 2023-11-27T15:27:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.502+0000 7f5fb63ff700 1 --2- 172.21.15.133:0/2103539989 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f5f980a1150 0x7f5f980a3600 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.502+0000 7f5fb63ff700 1 --2- 172.21.15.133:0/2103539989 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fb00770a0 0x7f5fb019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.502+0000 7f5fb63ff700 1 --2- 172.21.15.133:0/2103539989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5fb00754b0 0x7f5fb0193db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.502+0000 7f5fb63ff700 1 --2- 172.21.15.133:0/2103539989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fb00766f0 0x7f5fb01942f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.502+0000 7f5fb63ff700 1 -- 172.21.15.133:0/2103539989 >> 172.21.15.133:0/2103539989 conn(0x7f5fb00fdac0 msgr2=0x7f5fb00ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.502+0000 7f5fb63ff700 1 -- 172.21.15.133:0/2103539989 shutdown_connections 2023-11-27T15:27:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:27.502+0000 7f5fb63ff700 1 -- 172.21.15.133:0/2103539989 wait complete. 2023-11-27T15:27:29.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:28 smithi133 ceph-mon[97860]: pgmap v485: 129 pgs: 129 active+clean; 328 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 573 op/s 2023-11-27T15:27:29.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:28 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:27:29.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:28 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:29.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:28 smithi177 ceph-mon[131504]: pgmap v485: 129 pgs: 129 active+clean; 328 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 573 op/s 2023-11-27T15:27:29.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:28 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:27:29.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:28 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:29.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:28 smithi174 ceph-mon[131110]: pgmap v485: 129 pgs: 129 active+clean; 328 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.2 MiB/s wr, 573 op/s 2023-11-27T15:27:29.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:28 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:27:29.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:28 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:31.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:30 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:27:31.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:30 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:31.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:30 smithi174 ceph-mon[131110]: pgmap v486: 129 pgs: 129 active+clean; 325 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 645 op/s 2023-11-27T15:27:31.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:30 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:27:31.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:30 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:31.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:30 smithi133 ceph-mon[97860]: pgmap v486: 129 pgs: 129 active+clean; 325 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 645 op/s 2023-11-27T15:27:31.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:30 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:27:31.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:30 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:31.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:30 smithi177 ceph-mon[131504]: pgmap v486: 129 pgs: 129 active+clean; 325 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.6 MiB/s wr, 645 op/s 2023-11-27T15:27:33.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:32 smithi174 ceph-mon[131110]: pgmap v487: 129 pgs: 129 active+clean; 331 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 667 op/s 2023-11-27T15:27:33.276 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:32 smithi133 ceph-mon[97860]: pgmap v487: 129 pgs: 129 active+clean; 331 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 667 op/s 2023-11-27T15:27:33.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:32 smithi177 ceph-mon[131504]: pgmap v487: 129 pgs: 129 active+clean; 331 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 667 op/s 2023-11-27T15:27:35.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:34 smithi174 ceph-mon[131110]: pgmap v488: 129 pgs: 129 active+clean; 334 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 656 op/s 2023-11-27T15:27:35.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:34 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:27:35.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:34 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:35.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:34 smithi133 ceph-mon[97860]: pgmap v488: 129 pgs: 129 active+clean; 334 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 656 op/s 2023-11-27T15:27:35.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:34 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:27:35.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:34 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:35.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:34 smithi177 ceph-mon[131504]: pgmap v488: 129 pgs: 129 active+clean; 334 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 656 op/s 2023-11-27T15:27:35.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:34 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:27:35.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:34 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:37.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:36 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:37.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:36 smithi174 ceph-mon[131110]: pgmap v489: 129 pgs: 129 active+clean; 339 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 728 op/s 2023-11-27T15:27:37.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:36 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:37.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:36 smithi133 ceph-mon[97860]: pgmap v489: 129 pgs: 129 active+clean; 339 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 728 op/s 2023-11-27T15:27:37.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:36 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:37.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:36 smithi177 ceph-mon[131504]: pgmap v489: 129 pgs: 129 active+clean; 339 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.5 MiB/s wr, 728 op/s 2023-11-27T15:27:39.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:38 smithi133 ceph-mon[97860]: pgmap v490: 129 pgs: 129 active+clean; 350 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 684 op/s 2023-11-27T15:27:39.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:38 smithi177 ceph-mon[131504]: pgmap v490: 129 pgs: 129 active+clean; 350 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 684 op/s 2023-11-27T15:27:39.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:38 smithi174 ceph-mon[131110]: pgmap v490: 129 pgs: 129 active+clean; 350 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.5 MiB/s wr, 684 op/s 2023-11-27T15:27:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.728+0000 7fba815ff700 1 -- 172.21.15.133:0/2213601155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba7c1036a0 msgr2=0x7fba7c105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.728+0000 7fba815ff700 1 --2- 172.21.15.133:0/2213601155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba7c1036a0 0x7fba7c105a80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fba700098f0 tx=0x7fba70009c00 comp rx=0 tx=0).stop 2023-11-27T15:27:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.728+0000 7fba815ff700 1 -- 172.21.15.133:0/2213601155 shutdown_connections 2023-11-27T15:27:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.728+0000 7fba815ff700 1 --2- 172.21.15.133:0/2213601155 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba7c105fc0 0x7fba7c1083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.728+0000 7fba815ff700 1 --2- 172.21.15.133:0/2213601155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba7c100d80 0x7fba7c103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.728+0000 7fba815ff700 1 --2- 172.21.15.133:0/2213601155 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba7c1036a0 0x7fba7c105a80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.728+0000 7fba815ff700 1 -- 172.21.15.133:0/2213601155 >> 172.21.15.133:0/2213601155 conn(0x7fba7c0fa720 msgr2=0x7fba7c0fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.728+0000 7fba815ff700 1 -- 172.21.15.133:0/2213601155 shutdown_connections 2023-11-27T15:27:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.729+0000 7fba815ff700 1 -- 172.21.15.133:0/2213601155 wait complete. 2023-11-27T15:27:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.730+0000 7fba815ff700 1 Processor -- start 2023-11-27T15:27:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.730+0000 7fba815ff700 1 -- start start 2023-11-27T15:27:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.730+0000 7fba815ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba7c100d80 0x7fba7c1006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.730+0000 7fba815ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba7c1036a0 0x7fba7c0fed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.730+0000 7fba815ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba7c105fc0 0x7fba7c0ff270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.730+0000 7fba815ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c0ffa10 con 0x7fba7c1036a0 2023-11-27T15:27:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.730+0000 7fba815ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c0ffb80 con 0x7fba7c105fc0 2023-11-27T15:27:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.730+0000 7fba815ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c1a0090 con 0x7fba7c100d80 2023-11-27T15:27:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.731+0000 7fba7affd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba7c100d80 0x7fba7c1006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.731+0000 7fba7b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba7c105fc0 0x7fba7c0ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:40.739 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.731+0000 7fba7affd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba7c100d80 0x7fba7c1006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:46246/0 (socket says 172.21.15.133:46246) 2023-11-27T15:27:40.740 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.731+0000 7fba7affd700 1 -- 172.21.15.133:0/3486255335 learned_addr learned my addr 172.21.15.133:0/3486255335 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:40.740 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.731+0000 7fba7a7fc700 1 --2- 172.21.15.133:0/3486255335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba7c1036a0 0x7fba7c0fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:40.740 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.731+0000 7fba7affd700 1 -- 172.21.15.133:0/3486255335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba7c105fc0 msgr2=0x7fba7c0ff270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:40.740 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.731+0000 7fba7affd700 1 --2- 172.21.15.133:0/3486255335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba7c105fc0 0x7fba7c0ff270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.740 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.731+0000 7fba7affd700 1 -- 172.21.15.133:0/3486255335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba7c1036a0 msgr2=0x7fba7c0fed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:40.740 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.731+0000 7fba7affd700 1 --2- 172.21.15.133:0/3486255335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba7c1036a0 0x7fba7c0fed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.740 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.731+0000 7fba7affd700 1 -- 172.21.15.133:0/3486255335 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba70009520 con 0x7fba7c100d80 2023-11-27T15:27:40.740 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.731+0000 7fba7a7fc700 1 --2- 172.21.15.133:0/3486255335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba7c1036a0 0x7fba7c0fed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.731+0000 7fba7b7fe700 1 --2- 172.21.15.133:0/3486255335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba7c105fc0 0x7fba7c0ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:27:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.731+0000 7fba7affd700 1 --2- 172.21.15.133:0/3486255335 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba7c100d80 0x7fba7c1006f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00e9c0 tx=0x7fba6c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.732+0000 7fba63fff700 1 -- 172.21.15.133:0/3486255335 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c00cbd0 con 0x7fba7c100d80 2023-11-27T15:27:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.732+0000 7fba63fff700 1 -- 172.21.15.133:0/3486255335 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba6c00eeb0 con 0x7fba7c100d80 2023-11-27T15:27:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.732+0000 7fba63fff700 1 -- 172.21.15.133:0/3486255335 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c005580 con 0x7fba7c100d80 2023-11-27T15:27:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.732+0000 7fba815ff700 1 -- 172.21.15.133:0/3486255335 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7c10bc10 con 0x7fba7c100d80 2023-11-27T15:27:40.741 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.732+0000 7fba815ff700 1 -- 172.21.15.133:0/3486255335 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7c10ba80 con 0x7fba7c100d80 2023-11-27T15:27:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.736+0000 7fba63fff700 1 -- 172.21.15.133:0/3486255335 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fba6c004750 con 0x7fba7c100d80 2023-11-27T15:27:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.737+0000 7fba94ea7b80 1 Processor -- start 2023-11-27T15:27:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.737+0000 7fba94ea7b80 1 -- start start 2023-11-27T15:27:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.737+0000 7fba63fff700 1 --2- 172.21.15.133:0/3486255335 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fba640a12a0 0x7fba640a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.737+0000 7fba94ea7b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacaa46b0 0x55efacc48870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.737+0000 7fba94ea7b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 0x55efacc495d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.737+0000 7fba94ea7b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacc49b10 0x55efacc49f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:40.745 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.737+0000 7fba94ea7b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efacb4e000 con 0x55efacc49b10 2023-11-27T15:27:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.737+0000 7fba94ea7b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efacba0840 con 0x55efacaa46b0 2023-11-27T15:27:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.737+0000 7fba94ea7b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efacb99cd0 con 0x55efacc48db0 2023-11-27T15:27:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.738+0000 7fba63fff700 1 -- 172.21.15.133:0/3486255335 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fba6c014070 con 0x7fba7c100d80 2023-11-27T15:27:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.738+0000 7fba7a7fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 0x55efacc495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.738+0000 7fba7affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacaa46b0 0x55efacc48870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.738+0000 7fba7a7fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 0x55efacc495d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:46254/0 (socket says 172.21.15.133:46254) 2023-11-27T15:27:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.738+0000 7fba7affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacaa46b0 0x55efacc48870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:49472/0 (socket says 172.21.15.133:49472) 2023-11-27T15:27:40.746 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.738+0000 7fba7a7fc700 1 -- 172.21.15.133:0/2279174832 learned_addr learned my addr 172.21.15.133:0/2279174832 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.738+0000 7fba7b7fe700 1 --2- 172.21.15.133:0/3486255335 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fba640a12a0 0x7fba640a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.738+0000 7fba7b7fe700 1 --2- 172.21.15.133:0/2279174832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacc49b10 0x55efacc49f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.738+0000 7fba7affd700 1 -- 172.21.15.133:0/2279174832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 msgr2=0x55efacc495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.738+0000 7fba7affd700 1 --2- 172.21.15.133:0/2279174832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 0x55efacc495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.738+0000 7fba7affd700 1 -- 172.21.15.133:0/2279174832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacc49b10 msgr2=0x55efacc49f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.738+0000 7fba7affd700 1 --2- 172.21.15.133:0/2279174832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacc49b10 0x55efacc49f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.738+0000 7fba7affd700 1 -- 172.21.15.133:0/2279174832 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efaca1b5d0 con 0x55efacaa46b0 2023-11-27T15:27:40.747 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.738+0000 7fba7b7fe700 1 --2- 172.21.15.133:0/2279174832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacc49b10 0x55efacc49f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.738+0000 7fba7affd700 1 --2- 172.21.15.133:0/2279174832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacaa46b0 0x55efacc48870 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fba6c000c00 tx=0x7fba6c099ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=556989ddb20fd27 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.739+0000 7fba617fa700 1 -- 172.21.15.133:0/2279174832 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c00cd30 con 0x55efacaa46b0 2023-11-27T15:27:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.739+0000 7fba617fa700 1 -- 172.21.15.133:0/2279174832 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba6c099fd0 con 0x55efacaa46b0 2023-11-27T15:27:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.739+0000 7fba617fa700 1 -- 172.21.15.133:0/2279174832 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0adcd0 con 0x55efacaa46b0 2023-11-27T15:27:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.739+0000 7fba94ea7b80 1 -- 172.21.15.133:0/2279174832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacaa46b0 msgr2=0x55efacc48870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.739+0000 7fba94ea7b80 1 --2- 172.21.15.133:0/2279174832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacaa46b0 0x55efacc48870 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fba6c000c00 tx=0x7fba6c099ed0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.739+0000 7fba7b7fe700 1 --2- 172.21.15.133:0/3486255335 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fba640a12a0 0x7fba640a3750 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fba7c100d50 tx=0x7fba7400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.739+0000 7fba94ea7b80 1 -- 172.21.15.133:0/2279174832 shutdown_connections 2023-11-27T15:27:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.740+0000 7fba94ea7b80 1 --2- 172.21.15.133:0/2279174832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacc49b10 0x55efacc49f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.740+0000 7fba94ea7b80 1 --2- 172.21.15.133:0/2279174832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacaa46b0 0x55efacc48870 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.740+0000 7fba94ea7b80 1 --2- 172.21.15.133:0/2279174832 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 0x55efacc495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.740+0000 7fba94ea7b80 1 -- 172.21.15.133:0/2279174832 >> 172.21.15.133:0/2279174832 conn(0x55efacc3df50 msgr2=0x55efacc43100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.740+0000 7fba94ea7b80 1 -- 172.21.15.133:0/2279174832 shutdown_connections 2023-11-27T15:27:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.740+0000 7fba94ea7b80 1 -- 172.21.15.133:0/2279174832 wait complete. 2023-11-27T15:27:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.740+0000 7fba94ea7b80 1 -- start start 2023-11-27T15:27:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.740+0000 7fba94ea7b80 1 Processor -- start 2023-11-27T15:27:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.741+0000 7fba94ea7b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacaa46b0 0x55efacc51c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.741+0000 7fba94ea7b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 0x55efacc521b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:40.755 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.741+0000 7fba94ea7b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacc49b10 0x55efacc591f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.741+0000 7fba94ea7b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efacb3b880 con 0x55efacaa46b0 2023-11-27T15:27:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.741+0000 7fba94ea7b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efacb3b520 con 0x55efacc49b10 2023-11-27T15:27:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.741+0000 7fba94ea7b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efaca1d8c0 con 0x55efacc48db0 2023-11-27T15:27:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.741+0000 7fba7a7fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 0x55efacc521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.741+0000 7fba94ea7b80 10 client.0 _ll_register_callbacks cb 0x55efacb59720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.741+0000 7fba7a7fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 0x55efacc521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:46256/0 (socket says 172.21.15.133:46256) 2023-11-27T15:27:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.741+0000 7fba7a7fc700 1 -- 172.21.15.133:0/2110833558 learned_addr learned my addr 172.21.15.133:0/2110833558 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.741+0000 7fba7affd700 1 --2- 172.21.15.133:0/2110833558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacaa46b0 0x55efacc51c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.741+0000 7fba7b7fe700 1 --2- 172.21.15.133:0/2110833558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacc49b10 0x55efacc591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.742+0000 7fba7a7fc700 1 -- 172.21.15.133:0/2110833558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacc49b10 msgr2=0x55efacc591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.742+0000 7fba7a7fc700 1 --2- 172.21.15.133:0/2110833558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacc49b10 0x55efacc591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.756 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.742+0000 7fba7a7fc700 1 -- 172.21.15.133:0/2110833558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacaa46b0 msgr2=0x55efacc51c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.742+0000 7fba7a7fc700 1 --2- 172.21.15.133:0/2110833558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacaa46b0 0x55efacc51c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.742+0000 7fba7a7fc700 1 -- 172.21.15.133:0/2110833558 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7c10ba80 con 0x55efacc48db0 2023-11-27T15:27:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.742+0000 7fba7a7fc700 1 --2- 172.21.15.133:0/2110833558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 0x55efacc521b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fba7000fa40 tx=0x7fba70024b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ea549f870a1ec6b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.742+0000 7fba5eff5700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:27:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.742+0000 7fba5eff5700 1 -- 172.21.15.133:0/2110833558 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7000b970 con 0x55efacc48db0 2023-11-27T15:27:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.742+0000 7fba94ea7b80 1 -- 172.21.15.133:0/2110833558 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55efacc3b710 con 0x55efacc48db0 2023-11-27T15:27:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.743+0000 7fba5eff5700 1 -- 172.21.15.133:0/2110833558 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fba7000bad0 con 0x55efacc48db0 2023-11-27T15:27:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.743+0000 7fba5eff5700 1 -- 172.21.15.133:0/2110833558 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba70005040 con 0x55efacc48db0 2023-11-27T15:27:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.743+0000 7fba94ea7b80 10 client.26296 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.743+0000 7fba94ea7b80 1 -- 172.21.15.133:0/2110833558 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55efacb60130 con 0x55efacc48db0 2023-11-27T15:27:40.757 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.743+0000 7fba5eff5700 1 -- 172.21.15.133:0/2110833558 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55efacc3b710 con 0x55efacc48db0 2023-11-27T15:27:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.743+0000 7fba5eff5700 1 -- 172.21.15.133:0/2110833558 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fba70015020 con 0x55efacc48db0 2023-11-27T15:27:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.744+0000 7fba94ea7b80 10 client.26296 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.744+0000 7fba94ea7b80 10 client.26296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.744+0000 7fba94ea7b80 20 client.26296 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.744+0000 7fba94ea7b80 1 --2- 172.21.15.133:0/2110833558 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55efacc5da60 0x55efacc5fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.744+0000 7fba94ea7b80 4 client.26296 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.744+0000 7fba94ea7b80 1 -- 172.21.15.133:0/2110833558 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55efacb5c040 con 0x55efacc5da60 2023-11-27T15:27:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.744+0000 7fba7affd700 1 --2- 172.21.15.133:0/2110833558 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55efacc5da60 0x55efacc5fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.745+0000 7fba5eff5700 10 client.26296 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.745+0000 7fba7affd700 1 --2- 172.21.15.133:0/2110833558 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55efacc5da60 0x55efacc5fe70 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=467d01ff58295f9c server_cookie=5dea2fb67c1d952f in_seq=0 out_seq=0 2023-11-27T15:27:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.746+0000 7fba5eff5700 1 -- 172.21.15.133:0/2110833558 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55efacb5c040 con 0x55efacc5da60 2023-11-27T15:27:40.758 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.746+0000 7fba5eff5700 10 client.26296 handle_command_reply: tid=0 2023-11-27T15:27:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.746+0000 7fba94ea7b80 1 client.26296 shutdown 2023-11-27T15:27:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.746+0000 7fba94ea7b80 1 -- 172.21.15.133:0/2110833558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 msgr2=0x55efacc521b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.746+0000 7fba94ea7b80 1 --2- 172.21.15.133:0/2110833558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 0x55efacc521b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fba7000fa40 tx=0x7fba70024b80 comp rx=0 tx=0).stop 2023-11-27T15:27:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.746+0000 7fba94ea7b80 1 -- 172.21.15.133:0/2110833558 shutdown_connections 2023-11-27T15:27:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.746+0000 7fba94ea7b80 1 --2- 172.21.15.133:0/2110833558 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55efacc5da60 0x55efacc5fe70 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.746+0000 7fba94ea7b80 1 --2- 172.21.15.133:0/2110833558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacc49b10 0x55efacc591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.746+0000 7fba94ea7b80 1 --2- 172.21.15.133:0/2110833558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacaa46b0 0x55efacc51c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.746+0000 7fba94ea7b80 1 --2- 172.21.15.133:0/2110833558 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 0x55efacc521b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.746+0000 7fba94ea7b80 1 -- 172.21.15.133:0/2110833558 >> 172.21.15.133:0/2110833558 conn(0x55efacc3df50 msgr2=0x55efacc427e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.746+0000 7fba5eff5700 0 client.26296 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:40.759 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.746+0000 7fba94ea7b80 1 -- 172.21.15.133:0/2110833558 shutdown_connections 2023-11-27T15:27:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.746+0000 7fba94ea7b80 1 -- 172.21.15.133:0/2110833558 wait complete. 2023-11-27T15:27:40.760 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.747+0000 7fba94ea7b80 20 client.26296 trim_cache size 0 max 16384 2023-11-27T15:27:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.764+0000 7fba94ea7b80 1 Processor -- start 2023-11-27T15:27:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.764+0000 7fba94ea7b80 1 -- start start 2023-11-27T15:27:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.764+0000 7fba94ea7b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacaa46b0 0x55efacc4c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.764+0000 7fba94ea7b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 0x55efacc4cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.764+0000 7fba94ea7b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacc49b10 0x55efacc591b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.764+0000 7fba94ea7b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efacc44930 con 0x55efacc49b10 2023-11-27T15:27:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.764+0000 7fba94ea7b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efacc528f0 con 0x55efacaa46b0 2023-11-27T15:27:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.764+0000 7fba94ea7b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efacc50740 con 0x55efacc48db0 2023-11-27T15:27:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.765+0000 7fba7a7fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 0x55efacc4cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.765+0000 7fba7affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacaa46b0 0x55efacc4c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.765+0000 7fba7affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacaa46b0 0x55efacc4c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:49484/0 (socket says 172.21.15.133:49484) 2023-11-27T15:27:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.765+0000 7fba7a7fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 0x55efacc4cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:46272/0 (socket says 172.21.15.133:46272) 2023-11-27T15:27:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.765+0000 7fba7affd700 1 -- 172.21.15.133:0/389698959 learned_addr learned my addr 172.21.15.133:0/389698959 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.765+0000 7fba7b7fe700 1 --2- 172.21.15.133:0/389698959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacc49b10 0x55efacc591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.765+0000 7fba7affd700 1 -- 172.21.15.133:0/389698959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 msgr2=0x55efacc4cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.765+0000 7fba7affd700 1 --2- 172.21.15.133:0/389698959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 0x55efacc4cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.765+0000 7fba7affd700 1 -- 172.21.15.133:0/389698959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacc49b10 msgr2=0x55efacc591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.765+0000 7fba7affd700 1 --2- 172.21.15.133:0/389698959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacc49b10 0x55efacc591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.765+0000 7fba7affd700 1 -- 172.21.15.133:0/389698959 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55efacb5fe80 con 0x55efacaa46b0 2023-11-27T15:27:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.765+0000 7fba7b7fe700 1 --2- 172.21.15.133:0/389698959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacc49b10 0x55efacc591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.765+0000 7fba7affd700 1 --2- 172.21.15.133:0/389698959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacaa46b0 0x55efacc4c9e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0ca710 tx=0x7fba6c00cf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5d924780e70a9ff server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.765+0000 7fba61ffb700 1 -- 172.21.15.133:0/389698959 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c00cd30 con 0x55efacaa46b0 2023-11-27T15:27:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.766+0000 7fba61ffb700 1 -- 172.21.15.133:0/389698959 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba6c0b7070 con 0x55efacaa46b0 2023-11-27T15:27:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.766+0000 7fba61ffb700 1 -- 172.21.15.133:0/389698959 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0b8420 con 0x55efacaa46b0 2023-11-27T15:27:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.766+0000 7fba94ea7b80 1 -- 172.21.15.133:0/389698959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacaa46b0 msgr2=0x55efacc4c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.766+0000 7fba94ea7b80 1 --2- 172.21.15.133:0/389698959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacaa46b0 0x55efacc4c9e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0ca710 tx=0x7fba6c00cf10 comp rx=0 tx=0).stop 2023-11-27T15:27:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.766+0000 7fba94ea7b80 1 -- 172.21.15.133:0/389698959 shutdown_connections 2023-11-27T15:27:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.766+0000 7fba94ea7b80 1 --2- 172.21.15.133:0/389698959 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacc49b10 0x55efacc591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.766+0000 7fba94ea7b80 1 --2- 172.21.15.133:0/389698959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacaa46b0 0x55efacc4c9e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.766+0000 7fba94ea7b80 1 --2- 172.21.15.133:0/389698959 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 0x55efacc4cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.766+0000 7fba94ea7b80 1 -- 172.21.15.133:0/389698959 >> 172.21.15.133:0/389698959 conn(0x55efacc3df50 msgr2=0x55efacc64790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.766+0000 7fba94ea7b80 1 -- 172.21.15.133:0/389698959 shutdown_connections 2023-11-27T15:27:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.767+0000 7fba94ea7b80 1 -- 172.21.15.133:0/389698959 wait complete. 2023-11-27T15:27:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.767+0000 7fba94ea7b80 1 -- start start 2023-11-27T15:27:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.767+0000 7fba94ea7b80 1 Processor -- start 2023-11-27T15:27:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.768+0000 7fba94ea7b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacaa46b0 0x55efacc54cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.768+0000 7fba94ea7b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 0x55efacc55210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.768+0000 7fba94ea7b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacc49b10 0x55efacc58230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.768+0000 7fba94ea7b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efacc58770 con 0x55efacc49b10 2023-11-27T15:27:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.768+0000 7fba94ea7b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efacc588e0 con 0x55efacaa46b0 2023-11-27T15:27:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.768+0000 7fba94ea7b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55efacc58aa0 con 0x55efacc48db0 2023-11-27T15:27:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.768+0000 7fba94ea7b80 10 client.0 _ll_register_callbacks cb 0x7fba7000f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.768+0000 7fba7affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacaa46b0 0x55efacc54cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.768+0000 7fba7affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacaa46b0 0x55efacc54cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:49494/0 (socket says 172.21.15.133:49494) 2023-11-27T15:27:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.768+0000 7fba7affd700 1 -- 172.21.15.133:0/1923693378 learned_addr learned my addr 172.21.15.133:0/1923693378 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.768+0000 7fba7a7fc700 1 --2- 172.21.15.133:0/1923693378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 0x55efacc55210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.768+0000 7fba7b7fe700 1 --2- 172.21.15.133:0/1923693378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacc49b10 0x55efacc58230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.768+0000 7fba7affd700 1 -- 172.21.15.133:0/1923693378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 msgr2=0x55efacc55210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.768+0000 7fba7affd700 1 --2- 172.21.15.133:0/1923693378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 0x55efacc55210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.768+0000 7fba7affd700 1 -- 172.21.15.133:0/1923693378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacc49b10 msgr2=0x55efacc58230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.768+0000 7fba7affd700 1 --2- 172.21.15.133:0/1923693378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacc49b10 0x55efacc58230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.768+0000 7fba7affd700 1 -- 172.21.15.133:0/1923693378 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55efaca1b5d0 con 0x55efacaa46b0 2023-11-27T15:27:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.769+0000 7fba7affd700 1 --2- 172.21.15.133:0/1923693378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacaa46b0 0x55efacc54cd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0a1ef0 tx=0x7fba6c009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b3d36f8d2d3bcfb server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.769+0000 7fba5eff5700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:27:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.769+0000 7fba5eff5700 1 -- 172.21.15.133:0/1923693378 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0b83c0 con 0x55efacaa46b0 2023-11-27T15:27:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.769+0000 7fba5eff5700 1 -- 172.21.15.133:0/1923693378 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fba6c0a2420 con 0x55efacaa46b0 2023-11-27T15:27:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.769+0000 7fba94ea7b80 1 -- 172.21.15.133:0/1923693378 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba6c099fd0 con 0x55efacaa46b0 2023-11-27T15:27:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.769+0000 7fba5eff5700 1 -- 172.21.15.133:0/1923693378 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba6c098370 con 0x55efacaa46b0 2023-11-27T15:27:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.769+0000 7fba94ea7b80 10 client.26318 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.769+0000 7fba94ea7b80 1 -- 172.21.15.133:0/1923693378 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55efac9b0c00 con 0x55efacaa46b0 2023-11-27T15:27:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.769+0000 7fba5eff5700 1 -- 172.21.15.133:0/1923693378 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba6c099fd0 con 0x55efacaa46b0 2023-11-27T15:27:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.769+0000 7fba5eff5700 1 -- 172.21.15.133:0/1923693378 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fba6c0994c0 con 0x55efacaa46b0 2023-11-27T15:27:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.770+0000 7fba94ea7b80 10 client.26318 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.770+0000 7fba94ea7b80 10 client.26318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.770+0000 7fba94ea7b80 20 client.26318 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.770+0000 7fba94ea7b80 1 --2- 172.21.15.133:0/1923693378 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55efaccb2140 0x55efaccb4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.770+0000 7fba94ea7b80 4 client.26318 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:27:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.770+0000 7fba94ea7b80 1 -- 172.21.15.133:0/1923693378 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55efacb68600 con 0x55efaccb2140 2023-11-27T15:27:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.770+0000 7fba7a7fc700 1 --2- 172.21.15.133:0/1923693378 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55efaccb2140 0x55efaccb4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.770+0000 7fba5eff5700 10 client.26318 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:40.770+0000 7fba7a7fc700 1 --2- 172.21.15.133:0/1923693378 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55efaccb2140 0x55efaccb4520 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2636731968a3fc5e server_cookie=c76c50cb12aa98ba in_seq=0 out_seq=0 2023-11-27T15:27:41.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:40 smithi174 ceph-mon[131110]: pgmap v491: 129 pgs: 129 active+clean; 350 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 684 op/s 2023-11-27T15:27:41.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:40 smithi133 ceph-mon[97860]: pgmap v491: 129 pgs: 129 active+clean; 350 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 684 op/s 2023-11-27T15:27:41.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:40 smithi177 ceph-mon[131504]: pgmap v491: 129 pgs: 129 active+clean; 350 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 684 op/s 2023-11-27T15:27:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.984+0000 7fba5eff5700 1 -- 172.21.15.133:0/1923693378 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55efacb68600 con 0x55efaccb2140 2023-11-27T15:27:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.984+0000 7fba5eff5700 10 client.26318 handle_command_reply: tid=0 2023-11-27T15:27:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.984+0000 7fba94ea7b80 1 client.26318 shutdown 2023-11-27T15:27:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.985+0000 7fba94ea7b80 1 -- 172.21.15.133:0/1923693378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacaa46b0 msgr2=0x55efacc54cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.985+0000 7fba94ea7b80 1 --2- 172.21.15.133:0/1923693378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacaa46b0 0x55efacc54cd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0a1ef0 tx=0x7fba6c009550 comp rx=0 tx=0).stop 2023-11-27T15:27:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.985+0000 7fba94ea7b80 1 -- 172.21.15.133:0/1923693378 shutdown_connections 2023-11-27T15:27:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.985+0000 7fba94ea7b80 1 --2- 172.21.15.133:0/1923693378 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55efaccb2140 0x55efaccb4520 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.985+0000 7fba94ea7b80 1 --2- 172.21.15.133:0/1923693378 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55efacc49b10 0x55efacc58230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.985+0000 7fba94ea7b80 1 --2- 172.21.15.133:0/1923693378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efacaa46b0 0x55efacc54cd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.985+0000 7fba5eff5700 0 client.26318 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.985+0000 7fba94ea7b80 1 --2- 172.21.15.133:0/1923693378 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55efacc48db0 0x55efacc55210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.985+0000 7fba94ea7b80 1 -- 172.21.15.133:0/1923693378 >> 172.21.15.133:0/1923693378 conn(0x55efacc3df50 msgr2=0x55efacc4c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.985+0000 7fba94ea7b80 1 -- 172.21.15.133:0/1923693378 shutdown_connections 2023-11-27T15:27:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.985+0000 7fba94ea7b80 1 -- 172.21.15.133:0/1923693378 wait complete. 2023-11-27T15:27:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.985+0000 7fba94ea7b80 20 client.26318 trim_cache size 0 max 16384 2023-11-27T15:27:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:27:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:27:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:27:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:27:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.986+0000 7fba815ff700 1 -- 172.21.15.133:0/3486255335 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fba640a12a0 msgr2=0x7fba640a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.986+0000 7fba815ff700 1 --2- 172.21.15.133:0/3486255335 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fba640a12a0 0x7fba640a3750 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fba7c100d50 tx=0x7fba7400b040 comp rx=0 tx=0).stop 2023-11-27T15:27:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.986+0000 7fba815ff700 1 -- 172.21.15.133:0/3486255335 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba7c100d80 msgr2=0x7fba7c1006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.986+0000 7fba815ff700 1 --2- 172.21.15.133:0/3486255335 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba7c100d80 0x7fba7c1006f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00e9c0 tx=0x7fba6c00ecd0 comp rx=0 tx=0).stop 2023-11-27T15:27:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.986+0000 7fba815ff700 1 -- 172.21.15.133:0/3486255335 shutdown_connections 2023-11-27T15:27:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.986+0000 7fba815ff700 1 --2- 172.21.15.133:0/3486255335 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fba640a12a0 0x7fba640a3750 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.986+0000 7fba815ff700 1 --2- 172.21.15.133:0/3486255335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba7c105fc0 0x7fba7c0ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.986+0000 7fba815ff700 1 --2- 172.21.15.133:0/3486255335 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fba7c100d80 0x7fba7c1006f0 secure :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00e9c0 tx=0x7fba6c00ecd0 comp rx=0 tx=0).stop 2023-11-27T15:27:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.986+0000 7fba815ff700 1 --2- 172.21.15.133:0/3486255335 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fba7c1036a0 0x7fba7c0fed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.986+0000 7fba815ff700 1 -- 172.21.15.133:0/3486255335 >> 172.21.15.133:0/3486255335 conn(0x7fba7c0fa720 msgr2=0x7fba7c102220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.986+0000 7fba815ff700 1 -- 172.21.15.133:0/3486255335 shutdown_connections 2023-11-27T15:27:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:41.986+0000 7fba815ff700 1 -- 172.21.15.133:0/3486255335 wait complete. 2023-11-27T15:27:43.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:42 smithi133 ceph-mon[97860]: pgmap v492: 129 pgs: 129 active+clean; 353 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 647 op/s 2023-11-27T15:27:43.283 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:42 smithi177 ceph-mon[131504]: pgmap v492: 129 pgs: 129 active+clean; 353 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 647 op/s 2023-11-27T15:27:43.406 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:42 smithi174 ceph-mon[131110]: pgmap v492: 129 pgs: 129 active+clean; 353 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.7 MiB/s wr, 647 op/s 2023-11-27T15:27:43.587 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:27:43 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-c[157240]: *** Caught signal (Segmentation fault) ** 2023-11-27T15:27:43.587 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:27:43 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-c[157240]: in thread 7f1258388700 thread_name:MR_Finisher 2023-11-27T15:27:45.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:44 smithi133 ceph-mon[97860]: pgmap v493: 129 pgs: 129 active+clean; 360 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 618 op/s 2023-11-27T15:27:45.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:44 smithi177 ceph-mon[131504]: pgmap v493: 129 pgs: 129 active+clean; 360 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 618 op/s 2023-11-27T15:27:45.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:44 smithi174 ceph-mon[131110]: pgmap v493: 129 pgs: 129 active+clean; 360 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 618 op/s 2023-11-27T15:27:46.437 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:27:46 smithi177 systemd[1]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.c.service: Main process exited, code=exited, status=139/n/a 2023-11-27T15:27:46.444 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:27:46 smithi177 systemd[1]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.c.service: Failed with result 'exit-code'. 2023-11-27T15:27:47.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:46 smithi133 ceph-mon[97860]: pgmap v494: 129 pgs: 129 active+clean; 365 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 706 op/s 2023-11-27T15:27:47.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:46 smithi133 ceph-mon[97860]: scrub summary: idle 2023-11-27T15:27:47.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:46 smithi177 ceph-mon[131504]: pgmap v494: 129 pgs: 129 active+clean; 365 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 706 op/s 2023-11-27T15:27:47.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:46 smithi177 ceph-mon[131504]: scrub summary: idle 2023-11-27T15:27:47.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:46 smithi174 ceph-mon[131110]: pgmap v494: 129 pgs: 129 active+clean; 365 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 706 op/s 2023-11-27T15:27:47.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:46 smithi174 ceph-mon[131110]: scrub summary: idle 2023-11-27T15:27:47.407 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:27:48.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.053+0000 7f400d9ff700 1 -- 172.21.15.133:0/2941591034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4008074b30 msgr2=0x7f4008072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:48.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.053+0000 7f400d9ff700 1 --2- 172.21.15.133:0/2941591034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4008074b30 0x7f4008072f80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009990 tx=0x7f3ff8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.053+0000 7f400d9ff700 1 -- 172.21.15.133:0/2941591034 shutdown_connections 2023-11-27T15:27:48.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.053+0000 7f400d9ff700 1 --2- 172.21.15.133:0/2941591034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f400810c890 0x7f400810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.053+0000 7f400d9ff700 1 --2- 172.21.15.133:0/2941591034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4008074b30 0x7f4008072f80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.053+0000 7f400d9ff700 1 --2- 172.21.15.133:0/2941591034 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4008073550 0x7f40080739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.053+0000 7f400d9ff700 1 -- 172.21.15.133:0/2941591034 >> 172.21.15.133:0/2941591034 conn(0x7f40080fb8a0 msgr2=0x7f40080fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:48.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.053+0000 7f400d9ff700 1 -- 172.21.15.133:0/2941591034 shutdown_connections 2023-11-27T15:27:48.058 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.053+0000 7f400d9ff700 1 -- 172.21.15.133:0/2941591034 wait complete. 2023-11-27T15:27:48.060 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.054+0000 7f400d9ff700 1 Processor -- start 2023-11-27T15:27:48.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.054+0000 7f400d9ff700 1 -- start start 2023-11-27T15:27:48.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.055+0000 7f400d9ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4008073550 0x7f400819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.055+0000 7f400d9ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4008074b30 0x7f400819cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.055+0000 7f400d9ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f400810c890 0x7f40081a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.055+0000 7f400d9ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40081a4220 con 0x7f4008074b30 2023-11-27T15:27:48.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.055+0000 7f400d9ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40081a4360 con 0x7f4008073550 2023-11-27T15:27:48.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.055+0000 7f400d9ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40081a44f0 con 0x7f400810c890 2023-11-27T15:27:48.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.055+0000 7f4006ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4008073550 0x7f400819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:48.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.055+0000 7f4006ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4008073550 0x7f400819c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:46752/0 (socket says 172.21.15.133:46752) 2023-11-27T15:27:48.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.055+0000 7f4006ffd700 1 -- 172.21.15.133:0/3746875727 learned_addr learned my addr 172.21.15.133:0/3746875727 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:48.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.055+0000 7f40077fe700 1 --2- 172.21.15.133:0/3746875727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f400810c890 0x7f40081a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:48.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.055+0000 7f40067fc700 1 --2- 172.21.15.133:0/3746875727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4008074b30 0x7f400819cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:48.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.055+0000 7f4006ffd700 1 -- 172.21.15.133:0/3746875727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f400810c890 msgr2=0x7f40081a3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:48.061 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.055+0000 7f4006ffd700 1 --2- 172.21.15.133:0/3746875727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f400810c890 0x7f40081a3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.055+0000 7f4006ffd700 1 -- 172.21.15.133:0/3746875727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4008074b30 msgr2=0x7f400819cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:48.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.055+0000 7f4006ffd700 1 --2- 172.21.15.133:0/3746875727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4008074b30 0x7f400819cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.055+0000 7f4006ffd700 1 -- 172.21.15.133:0/3746875727 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ffc009520 con 0x7f4008073550 2023-11-27T15:27:48.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.055+0000 7f40067fc700 1 --2- 172.21.15.133:0/3746875727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4008074b30 0x7f400819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:48.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.056+0000 7f4006ffd700 1 --2- 172.21.15.133:0/3746875727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4008073550 0x7f400819c6a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800b680 tx=0x7f3ff8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:48.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.056+0000 7f400c9fd700 1 -- 172.21.15.133:0/3746875727 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff801d070 con 0x7f4008073550 2023-11-27T15:27:48.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.056+0000 7f400c9fd700 1 -- 172.21.15.133:0/3746875727 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ff8021410 con 0x7f4008073550 2023-11-27T15:27:48.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.056+0000 7f400d9ff700 1 -- 172.21.15.133:0/3746875727 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff80095f0 con 0x7f4008073550 2023-11-27T15:27:48.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.056+0000 7f400c9fd700 1 -- 172.21.15.133:0/3746875727 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8017650 con 0x7f4008073550 2023-11-27T15:27:48.062 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.056+0000 7f400d9ff700 1 -- 172.21.15.133:0/3746875727 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4008112280 con 0x7f4008073550 2023-11-27T15:27:48.063 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.059+0000 7f400c9fd700 1 -- 172.21.15.133:0/3746875727 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f3ff8021580 con 0x7f4008073550 2023-11-27T15:27:48.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.061+0000 7f4021304b80 1 Processor -- start 2023-11-27T15:27:48.065 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.061+0000 7f4021304b80 1 -- start start 2023-11-27T15:27:48.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.061+0000 7f4021304b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648bc00 0x55e08648c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.061+0000 7f4021304b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648c550 0x55e08648cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.061+0000 7f4021304b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648d110 0x55e08648d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.066 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.061+0000 7f4021304b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e086391060 con 0x55e08648c550 2023-11-27T15:27:48.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.061+0000 7f4021304b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0863e38a0 con 0x55e08648d110 2023-11-27T15:27:48.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.061+0000 7f4021304b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0863dcd30 con 0x55e08648bc00 2023-11-27T15:27:48.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.061+0000 7f4006ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648c550 0x55e08648cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:48.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.061+0000 7f40077fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648bc00 0x55e08648c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:48.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.061+0000 7f400c9fd700 1 --2- 172.21.15.133:0/3746875727 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3ff00a1150 0x7f3ff00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.062+0000 7f40077fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648bc00 0x55e08648c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48364/0 (socket says 172.21.15.133:48364) 2023-11-27T15:27:48.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.062+0000 7f40077fe700 1 -- 172.21.15.133:0/1377172168 learned_addr learned my addr 172.21.15.133:0/1377172168 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:48.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.062+0000 7f40067fc700 1 --2- 172.21.15.133:0/1377172168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648d110 0x55e08648d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:48.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.062+0000 7f40077fe700 1 --2- 172.21.15.133:0/3746875727 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3ff00a1150 0x7f3ff00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:48.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.062+0000 7f400c9fd700 1 -- 172.21.15.133:0/3746875727 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f3ff80ce210 con 0x7f4008073550 2023-11-27T15:27:48.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.062+0000 7f40077fe700 1 -- 172.21.15.133:0/1377172168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648d110 msgr2=0x55e08648d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:48.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.062+0000 7f40077fe700 1 --2- 172.21.15.133:0/1377172168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648d110 0x55e08648d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.067 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.062+0000 7f40077fe700 1 -- 172.21.15.133:0/1377172168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648c550 msgr2=0x55e08648cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:48.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.062+0000 7f40077fe700 1 --2- 172.21.15.133:0/1377172168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648c550 0x55e08648cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.062+0000 7f40077fe700 1 -- 172.21.15.133:0/1377172168 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e08625e630 con 0x55e08648bc00 2023-11-27T15:27:48.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.062+0000 7f40077fe700 1 --2- 172.21.15.133:0/1377172168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648bc00 0x55e08648c010 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f40000040c0 tx=0x7f40000043d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f7bcb7872e8e2df server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:48.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.063+0000 7f40077fe700 1 --2- 172.21.15.133:0/3746875727 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3ff00a1150 0x7f3ff00a3600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4000004720 tx=0x7f400001d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:48.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.063+0000 7f3fedffb700 1 -- 172.21.15.133:0/1377172168 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4000009a60 con 0x55e08648bc00 2023-11-27T15:27:48.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.063+0000 7f3fedffb700 1 -- 172.21.15.133:0/1377172168 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f400000f040 con 0x55e08648bc00 2023-11-27T15:27:48.069 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.063+0000 7f3fedffb700 1 -- 172.21.15.133:0/1377172168 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400001ac70 con 0x55e08648bc00 2023-11-27T15:27:48.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.064+0000 7f4021304b80 1 -- 172.21.15.133:0/1377172168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648bc00 msgr2=0x55e08648c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:48.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.064+0000 7f4021304b80 1 --2- 172.21.15.133:0/1377172168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648bc00 0x55e08648c010 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f40000040c0 tx=0x7f40000043d0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.064+0000 7f4021304b80 1 -- 172.21.15.133:0/1377172168 shutdown_connections 2023-11-27T15:27:48.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.064+0000 7f4021304b80 1 --2- 172.21.15.133:0/1377172168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648d110 0x55e08648d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.064+0000 7f4021304b80 1 --2- 172.21.15.133:0/1377172168 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648bc00 0x55e08648c010 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.064+0000 7f4021304b80 1 --2- 172.21.15.133:0/1377172168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648c550 0x55e08648cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.064+0000 7f4021304b80 1 -- 172.21.15.133:0/1377172168 >> 172.21.15.133:0/1377172168 conn(0x55e086481360 msgr2=0x55e086486670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:48.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.064+0000 7f4021304b80 1 -- 172.21.15.133:0/1377172168 shutdown_connections 2023-11-27T15:27:48.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.064+0000 7f4021304b80 1 -- 172.21.15.133:0/1377172168 wait complete. 2023-11-27T15:27:48.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.065+0000 7f4021304b80 1 -- start start 2023-11-27T15:27:48.070 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.065+0000 7f4021304b80 1 Processor -- start 2023-11-27T15:27:48.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.065+0000 7f4021304b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648bc00 0x55e086495560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.065+0000 7f4021304b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648c550 0x55e086495aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.065+0000 7f4021304b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648d110 0x55e08649cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.065+0000 7f4021304b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e08637e8e0 con 0x55e08648d110 2023-11-27T15:27:48.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.065+0000 7f4021304b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e08637e580 con 0x55e08648bc00 2023-11-27T15:27:48.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.065+0000 7f4021304b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e086260920 con 0x55e08648c550 2023-11-27T15:27:48.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.065+0000 7f4021304b80 10 client.0 _ll_register_callbacks cb 0x55e08639c780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:48.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.066+0000 7f4006ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648bc00 0x55e086495560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:48.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.066+0000 7f4006ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648bc00 0x55e086495560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:46762/0 (socket says 172.21.15.133:46762) 2023-11-27T15:27:48.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.066+0000 7f4006ffd700 1 -- 172.21.15.133:0/2975537452 learned_addr learned my addr 172.21.15.133:0/2975537452 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:48.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.066+0000 7f40067fc700 1 --2- 172.21.15.133:0/2975537452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648c550 0x55e086495aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:48.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.066+0000 7f40077fe700 1 --2- 172.21.15.133:0/2975537452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648d110 0x55e08649cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:48.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.066+0000 7f4006ffd700 1 -- 172.21.15.133:0/2975537452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648c550 msgr2=0x55e086495aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:48.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.066+0000 7f4006ffd700 1 --2- 172.21.15.133:0/2975537452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648c550 0x55e086495aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.066+0000 7f4006ffd700 1 -- 172.21.15.133:0/2975537452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648d110 msgr2=0x55e08649cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:48.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.066+0000 7f4006ffd700 1 --2- 172.21.15.133:0/2975537452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648d110 0x55e08649cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.066+0000 7f4006ffd700 1 -- 172.21.15.133:0/2975537452 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4000009c00 con 0x55e08648bc00 2023-11-27T15:27:48.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.066+0000 7f4006ffd700 1 --2- 172.21.15.133:0/2975537452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648bc00 0x55e086495560 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8000c00 tx=0x7f3ff8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a7cbe1f1d5980d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:48.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.066+0000 7f3fe6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:27:48.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.066+0000 7f3fe6ffd700 1 -- 172.21.15.133:0/2975537452 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff809ac60 con 0x55e08648bc00 2023-11-27T15:27:48.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.066+0000 7f3fe6ffd700 1 -- 172.21.15.133:0/2975537452 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f3ff809b300 con 0x55e08648bc00 2023-11-27T15:27:48.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.066+0000 7f4021304b80 1 -- 172.21.15.133:0/2975537452 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e08647eb20 con 0x55e08648bc00 2023-11-27T15:27:48.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.067+0000 7f3fe6ffd700 1 -- 172.21.15.133:0/2975537452 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ff809f140 con 0x55e08648bc00 2023-11-27T15:27:48.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.067+0000 7f4021304b80 10 client.26330 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:48.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.067+0000 7f4021304b80 1 -- 172.21.15.133:0/2975537452 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e0863a3190 con 0x55e08648bc00 2023-11-27T15:27:48.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.067+0000 7f3fe6ffd700 1 -- 172.21.15.133:0/2975537452 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e08647eb20 con 0x55e08648bc00 2023-11-27T15:27:48.074 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.067+0000 7f3fe6ffd700 1 -- 172.21.15.133:0/2975537452 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f3ff809a880 con 0x55e08648bc00 2023-11-27T15:27:48.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.067+0000 7f4021304b80 10 client.26330 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:48.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.067+0000 7f4021304b80 10 client.26330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:48.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.067+0000 7f4021304b80 20 client.26330 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:48.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.067+0000 7f4021304b80 1 --2- 172.21.15.133:0/2975537452 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55e0864a1000 0x55e0864a3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.067+0000 7f4021304b80 4 client.26330 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:48.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.067+0000 7f4021304b80 1 -- 172.21.15.133:0/2975537452 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e08639f0a0 con 0x55e0864a1000 2023-11-27T15:27:48.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.068+0000 7f40067fc700 1 --2- 172.21.15.133:0/2975537452 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55e0864a1000 0x55e0864a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:48.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.068+0000 7f3fe6ffd700 10 client.26330 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:48.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.068+0000 7f40067fc700 1 --2- 172.21.15.133:0/2975537452 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55e0864a1000 0x55e0864a3410 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9161ff78ac10e300 server_cookie=ec7ba70d223db577 in_seq=0 out_seq=0 2023-11-27T15:27:48.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.071+0000 7f3fe6ffd700 1 -- 172.21.15.133:0/2975537452 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e08639f0a0 con 0x55e0864a1000 2023-11-27T15:27:48.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.071+0000 7f3fe6ffd700 10 client.26330 handle_command_reply: tid=0 2023-11-27T15:27:48.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.071+0000 7f4021304b80 1 client.26330 shutdown 2023-11-27T15:27:48.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.071+0000 7f4021304b80 1 -- 172.21.15.133:0/2975537452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648bc00 msgr2=0x55e086495560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:48.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.071+0000 7f4021304b80 1 --2- 172.21.15.133:0/2975537452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648bc00 0x55e086495560 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8000c00 tx=0x7f3ff8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.071+0000 7f4021304b80 1 -- 172.21.15.133:0/2975537452 shutdown_connections 2023-11-27T15:27:48.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.071+0000 7f4021304b80 1 --2- 172.21.15.133:0/2975537452 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55e0864a1000 0x55e0864a3410 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.071+0000 7f4021304b80 1 --2- 172.21.15.133:0/2975537452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648d110 0x55e08649cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.071+0000 7f4021304b80 1 --2- 172.21.15.133:0/2975537452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648bc00 0x55e086495560 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.071+0000 7f4021304b80 1 --2- 172.21.15.133:0/2975537452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648c550 0x55e086495aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.071+0000 7f3fe6ffd700 0 client.26330 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:48.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.071+0000 7f4021304b80 1 -- 172.21.15.133:0/2975537452 >> 172.21.15.133:0/2975537452 conn(0x55e086481360 msgr2=0x55e086485de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:48.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.071+0000 7f4021304b80 1 -- 172.21.15.133:0/2975537452 shutdown_connections 2023-11-27T15:27:48.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.071+0000 7f4021304b80 1 -- 172.21.15.133:0/2975537452 wait complete. 2023-11-27T15:27:48.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.072+0000 7f4021304b80 20 client.26330 trim_cache size 0 max 16384 2023-11-27T15:27:48.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.091+0000 7f4021304b80 1 Processor -- start 2023-11-27T15:27:48.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.092+0000 7f4021304b80 1 -- start start 2023-11-27T15:27:48.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.092+0000 7f4021304b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648bc00 0x55e08648ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.092+0000 7f4021304b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648c550 0x55e08649caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.092+0000 7f4021304b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648d110 0x55e0864904c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.092+0000 7f4021304b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e08648da00 con 0x55e08648bc00 2023-11-27T15:27:48.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.092+0000 7f4021304b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e086494a10 con 0x55e08648c550 2023-11-27T15:27:48.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.092+0000 7f4021304b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e086496590 con 0x55e08648d110 2023-11-27T15:27:48.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.092+0000 7f40077fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648d110 0x55e0864904c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:48.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.092+0000 7f40077fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648d110 0x55e0864904c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48378/0 (socket says 172.21.15.133:48378) 2023-11-27T15:27:48.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.092+0000 7f40077fe700 1 -- 172.21.15.133:0/2264702652 learned_addr learned my addr 172.21.15.133:0/2264702652 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:48.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.092+0000 7f40067fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648c550 0x55e08649caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:48.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.092+0000 7f4006ffd700 1 --2- 172.21.15.133:0/2264702652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648bc00 0x55e08648ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:48.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.093+0000 7f40077fe700 1 -- 172.21.15.133:0/2264702652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648c550 msgr2=0x55e08649caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:48.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.093+0000 7f40077fe700 1 --2- 172.21.15.133:0/2264702652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648c550 0x55e08649caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.098 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.093+0000 7f40077fe700 1 -- 172.21.15.133:0/2264702652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648bc00 msgr2=0x55e08648ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:48.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.093+0000 7f40077fe700 1 --2- 172.21.15.133:0/2264702652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648bc00 0x55e08648ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.093+0000 7f40077fe700 1 -- 172.21.15.133:0/2264702652 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0863a2ee0 con 0x55e08648d110 2023-11-27T15:27:48.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.093+0000 7f40077fe700 1 --2- 172.21.15.133:0/2264702652 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648d110 0x55e0864904c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f40000040c0 tx=0x7f40000095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4361b9ad7c44c05e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:48.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.093+0000 7f3fee7fc700 1 -- 172.21.15.133:0/2264702652 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4000004c60 con 0x55e08648d110 2023-11-27T15:27:48.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.093+0000 7f3fee7fc700 1 -- 172.21.15.133:0/2264702652 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4000004dc0 con 0x55e08648d110 2023-11-27T15:27:48.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.093+0000 7f3fee7fc700 1 -- 172.21.15.133:0/2264702652 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4000023680 con 0x55e08648d110 2023-11-27T15:27:48.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.094+0000 7f4021304b80 1 -- 172.21.15.133:0/2264702652 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648d110 msgr2=0x55e0864904c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:48.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.094+0000 7f4021304b80 1 --2- 172.21.15.133:0/2264702652 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648d110 0x55e0864904c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f40000040c0 tx=0x7f40000095e0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.094+0000 7f4021304b80 1 -- 172.21.15.133:0/2264702652 shutdown_connections 2023-11-27T15:27:48.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.094+0000 7f4021304b80 1 --2- 172.21.15.133:0/2264702652 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648d110 0x55e0864904c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.094+0000 7f4021304b80 1 --2- 172.21.15.133:0/2264702652 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648bc00 0x55e08648ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.094+0000 7f4021304b80 1 --2- 172.21.15.133:0/2264702652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648c550 0x55e08649caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.094+0000 7f4021304b80 1 -- 172.21.15.133:0/2264702652 >> 172.21.15.133:0/2264702652 conn(0x55e086481360 msgr2=0x55e0864a7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:48.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.094+0000 7f4021304b80 1 -- 172.21.15.133:0/2264702652 shutdown_connections 2023-11-27T15:27:48.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.095+0000 7f4021304b80 1 -- 172.21.15.133:0/2264702652 wait complete. 2023-11-27T15:27:48.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.095+0000 7f4021304b80 1 -- start start 2023-11-27T15:27:48.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.095+0000 7f4021304b80 1 Processor -- start 2023-11-27T15:27:48.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.096+0000 7f4021304b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648bc00 0x55e086498a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.096+0000 7f4021304b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648c550 0x55e086498fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.096+0000 7f4021304b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648d110 0x55e08649bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.096+0000 7f4021304b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e08649c500 con 0x55e08648bc00 2023-11-27T15:27:48.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.096+0000 7f4021304b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e08649c670 con 0x55e08648d110 2023-11-27T15:27:48.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.096+0000 7f4021304b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e08649c830 con 0x55e08648c550 2023-11-27T15:27:48.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.096+0000 7f4021304b80 10 client.0 _ll_register_callbacks cb 0x55e08639c780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:48.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.096+0000 7f4006ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648bc00 0x55e086498a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:48.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.096+0000 7f4006ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648bc00 0x55e086498a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57066/0 (socket says 172.21.15.133:57066) 2023-11-27T15:27:48.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.096+0000 7f4006ffd700 1 -- 172.21.15.133:0/3809542829 learned_addr learned my addr 172.21.15.133:0/3809542829 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:48.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.096+0000 7f40067fc700 1 --2- 172.21.15.133:0/3809542829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648c550 0x55e086498fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:48.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.096+0000 7f40077fe700 1 --2- 172.21.15.133:0/3809542829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648d110 0x55e08649bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:48.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.096+0000 7f40067fc700 1 -- 172.21.15.133:0/3809542829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648d110 msgr2=0x55e08649bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:48.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.096+0000 7f40067fc700 1 --2- 172.21.15.133:0/3809542829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648d110 0x55e08649bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.096+0000 7f40067fc700 1 -- 172.21.15.133:0/3809542829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648bc00 msgr2=0x55e086498a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:48.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.096+0000 7f40067fc700 1 --2- 172.21.15.133:0/3809542829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648bc00 0x55e086498a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.096+0000 7f40067fc700 1 -- 172.21.15.133:0/3809542829 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e08625e630 con 0x55e08648c550 2023-11-27T15:27:48.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.097+0000 7f4006ffd700 1 --2- 172.21.15.133:0/3809542829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648bc00 0x55e086498a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:27:48.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.097+0000 7f40077fe700 1 --2- 172.21.15.133:0/3809542829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648d110 0x55e08649bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:48.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.097+0000 7f40067fc700 1 --2- 172.21.15.133:0/3809542829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648c550 0x55e086498fa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55e0864a13d0 tx=0x7f3ffc01e500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa4a7f9158d3106d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:48.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.097+0000 7f3fe6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:27:48.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.097+0000 7f3fe6ffd700 1 -- 172.21.15.133:0/3809542829 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc017040 con 0x55e08648c550 2023-11-27T15:27:48.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.097+0000 7f3fe6ffd700 1 -- 172.21.15.133:0/3809542829 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f3ffc004220 con 0x55e08648c550 2023-11-27T15:27:48.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.097+0000 7f4021304b80 1 -- 172.21.15.133:0/3809542829 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f400000f040 con 0x55e08648c550 2023-11-27T15:27:48.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.097+0000 7f3fe6ffd700 1 -- 172.21.15.133:0/3809542829 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ffc0043e0 con 0x55e08648c550 2023-11-27T15:27:48.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.097+0000 7f4021304b80 10 client.26320 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:48.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.097+0000 7f4021304b80 1 -- 172.21.15.133:0/3809542829 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e0861f3c60 con 0x55e08648c550 2023-11-27T15:27:48.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.098+0000 7f3fe6ffd700 1 -- 172.21.15.133:0/3809542829 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f400000f040 con 0x55e08648c550 2023-11-27T15:27:48.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.098+0000 7f3fe6ffd700 1 -- 172.21.15.133:0/3809542829 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f3ffc005400 con 0x55e08648c550 2023-11-27T15:27:48.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.098+0000 7f4021304b80 10 client.26320 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:48.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.098+0000 7f4021304b80 10 client.26320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:48.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.098+0000 7f4021304b80 20 client.26320 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:48.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.098+0000 7f4021304b80 1 --2- 172.21.15.133:0/3809542829 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55e0864f6060 0x55e0864f8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.098+0000 7f4021304b80 4 client.26320 mds_command: new command op to 24560 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:27:48.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.098+0000 7f4021304b80 1 -- 172.21.15.133:0/3809542829 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e0863ab660 con 0x55e0864f6060 2023-11-27T15:27:48.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.098+0000 7f4006ffd700 1 --2- 172.21.15.133:0/3809542829 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55e0864f6060 0x55e0864f8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:48.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.099+0000 7f4006ffd700 1 --2- 172.21.15.133:0/3809542829 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55e0864f6060 0x55e0864f8470 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51ff23dcc31bcf97 server_cookie=59faa21ac3680fb3 in_seq=0 out_seq=0 2023-11-27T15:27:48.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.099+0000 7f3fe6ffd700 10 client.26320 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:48.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.102+0000 7f3fe6ffd700 1 -- 172.21.15.133:0/3809542829 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e0863ab660 con 0x55e0864f6060 2023-11-27T15:27:48.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.102+0000 7f3fe6ffd700 10 client.26320 handle_command_reply: tid=0 2023-11-27T15:27:48.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.102+0000 7f4021304b80 1 client.26320 shutdown 2023-11-27T15:27:48.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.103+0000 7f4021304b80 1 -- 172.21.15.133:0/3809542829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648c550 msgr2=0x55e086498fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:48.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.103+0000 7f4021304b80 1 --2- 172.21.15.133:0/3809542829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648c550 0x55e086498fa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55e0864a13d0 tx=0x7f3ffc01e500 comp rx=0 tx=0).stop 2023-11-27T15:27:48.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.103+0000 7f4021304b80 1 -- 172.21.15.133:0/3809542829 shutdown_connections 2023-11-27T15:27:48.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.103+0000 7f4021304b80 1 --2- 172.21.15.133:0/3809542829 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55e0864f6060 0x55e0864f8470 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.103+0000 7f4021304b80 1 --2- 172.21.15.133:0/3809542829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08648d110 0x55e08649bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.103+0000 7f4021304b80 1 --2- 172.21.15.133:0/3809542829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e08648bc00 0x55e086498a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.103+0000 7f4021304b80 1 --2- 172.21.15.133:0/3809542829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e08648c550 0x55e086498fa0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.103+0000 7f4021304b80 1 -- 172.21.15.133:0/3809542829 >> 172.21.15.133:0/3809542829 conn(0x55e086481360 msgr2=0x55e08648ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:48.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.103+0000 7f3fe6ffd700 0 client.26320 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:48.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.103+0000 7f4021304b80 1 -- 172.21.15.133:0/3809542829 shutdown_connections 2023-11-27T15:27:48.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.103+0000 7f4021304b80 1 -- 172.21.15.133:0/3809542829 wait complete. 2023-11-27T15:27:48.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.103+0000 7f4021304b80 20 client.26320 trim_cache size 0 max 16384 2023-11-27T15:27:48.109 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:27:48.109 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2023-11-27T15:27:48.109 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2023-11-27T15:27:48.109 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:27:48.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.104+0000 7f400d9ff700 1 -- 172.21.15.133:0/3746875727 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3ff00a1150 msgr2=0x7f3ff00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:48.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.104+0000 7f400d9ff700 1 --2- 172.21.15.133:0/3746875727 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3ff00a1150 0x7f3ff00a3600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4000004720 tx=0x7f400001d000 comp rx=0 tx=0).stop 2023-11-27T15:27:48.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.104+0000 7f400d9ff700 1 -- 172.21.15.133:0/3746875727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4008073550 msgr2=0x7f400819c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:48.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.104+0000 7f400d9ff700 1 --2- 172.21.15.133:0/3746875727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4008073550 0x7f400819c6a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800b680 tx=0x7f3ff8005740 comp rx=0 tx=0).stop 2023-11-27T15:27:48.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.104+0000 7f400d9ff700 1 -- 172.21.15.133:0/3746875727 shutdown_connections 2023-11-27T15:27:48.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.104+0000 7f400d9ff700 1 --2- 172.21.15.133:0/3746875727 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f3ff00a1150 0x7f3ff00a3600 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.104+0000 7f400d9ff700 1 --2- 172.21.15.133:0/3746875727 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f400810c890 0x7f40081a3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.104+0000 7f400d9ff700 1 --2- 172.21.15.133:0/3746875727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4008073550 0x7f400819c6a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.104+0000 7f400d9ff700 1 --2- 172.21.15.133:0/3746875727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4008074b30 0x7f400819cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.104+0000 7f400d9ff700 1 -- 172.21.15.133:0/3746875727 >> 172.21.15.133:0/3746875727 conn(0x7f40080fb8a0 msgr2=0x7f40080fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:48.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.104+0000 7f400d9ff700 1 -- 172.21.15.133:0/3746875727 shutdown_connections 2023-11-27T15:27:48.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.104+0000 7f400d9ff700 1 -- 172.21.15.133:0/3746875727 wait complete. 2023-11-27T15:27:48.129 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:27:48.129 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:27:48.202 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:47 smithi133 ceph-mon[97860]: scrub summary: idle 2023-11-27T15:27:48.202 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:47 smithi133 ceph-mon[97860]: scrub summary: idle 2023-11-27T15:27:48.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:47 smithi177 ceph-mon[131504]: scrub summary: idle 2023-11-27T15:27:48.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:47 smithi177 ceph-mon[131504]: scrub summary: idle 2023-11-27T15:27:48.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:47 smithi174 ceph-mon[131110]: scrub summary: idle 2023-11-27T15:27:48.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:47 smithi174 ceph-mon[131110]: scrub summary: idle 2023-11-27T15:27:48.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.461+0000 7f78e9fff700 1 -- 172.21.15.133:0/2104902635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78e4071ac0 msgr2=0x7f78e4071ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:48.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.461+0000 7f78e9fff700 1 --2- 172.21.15.133:0/2104902635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78e4071ac0 0x7f78e4071ed0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f78dc00b0a0 tx=0x7f78dc00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.461+0000 7f78e9fff700 1 -- 172.21.15.133:0/2104902635 shutdown_connections 2023-11-27T15:27:48.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.461+0000 7f78e9fff700 1 --2- 172.21.15.133:0/2104902635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78e41084a0 0x7f78e4117a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.461+0000 7f78e9fff700 1 --2- 172.21.15.133:0/2104902635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78e4071ac0 0x7f78e4071ed0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.461+0000 7f78e9fff700 1 --2- 172.21.15.133:0/2104902635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78e4107af0 0x7f78e4107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.466 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.461+0000 7f78e9fff700 1 -- 172.21.15.133:0/2104902635 >> 172.21.15.133:0/2104902635 conn(0x7f78e406d1c0 msgr2=0x7f78e406f610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:48.468 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.463+0000 7f78e9fff700 1 -- 172.21.15.133:0/2104902635 shutdown_connections 2023-11-27T15:27:48.469 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.464+0000 7f78e9fff700 1 -- 172.21.15.133:0/2104902635 wait complete. 2023-11-27T15:27:48.471 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.467+0000 7f78e9fff700 1 Processor -- start 2023-11-27T15:27:48.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.468+0000 7f78e9fff700 1 -- start start 2023-11-27T15:27:48.472 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.468+0000 7f78e9fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78e4071ac0 0x7f78e41a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.468+0000 7f78e9fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78e4107af0 0x7f78e41a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.469+0000 7f78e9fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78e41084a0 0x7f78e41ac520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.469+0000 7f78e9fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e41acaf0 con 0x7f78e4071ac0 2023-11-27T15:27:48.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.469+0000 7f78e9fff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e41acc30 con 0x7f78e41084a0 2023-11-27T15:27:48.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.469+0000 7f78e9fff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e41acdc0 con 0x7f78e4107af0 2023-11-27T15:27:48.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.470+0000 7f78e37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78e4071ac0 0x7f78e41a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:48.474 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.470+0000 7f78e3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78e41084a0 0x7f78e41ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:48.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.470+0000 7f78e37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78e4071ac0 0x7f78e41a4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57080/0 (socket says 172.21.15.133:57080) 2023-11-27T15:27:48.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.470+0000 7f78e37fe700 1 -- 172.21.15.133:0/584639243 learned_addr learned my addr 172.21.15.133:0/584639243 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:48.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.471+0000 7f78e3fff700 1 -- 172.21.15.133:0/584639243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78e4107af0 msgr2=0x7f78e41a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:48.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.471+0000 7f78e3fff700 1 --2- 172.21.15.133:0/584639243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78e4107af0 0x7f78e41a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.475 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.471+0000 7f78e3fff700 1 -- 172.21.15.133:0/584639243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78e4071ac0 msgr2=0x7f78e41a4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:48.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.471+0000 7f78e3fff700 1 --2- 172.21.15.133:0/584639243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78e4071ac0 0x7f78e41a4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:48.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.471+0000 7f78e3fff700 1 -- 172.21.15.133:0/584639243 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78dc009d00 con 0x7f78e41084a0 2023-11-27T15:27:48.476 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.471+0000 7f78e37fe700 1 --2- 172.21.15.133:0/584639243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78e4071ac0 0x7f78e41a4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:27:48.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.471+0000 7f78e3fff700 1 --2- 172.21.15.133:0/584639243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78e41084a0 0x7f78e41ac520 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f78d800b3f0 tx=0x7f78d800b700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:48.477 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.472+0000 7f78e0ff9700 1 -- 172.21.15.133:0/584639243 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d8011830 con 0x7f78e41084a0 2023-11-27T15:27:48.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.472+0000 7f78e0ff9700 1 -- 172.21.15.133:0/584639243 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78d8011e50 con 0x7f78e41084a0 2023-11-27T15:27:48.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.472+0000 7f78e9fff700 1 -- 172.21.15.133:0/584639243 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e411aab0 con 0x7f78e41084a0 2023-11-27T15:27:48.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.472+0000 7f78e9fff700 1 -- 172.21.15.133:0/584639243 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e411a920 con 0x7f78e41084a0 2023-11-27T15:27:48.478 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.472+0000 7f78e0ff9700 1 -- 172.21.15.133:0/584639243 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d8010db0 con 0x7f78e41084a0 2023-11-27T15:27:48.480 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.476+0000 7f78e9fff700 1 -- 172.21.15.133:0/584639243 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d0005310 con 0x7f78e41084a0 2023-11-27T15:27:48.487 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.480+0000 7f78e0ff9700 1 -- 172.21.15.133:0/584639243 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f78d8011990 con 0x7f78e41084a0 2023-11-27T15:27:48.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.485+0000 7f78e0ff9700 1 --2- 172.21.15.133:0/584639243 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f78cc0a1320 0x7f78cc0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:48.489 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.485+0000 7f78e0ff9700 1 -- 172.21.15.133:0/584639243 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f78d80cc400 con 0x7f78e41084a0 2023-11-27T15:27:48.490 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.485+0000 7f78e37fe700 1 --2- 172.21.15.133:0/584639243 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f78cc0a1320 0x7f78cc0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:48.491 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.486+0000 7f78e0ff9700 1 -- 172.21.15.133:0/584639243 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f78d80957a0 con 0x7f78e41084a0 2023-11-27T15:27:48.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:48.488+0000 7f78e37fe700 1 --2- 172.21.15.133:0/584639243 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f78cc0a1320 0x7f78cc0a37d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f78dc007a10 tx=0x7f78dc006210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:49.048 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.044+0000 7f78e9fff700 1 -- 172.21.15.133:0/584639243 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78d0005f60 con 0x7f78e41084a0 2023-11-27T15:27:49.049 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.045+0000 7f78e0ff9700 1 -- 172.21.15.133:0/584639243 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10838 (secure 0 0 0) 0x7f78d8098dc0 con 0x7f78e41084a0 2023-11-27T15:27:49.049 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:27:49.049 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14493,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2381290259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2381290259},{"type":"v1","addr":"172.21.15.133:6839","nonce":2381290259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1704875895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1704875895},{"type":"v1","addr":"172.21.15.133:6841","nonce":1704875895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:20:33.201023+0000","modified":"2023-11-27T15:26:38.737228+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24521,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/3492215689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":3492215689},{"type":"v1","addr":"172.21.15.177:6837","nonce":3492215689}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6839/2781994447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2781994447},{"type":"v1","addr":"172.21.15.174:6839","nonce":2781994447}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:27:49.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.051+0000 7f78e9fff700 1 -- 172.21.15.133:0/584639243 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f78cc0a1320 msgr2=0x7f78cc0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:49.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.051+0000 7f78e9fff700 1 --2- 172.21.15.133:0/584639243 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f78cc0a1320 0x7f78cc0a37d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f78dc007a10 tx=0x7f78dc006210 comp rx=0 tx=0).stop 2023-11-27T15:27:49.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.051+0000 7f78e9fff700 1 -- 172.21.15.133:0/584639243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78e41084a0 msgr2=0x7f78e41ac520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:49.054 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.051+0000 7f78e9fff700 1 --2- 172.21.15.133:0/584639243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78e41084a0 0x7f78e41ac520 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f78d800b3f0 tx=0x7f78d800b700 comp rx=0 tx=0).stop 2023-11-27T15:27:49.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.051+0000 7f78e9fff700 1 -- 172.21.15.133:0/584639243 shutdown_connections 2023-11-27T15:27:49.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.051+0000 7f78e9fff700 1 --2- 172.21.15.133:0/584639243 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f78cc0a1320 0x7f78cc0a37d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.051+0000 7f78e9fff700 1 --2- 172.21.15.133:0/584639243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78e41084a0 0x7f78e41ac520 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.051+0000 7f78e9fff700 1 --2- 172.21.15.133:0/584639243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78e4071ac0 0x7f78e41a4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.051+0000 7f78e9fff700 1 --2- 172.21.15.133:0/584639243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f78e4107af0 0x7f78e41a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.055 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.051+0000 7f78e9fff700 1 -- 172.21.15.133:0/584639243 >> 172.21.15.133:0/584639243 conn(0x7f78e406d1c0 msgr2=0x7f78e406ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:49.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.052+0000 7f78e9fff700 1 -- 172.21.15.133:0/584639243 shutdown_connections 2023-11-27T15:27:49.056 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.052+0000 7f78e9fff700 1 -- 172.21.15.133:0/584639243 wait complete. 2023-11-27T15:27:49.061 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 27 2023-11-27T15:27:49.102 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:27:49.220 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:48 smithi133 ceph-mon[97860]: pgmap v495: 129 pgs: 129 active+clean; 362 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 657 op/s 2023-11-27T15:27:49.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:48 smithi177 ceph-mon[131504]: pgmap v495: 129 pgs: 129 active+clean; 362 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 657 op/s 2023-11-27T15:27:49.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:48 smithi174 ceph-mon[131110]: pgmap v495: 129 pgs: 129 active+clean; 362 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.6 MiB/s wr, 657 op/s 2023-11-27T15:27:49.557 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.554+0000 7ff8bbfff700 1 -- 172.21.15.133:0/3086507856 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c000bb10 con 0x7ff8c4071d60 2023-11-27T15:27:49.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.555+0000 7ff8ca19d700 1 -- 172.21.15.133:0/3086507856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8c4071d60 msgr2=0x7ff8c4072170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:49.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.555+0000 7ff8ca19d700 1 --2- 172.21.15.133:0/3086507856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8c4071d60 0x7ff8c4072170 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0009990 tx=0x7ff8c0009ca0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.559 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.556+0000 7ff8ca19d700 1 -- 172.21.15.133:0/3086507856 shutdown_connections 2023-11-27T15:27:49.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.556+0000 7ff8ca19d700 1 --2- 172.21.15.133:0/3086507856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8c4108770 0x7ff8c410f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.556+0000 7ff8ca19d700 1 --2- 172.21.15.133:0/3086507856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8c4071d60 0x7ff8c4072170 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.556+0000 7ff8ca19d700 1 --2- 172.21.15.133:0/3086507856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c4107dc0 0x7ff8c4108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.560 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.556+0000 7ff8ca19d700 1 -- 172.21.15.133:0/3086507856 >> 172.21.15.133:0/3086507856 conn(0x7ff8c406d0e0 msgr2=0x7ff8c406f550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:49.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.556+0000 7ff8ca19d700 1 -- 172.21.15.133:0/3086507856 shutdown_connections 2023-11-27T15:27:49.561 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.557+0000 7ff8ca19d700 1 -- 172.21.15.133:0/3086507856 wait complete. 2023-11-27T15:27:49.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.558+0000 7ff8ca19d700 1 Processor -- start 2023-11-27T15:27:49.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.560+0000 7ff8ca19d700 1 -- start start 2023-11-27T15:27:49.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.561+0000 7ff8ca19d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8c4071d60 0x7ff8c419e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:49.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.561+0000 7ff8ca19d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c4107dc0 0x7ff8c419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:49.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.561+0000 7ff8ca19d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8c4108770 0x7ff8c41a7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:49.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.561+0000 7ff8ca19d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c41a8460 con 0x7ff8c4107dc0 2023-11-27T15:27:49.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.561+0000 7ff8ca19d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c41a85a0 con 0x7ff8c4108770 2023-11-27T15:27:49.565 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.561+0000 7ff8c999c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8c4108770 0x7ff8c41a7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:49.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.561+0000 7ff8c999c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8c4108770 0x7ff8c41a7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:46824/0 (socket says 172.21.15.133:46824) 2023-11-27T15:27:49.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.561+0000 7ff8c999c700 1 -- 172.21.15.133:0/1553844556 learned_addr learned my addr 172.21.15.133:0/1553844556 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:49.566 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.562+0000 7ff8ca19d700 1 -- 172.21.15.133:0/1553844556 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c41a86e0 con 0x7ff8c4071d60 2023-11-27T15:27:49.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.562+0000 7ff8c919b700 1 --2- 172.21.15.133:0/1553844556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8c4071d60 0x7ff8c419e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:49.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.563+0000 7ff8c899a700 1 --2- 172.21.15.133:0/1553844556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c4107dc0 0x7ff8c419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:49.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.564+0000 7ff8c999c700 1 -- 172.21.15.133:0/1553844556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8c4071d60 msgr2=0x7ff8c419e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:49.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.564+0000 7ff8c999c700 1 --2- 172.21.15.133:0/1553844556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8c4071d60 0x7ff8c419e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.564+0000 7ff8c999c700 1 -- 172.21.15.133:0/1553844556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c4107dc0 msgr2=0x7ff8c419ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:49.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.564+0000 7ff8c999c700 1 --2- 172.21.15.133:0/1553844556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c4107dc0 0x7ff8c419ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.564+0000 7ff8c999c700 1 -- 172.21.15.133:0/1553844556 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c00095f0 con 0x7ff8c4108770 2023-11-27T15:27:49.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.564+0000 7ff8c899a700 1 --2- 172.21.15.133:0/1553844556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c4107dc0 0x7ff8c419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:49.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.564+0000 7ff8c999c700 1 --2- 172.21.15.133:0/1553844556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8c4108770 0x7ff8c41a7f20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00ef60 tx=0x7ff8bc00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:49.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.564+0000 7ff8ba7fc700 1 -- 172.21.15.133:0/1553844556 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc018070 con 0x7ff8c4108770 2023-11-27T15:27:49.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.565+0000 7ff8ca19d700 1 -- 172.21.15.133:0/1553844556 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c4114270 con 0x7ff8c4108770 2023-11-27T15:27:49.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.565+0000 7ff8ca19d700 1 -- 172.21.15.133:0/1553844556 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c41140e0 con 0x7ff8c4108770 2023-11-27T15:27:49.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.565+0000 7ff8ba7fc700 1 -- 172.21.15.133:0/1553844556 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8bc00f040 con 0x7ff8c4108770 2023-11-27T15:27:49.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.565+0000 7ff8ba7fc700 1 -- 172.21.15.133:0/1553844556 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc003e30 con 0x7ff8c4108770 2023-11-27T15:27:49.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.576+0000 7ff8ded00b80 1 Processor -- start 2023-11-27T15:27:49.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.580+0000 7ff8ded00b80 1 -- start start 2023-11-27T15:27:49.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.581+0000 7ff8ded00b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02d6a0 0x55ea7b02dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:49.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.581+0000 7ff8ded00b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea7b02dff0 0x55ea7b02e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:49.589 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.581+0000 7ff8ded00b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02eac0 0x55ea7b02ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:49.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.581+0000 7ff8ded00b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7af2a060 con 0x55ea7b02dff0 2023-11-27T15:27:49.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.581+0000 7ff8ded00b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7af7c8a0 con 0x55ea7b02eac0 2023-11-27T15:27:49.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.581+0000 7ff8ded00b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7af75d30 con 0x55ea7b02d6a0 2023-11-27T15:27:49.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.581+0000 7ff8c899a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02d6a0 0x55ea7b02dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:49.590 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.581+0000 7ff8c899a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02d6a0 0x55ea7b02dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48444/0 (socket says 172.21.15.133:48444) 2023-11-27T15:27:49.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.581+0000 7ff8c899a700 1 -- 172.21.15.133:0/3497662063 learned_addr learned my addr 172.21.15.133:0/3497662063 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:49.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.581+0000 7ff8c899a700 1 --2- 172.21.15.133:0/3497662063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02eac0 0x55ea7b02ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:49.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.582+0000 7ff8c919b700 1 --2- 172.21.15.133:0/3497662063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea7b02dff0 0x55ea7b02e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:49.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.582+0000 7ff8ba7fc700 1 -- 172.21.15.133:0/1553844556 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff8bc00ca90 con 0x7ff8c4108770 2023-11-27T15:27:49.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.582+0000 7ff8c899a700 1 -- 172.21.15.133:0/3497662063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02eac0 msgr2=0x55ea7b02ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:49.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.582+0000 7ff8c899a700 1 --2- 172.21.15.133:0/3497662063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02eac0 0x55ea7b02ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.591 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.582+0000 7ff8c899a700 1 -- 172.21.15.133:0/3497662063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea7b02dff0 msgr2=0x55ea7b02e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:49.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.582+0000 7ff8c899a700 1 --2- 172.21.15.133:0/3497662063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea7b02dff0 0x55ea7b02e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.583+0000 7ff8c899a700 1 -- 172.21.15.133:0/3497662063 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea7adf7630 con 0x55ea7b02d6a0 2023-11-27T15:27:49.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.585+0000 7ff8c899a700 1 --2- 172.21.15.133:0/3497662063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02d6a0 0x55ea7b02dab0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400a250 tx=0x7ff8b400a450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff734ee2d8b6cec8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:49.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.585+0000 7ff8af7fe700 1 -- 172.21.15.133:0/3497662063 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4012e30 con 0x55ea7b02d6a0 2023-11-27T15:27:49.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.585+0000 7ff8af7fe700 1 -- 172.21.15.133:0/3497662063 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8b4004cf0 con 0x55ea7b02d6a0 2023-11-27T15:27:49.592 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.585+0000 7ff8af7fe700 1 -- 172.21.15.133:0/3497662063 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b40105e0 con 0x55ea7b02d6a0 2023-11-27T15:27:49.593 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.586+0000 7ff8ba7fc700 1 --2- 172.21.15.133:0/1553844556 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff8b00a1250 0x7ff8b00a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:49.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.586+0000 7ff8ba7fc700 1 -- 172.21.15.133:0/1553844556 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ff8bc0cc7b0 con 0x7ff8c4108770 2023-11-27T15:27:49.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.587+0000 7ff8ded00b80 1 -- 172.21.15.133:0/3497662063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02d6a0 msgr2=0x55ea7b02dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:49.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.587+0000 7ff8ded00b80 1 --2- 172.21.15.133:0/3497662063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02d6a0 0x55ea7b02dab0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400a250 tx=0x7ff8b400a450 comp rx=0 tx=0).stop 2023-11-27T15:27:49.594 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.587+0000 7ff8ded00b80 1 -- 172.21.15.133:0/3497662063 shutdown_connections 2023-11-27T15:27:49.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.587+0000 7ff8ded00b80 1 --2- 172.21.15.133:0/3497662063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02eac0 0x55ea7b02ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.587+0000 7ff8ded00b80 1 --2- 172.21.15.133:0/3497662063 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02d6a0 0x55ea7b02dab0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.587+0000 7ff8ded00b80 1 --2- 172.21.15.133:0/3497662063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea7b02dff0 0x55ea7b02e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.587+0000 7ff8ded00b80 1 -- 172.21.15.133:0/3497662063 >> 172.21.15.133:0/3497662063 conn(0x55ea7b01a5b0 msgr2=0x55ea7b01f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:49.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.588+0000 7ff8ded00b80 1 -- 172.21.15.133:0/3497662063 shutdown_connections 2023-11-27T15:27:49.595 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.588+0000 7ff8c919b700 1 --2- 172.21.15.133:0/1553844556 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff8b00a1250 0x7ff8b00a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:49.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.588+0000 7ff8ded00b80 1 -- 172.21.15.133:0/3497662063 wait complete. 2023-11-27T15:27:49.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.589+0000 7ff8c919b700 1 --2- 172.21.15.133:0/1553844556 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff8b00a1250 0x7ff8b00a3700 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0006010 tx=0x7ff8c000b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:49.596 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.590+0000 7ff8ded00b80 1 -- start start 2023-11-27T15:27:49.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.592+0000 7ff8ded00b80 1 Processor -- start 2023-11-27T15:27:49.597 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.592+0000 7ff8ded00b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea7b02d6a0 0x55ea7b02c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:49.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.592+0000 7ff8ded00b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02dff0 0x55ea7b02ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:49.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.592+0000 7ff8ded00b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02eac0 0x55ea7b035d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:49.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.593+0000 7ff8ded00b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7af178e0 con 0x55ea7b02d6a0 2023-11-27T15:27:49.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.593+0000 7ff8ded00b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7af17580 con 0x55ea7b02dff0 2023-11-27T15:27:49.598 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.593+0000 7ff8ded00b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7adf9920 con 0x55ea7b02eac0 2023-11-27T15:27:49.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.593+0000 7ff8c999c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02eac0 0x55ea7b035d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:49.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.593+0000 7ff8c899a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02dff0 0x55ea7b02ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:49.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.593+0000 7ff8c999c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02eac0 0x55ea7b035d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48450/0 (socket says 172.21.15.133:48450) 2023-11-27T15:27:49.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.593+0000 7ff8c999c700 1 -- 172.21.15.133:0/2096991840 learned_addr learned my addr 172.21.15.133:0/2096991840 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:49.599 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.593+0000 7ff8ded00b80 10 client.0 _ll_register_callbacks cb 0x55ea7af35780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:49.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.594+0000 7ff8c999c700 1 -- 172.21.15.133:0/2096991840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02dff0 msgr2=0x55ea7b02ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:49.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.594+0000 7ff8c999c700 1 --2- 172.21.15.133:0/2096991840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02dff0 0x55ea7b02ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.600 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.594+0000 7ff8c999c700 1 -- 172.21.15.133:0/2096991840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea7b02d6a0 msgr2=0x55ea7b02c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:49.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.594+0000 7ff8c999c700 1 --2- 172.21.15.133:0/2096991840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea7b02d6a0 0x55ea7b02c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.594+0000 7ff8c999c700 1 -- 172.21.15.133:0/2096991840 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b40123f0 con 0x55ea7b02eac0 2023-11-27T15:27:49.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.595+0000 7ff8c999c700 1 --2- 172.21.15.133:0/2096991840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02eac0 0x55ea7b035d70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc095da0 tx=0x7ff8bc004db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=893a04ea5b4bc397 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:49.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.595+0000 7ff8c899a700 1 --2- 172.21.15.133:0/2096991840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02dff0 0x55ea7b02ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:49.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.595+0000 7ff8acff9700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:27:49.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.595+0000 7ff8acff9700 1 -- 172.21.15.133:0/2096991840 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc0a1c60 con 0x55ea7b02eac0 2023-11-27T15:27:49.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.595+0000 7ff8acff9700 1 -- 172.21.15.133:0/2096991840 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ff8bc0b7e40 con 0x55ea7b02eac0 2023-11-27T15:27:49.601 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.595+0000 7ff8ded00b80 1 -- 172.21.15.133:0/2096991840 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea7b017d70 con 0x55ea7b02eac0 2023-11-27T15:27:49.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.596+0000 7ff8acff9700 1 -- 172.21.15.133:0/2096991840 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8bc00cd40 con 0x55ea7b02eac0 2023-11-27T15:27:49.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.596+0000 7ff8ded00b80 10 client.26335 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:49.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.596+0000 7ff8ded00b80 1 -- 172.21.15.133:0/2096991840 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ea7af3c190 con 0x55ea7b02eac0 2023-11-27T15:27:49.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.596+0000 7ff8acff9700 1 -- 172.21.15.133:0/2096991840 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ea7b017d70 con 0x55ea7b02eac0 2023-11-27T15:27:49.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.597+0000 7ff8acff9700 1 -- 172.21.15.133:0/2096991840 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7ff8bc0a17d0 con 0x55ea7b02eac0 2023-11-27T15:27:49.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.597+0000 7ff8ded00b80 10 client.26335 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:49.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.597+0000 7ff8ded00b80 10 client.26335 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-11-27T15:27:49.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.597+0000 7ff8ded00b80 20 client.26335 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:49.602 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.597+0000 7ff8ded00b80 1 --2- 172.21.15.133:0/2096991840 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55ea7b03a5e0 0x55ea7b03c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:49.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.597+0000 7ff8ded00b80 4 client.26335 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:49.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.597+0000 7ff8ded00b80 1 -- 172.21.15.133:0/2096991840 --> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea7af380a0 con 0x55ea7b03a5e0 2023-11-27T15:27:49.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.597+0000 7ff8c919b700 1 --2- 172.21.15.133:0/2096991840 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55ea7b03a5e0 0x55ea7b03c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:49.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.598+0000 7ff8c919b700 1 --2- 172.21.15.133:0/2096991840 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55ea7b03a5e0 0x55ea7b03c9f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46d5513d223d01e3 server_cookie=ccea0325fb44bdaa in_seq=0 out_seq=0 2023-11-27T15:27:49.603 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.598+0000 7ff8acff9700 10 client.26335 ms_handle_connect on v2:172.21.15.177:6838/1463687598 2023-11-27T15:27:49.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.600+0000 7ff8acff9700 1 -- 172.21.15.133:0/2096991840 <== mds.2 v2:172.21.15.177:6838/1463687598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ea7af380a0 con 0x55ea7b03a5e0 2023-11-27T15:27:49.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.600+0000 7ff8acff9700 10 client.26335 handle_command_reply: tid=0 2023-11-27T15:27:49.604 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.600+0000 7ff8ded00b80 1 client.26335 shutdown 2023-11-27T15:27:49.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.601+0000 7ff8ded00b80 1 -- 172.21.15.133:0/2096991840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02eac0 msgr2=0x55ea7b035d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:49.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.601+0000 7ff8ded00b80 1 --2- 172.21.15.133:0/2096991840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02eac0 0x55ea7b035d70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc095da0 tx=0x7ff8bc004db0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.601+0000 7ff8ded00b80 1 -- 172.21.15.133:0/2096991840 shutdown_connections 2023-11-27T15:27:49.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.601+0000 7ff8ded00b80 1 --2- 172.21.15.133:0/2096991840 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55ea7b03a5e0 0x55ea7b03c9f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.601+0000 7ff8ded00b80 1 --2- 172.21.15.133:0/2096991840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02eac0 0x55ea7b035d70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.601+0000 7ff8ded00b80 1 --2- 172.21.15.133:0/2096991840 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea7b02d6a0 0x55ea7b02c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.601+0000 7ff8ded00b80 1 --2- 172.21.15.133:0/2096991840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02dff0 0x55ea7b02ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.605 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.602+0000 7ff8ded00b80 1 -- 172.21.15.133:0/2096991840 >> 172.21.15.133:0/2096991840 conn(0x55ea7b01a5b0 msgr2=0x55ea7b023bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:49.607 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.603+0000 7ff8acff9700 0 client.26335 ms_handle_reset on v2:172.21.15.177:6838/1463687598 2023-11-27T15:27:49.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.604+0000 7ff8ded00b80 1 -- 172.21.15.133:0/2096991840 shutdown_connections 2023-11-27T15:27:49.608 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.604+0000 7ff8ded00b80 1 -- 172.21.15.133:0/2096991840 wait complete. 2023-11-27T15:27:49.614 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.610+0000 7ff8ded00b80 20 client.26335 trim_cache size 0 max 16384 2023-11-27T15:27:49.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.642+0000 7ff8ded00b80 1 Processor -- start 2023-11-27T15:27:49.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.642+0000 7ff8ded00b80 1 -- start start 2023-11-27T15:27:49.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.642+0000 7ff8ded00b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02d6a0 0x55ea7b01f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:49.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.642+0000 7ff8ded00b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea7b02dff0 0x55ea7b035d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:49.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.642+0000 7ff8ded00b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02eac0 0x55ea7b036690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:49.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.642+0000 7ff8ded00b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7b037a10 con 0x55ea7b02dff0 2023-11-27T15:27:49.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.642+0000 7ff8ded00b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7b027f90 con 0x55ea7b02d6a0 2023-11-27T15:27:49.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.642+0000 7ff8ded00b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7b021080 con 0x55ea7b02eac0 2023-11-27T15:27:49.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.644+0000 7ff8c999c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02eac0 0x55ea7b036690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:49.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.644+0000 7ff8c999c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02eac0 0x55ea7b036690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48462/0 (socket says 172.21.15.133:48462) 2023-11-27T15:27:49.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.644+0000 7ff8c999c700 1 -- 172.21.15.133:0/2893739258 learned_addr learned my addr 172.21.15.133:0/2893739258 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:49.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.644+0000 7ff8c919b700 1 --2- 172.21.15.133:0/2893739258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02d6a0 0x55ea7b01f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:49.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.645+0000 7ff8c999c700 1 -- 172.21.15.133:0/2893739258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02d6a0 msgr2=0x55ea7b01f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:49.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.645+0000 7ff8c999c700 1 --2- 172.21.15.133:0/2893739258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02d6a0 0x55ea7b01f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.645+0000 7ff8c999c700 1 -- 172.21.15.133:0/2893739258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea7b02dff0 msgr2=0x55ea7b035d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:49.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.647+0000 7ff8c899a700 1 --2- 172.21.15.133:0/2893739258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea7b02dff0 0x55ea7b035d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:49.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.647+0000 7ff8c999c700 1 --2- 172.21.15.133:0/2893739258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea7b02dff0 0x55ea7b035d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.647+0000 7ff8c999c700 1 -- 172.21.15.133:0/2893739258 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea7af3bee0 con 0x55ea7b02eac0 2023-11-27T15:27:49.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.648+0000 7ff8c999c700 1 --2- 172.21.15.133:0/2893739258 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02eac0 0x55ea7b036690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc097bf0 tx=0x7ff8bc097e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1269044d0fb73ca server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:49.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.649+0000 7ff8affff700 1 -- 172.21.15.133:0/2893739258 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc09f0b0 con 0x55ea7b02eac0 2023-11-27T15:27:49.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.649+0000 7ff8affff700 1 -- 172.21.15.133:0/2893739258 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8bc0b7410 con 0x55ea7b02eac0 2023-11-27T15:27:49.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.649+0000 7ff8affff700 1 -- 172.21.15.133:0/2893739258 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc004080 con 0x55ea7b02eac0 2023-11-27T15:27:49.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.650+0000 7ff8ded00b80 1 -- 172.21.15.133:0/2893739258 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02eac0 msgr2=0x55ea7b036690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:49.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.650+0000 7ff8ded00b80 1 --2- 172.21.15.133:0/2893739258 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02eac0 0x55ea7b036690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc097bf0 tx=0x7ff8bc097e50 comp rx=0 tx=0).stop 2023-11-27T15:27:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.651+0000 7ff8ded00b80 1 -- 172.21.15.133:0/2893739258 shutdown_connections 2023-11-27T15:27:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.651+0000 7ff8ded00b80 1 --2- 172.21.15.133:0/2893739258 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02eac0 0x55ea7b036690 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.651+0000 7ff8ded00b80 1 --2- 172.21.15.133:0/2893739258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02d6a0 0x55ea7b01f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.654 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.651+0000 7ff8ded00b80 1 --2- 172.21.15.133:0/2893739258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea7b02dff0 0x55ea7b035d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.655 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.651+0000 7ff8ded00b80 1 -- 172.21.15.133:0/2893739258 >> 172.21.15.133:0/2893739258 conn(0x55ea7b01a5b0 msgr2=0x55ea7b041350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:49.655 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.651+0000 7ff8ded00b80 1 -- 172.21.15.133:0/2893739258 shutdown_connections 2023-11-27T15:27:49.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.652+0000 7ff8ded00b80 1 -- 172.21.15.133:0/2893739258 wait complete. 2023-11-27T15:27:49.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.652+0000 7ff8ded00b80 1 -- start start 2023-11-27T15:27:49.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.654+0000 7ff8ded00b80 1 Processor -- start 2023-11-27T15:27:49.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.655+0000 7ff8ded00b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea7b02d6a0 0x55ea7b033090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:49.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.655+0000 7ff8ded00b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02dff0 0x55ea7b0335d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:49.658 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.655+0000 7ff8ded00b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02eac0 0x55ea7b0345e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:49.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.655+0000 7ff8c899a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02dff0 0x55ea7b0335d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:49.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.655+0000 7ff8c899a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02dff0 0x55ea7b0335d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48478/0 (socket says 172.21.15.133:48478) 2023-11-27T15:27:49.659 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.656+0000 7ff8c899a700 1 -- 172.21.15.133:0/1789827041 learned_addr learned my addr 172.21.15.133:0/1789827041 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:49.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.656+0000 7ff8ded00b80 1 -- 172.21.15.133:0/1789827041 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7b034b20 con 0x55ea7b02d6a0 2023-11-27T15:27:49.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.656+0000 7ff8ded00b80 1 -- 172.21.15.133:0/1789827041 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7b034c90 con 0x55ea7b02eac0 2023-11-27T15:27:49.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.656+0000 7ff8c919b700 1 --2- 172.21.15.133:0/1789827041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea7b02d6a0 0x55ea7b033090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:49.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.656+0000 7ff8ded00b80 1 -- 172.21.15.133:0/1789827041 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7b034e50 con 0x55ea7b02dff0 2023-11-27T15:27:49.660 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.656+0000 7ff8c999c700 1 --2- 172.21.15.133:0/1789827041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02eac0 0x55ea7b0345e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:49.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.657+0000 7ff8c899a700 1 -- 172.21.15.133:0/1789827041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02eac0 msgr2=0x55ea7b0345e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:49.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.657+0000 7ff8c899a700 1 --2- 172.21.15.133:0/1789827041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02eac0 0x55ea7b0345e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.657+0000 7ff8ded00b80 10 client.0 _ll_register_callbacks cb 0x55ea7af35780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:49.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.657+0000 7ff8c899a700 1 -- 172.21.15.133:0/1789827041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea7b02d6a0 msgr2=0x55ea7b033090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:49.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.657+0000 7ff8c899a700 1 --2- 172.21.15.133:0/1789827041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea7b02d6a0 0x55ea7b033090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.657+0000 7ff8c899a700 1 -- 172.21.15.133:0/1789827041 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea7af3c190 con 0x55ea7b02dff0 2023-11-27T15:27:49.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.657+0000 7ff8ded00b80 1 -- 172.21.15.133:0/1789827041 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8b4004cf0 con 0x55ea7b02dff0 2023-11-27T15:27:49.661 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.657+0000 7ff8c999c700 1 --2- 172.21.15.133:0/1789827041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02eac0 0x55ea7b0345e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:49.662 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.658+0000 7ff8c899a700 1 --2- 172.21.15.133:0/1789827041 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02dff0 0x55ea7b0335d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400ace0 tx=0x7ff8b4004920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=461515fd0c5a209e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:49.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.661+0000 7ff8acff9700 10 client.26341 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:27:49.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.661+0000 7ff8acff9700 1 -- 172.21.15.133:0/1789827041 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4045070 con 0x55ea7b02dff0 2023-11-27T15:27:49.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.661+0000 7ff8acff9700 1 -- 172.21.15.133:0/1789827041 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ff8b4004e60 con 0x55ea7b02dff0 2023-11-27T15:27:49.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.661+0000 7ff8acff9700 1 -- 172.21.15.133:0/1789827041 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8b4005080 con 0x55ea7b02dff0 2023-11-27T15:27:49.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.662+0000 7ff8ded00b80 10 client.26341 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:49.665 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.662+0000 7ff8ded00b80 1 -- 172.21.15.133:0/1789827041 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x55ea7ad8cc60 con 0x55ea7b02dff0 2023-11-27T15:27:49.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.663+0000 7ff8acff9700 1 -- 172.21.15.133:0/1789827041 <== mon.1 v2:172.21.15.177:3300/0 4 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7ff8b40051e0 con 0x55ea7b02dff0 2023-11-27T15:27:49.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.664+0000 7ff8ded00b80 10 client.26341 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:49.667 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.664+0000 7ff8ded00b80 10 client.26341 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-11-27T15:27:49.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.664+0000 7ff8ded00b80 20 client.26341 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:49.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.664+0000 7ff8ded00b80 1 --2- 172.21.15.133:0/1789827041 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55ea7b08e6a0 0x55ea7b090ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:49.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.664+0000 7ff8ded00b80 4 client.26341 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:27:49.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.664+0000 7ff8ded00b80 1 -- 172.21.15.133:0/1789827041 --> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ea7af44660 con 0x55ea7b08e6a0 2023-11-27T15:27:49.668 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.665+0000 7ff8c919b700 1 --2- 172.21.15.133:0/1789827041 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55ea7b08e6a0 0x55ea7b090ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:49.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.666+0000 7ff8c919b700 1 --2- 172.21.15.133:0/1789827041 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55ea7b08e6a0 0x55ea7b090ab0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=98d3940af1ecd47c server_cookie=ab102115b39a4d5f in_seq=0 out_seq=0 2023-11-27T15:27:49.669 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.666+0000 7ff8acff9700 10 client.26341 ms_handle_connect on v2:172.21.15.177:6838/1463687598 2023-11-27T15:27:49.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.677+0000 7ff8acff9700 1 -- 172.21.15.133:0/1789827041 <== mds.2 v2:172.21.15.177:6838/1463687598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ea7af44660 con 0x55ea7b08e6a0 2023-11-27T15:27:49.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.677+0000 7ff8acff9700 10 client.26341 handle_command_reply: tid=0 2023-11-27T15:27:49.680 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.677+0000 7ff8ded00b80 1 client.26341 shutdown 2023-11-27T15:27:49.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.678+0000 7ff8ded00b80 1 -- 172.21.15.133:0/1789827041 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02dff0 msgr2=0x55ea7b0335d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:49.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.678+0000 7ff8ded00b80 1 --2- 172.21.15.133:0/1789827041 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02dff0 0x55ea7b0335d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400ace0 tx=0x7ff8b4004920 comp rx=0 tx=0).stop 2023-11-27T15:27:49.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.678+0000 7ff8ded00b80 1 -- 172.21.15.133:0/1789827041 shutdown_connections 2023-11-27T15:27:49.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.678+0000 7ff8ded00b80 1 --2- 172.21.15.133:0/1789827041 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55ea7b08e6a0 0x55ea7b090ab0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.678+0000 7ff8ded00b80 1 --2- 172.21.15.133:0/1789827041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea7b02eac0 0x55ea7b0345e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.678+0000 7ff8ded00b80 1 --2- 172.21.15.133:0/1789827041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ea7b02d6a0 0x55ea7b033090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.678+0000 7ff8ded00b80 1 --2- 172.21.15.133:0/1789827041 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55ea7b02dff0 0x55ea7b0335d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.678+0000 7ff8ded00b80 1 -- 172.21.15.133:0/1789827041 >> 172.21.15.133:0/1789827041 conn(0x55ea7b01a5b0 msgr2=0x55ea7b01f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:49.682 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.678+0000 7ff8acff9700 0 client.26341 ms_handle_reset on v2:172.21.15.177:6838/1463687598 2023-11-27T15:27:49.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.679+0000 7ff8ded00b80 1 -- 172.21.15.133:0/1789827041 shutdown_connections 2023-11-27T15:27:49.683 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.680+0000 7ff8ded00b80 1 -- 172.21.15.133:0/1789827041 wait complete. 2023-11-27T15:27:49.684 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.680+0000 7ff8ded00b80 20 client.26341 trim_cache size 0 max 16384 2023-11-27T15:27:49.684 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2023-11-27T15:27:49.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.681+0000 7ff8ca19d700 1 -- 172.21.15.133:0/1553844556 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff8b00a1250 msgr2=0x7ff8b00a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:49.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.681+0000 7ff8ca19d700 1 --2- 172.21.15.133:0/1553844556 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff8b00a1250 0x7ff8b00a3700 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0006010 tx=0x7ff8c000b260 comp rx=0 tx=0).stop 2023-11-27T15:27:49.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.681+0000 7ff8ca19d700 1 -- 172.21.15.133:0/1553844556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8c4108770 msgr2=0x7ff8c41a7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:49.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.681+0000 7ff8ca19d700 1 --2- 172.21.15.133:0/1553844556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8c4108770 0x7ff8c41a7f20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00ef60 tx=0x7ff8bc00ef90 comp rx=0 tx=0).stop 2023-11-27T15:27:49.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.681+0000 7ff8ca19d700 1 -- 172.21.15.133:0/1553844556 shutdown_connections 2023-11-27T15:27:49.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.681+0000 7ff8ca19d700 1 --2- 172.21.15.133:0/1553844556 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff8b00a1250 0x7ff8b00a3700 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.685 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.681+0000 7ff8ca19d700 1 --2- 172.21.15.133:0/1553844556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8c4108770 0x7ff8c41a7f20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.681+0000 7ff8ca19d700 1 --2- 172.21.15.133:0/1553844556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff8c4071d60 0x7ff8c419e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.681+0000 7ff8ca19d700 1 --2- 172.21.15.133:0/1553844556 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c4107dc0 0x7ff8c419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:49.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.681+0000 7ff8ca19d700 1 -- 172.21.15.133:0/1553844556 >> 172.21.15.133:0/1553844556 conn(0x7ff8c406d0e0 msgr2=0x7ff8c4070290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:49.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.682+0000 7ff8ca19d700 1 -- 172.21.15.133:0/1553844556 shutdown_connections 2023-11-27T15:27:49.686 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:49.682+0000 7ff8ca19d700 1 -- 172.21.15.133:0/1553844556 wait complete. 2023-11-27T15:27:49.730 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:27:50.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:49 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/584639243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:27:50.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.079+0000 7fc8f25ff700 1 -- 172.21.15.133:0/1634912490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec107d60 msgr2=0x7fc8ec108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:50.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.079+0000 7fc8f25ff700 1 --2- 172.21.15.133:0/1634912490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec107d60 0x7fc8ec108170 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009990 tx=0x7fc8dc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.079+0000 7fc8f25ff700 1 -- 172.21.15.133:0/1634912490 shutdown_connections 2023-11-27T15:27:50.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.079+0000 7fc8f25ff700 1 --2- 172.21.15.133:0/1634912490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8ec072670 0x7fc8ec111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.079+0000 7fc8f25ff700 1 --2- 172.21.15.133:0/1634912490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec107d60 0x7fc8ec108170 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.079+0000 7fc8f25ff700 1 --2- 172.21.15.133:0/1634912490 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8ec071ce0 0x7fc8ec072130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.079+0000 7fc8f25ff700 1 -- 172.21.15.133:0/1634912490 >> 172.21.15.133:0/1634912490 conn(0x7fc8ec06d0d0 msgr2=0x7fc8ec06f520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:50.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.080+0000 7fc8f25ff700 1 -- 172.21.15.133:0/1634912490 shutdown_connections 2023-11-27T15:27:50.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.080+0000 7fc8f25ff700 1 -- 172.21.15.133:0/1634912490 wait complete. 2023-11-27T15:27:50.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.081+0000 7fc8f25ff700 1 Processor -- start 2023-11-27T15:27:50.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.081+0000 7fc8f25ff700 1 -- start start 2023-11-27T15:27:50.086 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.081+0000 7fc8f25ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8ec071ce0 0x7fc8ec10a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.081+0000 7fc8f25ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8ec072670 0x7fc8ec108dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.081+0000 7fc8f25ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec107d60 0x7fc8ec109300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.081+0000 7fc8f25ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec109aa0 con 0x7fc8ec071ce0 2023-11-27T15:27:50.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.081+0000 7fc8f25ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec109c10 con 0x7fc8ec107d60 2023-11-27T15:27:50.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.081+0000 7fc8f25ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec1a88b0 con 0x7fc8ec072670 2023-11-27T15:27:50.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.081+0000 7fc8eb7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8ec072670 0x7fc8ec108dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.081+0000 7fc8f0b9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec107d60 0x7fc8ec109300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.081+0000 7fc8eb7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8ec072670 0x7fc8ec108dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48500/0 (socket says 172.21.15.133:48500) 2023-11-27T15:27:50.087 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.081+0000 7fc8f0b9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec107d60 0x7fc8ec109300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:46876/0 (socket says 172.21.15.133:46876) 2023-11-27T15:27:50.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.081+0000 7fc8eb7fe700 1 -- 172.21.15.133:0/4223426327 learned_addr learned my addr 172.21.15.133:0/4223426327 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:50.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.081+0000 7fc8ebfff700 1 --2- 172.21.15.133:0/4223426327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8ec071ce0 0x7fc8ec10a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.082+0000 7fc8f0b9c700 1 -- 172.21.15.133:0/4223426327 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8ec072670 msgr2=0x7fc8ec108dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:50.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.082+0000 7fc8f0b9c700 1 --2- 172.21.15.133:0/4223426327 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8ec072670 0x7fc8ec108dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.082+0000 7fc8f0b9c700 1 -- 172.21.15.133:0/4223426327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8ec071ce0 msgr2=0x7fc8ec10a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:50.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.082+0000 7fc8f0b9c700 1 --2- 172.21.15.133:0/4223426327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8ec071ce0 0x7fc8ec10a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.082+0000 7fc8f0b9c700 1 -- 172.21.15.133:0/4223426327 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8dc0095f0 con 0x7fc8ec107d60 2023-11-27T15:27:50.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.082+0000 7fc8f0b9c700 1 --2- 172.21.15.133:0/4223426327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec107d60 0x7fc8ec109300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4007f10 tx=0x7fc8e400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:50.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.082+0000 7fc8e97fa700 1 -- 172.21.15.133:0/4223426327 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4004db0 con 0x7fc8ec107d60 2023-11-27T15:27:50.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.083+0000 7fc8e97fa700 1 -- 172.21.15.133:0/4223426327 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8e400f040 con 0x7fc8ec107d60 2023-11-27T15:27:50.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.083+0000 7fc8e97fa700 1 -- 172.21.15.133:0/4223426327 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4005990 con 0x7fc8ec107d60 2023-11-27T15:27:50.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.083+0000 7fc8f25ff700 1 -- 172.21.15.133:0/4223426327 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ec114450 con 0x7fc8ec107d60 2023-11-27T15:27:50.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.083+0000 7fc8f25ff700 1 -- 172.21.15.133:0/4223426327 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ec1142c0 con 0x7fc8ec107d60 2023-11-27T15:27:50.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.087+0000 7fc8e97fa700 1 -- 172.21.15.133:0/4223426327 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fc8e400db20 con 0x7fc8ec107d60 2023-11-27T15:27:50.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.089+0000 7fc8e97fa700 1 --2- 172.21.15.133:0/4223426327 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc8d40a11d0 0x7fc8d40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.089+0000 7fc8e97fa700 1 -- 172.21.15.133:0/4223426327 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc8e40cc500 con 0x7fc8ec107d60 2023-11-27T15:27:50.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.089+0000 7fc905e43b80 1 Processor -- start 2023-11-27T15:27:50.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.090+0000 7fc8ebfff700 1 --2- 172.21.15.133:0/4223426327 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc8d40a11d0 0x7fc8d40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.090+0000 7fc905e43b80 1 -- start start 2023-11-27T15:27:50.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.090+0000 7fc905e43b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6605c00 0x5591f6606010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.090+0000 7fc905e43b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6606550 0x5591f6606b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.090+0000 7fc905e43b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6607110 0x5591f6607580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.090+0000 7fc905e43b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591f650b060 con 0x5591f6607110 2023-11-27T15:27:50.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.090+0000 7fc905e43b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591f655d8a0 con 0x5591f6606550 2023-11-27T15:27:50.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.090+0000 7fc905e43b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591f6556d30 con 0x5591f6605c00 2023-11-27T15:27:50.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.091+0000 7fc8ebfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6605c00 0x5591f6606010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.091+0000 7fc8ebfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6605c00 0x5591f6606010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48508/0 (socket says 172.21.15.133:48508) 2023-11-27T15:27:50.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.091+0000 7fc8ebfff700 1 -- 172.21.15.133:0/2711436697 learned_addr learned my addr 172.21.15.133:0/2711436697 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:50.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.091+0000 7fc8f0b9c700 1 --2- 172.21.15.133:0/2711436697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6607110 0x5591f6607580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.091+0000 7fc8eb7fe700 1 --2- 172.21.15.133:0/2711436697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6606550 0x5591f6606b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.091+0000 7fc8ebfff700 1 --2- 172.21.15.133:0/4223426327 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc8d40a11d0 0x7fc8d40a3680 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc003790 tx=0x7fc8dc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:50.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.091+0000 7fc8ebfff700 1 -- 172.21.15.133:0/2711436697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6606550 msgr2=0x5591f6606b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:50.096 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.091+0000 7fc8ebfff700 1 --2- 172.21.15.133:0/2711436697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6606550 0x5591f6606b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.091+0000 7fc8ebfff700 1 -- 172.21.15.133:0/2711436697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6607110 msgr2=0x5591f6607580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:50.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.091+0000 7fc8ebfff700 1 --2- 172.21.15.133:0/2711436697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6607110 0x5591f6607580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.091+0000 7fc8ebfff700 1 -- 172.21.15.133:0/2711436697 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591f63d8630 con 0x5591f6605c00 2023-11-27T15:27:50.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.091+0000 7fc8f0b9c700 1 --2- 172.21.15.133:0/2711436697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6607110 0x5591f6607580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:50.097 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.092+0000 7fc8ebfff700 1 --2- 172.21.15.133:0/2711436697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6605c00 0x5591f6606010 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc015000 tx=0x7fc8dc028000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70ec42430a352c71 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:50.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.092+0000 7fc8d27fc700 1 -- 172.21.15.133:0/2711436697 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc00b9d0 con 0x5591f6605c00 2023-11-27T15:27:50.099 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.092+0000 7fc8d27fc700 1 -- 172.21.15.133:0/2711436697 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8dc01e040 con 0x5591f6605c00 2023-11-27T15:27:50.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.092+0000 7fc8d27fc700 1 -- 172.21.15.133:0/2711436697 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc004750 con 0x5591f6605c00 2023-11-27T15:27:50.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.093+0000 7fc905e43b80 1 -- 172.21.15.133:0/2711436697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6605c00 msgr2=0x5591f6606010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:50.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.093+0000 7fc905e43b80 1 --2- 172.21.15.133:0/2711436697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6605c00 0x5591f6606010 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc015000 tx=0x7fc8dc028000 comp rx=0 tx=0).stop 2023-11-27T15:27:50.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.093+0000 7fc905e43b80 1 -- 172.21.15.133:0/2711436697 shutdown_connections 2023-11-27T15:27:50.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.093+0000 7fc905e43b80 1 --2- 172.21.15.133:0/2711436697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6607110 0x5591f6607580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.093+0000 7fc905e43b80 1 --2- 172.21.15.133:0/2711436697 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6605c00 0x5591f6606010 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.093+0000 7fc905e43b80 1 --2- 172.21.15.133:0/2711436697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6606550 0x5591f6606b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.093+0000 7fc905e43b80 1 -- 172.21.15.133:0/2711436697 >> 172.21.15.133:0/2711436697 conn(0x5591f65fb360 msgr2=0x5591f6600670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:50.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.093+0000 7fc905e43b80 1 -- 172.21.15.133:0/2711436697 shutdown_connections 2023-11-27T15:27:50.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.093+0000 7fc905e43b80 1 -- 172.21.15.133:0/2711436697 wait complete. 2023-11-27T15:27:50.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.094+0000 7fc905e43b80 1 -- start start 2023-11-27T15:27:50.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.094+0000 7fc905e43b80 1 Processor -- start 2023-11-27T15:27:50.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.095+0000 7fc905e43b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6605c00 0x5591f660f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.095+0000 7fc905e43b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6606550 0x5591f660fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.095+0000 7fc905e43b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6607110 0x5591f6616b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.095+0000 7fc905e43b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591f64f88e0 con 0x5591f6605c00 2023-11-27T15:27:50.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.095+0000 7fc905e43b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591f64f8580 con 0x5591f6607110 2023-11-27T15:27:50.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.095+0000 7fc905e43b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591f63da920 con 0x5591f6606550 2023-11-27T15:27:50.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.095+0000 7fc905e43b80 10 client.0 _ll_register_callbacks cb 0x5591f6516780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:50.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.095+0000 7fc8eb7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6606550 0x5591f660fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.095+0000 7fc8eb7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6606550 0x5591f660fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48512/0 (socket says 172.21.15.133:48512) 2023-11-27T15:27:50.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.095+0000 7fc8eb7fe700 1 -- 172.21.15.133:0/3193648312 learned_addr learned my addr 172.21.15.133:0/3193648312 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:50.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.095+0000 7fc8f0b9c700 1 --2- 172.21.15.133:0/3193648312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6607110 0x5591f6616b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.102 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.095+0000 7fc8ebfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6605c00 0x5591f660f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.095+0000 7fc8eb7fe700 1 -- 172.21.15.133:0/3193648312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6607110 msgr2=0x5591f6616b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:50.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.095+0000 7fc8eb7fe700 1 --2- 172.21.15.133:0/3193648312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6607110 0x5591f6616b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.095+0000 7fc8eb7fe700 1 -- 172.21.15.133:0/3193648312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6605c00 msgr2=0x5591f660f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:50.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.095+0000 7fc8eb7fe700 1 --2- 172.21.15.133:0/3193648312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6605c00 0x5591f660f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.095+0000 7fc8eb7fe700 1 -- 172.21.15.133:0/3193648312 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ec111810 con 0x5591f6606550 2023-11-27T15:27:50.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.095+0000 7fc8f0b9c700 1 --2- 172.21.15.133:0/3193648312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6607110 0x5591f6616b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:50.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.096+0000 7fc8eb7fe700 1 --2- 172.21.15.133:0/3193648312 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6606550 0x5591f660fae0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0012700 tx=0x7fc8e0012ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=765c86610f843cb1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:50.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.096+0000 7fc8cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:27:50.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.096+0000 7fc8cb7fe700 1 -- 172.21.15.133:0/3193648312 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00108a0 con 0x5591f6606550 2023-11-27T15:27:50.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.097+0000 7fc8cb7fe700 1 -- 172.21.15.133:0/3193648312 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc8e0010a00 con 0x5591f6606550 2023-11-27T15:27:50.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.097+0000 7fc905e43b80 1 -- 172.21.15.133:0/3193648312 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591f65f8b20 con 0x5591f6606550 2023-11-27T15:27:50.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.097+0000 7fc8cb7fe700 1 -- 172.21.15.133:0/3193648312 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8e0010bc0 con 0x5591f6606550 2023-11-27T15:27:50.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.097+0000 7fc905e43b80 10 client.26353 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:50.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.097+0000 7fc905e43b80 1 -- 172.21.15.133:0/3193648312 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5591f651d190 con 0x5591f6606550 2023-11-27T15:27:50.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.097+0000 7fc8cb7fe700 1 -- 172.21.15.133:0/3193648312 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5591f65f8b20 con 0x5591f6606550 2023-11-27T15:27:50.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.098+0000 7fc8cb7fe700 1 -- 172.21.15.133:0/3193648312 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fc8e00253d0 con 0x5591f6606550 2023-11-27T15:27:50.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.098+0000 7fc905e43b80 10 client.26353 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:50.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.098+0000 7fc905e43b80 10 client.26353 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:27:50.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.098+0000 7fc905e43b80 20 client.26353 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:50.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.098+0000 7fc905e43b80 1 --2- 172.21.15.133:0/3193648312 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x5591f661b480 0x5591f661d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.098+0000 7fc905e43b80 4 client.26353 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:50.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.098+0000 7fc905e43b80 1 -- 172.21.15.133:0/3193648312 --> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5591f65190a0 con 0x5591f661b480 2023-11-27T15:27:50.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.098+0000 7fc8ebfff700 1 --2- 172.21.15.133:0/3193648312 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x5591f661b480 0x5591f661d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.099+0000 7fc8ebfff700 1 --2- 172.21.15.133:0/3193648312 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x5591f661b480 0x5591f661d890 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b9cc9adc68666503 server_cookie=8dc9c4243bb37b63 in_seq=0 out_seq=0 2023-11-27T15:27:50.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.099+0000 7fc8cb7fe700 10 client.26353 ms_handle_connect on v2:172.21.15.174:6836/1955090548 2023-11-27T15:27:50.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.099+0000 7fc8cb7fe700 1 -- 172.21.15.133:0/3193648312 <== mds.1 v2:172.21.15.174:6836/1955090548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5591f65190a0 con 0x5591f661b480 2023-11-27T15:27:50.107 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.099+0000 7fc8cb7fe700 10 client.26353 handle_command_reply: tid=0 2023-11-27T15:27:50.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.099+0000 7fc905e43b80 1 client.26353 shutdown 2023-11-27T15:27:50.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.100+0000 7fc905e43b80 1 -- 172.21.15.133:0/3193648312 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6606550 msgr2=0x5591f660fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:50.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.100+0000 7fc905e43b80 1 --2- 172.21.15.133:0/3193648312 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6606550 0x5591f660fae0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0012700 tx=0x7fc8e0012ac0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.100+0000 7fc905e43b80 1 -- 172.21.15.133:0/3193648312 shutdown_connections 2023-11-27T15:27:50.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.100+0000 7fc905e43b80 1 --2- 172.21.15.133:0/3193648312 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x5591f661b480 0x5591f661d890 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.100+0000 7fc8cb7fe700 0 client.26353 ms_handle_reset on v2:172.21.15.174:6836/1955090548 2023-11-27T15:27:50.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.100+0000 7fc905e43b80 1 --2- 172.21.15.133:0/3193648312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6607110 0x5591f6616b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.100+0000 7fc905e43b80 1 --2- 172.21.15.133:0/3193648312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6605c00 0x5591f660f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.100+0000 7fc905e43b80 1 --2- 172.21.15.133:0/3193648312 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6606550 0x5591f660fae0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.100+0000 7fc905e43b80 1 -- 172.21.15.133:0/3193648312 >> 172.21.15.133:0/3193648312 conn(0x5591f65fb360 msgr2=0x5591f65ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:50.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.100+0000 7fc905e43b80 1 -- 172.21.15.133:0/3193648312 shutdown_connections 2023-11-27T15:27:50.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.100+0000 7fc905e43b80 1 -- 172.21.15.133:0/3193648312 wait complete. 2023-11-27T15:27:50.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.100+0000 7fc905e43b80 20 client.26353 trim_cache size 0 max 16384 2023-11-27T15:27:50.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.120+0000 7fc905e43b80 1 Processor -- start 2023-11-27T15:27:50.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.120+0000 7fc905e43b80 1 -- start start 2023-11-27T15:27:50.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.120+0000 7fc905e43b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6605c00 0x5591f660a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.120+0000 7fc905e43b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6606550 0x5591f660a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.120+0000 7fc905e43b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6607110 0x5591f6616ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.125 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.120+0000 7fc905e43b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591f660f110 con 0x5591f6606550 2023-11-27T15:27:50.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.120+0000 7fc905e43b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591f6618790 con 0x5591f6607110 2023-11-27T15:27:50.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.120+0000 7fc905e43b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591f66021a0 con 0x5591f6605c00 2023-11-27T15:27:50.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.120+0000 7fc8ebfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6605c00 0x5591f660a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.121+0000 7fc8ebfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6605c00 0x5591f660a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48522/0 (socket says 172.21.15.133:48522) 2023-11-27T15:27:50.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.121+0000 7fc8ebfff700 1 -- 172.21.15.133:0/928366140 learned_addr learned my addr 172.21.15.133:0/928366140 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:50.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.121+0000 7fc8f0b9c700 1 --2- 172.21.15.133:0/928366140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6607110 0x5591f6616ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.121+0000 7fc8ebfff700 1 -- 172.21.15.133:0/928366140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6607110 msgr2=0x5591f6616ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:50.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.121+0000 7fc8ebfff700 1 --2- 172.21.15.133:0/928366140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6607110 0x5591f6616ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.121+0000 7fc8ebfff700 1 -- 172.21.15.133:0/928366140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6606550 msgr2=0x5591f660a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:50.126 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.121+0000 7fc8eb7fe700 1 --2- 172.21.15.133:0/928366140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6606550 0x5591f660a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.121+0000 7fc8ebfff700 1 --2- 172.21.15.133:0/928366140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6606550 0x5591f660a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.121+0000 7fc8ebfff700 1 -- 172.21.15.133:0/928366140 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591f651cee0 con 0x5591f6605c00 2023-11-27T15:27:50.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.121+0000 7fc8eb7fe700 1 --2- 172.21.15.133:0/928366140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6606550 0x5591f660a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:50.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.121+0000 7fc8ebfff700 1 --2- 172.21.15.133:0/928366140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6605c00 0x5591f660a060 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc000c00 tx=0x7fc8dc028000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bade16dba9f3450 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:50.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.122+0000 7fc8d2ffd700 1 -- 172.21.15.133:0/928366140 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc03d040 con 0x5591f6605c00 2023-11-27T15:27:50.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.122+0000 7fc8d2ffd700 1 -- 172.21.15.133:0/928366140 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8dc0405f0 con 0x5591f6605c00 2023-11-27T15:27:50.127 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.122+0000 7fc8d2ffd700 1 -- 172.21.15.133:0/928366140 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc00be90 con 0x5591f6605c00 2023-11-27T15:27:50.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.122+0000 7fc905e43b80 1 -- 172.21.15.133:0/928366140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6605c00 msgr2=0x5591f660a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:50.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.122+0000 7fc905e43b80 1 --2- 172.21.15.133:0/928366140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6605c00 0x5591f660a060 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc000c00 tx=0x7fc8dc028000 comp rx=0 tx=0).stop 2023-11-27T15:27:50.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.123+0000 7fc905e43b80 1 -- 172.21.15.133:0/928366140 shutdown_connections 2023-11-27T15:27:50.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.123+0000 7fc905e43b80 1 --2- 172.21.15.133:0/928366140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6607110 0x5591f6616ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.123+0000 7fc905e43b80 1 --2- 172.21.15.133:0/928366140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6605c00 0x5591f660a060 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.129 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.123+0000 7fc905e43b80 1 --2- 172.21.15.133:0/928366140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6606550 0x5591f660a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.123+0000 7fc905e43b80 1 -- 172.21.15.133:0/928366140 >> 172.21.15.133:0/928366140 conn(0x5591f65fb360 msgr2=0x5591f660d500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:50.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.123+0000 7fc905e43b80 1 -- 172.21.15.133:0/928366140 shutdown_connections 2023-11-27T15:27:50.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.123+0000 7fc905e43b80 1 -- 172.21.15.133:0/928366140 wait complete. 2023-11-27T15:27:50.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.123+0000 7fc905e43b80 1 -- start start 2023-11-27T15:27:50.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.124+0000 7fc905e43b80 1 Processor -- start 2023-11-27T15:27:50.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.124+0000 7fc905e43b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6605c00 0x5591f6614390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.125+0000 7fc905e43b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6606550 0x5591f66148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.125+0000 7fc905e43b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6607110 0x5591f66158e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.130 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.125+0000 7fc905e43b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591f6615e20 con 0x5591f6607110 2023-11-27T15:27:50.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.125+0000 7fc905e43b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591f6615f90 con 0x5591f6606550 2023-11-27T15:27:50.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.125+0000 7fc905e43b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591f6616150 con 0x5591f6605c00 2023-11-27T15:27:50.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.125+0000 7fc905e43b80 10 client.0 _ll_register_callbacks cb 0x5591f6516780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:50.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.125+0000 7fc8f0b9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6607110 0x5591f66158e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.125+0000 7fc8eb7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6606550 0x5591f66148d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.125+0000 7fc8f0b9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6607110 0x5591f66158e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57190/0 (socket says 172.21.15.133:57190) 2023-11-27T15:27:50.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.125+0000 7fc8eb7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6606550 0x5591f66148d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:46922/0 (socket says 172.21.15.133:46922) 2023-11-27T15:27:50.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.125+0000 7fc8eb7fe700 1 -- 172.21.15.133:0/3283956538 learned_addr learned my addr 172.21.15.133:0/3283956538 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:50.131 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.126+0000 7fc8eb7fe700 1 -- 172.21.15.133:0/3283956538 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6605c00 msgr2=0x5591f6614390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-27T15:27:50.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.126+0000 7fc8eb7fe700 1 --2- 172.21.15.133:0/3283956538 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6605c00 0x5591f6614390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.126+0000 7fc8eb7fe700 1 -- 172.21.15.133:0/3283956538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6607110 msgr2=0x5591f66158e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:50.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.126+0000 7fc8eb7fe700 1 --2- 172.21.15.133:0/3283956538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6607110 0x5591f66158e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.132 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.126+0000 7fc8eb7fe700 1 -- 172.21.15.133:0/3283956538 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5591f63d8630 con 0x5591f6606550 2023-11-27T15:27:50.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.126+0000 7fc8eb7fe700 1 --2- 172.21.15.133:0/3283956538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6606550 0x5591f66148d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc8e000bd30 tx=0x7fc8e0005c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4f94cc1c259ead8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:50.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.126+0000 7fc8cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:27:50.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.127+0000 7fc8cb7fe700 1 -- 172.21.15.133:0/3283956538 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0005910 con 0x5591f6606550 2023-11-27T15:27:50.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.127+0000 7fc8cb7fe700 1 -- 172.21.15.133:0/3283956538 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc8e0000e40 con 0x5591f6606550 2023-11-27T15:27:50.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.127+0000 7fc905e43b80 1 -- 172.21.15.133:0/3283956538 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8dc01e040 con 0x5591f6606550 2023-11-27T15:27:50.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.127+0000 7fc8cb7fe700 1 -- 172.21.15.133:0/3283956538 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8e0025bb0 con 0x5591f6606550 2023-11-27T15:27:50.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.127+0000 7fc905e43b80 10 client.26369 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:50.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.127+0000 7fc905e43b80 1 -- 172.21.15.133:0/3283956538 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5591f636dc60 con 0x5591f6606550 2023-11-27T15:27:50.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.127+0000 7fc8cb7fe700 1 -- 172.21.15.133:0/3283956538 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8dc01e040 con 0x5591f6606550 2023-11-27T15:27:50.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.127+0000 7fc8cb7fe700 1 -- 172.21.15.133:0/3283956538 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fc8e002d070 con 0x5591f6606550 2023-11-27T15:27:50.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.128+0000 7fc905e43b80 10 client.26369 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:50.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.128+0000 7fc905e43b80 10 client.26369 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:27:50.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.128+0000 7fc905e43b80 20 client.26369 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:50.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.128+0000 7fc905e43b80 1 --2- 172.21.15.133:0/3283956538 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x5591f666fd10 0x5591f6672120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.128+0000 7fc905e43b80 4 client.26369 mds_command: new command op to 24521 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:27:50.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.128+0000 7fc905e43b80 1 -- 172.21.15.133:0/3283956538 --> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5591f6525660 con 0x5591f666fd10 2023-11-27T15:27:50.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.128+0000 7fc8ebfff700 1 --2- 172.21.15.133:0/3283956538 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x5591f666fd10 0x5591f6672120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.129+0000 7fc8ebfff700 1 --2- 172.21.15.133:0/3283956538 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x5591f666fd10 0x5591f6672120 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c7d613de5023b64 server_cookie=90c79b437e28cc10 in_seq=0 out_seq=0 2023-11-27T15:27:50.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.129+0000 7fc8cb7fe700 10 client.26369 ms_handle_connect on v2:172.21.15.174:6836/1955090548 2023-11-27T15:27:50.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.129+0000 7fc8cb7fe700 1 -- 172.21.15.133:0/3283956538 <== mds.1 v2:172.21.15.174:6836/1955090548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5591f6525660 con 0x5591f666fd10 2023-11-27T15:27:50.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.129+0000 7fc8cb7fe700 10 client.26369 handle_command_reply: tid=0 2023-11-27T15:27:50.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.129+0000 7fc905e43b80 1 client.26369 shutdown 2023-11-27T15:27:50.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.129+0000 7fc905e43b80 1 -- 172.21.15.133:0/3283956538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6606550 msgr2=0x5591f66148d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:50.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.129+0000 7fc905e43b80 1 --2- 172.21.15.133:0/3283956538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6606550 0x5591f66148d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc8e000bd30 tx=0x7fc8e0005c40 comp rx=0 tx=0).stop 2023-11-27T15:27:50.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.129+0000 7fc905e43b80 1 -- 172.21.15.133:0/3283956538 shutdown_connections 2023-11-27T15:27:50.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.129+0000 7fc905e43b80 1 --2- 172.21.15.133:0/3283956538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5591f6607110 0x5591f66158e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.129+0000 7fc905e43b80 1 --2- 172.21.15.133:0/3283956538 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5591f6605c00 0x5591f6614390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.129+0000 7fc905e43b80 1 --2- 172.21.15.133:0/3283956538 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x5591f666fd10 0x5591f6672120 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.129+0000 7fc905e43b80 1 --2- 172.21.15.133:0/3283956538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591f6606550 0x5591f66148d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.129+0000 7fc905e43b80 1 -- 172.21.15.133:0/3283956538 >> 172.21.15.133:0/3283956538 conn(0x5591f65fb360 msgr2=0x5591f660a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:50.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.130+0000 7fc8cb7fe700 0 client.26369 ms_handle_reset on v2:172.21.15.174:6836/1955090548 2023-11-27T15:27:50.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.130+0000 7fc905e43b80 1 -- 172.21.15.133:0/3283956538 shutdown_connections 2023-11-27T15:27:50.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.130+0000 7fc905e43b80 1 -- 172.21.15.133:0/3283956538 wait complete. 2023-11-27T15:27:50.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.130+0000 7fc905e43b80 20 client.26369 trim_cache size 0 max 16384 2023-11-27T15:27:50.138 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2023-11-27T15:27:50.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.131+0000 7fc8f25ff700 1 -- 172.21.15.133:0/4223426327 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc8d40a11d0 msgr2=0x7fc8d40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:50.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.131+0000 7fc8f25ff700 1 --2- 172.21.15.133:0/4223426327 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc8d40a11d0 0x7fc8d40a3680 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc003790 tx=0x7fc8dc005fb0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.131+0000 7fc8f25ff700 1 -- 172.21.15.133:0/4223426327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec107d60 msgr2=0x7fc8ec109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:50.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.131+0000 7fc8f25ff700 1 --2- 172.21.15.133:0/4223426327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec107d60 0x7fc8ec109300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4007f10 tx=0x7fc8e400d3b0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.131+0000 7fc8f25ff700 1 -- 172.21.15.133:0/4223426327 shutdown_connections 2023-11-27T15:27:50.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.131+0000 7fc8f25ff700 1 --2- 172.21.15.133:0/4223426327 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc8d40a11d0 0x7fc8d40a3680 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.131+0000 7fc8f25ff700 1 --2- 172.21.15.133:0/4223426327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec107d60 0x7fc8ec109300 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.131+0000 7fc8f25ff700 1 --2- 172.21.15.133:0/4223426327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8ec071ce0 0x7fc8ec10a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.131+0000 7fc8f25ff700 1 --2- 172.21.15.133:0/4223426327 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc8ec072670 0x7fc8ec108dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.131+0000 7fc8f25ff700 1 -- 172.21.15.133:0/4223426327 >> 172.21.15.133:0/4223426327 conn(0x7fc8ec06d0d0 msgr2=0x7fc8ec10d300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:50.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.131+0000 7fc8f25ff700 1 -- 172.21.15.133:0/4223426327 shutdown_connections 2023-11-27T15:27:50.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.132+0000 7fc8f25ff700 1 -- 172.21.15.133:0/4223426327 wait complete. 2023-11-27T15:27:50.157 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:27:50.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:49 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/584639243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:27:50.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:49 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/584639243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:27:50.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.505+0000 7f7b84dff700 1 -- 172.21.15.133:0/1952876298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b80074b30 msgr2=0x7f7b80072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:50.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.505+0000 7f7b84dff700 1 --2- 172.21.15.133:0/1952876298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b80074b30 0x7f7b80072f80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7b700098f0 tx=0x7f7b70009c00 comp rx=0 tx=0).stop 2023-11-27T15:27:50.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.506+0000 7f7b84dff700 1 -- 172.21.15.133:0/1952876298 shutdown_connections 2023-11-27T15:27:50.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.506+0000 7f7b84dff700 1 --2- 172.21.15.133:0/1952876298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b8010c8a0 0x7f7b8010ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.506+0000 7f7b84dff700 1 --2- 172.21.15.133:0/1952876298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b80074b30 0x7f7b80072f80 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.506+0000 7f7b84dff700 1 --2- 172.21.15.133:0/1952876298 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b80073550 0x7f7b800739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.506+0000 7f7b84dff700 1 -- 172.21.15.133:0/1952876298 >> 172.21.15.133:0/1952876298 conn(0x7f7b800fb8b0 msgr2=0x7f7b800fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:50.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.506+0000 7f7b84dff700 1 -- 172.21.15.133:0/1952876298 shutdown_connections 2023-11-27T15:27:50.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.506+0000 7f7b84dff700 1 -- 172.21.15.133:0/1952876298 wait complete. 2023-11-27T15:27:50.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.507+0000 7f7b84dff700 1 Processor -- start 2023-11-27T15:27:50.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.508+0000 7f7b84dff700 1 -- start start 2023-11-27T15:27:50.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.508+0000 7f7b84dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b80073550 0x7f7b8019c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.508+0000 7f7b84dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b80074b30 0x7f7b8019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.508+0000 7f7b84dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b8010c8a0 0x7f7b801a3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.508+0000 7f7b84dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b801a42e0 con 0x7f7b80073550 2023-11-27T15:27:50.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.508+0000 7f7b84dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b801a4420 con 0x7f7b8010c8a0 2023-11-27T15:27:50.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.508+0000 7f7b84dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b801a45b0 con 0x7f7b80074b30 2023-11-27T15:27:50.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.509+0000 7f7b7ed9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b8010c8a0 0x7f7b801a3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.509+0000 7f7b77fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b80074b30 0x7f7b8019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.509+0000 7f7b7ed9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b8010c8a0 0x7f7b801a3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:46946/0 (socket says 172.21.15.133:46946) 2023-11-27T15:27:50.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.509+0000 7f7b7ed9d700 1 -- 172.21.15.133:0/2341869055 learned_addr learned my addr 172.21.15.133:0/2341869055 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:50.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.510+0000 7f7b7e59c700 1 --2- 172.21.15.133:0/2341869055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b80073550 0x7f7b8019c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.511+0000 7f7b7ed9d700 1 -- 172.21.15.133:0/2341869055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b80074b30 msgr2=0x7f7b8019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:50.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.511+0000 7f7b7ed9d700 1 --2- 172.21.15.133:0/2341869055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b80074b30 0x7f7b8019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.511+0000 7f7b7ed9d700 1 -- 172.21.15.133:0/2341869055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b80073550 msgr2=0x7f7b8019c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:50.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.511+0000 7f7b7ed9d700 1 --2- 172.21.15.133:0/2341869055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b80073550 0x7f7b8019c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.511+0000 7f7b7ed9d700 1 -- 172.21.15.133:0/2341869055 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b70009520 con 0x7f7b8010c8a0 2023-11-27T15:27:50.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.511+0000 7f7b7e59c700 1 --2- 172.21.15.133:0/2341869055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b80073550 0x7f7b8019c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:50.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.511+0000 7f7b77fff700 1 --2- 172.21.15.133:0/2341869055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b80074b30 0x7f7b8019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:27:50.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.511+0000 7f7b7ed9d700 1 --2- 172.21.15.133:0/2341869055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b8010c8a0 0x7f7b801a3d10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800c110 tx=0x7f7b7800c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:50.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.512+0000 7f7b777fe700 1 -- 172.21.15.133:0/2341869055 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b78017070 con 0x7f7b8010c8a0 2023-11-27T15:27:50.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.512+0000 7f7b777fe700 1 -- 172.21.15.133:0/2341869055 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b7800e050 con 0x7f7b8010c8a0 2023-11-27T15:27:50.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.512+0000 7f7b777fe700 1 -- 172.21.15.133:0/2341869055 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b78007980 con 0x7f7b8010c8a0 2023-11-27T15:27:50.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.512+0000 7f7b84dff700 1 -- 172.21.15.133:0/2341869055 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b80112290 con 0x7f7b8010c8a0 2023-11-27T15:27:50.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.512+0000 7f7b84dff700 1 -- 172.21.15.133:0/2341869055 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b80112100 con 0x7f7b8010c8a0 2023-11-27T15:27:50.526 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.522+0000 7f7b777fe700 1 -- 172.21.15.133:0/2341869055 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7b78007ae0 con 0x7f7b8010c8a0 2023-11-27T15:27:50.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.525+0000 7f7b98651b80 1 Processor -- start 2023-11-27T15:27:50.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.525+0000 7f7b98651b80 1 -- start start 2023-11-27T15:27:50.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.525+0000 7f7b98651b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a51c00 0x555738a52010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.526+0000 7f7b98651b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a52550 0x555738a52b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.526+0000 7f7b777fe700 1 --2- 172.21.15.133:0/2341869055 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7b6c0a12a0 0x7f7b6c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.526+0000 7f7b98651b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a53110 0x555738a53580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.526+0000 7f7b98651b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555738957060 con 0x555738a51c00 2023-11-27T15:27:50.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.526+0000 7f7b98651b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557389a98a0 con 0x555738a53110 2023-11-27T15:27:50.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.526+0000 7f7b98651b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557389a2d30 con 0x555738a52550 2023-11-27T15:27:50.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.526+0000 7f7b777fe700 1 -- 172.21.15.133:0/2341869055 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7b780cc5d0 con 0x7f7b8010c8a0 2023-11-27T15:27:50.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.527+0000 7f7b7e59c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a52550 0x555738a52b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.527+0000 7f7b7e59c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a52550 0x555738a52b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48558/0 (socket says 172.21.15.133:48558) 2023-11-27T15:27:50.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.527+0000 7f7b7e59c700 1 -- 172.21.15.133:0/2597406023 learned_addr learned my addr 172.21.15.133:0/2597406023 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:50.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.527+0000 7f7b77fff700 1 --2- 172.21.15.133:0/2597406023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a51c00 0x555738a52010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.527+0000 7f7b7e59c700 1 -- 172.21.15.133:0/2597406023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a53110 msgr2=0x555738a53580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-27T15:27:50.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.527+0000 7f7b7e59c700 1 --2- 172.21.15.133:0/2597406023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a53110 0x555738a53580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.527+0000 7f7b7e59c700 1 -- 172.21.15.133:0/2597406023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a51c00 msgr2=0x555738a52010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:50.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.527+0000 7f7b7e59c700 1 --2- 172.21.15.133:0/2597406023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a51c00 0x555738a52010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.527+0000 7f7b7e59c700 1 -- 172.21.15.133:0/2597406023 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555738824630 con 0x555738a52550 2023-11-27T15:27:50.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.527+0000 7f7b77fff700 1 --2- 172.21.15.133:0/2341869055 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7b6c0a12a0 0x7f7b6c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.527+0000 7f7b7e59c700 1 --2- 172.21.15.133:0/2597406023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a52550 0x555738a52b30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7b70016000 tx=0x7f7b70005bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ef6ddd8c6e393bc server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:50.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.528+0000 7f7b74ff9700 1 -- 172.21.15.133:0/2597406023 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7001c070 con 0x555738a52550 2023-11-27T15:27:50.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.528+0000 7f7b74ff9700 1 -- 172.21.15.133:0/2597406023 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b7000fc80 con 0x555738a52550 2023-11-27T15:27:50.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.528+0000 7f7b74ff9700 1 -- 172.21.15.133:0/2597406023 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7000b7c0 con 0x555738a52550 2023-11-27T15:27:50.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.529+0000 7f7b98651b80 1 -- 172.21.15.133:0/2597406023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a52550 msgr2=0x555738a52b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:50.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.529+0000 7f7b98651b80 1 --2- 172.21.15.133:0/2597406023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a52550 0x555738a52b30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7b70016000 tx=0x7f7b70005bf0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.529+0000 7f7b77fff700 1 --2- 172.21.15.133:0/2341869055 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7b6c0a12a0 0x7f7b6c0a3750 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7b80074850 tx=0x7f7b680047c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:50.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.529+0000 7f7b98651b80 1 -- 172.21.15.133:0/2597406023 shutdown_connections 2023-11-27T15:27:50.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.529+0000 7f7b98651b80 1 --2- 172.21.15.133:0/2597406023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a53110 0x555738a53580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.529+0000 7f7b98651b80 1 --2- 172.21.15.133:0/2597406023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a51c00 0x555738a52010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.529+0000 7f7b98651b80 1 --2- 172.21.15.133:0/2597406023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a52550 0x555738a52b30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.529+0000 7f7b98651b80 1 -- 172.21.15.133:0/2597406023 >> 172.21.15.133:0/2597406023 conn(0x555738a47360 msgr2=0x555738a4c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:50.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.530+0000 7f7b98651b80 1 -- 172.21.15.133:0/2597406023 shutdown_connections 2023-11-27T15:27:50.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.530+0000 7f7b98651b80 1 -- 172.21.15.133:0/2597406023 wait complete. 2023-11-27T15:27:50.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.530+0000 7f7b98651b80 1 -- start start 2023-11-27T15:27:50.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.530+0000 7f7b98651b80 1 Processor -- start 2023-11-27T15:27:50.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.531+0000 7f7b98651b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a51c00 0x555738a5b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.531+0000 7f7b98651b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a52550 0x555738a5bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.531+0000 7f7b98651b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a53110 0x555738a62b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.531+0000 7f7b98651b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557389448e0 con 0x555738a53110 2023-11-27T15:27:50.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.531+0000 7f7b98651b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x555738944580 con 0x555738a51c00 2023-11-27T15:27:50.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.531+0000 7f7b98651b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x555738826920 con 0x555738a52550 2023-11-27T15:27:50.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.531+0000 7f7b98651b80 10 client.0 _ll_register_callbacks cb 0x555738962780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:50.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.531+0000 7f7b7e59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a51c00 0x555738a5b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.531+0000 7f7b7e59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a51c00 0x555738a5b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:46978/0 (socket says 172.21.15.133:46978) 2023-11-27T15:27:50.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.531+0000 7f7b7e59c700 1 -- 172.21.15.133:0/2675560169 learned_addr learned my addr 172.21.15.133:0/2675560169 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:50.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.532+0000 7f7b7e59c700 1 -- 172.21.15.133:0/2675560169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a52550 msgr2=0x555738a5bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:50.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.532+0000 7f7b77fff700 1 --2- 172.21.15.133:0/2675560169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a52550 0x555738a5bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.532+0000 7f7b7e59c700 1 --2- 172.21.15.133:0/2675560169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a52550 0x555738a5bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.532+0000 7f7b7e59c700 1 -- 172.21.15.133:0/2675560169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a53110 msgr2=0x555738a62b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:50.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.532+0000 7f7b7ed9d700 1 --2- 172.21.15.133:0/2675560169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a53110 0x555738a62b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.532+0000 7f7b7e59c700 1 --2- 172.21.15.133:0/2675560169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a53110 0x555738a62b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.532+0000 7f7b7e59c700 1 -- 172.21.15.133:0/2675560169 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b8010f630 con 0x555738a51c00 2023-11-27T15:27:50.542 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.532+0000 7f7b7ed9d700 1 --2- 172.21.15.133:0/2675560169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a53110 0x555738a62b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:50.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.532+0000 7f7b7e59c700 1 --2- 172.21.15.133:0/2675560169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a51c00 0x555738a5b600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7b70000c00 tx=0x7f7b7000f860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25b6e781ff571b90 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:50.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.532+0000 7f7b65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:27:50.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.533+0000 7f7b65ffb700 1 -- 172.21.15.133:0/2675560169 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7001c040 con 0x555738a51c00 2023-11-27T15:27:50.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.533+0000 7f7b65ffb700 1 -- 172.21.15.133:0/2675560169 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7b7000b860 con 0x555738a51c00 2023-11-27T15:27:50.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.533+0000 7f7b98651b80 1 -- 172.21.15.133:0/2675560169 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555738a44b20 con 0x555738a51c00 2023-11-27T15:27:50.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.533+0000 7f7b65ffb700 1 -- 172.21.15.133:0/2675560169 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b7000ba20 con 0x555738a51c00 2023-11-27T15:27:50.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.533+0000 7f7b98651b80 10 client.26378 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:50.543 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.533+0000 7f7b98651b80 1 -- 172.21.15.133:0/2675560169 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555738969190 con 0x555738a51c00 2023-11-27T15:27:50.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.534+0000 7f7b65ffb700 1 -- 172.21.15.133:0/2675560169 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555738a44b20 con 0x555738a51c00 2023-11-27T15:27:50.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.534+0000 7f7b65ffb700 1 -- 172.21.15.133:0/2675560169 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f7b70032070 con 0x555738a51c00 2023-11-27T15:27:50.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.534+0000 7f7b98651b80 10 client.26378 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:50.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.534+0000 7f7b98651b80 10 client.26378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:50.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.534+0000 7f7b98651b80 20 client.26378 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:50.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.534+0000 7f7b98651b80 1 --2- 172.21.15.133:0/2675560169 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x555738a66fb0 0x555738a693c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.534+0000 7f7b98651b80 4 client.26378 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:50.544 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.534+0000 7f7b98651b80 1 -- 172.21.15.133:0/2675560169 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557389650a0 con 0x555738a66fb0 2023-11-27T15:27:50.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.535+0000 7f7b77fff700 1 --2- 172.21.15.133:0/2675560169 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x555738a66fb0 0x555738a693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.535+0000 7f7b77fff700 1 --2- 172.21.15.133:0/2675560169 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x555738a66fb0 0x555738a693c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e86d0ddd7da21b13 server_cookie=b116fdfaf49f9e65 in_seq=0 out_seq=0 2023-11-27T15:27:50.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.535+0000 7f7b65ffb700 10 client.26378 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:50.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.536+0000 7f7b65ffb700 1 -- 172.21.15.133:0/2675560169 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5557389650a0 con 0x555738a66fb0 2023-11-27T15:27:50.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.536+0000 7f7b65ffb700 10 client.26378 handle_command_reply: tid=0 2023-11-27T15:27:50.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.536+0000 7f7b98651b80 1 client.26378 shutdown 2023-11-27T15:27:50.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.537+0000 7f7b98651b80 1 -- 172.21.15.133:0/2675560169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a51c00 msgr2=0x555738a5b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:50.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.537+0000 7f7b98651b80 1 --2- 172.21.15.133:0/2675560169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a51c00 0x555738a5b600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7b70000c00 tx=0x7f7b7000f860 comp rx=0 tx=0).stop 2023-11-27T15:27:50.545 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.537+0000 7f7b98651b80 1 -- 172.21.15.133:0/2675560169 shutdown_connections 2023-11-27T15:27:50.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.537+0000 7f7b98651b80 1 --2- 172.21.15.133:0/2675560169 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x555738a66fb0 0x555738a693c0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.537+0000 7f7b98651b80 1 --2- 172.21.15.133:0/2675560169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a53110 0x555738a62b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.537+0000 7f7b98651b80 1 --2- 172.21.15.133:0/2675560169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a51c00 0x555738a5b600 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.537+0000 7f7b98651b80 1 --2- 172.21.15.133:0/2675560169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a52550 0x555738a5bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.537+0000 7f7b98651b80 1 -- 172.21.15.133:0/2675560169 >> 172.21.15.133:0/2675560169 conn(0x555738a47360 msgr2=0x555738a4bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:50.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.537+0000 7f7b65ffb700 0 client.26378 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:50.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.537+0000 7f7b98651b80 1 -- 172.21.15.133:0/2675560169 shutdown_connections 2023-11-27T15:27:50.546 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.538+0000 7f7b98651b80 1 -- 172.21.15.133:0/2675560169 wait complete. 2023-11-27T15:27:50.547 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.538+0000 7f7b98651b80 20 client.26378 trim_cache size 0 max 16384 2023-11-27T15:27:50.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.557+0000 7f7b98651b80 1 Processor -- start 2023-11-27T15:27:50.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.557+0000 7f7b98651b80 1 -- start start 2023-11-27T15:27:50.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.558+0000 7f7b98651b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a51c00 0x555738a55fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.558+0000 7f7b98651b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a52550 0x555738a56500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.562 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.558+0000 7f7b98651b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a53110 0x555738a62b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.558+0000 7f7b98651b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555738a5c7f0 con 0x555738a53110 2023-11-27T15:27:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.558+0000 7f7b98651b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x555738a52f90 con 0x555738a51c00 2023-11-27T15:27:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.558+0000 7f7b98651b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x555738a4e140 con 0x555738a52550 2023-11-27T15:27:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.558+0000 7f7b77fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a52550 0x555738a56500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.558+0000 7f7b7e59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a51c00 0x555738a55fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.558+0000 7f7b77fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a52550 0x555738a56500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48572/0 (socket says 172.21.15.133:48572) 2023-11-27T15:27:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.558+0000 7f7b77fff700 1 -- 172.21.15.133:0/82102411 learned_addr learned my addr 172.21.15.133:0/82102411 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:50.563 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.558+0000 7f7b7ed9d700 1 --2- 172.21.15.133:0/82102411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a53110 0x555738a62b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.558+0000 7f7b7e59c700 1 -- 172.21.15.133:0/82102411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a52550 msgr2=0x555738a56500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:50.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.558+0000 7f7b7e59c700 1 --2- 172.21.15.133:0/82102411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a52550 0x555738a56500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.558+0000 7f7b7e59c700 1 -- 172.21.15.133:0/82102411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a53110 msgr2=0x555738a62b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:50.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.558+0000 7f7b7e59c700 1 --2- 172.21.15.133:0/82102411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a53110 0x555738a62b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.558+0000 7f7b7e59c700 1 -- 172.21.15.133:0/82102411 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555738968ee0 con 0x555738a51c00 2023-11-27T15:27:50.564 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.559+0000 7f7b7e59c700 1 --2- 172.21.15.133:0/82102411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a51c00 0x555738a55fc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7b70000c00 tx=0x7f7b7000f8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=415e7403baac9d29 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:50.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.559+0000 7f7b757fa700 1 -- 172.21.15.133:0/82102411 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b70033e60 con 0x555738a51c00 2023-11-27T15:27:50.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.559+0000 7f7b757fa700 1 -- 172.21.15.133:0/82102411 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b70021e90 con 0x555738a51c00 2023-11-27T15:27:50.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.559+0000 7f7b757fa700 1 -- 172.21.15.133:0/82102411 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b70033600 con 0x555738a51c00 2023-11-27T15:27:50.567 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.560+0000 7f7b98651b80 1 -- 172.21.15.133:0/82102411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a51c00 msgr2=0x555738a55fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:50.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.560+0000 7f7b98651b80 1 --2- 172.21.15.133:0/82102411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a51c00 0x555738a55fc0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7b70000c00 tx=0x7f7b7000f8d0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.560+0000 7f7b98651b80 1 -- 172.21.15.133:0/82102411 shutdown_connections 2023-11-27T15:27:50.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.560+0000 7f7b98651b80 1 --2- 172.21.15.133:0/82102411 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a53110 0x555738a62b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.560+0000 7f7b98651b80 1 --2- 172.21.15.133:0/82102411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a51c00 0x555738a55fc0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.560+0000 7f7b98651b80 1 --2- 172.21.15.133:0/82102411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a52550 0x555738a56500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.560+0000 7f7b98651b80 1 -- 172.21.15.133:0/82102411 >> 172.21.15.133:0/82102411 conn(0x555738a47360 msgr2=0x555738a63470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:50.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.560+0000 7f7b98651b80 1 -- 172.21.15.133:0/82102411 shutdown_connections 2023-11-27T15:27:50.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.560+0000 7f7b98651b80 1 -- 172.21.15.133:0/82102411 wait complete. 2023-11-27T15:27:50.568 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.561+0000 7f7b98651b80 1 -- start start 2023-11-27T15:27:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.561+0000 7f7b98651b80 1 Processor -- start 2023-11-27T15:27:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.562+0000 7f7b98651b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a51c00 0x555738a5e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.562+0000 7f7b98651b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a52550 0x555738a5e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.562+0000 7f7b98651b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a53110 0x555738a61960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.562+0000 7f7b98651b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555738a61ea0 con 0x555738a53110 2023-11-27T15:27:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.562+0000 7f7b98651b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x555738a62010 con 0x555738a52550 2023-11-27T15:27:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.562+0000 7f7b98651b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x555738a621d0 con 0x555738a51c00 2023-11-27T15:27:50.569 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.562+0000 7f7b98651b80 10 client.0 _ll_register_callbacks cb 0x7f7b78009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:50.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.562+0000 7f7b7e59c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a51c00 0x555738a5e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.562+0000 7f7b7e59c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a51c00 0x555738a5e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48582/0 (socket says 172.21.15.133:48582) 2023-11-27T15:27:50.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.562+0000 7f7b77fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a52550 0x555738a5e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.562+0000 7f7b7e59c700 1 -- 172.21.15.133:0/2444200459 learned_addr learned my addr 172.21.15.133:0/2444200459 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:50.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.563+0000 7f7b7ed9d700 1 --2- 172.21.15.133:0/2444200459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a53110 0x555738a61960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.563+0000 7f7b7e59c700 1 -- 172.21.15.133:0/2444200459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a52550 msgr2=0x555738a5e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:50.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.563+0000 7f7b7e59c700 1 --2- 172.21.15.133:0/2444200459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a52550 0x555738a5e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.570 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.563+0000 7f7b7e59c700 1 -- 172.21.15.133:0/2444200459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a53110 msgr2=0x555738a61960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:50.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.563+0000 7f7b7e59c700 1 --2- 172.21.15.133:0/2444200459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a53110 0x555738a61960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.563+0000 7f7b7e59c700 1 -- 172.21.15.133:0/2444200459 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555738969190 con 0x555738a51c00 2023-11-27T15:27:50.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.563+0000 7f7b7ed9d700 1 --2- 172.21.15.133:0/2444200459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a53110 0x555738a61960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:50.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.563+0000 7f7b77fff700 1 --2- 172.21.15.133:0/2444200459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a52550 0x555738a5e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:50.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.563+0000 7f7b7e59c700 1 --2- 172.21.15.133:0/2444200459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a51c00 0x555738a5e400 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7b70021800 tx=0x7f7b70009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21087cd4e455781b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:50.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.563+0000 7f7b65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:27:50.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.563+0000 7f7b65ffb700 1 -- 172.21.15.133:0/2444200459 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7001c030 con 0x555738a51c00 2023-11-27T15:27:50.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.564+0000 7f7b65ffb700 1 -- 172.21.15.133:0/2444200459 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7b70000e40 con 0x555738a51c00 2023-11-27T15:27:50.571 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.564+0000 7f7b98651b80 1 -- 172.21.15.133:0/2444200459 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b7000fc80 con 0x555738a51c00 2023-11-27T15:27:50.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.564+0000 7f7b65ffb700 1 -- 172.21.15.133:0/2444200459 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b700053c0 con 0x555738a51c00 2023-11-27T15:27:50.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.564+0000 7f7b98651b80 10 client.26371 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:50.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.564+0000 7f7b98651b80 1 -- 172.21.15.133:0/2444200459 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5557387b9c60 con 0x555738a51c00 2023-11-27T15:27:50.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.564+0000 7f7b65ffb700 1 -- 172.21.15.133:0/2444200459 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b7000fc80 con 0x555738a51c00 2023-11-27T15:27:50.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.565+0000 7f7b65ffb700 1 -- 172.21.15.133:0/2444200459 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f7b70047070 con 0x555738a51c00 2023-11-27T15:27:50.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.565+0000 7f7b98651b80 10 client.26371 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:50.574 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.565+0000 7f7b98651b80 10 client.26371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:50.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.565+0000 7f7b98651b80 20 client.26371 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:50.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.565+0000 7f7b98651b80 1 --2- 172.21.15.133:0/2444200459 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x555738abb4e0 0x555738abd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:50.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.565+0000 7f7b98651b80 4 client.26371 mds_command: new command op to 24560 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:27:50.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.565+0000 7f7b98651b80 1 -- 172.21.15.133:0/2444200459 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555738971660 con 0x555738abb4e0 2023-11-27T15:27:50.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.565+0000 7f7b77fff700 1 --2- 172.21.15.133:0/2444200459 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x555738abb4e0 0x555738abd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:50.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.566+0000 7f7b77fff700 1 --2- 172.21.15.133:0/2444200459 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x555738abb4e0 0x555738abd8c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd3c74bd4339d8a4 server_cookie=63cfda9036f15e7 in_seq=0 out_seq=0 2023-11-27T15:27:50.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.566+0000 7f7b65ffb700 10 client.26371 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:50.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.566+0000 7f7b65ffb700 1 -- 172.21.15.133:0/2444200459 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555738971660 con 0x555738abb4e0 2023-11-27T15:27:50.575 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.566+0000 7f7b65ffb700 10 client.26371 handle_command_reply: tid=0 2023-11-27T15:27:50.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.566+0000 7f7b98651b80 1 client.26371 shutdown 2023-11-27T15:27:50.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.566+0000 7f7b98651b80 1 -- 172.21.15.133:0/2444200459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a51c00 msgr2=0x555738a5e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:50.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.566+0000 7f7b98651b80 1 --2- 172.21.15.133:0/2444200459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a51c00 0x555738a5e400 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7b70021800 tx=0x7f7b70009260 comp rx=0 tx=0).stop 2023-11-27T15:27:50.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.566+0000 7f7b98651b80 1 -- 172.21.15.133:0/2444200459 shutdown_connections 2023-11-27T15:27:50.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.566+0000 7f7b98651b80 1 --2- 172.21.15.133:0/2444200459 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x555738abb4e0 0x555738abd8c0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.566+0000 7f7b98651b80 1 --2- 172.21.15.133:0/2444200459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555738a53110 0x555738a61960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.567+0000 7f7b65ffb700 0 client.26371 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:50.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.567+0000 7f7b98651b80 1 --2- 172.21.15.133:0/2444200459 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555738a51c00 0x555738a5e400 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.576 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.567+0000 7f7b98651b80 1 --2- 172.21.15.133:0/2444200459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555738a52550 0x555738a5e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.567+0000 7f7b98651b80 1 -- 172.21.15.133:0/2444200459 >> 172.21.15.133:0/2444200459 conn(0x555738a47360 msgr2=0x555738a594b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:50.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.567+0000 7f7b98651b80 1 -- 172.21.15.133:0/2444200459 shutdown_connections 2023-11-27T15:27:50.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.567+0000 7f7b98651b80 1 -- 172.21.15.133:0/2444200459 wait complete. 2023-11-27T15:27:50.577 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.567+0000 7f7b98651b80 20 client.26371 trim_cache size 0 max 16384 2023-11-27T15:27:50.577 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2023-11-27T15:27:50.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.568+0000 7f7b84dff700 1 -- 172.21.15.133:0/2341869055 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7b6c0a12a0 msgr2=0x7f7b6c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:50.578 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.568+0000 7f7b84dff700 1 --2- 172.21.15.133:0/2341869055 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7b6c0a12a0 0x7f7b6c0a3750 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7b80074850 tx=0x7f7b680047c0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.568+0000 7f7b84dff700 1 -- 172.21.15.133:0/2341869055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b8010c8a0 msgr2=0x7f7b801a3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:50.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.568+0000 7f7b84dff700 1 --2- 172.21.15.133:0/2341869055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b8010c8a0 0x7f7b801a3d10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7b7800c110 tx=0x7f7b7800c420 comp rx=0 tx=0).stop 2023-11-27T15:27:50.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.568+0000 7f7b84dff700 1 -- 172.21.15.133:0/2341869055 shutdown_connections 2023-11-27T15:27:50.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.568+0000 7f7b84dff700 1 --2- 172.21.15.133:0/2341869055 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7b6c0a12a0 0x7f7b6c0a3750 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.568+0000 7f7b84dff700 1 --2- 172.21.15.133:0/2341869055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b8010c8a0 0x7f7b801a3d10 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.568+0000 7f7b84dff700 1 --2- 172.21.15.133:0/2341869055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b80073550 0x7f7b8019c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.568+0000 7f7b84dff700 1 --2- 172.21.15.133:0/2341869055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7b80074b30 0x7f7b8019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:50.579 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.568+0000 7f7b84dff700 1 -- 172.21.15.133:0/2341869055 >> 172.21.15.133:0/2341869055 conn(0x7f7b800fb8b0 msgr2=0x7f7b800fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:50.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.568+0000 7f7b84dff700 1 -- 172.21.15.133:0/2341869055 shutdown_connections 2023-11-27T15:27:50.580 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:50.568+0000 7f7b84dff700 1 -- 172.21.15.133:0/2341869055 wait complete. 2023-11-27T15:27:51.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:50 smithi133 ceph-mon[97860]: pgmap v496: 129 pgs: 129 active+clean; 362 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 649 op/s 2023-11-27T15:27:51.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:50 smithi177 ceph-mon[131504]: pgmap v496: 129 pgs: 129 active+clean; 362 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 649 op/s 2023-11-27T15:27:51.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:50 smithi174 ceph-mon[131110]: pgmap v496: 129 pgs: 129 active+clean; 362 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 649 op/s 2023-11-27T15:27:51.596 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-11-27T15:27:51.596 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-11-27T15:27:51.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.948+0000 7fa9277ff700 1 -- 172.21.15.133:0/47226026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9200feb90 msgr2=0x7fa9200fefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:51.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.948+0000 7fa9277ff700 1 --2- 172.21.15.133:0/47226026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9200feb90 0x7fa9200fefa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa914009990 tx=0x7fa914009ca0 comp rx=0 tx=0).stop 2023-11-27T15:27:51.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.948+0000 7fa9277ff700 1 -- 172.21.15.133:0/47226026 shutdown_connections 2023-11-27T15:27:51.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.948+0000 7fa9277ff700 1 --2- 172.21.15.133:0/47226026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9200ffe90 0x7fa920100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:51.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.948+0000 7fa9277ff700 1 --2- 172.21.15.133:0/47226026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9200feb90 0x7fa9200fefa0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:51.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.948+0000 7fa9277ff700 1 --2- 172.21.15.133:0/47226026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9200ff4e0 0x7fa9200ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:51.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.948+0000 7fa9277ff700 1 -- 172.21.15.133:0/47226026 >> 172.21.15.133:0/47226026 conn(0x7fa9200fa7c0 msgr2=0x7fa9200fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:51.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.949+0000 7fa9277ff700 1 -- 172.21.15.133:0/47226026 shutdown_connections 2023-11-27T15:27:51.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.949+0000 7fa9277ff700 1 -- 172.21.15.133:0/47226026 wait complete. 2023-11-27T15:27:51.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.950+0000 7fa9277ff700 1 Processor -- start 2023-11-27T15:27:51.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.950+0000 7fa9277ff700 1 -- start start 2023-11-27T15:27:51.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.950+0000 7fa9277ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9200feb90 0x7fa920104710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:51.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.950+0000 7fa9277ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9200ff4e0 0x7fa920104c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:51.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.950+0000 7fa9277ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9200ffe90 0x7fa9201012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:51.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.950+0000 7fa9277ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa920101a40 con 0x7fa9200ff4e0 2023-11-27T15:27:51.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.951+0000 7fa9277ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa920101bb0 con 0x7fa9200ffe90 2023-11-27T15:27:51.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.951+0000 7fa9277ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa920101d70 con 0x7fa9200feb90 2023-11-27T15:27:51.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.951+0000 7fa92559b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9200feb90 0x7fa920104710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:51.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.951+0000 7fa92559b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9200feb90 0x7fa920104710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48598/0 (socket says 172.21.15.133:48598) 2023-11-27T15:27:51.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.951+0000 7fa92559b700 1 -- 172.21.15.133:0/76596018 learned_addr learned my addr 172.21.15.133:0/76596018 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:51.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.951+0000 7fa924d9a700 1 --2- 172.21.15.133:0/76596018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9200ff4e0 0x7fa920104c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:51.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.951+0000 7fa925d9c700 1 --2- 172.21.15.133:0/76596018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9200ffe90 0x7fa9201012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:51.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.951+0000 7fa92559b700 1 -- 172.21.15.133:0/76596018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9200ffe90 msgr2=0x7fa9201012a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:51.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.951+0000 7fa92559b700 1 --2- 172.21.15.133:0/76596018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9200ffe90 0x7fa9201012a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:51.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.951+0000 7fa92559b700 1 -- 172.21.15.133:0/76596018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9200ff4e0 msgr2=0x7fa920104c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:51.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.951+0000 7fa92559b700 1 --2- 172.21.15.133:0/76596018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9200ff4e0 0x7fa920104c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:51.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.951+0000 7fa92559b700 1 -- 172.21.15.133:0/76596018 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9140095f0 con 0x7fa9200feb90 2023-11-27T15:27:51.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.952+0000 7fa92559b700 1 --2- 172.21.15.133:0/76596018 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9200feb90 0x7fa920104710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa914006010 tx=0x7fa914009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:51.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.952+0000 7fa9127fc700 1 -- 172.21.15.133:0/76596018 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91401d070 con 0x7fa9200feb90 2023-11-27T15:27:51.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.952+0000 7fa9277ff700 1 -- 172.21.15.133:0/76596018 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92010de60 con 0x7fa9200feb90 2023-11-27T15:27:51.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.952+0000 7fa9127fc700 1 -- 172.21.15.133:0/76596018 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa914022470 con 0x7fa9200feb90 2023-11-27T15:27:51.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.952+0000 7fa9127fc700 1 -- 172.21.15.133:0/76596018 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914005720 con 0x7fa9200feb90 2023-11-27T15:27:51.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.952+0000 7fa9277ff700 1 -- 172.21.15.133:0/76596018 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92010dcd0 con 0x7fa9200feb90 2023-11-27T15:27:51.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.956+0000 7fa9127fc700 1 -- 172.21.15.133:0/76596018 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fa9140225e0 con 0x7fa9200feb90 2023-11-27T15:27:51.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.957+0000 7fa9127fc700 1 --2- 172.21.15.133:0/76596018 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa90c0a1250 0x7fa90c0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:51.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.957+0000 7fa9127fc700 1 -- 172.21.15.133:0/76596018 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa9140cde80 con 0x7fa9200feb90 2023-11-27T15:27:51.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.957+0000 7fa93b191b80 1 Processor -- start 2023-11-27T15:27:51.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.957+0000 7fa924d9a700 1 --2- 172.21.15.133:0/76596018 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa90c0a1250 0x7fa90c0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:51.961 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.958+0000 7fa93b191b80 1 -- start start 2023-11-27T15:27:51.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.959+0000 7fa93b191b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b4ffc80 0x559b5b500090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:51.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.959+0000 7fa93b191b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b5005d0 0x559b5b500bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:51.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.959+0000 7fa93b191b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b501190 0x559b5b501600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:51.963 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.959+0000 7fa93b191b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5b4050e0 con 0x559b5b501190 2023-11-27T15:27:51.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.959+0000 7fa93b191b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5b457920 con 0x559b5b5005d0 2023-11-27T15:27:51.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.959+0000 7fa93b191b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5b450db0 con 0x559b5b4ffc80 2023-11-27T15:27:51.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.959+0000 7fa924d9a700 1 --2- 172.21.15.133:0/76596018 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa90c0a1250 0x7fa90c0a3700 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009c60 tx=0x7fa91c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:51.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.959+0000 7fa92559b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b4ffc80 0x559b5b500090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:51.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.959+0000 7fa92559b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b4ffc80 0x559b5b500090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48602/0 (socket says 172.21.15.133:48602) 2023-11-27T15:27:51.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.959+0000 7fa92559b700 1 -- 172.21.15.133:0/725292234 learned_addr learned my addr 172.21.15.133:0/725292234 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:51.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.959+0000 7fa925d9c700 1 --2- 172.21.15.133:0/725292234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b501190 0x559b5b501600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:51.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.959+0000 7fa924d9a700 1 --2- 172.21.15.133:0/725292234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b5005d0 0x559b5b500bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:51.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.959+0000 7fa92559b700 1 -- 172.21.15.133:0/725292234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b5005d0 msgr2=0x559b5b500bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:51.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.959+0000 7fa92559b700 1 --2- 172.21.15.133:0/725292234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b5005d0 0x559b5b500bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:51.964 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.959+0000 7fa92559b700 1 -- 172.21.15.133:0/725292234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b501190 msgr2=0x559b5b501600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:51.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.959+0000 7fa92559b700 1 --2- 172.21.15.133:0/725292234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b501190 0x559b5b501600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:51.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.959+0000 7fa92559b700 1 -- 172.21.15.133:0/725292234 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b5b2d26b0 con 0x559b5b4ffc80 2023-11-27T15:27:51.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.960+0000 7fa92559b700 1 --2- 172.21.15.133:0/725292234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b4ffc80 0x559b5b500090 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa914097470 tx=0x7fa914009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f0bf823397056ac server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:51.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.960+0000 7fa90b7fe700 1 -- 172.21.15.133:0/725292234 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91409efa0 con 0x559b5b4ffc80 2023-11-27T15:27:51.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.960+0000 7fa90b7fe700 1 -- 172.21.15.133:0/725292234 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9140a3330 con 0x559b5b4ffc80 2023-11-27T15:27:51.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.960+0000 7fa90b7fe700 1 -- 172.21.15.133:0/725292234 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91409a6c0 con 0x559b5b4ffc80 2023-11-27T15:27:51.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.961+0000 7fa93b191b80 1 -- 172.21.15.133:0/725292234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b4ffc80 msgr2=0x559b5b500090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:51.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.961+0000 7fa93b191b80 1 --2- 172.21.15.133:0/725292234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b4ffc80 0x559b5b500090 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa914097470 tx=0x7fa914009ca0 comp rx=0 tx=0).stop 2023-11-27T15:27:51.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.961+0000 7fa93b191b80 1 -- 172.21.15.133:0/725292234 shutdown_connections 2023-11-27T15:27:51.967 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.961+0000 7fa93b191b80 1 --2- 172.21.15.133:0/725292234 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b501190 0x559b5b501600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:51.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.961+0000 7fa93b191b80 1 --2- 172.21.15.133:0/725292234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b4ffc80 0x559b5b500090 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:51.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.961+0000 7fa93b191b80 1 --2- 172.21.15.133:0/725292234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b5005d0 0x559b5b500bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:51.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.961+0000 7fa93b191b80 1 -- 172.21.15.133:0/725292234 >> 172.21.15.133:0/725292234 conn(0x559b5b4f53e0 msgr2=0x559b5b4fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:51.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.961+0000 7fa93b191b80 1 -- 172.21.15.133:0/725292234 shutdown_connections 2023-11-27T15:27:51.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.961+0000 7fa93b191b80 1 -- 172.21.15.133:0/725292234 wait complete. 2023-11-27T15:27:51.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.961+0000 7fa93b191b80 1 -- start start 2023-11-27T15:27:51.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.962+0000 7fa93b191b80 1 Processor -- start 2023-11-27T15:27:51.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.962+0000 7fa93b191b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b4ffc80 0x559b5b509340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:51.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.962+0000 7fa93b191b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b5005d0 0x559b5b509880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:51.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.962+0000 7fa93b191b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b501190 0x559b5b5108c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:51.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.962+0000 7fa93b191b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5b3f2960 con 0x559b5b4ffc80 2023-11-27T15:27:51.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.962+0000 7fa93b191b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5b3f2600 con 0x559b5b5005d0 2023-11-27T15:27:51.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.962+0000 7fa93b191b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5b2d49a0 con 0x559b5b501190 2023-11-27T15:27:51.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.962+0000 7fa93b191b80 10 client.0 _ll_register_callbacks cb 0x559b5b410800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:51.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.963+0000 7fa924d9a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b5005d0 0x559b5b509880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:51.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.963+0000 7fa924d9a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b5005d0 0x559b5b509880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:47044/0 (socket says 172.21.15.133:47044) 2023-11-27T15:27:51.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.963+0000 7fa924d9a700 1 -- 172.21.15.133:0/2961592179 learned_addr learned my addr 172.21.15.133:0/2961592179 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:51.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.963+0000 7fa92559b700 1 --2- 172.21.15.133:0/2961592179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b4ffc80 0x559b5b509340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:51.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.963+0000 7fa925d9c700 1 --2- 172.21.15.133:0/2961592179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b501190 0x559b5b5108c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:51.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.963+0000 7fa924d9a700 1 -- 172.21.15.133:0/2961592179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b501190 msgr2=0x559b5b5108c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:51.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.963+0000 7fa924d9a700 1 --2- 172.21.15.133:0/2961592179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b501190 0x559b5b5108c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:51.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.963+0000 7fa924d9a700 1 -- 172.21.15.133:0/2961592179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b4ffc80 msgr2=0x559b5b509340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:51.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.963+0000 7fa924d9a700 1 --2- 172.21.15.133:0/2961592179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b4ffc80 0x559b5b509340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:51.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.963+0000 7fa924d9a700 1 -- 172.21.15.133:0/2961592179 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92010dcd0 con 0x559b5b5005d0 2023-11-27T15:27:51.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.963+0000 7fa925d9c700 1 --2- 172.21.15.133:0/2961592179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b501190 0x559b5b5108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:51.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.963+0000 7fa924d9a700 1 --2- 172.21.15.133:0/2961592179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b5005d0 0x559b5b509880 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa91c01dbd0 tx=0x7fa91c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=855713461a0ab88d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:51.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.964+0000 7fa908ff9700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:27:51.970 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.964+0000 7fa908ff9700 1 -- 172.21.15.133:0/2961592179 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c00ec50 con 0x559b5b5005d0 2023-11-27T15:27:51.971 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.964+0000 7fa908ff9700 1 -- 172.21.15.133:0/2961592179 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa91c00edb0 con 0x559b5b5005d0 2023-11-27T15:27:51.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.964+0000 7fa93b191b80 1 -- 172.21.15.133:0/2961592179 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b5b4f2ba0 con 0x559b5b5005d0 2023-11-27T15:27:51.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.965+0000 7fa908ff9700 1 -- 172.21.15.133:0/2961592179 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa91c028450 con 0x559b5b5005d0 2023-11-27T15:27:51.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.965+0000 7fa93b191b80 10 client.26393 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:51.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.965+0000 7fa93b191b80 1 -- 172.21.15.133:0/2961592179 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559b5b417210 con 0x559b5b5005d0 2023-11-27T15:27:51.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.965+0000 7fa908ff9700 1 -- 172.21.15.133:0/2961592179 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559b5b4f2ba0 con 0x559b5b5005d0 2023-11-27T15:27:51.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.965+0000 7fa908ff9700 1 -- 172.21.15.133:0/2961592179 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fa91c01a580 con 0x559b5b5005d0 2023-11-27T15:27:51.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.966+0000 7fa93b191b80 10 client.26393 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:51.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.966+0000 7fa93b191b80 10 client.26393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:51.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.966+0000 7fa93b191b80 20 client.26393 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:51.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.966+0000 7fa93b191b80 1 --2- 172.21.15.133:0/2961592179 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x559b5b515180 0x559b5b517590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:51.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.966+0000 7fa93b191b80 4 client.26393 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:51.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.966+0000 7fa93b191b80 1 -- 172.21.15.133:0/2961592179 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b5b413120 con 0x559b5b515180 2023-11-27T15:27:51.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.966+0000 7fa92559b700 1 --2- 172.21.15.133:0/2961592179 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x559b5b515180 0x559b5b517590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:51.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.966+0000 7fa908ff9700 10 client.26393 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:51.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.966+0000 7fa92559b700 1 --2- 172.21.15.133:0/2961592179 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x559b5b515180 0x559b5b517590 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d1fa3c57bf4ff0 server_cookie=6d021888952aa904 in_seq=0 out_seq=0 2023-11-27T15:27:51.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.967+0000 7fa908ff9700 1 -- 172.21.15.133:0/2961592179 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559b5b413120 con 0x559b5b515180 2023-11-27T15:27:51.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.967+0000 7fa908ff9700 10 client.26393 handle_command_reply: tid=0 2023-11-27T15:27:51.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.967+0000 7fa93b191b80 1 client.26393 shutdown 2023-11-27T15:27:51.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.968+0000 7fa93b191b80 1 -- 172.21.15.133:0/2961592179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b5005d0 msgr2=0x559b5b509880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:51.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.968+0000 7fa93b191b80 1 --2- 172.21.15.133:0/2961592179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b5005d0 0x559b5b509880 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa91c01dbd0 tx=0x7fa91c0047c0 comp rx=0 tx=0).stop 2023-11-27T15:27:51.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.968+0000 7fa93b191b80 1 -- 172.21.15.133:0/2961592179 shutdown_connections 2023-11-27T15:27:51.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.968+0000 7fa93b191b80 1 --2- 172.21.15.133:0/2961592179 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x559b5b515180 0x559b5b517590 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:51.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.968+0000 7fa93b191b80 1 --2- 172.21.15.133:0/2961592179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b501190 0x559b5b5108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:51.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.968+0000 7fa908ff9700 0 client.26393 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:51.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.968+0000 7fa93b191b80 1 --2- 172.21.15.133:0/2961592179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b4ffc80 0x559b5b509340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:51.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.968+0000 7fa93b191b80 1 --2- 172.21.15.133:0/2961592179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b5005d0 0x559b5b509880 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:51.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.968+0000 7fa93b191b80 1 -- 172.21.15.133:0/2961592179 >> 172.21.15.133:0/2961592179 conn(0x559b5b4f53e0 msgr2=0x559b5b4f9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:51.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.968+0000 7fa93b191b80 1 -- 172.21.15.133:0/2961592179 shutdown_connections 2023-11-27T15:27:51.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.969+0000 7fa93b191b80 1 -- 172.21.15.133:0/2961592179 wait complete. 2023-11-27T15:27:51.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.969+0000 7fa93b191b80 20 client.26393 trim_cache size 0 max 16384 2023-11-27T15:27:51.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.991+0000 7fa93b191b80 1 Processor -- start 2023-11-27T15:27:51.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.991+0000 7fa93b191b80 1 -- start start 2023-11-27T15:27:51.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.992+0000 7fa93b191b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b4ffc80 0x559b5b504040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:51.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.992+0000 7fa93b191b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b5005d0 0x559b5b510880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:51.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.992+0000 7fa93b191b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b501190 0x559b5b504580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:51.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.992+0000 7fa93b191b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5b50a280 con 0x559b5b5005d0 2023-11-27T15:27:51.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.992+0000 7fa93b191b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5b4fb600 con 0x559b5b501190 2023-11-27T15:27:51.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.992+0000 7fa93b191b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5b512550 con 0x559b5b4ffc80 2023-11-27T15:27:51.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.992+0000 7fa92559b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b4ffc80 0x559b5b504040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:51.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.992+0000 7fa92559b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b4ffc80 0x559b5b504040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48610/0 (socket says 172.21.15.133:48610) 2023-11-27T15:27:51.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.992+0000 7fa92559b700 1 -- 172.21.15.133:0/3622540676 learned_addr learned my addr 172.21.15.133:0/3622540676 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:51.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.992+0000 7fa925d9c700 1 --2- 172.21.15.133:0/3622540676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b501190 0x559b5b504580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:51.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.992+0000 7fa924d9a700 1 --2- 172.21.15.133:0/3622540676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b5005d0 0x559b5b510880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:51.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.993+0000 7fa92559b700 1 -- 172.21.15.133:0/3622540676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b501190 msgr2=0x559b5b504580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:51.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.993+0000 7fa92559b700 1 --2- 172.21.15.133:0/3622540676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b501190 0x559b5b504580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:51.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.993+0000 7fa92559b700 1 -- 172.21.15.133:0/3622540676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b5005d0 msgr2=0x559b5b510880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:51.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.993+0000 7fa92559b700 1 --2- 172.21.15.133:0/3622540676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b5005d0 0x559b5b510880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:51.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.993+0000 7fa92559b700 1 -- 172.21.15.133:0/3622540676 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b5b4f6b80 con 0x559b5b4ffc80 2023-11-27T15:27:51.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.993+0000 7fa924d9a700 1 --2- 172.21.15.133:0/3622540676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b5005d0 0x559b5b510880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:51.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.993+0000 7fa92559b700 1 --2- 172.21.15.133:0/3622540676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b4ffc80 0x559b5b504040 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa91409ac70 tx=0x7fa914005910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9c35157251aa61 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:51.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.993+0000 7fa90bfff700 1 -- 172.21.15.133:0/3622540676 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914022950 con 0x559b5b4ffc80 2023-11-27T15:27:51.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.993+0000 7fa90bfff700 1 -- 172.21.15.133:0/3622540676 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9140cb5c0 con 0x559b5b4ffc80 2023-11-27T15:27:51.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.993+0000 7fa90bfff700 1 -- 172.21.15.133:0/3622540676 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91409a6c0 con 0x559b5b4ffc80 2023-11-27T15:27:52.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.994+0000 7fa93b191b80 1 -- 172.21.15.133:0/3622540676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b4ffc80 msgr2=0x559b5b504040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:52.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.994+0000 7fa93b191b80 1 --2- 172.21.15.133:0/3622540676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b4ffc80 0x559b5b504040 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa91409ac70 tx=0x7fa914005910 comp rx=0 tx=0).stop 2023-11-27T15:27:52.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.994+0000 7fa93b191b80 1 -- 172.21.15.133:0/3622540676 shutdown_connections 2023-11-27T15:27:52.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.994+0000 7fa93b191b80 1 --2- 172.21.15.133:0/3622540676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b501190 0x559b5b504580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.994+0000 7fa93b191b80 1 --2- 172.21.15.133:0/3622540676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b4ffc80 0x559b5b504040 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.994+0000 7fa93b191b80 1 --2- 172.21.15.133:0/3622540676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b5005d0 0x559b5b510880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.994+0000 7fa93b191b80 1 -- 172.21.15.133:0/3622540676 >> 172.21.15.133:0/3622540676 conn(0x559b5b4f53e0 msgr2=0x559b5b509300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:52.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.994+0000 7fa93b191b80 1 -- 172.21.15.133:0/3622540676 shutdown_connections 2023-11-27T15:27:52.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.994+0000 7fa93b191b80 1 -- 172.21.15.133:0/3622540676 wait complete. 2023-11-27T15:27:52.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.995+0000 7fa93b191b80 1 -- start start 2023-11-27T15:27:52.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.995+0000 7fa93b191b80 1 Processor -- start 2023-11-27T15:27:52.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.995+0000 7fa93b191b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b4ffc80 0x559b5b510400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.995+0000 7fa93b191b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b5005d0 0x559b5b50cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.995+0000 7fa93b191b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b501190 0x559b5b50d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.995+0000 7fa93b191b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5b510c20 con 0x559b5b501190 2023-11-27T15:27:52.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.995+0000 7fa93b191b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5b50d920 con 0x559b5b4ffc80 2023-11-27T15:27:52.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.995+0000 7fa93b191b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b5b50dab0 con 0x559b5b5005d0 2023-11-27T15:27:52.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.995+0000 7fa93b191b80 10 client.0 _ll_register_callbacks cb 0x559b5b410800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:52.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.996+0000 7fa92559b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b4ffc80 0x559b5b510400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:52.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.996+0000 7fa92559b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b4ffc80 0x559b5b510400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:47066/0 (socket says 172.21.15.133:47066) 2023-11-27T15:27:52.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.996+0000 7fa92559b700 1 -- 172.21.15.133:0/3454766249 learned_addr learned my addr 172.21.15.133:0/3454766249 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:52.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.996+0000 7fa924d9a700 1 --2- 172.21.15.133:0/3454766249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b5005d0 0x559b5b50cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:52.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.996+0000 7fa925d9c700 1 --2- 172.21.15.133:0/3454766249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b501190 0x559b5b50d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:52.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.996+0000 7fa92559b700 1 -- 172.21.15.133:0/3454766249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b5005d0 msgr2=0x559b5b50cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:52.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.996+0000 7fa92559b700 1 --2- 172.21.15.133:0/3454766249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b5005d0 0x559b5b50cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.996+0000 7fa92559b700 1 -- 172.21.15.133:0/3454766249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b501190 msgr2=0x559b5b50d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:52.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.996+0000 7fa92559b700 1 --2- 172.21.15.133:0/3454766249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b501190 0x559b5b50d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.996+0000 7fa92559b700 1 -- 172.21.15.133:0/3454766249 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b5b2d26b0 con 0x559b5b4ffc80 2023-11-27T15:27:52.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.996+0000 7fa925d9c700 1 --2- 172.21.15.133:0/3454766249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b501190 0x559b5b50d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:52.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.996+0000 7fa92559b700 1 --2- 172.21.15.133:0/3454766249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b4ffc80 0x559b5b510400 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa91409f160 tx=0x7fa9140aebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a12d36c8ba1d21f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:52.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.996+0000 7fa908ff9700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:27:52.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.997+0000 7fa908ff9700 1 -- 172.21.15.133:0/3454766249 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9140b9310 con 0x559b5b4ffc80 2023-11-27T15:27:52.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.997+0000 7fa908ff9700 1 -- 172.21.15.133:0/3454766249 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fa9140228e0 con 0x559b5b4ffc80 2023-11-27T15:27:52.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.997+0000 7fa93b191b80 1 -- 172.21.15.133:0/3454766249 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9140a3330 con 0x559b5b4ffc80 2023-11-27T15:27:52.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.997+0000 7fa908ff9700 1 -- 172.21.15.133:0/3454766249 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9140cb2d0 con 0x559b5b4ffc80 2023-11-27T15:27:52.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.997+0000 7fa93b191b80 10 client.26399 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:52.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.997+0000 7fa93b191b80 1 -- 172.21.15.133:0/3454766249 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559b5b27ee60 con 0x559b5b4ffc80 2023-11-27T15:27:52.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.997+0000 7fa908ff9700 1 -- 172.21.15.133:0/3454766249 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa914079e90 con 0x559b5b4ffc80 2023-11-27T15:27:52.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.997+0000 7fa908ff9700 1 -- 172.21.15.133:0/3454766249 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fa9140b9030 con 0x559b5b4ffc80 2023-11-27T15:27:52.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.997+0000 7fa93b191b80 10 client.26399 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:52.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.997+0000 7fa93b191b80 10 client.26399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:52.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.997+0000 7fa93b191b80 20 client.26399 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:52.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.997+0000 7fa93b191b80 1 --2- 172.21.15.133:0/3454766249 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x559b5b56a400 0x559b5b56c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.997+0000 7fa93b191b80 4 client.26399 mds_command: new command op to 24560 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-11-27T15:27:52.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.997+0000 7fa93b191b80 1 -- 172.21.15.133:0/3454766249 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559b5b41f6e0 con 0x559b5b56a400 2023-11-27T15:27:52.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.998+0000 7fa924d9a700 1 --2- 172.21.15.133:0/3454766249 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x559b5b56a400 0x559b5b56c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:52.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.998+0000 7fa924d9a700 1 --2- 172.21.15.133:0/3454766249 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x559b5b56a400 0x559b5b56c810 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed5ed89aa64571c7 server_cookie=8cff3069d78c98d0 in_seq=0 out_seq=0 2023-11-27T15:27:52.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:51.998+0000 7fa908ff9700 10 client.26399 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:52.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.002+0000 7fa908ff9700 1 -- 172.21.15.133:0/3454766249 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559b5b41f6e0 con 0x559b5b56a400 2023-11-27T15:27:52.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.002+0000 7fa908ff9700 10 client.26399 handle_command_reply: tid=0 2023-11-27T15:27:52.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.002+0000 7fa93b191b80 1 client.26399 shutdown 2023-11-27T15:27:52.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.002+0000 7fa93b191b80 1 -- 172.21.15.133:0/3454766249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b4ffc80 msgr2=0x559b5b510400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:52.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.002+0000 7fa93b191b80 1 --2- 172.21.15.133:0/3454766249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b4ffc80 0x559b5b510400 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa91409f160 tx=0x7fa9140aebe0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.002+0000 7fa93b191b80 1 -- 172.21.15.133:0/3454766249 shutdown_connections 2023-11-27T15:27:52.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.002+0000 7fa93b191b80 1 --2- 172.21.15.133:0/3454766249 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x559b5b56a400 0x559b5b56c810 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.002+0000 7fa93b191b80 1 --2- 172.21.15.133:0/3454766249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559b5b501190 0x559b5b50d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.002+0000 7fa93b191b80 1 --2- 172.21.15.133:0/3454766249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b5b4ffc80 0x559b5b510400 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.002+0000 7fa93b191b80 1 --2- 172.21.15.133:0/3454766249 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559b5b5005d0 0x559b5b50cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.002+0000 7fa93b191b80 1 -- 172.21.15.133:0/3454766249 >> 172.21.15.133:0/3454766249 conn(0x559b5b4f53e0 msgr2=0x559b5b5092e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:52.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.002+0000 7fa908ff9700 0 client.26399 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:52.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.002+0000 7fa93b191b80 1 -- 172.21.15.133:0/3454766249 shutdown_connections 2023-11-27T15:27:52.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.003+0000 7fa93b191b80 1 -- 172.21.15.133:0/3454766249 wait complete. 2023-11-27T15:27:52.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.003+0000 7fa93b191b80 20 client.26399 trim_cache size 0 max 16384 2023-11-27T15:27:52.008 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:27:52.008 INFO:teuthology.orchestra.run.smithi133.stdout: "return_code": 0, 2023-11-27T15:27:52.008 INFO:teuthology.orchestra.run.smithi133.stdout: "scrub_tag": "51b5fea8-bb0d-45b1-9654-3423e74f8a20", 2023-11-27T15:27:52.008 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": "asynchronous" 2023-11-27T15:27:52.008 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:27:52.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.004+0000 7fa9277ff700 1 -- 172.21.15.133:0/76596018 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa90c0a1250 msgr2=0x7fa90c0a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:52.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.004+0000 7fa9277ff700 1 --2- 172.21.15.133:0/76596018 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa90c0a1250 0x7fa90c0a3700 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009c60 tx=0x7fa91c009260 comp rx=0 tx=0).stop 2023-11-27T15:27:52.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.004+0000 7fa9277ff700 1 -- 172.21.15.133:0/76596018 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9200feb90 msgr2=0x7fa920104710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:52.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.004+0000 7fa9277ff700 1 --2- 172.21.15.133:0/76596018 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9200feb90 0x7fa920104710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa914006010 tx=0x7fa914009580 comp rx=0 tx=0).stop 2023-11-27T15:27:52.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.004+0000 7fa9277ff700 1 -- 172.21.15.133:0/76596018 shutdown_connections 2023-11-27T15:27:52.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.004+0000 7fa9277ff700 1 --2- 172.21.15.133:0/76596018 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fa90c0a1250 0x7fa90c0a3700 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.004+0000 7fa9277ff700 1 --2- 172.21.15.133:0/76596018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9200ffe90 0x7fa9201012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.004+0000 7fa9277ff700 1 --2- 172.21.15.133:0/76596018 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa9200feb90 0x7fa920104710 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.004+0000 7fa9277ff700 1 --2- 172.21.15.133:0/76596018 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9200ff4e0 0x7fa920104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.004+0000 7fa9277ff700 1 -- 172.21.15.133:0/76596018 >> 172.21.15.133:0/76596018 conn(0x7fa9200fa7c0 msgr2=0x7fa9200fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:52.010 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.004+0000 7fa9277ff700 1 -- 172.21.15.133:0/76596018 shutdown_connections 2023-11-27T15:27:52.010 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.004+0000 7fa9277ff700 1 -- 172.21.15.133:0/76596018 wait complete. 2023-11-27T15:27:52.030 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:27:52.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.298+0000 7fc5527ff700 1 -- 172.21.15.133:0/821398630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc54c107af0 msgr2=0x7fc54c107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:52.301 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.298+0000 7fc5527ff700 1 --2- 172.21.15.133:0/821398630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc54c107af0 0x7fc54c107f60 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc53c009990 tx=0x7fc53c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.298+0000 7fc5527ff700 1 -- 172.21.15.133:0/821398630 shutdown_connections 2023-11-27T15:27:52.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.298+0000 7fc5527ff700 1 --2- 172.21.15.133:0/821398630 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc54c1084a0 0x7fc54c117a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.298+0000 7fc5527ff700 1 --2- 172.21.15.133:0/821398630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc54c071bd0 0x7fc54c071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.298+0000 7fc5527ff700 1 --2- 172.21.15.133:0/821398630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc54c107af0 0x7fc54c107f60 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.298+0000 7fc5527ff700 1 -- 172.21.15.133:0/821398630 >> 172.21.15.133:0/821398630 conn(0x7fc54c06d0d0 msgr2=0x7fc54c06f520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:52.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.299+0000 7fc5527ff700 1 -- 172.21.15.133:0/821398630 shutdown_connections 2023-11-27T15:27:52.302 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.299+0000 7fc5527ff700 1 -- 172.21.15.133:0/821398630 wait complete. 2023-11-27T15:27:52.307 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.304+0000 7fc5527ff700 1 Processor -- start 2023-11-27T15:27:52.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.304+0000 7fc5527ff700 1 -- start start 2023-11-27T15:27:52.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.305+0000 7fc5527ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc54c071bd0 0x7fc54c114000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.311 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.305+0000 7fc5527ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc54c107af0 0x7fc54c114540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.305+0000 7fc5527ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc54c1084a0 0x7fc54c1b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.305+0000 7fc5527ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c117620 con 0x7fc54c1084a0 2023-11-27T15:27:52.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.305+0000 7fc5527ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c117790 con 0x7fc54c071bd0 2023-11-27T15:27:52.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.305+0000 7fc5527ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c1b5530 con 0x7fc54c107af0 2023-11-27T15:27:52.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.305+0000 7fc54bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc54c071bd0 0x7fc54c114000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:52.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.305+0000 7fc54bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc54c071bd0 0x7fc54c114000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:47074/0 (socket says 172.21.15.133:47074) 2023-11-27T15:27:52.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.305+0000 7fc54bfff700 1 -- 172.21.15.133:0/1441522001 learned_addr learned my addr 172.21.15.133:0/1441522001 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:52.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.306+0000 7fc54b7fe700 1 --2- 172.21.15.133:0/1441522001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc54c107af0 0x7fc54c114540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:52.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.308+0000 7fc54b7fe700 1 -- 172.21.15.133:0/1441522001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc54c071bd0 msgr2=0x7fc54c114000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:52.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.308+0000 7fc54b7fe700 1 --2- 172.21.15.133:0/1441522001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc54c071bd0 0x7fc54c114000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.308+0000 7fc54b7fe700 1 -- 172.21.15.133:0/1441522001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc54c1084a0 msgr2=0x7fc54c1b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:52.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.308+0000 7fc54b7fe700 1 --2- 172.21.15.133:0/1441522001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc54c1084a0 0x7fc54c1b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.312 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.308+0000 7fc54b7fe700 1 -- 172.21.15.133:0/1441522001 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53c0095f0 con 0x7fc54c107af0 2023-11-27T15:27:52.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.308+0000 7fc54b7fe700 1 --2- 172.21.15.133:0/1441522001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc54c107af0 0x7fc54c114540 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc53c015000 tx=0x7fc53c0038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:52.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.308+0000 7fc5497fa700 1 -- 172.21.15.133:0/1441522001 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c01d070 con 0x7fc54c107af0 2023-11-27T15:27:52.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.308+0000 7fc5497fa700 1 -- 172.21.15.133:0/1441522001 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc53c00be70 con 0x7fc54c107af0 2023-11-27T15:27:52.313 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.308+0000 7fc5497fa700 1 -- 172.21.15.133:0/1441522001 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c004230 con 0x7fc54c107af0 2023-11-27T15:27:52.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.308+0000 7fc5527ff700 1 -- 172.21.15.133:0/1441522001 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54c11aa00 con 0x7fc54c107af0 2023-11-27T15:27:52.314 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.309+0000 7fc5527ff700 1 -- 172.21.15.133:0/1441522001 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54c11a870 con 0x7fc54c107af0 2023-11-27T15:27:52.317 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.314+0000 7fc5497fa700 1 -- 172.21.15.133:0/1441522001 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fc53c00fa20 con 0x7fc54c107af0 2023-11-27T15:27:52.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.317+0000 7fc5497fa700 1 --2- 172.21.15.133:0/1441522001 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc5340a11d0 0x7fc5340a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.320 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.317+0000 7fc5497fa700 1 -- 172.21.15.133:0/1441522001 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc53c0cdbe0 con 0x7fc54c107af0 2023-11-27T15:27:52.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.318+0000 7fc566037b80 1 Processor -- start 2023-11-27T15:27:52.322 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.318+0000 7fc566037b80 1 -- start start 2023-11-27T15:27:52.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.320+0000 7fc566037b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78fbc00 0x5633b78fc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.320+0000 7fc566037b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fc550 0x5633b78fcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.320+0000 7fc566037b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fd110 0x5633b78fd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.321+0000 7fc54bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78fbc00 0x5633b78fc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:52.324 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.321+0000 7fc54bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78fbc00 0x5633b78fc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:47090/0 (socket says 172.21.15.133:47090) 2023-11-27T15:27:52.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.321+0000 7fc54b7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fc550 0x5633b78fcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:52.325 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.321+0000 7fc54b7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fc550 0x5633b78fcb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:48634/0 (socket says 172.21.15.133:48634) 2023-11-27T15:27:52.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.322+0000 7fc550d9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fd110 0x5633b78fd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:52.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.323+0000 7fc550d9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fd110 0x5633b78fd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57318/0 (socket says 172.21.15.133:57318) 2023-11-27T15:27:52.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.323+0000 7fc566037b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b7801060 con 0x5633b78fd110 2023-11-27T15:27:52.326 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.323+0000 7fc566037b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b78538a0 con 0x5633b78fbc00 2023-11-27T15:27:52.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.323+0000 7fc566037b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b784cd30 con 0x5633b78fc550 2023-11-27T15:27:52.327 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.323+0000 7fc54bfff700 1 -- 172.21.15.133:0/1101679347 learned_addr learned my addr 172.21.15.133:0/1101679347 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:52.328 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.325+0000 7fc54bfff700 1 --2- 172.21.15.133:0/1441522001 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc5340a11d0 0x7fc5340a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:52.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.326+0000 7fc54b7fe700 1 -- 172.21.15.133:0/1101679347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78fbc00 msgr2=0x5633b78fc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:52.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.326+0000 7fc54b7fe700 1 --2- 172.21.15.133:0/1101679347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78fbc00 0x5633b78fc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.326+0000 7fc54b7fe700 1 -- 172.21.15.133:0/1101679347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fd110 msgr2=0x5633b78fd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:52.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.326+0000 7fc54b7fe700 1 --2- 172.21.15.133:0/1101679347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fd110 0x5633b78fd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.330 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.326+0000 7fc54b7fe700 1 -- 172.21.15.133:0/1101679347 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633b76ce630 con 0x5633b78fc550 2023-11-27T15:27:52.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.326+0000 7fc550d9c700 1 --2- 172.21.15.133:0/1101679347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fd110 0x5633b78fd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:52.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.326+0000 7fc54b7fe700 1 --2- 172.21.15.133:0/1101679347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fc550 0x5633b78fcb30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc53c0cb330 tx=0x7fc53c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1fe0d947465ae31 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:52.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.327+0000 7fc5327fc700 1 -- 172.21.15.133:0/1101679347 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c09d070 con 0x5633b78fc550 2023-11-27T15:27:52.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.327+0000 7fc5327fc700 1 -- 172.21.15.133:0/1101679347 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc53c09acd0 con 0x5633b78fc550 2023-11-27T15:27:52.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.327+0000 7fc5327fc700 1 -- 172.21.15.133:0/1101679347 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c09a420 con 0x5633b78fc550 2023-11-27T15:27:52.331 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.328+0000 7fc566037b80 1 -- 172.21.15.133:0/1101679347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fc550 msgr2=0x5633b78fcb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:52.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.328+0000 7fc566037b80 1 --2- 172.21.15.133:0/1101679347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fc550 0x5633b78fcb30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc53c0cb330 tx=0x7fc53c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.328+0000 7fc54bfff700 1 --2- 172.21.15.133:0/1441522001 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc5340a11d0 0x7fc5340a3680 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc54000bfd0 tx=0x7fc54000b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:52.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.328+0000 7fc566037b80 1 -- 172.21.15.133:0/1101679347 shutdown_connections 2023-11-27T15:27:52.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.328+0000 7fc566037b80 1 --2- 172.21.15.133:0/1101679347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fd110 0x5633b78fd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.328+0000 7fc566037b80 1 --2- 172.21.15.133:0/1101679347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78fbc00 0x5633b78fc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.328+0000 7fc566037b80 1 --2- 172.21.15.133:0/1101679347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fc550 0x5633b78fcb30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.332 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.328+0000 7fc566037b80 1 -- 172.21.15.133:0/1101679347 >> 172.21.15.133:0/1101679347 conn(0x5633b78f1360 msgr2=0x5633b78f6670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:52.333 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.329+0000 7fc566037b80 1 -- 172.21.15.133:0/1101679347 shutdown_connections 2023-11-27T15:27:52.333 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.329+0000 7fc566037b80 1 -- 172.21.15.133:0/1101679347 wait complete. 2023-11-27T15:27:52.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.330+0000 7fc566037b80 1 -- start start 2023-11-27T15:27:52.334 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.331+0000 7fc566037b80 1 Processor -- start 2023-11-27T15:27:52.338 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.335+0000 7fc566037b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fbc00 0x5633b7905240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.335+0000 7fc566037b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fc550 0x5633b7905780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.335+0000 7fc566037b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78fd110 0x5633b790c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.335+0000 7fc566037b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b77ee8e0 con 0x5633b78fc550 2023-11-27T15:27:52.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.335+0000 7fc566037b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b77ee580 con 0x5633b78fd110 2023-11-27T15:27:52.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.335+0000 7fc566037b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b76d0920 con 0x5633b78fbc00 2023-11-27T15:27:52.339 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.335+0000 7fc566037b80 10 client.0 _ll_register_callbacks cb 0x5633b780c780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:52.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.335+0000 7fc54b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fc550 0x5633b7905780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:52.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.336+0000 7fc54b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fc550 0x5633b7905780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57330/0 (socket says 172.21.15.133:57330) 2023-11-27T15:27:52.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.336+0000 7fc54b7fe700 1 -- 172.21.15.133:0/2983919066 learned_addr learned my addr 172.21.15.133:0/2983919066 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:52.340 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.336+0000 7fc550d9c700 1 --2- 172.21.15.133:0/2983919066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78fd110 0x5633b790c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.336+0000 7fc550d9c700 1 -- 172.21.15.133:0/2983919066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fbc00 msgr2=0x5633b7905240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.336+0000 7fc550d9c700 1 --2- 172.21.15.133:0/2983919066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fbc00 0x5633b7905240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.336+0000 7fc550d9c700 1 -- 172.21.15.133:0/2983919066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fc550 msgr2=0x5633b7905780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.336+0000 7fc550d9c700 1 --2- 172.21.15.133:0/2983919066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fc550 0x5633b7905780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.336+0000 7fc550d9c700 1 -- 172.21.15.133:0/2983919066 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54c11a870 con 0x5633b78fd110 2023-11-27T15:27:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.337+0000 7fc550d9c700 1 --2- 172.21.15.133:0/2983919066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78fd110 0x5633b790c7c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc544007ab0 tx=0x7fc544007e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=302432caf075445c server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:52.341 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.337+0000 7fc52b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:27:52.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.337+0000 7fc52b7fe700 1 -- 172.21.15.133:0/2983919066 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54400e070 con 0x5633b78fd110 2023-11-27T15:27:52.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.337+0000 7fc52b7fe700 1 -- 172.21.15.133:0/2983919066 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc544003e10 con 0x5633b78fd110 2023-11-27T15:27:52.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.337+0000 7fc566037b80 1 -- 172.21.15.133:0/2983919066 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633b78eeb20 con 0x5633b78fd110 2023-11-27T15:27:52.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.338+0000 7fc52b7fe700 1 -- 172.21.15.133:0/2983919066 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc544010630 con 0x5633b78fd110 2023-11-27T15:27:52.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.338+0000 7fc566037b80 10 client.26411 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:52.342 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.338+0000 7fc566037b80 1 -- 172.21.15.133:0/2983919066 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5633b7813190 con 0x5633b78fd110 2023-11-27T15:27:52.343 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.339+0000 7fc52b7fe700 1 -- 172.21.15.133:0/2983919066 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5633b78eeb20 con 0x5633b78fd110 2023-11-27T15:27:52.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.340+0000 7fc52b7fe700 1 -- 172.21.15.133:0/2983919066 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fc544004660 con 0x5633b78fd110 2023-11-27T15:27:52.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.341+0000 7fc566037b80 10 client.26411 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:52.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.341+0000 7fc566037b80 10 client.26411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:52.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.341+0000 7fc566037b80 20 client.26411 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:52.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.341+0000 7fc566037b80 1 --2- 172.21.15.133:0/2983919066 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5633b7911170 0x5633b7913580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.341+0000 7fc566037b80 4 client.26411 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:52.344 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.341+0000 7fc566037b80 1 -- 172.21.15.133:0/2983919066 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633b780f0a0 con 0x5633b7911170 2023-11-27T15:27:52.346 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.342+0000 7fc54bfff700 1 --2- 172.21.15.133:0/2983919066 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5633b7911170 0x5633b7913580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:52.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.345+0000 7fc54bfff700 1 --2- 172.21.15.133:0/2983919066 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5633b7911170 0x5633b7913580 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b0c8782baf898dc server_cookie=ab48b3a536f41994 in_seq=0 out_seq=0 2023-11-27T15:27:52.349 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.345+0000 7fc52b7fe700 10 client.26411 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:52.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.347+0000 7fc52b7fe700 1 -- 172.21.15.133:0/2983919066 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5633b780f0a0 con 0x5633b7911170 2023-11-27T15:27:52.350 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.347+0000 7fc52b7fe700 10 client.26411 handle_command_reply: tid=0 2023-11-27T15:27:52.351 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.347+0000 7fc566037b80 1 client.26411 shutdown 2023-11-27T15:27:52.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.349+0000 7fc566037b80 1 -- 172.21.15.133:0/2983919066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78fd110 msgr2=0x5633b790c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:52.352 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.349+0000 7fc566037b80 1 --2- 172.21.15.133:0/2983919066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78fd110 0x5633b790c7c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc544007ab0 tx=0x7fc544007e70 comp rx=0 tx=0).stop 2023-11-27T15:27:52.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.349+0000 7fc566037b80 1 -- 172.21.15.133:0/2983919066 shutdown_connections 2023-11-27T15:27:52.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.349+0000 7fc566037b80 1 --2- 172.21.15.133:0/2983919066 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5633b7911170 0x5633b7913580 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.349+0000 7fc566037b80 1 --2- 172.21.15.133:0/2983919066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78fd110 0x5633b790c7c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.349+0000 7fc566037b80 1 --2- 172.21.15.133:0/2983919066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fbc00 0x5633b7905240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.349+0000 7fc566037b80 1 --2- 172.21.15.133:0/2983919066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fc550 0x5633b7905780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.349+0000 7fc566037b80 1 -- 172.21.15.133:0/2983919066 >> 172.21.15.133:0/2983919066 conn(0x5633b78f1360 msgr2=0x5633b78f5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:52.353 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.350+0000 7fc52b7fe700 0 client.26411 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:52.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.350+0000 7fc566037b80 1 -- 172.21.15.133:0/2983919066 shutdown_connections 2023-11-27T15:27:52.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.350+0000 7fc566037b80 1 -- 172.21.15.133:0/2983919066 wait complete. 2023-11-27T15:27:52.354 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.351+0000 7fc566037b80 20 client.26411 trim_cache size 0 max 16384 2023-11-27T15:27:52.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.397+0000 7fc566037b80 1 Processor -- start 2023-11-27T15:27:52.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.397+0000 7fc566037b80 1 -- start start 2023-11-27T15:27:52.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.397+0000 7fc566037b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78f1360 0x5633b790d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.397+0000 7fc566037b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fbc00 0x5633b790c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.397+0000 7fc566037b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fc550 0x5633b78fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.397+0000 7fc566037b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b78fcf90 con 0x5633b78fc550 2023-11-27T15:27:52.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.397+0000 7fc566037b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b79046f0 con 0x5633b78f1360 2023-11-27T15:27:52.401 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.397+0000 7fc566037b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b7903d00 con 0x5633b78fbc00 2023-11-27T15:27:52.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.398+0000 7fc54bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78f1360 0x5633b790d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:52.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.398+0000 7fc550d9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fc550 0x5633b78fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:52.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.398+0000 7fc54bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78f1360 0x5633b790d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:47106/0 (socket says 172.21.15.133:47106) 2023-11-27T15:27:52.402 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.398+0000 7fc54bfff700 1 -- 172.21.15.133:0/2628498608 learned_addr learned my addr 172.21.15.133:0/2628498608 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:52.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.399+0000 7fc54b7fe700 1 --2- 172.21.15.133:0/2628498608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fbc00 0x5633b790c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:52.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.399+0000 7fc54bfff700 1 -- 172.21.15.133:0/2628498608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fbc00 msgr2=0x5633b790c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:52.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.399+0000 7fc54bfff700 1 --2- 172.21.15.133:0/2628498608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fbc00 0x5633b790c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.399+0000 7fc54bfff700 1 -- 172.21.15.133:0/2628498608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fc550 msgr2=0x5633b78fffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:52.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.399+0000 7fc54bfff700 1 --2- 172.21.15.133:0/2628498608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fc550 0x5633b78fffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.399+0000 7fc54bfff700 1 -- 172.21.15.133:0/2628498608 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633b7812ee0 con 0x5633b78f1360 2023-11-27T15:27:52.403 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.399+0000 7fc550d9c700 1 --2- 172.21.15.133:0/2628498608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fc550 0x5633b78fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:27:52.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.399+0000 7fc54b7fe700 1 --2- 172.21.15.133:0/2628498608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fbc00 0x5633b790c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:52.404 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.400+0000 7fc54bfff700 1 --2- 172.21.15.133:0/2628498608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78f1360 0x5633b790d0b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc5400124d0 tx=0x7fc54000b920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a3ebfbe32445254 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:52.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.402+0000 7fc532ffd700 1 -- 172.21.15.133:0/2628498608 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc540025040 con 0x5633b78f1360 2023-11-27T15:27:52.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.402+0000 7fc532ffd700 1 -- 172.21.15.133:0/2628498608 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5633b780f0a0 con 0x5633b78f1360 2023-11-27T15:27:52.405 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.402+0000 7fc532ffd700 1 -- 172.21.15.133:0/2628498608 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5400288d0 con 0x5633b78f1360 2023-11-27T15:27:52.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.403+0000 7fc566037b80 1 -- 172.21.15.133:0/2628498608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78f1360 msgr2=0x5633b790d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:52.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.403+0000 7fc566037b80 1 --2- 172.21.15.133:0/2628498608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78f1360 0x5633b790d0b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc5400124d0 tx=0x7fc54000b920 comp rx=0 tx=0).stop 2023-11-27T15:27:52.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.406+0000 7fc566037b80 1 -- 172.21.15.133:0/2628498608 shutdown_connections 2023-11-27T15:27:52.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.406+0000 7fc566037b80 1 --2- 172.21.15.133:0/2628498608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fc550 0x5633b78fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.406+0000 7fc566037b80 1 --2- 172.21.15.133:0/2628498608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78f1360 0x5633b790d0b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.406+0000 7fc566037b80 1 --2- 172.21.15.133:0/2628498608 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fbc00 0x5633b790c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.406+0000 7fc566037b80 1 -- 172.21.15.133:0/2628498608 >> 172.21.15.133:0/2628498608 conn(0x5633b79042f0 msgr2=0x5633b7917ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:52.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.406+0000 7fc566037b80 1 -- 172.21.15.133:0/2628498608 shutdown_connections 2023-11-27T15:27:52.410 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.407+0000 7fc566037b80 1 -- 172.21.15.133:0/2628498608 wait complete. 2023-11-27T15:27:52.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.407+0000 7fc566037b80 1 -- start start 2023-11-27T15:27:52.411 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.408+0000 7fc566037b80 1 Processor -- start 2023-11-27T15:27:52.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.408+0000 7fc566037b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78f1360 0x5633b7908910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.409+0000 7fc566037b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fbc00 0x5633b7908e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.409+0000 7fc566037b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fc550 0x5633b790bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.412 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.409+0000 7fc54bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78f1360 0x5633b7908910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:52.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.409+0000 7fc54bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78f1360 0x5633b7908910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:47112/0 (socket says 172.21.15.133:47112) 2023-11-27T15:27:52.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.409+0000 7fc566037b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b790c1c0 con 0x5633b78fbc00 2023-11-27T15:27:52.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.409+0000 7fc566037b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b790c330 con 0x5633b78f1360 2023-11-27T15:27:52.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.409+0000 7fc566037b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b790c4f0 con 0x5633b78fc550 2023-11-27T15:27:52.413 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.409+0000 7fc566037b80 10 client.0 _ll_register_callbacks cb 0x7fc5440093f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:52.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.410+0000 7fc54bfff700 1 -- 172.21.15.133:0/18386407 learned_addr learned my addr 172.21.15.133:0/18386407 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:52.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.410+0000 7fc550d9c700 1 --2- 172.21.15.133:0/18386407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fc550 0x5633b790bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:52.415 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.411+0000 7fc54b7fe700 1 --2- 172.21.15.133:0/18386407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fbc00 0x5633b7908e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:52.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.412+0000 7fc54b7fe700 1 -- 172.21.15.133:0/18386407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fc550 msgr2=0x5633b790bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:52.416 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.412+0000 7fc54b7fe700 1 --2- 172.21.15.133:0/18386407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fc550 0x5633b790bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.413+0000 7fc54b7fe700 1 -- 172.21.15.133:0/18386407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78f1360 msgr2=0x5633b7908910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:52.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.413+0000 7fc54b7fe700 1 --2- 172.21.15.133:0/18386407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78f1360 0x5633b7908910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.413+0000 7fc54b7fe700 1 -- 172.21.15.133:0/18386407 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633b7813190 con 0x5633b78fbc00 2023-11-27T15:27:52.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.413+0000 7fc550d9c700 1 --2- 172.21.15.133:0/18386407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fc550 0x5633b790bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:27:52.417 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.413+0000 7fc54bfff700 1 --2- 172.21.15.133:0/18386407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78f1360 0x5633b7908910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:27:52.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.415+0000 7fc54b7fe700 1 --2- 172.21.15.133:0/18386407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fbc00 0x5633b7908e50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc53c0992c0 tx=0x7fc53c00b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4130bcfdc8cda33e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:52.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.415+0000 7fc52b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:27:52.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.416+0000 7fc52b7fe700 1 -- 172.21.15.133:0/18386407 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c09d030 con 0x5633b78fbc00 2023-11-27T15:27:52.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.416+0000 7fc52b7fe700 1 -- 172.21.15.133:0/18386407 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fc53c09e070 con 0x5633b78fbc00 2023-11-27T15:27:52.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.416+0000 7fc566037b80 1 -- 172.21.15.133:0/18386407 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc53c09acd0 con 0x5633b78fbc00 2023-11-27T15:27:52.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.417+0000 7fc52b7fe700 1 -- 172.21.15.133:0/18386407 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc53c0a3090 con 0x5633b78fbc00 2023-11-27T15:27:52.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.417+0000 7fc566037b80 10 client.16293 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:52.421 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.417+0000 7fc566037b80 1 -- 172.21.15.133:0/18386407 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fc54001a040 con 0x5633b78fbc00 2023-11-27T15:27:52.422 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.419+0000 7fc52b7fe700 1 -- 172.21.15.133:0/18386407 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc53c09acd0 con 0x5633b78fbc00 2023-11-27T15:27:52.422 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.419+0000 7fc52b7fe700 1 -- 172.21.15.133:0/18386407 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fc53c0cafb0 con 0x5633b78fbc00 2023-11-27T15:27:52.422 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.419+0000 7fc566037b80 10 client.16293 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:52.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.419+0000 7fc566037b80 10 client.16293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:52.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.419+0000 7fc566037b80 20 client.16293 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:52.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.419+0000 7fc566037b80 1 --2- 172.21.15.133:0/18386407 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5633b78fd110 0x5633b7967e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:52.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.420+0000 7fc566037b80 4 client.16293 mds_command: new command op to 24560 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:27:52.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.420+0000 7fc566037b80 1 -- 172.21.15.133:0/18386407 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5633b781b660 con 0x5633b78fd110 2023-11-27T15:27:52.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.420+0000 7fc54bfff700 1 --2- 172.21.15.133:0/18386407 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5633b78fd110 0x5633b7967e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:52.427 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.423+0000 7fc54bfff700 1 --2- 172.21.15.133:0/18386407 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5633b78fd110 0x5633b7967e60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c403da00a1b0ee1 server_cookie=5891263ae0869f80 in_seq=0 out_seq=0 2023-11-27T15:27:52.427 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.423+0000 7fc52b7fe700 10 client.16293 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:52.427 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.424+0000 7fc52b7fe700 1 -- 172.21.15.133:0/18386407 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+268 (crc 0 0 0) 0x5633b781b660 con 0x5633b78fd110 2023-11-27T15:27:52.428 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.424+0000 7fc52b7fe700 10 client.16293 handle_command_reply: tid=0 2023-11-27T15:27:52.428 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.424+0000 7fc566037b80 1 client.16293 shutdown 2023-11-27T15:27:52.428 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.425+0000 7fc566037b80 1 -- 172.21.15.133:0/18386407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fbc00 msgr2=0x5633b7908e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:52.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.425+0000 7fc566037b80 1 --2- 172.21.15.133:0/18386407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fbc00 0x5633b7908e50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc53c0992c0 tx=0x7fc53c00b490 comp rx=0 tx=0).stop 2023-11-27T15:27:52.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.426+0000 7fc566037b80 1 -- 172.21.15.133:0/18386407 shutdown_connections 2023-11-27T15:27:52.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.426+0000 7fc566037b80 1 --2- 172.21.15.133:0/18386407 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5633b78fd110 0x5633b7967e60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.429 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.426+0000 7fc566037b80 1 --2- 172.21.15.133:0/18386407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5633b78fc550 0x5633b790bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.430 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.426+0000 7fc52b7fe700 0 client.16293 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:52.430 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.426+0000 7fc566037b80 1 --2- 172.21.15.133:0/18386407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633b78f1360 0x5633b7908910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.430 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.426+0000 7fc566037b80 1 --2- 172.21.15.133:0/18386407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5633b78fbc00 0x5633b7908e50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.430 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.426+0000 7fc566037b80 1 -- 172.21.15.133:0/18386407 >> 172.21.15.133:0/18386407 conn(0x5633b79042f0 msgr2=0x5633b790d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:52.431 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.428+0000 7fc566037b80 1 -- 172.21.15.133:0/18386407 shutdown_connections 2023-11-27T15:27:52.431 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.428+0000 7fc566037b80 1 -- 172.21.15.133:0/18386407 wait complete. 2023-11-27T15:27:52.432 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.429+0000 7fc566037b80 20 client.16293 trim_cache size 0 max 16384 2023-11-27T15:27:52.433 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:27:52.433 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "scrub active (1416 inodes in the stack)", 2023-11-27T15:27:52.433 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-11-27T15:27:52.433 INFO:teuthology.orchestra.run.smithi133.stdout: "51b5fea8-bb0d-45b1-9654-3423e74f8a20": { 2023-11-27T15:27:52.433 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-11-27T15:27:52.433 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "51b5fea8-bb0d-45b1-9654-3423e74f8a20", 2023-11-27T15:27:52.434 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-11-27T15:27:52.434 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:27:52.434 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:27:52.434 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:27:52.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.430+0000 7fc5527ff700 1 -- 172.21.15.133:0/1441522001 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc5340a11d0 msgr2=0x7fc5340a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:52.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.430+0000 7fc5527ff700 1 --2- 172.21.15.133:0/1441522001 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc5340a11d0 0x7fc5340a3680 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc54000bfd0 tx=0x7fc54000b4a0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.431+0000 7fc5527ff700 1 -- 172.21.15.133:0/1441522001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc54c107af0 msgr2=0x7fc54c114540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:52.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.431+0000 7fc5527ff700 1 --2- 172.21.15.133:0/1441522001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc54c107af0 0x7fc54c114540 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc53c015000 tx=0x7fc53c0038b0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.431+0000 7fc5527ff700 1 -- 172.21.15.133:0/1441522001 shutdown_connections 2023-11-27T15:27:52.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.431+0000 7fc5527ff700 1 --2- 172.21.15.133:0/1441522001 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fc5340a11d0 0x7fc5340a3680 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.431+0000 7fc5527ff700 1 --2- 172.21.15.133:0/1441522001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc54c1084a0 0x7fc54c1b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.431+0000 7fc5527ff700 1 --2- 172.21.15.133:0/1441522001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc54c071bd0 0x7fc54c114000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.431+0000 7fc5527ff700 1 --2- 172.21.15.133:0/1441522001 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc54c107af0 0x7fc54c114540 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:52.435 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.431+0000 7fc5527ff700 1 -- 172.21.15.133:0/1441522001 >> 172.21.15.133:0/1441522001 conn(0x7fc54c06d0d0 msgr2=0x7fc54c06ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:52.436 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.433+0000 7fc5527ff700 1 -- 172.21.15.133:0/1441522001 shutdown_connections 2023-11-27T15:27:52.436 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:27:52.433+0000 7fc5527ff700 1 -- 172.21.15.133:0/1441522001 wait complete. 2023-11-27T15:27:52.482 INFO:tasks.cephfs.filesystem:scrub status for tag:51b5fea8-bb0d-45b1-9654-3423e74f8a20 - {'path': '/', 'tag': '51b5fea8-bb0d-45b1-9654-3423e74f8a20', 'options': 'recursive,force'} 2023-11-27T15:27:53.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:52 smithi133 ceph-mon[97860]: pgmap v497: 129 pgs: 129 active+clean; 371 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 631 op/s 2023-11-27T15:27:53.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:52 smithi133 ceph-mon[97860]: scrub queued for path: / 2023-11-27T15:27:53.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:52 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:53.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:52 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:27:53.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:52 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:27:53.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:52 smithi177 ceph-mon[131504]: pgmap v497: 129 pgs: 129 active+clean; 371 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 631 op/s 2023-11-27T15:27:53.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:52 smithi177 ceph-mon[131504]: scrub queued for path: / 2023-11-27T15:27:53.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:52 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:53.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:52 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:27:53.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:52 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:27:53.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:52 smithi174 ceph-mon[131110]: pgmap v497: 129 pgs: 129 active+clean; 371 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 631 op/s 2023-11-27T15:27:53.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:52 smithi174 ceph-mon[131110]: scrub queued for path: / 2023-11-27T15:27:53.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:52 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:53.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:52 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:27:53.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:52 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:27:54.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:53 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:27:54.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:53 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:54.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:53 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:27:54.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:53 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:54.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:53 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:27:54.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:53 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:27:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.178+0000 7f981179d700 1 -- 172.21.15.133:0/1702126761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f980c100070 msgr2=0x7f980c100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.178+0000 7f981179d700 1 --2- 172.21.15.133:0/1702126761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f980c100070 0x7f980c100480 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009990 tx=0x7f97fc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.178+0000 7f981179d700 1 -- 172.21.15.133:0/1702126761 shutdown_connections 2023-11-27T15:27:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.178+0000 7f981179d700 1 --2- 172.21.15.133:0/1702126761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f980c101bf0 0x7f980c1084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.178+0000 7f981179d700 1 --2- 172.21.15.133:0/1702126761 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f980c100070 0x7f980c100480 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.178+0000 7f981179d700 1 --2- 172.21.15.133:0/1702126761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f980c101260 0x7f980c1016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.178+0000 7f981179d700 1 -- 172.21.15.133:0/1702126761 >> 172.21.15.133:0/1702126761 conn(0x7f980c0fb670 msgr2=0x7f980c0fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.179+0000 7f981179d700 1 -- 172.21.15.133:0/1702126761 shutdown_connections 2023-11-27T15:27:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.179+0000 7f981179d700 1 -- 172.21.15.133:0/1702126761 wait complete. 2023-11-27T15:27:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.180+0000 7f981179d700 1 Processor -- start 2023-11-27T15:27:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.180+0000 7f981179d700 1 -- start start 2023-11-27T15:27:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.181+0000 7f981179d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f980c100070 0x7f980c06f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.181+0000 7f981179d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f980c101260 0x7f980c074920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.181+0000 7f981179d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f980c101bf0 0x7f980c06f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.181+0000 7f981179d700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c1a83c0 con 0x7f980c101260 2023-11-27T15:27:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.181+0000 7f981179d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c1a8500 con 0x7f980c101bf0 2023-11-27T15:27:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.181+0000 7f981179d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c1a8670 con 0x7f980c100070 2023-11-27T15:27:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.181+0000 7f980bfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f980c100070 0x7f980c06f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.181+0000 7f980bfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f980c100070 0x7f980c06f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:59788/0 (socket says 172.21.15.133:59788) 2023-11-27T15:27:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.181+0000 7f9810f9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f980c101bf0 0x7f980c06f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.181+0000 7f980bfff700 1 -- 172.21.15.133:0/704618356 learned_addr learned my addr 172.21.15.133:0/704618356 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.181+0000 7f980b7fe700 1 --2- 172.21.15.133:0/704618356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f980c101260 0x7f980c074920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.181+0000 7f9810f9c700 1 -- 172.21.15.133:0/704618356 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f980c100070 msgr2=0x7f980c06f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.181+0000 7f9810f9c700 1 --2- 172.21.15.133:0/704618356 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f980c100070 0x7f980c06f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.182+0000 7f9810f9c700 1 -- 172.21.15.133:0/704618356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f980c101260 msgr2=0x7f980c074920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.182+0000 7f9810f9c700 1 --2- 172.21.15.133:0/704618356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f980c101260 0x7f980c074920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.182+0000 7f9810f9c700 1 -- 172.21.15.133:0/704618356 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97fc0095f0 con 0x7f980c101bf0 2023-11-27T15:27:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.182+0000 7f980b7fe700 1 --2- 172.21.15.133:0/704618356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f980c101260 0x7f980c074920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.182+0000 7f9810f9c700 1 --2- 172.21.15.133:0/704618356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f980c101bf0 0x7f980c06f940 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f980400efc0 tx=0x7f980400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.182+0000 7f98097fa700 1 -- 172.21.15.133:0/704618356 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9804018070 con 0x7f980c101bf0 2023-11-27T15:27:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.182+0000 7f98097fa700 1 -- 172.21.15.133:0/704618356 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f980400f040 con 0x7f980c101bf0 2023-11-27T15:27:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.182+0000 7f98097fa700 1 -- 172.21.15.133:0/704618356 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980400cdf0 con 0x7f980c101bf0 2023-11-27T15:27:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.183+0000 7f981179d700 1 -- 172.21.15.133:0/704618356 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980c10b9d0 con 0x7f980c101bf0 2023-11-27T15:27:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.183+0000 7f981179d700 1 -- 172.21.15.133:0/704618356 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980c10b840 con 0x7f980c101bf0 2023-11-27T15:27:55.191 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.188+0000 7f98097fa700 1 -- 172.21.15.133:0/704618356 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f9804007500 con 0x7f980c101bf0 2023-11-27T15:27:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.190+0000 7f9826418b80 1 Processor -- start 2023-11-27T15:27:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.190+0000 7f98097fa700 1 --2- 172.21.15.133:0/704618356 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f97f40a1250 0x7f97f40a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.190+0000 7f98097fa700 1 -- 172.21.15.133:0/704618356 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f98040cc850 con 0x7f980c101bf0 2023-11-27T15:27:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.191+0000 7f9826418b80 1 -- start start 2023-11-27T15:27:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.191+0000 7f980b7fe700 1 --2- 172.21.15.133:0/704618356 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f97f40a1250 0x7f97f40a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.191+0000 7f9826418b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556158ed6b0 0x555615a9a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.191+0000 7f9826418b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555615a9a7c0 0x555615a9b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.191+0000 7f9826418b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555615a9b570 0x555615a9ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.191+0000 7f9826418b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555615997000 con 0x5556158ed6b0 2023-11-27T15:27:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.191+0000 7f9826418b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556159e9840 con 0x555615a9b570 2023-11-27T15:27:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.191+0000 7f9826418b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556159e2cd0 con 0x555615a9a7c0 2023-11-27T15:27:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.192+0000 7f980b7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555615a9a7c0 0x555615a9b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.192+0000 7f980b7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555615a9a7c0 0x555615a9b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:59800/0 (socket says 172.21.15.133:59800) 2023-11-27T15:27:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.192+0000 7f980b7fe700 1 -- 172.21.15.133:0/1049671365 learned_addr learned my addr 172.21.15.133:0/1049671365 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.192+0000 7f9810f9c700 1 --2- 172.21.15.133:0/1049671365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555615a9b570 0x555615a9ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.192+0000 7f980bfff700 1 --2- 172.21.15.133:0/1049671365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556158ed6b0 0x555615a9a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.192+0000 7f980b7fe700 1 --2- 172.21.15.133:0/704618356 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f97f40a1250 0x7f97f40a3700 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9800015610 tx=0x7f9800015920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.192+0000 7f980b7fe700 1 -- 172.21.15.133:0/1049671365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555615a9b570 msgr2=0x555615a9ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.192+0000 7f980b7fe700 1 --2- 172.21.15.133:0/1049671365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555615a9b570 0x555615a9ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.192+0000 7f980b7fe700 1 -- 172.21.15.133:0/1049671365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556158ed6b0 msgr2=0x555615a9a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.193+0000 7f980b7fe700 1 --2- 172.21.15.133:0/1049671365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556158ed6b0 0x555615a9a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.193+0000 7f980b7fe700 1 -- 172.21.15.133:0/1049671365 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556158645d0 con 0x555615a9a7c0 2023-11-27T15:27:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.193+0000 7f9810f9c700 1 --2- 172.21.15.133:0/1049671365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555615a9b570 0x555615a9ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:27:55.198 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.193+0000 7f980bfff700 1 --2- 172.21.15.133:0/1049671365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556158ed6b0 0x555615a9a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.193+0000 7f980b7fe700 1 --2- 172.21.15.133:0/1049671365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555615a9a7c0 0x555615a9b030 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9800015d20 tx=0x7f980001c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cbb360fdac998d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.193+0000 7f97f27fc700 1 -- 172.21.15.133:0/1049671365 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800015230 con 0x555615a9a7c0 2023-11-27T15:27:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.193+0000 7f97f27fc700 1 -- 172.21.15.133:0/1049671365 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9800004cf0 con 0x555615a9a7c0 2023-11-27T15:27:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.193+0000 7f97f27fc700 1 -- 172.21.15.133:0/1049671365 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800005480 con 0x555615a9a7c0 2023-11-27T15:27:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.194+0000 7f9826418b80 1 -- 172.21.15.133:0/1049671365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555615a9a7c0 msgr2=0x555615a9b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.194+0000 7f9826418b80 1 --2- 172.21.15.133:0/1049671365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555615a9a7c0 0x555615a9b030 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9800015d20 tx=0x7f980001c000 comp rx=0 tx=0).stop 2023-11-27T15:27:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.194+0000 7f9826418b80 1 -- 172.21.15.133:0/1049671365 shutdown_connections 2023-11-27T15:27:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.194+0000 7f9826418b80 1 --2- 172.21.15.133:0/1049671365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555615a9b570 0x555615a9ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.194+0000 7f9826418b80 1 --2- 172.21.15.133:0/1049671365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556158ed6b0 0x555615a9a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.194+0000 7f9826418b80 1 --2- 172.21.15.133:0/1049671365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555615a9a7c0 0x555615a9b030 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.194+0000 7f9826418b80 1 -- 172.21.15.133:0/1049671365 >> 172.21.15.133:0/1049671365 conn(0x555615a871a0 msgr2=0x555615a8c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:55.199 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.194+0000 7f9826418b80 1 -- 172.21.15.133:0/1049671365 shutdown_connections 2023-11-27T15:27:55.200 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.194+0000 7f9826418b80 1 -- 172.21.15.133:0/1049671365 wait complete. 2023-11-27T15:27:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.195+0000 7f9826418b80 1 -- start start 2023-11-27T15:27:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.195+0000 7f9826418b80 1 Processor -- start 2023-11-27T15:27:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.196+0000 7f9826418b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556158ed6b0 0x555615aa3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:55.201 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.196+0000 7f9826418b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555615a9a7c0 0x555615aa3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.196+0000 7f9826418b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555615a9b570 0x555615aaabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.196+0000 7f9826418b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555615984880 con 0x555615a9b570 2023-11-27T15:27:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.196+0000 7f980bfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556158ed6b0 0x555615aa3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.196+0000 7f980bfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556158ed6b0 0x555615aa3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:59812/0 (socket says 172.21.15.133:59812) 2023-11-27T15:27:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.196+0000 7f9826418b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x555615984520 con 0x555615a9a7c0 2023-11-27T15:27:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.196+0000 7f9826418b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556158668c0 con 0x5556158ed6b0 2023-11-27T15:27:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.196+0000 7f9826418b80 10 client.0 _ll_register_callbacks cb 0x5556159a2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.196+0000 7f980b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555615a9a7c0 0x555615aa3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.196+0000 7f980bfff700 1 -- 172.21.15.133:0/272421482 learned_addr learned my addr 172.21.15.133:0/272421482 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.196+0000 7f980b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555615a9a7c0 0x555615aa3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:43340/0 (socket says 172.21.15.133:43340) 2023-11-27T15:27:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.197+0000 7f980bfff700 1 -- 172.21.15.133:0/272421482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555615a9a7c0 msgr2=0x555615aa3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:55.202 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.197+0000 7f980bfff700 1 --2- 172.21.15.133:0/272421482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555615a9a7c0 0x555615aa3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.197+0000 7f980bfff700 1 -- 172.21.15.133:0/272421482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555615a9b570 msgr2=0x555615aaabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.197+0000 7f980bfff700 1 --2- 172.21.15.133:0/272421482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555615a9b570 0x555615aaabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.197+0000 7f980bfff700 1 -- 172.21.15.133:0/272421482 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980001b070 con 0x5556158ed6b0 2023-11-27T15:27:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.197+0000 7f980bfff700 1 --2- 172.21.15.133:0/272421482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556158ed6b0 0x555615aa3640 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f97fc005900 tx=0x7f97fc005930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=984b45870a5895c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.197+0000 7f97eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:27:55.203 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.198+0000 7f97eb7fe700 1 -- 172.21.15.133:0/272421482 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc028070 con 0x5556158ed6b0 2023-11-27T15:27:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.198+0000 7f97eb7fe700 1 -- 172.21.15.133:0/272421482 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f97fc004500 con 0x5556158ed6b0 2023-11-27T15:27:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.199+0000 7f9826418b80 1 -- 172.21.15.133:0/272421482 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555615a84960 con 0x5556158ed6b0 2023-11-27T15:27:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.199+0000 7f97eb7fe700 1 -- 172.21.15.133:0/272421482 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97fc01d070 con 0x5556158ed6b0 2023-11-27T15:27:55.204 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.199+0000 7f9826418b80 10 client.26410 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.199+0000 7f9826418b80 1 -- 172.21.15.133:0/272421482 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5556159a9130 con 0x5556158ed6b0 2023-11-27T15:27:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.199+0000 7f97eb7fe700 1 -- 172.21.15.133:0/272421482 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555615a84960 con 0x5556158ed6b0 2023-11-27T15:27:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.199+0000 7f97eb7fe700 1 -- 172.21.15.133:0/272421482 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f97fc02b020 con 0x5556158ed6b0 2023-11-27T15:27:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.200+0000 7f9826418b80 10 client.26410 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.200+0000 7f9826418b80 10 client.26410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.200+0000 7f9826418b80 20 client.26410 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.200+0000 7f9826418b80 1 --2- 172.21.15.133:0/272421482 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x555615aaf470 0x555615ab1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.200+0000 7f9826418b80 4 client.26410 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:27:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.200+0000 7f9826418b80 1 -- 172.21.15.133:0/272421482 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556159a5040 con 0x555615aaf470 2023-11-27T15:27:55.205 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.201+0000 7f980b7fe700 1 --2- 172.21.15.133:0/272421482 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x555615aaf470 0x555615ab1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.202+0000 7f980b7fe700 1 --2- 172.21.15.133:0/272421482 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x555615aaf470 0x555615ab1880 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c40ca13a222aa25a server_cookie=5141597e6db0f403 in_seq=0 out_seq=0 2023-11-27T15:27:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.202+0000 7f97eb7fe700 10 client.26410 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.203+0000 7f97eb7fe700 1 -- 172.21.15.133:0/272421482 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5556159a5040 con 0x555615aaf470 2023-11-27T15:27:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.203+0000 7f97eb7fe700 10 client.26410 handle_command_reply: tid=0 2023-11-27T15:27:55.206 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.203+0000 7f9826418b80 1 client.26410 shutdown 2023-11-27T15:27:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.203+0000 7f9826418b80 1 -- 172.21.15.133:0/272421482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556158ed6b0 msgr2=0x555615aa3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.203+0000 7f9826418b80 1 --2- 172.21.15.133:0/272421482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556158ed6b0 0x555615aa3640 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f97fc005900 tx=0x7f97fc005930 comp rx=0 tx=0).stop 2023-11-27T15:27:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.203+0000 7f9826418b80 1 -- 172.21.15.133:0/272421482 shutdown_connections 2023-11-27T15:27:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.203+0000 7f9826418b80 1 --2- 172.21.15.133:0/272421482 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x555615aaf470 0x555615ab1880 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.203+0000 7f9826418b80 1 --2- 172.21.15.133:0/272421482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555615a9b570 0x555615aaabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.203+0000 7f9826418b80 1 --2- 172.21.15.133:0/272421482 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5556158ed6b0 0x555615aa3640 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.203+0000 7f9826418b80 1 --2- 172.21.15.133:0/272421482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555615a9a7c0 0x555615aa3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.203+0000 7f9826418b80 1 -- 172.21.15.133:0/272421482 >> 172.21.15.133:0/272421482 conn(0x555615a871a0 msgr2=0x555615a8ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.203+0000 7f97eb7fe700 0 client.26410 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.203+0000 7f9826418b80 1 -- 172.21.15.133:0/272421482 shutdown_connections 2023-11-27T15:27:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.203+0000 7f9826418b80 1 -- 172.21.15.133:0/272421482 wait complete. 2023-11-27T15:27:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.204+0000 7f9826418b80 20 client.26410 trim_cache size 0 max 16384 2023-11-27T15:27:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.224+0000 7f9826418b80 1 Processor -- start 2023-11-27T15:27:55.228 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.224+0000 7f9826418b80 1 -- start start 2023-11-27T15:27:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.225+0000 7f9826418b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556158ed6b0 0x555615a9b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.225+0000 7f9826418b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555615a9a7c0 0x555615aaaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:55.229 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.225+0000 7f9826418b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555615a9b570 0x555615aa2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.225+0000 7f9826418b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555615a86dc0 con 0x555615a9a7c0 2023-11-27T15:27:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.225+0000 7f9826418b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x555615aac850 con 0x5556158ed6b0 2023-11-27T15:27:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.225+0000 7f9826418b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x555615a9be80 con 0x555615a9b570 2023-11-27T15:27:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.225+0000 7f9810f9c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555615a9b570 0x555615aa2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.225+0000 7f980b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555615a9a7c0 0x555615aaaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.225+0000 7f9810f9c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555615a9b570 0x555615aa2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:59822/0 (socket says 172.21.15.133:59822) 2023-11-27T15:27:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.225+0000 7f9810f9c700 1 -- 172.21.15.133:0/313160058 learned_addr learned my addr 172.21.15.133:0/313160058 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:55.230 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.225+0000 7f980bfff700 1 --2- 172.21.15.133:0/313160058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556158ed6b0 0x555615a9b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.225+0000 7f9810f9c700 1 -- 172.21.15.133:0/313160058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556158ed6b0 msgr2=0x555615a9b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.225+0000 7f9810f9c700 1 --2- 172.21.15.133:0/313160058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556158ed6b0 0x555615a9b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.225+0000 7f9810f9c700 1 -- 172.21.15.133:0/313160058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555615a9a7c0 msgr2=0x555615aaaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.225+0000 7f9810f9c700 1 --2- 172.21.15.133:0/313160058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555615a9a7c0 0x555615aaaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.225+0000 7f9810f9c700 1 -- 172.21.15.133:0/313160058 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556159a8e80 con 0x555615a9b570 2023-11-27T15:27:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.225+0000 7f980bfff700 1 --2- 172.21.15.133:0/313160058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556158ed6b0 0x555615a9b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:27:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.226+0000 7f9810f9c700 1 --2- 172.21.15.133:0/313160058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555615a9b570 0x555615aa2380 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f98040998d0 tx=0x7f98040092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=728afd8c3bededda server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:55.231 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.226+0000 7f97f2ffd700 1 -- 172.21.15.133:0/313160058 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98040992a0 con 0x555615a9b570 2023-11-27T15:27:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.226+0000 7f97f2ffd700 1 -- 172.21.15.133:0/313160058 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9804004ad0 con 0x555615a9b570 2023-11-27T15:27:55.233 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.226+0000 7f97f2ffd700 1 -- 172.21.15.133:0/313160058 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980409dbd0 con 0x555615a9b570 2023-11-27T15:27:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.227+0000 7f9826418b80 1 -- 172.21.15.133:0/313160058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555615a9b570 msgr2=0x555615aa2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.227+0000 7f9826418b80 1 --2- 172.21.15.133:0/313160058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555615a9b570 0x555615aa2380 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f98040998d0 tx=0x7f98040092e0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.227+0000 7f9826418b80 1 -- 172.21.15.133:0/313160058 shutdown_connections 2023-11-27T15:27:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.227+0000 7f9826418b80 1 --2- 172.21.15.133:0/313160058 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555615a9b570 0x555615aa2380 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.227+0000 7f9826418b80 1 --2- 172.21.15.133:0/313160058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556158ed6b0 0x555615a9b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.227+0000 7f9826418b80 1 --2- 172.21.15.133:0/313160058 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x555615a9a7c0 0x555615aaaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.227+0000 7f9826418b80 1 -- 172.21.15.133:0/313160058 >> 172.21.15.133:0/313160058 conn(0x555615a871a0 msgr2=0x555615ab61e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.227+0000 7f9826418b80 1 -- 172.21.15.133:0/313160058 shutdown_connections 2023-11-27T15:27:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.227+0000 7f9826418b80 1 -- 172.21.15.133:0/313160058 wait complete. 2023-11-27T15:27:55.234 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.228+0000 7f9826418b80 1 -- start start 2023-11-27T15:27:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.228+0000 7f9826418b80 1 Processor -- start 2023-11-27T15:27:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.229+0000 7f9826418b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556158ed6b0 0x555615aa6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.229+0000 7f9826418b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555615a9a7c0 0x555615aa7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.229+0000 7f9826418b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555615a9b570 0x555615aa9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.229+0000 7f9826418b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x555615aaa060 con 0x5556158ed6b0 2023-11-27T15:27:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.229+0000 7f9826418b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x555615aaa1d0 con 0x555615a9b570 2023-11-27T15:27:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.229+0000 7f9826418b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x555615aaa390 con 0x555615a9a7c0 2023-11-27T15:27:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.229+0000 7f9826418b80 10 client.0 _ll_register_callbacks cb 0x7f97fc00bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:27:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.229+0000 7f980b7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555615a9a7c0 0x555615aa7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.229+0000 7f980b7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555615a9a7c0 0x555615aa7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:59832/0 (socket says 172.21.15.133:59832) 2023-11-27T15:27:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.229+0000 7f980b7fe700 1 -- 172.21.15.133:0/1553183438 learned_addr learned my addr 172.21.15.133:0/1553183438 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:27:55.235 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.229+0000 7f9810f9c700 1 --2- 172.21.15.133:0/1553183438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555615a9b570 0x555615aa9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:55.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.229+0000 7f980bfff700 1 --2- 172.21.15.133:0/1553183438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556158ed6b0 0x555615aa6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:55.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.229+0000 7f980b7fe700 1 -- 172.21.15.133:0/1553183438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555615a9b570 msgr2=0x555615aa9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:55.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.229+0000 7f980b7fe700 1 --2- 172.21.15.133:0/1553183438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555615a9b570 0x555615aa9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.229+0000 7f980b7fe700 1 -- 172.21.15.133:0/1553183438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556158ed6b0 msgr2=0x555615aa6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:27:55.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.229+0000 7f980b7fe700 1 --2- 172.21.15.133:0/1553183438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556158ed6b0 0x555615aa6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:55.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.229+0000 7f980b7fe700 1 -- 172.21.15.133:0/1553183438 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980c10b9d0 con 0x555615a9a7c0 2023-11-27T15:27:55.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.229+0000 7f980bfff700 1 --2- 172.21.15.133:0/1553183438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556158ed6b0 0x555615aa6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:27:55.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.230+0000 7f980b7fe700 1 --2- 172.21.15.133:0/1553183438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555615a9a7c0 0x555615aa7250 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9800005f20 tx=0x7f9800009eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ee2b92cf24dfc87 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:27:55.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.230+0000 7f97eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:27:55.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.230+0000 7f97eb7fe700 1 -- 172.21.15.133:0/1553183438 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98000152c0 con 0x555615a9a7c0 2023-11-27T15:27:55.236 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.230+0000 7f97eb7fe700 1 -- 172.21.15.133:0/1553183438 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f980002f040 con 0x555615a9a7c0 2023-11-27T15:27:55.237 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.230+0000 7f9826418b80 1 -- 172.21.15.133:0/1553183438 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9800004cf0 con 0x555615a9a7c0 2023-11-27T15:27:55.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.231+0000 7f97eb7fe700 1 -- 172.21.15.133:0/1553183438 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9800004e60 con 0x555615a9a7c0 2023-11-27T15:27:55.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.231+0000 7f9826418b80 10 client.26416 fetch_fsmap learned FSMap version 27 2023-11-27T15:27:55.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.231+0000 7f9826418b80 1 -- 172.21.15.133:0/1553183438 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5556157f9c00 con 0x555615a9a7c0 2023-11-27T15:27:55.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.231+0000 7f97eb7fe700 1 -- 172.21.15.133:0/1553183438 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9800004cf0 con 0x555615a9a7c0 2023-11-27T15:27:55.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.231+0000 7f97eb7fe700 1 -- 172.21.15.133:0/1553183438 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 27) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f980003a060 con 0x555615a9a7c0 2023-11-27T15:27:55.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.231+0000 7f9826418b80 10 client.26416 fetch_fsmap finished waiting for FSMap version 27 2023-11-27T15:27:55.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.231+0000 7f9826418b80 10 client.26416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:27:55.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.231+0000 7f9826418b80 20 client.26416 populate_metadata read hostname 'smithi133' 2023-11-27T15:27:55.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.231+0000 7f9826418b80 1 --2- 172.21.15.133:0/1553183438 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x555615b040b0 0x555615b064c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:27:55.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.231+0000 7f9826418b80 4 client.26416 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:27:55.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.231+0000 7f9826418b80 1 -- 172.21.15.133:0/1553183438 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5556159b1600 con 0x555615b040b0 2023-11-27T15:27:55.238 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.232+0000 7f980bfff700 1 --2- 172.21.15.133:0/1553183438 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x555615b040b0 0x555615b064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:27:55.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.232+0000 7f97eb7fe700 10 client.26416 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:55.239 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:55.232+0000 7f980bfff700 1 --2- 172.21.15.133:0/1553183438 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x555615b040b0 0x555615b064c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39b7dd39c55defd server_cookie=f0c8502d9af23ec5 in_seq=0 out_seq=0 2023-11-27T15:27:55.276 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:54 smithi133 ceph-mon[97860]: pgmap v498: 129 pgs: 129 active+clean; 374 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 639 op/s 2023-11-27T15:27:55.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:54 smithi177 ceph-mon[131504]: pgmap v498: 129 pgs: 129 active+clean; 374 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 639 op/s 2023-11-27T15:27:55.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:54 smithi174 ceph-mon[131110]: pgmap v498: 129 pgs: 129 active+clean; 374 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 639 op/s 2023-11-27T15:27:56.871 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.867+0000 7f97eb7fe700 1 -- 172.21.15.133:0/1553183438 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5556159b1600 con 0x555615b040b0 2023-11-27T15:27:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.867+0000 7f97eb7fe700 10 client.26416 handle_command_reply: tid=0 2023-11-27T15:27:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.867+0000 7f9826418b80 1 client.26416 shutdown 2023-11-27T15:27:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.868+0000 7f9826418b80 1 -- 172.21.15.133:0/1553183438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555615a9a7c0 msgr2=0x555615aa7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.868+0000 7f9826418b80 1 --2- 172.21.15.133:0/1553183438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555615a9a7c0 0x555615aa7250 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9800005f20 tx=0x7f9800009eb0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.868+0000 7f9826418b80 1 -- 172.21.15.133:0/1553183438 shutdown_connections 2023-11-27T15:27:56.872 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.868+0000 7f9826418b80 1 --2- 172.21.15.133:0/1553183438 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x555615b040b0 0x555615b064c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.868+0000 7f9826418b80 1 --2- 172.21.15.133:0/1553183438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555615a9b570 0x555615aa9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.868+0000 7f9826418b80 1 --2- 172.21.15.133:0/1553183438 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5556158ed6b0 0x555615aa6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.868+0000 7f97eb7fe700 0 client.26416 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:27:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.868+0000 7f9826418b80 1 --2- 172.21.15.133:0/1553183438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x555615a9a7c0 0x555615aa7250 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.868+0000 7f9826418b80 1 -- 172.21.15.133:0/1553183438 >> 172.21.15.133:0/1553183438 conn(0x555615a871a0 msgr2=0x555615aa2380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.868+0000 7f9826418b80 1 -- 172.21.15.133:0/1553183438 shutdown_connections 2023-11-27T15:27:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.868+0000 7f9826418b80 1 -- 172.21.15.133:0/1553183438 wait complete. 2023-11-27T15:27:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.869+0000 7f9826418b80 20 client.26416 trim_cache size 0 max 16384 2023-11-27T15:27:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:27:56.873 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:27:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:27:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:27:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.870+0000 7f981179d700 1 -- 172.21.15.133:0/704618356 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f97f40a1250 msgr2=0x7f97f40a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.870+0000 7f981179d700 1 --2- 172.21.15.133:0/704618356 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f97f40a1250 0x7f97f40a3700 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9800015610 tx=0x7f9800015920 comp rx=0 tx=0).stop 2023-11-27T15:27:56.874 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.870+0000 7f981179d700 1 -- 172.21.15.133:0/704618356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f980c101bf0 msgr2=0x7f980c06f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:27:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.870+0000 7f981179d700 1 --2- 172.21.15.133:0/704618356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f980c101bf0 0x7f980c06f940 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f980400efc0 tx=0x7f980400c3a0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.870+0000 7f981179d700 1 -- 172.21.15.133:0/704618356 shutdown_connections 2023-11-27T15:27:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.870+0000 7f981179d700 1 --2- 172.21.15.133:0/704618356 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f97f40a1250 0x7f97f40a3700 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.870+0000 7f981179d700 1 --2- 172.21.15.133:0/704618356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f980c101bf0 0x7f980c06f940 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.870+0000 7f981179d700 1 --2- 172.21.15.133:0/704618356 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f980c100070 0x7f980c06f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.870+0000 7f981179d700 1 --2- 172.21.15.133:0/704618356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f980c101260 0x7f980c074920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:27:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.870+0000 7f981179d700 1 -- 172.21.15.133:0/704618356 >> 172.21.15.133:0/704618356 conn(0x7f980c0fb670 msgr2=0x7f980c104770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:27:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.870+0000 7f981179d700 1 -- 172.21.15.133:0/704618356 shutdown_connections 2023-11-27T15:27:56.875 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:27:56.870+0000 7f981179d700 1 -- 172.21.15.133:0/704618356 wait complete. 2023-11-27T15:27:56.956 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:27:56 smithi177 systemd[1]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.c.service: Service RestartSec=10s expired, scheduling restart. 2023-11-27T15:27:56.956 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:27:56 smithi177 systemd[1]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.c.service: Scheduled restart job, restart counter is at 1. 2023-11-27T15:27:56.957 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:27:56 smithi177 systemd[1]: Stopped Ceph mds.c for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:27:56.957 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:27:56 smithi177 systemd[1]: Starting Ceph mds.c for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:27:57.233 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:56 smithi177 ceph-mon[131504]: pgmap v499: 129 pgs: 129 active+clean; 376 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 870 op/s 2023-11-27T15:27:57.233 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:27:56 smithi177 podman[160593]: 2023-11-27T15:27:57.276 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:56 smithi133 ceph-mon[97860]: pgmap v499: 129 pgs: 129 active+clean; 376 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 870 op/s 2023-11-27T15:27:57.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:56 smithi174 ceph-mon[131110]: pgmap v499: 129 pgs: 129 active+clean; 376 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 870 op/s 2023-11-27T15:27:57.587 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:27:57 smithi177 bash[160593]: fb9624bc029e76fa8a2e57439ebf8254f7b54fa7cf36dafadae4916fbe700028 2023-11-27T15:27:57.588 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:27:57 smithi177 ceph-mds[160632]: -- 172.21.15.177:0/31694871 <== mon.1 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557d06ac5520 con 0x557d06a9ac00 2023-11-27T15:27:57.588 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:27:57 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-c[160608]: starting mds.c at 2023-11-27T15:27:57.588 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:27:57 smithi177 systemd[1]: Started Ceph mds.c for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:27:58.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:57 smithi133 ceph-mon[97860]: daemon mds.c restarted 2023-11-27T15:27:58.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:57 smithi133 ceph-mon[97860]: osdmap e93: 12 total, 12 up, 12 in 2023-11-27T15:27:58.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:57 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.177:6836/3131631796,v1:172.21.15.177:6837/3131631796] up:boot 2023-11-27T15:27:58.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:57 smithi133 ceph-mon[97860]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:27:58.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:57 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:27:58.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:57 smithi177 ceph-mon[131504]: daemon mds.c restarted 2023-11-27T15:27:58.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:57 smithi177 ceph-mon[131504]: osdmap e93: 12 total, 12 up, 12 in 2023-11-27T15:27:58.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:57 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.177:6836/3131631796,v1:172.21.15.177:6837/3131631796] up:boot 2023-11-27T15:27:58.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:57 smithi177 ceph-mon[131504]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:27:58.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:57 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:27:58.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:57 smithi174 ceph-mon[131110]: daemon mds.c restarted 2023-11-27T15:27:58.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:57 smithi174 ceph-mon[131110]: osdmap e93: 12 total, 12 up, 12 in 2023-11-27T15:27:58.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:57 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.177:6836/3131631796,v1:172.21.15.177:6837/3131631796] up:boot 2023-11-27T15:27:58.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:57 smithi174 ceph-mon[131110]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:27:58.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:57 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:27:59.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:27:58 smithi133 ceph-mon[97860]: pgmap v501: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 963 op/s 2023-11-27T15:27:59.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:27:58 smithi177 ceph-mon[131504]: pgmap v501: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 963 op/s 2023-11-27T15:27:59.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:27:58 smithi174 ceph-mon[131110]: pgmap v501: 129 pgs: 129 active+clean; 375 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 963 op/s 2023-11-27T15:28:01.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:00 smithi133 ceph-mon[97860]: pgmap v502: 129 pgs: 129 active+clean; 384 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 1.06k op/s 2023-11-27T15:28:01.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:00 smithi177 ceph-mon[131504]: pgmap v502: 129 pgs: 129 active+clean; 384 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 1.06k op/s 2023-11-27T15:28:01.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:00 smithi174 ceph-mon[131110]: pgmap v502: 129 pgs: 129 active+clean; 384 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 1.06k op/s 2023-11-27T15:28:03.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:02 smithi133 ceph-mon[97860]: pgmap v503: 129 pgs: 129 active+clean; 381 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 1.11k op/s 2023-11-27T15:28:03.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:02 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:28:03.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:02 smithi177 ceph-mon[131504]: pgmap v503: 129 pgs: 129 active+clean; 381 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 1.11k op/s 2023-11-27T15:28:03.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:02 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:28:03.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:02 smithi174 ceph-mon[131110]: pgmap v503: 129 pgs: 129 active+clean; 381 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 1.11k op/s 2023-11-27T15:28:03.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:02 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:28:05.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:04 smithi133 ceph-mon[97860]: pgmap v504: 129 pgs: 129 active+clean; 382 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 1.11k op/s 2023-11-27T15:28:05.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:04 smithi177 ceph-mon[131504]: pgmap v504: 129 pgs: 129 active+clean; 382 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 1.11k op/s 2023-11-27T15:28:05.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:04 smithi174 ceph-mon[131110]: pgmap v504: 129 pgs: 129 active+clean; 382 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 1.11k op/s 2023-11-27T15:28:07.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:06 smithi133 ceph-mon[97860]: pgmap v505: 129 pgs: 129 active+clean; 386 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 907 op/s 2023-11-27T15:28:07.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:06 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:28:07.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:06 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:28:07.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:06 smithi177 ceph-mon[131504]: pgmap v505: 129 pgs: 129 active+clean; 386 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 907 op/s 2023-11-27T15:28:07.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:06 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:28:07.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:06 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:28:07.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:06 smithi174 ceph-mon[131110]: pgmap v505: 129 pgs: 129 active+clean; 386 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 907 op/s 2023-11-27T15:28:07.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:06 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:28:07.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:06 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:28:09.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:08 smithi177 ceph-mon[131504]: pgmap v506: 129 pgs: 129 active+clean; 393 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 771 op/s 2023-11-27T15:28:09.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:08 smithi174 ceph-mon[131110]: pgmap v506: 129 pgs: 129 active+clean; 393 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 771 op/s 2023-11-27T15:28:09.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:08 smithi133 ceph-mon[97860]: pgmap v506: 129 pgs: 129 active+clean; 393 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 771 op/s 2023-11-27T15:28:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.039+0000 7f527d7ff700 1 -- 172.21.15.133:0/2692698998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5278101dc0 msgr2=0x7f5278108680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.039+0000 7f527d7ff700 1 --2- 172.21.15.133:0/2692698998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5278101dc0 0x7f5278108680 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f527000b0a0 tx=0x7f527000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.039+0000 7f527d7ff700 1 -- 172.21.15.133:0/2692698998 shutdown_connections 2023-11-27T15:28:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.039+0000 7f527d7ff700 1 --2- 172.21.15.133:0/2692698998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5278101dc0 0x7f5278108680 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.039+0000 7f527d7ff700 1 --2- 172.21.15.133:0/2692698998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5278100240 0x7f5278100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.039+0000 7f527d7ff700 1 --2- 172.21.15.133:0/2692698998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5278101430 0x7f5278101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.039+0000 7f527d7ff700 1 -- 172.21.15.133:0/2692698998 >> 172.21.15.133:0/2692698998 conn(0x7f52780fb840 msgr2=0x7f52780fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.039+0000 7f527d7ff700 1 -- 172.21.15.133:0/2692698998 shutdown_connections 2023-11-27T15:28:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.039+0000 7f527d7ff700 1 -- 172.21.15.133:0/2692698998 wait complete. 2023-11-27T15:28:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.041+0000 7f527d7ff700 1 Processor -- start 2023-11-27T15:28:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.041+0000 7f527d7ff700 1 -- start start 2023-11-27T15:28:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.041+0000 7f527d7ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5278100240 0x7f5278198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.042+0000 7f527d7ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5278101430 0x7f52781985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.042+0000 7f527d7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5278101dc0 0x7f527819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.042+0000 7f527d7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527819fc00 con 0x7f5278101dc0 2023-11-27T15:28:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.042+0000 7f527d7ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527819fd40 con 0x7f5278100240 2023-11-27T15:28:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.042+0000 7f527d7ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527819fed0 con 0x7f5278101430 2023-11-27T15:28:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.042+0000 7f5276ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5278100240 0x7f5278198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.042+0000 7f5276ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5278100240 0x7f5278198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:45538/0 (socket says 172.21.15.133:45538) 2023-11-27T15:28:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.042+0000 7f5276ffd700 1 -- 172.21.15.133:0/911394636 learned_addr learned my addr 172.21.15.133:0/911394636 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.042+0000 7f52767fc700 1 --2- 172.21.15.133:0/911394636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5278101430 0x7f52781985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.042+0000 7f52777fe700 1 --2- 172.21.15.133:0/911394636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5278101dc0 0x7f527819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.042+0000 7f5276ffd700 1 -- 172.21.15.133:0/911394636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5278101430 msgr2=0x7f52781985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.042+0000 7f5276ffd700 1 --2- 172.21.15.133:0/911394636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5278101430 0x7f52781985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.042+0000 7f5276ffd700 1 -- 172.21.15.133:0/911394636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5278101dc0 msgr2=0x7f527819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.042+0000 7f5276ffd700 1 --2- 172.21.15.133:0/911394636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5278101dc0 0x7f527819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.042+0000 7f5276ffd700 1 -- 172.21.15.133:0/911394636 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5268009520 con 0x7f5278100240 2023-11-27T15:28:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.042+0000 7f52767fc700 1 --2- 172.21.15.133:0/911394636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5278101430 0x7f52781985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.043+0000 7f5276ffd700 1 --2- 172.21.15.133:0/911394636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5278100240 0x7f5278198080 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f526800ea50 tx=0x7f526800ee10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.043+0000 7f525ffff700 1 -- 172.21.15.133:0/911394636 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526800cc90 con 0x7f5278100240 2023-11-27T15:28:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.043+0000 7f525ffff700 1 -- 172.21.15.133:0/911394636 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f526800cdf0 con 0x7f5278100240 2023-11-27T15:28:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.043+0000 7f527d7ff700 1 -- 172.21.15.133:0/911394636 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5270009d00 con 0x7f5278100240 2023-11-27T15:28:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.043+0000 7f525ffff700 1 -- 172.21.15.133:0/911394636 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52680187a0 con 0x7f5278100240 2023-11-27T15:28:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.043+0000 7f527d7ff700 1 -- 172.21.15.133:0/911394636 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52780749f0 con 0x7f5278100240 2023-11-27T15:28:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.047+0000 7f525ffff700 1 -- 172.21.15.133:0/911394636 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f5268010430 con 0x7f5278100240 2023-11-27T15:28:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.048+0000 7f52911f6b80 1 Processor -- start 2023-11-27T15:28:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.048+0000 7f525ffff700 1 --2- 172.21.15.133:0/911394636 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f52600a1150 0x7f52600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.049+0000 7f52911f6b80 1 -- start start 2023-11-27T15:28:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.049+0000 7f525ffff700 1 -- 172.21.15.133:0/911394636 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f5268014070 con 0x7f5278100240 2023-11-27T15:28:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.049+0000 7f52911f6b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771ab870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.049+0000 7f52911f6b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652771abdb0 0x5652771ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.049+0000 7f52911f6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771acb10 0x5652771acf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.049+0000 7f52777fe700 1 --2- 172.21.15.133:0/911394636 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f52600a1150 0x7f52600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.049+0000 7f52911f6b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652770b1000 con 0x5652771abdb0 2023-11-27T15:28:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.049+0000 7f52911f6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x565277103840 con 0x5652771acb10 2023-11-27T15:28:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.049+0000 7f52911f6b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652770fccd0 con 0x5652770076b0 2023-11-27T15:28:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.049+0000 7f5276ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.049+0000 7f52777fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771acb10 0x5652771acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.049+0000 7f5276ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771ab870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:36636/0 (socket says 172.21.15.133:36636) 2023-11-27T15:28:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.049+0000 7f5276ffd700 1 -- 172.21.15.133:0/279859568 learned_addr learned my addr 172.21.15.133:0/279859568 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.050+0000 7f52767fc700 1 --2- 172.21.15.133:0/279859568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652771abdb0 0x5652771ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.050+0000 7f52777fe700 1 -- 172.21.15.133:0/279859568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 msgr2=0x5652771ab870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.050+0000 7f52777fe700 1 --2- 172.21.15.133:0/279859568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771ab870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.054 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.050+0000 7f52777fe700 1 -- 172.21.15.133:0/279859568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652771abdb0 msgr2=0x5652771ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.050+0000 7f52777fe700 1 --2- 172.21.15.133:0/279859568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652771abdb0 0x5652771ac5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.050+0000 7f52777fe700 1 -- 172.21.15.133:0/279859568 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565276f7e5d0 con 0x5652771acb10 2023-11-27T15:28:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.050+0000 7f5276ffd700 1 --2- 172.21.15.133:0/279859568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771ab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:28:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.050+0000 7f52777fe700 1 --2- 172.21.15.133:0/911394636 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f52600a1150 0x7f52600a3600 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f52700062a0 tx=0x7f527000bab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.050+0000 7f52777fe700 1 --2- 172.21.15.133:0/279859568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771acb10 0x5652771acf80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5270009fd0 tx=0x7f5270024000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1e65f37ef5443ec server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.050+0000 7f525d7fa700 1 -- 172.21.15.133:0/279859568 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527000e070 con 0x5652771acb10 2023-11-27T15:28:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.051+0000 7f525d7fa700 1 -- 172.21.15.133:0/279859568 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5270007870 con 0x5652771acb10 2023-11-27T15:28:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.051+0000 7f525d7fa700 1 -- 172.21.15.133:0/279859568 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527002b820 con 0x5652771acb10 2023-11-27T15:28:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.052+0000 7f52911f6b80 1 -- 172.21.15.133:0/279859568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771acb10 msgr2=0x5652771acf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.052+0000 7f52911f6b80 1 --2- 172.21.15.133:0/279859568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771acb10 0x5652771acf80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5270009fd0 tx=0x7f5270024000 comp rx=0 tx=0).stop 2023-11-27T15:28:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.052+0000 7f52911f6b80 1 -- 172.21.15.133:0/279859568 shutdown_connections 2023-11-27T15:28:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.052+0000 7f52911f6b80 1 --2- 172.21.15.133:0/279859568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771acb10 0x5652771acf80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.052+0000 7f52911f6b80 1 --2- 172.21.15.133:0/279859568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771ab870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.052+0000 7f52911f6b80 1 --2- 172.21.15.133:0/279859568 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652771abdb0 0x5652771ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.052+0000 7f52911f6b80 1 -- 172.21.15.133:0/279859568 >> 172.21.15.133:0/279859568 conn(0x5652771a0f50 msgr2=0x5652771a6100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.052+0000 7f52911f6b80 1 -- 172.21.15.133:0/279859568 shutdown_connections 2023-11-27T15:28:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.052+0000 7f52911f6b80 1 -- 172.21.15.133:0/279859568 wait complete. 2023-11-27T15:28:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.052+0000 7f52911f6b80 1 -- start start 2023-11-27T15:28:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.053+0000 7f52911f6b80 1 Processor -- start 2023-11-27T15:28:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.053+0000 7f52911f6b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771bd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.053+0000 7f52911f6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771abdb0 0x5652771bd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.053+0000 7f52911f6b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652771acb10 0x5652771c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.053+0000 7f52911f6b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527709e880 con 0x5652771acb10 2023-11-27T15:28:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.053+0000 7f52911f6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527709e520 con 0x5652771abdb0 2023-11-27T15:28:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.053+0000 7f52911f6b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x565276f808c0 con 0x5652770076b0 2023-11-27T15:28:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.053+0000 7f52911f6b80 10 client.0 _ll_register_callbacks cb 0x5652770bc720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:10.059 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.053+0000 7f5276ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771bd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.053+0000 7f52767fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771abdb0 0x5652771bd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.053+0000 7f5276ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771bd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:36644/0 (socket says 172.21.15.133:36644) 2023-11-27T15:28:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.053+0000 7f5276ffd700 1 -- 172.21.15.133:0/1533137926 learned_addr learned my addr 172.21.15.133:0/1533137926 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.053+0000 7f52777fe700 1 --2- 172.21.15.133:0/1533137926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652771acb10 0x5652771c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.054+0000 7f5276ffd700 1 -- 172.21.15.133:0/1533137926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771abdb0 msgr2=0x5652771bd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.054+0000 7f5276ffd700 1 --2- 172.21.15.133:0/1533137926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771abdb0 0x5652771bd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.054+0000 7f5276ffd700 1 -- 172.21.15.133:0/1533137926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652771acb10 msgr2=0x5652771c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.054+0000 7f5276ffd700 1 --2- 172.21.15.133:0/1533137926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652771acb10 0x5652771c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.054+0000 7f5276ffd700 1 -- 172.21.15.133:0/1533137926 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5270027050 con 0x5652770076b0 2023-11-27T15:28:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.054+0000 7f52767fc700 1 --2- 172.21.15.133:0/1533137926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771abdb0 0x5652771bd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:28:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.054+0000 7f5276ffd700 1 --2- 172.21.15.133:0/1533137926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771bd490 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5268010d80 tx=0x7f526809a8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=237245b8b8b6387d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.054+0000 7f525aff5700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:28:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.054+0000 7f525aff5700 1 -- 172.21.15.133:0/1533137926 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52680cd150 con 0x5652770076b0 2023-11-27T15:28:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.054+0000 7f52911f6b80 1 -- 172.21.15.133:0/1533137926 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56527719e710 con 0x5652770076b0 2023-11-27T15:28:10.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.054+0000 7f525aff5700 1 -- 172.21.15.133:0/1533137926 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f526809e3e0 con 0x5652770076b0 2023-11-27T15:28:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.054+0000 7f525aff5700 1 -- 172.21.15.133:0/1533137926 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52680a2800 con 0x5652770076b0 2023-11-27T15:28:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.054+0000 7f52911f6b80 10 client.26434 fetch_fsmap learned FSMap version 28 2023-11-27T15:28:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.054+0000 7f52911f6b80 1 -- 172.21.15.133:0/1533137926 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5652770c3130 con 0x5652770076b0 2023-11-27T15:28:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.055+0000 7f525aff5700 1 -- 172.21.15.133:0/1533137926 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56527719e710 con 0x5652770076b0 2023-11-27T15:28:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.055+0000 7f525aff5700 1 -- 172.21.15.133:0/1533137926 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f526801eb90 con 0x5652770076b0 2023-11-27T15:28:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.055+0000 7f52911f6b80 10 client.26434 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:28:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.055+0000 7f52911f6b80 10 client.26434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:28:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.055+0000 7f52911f6b80 20 client.26434 populate_metadata read hostname 'smithi133' 2023-11-27T15:28:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.055+0000 7f52911f6b80 1 --2- 172.21.15.133:0/1533137926 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5652771c92f0 0x5652771cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.055+0000 7f52911f6b80 4 client.26434 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:28:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.055+0000 7f52911f6b80 1 -- 172.21.15.133:0/1533137926 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652770bf040 con 0x5652771c92f0 2023-11-27T15:28:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.055+0000 7f52767fc700 1 --2- 172.21.15.133:0/1533137926 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5652771c92f0 0x5652771cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.056+0000 7f525aff5700 10 client.26434 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:10.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.056+0000 7f52767fc700 1 --2- 172.21.15.133:0/1533137926 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5652771c92f0 0x5652771cb6d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b849f5af1e2513e server_cookie=da12fee3c1efca16 in_seq=0 out_seq=0 2023-11-27T15:28:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.059+0000 7f525aff5700 1 -- 172.21.15.133:0/1533137926 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5652770bf040 con 0x5652771c92f0 2023-11-27T15:28:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.059+0000 7f525aff5700 10 client.26434 handle_command_reply: tid=0 2023-11-27T15:28:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.059+0000 7f52911f6b80 1 client.26434 shutdown 2023-11-27T15:28:10.062 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.059+0000 7f52911f6b80 1 -- 172.21.15.133:0/1533137926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 msgr2=0x5652771bd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.059+0000 7f52911f6b80 1 --2- 172.21.15.133:0/1533137926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771bd490 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5268010d80 tx=0x7f526809a8f0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.059+0000 7f52911f6b80 1 -- 172.21.15.133:0/1533137926 shutdown_connections 2023-11-27T15:28:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.059+0000 7f52911f6b80 1 --2- 172.21.15.133:0/1533137926 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x5652771c92f0 0x5652771cb6d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.059+0000 7f52911f6b80 1 --2- 172.21.15.133:0/1533137926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652771acb10 0x5652771c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.063 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.059+0000 7f52911f6b80 1 --2- 172.21.15.133:0/1533137926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771bd490 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.059+0000 7f52911f6b80 1 --2- 172.21.15.133:0/1533137926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771abdb0 0x5652771bd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.059+0000 7f52911f6b80 1 -- 172.21.15.133:0/1533137926 >> 172.21.15.133:0/1533137926 conn(0x5652771a0f50 msgr2=0x5652771a57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.059+0000 7f525aff5700 0 client.26434 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.060+0000 7f52911f6b80 1 -- 172.21.15.133:0/1533137926 shutdown_connections 2023-11-27T15:28:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.060+0000 7f52911f6b80 1 -- 172.21.15.133:0/1533137926 wait complete. 2023-11-27T15:28:10.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.060+0000 7f52911f6b80 20 client.26434 trim_cache size 0 max 16384 2023-11-27T15:28:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.078+0000 7f52911f6b80 1 Processor -- start 2023-11-27T15:28:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.078+0000 7f52911f6b80 1 -- start start 2023-11-27T15:28:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.078+0000 7f52911f6b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.078+0000 7f52911f6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771abdb0 0x5652771c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.078+0000 7f52911f6b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652771acb10 0x5652771af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.078+0000 7f52911f6b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652771bc970 con 0x5652771acb10 2023-11-27T15:28:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.078+0000 7f52911f6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652771c66b0 con 0x5652771abdb0 2023-11-27T15:28:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.078+0000 7f52911f6b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652771a7be0 con 0x5652770076b0 2023-11-27T15:28:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.079+0000 7f5276ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.079+0000 7f5276ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771b2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:36646/0 (socket says 172.21.15.133:36646) 2023-11-27T15:28:10.083 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.079+0000 7f5276ffd700 1 -- 172.21.15.133:0/2171155537 learned_addr learned my addr 172.21.15.133:0/2171155537 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.079+0000 7f52767fc700 1 --2- 172.21.15.133:0/2171155537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771abdb0 0x5652771c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.079+0000 7f52777fe700 1 --2- 172.21.15.133:0/2171155537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652771acb10 0x5652771af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.079+0000 7f5276ffd700 1 -- 172.21.15.133:0/2171155537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771abdb0 msgr2=0x5652771c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.079+0000 7f5276ffd700 1 --2- 172.21.15.133:0/2171155537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771abdb0 0x5652771c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.079+0000 7f5276ffd700 1 -- 172.21.15.133:0/2171155537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652771acb10 msgr2=0x5652771af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.079+0000 7f5276ffd700 1 --2- 172.21.15.133:0/2171155537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652771acb10 0x5652771af9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.079+0000 7f5276ffd700 1 -- 172.21.15.133:0/2171155537 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652770c2e80 con 0x5652770076b0 2023-11-27T15:28:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.079+0000 7f5276ffd700 1 --2- 172.21.15.133:0/2171155537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771b2e90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f52680cae10 tx=0x7f526800cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4c2551831bae930 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.080+0000 7f525dffb700 1 -- 172.21.15.133:0/2171155537 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52680993b0 con 0x5652770076b0 2023-11-27T15:28:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.080+0000 7f525dffb700 1 -- 172.21.15.133:0/2171155537 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f526809e3e0 con 0x5652770076b0 2023-11-27T15:28:10.084 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.080+0000 7f525dffb700 1 -- 172.21.15.133:0/2171155537 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526801eb90 con 0x5652770076b0 2023-11-27T15:28:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.081+0000 7f52911f6b80 1 -- 172.21.15.133:0/2171155537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 msgr2=0x5652771b2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.081+0000 7f52911f6b80 1 --2- 172.21.15.133:0/2171155537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771b2e90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f52680cae10 tx=0x7f526800cf60 comp rx=0 tx=0).stop 2023-11-27T15:28:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.081+0000 7f52911f6b80 1 -- 172.21.15.133:0/2171155537 shutdown_connections 2023-11-27T15:28:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.081+0000 7f52911f6b80 1 --2- 172.21.15.133:0/2171155537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652771acb10 0x5652771af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.081+0000 7f52911f6b80 1 --2- 172.21.15.133:0/2171155537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771b2e90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.081+0000 7f52911f6b80 1 --2- 172.21.15.133:0/2171155537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771abdb0 0x5652771c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.081+0000 7f52911f6b80 1 -- 172.21.15.133:0/2171155537 >> 172.21.15.133:0/2171155537 conn(0x5652771a0f50 msgr2=0x5652771d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.081+0000 7f52911f6b80 1 -- 172.21.15.133:0/2171155537 shutdown_connections 2023-11-27T15:28:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.081+0000 7f52911f6b80 1 -- 172.21.15.133:0/2171155537 wait complete. 2023-11-27T15:28:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.081+0000 7f52911f6b80 1 -- start start 2023-11-27T15:28:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.081+0000 7f52911f6b80 1 Processor -- start 2023-11-27T15:28:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.082+0000 7f52911f6b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.082+0000 7f52911f6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771abdb0 0x565277219150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.082+0000 7f52911f6b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652771acb10 0x56527721b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:10.087 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.082+0000 7f52911f6b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527721bbe0 con 0x5652771acb10 2023-11-27T15:28:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.082+0000 7f52911f6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527721bd20 con 0x5652771abdb0 2023-11-27T15:28:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.082+0000 7f52911f6b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527721beb0 con 0x5652770076b0 2023-11-27T15:28:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.082+0000 7f52911f6b80 10 client.0 _ll_register_callbacks cb 0x7f5268099f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.082+0000 7f5276ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.082+0000 7f52767fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771abdb0 0x565277219150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.082+0000 7f52777fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652771acb10 0x56527721b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.082+0000 7f52767fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771abdb0 0x565277219150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:45568/0 (socket says 172.21.15.133:45568) 2023-11-27T15:28:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.082+0000 7f52767fc700 1 -- 172.21.15.133:0/1056434985 learned_addr learned my addr 172.21.15.133:0/1056434985 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.082+0000 7f5276ffd700 1 -- 172.21.15.133:0/1056434985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771abdb0 msgr2=0x565277219150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.082+0000 7f5276ffd700 1 --2- 172.21.15.133:0/1056434985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771abdb0 0x565277219150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.082+0000 7f5276ffd700 1 -- 172.21.15.133:0/1056434985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652771acb10 msgr2=0x56527721b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.082+0000 7f5276ffd700 1 --2- 172.21.15.133:0/1056434985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652771acb10 0x56527721b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.082+0000 7f5276ffd700 1 -- 172.21.15.133:0/1056434985 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652770c3130 con 0x5652770076b0 2023-11-27T15:28:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.083+0000 7f5276ffd700 1 --2- 172.21.15.133:0/1056434985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771c2920 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5652771a0a80 tx=0x7f526809a1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12a95cf20f87daf server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.083+0000 7f525aff5700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:28:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.083+0000 7f525aff5700 1 -- 172.21.15.133:0/1056434985 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52680a2800 con 0x5652770076b0 2023-11-27T15:28:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.083+0000 7f525aff5700 1 -- 172.21.15.133:0/1056434985 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f52680c4be0 con 0x5652770076b0 2023-11-27T15:28:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.083+0000 7f52911f6b80 1 -- 172.21.15.133:0/1056434985 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5270007870 con 0x5652770076b0 2023-11-27T15:28:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.083+0000 7f525aff5700 1 -- 172.21.15.133:0/1056434985 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52680c4da0 con 0x5652770076b0 2023-11-27T15:28:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.083+0000 7f52911f6b80 10 client.26440 fetch_fsmap learned FSMap version 28 2023-11-27T15:28:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.083+0000 7f52911f6b80 1 -- 172.21.15.133:0/1056434985 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x565276f13c00 con 0x5652770076b0 2023-11-27T15:28:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.083+0000 7f525aff5700 1 -- 172.21.15.133:0/1056434985 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5270007870 con 0x5652770076b0 2023-11-27T15:28:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.084+0000 7f525aff5700 1 -- 172.21.15.133:0/1056434985 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5268018900 con 0x5652770076b0 2023-11-27T15:28:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.084+0000 7f52911f6b80 10 client.26440 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:28:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.084+0000 7f52911f6b80 10 client.26440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:28:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.084+0000 7f52911f6b80 20 client.26440 populate_metadata read hostname 'smithi133' 2023-11-27T15:28:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.084+0000 7f52911f6b80 1 --2- 172.21.15.133:0/1056434985 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56527721d400 0x56527721f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.084+0000 7f52911f6b80 4 client.26440 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:28:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.084+0000 7f52911f6b80 1 -- 172.21.15.133:0/1056434985 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5652770cb600 con 0x56527721d400 2023-11-27T15:28:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.084+0000 7f52767fc700 1 --2- 172.21.15.133:0/1056434985 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56527721d400 0x56527721f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.084+0000 7f52767fc700 1 --2- 172.21.15.133:0/1056434985 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56527721d400 0x56527721f7e0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6db61f7be8f5482 server_cookie=9ab4cc4fbcd2695c in_seq=0 out_seq=0 2023-11-27T15:28:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:10.084+0000 7f525aff5700 10 client.26440 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:11.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:10 smithi133 ceph-mon[97860]: pgmap v507: 129 pgs: 129 active+clean; 395 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 729 op/s 2023-11-27T15:28:11.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:10 smithi177 ceph-mon[131504]: pgmap v507: 129 pgs: 129 active+clean; 395 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 729 op/s 2023-11-27T15:28:11.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:10 smithi174 ceph-mon[131110]: pgmap v507: 129 pgs: 129 active+clean; 395 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 729 op/s 2023-11-27T15:28:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.762+0000 7f525aff5700 1 -- 172.21.15.133:0/1056434985 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5652770cb600 con 0x56527721d400 2023-11-27T15:28:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.762+0000 7f525aff5700 10 client.26440 handle_command_reply: tid=0 2023-11-27T15:28:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.762+0000 7f52911f6b80 1 client.26440 shutdown 2023-11-27T15:28:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.762+0000 7f52911f6b80 1 -- 172.21.15.133:0/1056434985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 msgr2=0x5652771c2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.762+0000 7f52911f6b80 1 --2- 172.21.15.133:0/1056434985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771c2920 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5652771a0a80 tx=0x7f526809a1f0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.763+0000 7f52911f6b80 1 -- 172.21.15.133:0/1056434985 shutdown_connections 2023-11-27T15:28:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.763+0000 7f52911f6b80 1 --2- 172.21.15.133:0/1056434985 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x56527721d400 0x56527721f7e0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.763+0000 7f52911f6b80 1 --2- 172.21.15.133:0/1056434985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5652771acb10 0x56527721b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.763+0000 7f52911f6b80 1 --2- 172.21.15.133:0/1056434985 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652770076b0 0x5652771c2920 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.763+0000 7f525aff5700 0 client.26440 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.763+0000 7f52911f6b80 1 --2- 172.21.15.133:0/1056434985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5652771abdb0 0x565277219150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.763+0000 7f52911f6b80 1 -- 172.21.15.133:0/1056434985 >> 172.21.15.133:0/1056434985 conn(0x5652771a0f50 msgr2=0x5652771c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.764+0000 7f52911f6b80 1 -- 172.21.15.133:0/1056434985 shutdown_connections 2023-11-27T15:28:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.764+0000 7f52911f6b80 1 -- 172.21.15.133:0/1056434985 wait complete. 2023-11-27T15:28:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.764+0000 7f52911f6b80 20 client.26440 trim_cache size 0 max 16384 2023-11-27T15:28:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:28:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:28:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:28:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:28:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.766+0000 7f527d7ff700 1 -- 172.21.15.133:0/911394636 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f52600a1150 msgr2=0x7f52600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.766+0000 7f527d7ff700 1 --2- 172.21.15.133:0/911394636 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f52600a1150 0x7f52600a3600 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f52700062a0 tx=0x7f527000bab0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.766+0000 7f527d7ff700 1 -- 172.21.15.133:0/911394636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5278100240 msgr2=0x7f5278198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.766+0000 7f527d7ff700 1 --2- 172.21.15.133:0/911394636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5278100240 0x7f5278198080 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f526800ea50 tx=0x7f526800ee10 comp rx=0 tx=0).stop 2023-11-27T15:28:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.768+0000 7f527d7ff700 1 -- 172.21.15.133:0/911394636 shutdown_connections 2023-11-27T15:28:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.768+0000 7f527d7ff700 1 --2- 172.21.15.133:0/911394636 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f52600a1150 0x7f52600a3600 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.768+0000 7f527d7ff700 1 --2- 172.21.15.133:0/911394636 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5278101dc0 0x7f527819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.768+0000 7f527d7ff700 1 --2- 172.21.15.133:0/911394636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5278100240 0x7f5278198080 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.768+0000 7f527d7ff700 1 --2- 172.21.15.133:0/911394636 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5278101430 0x7f52781985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.768+0000 7f527d7ff700 1 -- 172.21.15.133:0/911394636 >> 172.21.15.133:0/911394636 conn(0x7f52780fb840 msgr2=0x7f52781048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:11.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.768+0000 7f527d7ff700 1 -- 172.21.15.133:0/911394636 shutdown_connections 2023-11-27T15:28:11.772 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:11.769+0000 7f527d7ff700 1 -- 172.21.15.133:0/911394636 wait complete. 2023-11-27T15:28:13.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:12 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:28:13.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:12 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:28:13.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:12 smithi133 ceph-mon[97860]: pgmap v508: 129 pgs: 129 active+clean; 402 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 617 op/s 2023-11-27T15:28:13.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:12 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:28:13.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:12 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:28:13.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:12 smithi177 ceph-mon[131504]: pgmap v508: 129 pgs: 129 active+clean; 402 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 617 op/s 2023-11-27T15:28:13.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:12 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:28:13.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:12 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:28:13.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:12 smithi174 ceph-mon[131110]: pgmap v508: 129 pgs: 129 active+clean; 402 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 617 op/s 2023-11-27T15:28:14.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:13 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:28:14.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:13 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:28:14.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:13 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:28:14.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:13 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:28:14.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:13 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:28:14.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:13 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:28:15.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:14 smithi133 ceph-mon[97860]: pgmap v509: 129 pgs: 129 active+clean; 405 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 553 op/s 2023-11-27T15:28:15.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:14 smithi177 ceph-mon[131504]: pgmap v509: 129 pgs: 129 active+clean; 405 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 553 op/s 2023-11-27T15:28:15.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:14 smithi174 ceph-mon[131110]: pgmap v509: 129 pgs: 129 active+clean; 405 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 553 op/s 2023-11-27T15:28:15.838 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:28:15 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-c[160608]: *** Caught signal (Segmentation fault) ** 2023-11-27T15:28:15.838 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:28:15 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-c[160608]: in thread 7fb074413700 thread_name:MR_Finisher 2023-11-27T15:28:16.337 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:28:15 smithi177 systemd[1]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.c.service: Main process exited, code=exited, status=139/n/a 2023-11-27T15:28:16.338 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:28:16 smithi177 systemd[1]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.c.service: Failed with result 'exit-code'. 2023-11-27T15:28:17.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:16 smithi133 ceph-mon[97860]: pgmap v510: 129 pgs: 129 active+clean; 397 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 551 op/s 2023-11-27T15:28:17.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:16 smithi177 ceph-mon[131504]: pgmap v510: 129 pgs: 129 active+clean; 397 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 551 op/s 2023-11-27T15:28:17.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:16 smithi174 ceph-mon[131110]: pgmap v510: 129 pgs: 129 active+clean; 397 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 551 op/s 2023-11-27T15:28:19.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:18 smithi177 ceph-mon[131504]: pgmap v511: 129 pgs: 129 active+clean; 403 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 450 op/s 2023-11-27T15:28:19.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:18 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:28:19.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:18 smithi174 ceph-mon[131110]: pgmap v511: 129 pgs: 129 active+clean; 403 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 450 op/s 2023-11-27T15:28:19.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:18 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:28:19.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:18 smithi133 ceph-mon[97860]: pgmap v511: 129 pgs: 129 active+clean; 403 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 450 op/s 2023-11-27T15:28:19.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:18 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:28:20.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:28:20.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:28:20.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:19 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:28:20.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:28:20.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:19 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:28:20.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:19 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:28:20.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:19 smithi177 ceph-mon[131504]: Service mds.all has no related daemon already placed. 2023-11-27T15:28:20.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:28:20.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:28:20.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:19 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:28:20.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:28:20.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:19 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:28:20.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:19 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:28:20.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:19 smithi174 ceph-mon[131110]: Service mds.all has no related daemon already placed. 2023-11-27T15:28:20.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:28:20.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:28:20.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:19 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:28:20.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:28:20.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:19 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:28:20.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:19 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:28:20.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:19 smithi133 ceph-mon[97860]: Service mds.all has no related daemon already placed. 2023-11-27T15:28:21.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:20 smithi177 ceph-mon[131504]: scrub summary: active paths [/] 2023-11-27T15:28:21.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:20 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:28:21.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:20 smithi177 ceph-mon[131504]: pgmap v512: 129 pgs: 129 active+clean; 407 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 424 op/s 2023-11-27T15:28:21.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:20 smithi174 ceph-mon[131110]: scrub summary: active paths [/] 2023-11-27T15:28:21.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:20 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:28:21.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:20 smithi174 ceph-mon[131110]: pgmap v512: 129 pgs: 129 active+clean; 407 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 424 op/s 2023-11-27T15:28:21.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:20 smithi133 ceph-mon[97860]: scrub summary: active paths [/] 2023-11-27T15:28:21.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:20 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:28:21.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:20 smithi133 ceph-mon[97860]: pgmap v512: 129 pgs: 129 active+clean; 407 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 424 op/s 2023-11-27T15:28:22.483 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:28:22.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.748+0000 7feac3fff700 1 -- 172.21.15.133:0/3337697537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feabc0ffe30 msgr2=0x7feabc1002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:22.751 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.748+0000 7feac3fff700 1 --2- 172.21.15.133:0/3337697537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feabc0ffe30 0x7feabc1002a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7feab80098f0 tx=0x7feab8009c00 comp rx=0 tx=0).stop 2023-11-27T15:28:22.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.748+0000 7feac3fff700 1 -- 172.21.15.133:0/3337697537 shutdown_connections 2023-11-27T15:28:22.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.748+0000 7feac3fff700 1 --2- 172.21.15.133:0/3337697537 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feabc0ffe30 0x7feabc1002a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.748+0000 7feac3fff700 1 --2- 172.21.15.133:0/3337697537 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feabc0feb30 0x7feabc0fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.748+0000 7feac3fff700 1 --2- 172.21.15.133:0/3337697537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feabc0ff480 0x7feabc0ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.752 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.748+0000 7feac3fff700 1 -- 172.21.15.133:0/3337697537 >> 172.21.15.133:0/3337697537 conn(0x7feabc0fa7c0 msgr2=0x7feabc0fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:22.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.749+0000 7feac3fff700 1 -- 172.21.15.133:0/3337697537 shutdown_connections 2023-11-27T15:28:22.753 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.749+0000 7feac3fff700 1 -- 172.21.15.133:0/3337697537 wait complete. 2023-11-27T15:28:22.755 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.752+0000 7feac3fff700 1 Processor -- start 2023-11-27T15:28:22.756 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.753+0000 7feac3fff700 1 -- start start 2023-11-27T15:28:22.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.753+0000 7feac3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feabc0feb30 0x7feabc198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:22.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.753+0000 7feac3fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feabc0ff480 0x7feabc1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:22.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.753+0000 7feac3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feabc0ffe30 0x7feabc19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:22.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.753+0000 7feac3fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc19fe10 con 0x7feabc0feb30 2023-11-27T15:28:22.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.753+0000 7feac3fff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc06a510 con 0x7feabc0ffe30 2023-11-27T15:28:22.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.753+0000 7feac3fff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc06a6a0 con 0x7feabc0ff480 2023-11-27T15:28:22.757 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.753+0000 7feac1d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feabc0feb30 0x7feabc198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:22.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.753+0000 7feac1d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feabc0feb30 0x7feabc198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:34478/0 (socket says 172.21.15.133:34478) 2023-11-27T15:28:22.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.753+0000 7feac1d9b700 1 -- 172.21.15.133:0/2091267925 learned_addr learned my addr 172.21.15.133:0/2091267925 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:22.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.754+0000 7feac159a700 1 --2- 172.21.15.133:0/2091267925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feabc0ff480 0x7feabc1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:22.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.754+0000 7feac259c700 1 --2- 172.21.15.133:0/2091267925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feabc0ffe30 0x7feabc19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:22.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.754+0000 7feac1d9b700 1 -- 172.21.15.133:0/2091267925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feabc0ff480 msgr2=0x7feabc1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:22.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.754+0000 7feac1d9b700 1 --2- 172.21.15.133:0/2091267925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feabc0ff480 0x7feabc1987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.758 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.754+0000 7feac1d9b700 1 -- 172.21.15.133:0/2091267925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feabc0ffe30 msgr2=0x7feabc19f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:22.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.754+0000 7feac1d9b700 1 --2- 172.21.15.133:0/2091267925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feabc0ffe30 0x7feabc19f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.754+0000 7feac1d9b700 1 -- 172.21.15.133:0/2091267925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab8009520 con 0x7feabc0feb30 2023-11-27T15:28:22.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.754+0000 7feac259c700 1 --2- 172.21.15.133:0/2091267925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feabc0ffe30 0x7feabc19f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:28:22.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.754+0000 7feac159a700 1 --2- 172.21.15.133:0/2091267925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feabc0ff480 0x7feabc1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:28:22.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.754+0000 7feac1d9b700 1 --2- 172.21.15.133:0/2091267925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feabc0feb30 0x7feabc198290 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7feab4007fa0 tx=0x7feab400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:22.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.755+0000 7feab2ffd700 1 -- 172.21.15.133:0/2091267925 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab400de60 con 0x7feabc0feb30 2023-11-27T15:28:22.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.755+0000 7feab2ffd700 1 -- 172.21.15.133:0/2091267925 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feab400f040 con 0x7feabc0feb30 2023-11-27T15:28:22.759 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.755+0000 7feab2ffd700 1 -- 172.21.15.133:0/2091267925 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4004510 con 0x7feabc0feb30 2023-11-27T15:28:22.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.755+0000 7feac3fff700 1 -- 172.21.15.133:0/2091267925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feabc10de50 con 0x7feabc0feb30 2023-11-27T15:28:22.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.756+0000 7feac3fff700 1 -- 172.21.15.133:0/2091267925 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feabc10dcc0 con 0x7feabc0feb30 2023-11-27T15:28:22.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.762+0000 7feab2ffd700 1 -- 172.21.15.133:0/2091267925 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7feab4004ad0 con 0x7feabc0feb30 2023-11-27T15:28:22.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.766+0000 7fead78a1b80 1 Processor -- start 2023-11-27T15:28:22.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.768+0000 7fead78a1b80 1 -- start start 2023-11-27T15:28:22.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.769+0000 7fead78a1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38d4a0 0x55936e38d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:22.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.769+0000 7fead78a1b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38e340 0x55936e38e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:22.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.769+0000 7fead78a1b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55936e38ecf0 0x55936e395590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:22.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.769+0000 7fead78a1b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936e28a060 con 0x55936e38ecf0 2023-11-27T15:28:22.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.769+0000 7fead78a1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936e2dc8a0 con 0x55936e38d4a0 2023-11-27T15:28:22.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.769+0000 7fead78a1b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936e2d5d30 con 0x55936e38e340 2023-11-27T15:28:22.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.769+0000 7feac1d9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38e340 0x55936e38e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:22.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.769+0000 7feab2ffd700 1 --2- 172.21.15.133:0/2091267925 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7feaa80a12a0 0x7feaa80a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:22.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.769+0000 7feac1d9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38e340 0x55936e38e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:60538/0 (socket says 172.21.15.133:60538) 2023-11-27T15:28:22.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.769+0000 7feac1d9b700 1 -- 172.21.15.133:0/1203145185 learned_addr learned my addr 172.21.15.133:0/1203145185 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:22.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.769+0000 7feac159a700 1 --2- 172.21.15.133:0/1203145185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55936e38ecf0 0x55936e395590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:22.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.770+0000 7feac259c700 1 --2- 172.21.15.133:0/1203145185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38d4a0 0x55936e38d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:22.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.771+0000 7feac1d9b700 1 -- 172.21.15.133:0/1203145185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38d4a0 msgr2=0x55936e38d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:22.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.771+0000 7feac1d9b700 1 --2- 172.21.15.133:0/1203145185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38d4a0 0x55936e38d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.771+0000 7feac1d9b700 1 -- 172.21.15.133:0/1203145185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55936e38ecf0 msgr2=0x55936e395590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:22.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.771+0000 7feac1d9b700 1 --2- 172.21.15.133:0/1203145185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55936e38ecf0 0x55936e395590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.771+0000 7feac1d9b700 1 -- 172.21.15.133:0/1203145185 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55936e157630 con 0x55936e38e340 2023-11-27T15:28:22.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.771+0000 7feac1d9b700 1 --2- 172.21.15.133:0/1203145185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38e340 0x55936e38e7b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feab409d9a0 tx=0x7feab4099240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5baab0ced7d26d76 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:22.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.772+0000 7feac259c700 1 --2- 172.21.15.133:0/2091267925 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7feaa80a12a0 0x7feaa80a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:22.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.772+0000 7feaa7fff700 1 -- 172.21.15.133:0/1203145185 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40a1c00 con 0x55936e38e340 2023-11-27T15:28:22.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.772+0000 7feaa7fff700 1 -- 172.21.15.133:0/1203145185 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feab4003d90 con 0x55936e38e340 2023-11-27T15:28:22.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.772+0000 7feab2ffd700 1 -- 172.21.15.133:0/2091267925 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7feab40cc6f0 con 0x7feabc0feb30 2023-11-27T15:28:22.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.772+0000 7feaa7fff700 1 -- 172.21.15.133:0/1203145185 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4004690 con 0x55936e38e340 2023-11-27T15:28:22.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.773+0000 7fead78a1b80 1 -- 172.21.15.133:0/1203145185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38e340 msgr2=0x55936e38e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:22.777 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.773+0000 7fead78a1b80 1 --2- 172.21.15.133:0/1203145185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38e340 0x55936e38e7b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feab409d9a0 tx=0x7feab4099240 comp rx=0 tx=0).stop 2023-11-27T15:28:22.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.774+0000 7fead78a1b80 1 -- 172.21.15.133:0/1203145185 shutdown_connections 2023-11-27T15:28:22.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.774+0000 7fead78a1b80 1 --2- 172.21.15.133:0/1203145185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55936e38ecf0 0x55936e395590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.774+0000 7fead78a1b80 1 --2- 172.21.15.133:0/1203145185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38d4a0 0x55936e38d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.774+0000 7fead78a1b80 1 --2- 172.21.15.133:0/1203145185 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38e340 0x55936e38e7b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.774+0000 7fead78a1b80 1 -- 172.21.15.133:0/1203145185 >> 172.21.15.133:0/1203145185 conn(0x55936e37a360 msgr2=0x55936e37f680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:22.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.774+0000 7feac259c700 1 --2- 172.21.15.133:0/2091267925 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7feaa80a12a0 0x7feaa80a3750 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feab800b2e0 tx=0x7feab801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:22.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.775+0000 7fead78a1b80 1 -- 172.21.15.133:0/1203145185 shutdown_connections 2023-11-27T15:28:22.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.775+0000 7fead78a1b80 1 -- 172.21.15.133:0/1203145185 wait complete. 2023-11-27T15:28:22.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.776+0000 7fead78a1b80 1 -- start start 2023-11-27T15:28:22.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.776+0000 7fead78a1b80 1 Processor -- start 2023-11-27T15:28:22.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.777+0000 7fead78a1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38d4a0 0x55936e396e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:22.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.777+0000 7fead78a1b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38e340 0x55936e397390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:22.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.777+0000 7fead78a1b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55936e38ecf0 0x55936e39e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:22.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.777+0000 7fead78a1b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936e2778e0 con 0x55936e38ecf0 2023-11-27T15:28:22.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.777+0000 7fead78a1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936e277580 con 0x55936e38d4a0 2023-11-27T15:28:22.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.777+0000 7fead78a1b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936e159920 con 0x55936e38e340 2023-11-27T15:28:22.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.777+0000 7fead78a1b80 10 client.0 _ll_register_callbacks cb 0x55936e295780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:22.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.777+0000 7feac1d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38d4a0 0x55936e396e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:22.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.777+0000 7feac1d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38d4a0 0x55936e396e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:55742/0 (socket says 172.21.15.133:55742) 2023-11-27T15:28:22.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.777+0000 7feac1d9b700 1 -- 172.21.15.133:0/1247373075 learned_addr learned my addr 172.21.15.133:0/1247373075 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:22.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.778+0000 7feac1d9b700 1 -- 172.21.15.133:0/1247373075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38e340 msgr2=0x55936e397390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-27T15:28:22.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.778+0000 7feac1d9b700 1 --2- 172.21.15.133:0/1247373075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38e340 0x55936e397390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.778+0000 7feac1d9b700 1 -- 172.21.15.133:0/1247373075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55936e38ecf0 msgr2=0x55936e39e3d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-27T15:28:22.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.778+0000 7feac1d9b700 1 --2- 172.21.15.133:0/1247373075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55936e38ecf0 0x55936e39e3d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.778+0000 7feac1d9b700 1 -- 172.21.15.133:0/1247373075 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feabc10dcc0 con 0x55936e38d4a0 2023-11-27T15:28:22.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.778+0000 7feac1d9b700 1 --2- 172.21.15.133:0/1247373075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38d4a0 0x55936e396e50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7feab4095cb0 tx=0x7feab409d910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ce45c9f956ac0ab server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:22.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.778+0000 7feaa57fa700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:28:22.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.778+0000 7feaa57fa700 1 -- 172.21.15.133:0/1247373075 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40a1c00 con 0x55936e38d4a0 2023-11-27T15:28:22.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.779+0000 7fead78a1b80 1 -- 172.21.15.133:0/1247373075 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55936e377b20 con 0x55936e38d4a0 2023-11-27T15:28:22.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.779+0000 7feaa57fa700 1 -- 172.21.15.133:0/1247373075 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7feab40b3060 con 0x55936e38d4a0 2023-11-27T15:28:22.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.780+0000 7feaa57fa700 1 -- 172.21.15.133:0/1247373075 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7feab4003a00 con 0x55936e38d4a0 2023-11-27T15:28:22.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.780+0000 7fead78a1b80 10 client.26468 fetch_fsmap learned FSMap version 28 2023-11-27T15:28:22.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.780+0000 7fead78a1b80 1 -- 172.21.15.133:0/1247373075 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55936e29c190 con 0x55936e38d4a0 2023-11-27T15:28:22.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.781+0000 7feaa57fa700 1 -- 172.21.15.133:0/1247373075 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55936e377b20 con 0x55936e38d4a0 2023-11-27T15:28:22.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.781+0000 7feaa57fa700 1 -- 172.21.15.133:0/1247373075 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7feab4098ca0 con 0x55936e38d4a0 2023-11-27T15:28:22.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.781+0000 7fead78a1b80 10 client.26468 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:28:22.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.781+0000 7fead78a1b80 10 client.26468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:28:22.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.781+0000 7fead78a1b80 20 client.26468 populate_metadata read hostname 'smithi133' 2023-11-27T15:28:22.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.781+0000 7fead78a1b80 1 --2- 172.21.15.133:0/1247373075 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55936e3a27f0 0x55936e3a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:22.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.781+0000 7fead78a1b80 4 client.26468 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:28:22.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.781+0000 7fead78a1b80 1 -- 172.21.15.133:0/1247373075 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55936e2980a0 con 0x55936e3a27f0 2023-11-27T15:28:22.785 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.782+0000 7feac159a700 1 --2- 172.21.15.133:0/1247373075 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55936e3a27f0 0x55936e3a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:22.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.786+0000 7feaa57fa700 10 client.26468 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:22.789 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.786+0000 7feac159a700 1 --2- 172.21.15.133:0/1247373075 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55936e3a27f0 0x55936e3a4c00 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de949563df68cfb5 server_cookie=2a471674990ac9d7 in_seq=0 out_seq=0 2023-11-27T15:28:22.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.787+0000 7feaa57fa700 1 -- 172.21.15.133:0/1247373075 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55936e2980a0 con 0x55936e3a27f0 2023-11-27T15:28:22.790 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.787+0000 7feaa57fa700 10 client.26468 handle_command_reply: tid=0 2023-11-27T15:28:22.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.787+0000 7fead78a1b80 1 client.26468 shutdown 2023-11-27T15:28:22.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.788+0000 7fead78a1b80 1 -- 172.21.15.133:0/1247373075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38d4a0 msgr2=0x55936e396e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:22.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.788+0000 7fead78a1b80 1 --2- 172.21.15.133:0/1247373075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38d4a0 0x55936e396e50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7feab4095cb0 tx=0x7feab409d910 comp rx=0 tx=0).stop 2023-11-27T15:28:22.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.788+0000 7fead78a1b80 1 -- 172.21.15.133:0/1247373075 shutdown_connections 2023-11-27T15:28:22.791 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.788+0000 7fead78a1b80 1 --2- 172.21.15.133:0/1247373075 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55936e3a27f0 0x55936e3a4c00 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.788+0000 7fead78a1b80 1 --2- 172.21.15.133:0/1247373075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55936e38ecf0 0x55936e39e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.788+0000 7fead78a1b80 1 --2- 172.21.15.133:0/1247373075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38d4a0 0x55936e396e50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.788+0000 7fead78a1b80 1 --2- 172.21.15.133:0/1247373075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38e340 0x55936e397390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.788+0000 7fead78a1b80 1 -- 172.21.15.133:0/1247373075 >> 172.21.15.133:0/1247373075 conn(0x55936e37a360 msgr2=0x55936e37ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:22.792 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.788+0000 7feaa57fa700 0 client.26468 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:22.793 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.789+0000 7fead78a1b80 1 -- 172.21.15.133:0/1247373075 shutdown_connections 2023-11-27T15:28:22.793 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.789+0000 7fead78a1b80 1 -- 172.21.15.133:0/1247373075 wait complete. 2023-11-27T15:28:22.793 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.789+0000 7fead78a1b80 20 client.26468 trim_cache size 0 max 16384 2023-11-27T15:28:22.824 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.821+0000 7fead78a1b80 1 Processor -- start 2023-11-27T15:28:22.825 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.821+0000 7fead78a1b80 1 -- start start 2023-11-27T15:28:22.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.822+0000 7fead78a1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38d4a0 0x55936e391c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:22.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.822+0000 7fead78a1b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55936e38e340 0x55936e3921a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:22.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.822+0000 7fead78a1b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38ecf0 0x55936e39e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:22.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.822+0000 7fead78a1b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936e2980a0 con 0x55936e38e340 2023-11-27T15:28:22.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.822+0000 7fead78a1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936e396300 con 0x55936e38d4a0 2023-11-27T15:28:22.826 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.822+0000 7fead78a1b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936e38dd10 con 0x55936e38ecf0 2023-11-27T15:28:22.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.822+0000 7feac1d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38d4a0 0x55936e391c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:22.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.822+0000 7feac1d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38d4a0 0x55936e391c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:55746/0 (socket says 172.21.15.133:55746) 2023-11-27T15:28:22.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.822+0000 7feac1d9b700 1 -- 172.21.15.133:0/1453269037 learned_addr learned my addr 172.21.15.133:0/1453269037 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:22.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.823+0000 7feac1d9b700 1 -- 172.21.15.133:0/1453269037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38ecf0 msgr2=0x55936e39e390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-27T15:28:22.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.823+0000 7feac1d9b700 1 --2- 172.21.15.133:0/1453269037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38ecf0 0x55936e39e390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.823+0000 7feac1d9b700 1 -- 172.21.15.133:0/1453269037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55936e38e340 msgr2=0x55936e3921a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:22.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.823+0000 7feac1d9b700 1 --2- 172.21.15.133:0/1453269037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55936e38e340 0x55936e3921a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.823+0000 7feac1d9b700 1 -- 172.21.15.133:0/1453269037 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55936e29bee0 con 0x55936e38d4a0 2023-11-27T15:28:22.827 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.824+0000 7feac1d9b700 1 --2- 172.21.15.133:0/1453269037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38d4a0 0x55936e391c60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7feab4099180 tx=0x7feab4099240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7f6ad1c47e61792 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:22.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.824+0000 7feab0ff9700 1 -- 172.21.15.133:0/1453269037 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4098580 con 0x55936e38d4a0 2023-11-27T15:28:22.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.824+0000 7feab0ff9700 1 -- 172.21.15.133:0/1453269037 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feab4097b50 con 0x55936e38d4a0 2023-11-27T15:28:22.828 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.824+0000 7feab0ff9700 1 -- 172.21.15.133:0/1453269037 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4098ca0 con 0x55936e38d4a0 2023-11-27T15:28:22.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.825+0000 7fead78a1b80 1 -- 172.21.15.133:0/1453269037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38d4a0 msgr2=0x55936e391c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:22.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.825+0000 7fead78a1b80 1 --2- 172.21.15.133:0/1453269037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38d4a0 0x55936e391c60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7feab4099180 tx=0x7feab4099240 comp rx=0 tx=0).stop 2023-11-27T15:28:22.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.825+0000 7fead78a1b80 1 -- 172.21.15.133:0/1453269037 shutdown_connections 2023-11-27T15:28:22.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.825+0000 7fead78a1b80 1 --2- 172.21.15.133:0/1453269037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38ecf0 0x55936e39e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.825+0000 7fead78a1b80 1 --2- 172.21.15.133:0/1453269037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38d4a0 0x55936e391c60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.831 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.825+0000 7fead78a1b80 1 --2- 172.21.15.133:0/1453269037 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55936e38e340 0x55936e3921a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.825+0000 7fead78a1b80 1 -- 172.21.15.133:0/1453269037 >> 172.21.15.133:0/1453269037 conn(0x55936e37a360 msgr2=0x55936e395b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:22.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.827+0000 7fead78a1b80 1 -- 172.21.15.133:0/1453269037 shutdown_connections 2023-11-27T15:28:22.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.828+0000 7fead78a1b80 1 -- 172.21.15.133:0/1453269037 wait complete. 2023-11-27T15:28:22.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.828+0000 7fead78a1b80 1 -- start start 2023-11-27T15:28:22.832 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.829+0000 7fead78a1b80 1 Processor -- start 2023-11-27T15:28:22.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.830+0000 7fead78a1b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38d4a0 0x55936e399dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:22.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.830+0000 7fead78a1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38e340 0x55936e39a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:22.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.830+0000 7fead78a1b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55936e38ecf0 0x55936e39d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:22.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.830+0000 7fead78a1b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936e39d860 con 0x55936e38ecf0 2023-11-27T15:28:22.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.830+0000 7fead78a1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936e39d9d0 con 0x55936e38e340 2023-11-27T15:28:22.833 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.830+0000 7fead78a1b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55936e39db90 con 0x55936e38d4a0 2023-11-27T15:28:22.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.830+0000 7fead78a1b80 10 client.0 _ll_register_callbacks cb 0x7feab409da40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:22.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.830+0000 7feac1d9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38d4a0 0x55936e399dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:22.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.830+0000 7feac159a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38e340 0x55936e39a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:22.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.830+0000 7feac1d9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38d4a0 0x55936e399dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:60554/0 (socket says 172.21.15.133:60554) 2023-11-27T15:28:22.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.830+0000 7feac1d9b700 1 -- 172.21.15.133:0/2920526552 learned_addr learned my addr 172.21.15.133:0/2920526552 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:22.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.831+0000 7feac1d9b700 1 -- 172.21.15.133:0/2920526552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38e340 msgr2=0x55936e39a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:22.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.831+0000 7feac1d9b700 1 --2- 172.21.15.133:0/2920526552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38e340 0x55936e39a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.834 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.831+0000 7feac1d9b700 1 -- 172.21.15.133:0/2920526552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55936e38ecf0 msgr2=0x55936e39d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:22.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.831+0000 7feac259c700 1 --2- 172.21.15.133:0/2920526552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55936e38ecf0 0x55936e39d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:22.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.831+0000 7feac1d9b700 1 --2- 172.21.15.133:0/2920526552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55936e38ecf0 0x55936e39d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.831+0000 7feac1d9b700 1 -- 172.21.15.133:0/2920526552 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55936e29c190 con 0x55936e38d4a0 2023-11-27T15:28:22.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.831+0000 7feac159a700 1 --2- 172.21.15.133:0/2920526552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38e340 0x55936e39a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:22.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.831+0000 7feac1d9b700 1 --2- 172.21.15.133:0/2920526552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38d4a0 0x55936e399dc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feab40a1a00 tx=0x7feab40cc670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbab75807cb38656 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:22.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.831+0000 7feaa57fa700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:28:22.835 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.831+0000 7feac259c700 1 --2- 172.21.15.133:0/2920526552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55936e38ecf0 0x55936e39d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:28:22.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.832+0000 7feaa57fa700 1 -- 172.21.15.133:0/2920526552 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4098ea0 con 0x55936e38d4a0 2023-11-27T15:28:22.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.832+0000 7feaa57fa700 1 -- 172.21.15.133:0/2920526552 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7feab40b3040 con 0x55936e38d4a0 2023-11-27T15:28:22.836 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.832+0000 7fead78a1b80 1 -- 172.21.15.133:0/2920526552 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feab4003d90 con 0x55936e38d4a0 2023-11-27T15:28:22.837 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.834+0000 7feaa57fa700 1 -- 172.21.15.133:0/2920526552 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7feab4098530 con 0x55936e38d4a0 2023-11-27T15:28:22.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.834+0000 7fead78a1b80 10 client.26449 fetch_fsmap learned FSMap version 28 2023-11-27T15:28:22.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.834+0000 7fead78a1b80 1 -- 172.21.15.133:0/2920526552 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55936e0ecc60 con 0x55936e38d4a0 2023-11-27T15:28:22.838 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.835+0000 7feaa57fa700 1 -- 172.21.15.133:0/2920526552 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feab4003d90 con 0x55936e38d4a0 2023-11-27T15:28:22.840 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.836+0000 7feaa57fa700 1 -- 172.21.15.133:0/2920526552 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7feab4097e10 con 0x55936e38d4a0 2023-11-27T15:28:22.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.837+0000 7fead78a1b80 10 client.26449 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:28:22.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.837+0000 7fead78a1b80 10 client.26449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:28:22.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.837+0000 7fead78a1b80 20 client.26449 populate_metadata read hostname 'smithi133' 2023-11-27T15:28:22.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.837+0000 7fead78a1b80 1 --2- 172.21.15.133:0/2920526552 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55936e3860a0 0x55936e3884b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:22.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.837+0000 7fead78a1b80 4 client.26449 mds_command: new command op to 24560 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:28:22.841 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.837+0000 7fead78a1b80 1 -- 172.21.15.133:0/2920526552 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55936e2a4660 con 0x55936e3860a0 2023-11-27T15:28:22.843 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.839+0000 7feac159a700 1 --2- 172.21.15.133:0/2920526552 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55936e3860a0 0x55936e3884b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:22.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.841+0000 7feaa57fa700 10 client.26449 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:22.844 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.841+0000 7feac159a700 1 --2- 172.21.15.133:0/2920526552 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55936e3860a0 0x55936e3884b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1693070ec31d1025 server_cookie=1749772a53378b53 in_seq=0 out_seq=0 2023-11-27T15:28:22.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.841+0000 7feaa57fa700 1 -- 172.21.15.133:0/2920526552 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55936e2a4660 con 0x55936e3860a0 2023-11-27T15:28:22.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.841+0000 7feaa57fa700 10 client.26449 handle_command_reply: tid=0 2023-11-27T15:28:22.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.841+0000 7fead78a1b80 1 client.26449 shutdown 2023-11-27T15:28:22.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.842+0000 7fead78a1b80 1 -- 172.21.15.133:0/2920526552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38d4a0 msgr2=0x55936e399dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:22.845 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.842+0000 7fead78a1b80 1 --2- 172.21.15.133:0/2920526552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38d4a0 0x55936e399dc0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feab40a1a00 tx=0x7feab40cc670 comp rx=0 tx=0).stop 2023-11-27T15:28:22.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.842+0000 7fead78a1b80 1 -- 172.21.15.133:0/2920526552 shutdown_connections 2023-11-27T15:28:22.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.842+0000 7fead78a1b80 1 --2- 172.21.15.133:0/2920526552 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55936e3860a0 0x55936e3884b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.842+0000 7fead78a1b80 1 --2- 172.21.15.133:0/2920526552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55936e38ecf0 0x55936e39d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.842+0000 7feaa57fa700 0 client.26449 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:22.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.842+0000 7fead78a1b80 1 --2- 172.21.15.133:0/2920526552 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55936e38d4a0 0x55936e399dc0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.842+0000 7fead78a1b80 1 --2- 172.21.15.133:0/2920526552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55936e38e340 0x55936e39a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.842+0000 7fead78a1b80 1 -- 172.21.15.133:0/2920526552 >> 172.21.15.133:0/2920526552 conn(0x55936e37a360 msgr2=0x55936e391c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:22.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.842+0000 7fead78a1b80 1 -- 172.21.15.133:0/2920526552 shutdown_connections 2023-11-27T15:28:22.846 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.843+0000 7fead78a1b80 1 -- 172.21.15.133:0/2920526552 wait complete. 2023-11-27T15:28:22.847 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.844+0000 7fead78a1b80 20 client.26449 trim_cache size 0 max 16384 2023-11-27T15:28:22.847 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:28:22.848 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-27T15:28:22.848 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": { 2023-11-27T15:28:22.848 INFO:teuthology.orchestra.run.smithi133.stdout: "51b5fea8-bb0d-45b1-9654-3423e74f8a20": { 2023-11-27T15:28:22.848 INFO:teuthology.orchestra.run.smithi133.stdout: "path": "/", 2023-11-27T15:28:22.848 INFO:teuthology.orchestra.run.smithi133.stdout: "tag": "51b5fea8-bb0d-45b1-9654-3423e74f8a20", 2023-11-27T15:28:22.848 INFO:teuthology.orchestra.run.smithi133.stdout: "options": "recursive,force" 2023-11-27T15:28:22.848 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:22.848 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:22.848 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:28:22.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.845+0000 7feac3fff700 1 -- 172.21.15.133:0/2091267925 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7feaa80a12a0 msgr2=0x7feaa80a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:22.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.845+0000 7feac3fff700 1 --2- 172.21.15.133:0/2091267925 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7feaa80a12a0 0x7feaa80a3750 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7feab800b2e0 tx=0x7feab801a040 comp rx=0 tx=0).stop 2023-11-27T15:28:22.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.845+0000 7feac3fff700 1 -- 172.21.15.133:0/2091267925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feabc0feb30 msgr2=0x7feabc198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:22.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.845+0000 7feac3fff700 1 --2- 172.21.15.133:0/2091267925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feabc0feb30 0x7feabc198290 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7feab4007fa0 tx=0x7feab400d3b0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.845+0000 7feac3fff700 1 -- 172.21.15.133:0/2091267925 shutdown_connections 2023-11-27T15:28:22.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.845+0000 7feac3fff700 1 --2- 172.21.15.133:0/2091267925 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7feaa80a12a0 0x7feaa80a3750 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.845+0000 7feac3fff700 1 --2- 172.21.15.133:0/2091267925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feabc0ffe30 0x7feabc19f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.845+0000 7feac3fff700 1 --2- 172.21.15.133:0/2091267925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feabc0feb30 0x7feabc198290 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.849 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.845+0000 7feac3fff700 1 --2- 172.21.15.133:0/2091267925 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feabc0ff480 0x7feabc1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:22.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.845+0000 7feac3fff700 1 -- 172.21.15.133:0/2091267925 >> 172.21.15.133:0/2091267925 conn(0x7feabc0fa7c0 msgr2=0x7feabc0fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:22.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.846+0000 7feac3fff700 1 -- 172.21.15.133:0/2091267925 shutdown_connections 2023-11-27T15:28:22.850 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:22.846+0000 7feac3fff700 1 -- 172.21.15.133:0/2091267925 wait complete. 2023-11-27T15:28:22.894 INFO:tasks.cephfs.filesystem:scrub status for tag:51b5fea8-bb0d-45b1-9654-3423e74f8a20 - {'path': '/', 'tag': '51b5fea8-bb0d-45b1-9654-3423e74f8a20', 'options': 'recursive,force'} 2023-11-27T15:28:23.276 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:22 smithi133 ceph-mon[97860]: pgmap v513: 129 pgs: 129 active+clean; 408 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 407 op/s 2023-11-27T15:28:23.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:22 smithi177 ceph-mon[131504]: pgmap v513: 129 pgs: 129 active+clean; 408 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 407 op/s 2023-11-27T15:28:23.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:22 smithi174 ceph-mon[131110]: pgmap v513: 129 pgs: 129 active+clean; 408 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 407 op/s 2023-11-27T15:28:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.018+0000 7fac21fff700 1 -- 172.21.15.133:0/1316202123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac1c0713f0 msgr2=0x7fac1c071860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.018+0000 7fac21fff700 1 --2- 172.21.15.133:0/1316202123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac1c0713f0 0x7fac1c071860 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fac04009990 tx=0x7fac04009ca0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.019+0000 7fac21fff700 1 -- 172.21.15.133:0/1316202123 shutdown_connections 2023-11-27T15:28:25.022 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.019+0000 7fac21fff700 1 --2- 172.21.15.133:0/1316202123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac1c071da0 0x7fac1c110e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.019+0000 7fac21fff700 1 --2- 172.21.15.133:0/1316202123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac1c108000 0x7fac1c108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.019+0000 7fac21fff700 1 --2- 172.21.15.133:0/1316202123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac1c0713f0 0x7fac1c071860 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.019+0000 7fac21fff700 1 -- 172.21.15.133:0/1316202123 >> 172.21.15.133:0/1316202123 conn(0x7fac1c06d1b0 msgr2=0x7fac1c06f620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.020+0000 7fac21fff700 1 -- 172.21.15.133:0/1316202123 shutdown_connections 2023-11-27T15:28:25.023 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.020+0000 7fac21fff700 1 -- 172.21.15.133:0/1316202123 wait complete. 2023-11-27T15:28:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.022+0000 7fac21fff700 1 Processor -- start 2023-11-27T15:28:25.025 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.022+0000 7fac21fff700 1 -- start start 2023-11-27T15:28:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.023+0000 7fac21fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac1c0713f0 0x7fac1c117a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.023+0000 7fac21fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac1c071da0 0x7fac1c117fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.023+0000 7fac21fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac1c108000 0x7fac1c11f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.023+0000 7fac21fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c11f600 con 0x7fac1c108000 2023-11-27T15:28:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.023+0000 7fac21fff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c11f770 con 0x7fac1c071da0 2023-11-27T15:28:25.026 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.023+0000 7fac21fff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c11f930 con 0x7fac1c0713f0 2023-11-27T15:28:25.027 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.023+0000 7fac1bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac1c108000 0x7fac1c11f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:25.027 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.023+0000 7fac1bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac1c108000 0x7fac1c11f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57250/0 (socket says 172.21.15.133:57250) 2023-11-27T15:28:25.027 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.023+0000 7fac1bfff700 1 -- 172.21.15.133:0/4105007790 learned_addr learned my addr 172.21.15.133:0/4105007790 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:25.027 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.023+0000 7fac12ffd700 1 --2- 172.21.15.133:0/4105007790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac1c071da0 0x7fac1c117fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:25.027 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.024+0000 7fac1b7fe700 1 --2- 172.21.15.133:0/4105007790 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac1c0713f0 0x7fac1c117a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.024+0000 7fac1bfff700 1 -- 172.21.15.133:0/4105007790 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac1c0713f0 msgr2=0x7fac1c117a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.024+0000 7fac1bfff700 1 --2- 172.21.15.133:0/4105007790 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac1c0713f0 0x7fac1c117a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.024+0000 7fac1bfff700 1 -- 172.21.15.133:0/4105007790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac1c071da0 msgr2=0x7fac1c117fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.024+0000 7fac1bfff700 1 --2- 172.21.15.133:0/4105007790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac1c071da0 0x7fac1c117fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.024+0000 7fac1bfff700 1 -- 172.21.15.133:0/4105007790 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac040095f0 con 0x7fac1c108000 2023-11-27T15:28:25.028 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.025+0000 7fac1bfff700 1 --2- 172.21.15.133:0/4105007790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac1c108000 0x7fac1c11f030 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fac1400d980 tx=0x7fac1400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.026+0000 7fac197fa700 1 -- 172.21.15.133:0/4105007790 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14004a30 con 0x7fac1c108000 2023-11-27T15:28:25.029 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.026+0000 7fac197fa700 1 -- 172.21.15.133:0/4105007790 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac140054e0 con 0x7fac1c108000 2023-11-27T15:28:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.026+0000 7fac197fa700 1 -- 172.21.15.133:0/4105007790 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1400fbc0 con 0x7fac1c108000 2023-11-27T15:28:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.026+0000 7fac21fff700 1 -- 172.21.15.133:0/4105007790 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac1c109510 con 0x7fac1c108000 2023-11-27T15:28:25.030 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.026+0000 7fac21fff700 1 -- 172.21.15.133:0/4105007790 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac1c109380 con 0x7fac1c108000 2023-11-27T15:28:25.035 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.032+0000 7fac197fa700 1 -- 172.21.15.133:0/4105007790 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fac14004c20 con 0x7fac1c108000 2023-11-27T15:28:25.037 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.034+0000 7fac197fa700 1 --2- 172.21.15.133:0/4105007790 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fac080a1320 0x7fac080a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.034+0000 7fac197fa700 1 -- 172.21.15.133:0/4105007790 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fac140cc8c0 con 0x7fac1c108000 2023-11-27T15:28:25.038 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.035+0000 7fac1b7fe700 1 --2- 172.21.15.133:0/4105007790 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fac080a1320 0x7fac080a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.037+0000 7fac358c1b80 1 Processor -- start 2023-11-27T15:28:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.037+0000 7fac1b7fe700 1 --2- 172.21.15.133:0/4105007790 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fac080a1320 0x7fac080a37d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00f540 tx=0x7fac0c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.038+0000 7fac358c1b80 1 -- start start 2023-11-27T15:28:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.038+0000 7fac358c1b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8a829f6b0 0x55d8a8443870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.038+0000 7fac358c1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8443db0 0x55d8a84445d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:25.042 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.038+0000 7fac358c1b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a8444b10 0x55d8a8444f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.038+0000 7fac358c1b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a8349000 con 0x55d8a8444b10 2023-11-27T15:28:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.038+0000 7fac358c1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a839b840 con 0x55d8a8443db0 2023-11-27T15:28:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.038+0000 7fac358c1b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a8394cd0 con 0x55d8a829f6b0 2023-11-27T15:28:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.039+0000 7fac1b7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8a829f6b0 0x55d8a8443870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.039+0000 7fac12ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8443db0 0x55d8a84445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:25.043 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.039+0000 7fac12ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8443db0 0x55d8a84445d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:34000/0 (socket says 172.21.15.133:34000) 2023-11-27T15:28:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.039+0000 7fac1bfff700 1 --2- 172.21.15.133:0/110882160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a8444b10 0x55d8a8444f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.039+0000 7fac12ffd700 1 -- 172.21.15.133:0/110882160 learned_addr learned my addr 172.21.15.133:0/110882160 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.040+0000 7fac1bfff700 1 -- 172.21.15.133:0/110882160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8a829f6b0 msgr2=0x55d8a8443870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.040+0000 7fac1bfff700 1 --2- 172.21.15.133:0/110882160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8a829f6b0 0x55d8a8443870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.040+0000 7fac1bfff700 1 -- 172.21.15.133:0/110882160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8443db0 msgr2=0x55d8a84445d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.040+0000 7fac1bfff700 1 --2- 172.21.15.133:0/110882160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8443db0 0x55d8a84445d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.040+0000 7fac1bfff700 1 -- 172.21.15.133:0/110882160 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8a82165d0 con 0x55d8a8444b10 2023-11-27T15:28:25.044 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.041+0000 7fac1bfff700 1 --2- 172.21.15.133:0/110882160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a8444b10 0x55d8a8444f80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fac1400d950 tx=0x7fac1400fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7d7dbdfda4765 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.042+0000 7fac11ffb700 1 -- 172.21.15.133:0/110882160 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14014070 con 0x55d8a8444b10 2023-11-27T15:28:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.042+0000 7fac11ffb700 1 -- 172.21.15.133:0/110882160 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac1409d950 con 0x55d8a8444b10 2023-11-27T15:28:25.045 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.042+0000 7fac11ffb700 1 -- 172.21.15.133:0/110882160 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14099070 con 0x55d8a8444b10 2023-11-27T15:28:25.046 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.043+0000 7fac358c1b80 1 -- 172.21.15.133:0/110882160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a8444b10 msgr2=0x55d8a8444f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.043+0000 7fac358c1b80 1 --2- 172.21.15.133:0/110882160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a8444b10 0x55d8a8444f80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fac1400d950 tx=0x7fac1400fda0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.043+0000 7fac358c1b80 1 -- 172.21.15.133:0/110882160 shutdown_connections 2023-11-27T15:28:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.043+0000 7fac358c1b80 1 --2- 172.21.15.133:0/110882160 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a8444b10 0x55d8a8444f80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.043+0000 7fac358c1b80 1 --2- 172.21.15.133:0/110882160 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8a829f6b0 0x55d8a8443870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.043+0000 7fac358c1b80 1 --2- 172.21.15.133:0/110882160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8443db0 0x55d8a84445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.043+0000 7fac358c1b80 1 -- 172.21.15.133:0/110882160 >> 172.21.15.133:0/110882160 conn(0x55d8a8438f50 msgr2=0x55d8a843e100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:25.047 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.044+0000 7fac358c1b80 1 -- 172.21.15.133:0/110882160 shutdown_connections 2023-11-27T15:28:25.048 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.044+0000 7fac358c1b80 1 -- 172.21.15.133:0/110882160 wait complete. 2023-11-27T15:28:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.045+0000 7fac358c1b80 1 -- start start 2023-11-27T15:28:25.049 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.046+0000 7fac358c1b80 1 Processor -- start 2023-11-27T15:28:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.046+0000 7fac358c1b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a829f6b0 0x55d8a844cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.047+0000 7fac358c1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8443db0 0x55d8a844d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.048+0000 7fac1b7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a829f6b0 0x55d8a844cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.048+0000 7fac12ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8443db0 0x55d8a844d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:25.051 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.048+0000 7fac12ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8443db0 0x55d8a844d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:34010/0 (socket says 172.21.15.133:34010) 2023-11-27T15:28:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.048+0000 7fac12ffd700 1 -- 172.21.15.133:0/1024083205 learned_addr learned my addr 172.21.15.133:0/1024083205 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.048+0000 7fac358c1b80 1 --2- 172.21.15.133:0/1024083205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8a8444b10 0x55d8a8454100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.048+0000 7fac358c1b80 1 -- 172.21.15.133:0/1024083205 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a8336880 con 0x55d8a829f6b0 2023-11-27T15:28:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.048+0000 7fac358c1b80 1 -- 172.21.15.133:0/1024083205 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a8336520 con 0x55d8a8443db0 2023-11-27T15:28:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.048+0000 7fac358c1b80 1 -- 172.21.15.133:0/1024083205 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a82188c0 con 0x55d8a8444b10 2023-11-27T15:28:25.052 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.049+0000 7fac358c1b80 10 client.0 _ll_register_callbacks cb 0x55d8a8354720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.049+0000 7fac12ffd700 1 -- 172.21.15.133:0/1024083205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8a8444b10 msgr2=0x55d8a8454100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.049+0000 7fac12ffd700 1 --2- 172.21.15.133:0/1024083205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8a8444b10 0x55d8a8454100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.049+0000 7fac12ffd700 1 -- 172.21.15.133:0/1024083205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a829f6b0 msgr2=0x55d8a844cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.049+0000 7fac12ffd700 1 --2- 172.21.15.133:0/1024083205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a829f6b0 0x55d8a844cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.053 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.049+0000 7fac12ffd700 1 -- 172.21.15.133:0/1024083205 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac1c109380 con 0x55d8a8443db0 2023-11-27T15:28:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.051+0000 7fac12ffd700 1 --2- 172.21.15.133:0/1024083205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8443db0 0x55d8a844d0c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fac04005cd0 tx=0x7fac040050f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a1a6f4e3299c508 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:25.055 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.051+0000 7fabfaffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:28:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.052+0000 7fabfaffd700 1 -- 172.21.15.133:0/1024083205 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0401d040 con 0x55d8a8443db0 2023-11-27T15:28:25.056 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.052+0000 7fabfaffd700 1 -- 172.21.15.133:0/1024083205 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fac040056b0 con 0x55d8a8443db0 2023-11-27T15:28:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.053+0000 7fac358c1b80 1 -- 172.21.15.133:0/1024083205 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8a8436710 con 0x55d8a8443db0 2023-11-27T15:28:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.054+0000 7fabfaffd700 1 -- 172.21.15.133:0/1024083205 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac04004910 con 0x55d8a8443db0 2023-11-27T15:28:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.054+0000 7fac358c1b80 10 client.26486 fetch_fsmap learned FSMap version 28 2023-11-27T15:28:25.058 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.054+0000 7fac358c1b80 1 -- 172.21.15.133:0/1024083205 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d8a835b130 con 0x55d8a8443db0 2023-11-27T15:28:25.059 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.055+0000 7fabfaffd700 1 -- 172.21.15.133:0/1024083205 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d8a8436710 con 0x55d8a8443db0 2023-11-27T15:28:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.055+0000 7fabfaffd700 1 -- 172.21.15.133:0/1024083205 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fac0400fce0 con 0x55d8a8443db0 2023-11-27T15:28:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.056+0000 7fac358c1b80 10 client.26486 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:28:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.056+0000 7fac358c1b80 10 client.26486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:28:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.056+0000 7fac358c1b80 20 client.26486 populate_metadata read hostname 'smithi133' 2023-11-27T15:28:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.056+0000 7fac358c1b80 1 --2- 172.21.15.133:0/1024083205 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d8a84589c0 0x55d8a845add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.056+0000 7fac358c1b80 4 client.26486 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:28:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.056+0000 7fac358c1b80 1 -- 172.21.15.133:0/1024083205 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8a8357040 con 0x55d8a84589c0 2023-11-27T15:28:25.060 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.057+0000 7fac1b7fe700 1 --2- 172.21.15.133:0/1024083205 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d8a84589c0 0x55d8a845add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:25.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.058+0000 7fac1b7fe700 1 --2- 172.21.15.133:0/1024083205 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d8a84589c0 0x55d8a845add0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=720f4d664d09905f server_cookie=baac6b4830bfe921 in_seq=0 out_seq=0 2023-11-27T15:28:25.061 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.058+0000 7fabfaffd700 10 client.26486 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.060+0000 7fabfaffd700 1 -- 172.21.15.133:0/1024083205 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d8a8357040 con 0x55d8a84589c0 2023-11-27T15:28:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.060+0000 7fabfaffd700 10 client.26486 handle_command_reply: tid=0 2023-11-27T15:28:25.064 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.061+0000 7fac358c1b80 1 client.26486 shutdown 2023-11-27T15:28:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.061+0000 7fac358c1b80 1 -- 172.21.15.133:0/1024083205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8443db0 msgr2=0x55d8a844d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.061+0000 7fac358c1b80 1 --2- 172.21.15.133:0/1024083205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8443db0 0x55d8a844d0c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fac04005cd0 tx=0x7fac040050f0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.062+0000 7fac358c1b80 1 -- 172.21.15.133:0/1024083205 shutdown_connections 2023-11-27T15:28:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.062+0000 7fac358c1b80 1 --2- 172.21.15.133:0/1024083205 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d8a84589c0 0x55d8a845add0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.062+0000 7fabfaffd700 0 client.26486 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.062+0000 7fac358c1b80 1 --2- 172.21.15.133:0/1024083205 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8a8444b10 0x55d8a8454100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.065 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.062+0000 7fac358c1b80 1 --2- 172.21.15.133:0/1024083205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a829f6b0 0x55d8a844cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.062+0000 7fac358c1b80 1 --2- 172.21.15.133:0/1024083205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8443db0 0x55d8a844d0c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.062+0000 7fac358c1b80 1 -- 172.21.15.133:0/1024083205 >> 172.21.15.133:0/1024083205 conn(0x55d8a8438f50 msgr2=0x55d8a843d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.063+0000 7fac358c1b80 1 -- 172.21.15.133:0/1024083205 shutdown_connections 2023-11-27T15:28:25.066 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.063+0000 7fac358c1b80 1 -- 172.21.15.133:0/1024083205 wait complete. 2023-11-27T15:28:25.067 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.064+0000 7fac358c1b80 20 client.26486 trim_cache size 0 max 16384 2023-11-27T15:28:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.091+0000 7fac358c1b80 1 Processor -- start 2023-11-27T15:28:25.094 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.091+0000 7fac358c1b80 1 -- start start 2023-11-27T15:28:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.091+0000 7fac358c1b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8a829f6b0 0x55d8a8454990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.092+0000 7fac358c1b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a8443db0 0x55d8a84540c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.092+0000 7fac358c1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8444b10 0x55d8a844c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.092+0000 7fac358c1b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a843f590 con 0x55d8a8443db0 2023-11-27T15:28:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.092+0000 7fac358c1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a844c6f0 con 0x55d8a8444b10 2023-11-27T15:28:25.095 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.092+0000 7fac358c1b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a8438b70 con 0x55d8a829f6b0 2023-11-27T15:28:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.092+0000 7fac12ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a8443db0 0x55d8a84540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.092+0000 7fac12ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a8443db0 0x55d8a84540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:57290/0 (socket says 172.21.15.133:57290) 2023-11-27T15:28:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.092+0000 7fac12ffd700 1 -- 172.21.15.133:0/2819521362 learned_addr learned my addr 172.21.15.133:0/2819521362 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:25.096 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.093+0000 7fac1bfff700 1 --2- 172.21.15.133:0/2819521362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8444b10 0x55d8a844c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.096+0000 7fac12ffd700 1 -- 172.21.15.133:0/2819521362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8a829f6b0 msgr2=0x55d8a8454990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-27T15:28:25.099 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.096+0000 7fac12ffd700 1 --2- 172.21.15.133:0/2819521362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8a829f6b0 0x55d8a8454990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.096+0000 7fac12ffd700 1 -- 172.21.15.133:0/2819521362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8444b10 msgr2=0x55d8a844c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.096+0000 7fac12ffd700 1 --2- 172.21.15.133:0/2819521362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8444b10 0x55d8a844c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.096+0000 7fac12ffd700 1 -- 172.21.15.133:0/2819521362 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8a835ae80 con 0x55d8a8443db0 2023-11-27T15:28:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.097+0000 7fac12ffd700 1 --2- 172.21.15.133:0/2819521362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a8443db0 0x55d8a84540c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fac0400b680 tx=0x7fac04009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f082453e2f1a86c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:25.100 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.097+0000 7fac127fc700 1 -- 172.21.15.133:0/2819521362 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0401d040 con 0x55d8a8443db0 2023-11-27T15:28:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.097+0000 7fac127fc700 1 -- 172.21.15.133:0/2819521362 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac0401e040 con 0x55d8a8443db0 2023-11-27T15:28:25.101 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.097+0000 7fac127fc700 1 -- 172.21.15.133:0/2819521362 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac04024990 con 0x55d8a8443db0 2023-11-27T15:28:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.099+0000 7fac358c1b80 1 -- 172.21.15.133:0/2819521362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a8443db0 msgr2=0x55d8a84540c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.099+0000 7fac358c1b80 1 --2- 172.21.15.133:0/2819521362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a8443db0 0x55d8a84540c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fac0400b680 tx=0x7fac04009ca0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.099+0000 7fac358c1b80 1 -- 172.21.15.133:0/2819521362 shutdown_connections 2023-11-27T15:28:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.099+0000 7fac358c1b80 1 --2- 172.21.15.133:0/2819521362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8444b10 0x55d8a844c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.102 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.099+0000 7fac358c1b80 1 --2- 172.21.15.133:0/2819521362 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8a829f6b0 0x55d8a8454990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.099+0000 7fac358c1b80 1 --2- 172.21.15.133:0/2819521362 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a8443db0 0x55d8a84540c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.103 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.099+0000 7fac358c1b80 1 -- 172.21.15.133:0/2819521362 >> 172.21.15.133:0/2819521362 conn(0x55d8a8438f50 msgr2=0x55d8a845f710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:25.104 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.101+0000 7fac358c1b80 1 -- 172.21.15.133:0/2819521362 shutdown_connections 2023-11-27T15:28:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.102+0000 7fac358c1b80 1 -- 172.21.15.133:0/2819521362 wait complete. 2023-11-27T15:28:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.103+0000 7fac358c1b80 1 -- start start 2023-11-27T15:28:25.106 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.103+0000 7fac358c1b80 1 Processor -- start 2023-11-27T15:28:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.105+0000 7fac358c1b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a829f6b0 0x55d8a844fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.105+0000 7fac358c1b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8a8443db0 0x55d8a8450100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.105+0000 7fac358c1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8444b10 0x55d8a8453120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.105+0000 7fac358c1b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a8453660 con 0x55d8a829f6b0 2023-11-27T15:28:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.105+0000 7fac358c1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a84537d0 con 0x55d8a8444b10 2023-11-27T15:28:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.105+0000 7fac358c1b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8a8453990 con 0x55d8a8443db0 2023-11-27T15:28:25.109 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.105+0000 7fac358c1b80 10 client.0 _ll_register_callbacks cb 0x7fac0400bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.106+0000 7fac12ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8a8443db0 0x55d8a8450100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.106+0000 7fac12ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8a8443db0 0x55d8a8450100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:39732/0 (socket says 172.21.15.133:39732) 2023-11-27T15:28:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.106+0000 7fac12ffd700 1 -- 172.21.15.133:0/2319505163 learned_addr learned my addr 172.21.15.133:0/2319505163 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.107+0000 7fac12ffd700 1 -- 172.21.15.133:0/2319505163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8444b10 msgr2=0x55d8a8453120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:25.110 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.107+0000 7fac12ffd700 1 --2- 172.21.15.133:0/2319505163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8444b10 0x55d8a8453120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.107+0000 7fac12ffd700 1 -- 172.21.15.133:0/2319505163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a829f6b0 msgr2=0x55d8a844fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.107+0000 7fac12ffd700 1 --2- 172.21.15.133:0/2319505163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a829f6b0 0x55d8a844fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.107+0000 7fac12ffd700 1 -- 172.21.15.133:0/2319505163 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8a835b130 con 0x55d8a8443db0 2023-11-27T15:28:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.107+0000 7fac12ffd700 1 --2- 172.21.15.133:0/2319505163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8a8443db0 0x55d8a8450100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fac04005130 tx=0x7fac0400bd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cafff80da9effd1e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:25.111 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.107+0000 7fabfaffd700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:28:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.108+0000 7fabfaffd700 1 -- 172.21.15.133:0/2319505163 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0401d070 con 0x55d8a8443db0 2023-11-27T15:28:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.108+0000 7fabfaffd700 1 -- 172.21.15.133:0/2319505163 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fac04000e40 con 0x55d8a8443db0 2023-11-27T15:28:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.108+0000 7fac358c1b80 1 -- 172.21.15.133:0/2319505163 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac1409d950 con 0x55d8a8443db0 2023-11-27T15:28:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.109+0000 7fabfaffd700 1 -- 172.21.15.133:0/2319505163 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac04005a00 con 0x55d8a8443db0 2023-11-27T15:28:25.112 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.109+0000 7fac358c1b80 10 client.26461 fetch_fsmap learned FSMap version 28 2023-11-27T15:28:25.113 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.109+0000 7fac358c1b80 1 -- 172.21.15.133:0/2319505163 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55d8a81abc00 con 0x55d8a8443db0 2023-11-27T15:28:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.110+0000 7fabfaffd700 1 -- 172.21.15.133:0/2319505163 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac1409d950 con 0x55d8a8443db0 2023-11-27T15:28:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.110+0000 7fabfaffd700 1 -- 172.21.15.133:0/2319505163 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 28) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fac04037480 con 0x55d8a8443db0 2023-11-27T15:28:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.110+0000 7fac358c1b80 10 client.26461 fetch_fsmap finished waiting for FSMap version 28 2023-11-27T15:28:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.110+0000 7fac358c1b80 10 client.26461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:28:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.110+0000 7fac358c1b80 20 client.26461 populate_metadata read hostname 'smithi133' 2023-11-27T15:28:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.110+0000 7fac358c1b80 1 --2- 172.21.15.133:0/2319505163 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d8a84ad170 0x55d8a84af550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.110+0000 7fac358c1b80 4 client.26461 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:28:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.110+0000 7fac358c1b80 1 -- 172.21.15.133:0/2319505163 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d8a8363600 con 0x55d8a84ad170 2023-11-27T15:28:25.114 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.110+0000 7fac1b7fe700 1 --2- 172.21.15.133:0/2319505163 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d8a84ad170 0x55d8a84af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.111+0000 7fabfaffd700 10 client.26461 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:25.115 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.111+0000 7fac1b7fe700 1 --2- 172.21.15.133:0/2319505163 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d8a84ad170 0x55d8a84af550 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3394ff0c27c85c2 server_cookie=84fb08e4661f91da in_seq=0 out_seq=0 2023-11-27T15:28:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.166+0000 7fabfaffd700 1 -- 172.21.15.133:0/2319505163 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d8a8363600 con 0x55d8a84ad170 2023-11-27T15:28:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.166+0000 7fabfaffd700 10 client.26461 handle_command_reply: tid=0 2023-11-27T15:28:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.167+0000 7fac358c1b80 1 client.26461 shutdown 2023-11-27T15:28:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.167+0000 7fac358c1b80 1 -- 172.21.15.133:0/2319505163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8a8443db0 msgr2=0x55d8a8450100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.168+0000 7fac358c1b80 1 --2- 172.21.15.133:0/2319505163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8a8443db0 0x55d8a8450100 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fac04005130 tx=0x7fac0400bd50 comp rx=0 tx=0).stop 2023-11-27T15:28:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.168+0000 7fac358c1b80 1 -- 172.21.15.133:0/2319505163 shutdown_connections 2023-11-27T15:28:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.168+0000 7fac358c1b80 1 --2- 172.21.15.133:0/2319505163 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55d8a84ad170 0x55d8a84af550 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.168+0000 7fac358c1b80 1 --2- 172.21.15.133:0/2319505163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8a8444b10 0x55d8a8453120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.168+0000 7fac358c1b80 1 --2- 172.21.15.133:0/2319505163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d8a829f6b0 0x55d8a844fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.168+0000 7fac358c1b80 1 --2- 172.21.15.133:0/2319505163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55d8a8443db0 0x55d8a8450100 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.168+0000 7fac358c1b80 1 -- 172.21.15.133:0/2319505163 >> 172.21.15.133:0/2319505163 conn(0x55d8a8438f50 msgr2=0x55d8a84479c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.168+0000 7fabfaffd700 0 client.26461 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.168+0000 7fac358c1b80 1 -- 172.21.15.133:0/2319505163 shutdown_connections 2023-11-27T15:28:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.168+0000 7fac358c1b80 1 -- 172.21.15.133:0/2319505163 wait complete. 2023-11-27T15:28:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.169+0000 7fac358c1b80 20 client.26461 trim_cache size 0 max 16384 2023-11-27T15:28:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:28:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:28:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:28:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:28:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.171+0000 7fac21fff700 1 -- 172.21.15.133:0/4105007790 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fac080a1320 msgr2=0x7fac080a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.171+0000 7fac21fff700 1 --2- 172.21.15.133:0/4105007790 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fac080a1320 0x7fac080a37d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00f540 tx=0x7fac0c00b040 comp rx=0 tx=0).stop 2023-11-27T15:28:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.171+0000 7fac21fff700 1 -- 172.21.15.133:0/4105007790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac1c108000 msgr2=0x7fac1c11f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.171+0000 7fac21fff700 1 --2- 172.21.15.133:0/4105007790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac1c108000 0x7fac1c11f030 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fac1400d980 tx=0x7fac1400dc90 comp rx=0 tx=0).stop 2023-11-27T15:28:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.171+0000 7fac21fff700 1 -- 172.21.15.133:0/4105007790 shutdown_connections 2023-11-27T15:28:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.171+0000 7fac21fff700 1 --2- 172.21.15.133:0/4105007790 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fac080a1320 0x7fac080a37d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.171+0000 7fac21fff700 1 --2- 172.21.15.133:0/4105007790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac1c108000 0x7fac1c11f030 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.171+0000 7fac21fff700 1 --2- 172.21.15.133:0/4105007790 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fac1c0713f0 0x7fac1c117a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.171+0000 7fac21fff700 1 --2- 172.21.15.133:0/4105007790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac1c071da0 0x7fac1c117fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.171+0000 7fac21fff700 1 -- 172.21.15.133:0/4105007790 >> 172.21.15.133:0/4105007790 conn(0x7fac1c06d1b0 msgr2=0x7fac1c10bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.173+0000 7fac21fff700 1 -- 172.21.15.133:0/4105007790 shutdown_connections 2023-11-27T15:28:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:25.173+0000 7fac21fff700 1 -- 172.21.15.133:0/4105007790 wait complete. 2023-11-27T15:28:25.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:24 smithi133 ceph-mon[97860]: pgmap v514: 129 pgs: 129 active+clean; 410 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 389 op/s 2023-11-27T15:28:25.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:24 smithi177 ceph-mon[131504]: pgmap v514: 129 pgs: 129 active+clean; 410 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 389 op/s 2023-11-27T15:28:25.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:24 smithi174 ceph-mon[131110]: pgmap v514: 129 pgs: 129 active+clean; 410 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 389 op/s 2023-11-27T15:28:26.414 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:28:26 smithi177 systemd[1]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.c.service: Service RestartSec=10s expired, scheduling restart. 2023-11-27T15:28:26.414 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:28:26 smithi177 systemd[1]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.c.service: Scheduled restart job, restart counter is at 2. 2023-11-27T15:28:26.414 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:28:26 smithi177 systemd[1]: Stopped Ceph mds.c for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:28:26.414 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:28:26 smithi177 systemd[1]: Starting Ceph mds.c for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:28:26.414 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:28:26 smithi177 podman[160936]: 2023-11-27T15:28:26.703 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:28:26 smithi177 bash[160936]: 1a4c5fbd0fe552740e2b6fcdecc43d33c6084b67f8bff4f38b07832be86bed24 2023-11-27T15:28:27.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:26 smithi177 ceph-mon[131504]: pgmap v515: 129 pgs: 129 active+clean; 414 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 390 op/s 2023-11-27T15:28:27.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:26 smithi177 ceph-mon[131504]: daemon mds.c restarted 2023-11-27T15:28:27.088 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:26 smithi177 ceph-mon[131504]: osdmap e94: 12 total, 12 up, 12 in 2023-11-27T15:28:27.088 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:28:26 smithi177 ceph-mds[160976]: -- 172.21.15.177:0/2557347307 <== mon.1 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561b186d7520 con 0x561b186ac800 2023-11-27T15:28:27.088 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:28:26 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-c[160952]: starting mds.c at 2023-11-27T15:28:27.088 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:28:26 smithi177 systemd[1]: Started Ceph mds.c for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:28:27.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:26 smithi133 ceph-mon[97860]: pgmap v515: 129 pgs: 129 active+clean; 414 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 390 op/s 2023-11-27T15:28:27.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:26 smithi133 ceph-mon[97860]: daemon mds.c restarted 2023-11-27T15:28:27.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:26 smithi133 ceph-mon[97860]: osdmap e94: 12 total, 12 up, 12 in 2023-11-27T15:28:27.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:26 smithi174 ceph-mon[131110]: pgmap v515: 129 pgs: 129 active+clean; 414 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 390 op/s 2023-11-27T15:28:27.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:26 smithi174 ceph-mon[131110]: daemon mds.c restarted 2023-11-27T15:28:27.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:26 smithi174 ceph-mon[131110]: osdmap e94: 12 total, 12 up, 12 in 2023-11-27T15:28:28.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:27 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.177:6836/1199749740,v1:172.21.15.177:6837/1199749740] up:boot 2023-11-27T15:28:28.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:27 smithi133 ceph-mon[97860]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-11-27T15:28:28.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:27 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:28:28.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:27 smithi133 ceph-mon[97860]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:28:28.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:27 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.177:6836/1199749740,v1:172.21.15.177:6837/1199749740] up:boot 2023-11-27T15:28:28.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:27 smithi177 ceph-mon[131504]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-11-27T15:28:28.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:27 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:28:28.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:27 smithi177 ceph-mon[131504]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:28:28.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:27 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.177:6836/1199749740,v1:172.21.15.177:6837/1199749740] up:boot 2023-11-27T15:28:28.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:27 smithi174 ceph-mon[131110]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-11-27T15:28:28.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:27 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:28:28.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:27 smithi174 ceph-mon[131110]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-11-27T15:28:29.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:28 smithi133 ceph-mon[97860]: pgmap v517: 129 pgs: 129 active+clean; 419 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 332 op/s 2023-11-27T15:28:29.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:28 smithi177 ceph-mon[131504]: pgmap v517: 129 pgs: 129 active+clean; 419 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 332 op/s 2023-11-27T15:28:29.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:28 smithi174 ceph-mon[131110]: pgmap v517: 129 pgs: 129 active+clean; 419 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 332 op/s 2023-11-27T15:28:30.490 INFO:tasks.workunit.client.0.smithi133.stdout:cleaned up hierarchy 2023-11-27T15:28:30.899 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-27T15:28:30.900 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-27T15:28:30.978 INFO:tasks.workunit:Running workunit fs/misc/filelock_deadlock.py... 2023-11-27T15:28:30.978 DEBUG:teuthology.orchestra.run.smithi133:workunit test fs/misc/filelock_deadlock.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_deadlock.py 2023-11-27T15:28:31.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:30 smithi133 ceph-mon[97860]: scrub summary: idle+waiting paths [/] 2023-11-27T15:28:31.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:30 smithi133 ceph-mon[97860]: pgmap v518: 129 pgs: 129 active+clean; 420 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 311 op/s 2023-11-27T15:28:31.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:30 smithi177 ceph-mon[131504]: scrub summary: idle+waiting paths [/] 2023-11-27T15:28:31.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:30 smithi177 ceph-mon[131504]: pgmap v518: 129 pgs: 129 active+clean; 420 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 311 op/s 2023-11-27T15:28:31.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:30 smithi174 ceph-mon[131110]: scrub summary: idle+waiting paths [/] 2023-11-27T15:28:31.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:30 smithi174 ceph-mon[131110]: pgmap v518: 129 pgs: 129 active+clean; 420 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 311 op/s 2023-11-27T15:28:33.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:32 smithi133 ceph-mon[97860]: pgmap v519: 129 pgs: 129 active+clean; 425 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 293 op/s 2023-11-27T15:28:33.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:32 smithi177 ceph-mon[131504]: pgmap v519: 129 pgs: 129 active+clean; 425 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 293 op/s 2023-11-27T15:28:33.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:32 smithi174 ceph-mon[131110]: pgmap v519: 129 pgs: 129 active+clean; 425 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 293 op/s 2023-11-27T15:28:35.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:34 smithi133 ceph-mon[97860]: pgmap v520: 129 pgs: 129 active+clean; 425 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 272 op/s 2023-11-27T15:28:35.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:34 smithi177 ceph-mon[131504]: pgmap v520: 129 pgs: 129 active+clean; 425 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 272 op/s 2023-11-27T15:28:35.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:34 smithi174 ceph-mon[131110]: pgmap v520: 129 pgs: 129 active+clean; 425 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 272 op/s 2023-11-27T15:28:37.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:36 smithi133 ceph-mon[97860]: pgmap v521: 129 pgs: 129 active+clean; 426 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 241 op/s 2023-11-27T15:28:37.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:36 smithi177 ceph-mon[131504]: pgmap v521: 129 pgs: 129 active+clean; 426 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 241 op/s 2023-11-27T15:28:37.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:36 smithi174 ceph-mon[131110]: pgmap v521: 129 pgs: 129 active+clean; 426 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 241 op/s 2023-11-27T15:28:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.349+0000 7f26dcfff700 1 -- 172.21.15.133:0/249634392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26d8101dc0 msgr2=0x7f26d8108680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.349+0000 7f26dcfff700 1 --2- 172.21.15.133:0/249634392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26d8101dc0 0x7f26d8108680 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f26d000b0a0 tx=0x7f26d000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.349+0000 7f26dcfff700 1 -- 172.21.15.133:0/249634392 shutdown_connections 2023-11-27T15:28:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.349+0000 7f26dcfff700 1 --2- 172.21.15.133:0/249634392 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26d8101dc0 0x7f26d8108680 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.349+0000 7f26dcfff700 1 --2- 172.21.15.133:0/249634392 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26d8100240 0x7f26d8100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.349+0000 7f26dcfff700 1 --2- 172.21.15.133:0/249634392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d8101430 0x7f26d8101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.349+0000 7f26dcfff700 1 -- 172.21.15.133:0/249634392 >> 172.21.15.133:0/249634392 conn(0x7f26d80fb840 msgr2=0x7f26d80fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.349+0000 7f26dcfff700 1 -- 172.21.15.133:0/249634392 shutdown_connections 2023-11-27T15:28:38.354 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.349+0000 7f26dcfff700 1 -- 172.21.15.133:0/249634392 wait complete. 2023-11-27T15:28:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.350+0000 7f26dcfff700 1 Processor -- start 2023-11-27T15:28:38.356 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.350+0000 7f26dcfff700 1 -- start start 2023-11-27T15:28:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.351+0000 7f26dcfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26d8100240 0x7f26d8195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.351+0000 7f26dcfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26d8101430 0x7f26d81963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.351+0000 7f26dcfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d8101dc0 0x7f26d819d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.351+0000 7f26dcfff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d819da30 con 0x7f26d8100240 2023-11-27T15:28:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.351+0000 7f26dcfff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d819db70 con 0x7f26d8101dc0 2023-11-27T15:28:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.351+0000 7f26dcfff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d819dd00 con 0x7f26d8101430 2023-11-27T15:28:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.351+0000 7f26d659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26d8100240 0x7f26d8195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.351+0000 7f26d659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26d8100240 0x7f26d8195eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:53982/0 (socket says 172.21.15.133:53982) 2023-11-27T15:28:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.351+0000 7f26d659c700 1 -- 172.21.15.133:0/3478164848 learned_addr learned my addr 172.21.15.133:0/3478164848 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.351+0000 7f26d5d9b700 1 --2- 172.21.15.133:0/3478164848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26d8101430 0x7f26d81963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:38.357 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.351+0000 7f26d6d9d700 1 --2- 172.21.15.133:0/3478164848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d8101dc0 0x7f26d819d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.351+0000 7f26d659c700 1 -- 172.21.15.133:0/3478164848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26d8101430 msgr2=0x7f26d81963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.351+0000 7f26d659c700 1 --2- 172.21.15.133:0/3478164848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26d8101430 0x7f26d81963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.351+0000 7f26d659c700 1 -- 172.21.15.133:0/3478164848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d8101dc0 msgr2=0x7f26d819d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.351+0000 7f26d659c700 1 --2- 172.21.15.133:0/3478164848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d8101dc0 0x7f26d819d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.352+0000 7f26d659c700 1 -- 172.21.15.133:0/3478164848 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d0009d00 con 0x7f26d8100240 2023-11-27T15:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.352+0000 7f26d6d9d700 1 --2- 172.21.15.133:0/3478164848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d8101dc0 0x7f26d819d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.352+0000 7f26d659c700 1 --2- 172.21.15.133:0/3478164848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26d8100240 0x7f26d8195eb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f26c800e970 tx=0x7f26c800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.352+0000 7f26c77fe700 1 -- 172.21.15.133:0/3478164848 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8009790 con 0x7f26d8100240 2023-11-27T15:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.352+0000 7f26c77fe700 1 -- 172.21.15.133:0/3478164848 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26c800eec0 con 0x7f26d8100240 2023-11-27T15:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.352+0000 7f26dcfff700 1 -- 172.21.15.133:0/3478164848 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d810bb50 con 0x7f26d8100240 2023-11-27T15:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.352+0000 7f26c77fe700 1 -- 172.21.15.133:0/3478164848 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c8003cf0 con 0x7f26d8100240 2023-11-27T15:28:38.358 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.352+0000 7f26dcfff700 1 -- 172.21.15.133:0/3478164848 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d810b9c0 con 0x7f26d8100240 2023-11-27T15:28:38.361 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.357+0000 7f26c77fe700 1 -- 172.21.15.133:0/3478164848 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f26c800ccd0 con 0x7f26d8100240 2023-11-27T15:28:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.358+0000 7f26f0963b80 1 Processor -- start 2023-11-27T15:28:38.362 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.358+0000 7f26f0963b80 1 -- start start 2023-11-27T15:28:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.358+0000 7f26f0963b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da115c6b0 0x562da1300870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.358+0000 7f26f0963b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562da1300db0 0x562da13015d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.358+0000 7f26f0963b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da1301b10 0x562da1301f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.358+0000 7f26f0963b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da1206000 con 0x562da115c6b0 2023-11-27T15:28:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.358+0000 7f26f0963b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da1258840 con 0x562da1301b10 2023-11-27T15:28:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.358+0000 7f26f0963b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da1251cd0 con 0x562da1300db0 2023-11-27T15:28:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.358+0000 7f26c77fe700 1 --2- 172.21.15.133:0/3478164848 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f26c00a1150 0x7f26c00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.358+0000 7f26d5d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da1301b10 0x562da1301f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.358+0000 7f26d5d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da1301b10 0x562da1301f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:57964/0 (socket says 172.21.15.133:57964) 2023-11-27T15:28:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.358+0000 7f26d5d9b700 1 -- 172.21.15.133:0/361014495 learned_addr learned my addr 172.21.15.133:0/361014495 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:38.363 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.358+0000 7f26d6d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da115c6b0 0x562da1300870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.359+0000 7f26d659c700 1 --2- 172.21.15.133:0/361014495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562da1300db0 0x562da13015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.358+0000 7f26c77fe700 1 -- 172.21.15.133:0/3478164848 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f26c8014070 con 0x7f26d8100240 2023-11-27T15:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.359+0000 7f26d6d9d700 1 --2- 172.21.15.133:0/3478164848 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f26c00a1150 0x7f26c00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.359+0000 7f26d5d9b700 1 -- 172.21.15.133:0/361014495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562da1300db0 msgr2=0x562da13015d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.359+0000 7f26d5d9b700 1 --2- 172.21.15.133:0/361014495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562da1300db0 0x562da13015d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.359+0000 7f26d5d9b700 1 -- 172.21.15.133:0/361014495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da115c6b0 msgr2=0x562da1300870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.359+0000 7f26d5d9b700 1 --2- 172.21.15.133:0/361014495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da115c6b0 0x562da1300870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.359+0000 7f26d5d9b700 1 -- 172.21.15.133:0/361014495 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562da10d35d0 con 0x562da1301b10 2023-11-27T15:28:38.365 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.359+0000 7f26d5d9b700 1 --2- 172.21.15.133:0/361014495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da1301b10 0x562da1301f80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009d80 tx=0x7f26cc011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbb2e2a76bb49c5a server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.359+0000 7f26d6d9d700 1 --2- 172.21.15.133:0/3478164848 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f26c00a1150 0x7f26c00a3600 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009cd0 tx=0x7f26d00099e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.359+0000 7f26c4ff9700 1 -- 172.21.15.133:0/361014495 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc004230 con 0x562da1301b10 2023-11-27T15:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.359+0000 7f26c4ff9700 1 -- 172.21.15.133:0/361014495 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26cc009e10 con 0x562da1301b10 2023-11-27T15:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.359+0000 7f26c4ff9700 1 -- 172.21.15.133:0/361014495 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc019880 con 0x562da1301b10 2023-11-27T15:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.360+0000 7f26f0963b80 1 -- 172.21.15.133:0/361014495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da1301b10 msgr2=0x562da1301f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.360+0000 7f26f0963b80 1 --2- 172.21.15.133:0/361014495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da1301b10 0x562da1301f80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009d80 tx=0x7f26cc011720 comp rx=0 tx=0).stop 2023-11-27T15:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.360+0000 7f26f0963b80 1 -- 172.21.15.133:0/361014495 shutdown_connections 2023-11-27T15:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.360+0000 7f26f0963b80 1 --2- 172.21.15.133:0/361014495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da1301b10 0x562da1301f80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.360+0000 7f26f0963b80 1 --2- 172.21.15.133:0/361014495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da115c6b0 0x562da1300870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.360+0000 7f26f0963b80 1 --2- 172.21.15.133:0/361014495 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562da1300db0 0x562da13015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.360+0000 7f26f0963b80 1 -- 172.21.15.133:0/361014495 >> 172.21.15.133:0/361014495 conn(0x562da12f5f50 msgr2=0x562da12fb100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.360+0000 7f26f0963b80 1 -- 172.21.15.133:0/361014495 shutdown_connections 2023-11-27T15:28:38.366 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.360+0000 7f26f0963b80 1 -- 172.21.15.133:0/361014495 wait complete. 2023-11-27T15:28:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.361+0000 7f26f0963b80 1 -- start start 2023-11-27T15:28:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.361+0000 7f26f0963b80 1 Processor -- start 2023-11-27T15:28:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.361+0000 7f26f0963b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da115c6b0 0x562da1309c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.362+0000 7f26f0963b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da1300db0 0x562da130a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:38.369 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.362+0000 7f26f0963b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562da1301b10 0x562da13111f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.362+0000 7f26f0963b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da11f3880 con 0x562da1300db0 2023-11-27T15:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.362+0000 7f26f0963b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da11f3520 con 0x562da115c6b0 2023-11-27T15:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.362+0000 7f26f0963b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da10d58c0 con 0x562da1301b10 2023-11-27T15:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.362+0000 7f26f0963b80 10 client.0 _ll_register_callbacks cb 0x562da1211720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.362+0000 7f26d5d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da1300db0 0x562da130a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.362+0000 7f26d5d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da1300db0 0x562da130a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54004/0 (socket says 172.21.15.133:54004) 2023-11-27T15:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.362+0000 7f26d5d9b700 1 -- 172.21.15.133:0/500530559 learned_addr learned my addr 172.21.15.133:0/500530559 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.362+0000 7f26d659c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da115c6b0 0x562da1309c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.362+0000 7f26d6d9d700 1 --2- 172.21.15.133:0/500530559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562da1301b10 0x562da13111f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.362+0000 7f26d5d9b700 1 -- 172.21.15.133:0/500530559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562da1301b10 msgr2=0x562da13111f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.362+0000 7f26d5d9b700 1 --2- 172.21.15.133:0/500530559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562da1301b10 0x562da13111f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.362+0000 7f26d5d9b700 1 -- 172.21.15.133:0/500530559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da115c6b0 msgr2=0x562da1309c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:38.370 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.362+0000 7f26d5d9b700 1 --2- 172.21.15.133:0/500530559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da115c6b0 0x562da1309c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.362+0000 7f26d5d9b700 1 -- 172.21.15.133:0/500530559 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26cc0099e0 con 0x562da1300db0 2023-11-27T15:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.362+0000 7f26d659c700 1 --2- 172.21.15.133:0/500530559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da115c6b0 0x562da1309c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.363+0000 7f26d5d9b700 1 --2- 172.21.15.133:0/500530559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da1300db0 0x562da130a1b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f26cc01b040 tx=0x7f26cc004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36d05e0e7893f66 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.363+0000 7f26bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.363+0000 7f26bdffb700 1 -- 172.21.15.133:0/500530559 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc003d60 con 0x562da1300db0 2023-11-27T15:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.363+0000 7f26bdffb700 1 -- 172.21.15.133:0/500530559 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f26cc005250 con 0x562da1300db0 2023-11-27T15:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.363+0000 7f26f0963b80 1 -- 172.21.15.133:0/500530559 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562da12f3710 con 0x562da1300db0 2023-11-27T15:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.363+0000 7f26bdffb700 1 -- 172.21.15.133:0/500530559 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26cc005410 con 0x562da1300db0 2023-11-27T15:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.363+0000 7f26f0963b80 10 client.16365 fetch_fsmap learned FSMap version 30 2023-11-27T15:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.363+0000 7f26f0963b80 1 -- 172.21.15.133:0/500530559 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562da1218130 con 0x562da1300db0 2023-11-27T15:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.363+0000 7f26bdffb700 1 -- 172.21.15.133:0/500530559 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562da12f3710 con 0x562da1300db0 2023-11-27T15:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.364+0000 7f26bdffb700 1 -- 172.21.15.133:0/500530559 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f26cc021040 con 0x562da1300db0 2023-11-27T15:28:38.371 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.364+0000 7f26f0963b80 10 client.16365 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.364+0000 7f26f0963b80 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.364+0000 7f26f0963b80 20 client.16365 populate_metadata read hostname 'smithi133' 2023-11-27T15:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.364+0000 7f26f0963b80 1 --2- 172.21.15.133:0/500530559 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x562da13159c0 0x562da1317dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.364+0000 7f26f0963b80 4 client.16365 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.364+0000 7f26f0963b80 1 -- 172.21.15.133:0/500530559 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562da1214040 con 0x562da13159c0 2023-11-27T15:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.364+0000 7f26d659c700 1 --2- 172.21.15.133:0/500530559 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x562da13159c0 0x562da1317dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.365+0000 7f26d659c700 1 --2- 172.21.15.133:0/500530559 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x562da13159c0 0x562da1317dd0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7f9ed8512cd87f1 server_cookie=34bdaf61cf9491ba in_seq=0 out_seq=0 2023-11-27T15:28:38.372 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.365+0000 7f26bdffb700 10 client.16365 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.365+0000 7f26bdffb700 1 -- 172.21.15.133:0/500530559 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562da1214040 con 0x562da13159c0 2023-11-27T15:28:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.365+0000 7f26bdffb700 10 client.16365 handle_command_reply: tid=0 2023-11-27T15:28:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.365+0000 7f26f0963b80 1 client.16365 shutdown 2023-11-27T15:28:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.366+0000 7f26f0963b80 1 -- 172.21.15.133:0/500530559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da1300db0 msgr2=0x562da130a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.366+0000 7f26f0963b80 1 --2- 172.21.15.133:0/500530559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da1300db0 0x562da130a1b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f26cc01b040 tx=0x7f26cc004250 comp rx=0 tx=0).stop 2023-11-27T15:28:38.373 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.366+0000 7f26f0963b80 1 -- 172.21.15.133:0/500530559 shutdown_connections 2023-11-27T15:28:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.366+0000 7f26f0963b80 1 --2- 172.21.15.133:0/500530559 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x562da13159c0 0x562da1317dd0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.366+0000 7f26f0963b80 1 --2- 172.21.15.133:0/500530559 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562da1301b10 0x562da13111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.366+0000 7f26bdffb700 0 client.16365 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.366+0000 7f26f0963b80 1 --2- 172.21.15.133:0/500530559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da115c6b0 0x562da1309c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.366+0000 7f26f0963b80 1 --2- 172.21.15.133:0/500530559 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da1300db0 0x562da130a1b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.366+0000 7f26f0963b80 1 -- 172.21.15.133:0/500530559 >> 172.21.15.133:0/500530559 conn(0x562da12f5f50 msgr2=0x562da12fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.366+0000 7f26f0963b80 1 -- 172.21.15.133:0/500530559 shutdown_connections 2023-11-27T15:28:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.366+0000 7f26f0963b80 1 -- 172.21.15.133:0/500530559 wait complete. 2023-11-27T15:28:38.374 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.366+0000 7f26f0963b80 20 client.16365 trim_cache size 0 max 16384 2023-11-27T15:28:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.381+0000 7f26f0963b80 1 Processor -- start 2023-11-27T15:28:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.381+0000 7f26f0963b80 1 -- start start 2023-11-27T15:28:38.385 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.381+0000 7f26f0963b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da115c6b0 0x562da1301ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.381+0000 7f26f0963b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da1300db0 0x562da1307e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.381+0000 7f26f0963b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562da1301b10 0x562da1308370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.381+0000 7f26f0963b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da130a8f0 con 0x562da1300db0 2023-11-27T15:28:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.381+0000 7f26f0963b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da1307be0 con 0x562da115c6b0 2023-11-27T15:28:38.386 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.381+0000 7f26f0963b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da130adc0 con 0x562da1301b10 2023-11-27T15:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.381+0000 7f26d5d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da1300db0 0x562da1307e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.381+0000 7f26d5d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da1300db0 0x562da1307e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54018/0 (socket says 172.21.15.133:54018) 2023-11-27T15:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.382+0000 7f26d5d9b700 1 -- 172.21.15.133:0/26667106 learned_addr learned my addr 172.21.15.133:0/26667106 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.382+0000 7f26d6d9d700 1 --2- 172.21.15.133:0/26667106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562da1301b10 0x562da1308370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.382+0000 7f26d659c700 1 --2- 172.21.15.133:0/26667106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da115c6b0 0x562da1301ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.382+0000 7f26d5d9b700 1 -- 172.21.15.133:0/26667106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562da1301b10 msgr2=0x562da1308370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.382+0000 7f26d5d9b700 1 --2- 172.21.15.133:0/26667106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562da1301b10 0x562da1308370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.382+0000 7f26d5d9b700 1 -- 172.21.15.133:0/26667106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da115c6b0 msgr2=0x562da1301ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.382+0000 7f26d5d9b700 1 --2- 172.21.15.133:0/26667106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da115c6b0 0x562da1301ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.382+0000 7f26d5d9b700 1 -- 172.21.15.133:0/26667106 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562da1217e80 con 0x562da1300db0 2023-11-27T15:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.382+0000 7f26d659c700 1 --2- 172.21.15.133:0/26667106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da115c6b0 0x562da1301ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.382+0000 7f26d5d9b700 1 --2- 172.21.15.133:0/26667106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da1300db0 0x562da1307e30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f26cc006010 tx=0x7f26cc019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a681b1b6836191 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:38.388 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.382+0000 7f26c57fa700 1 -- 172.21.15.133:0/26667106 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc019420 con 0x562da1300db0 2023-11-27T15:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.382+0000 7f26c57fa700 1 -- 172.21.15.133:0/26667106 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26cc0198b0 con 0x562da1300db0 2023-11-27T15:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.382+0000 7f26c57fa700 1 -- 172.21.15.133:0/26667106 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc005250 con 0x562da1300db0 2023-11-27T15:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.383+0000 7f26f0963b80 1 -- 172.21.15.133:0/26667106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da1300db0 msgr2=0x562da1307e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.383+0000 7f26f0963b80 1 --2- 172.21.15.133:0/26667106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da1300db0 0x562da1307e30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f26cc006010 tx=0x7f26cc019760 comp rx=0 tx=0).stop 2023-11-27T15:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.383+0000 7f26f0963b80 1 -- 172.21.15.133:0/26667106 shutdown_connections 2023-11-27T15:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.383+0000 7f26f0963b80 1 --2- 172.21.15.133:0/26667106 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562da1301b10 0x562da1308370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.383+0000 7f26f0963b80 1 --2- 172.21.15.133:0/26667106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da115c6b0 0x562da1301ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.383+0000 7f26f0963b80 1 --2- 172.21.15.133:0/26667106 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da1300db0 0x562da1307e30 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.383+0000 7f26f0963b80 1 -- 172.21.15.133:0/26667106 >> 172.21.15.133:0/26667106 conn(0x562da12f5f50 msgr2=0x562da131c730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.383+0000 7f26f0963b80 1 -- 172.21.15.133:0/26667106 shutdown_connections 2023-11-27T15:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.383+0000 7f26f0963b80 1 -- 172.21.15.133:0/26667106 wait complete. 2023-11-27T15:28:38.389 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.383+0000 7f26f0963b80 1 -- start start 2023-11-27T15:28:38.390 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.384+0000 7f26f0963b80 1 Processor -- start 2023-11-27T15:28:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.384+0000 7f26f0963b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da115c6b0 0x562da1367860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.384+0000 7f26f0963b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da1301b10 0x562da1369db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.384+0000 7f26f0963b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562da136a2f0 0x562da136c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.384+0000 7f26f0963b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da136ccf0 con 0x562da115c6b0 2023-11-27T15:28:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.384+0000 7f26f0963b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da136ce60 con 0x562da1301b10 2023-11-27T15:28:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.384+0000 7f26f0963b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x562da136d020 con 0x562da136a2f0 2023-11-27T15:28:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.384+0000 7f26f0963b80 10 client.0 _ll_register_callbacks cb 0x7f26cc005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.384+0000 7f26d659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da115c6b0 0x562da1367860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.384+0000 7f26d659c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da115c6b0 0x562da1367860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:54026/0 (socket says 172.21.15.133:54026) 2023-11-27T15:28:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.384+0000 7f26d659c700 1 -- 172.21.15.133:0/855278137 learned_addr learned my addr 172.21.15.133:0/855278137 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.384+0000 7f26d6d9d700 1 --2- 172.21.15.133:0/855278137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562da136a2f0 0x562da136c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:38.392 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.384+0000 7f26d5d9b700 1 --2- 172.21.15.133:0/855278137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da1301b10 0x562da1369db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.385+0000 7f26d659c700 1 -- 172.21.15.133:0/855278137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562da136a2f0 msgr2=0x562da136c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.385+0000 7f26d659c700 1 --2- 172.21.15.133:0/855278137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562da136a2f0 0x562da136c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.385+0000 7f26d659c700 1 -- 172.21.15.133:0/855278137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da1301b10 msgr2=0x562da1369db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.385+0000 7f26d659c700 1 --2- 172.21.15.133:0/855278137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da1301b10 0x562da1369db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.385+0000 7f26d659c700 1 -- 172.21.15.133:0/855278137 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562da1218130 con 0x562da115c6b0 2023-11-27T15:28:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.385+0000 7f26d6d9d700 1 --2- 172.21.15.133:0/855278137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562da136a2f0 0x562da136c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.385+0000 7f26d659c700 1 --2- 172.21.15.133:0/855278137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da115c6b0 0x562da1367860 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f26c800c5b0 tx=0x7f26c809a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1478b521b2941b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.385+0000 7f26bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:28:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.385+0000 7f26bdffb700 1 -- 172.21.15.133:0/855278137 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c80b2040 con 0x562da115c6b0 2023-11-27T15:28:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.385+0000 7f26bdffb700 1 -- 172.21.15.133:0/855278137 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f26c8098330 con 0x562da115c6b0 2023-11-27T15:28:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.385+0000 7f26f0963b80 1 -- 172.21.15.133:0/855278137 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f26cc009e10 con 0x562da115c6b0 2023-11-27T15:28:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.385+0000 7f26bdffb700 1 -- 172.21.15.133:0/855278137 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26c8099480 con 0x562da115c6b0 2023-11-27T15:28:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.385+0000 7f26f0963b80 10 client.16371 fetch_fsmap learned FSMap version 30 2023-11-27T15:28:38.393 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.385+0000 7f26f0963b80 1 -- 172.21.15.133:0/855278137 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562da1068c00 con 0x562da115c6b0 2023-11-27T15:28:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.385+0000 7f26bdffb700 1 -- 172.21.15.133:0/855278137 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26cc009e10 con 0x562da115c6b0 2023-11-27T15:28:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.386+0000 7f26bdffb700 1 -- 172.21.15.133:0/855278137 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f26c8094ba0 con 0x562da115c6b0 2023-11-27T15:28:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.386+0000 7f26f0963b80 10 client.16371 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:28:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.386+0000 7f26f0963b80 10 client.16371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:28:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.386+0000 7f26f0963b80 20 client.16371 populate_metadata read hostname 'smithi133' 2023-11-27T15:28:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.386+0000 7f26f0963b80 1 --2- 172.21.15.133:0/855278137 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x562da136ea60 0x562da1370e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.386+0000 7f26f0963b80 4 client.16371 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:28:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.386+0000 7f26f0963b80 1 -- 172.21.15.133:0/855278137 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562da1220600 con 0x562da136ea60 2023-11-27T15:28:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.386+0000 7f26d5d9b700 1 --2- 172.21.15.133:0/855278137 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x562da136ea60 0x562da1370e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.386+0000 7f26bdffb700 10 client.16371 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:38.394 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.386+0000 7f26d5d9b700 1 --2- 172.21.15.133:0/855278137 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x562da136ea60 0x562da1370e70 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d151f9b374814ef server_cookie=5abf16aa03d6eed2 in_seq=0 out_seq=0 2023-11-27T15:28:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.403+0000 7f26bdffb700 1 -- 172.21.15.133:0/855278137 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562da1220600 con 0x562da136ea60 2023-11-27T15:28:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.403+0000 7f26bdffb700 10 client.16371 handle_command_reply: tid=0 2023-11-27T15:28:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.403+0000 7f26f0963b80 1 client.16371 shutdown 2023-11-27T15:28:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.403+0000 7f26f0963b80 1 -- 172.21.15.133:0/855278137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da115c6b0 msgr2=0x562da1367860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.403+0000 7f26f0963b80 1 --2- 172.21.15.133:0/855278137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da115c6b0 0x562da1367860 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f26c800c5b0 tx=0x7f26c809a040 comp rx=0 tx=0).stop 2023-11-27T15:28:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.403+0000 7f26f0963b80 1 -- 172.21.15.133:0/855278137 shutdown_connections 2023-11-27T15:28:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.403+0000 7f26f0963b80 1 --2- 172.21.15.133:0/855278137 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x562da136ea60 0x562da1370e70 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.403+0000 7f26f0963b80 1 --2- 172.21.15.133:0/855278137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x562da136a2f0 0x562da136c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.408 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.403+0000 7f26f0963b80 1 --2- 172.21.15.133:0/855278137 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562da115c6b0 0x562da1367860 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.403+0000 7f26bdffb700 0 client.16371 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.403+0000 7f26f0963b80 1 --2- 172.21.15.133:0/855278137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562da1301b10 0x562da1369db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.403+0000 7f26f0963b80 1 -- 172.21.15.133:0/855278137 >> 172.21.15.133:0/855278137 conn(0x562da12f5f50 msgr2=0x562da1304980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.403+0000 7f26f0963b80 1 -- 172.21.15.133:0/855278137 shutdown_connections 2023-11-27T15:28:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.403+0000 7f26f0963b80 1 -- 172.21.15.133:0/855278137 wait complete. 2023-11-27T15:28:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.404+0000 7f26f0963b80 20 client.16371 trim_cache size 0 max 16384 2023-11-27T15:28:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:28:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:28:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:28:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:28:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.404+0000 7f26dcfff700 1 -- 172.21.15.133:0/3478164848 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f26c00a1150 msgr2=0x7f26c00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.404+0000 7f26dcfff700 1 --2- 172.21.15.133:0/3478164848 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f26c00a1150 0x7f26c00a3600 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009cd0 tx=0x7f26d00099e0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.404+0000 7f26dcfff700 1 -- 172.21.15.133:0/3478164848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26d8100240 msgr2=0x7f26d8195eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.404+0000 7f26dcfff700 1 --2- 172.21.15.133:0/3478164848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26d8100240 0x7f26d8195eb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f26c800e970 tx=0x7f26c800ec80 comp rx=0 tx=0).stop 2023-11-27T15:28:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.404+0000 7f26dcfff700 1 -- 172.21.15.133:0/3478164848 shutdown_connections 2023-11-27T15:28:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.404+0000 7f26dcfff700 1 --2- 172.21.15.133:0/3478164848 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f26c00a1150 0x7f26c00a3600 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.404+0000 7f26dcfff700 1 --2- 172.21.15.133:0/3478164848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d8101dc0 0x7f26d819d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.404+0000 7f26dcfff700 1 --2- 172.21.15.133:0/3478164848 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26d8100240 0x7f26d8195eb0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.404+0000 7f26dcfff700 1 --2- 172.21.15.133:0/3478164848 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26d8101430 0x7f26d81963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.404+0000 7f26dcfff700 1 -- 172.21.15.133:0/3478164848 >> 172.21.15.133:0/3478164848 conn(0x7f26d80fb840 msgr2=0x7f26d8104870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.404+0000 7f26dcfff700 1 -- 172.21.15.133:0/3478164848 shutdown_connections 2023-11-27T15:28:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:38.404+0000 7f26dcfff700 1 -- 172.21.15.133:0/3478164848 wait complete. 2023-11-27T15:28:39.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:38 smithi177 ceph-mon[131504]: pgmap v522: 129 pgs: 129 active+clean; 426 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 209 op/s 2023-11-27T15:28:39.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:38 smithi174 ceph-mon[131110]: pgmap v522: 129 pgs: 129 active+clean; 426 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 209 op/s 2023-11-27T15:28:39.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:38 smithi133 ceph-mon[97860]: pgmap v522: 129 pgs: 129 active+clean; 426 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 209 op/s 2023-11-27T15:28:41.117 INFO:tasks.workunit.client.0.smithi133.stdout:ok 2023-11-27T15:28:41.123 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-27T15:28:41.123 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-27T15:28:41.187 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:40 smithi133 ceph-mon[97860]: pgmap v523: 129 pgs: 129 active+clean; 426 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 640 KiB/s wr, 161 op/s 2023-11-27T15:28:41.214 INFO:tasks.workunit:Running workunit fs/misc/filelock_interrupt.py... 2023-11-27T15:28:41.214 DEBUG:teuthology.orchestra.run.smithi133:workunit test fs/misc/filelock_interrupt.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_interrupt.py 2023-11-27T15:28:41.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:40 smithi177 ceph-mon[131504]: pgmap v523: 129 pgs: 129 active+clean; 426 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 640 KiB/s wr, 161 op/s 2023-11-27T15:28:41.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:40 smithi174 ceph-mon[131110]: pgmap v523: 129 pgs: 129 active+clean; 426 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 640 KiB/s wr, 161 op/s 2023-11-27T15:28:42.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:41 smithi133 ceph-mon[97860]: scrub summary: idle 2023-11-27T15:28:42.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:41 smithi177 ceph-mon[131504]: scrub summary: idle 2023-11-27T15:28:42.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:41 smithi174 ceph-mon[131110]: scrub summary: idle 2023-11-27T15:28:43.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:42 smithi133 ceph-mon[97860]: scrub summary: idle 2023-11-27T15:28:43.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:42 smithi133 ceph-mon[97860]: scrub summary: idle 2023-11-27T15:28:43.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:42 smithi133 ceph-mon[97860]: pgmap v524: 129 pgs: 129 active+clean; 426 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 571 KiB/s wr, 123 op/s 2023-11-27T15:28:43.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:42 smithi177 ceph-mon[131504]: scrub summary: idle 2023-11-27T15:28:43.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:42 smithi177 ceph-mon[131504]: scrub summary: idle 2023-11-27T15:28:43.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:42 smithi177 ceph-mon[131504]: pgmap v524: 129 pgs: 129 active+clean; 426 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 571 KiB/s wr, 123 op/s 2023-11-27T15:28:43.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:42 smithi174 ceph-mon[131110]: scrub summary: idle 2023-11-27T15:28:43.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:42 smithi174 ceph-mon[131110]: scrub summary: idle 2023-11-27T15:28:43.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:42 smithi174 ceph-mon[131110]: pgmap v524: 129 pgs: 129 active+clean; 426 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 571 KiB/s wr, 123 op/s 2023-11-27T15:28:45.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:44 smithi133 ceph-mon[97860]: pgmap v525: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 84 op/s 2023-11-27T15:28:45.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:44 smithi177 ceph-mon[131504]: pgmap v525: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 84 op/s 2023-11-27T15:28:45.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:44 smithi174 ceph-mon[131110]: pgmap v525: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 123 KiB/s wr, 84 op/s 2023-11-27T15:28:47.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:46 smithi133 ceph-mon[97860]: pgmap v526: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 52 op/s 2023-11-27T15:28:47.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:46 smithi177 ceph-mon[131504]: pgmap v526: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 52 op/s 2023-11-27T15:28:47.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:46 smithi174 ceph-mon[131110]: pgmap v526: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 120 KiB/s wr, 52 op/s 2023-11-27T15:28:49.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:48 smithi177 ceph-mon[131504]: pgmap v527: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.7 KiB/s wr, 18 op/s 2023-11-27T15:28:49.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:48 smithi174 ceph-mon[131110]: pgmap v527: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.7 KiB/s wr, 18 op/s 2023-11-27T15:28:49.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:48 smithi133 ceph-mon[97860]: pgmap v527: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.7 KiB/s wr, 18 op/s 2023-11-27T15:28:51.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:50 smithi133 ceph-mon[97860]: pgmap v528: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2023-11-27T15:28:51.309 INFO:tasks.workunit.client.0.smithi133.stdout:ok 2023-11-27T15:28:51.315 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-11-27T15:28:51.315 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-11-27T15:28:51.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:50 smithi177 ceph-mon[131504]: pgmap v528: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2023-11-27T15:28:51.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:50 smithi174 ceph-mon[131110]: pgmap v528: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2023-11-27T15:28:51.404 INFO:tasks.workunit:Running workunit fs/misc/general_vxattrs.sh... 2023-11-27T15:28:51.404 DEBUG:teuthology.orchestra.run.smithi133:workunit test fs/misc/general_vxattrs.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh 2023-11-27T15:28:51.482 INFO:tasks.workunit.client.0.smithi133.stderr:+ mkdir -p dir 2023-11-27T15:28:51.487 INFO:tasks.workunit.client.0.smithi133.stderr:+ getfattr -n ceph.dir.pin dir 2023-11-27T15:28:51.487 INFO:tasks.workunit.client.0.smithi133.stderr:+ grep 'ceph.dir.pin="-1"' 2023-11-27T15:28:51.489 INFO:tasks.workunit.client.0.smithi133.stdout:ceph.dir.pin="-1" 2023-11-27T15:28:51.490 INFO:tasks.workunit.client.0.smithi133.stderr:+ setfattr -n ceph.dir.pin dir 2023-11-27T15:28:51.490 INFO:tasks.workunit.client.0.smithi133.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-27T15:28:51.492 INFO:tasks.workunit.client.0.smithi133.stdout:setfattr: dir: Invalid argument 2023-11-27T15:28:51.493 INFO:tasks.workunit.client.0.smithi133.stderr:+ setfattr -n ceph.dir.pin -v 1 dir 2023-11-27T15:28:51.496 INFO:tasks.workunit.client.0.smithi133.stderr:+ getfattr -n ceph.dir.pin dir 2023-11-27T15:28:51.496 INFO:tasks.workunit.client.0.smithi133.stderr:+ grep 'ceph.dir.pin="1"' 2023-11-27T15:28:51.499 INFO:tasks.workunit.client.0.smithi133.stdout:ceph.dir.pin="1" 2023-11-27T15:28:51.499 INFO:tasks.workunit.client.0.smithi133.stderr:+ setfattr -x ceph.dir.pin dir 2023-11-27T15:28:51.502 INFO:tasks.workunit.client.0.smithi133.stderr:+ getfattr -n ceph.dir.pin dir 2023-11-27T15:28:51.503 INFO:tasks.workunit.client.0.smithi133.stderr:+ grep 'ceph.dir.pin="-1"' 2023-11-27T15:28:51.505 INFO:tasks.workunit.client.0.smithi133.stdout:ceph.dir.pin="-1" 2023-11-27T15:28:51.506 INFO:tasks.workunit.client.0.smithi133.stderr:+ setfattr -n ceph.dir.pin.distributed dir 2023-11-27T15:28:51.507 INFO:tasks.workunit.client.0.smithi133.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-27T15:28:51.509 INFO:tasks.workunit.client.0.smithi133.stdout:setfattr: dir: Invalid argument 2023-11-27T15:28:51.509 INFO:tasks.workunit.client.0.smithi133.stderr:+ setfattr -n ceph.dir.pin.distributed -v 1 dir 2023-11-27T15:28:51.511 INFO:tasks.workunit.client.0.smithi133.stderr:+ setfattr -x ceph.dir.pin.distributed dir 2023-11-27T15:28:51.515 INFO:tasks.workunit.client.0.smithi133.stderr:+ setfattr -n ceph.dir.pin.random dir 2023-11-27T15:28:51.515 INFO:tasks.workunit.client.0.smithi133.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-27T15:28:51.517 INFO:tasks.workunit.client.0.smithi133.stdout:setfattr: dir: Invalid argument 2023-11-27T15:28:51.517 INFO:tasks.workunit.client.0.smithi133.stderr:+ setfattr -n ceph.dir.pin.random -v 0.01 dir 2023-11-27T15:28:51.520 INFO:tasks.workunit.client.0.smithi133.stderr:+ setfattr -x ceph.dir.pin.random dir 2023-11-27T15:28:51.524 INFO:tasks.workunit.client.0.smithi133.stderr:+ setfattr -n ceph.dir.subvolume dir 2023-11-27T15:28:51.524 INFO:tasks.workunit.client.0.smithi133.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-11-27T15:28:51.526 INFO:tasks.workunit.client.0.smithi133.stdout:setfattr: dir: Invalid argument 2023-11-27T15:28:51.526 INFO:tasks.workunit.client.0.smithi133.stderr:+ setfattr -n ceph.dir.subvolume -v 1 dir 2023-11-27T15:28:51.535 INFO:tasks.workunit.client.0.smithi133.stderr:setfattr: dir: Invalid argument 2023-11-27T15:28:51.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:28:51.536 INFO:tasks.workunit:Stopping ['fs/misc'] on client.0... 2023-11-27T15:28:51.536 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-11-27T15:28:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.560+0000 7f1482fff700 1 -- 172.21.15.133:0/631254475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f147c1002c0 msgr2=0x7f147c1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.560+0000 7f1482fff700 1 --2- 172.21.15.133:0/631254475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f147c1002c0 0x7f147c1006d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1470009990 tx=0x7f1470009ca0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.560+0000 7f1482fff700 1 -- 172.21.15.133:0/631254475 shutdown_connections 2023-11-27T15:28:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.560+0000 7f1482fff700 1 --2- 172.21.15.133:0/631254475 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f147c101e40 0x7f147c108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.560+0000 7f1482fff700 1 --2- 172.21.15.133:0/631254475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f147c1002c0 0x7f147c1006d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.560+0000 7f1482fff700 1 --2- 172.21.15.133:0/631254475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f147c1014b0 0x7f147c101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.560+0000 7f1482fff700 1 -- 172.21.15.133:0/631254475 >> 172.21.15.133:0/631254475 conn(0x7f147c0fb8e0 msgr2=0x7f147c0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.560+0000 7f1482fff700 1 -- 172.21.15.133:0/631254475 shutdown_connections 2023-11-27T15:28:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.560+0000 7f1482fff700 1 -- 172.21.15.133:0/631254475 wait complete. 2023-11-27T15:28:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.561+0000 7f1482fff700 1 Processor -- start 2023-11-27T15:28:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.561+0000 7f1482fff700 1 -- start start 2023-11-27T15:28:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.561+0000 7f1482fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f147c1002c0 0x7f147c198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.561+0000 7f1482fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f147c1014b0 0x7f147c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.561+0000 7f1482fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f147c101e40 0x7f147c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.561+0000 7f1482fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147c19fd90 con 0x7f147c1014b0 2023-11-27T15:28:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.561+0000 7f1482fff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147c19fed0 con 0x7f147c101e40 2023-11-27T15:28:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.561+0000 7f1482fff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147c1a0060 con 0x7f147c1002c0 2023-11-27T15:28:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.562+0000 7f147bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f147c1014b0 0x7f147c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.562+0000 7f147bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f147c1014b0 0x7f147c198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52678/0 (socket says 172.21.15.133:52678) 2023-11-27T15:28:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.562+0000 7f147bfff700 1 -- 172.21.15.133:0/2462223919 learned_addr learned my addr 172.21.15.133:0/2462223919 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.562+0000 7f1480d9b700 1 --2- 172.21.15.133:0/2462223919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f147c1002c0 0x7f147c198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.562+0000 7f148159c700 1 --2- 172.21.15.133:0/2462223919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f147c101e40 0x7f147c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.562+0000 7f147bfff700 1 -- 172.21.15.133:0/2462223919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f147c1002c0 msgr2=0x7f147c198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.562+0000 7f147bfff700 1 --2- 172.21.15.133:0/2462223919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f147c1002c0 0x7f147c198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.562+0000 7f147bfff700 1 -- 172.21.15.133:0/2462223919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f147c101e40 msgr2=0x7f147c19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.562+0000 7f147bfff700 1 --2- 172.21.15.133:0/2462223919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f147c101e40 0x7f147c19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.562+0000 7f147bfff700 1 -- 172.21.15.133:0/2462223919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14700095f0 con 0x7f147c1014b0 2023-11-27T15:28:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.562+0000 7f147bfff700 1 --2- 172.21.15.133:0/2462223919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f147c1014b0 0x7f147c198750 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f146800e9a0 tx=0x7f146800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.562+0000 7f1479ffb700 1 -- 172.21.15.133:0/2462223919 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468009790 con 0x7f147c1014b0 2023-11-27T15:28:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.562+0000 7f1479ffb700 1 -- 172.21.15.133:0/2462223919 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f146800cbd0 con 0x7f147c1014b0 2023-11-27T15:28:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.562+0000 7f1479ffb700 1 -- 172.21.15.133:0/2462223919 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468018680 con 0x7f147c1014b0 2023-11-27T15:28:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.562+0000 7f1482fff700 1 -- 172.21.15.133:0/2462223919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f147c10bc00 con 0x7f147c1014b0 2023-11-27T15:28:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.563+0000 7f1482fff700 1 -- 172.21.15.133:0/2462223919 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f147c10ba70 con 0x7f147c1014b0 2023-11-27T15:28:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.566+0000 7f1479ffb700 1 -- 172.21.15.133:0/2462223919 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f146800cd40 con 0x7f147c1014b0 2023-11-27T15:28:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.567+0000 7f149690bb80 1 Processor -- start 2023-11-27T15:28:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.567+0000 7f149690bb80 1 -- start start 2023-11-27T15:28:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.567+0000 7f149690bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf512d6b0 0x564bf52d1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.567+0000 7f149690bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf52d1db0 0x564bf52d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.567+0000 7f149690bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf52d2b10 0x564bf52d2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.567+0000 7f149690bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf51d7000 con 0x564bf52d1db0 2023-11-27T15:28:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.567+0000 7f149690bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf5229840 con 0x564bf512d6b0 2023-11-27T15:28:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.567+0000 7f149690bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf5222cd0 con 0x564bf52d2b10 2023-11-27T15:28:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.568+0000 7f1480d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf52d1db0 0x564bf52d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.568+0000 7f148159c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf512d6b0 0x564bf52d1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.568+0000 7f1480d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf52d1db0 0x564bf52d25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52692/0 (socket says 172.21.15.133:52692) 2023-11-27T15:28:51.572 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.568+0000 7f1480d9b700 1 -- 172.21.15.133:0/1636041112 learned_addr learned my addr 172.21.15.133:0/1636041112 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.568+0000 7f147bfff700 1 --2- 172.21.15.133:0/1636041112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf52d2b10 0x564bf52d2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.568+0000 7f148159c700 1 -- 172.21.15.133:0/1636041112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf52d2b10 msgr2=0x564bf52d2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.568+0000 7f148159c700 1 --2- 172.21.15.133:0/1636041112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf52d2b10 0x564bf52d2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.568+0000 7f148159c700 1 -- 172.21.15.133:0/1636041112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf52d1db0 msgr2=0x564bf52d25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.568+0000 7f148159c700 1 --2- 172.21.15.133:0/1636041112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf52d1db0 0x564bf52d25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.568+0000 7f148159c700 1 -- 172.21.15.133:0/1636041112 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bf50a45d0 con 0x564bf512d6b0 2023-11-27T15:28:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.568+0000 7f147bfff700 1 --2- 172.21.15.133:0/1636041112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf52d2b10 0x564bf52d2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.568+0000 7f1479ffb700 1 --2- 172.21.15.133:0/2462223919 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f146c0a11d0 0x7f146c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.568+0000 7f148159c700 1 --2- 172.21.15.133:0/1636041112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf512d6b0 0x564bf52d1870 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1474009cd0 tx=0x7f1474007500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=832636a27703448b server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.568+0000 7f1479ffb700 1 -- 172.21.15.133:0/2462223919 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1468014070 con 0x7f147c1014b0 2023-11-27T15:28:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.569+0000 7f1480d9b700 1 --2- 172.21.15.133:0/2462223919 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f146c0a11d0 0x7f146c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.569+0000 7f1466ffd700 1 -- 172.21.15.133:0/1636041112 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1474018070 con 0x564bf512d6b0 2023-11-27T15:28:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.569+0000 7f1466ffd700 1 -- 172.21.15.133:0/1636041112 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f147400f050 con 0x564bf512d6b0 2023-11-27T15:28:51.573 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.569+0000 7f1466ffd700 1 -- 172.21.15.133:0/1636041112 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1474005900 con 0x564bf512d6b0 2023-11-27T15:28:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.569+0000 7f149690bb80 1 -- 172.21.15.133:0/1636041112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf512d6b0 msgr2=0x564bf52d1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.569+0000 7f1480d9b700 1 --2- 172.21.15.133:0/2462223919 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f146c0a11d0 0x7f146c0a3680 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1470009990 tx=0x7f1470005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.569+0000 7f149690bb80 1 --2- 172.21.15.133:0/1636041112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf512d6b0 0x564bf52d1870 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1474009cd0 tx=0x7f1474007500 comp rx=0 tx=0).stop 2023-11-27T15:28:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.570+0000 7f149690bb80 1 -- 172.21.15.133:0/1636041112 shutdown_connections 2023-11-27T15:28:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.570+0000 7f149690bb80 1 --2- 172.21.15.133:0/1636041112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf52d2b10 0x564bf52d2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.570+0000 7f149690bb80 1 --2- 172.21.15.133:0/1636041112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf512d6b0 0x564bf52d1870 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.570+0000 7f149690bb80 1 --2- 172.21.15.133:0/1636041112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf52d1db0 0x564bf52d25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.570+0000 7f149690bb80 1 -- 172.21.15.133:0/1636041112 >> 172.21.15.133:0/1636041112 conn(0x564bf52c6f50 msgr2=0x564bf52cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.570+0000 7f149690bb80 1 -- 172.21.15.133:0/1636041112 shutdown_connections 2023-11-27T15:28:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.570+0000 7f149690bb80 1 -- 172.21.15.133:0/1636041112 wait complete. 2023-11-27T15:28:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.570+0000 7f149690bb80 1 -- start start 2023-11-27T15:28:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.570+0000 7f149690bb80 1 Processor -- start 2023-11-27T15:28:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.571+0000 7f149690bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf512d6b0 0x564bf52dac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.571+0000 7f149690bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf52d1db0 0x564bf52db1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.571+0000 7f149690bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf52d2b10 0x564bf52e21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.571+0000 7f149690bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf51c4880 con 0x564bf52d1db0 2023-11-27T15:28:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.571+0000 7f149690bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf51c4520 con 0x564bf52d2b10 2023-11-27T15:28:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.571+0000 7f149690bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf50a68c0 con 0x564bf512d6b0 2023-11-27T15:28:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.571+0000 7f149690bb80 10 client.0 _ll_register_callbacks cb 0x564bf51e2720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.571+0000 7f147bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf52d1db0 0x564bf52db1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.571+0000 7f1480d9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf512d6b0 0x564bf52dac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.571+0000 7f147bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf52d1db0 0x564bf52db1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52700/0 (socket says 172.21.15.133:52700) 2023-11-27T15:28:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.571+0000 7f1480d9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf512d6b0 0x564bf52dac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:36488/0 (socket says 172.21.15.133:36488) 2023-11-27T15:28:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.571+0000 7f147bfff700 1 -- 172.21.15.133:0/159412042 learned_addr learned my addr 172.21.15.133:0/159412042 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.571+0000 7f148159c700 1 --2- 172.21.15.133:0/159412042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf52d2b10 0x564bf52e21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.572+0000 7f147bfff700 1 -- 172.21.15.133:0/159412042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf512d6b0 msgr2=0x564bf52dac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.572+0000 7f147bfff700 1 --2- 172.21.15.133:0/159412042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf512d6b0 0x564bf52dac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.572+0000 7f147bfff700 1 -- 172.21.15.133:0/159412042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf52d2b10 msgr2=0x564bf52e21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.572+0000 7f147bfff700 1 --2- 172.21.15.133:0/159412042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf52d2b10 0x564bf52e21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.572+0000 7f147bfff700 1 -- 172.21.15.133:0/159412042 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1474009900 con 0x564bf52d1db0 2023-11-27T15:28:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.572+0000 7f148159c700 1 --2- 172.21.15.133:0/159412042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf52d2b10 0x564bf52e21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.572+0000 7f145bfff700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:28:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.572+0000 7f147bfff700 1 --2- 172.21.15.133:0/159412042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf52d1db0 0x564bf52db1a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f14680ca5e0 tx=0x7f146809a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a159c83b95736fa server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.572+0000 7f145bfff700 1 -- 172.21.15.133:0/159412042 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14680a20b0 con 0x564bf52d1db0 2023-11-27T15:28:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.572+0000 7f145bfff700 1 -- 172.21.15.133:0/159412042 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1468099ef0 con 0x564bf52d1db0 2023-11-27T15:28:51.578 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.572+0000 7f149690bb80 1 -- 172.21.15.133:0/159412042 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564bf52c4710 con 0x564bf52d1db0 2023-11-27T15:28:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.572+0000 7f145bfff700 1 -- 172.21.15.133:0/159412042 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f146809df20 con 0x564bf52d1db0 2023-11-27T15:28:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.573+0000 7f149690bb80 10 client.16383 fetch_fsmap learned FSMap version 30 2023-11-27T15:28:51.579 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.573+0000 7f149690bb80 1 -- 172.21.15.133:0/159412042 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564bf51e9130 con 0x564bf52d1db0 2023-11-27T15:28:51.580 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.574+0000 7f145bfff700 1 -- 172.21.15.133:0/159412042 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564bf52c4710 con 0x564bf52d1db0 2023-11-27T15:28:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.574+0000 7f145bfff700 1 -- 172.21.15.133:0/159412042 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1468098e50 con 0x564bf52d1db0 2023-11-27T15:28:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.574+0000 7f149690bb80 10 client.16383 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:28:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.574+0000 7f149690bb80 10 client.16383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:28:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.574+0000 7f149690bb80 20 client.16383 populate_metadata read hostname 'smithi133' 2023-11-27T15:28:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.574+0000 7f149690bb80 1 --2- 172.21.15.133:0/159412042 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564bf52e6a80 0x564bf52e8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.574+0000 7f149690bb80 4 client.16383 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:28:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.574+0000 7f149690bb80 1 -- 172.21.15.133:0/159412042 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564bf51e5040 con 0x564bf52e6a80 2023-11-27T15:28:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.574+0000 7f1480d9b700 1 --2- 172.21.15.133:0/159412042 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564bf52e6a80 0x564bf52e8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.575+0000 7f1480d9b700 1 --2- 172.21.15.133:0/159412042 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564bf52e6a80 0x564bf52e8e90 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=706b38f05d234294 server_cookie=25833aa27ef0c440 in_seq=0 out_seq=0 2023-11-27T15:28:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.575+0000 7f145bfff700 10 client.16383 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.576+0000 7f145bfff700 1 -- 172.21.15.133:0/159412042 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564bf51e5040 con 0x564bf52e6a80 2023-11-27T15:28:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.576+0000 7f145bfff700 10 client.16383 handle_command_reply: tid=0 2023-11-27T15:28:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.576+0000 7f149690bb80 1 client.16383 shutdown 2023-11-27T15:28:51.581 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.576+0000 7f149690bb80 1 -- 172.21.15.133:0/159412042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf52d1db0 msgr2=0x564bf52db1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.576+0000 7f149690bb80 1 --2- 172.21.15.133:0/159412042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf52d1db0 0x564bf52db1a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f14680ca5e0 tx=0x7f146809a430 comp rx=0 tx=0).stop 2023-11-27T15:28:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.576+0000 7f149690bb80 1 -- 172.21.15.133:0/159412042 shutdown_connections 2023-11-27T15:28:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.576+0000 7f149690bb80 1 --2- 172.21.15.133:0/159412042 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564bf52e6a80 0x564bf52e8e90 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.576+0000 7f149690bb80 1 --2- 172.21.15.133:0/159412042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf52d2b10 0x564bf52e21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.576+0000 7f149690bb80 1 --2- 172.21.15.133:0/159412042 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf512d6b0 0x564bf52dac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.576+0000 7f149690bb80 1 --2- 172.21.15.133:0/159412042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf52d1db0 0x564bf52db1a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.576+0000 7f149690bb80 1 -- 172.21.15.133:0/159412042 >> 172.21.15.133:0/159412042 conn(0x564bf52c6f50 msgr2=0x564bf52cb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.576+0000 7f145bfff700 0 client.16383 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.576+0000 7f149690bb80 1 -- 172.21.15.133:0/159412042 shutdown_connections 2023-11-27T15:28:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.576+0000 7f149690bb80 1 -- 172.21.15.133:0/159412042 wait complete. 2023-11-27T15:28:51.582 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.577+0000 7f149690bb80 20 client.16383 trim_cache size 0 max 16384 2023-11-27T15:28:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.593+0000 7f149690bb80 1 Processor -- start 2023-11-27T15:28:51.596 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.593+0000 7f149690bb80 1 -- start start 2023-11-27T15:28:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.594+0000 7f149690bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf512d6b0 0x564bf52d8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.594+0000 7f149690bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf52d1db0 0x564bf52e21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.594+0000 7f149690bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf52d2b10 0x564bf52dac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.594+0000 7f149690bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf52cd420 con 0x564bf52d1db0 2023-11-27T15:28:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.594+0000 7f149690bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf52dbb90 con 0x564bf512d6b0 2023-11-27T15:28:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.594+0000 7f149690bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf52da7d0 con 0x564bf52d2b10 2023-11-27T15:28:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.594+0000 7f147bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf52d1db0 0x564bf52e21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:51.598 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.594+0000 7f147bfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf52d1db0 0x564bf52e21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52706/0 (socket says 172.21.15.133:52706) 2023-11-27T15:28:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.594+0000 7f147bfff700 1 -- 172.21.15.133:0/1672253382 learned_addr learned my addr 172.21.15.133:0/1672253382 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.594+0000 7f148159c700 1 --2- 172.21.15.133:0/1672253382 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf52d2b10 0x564bf52dac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.594+0000 7f1480d9b700 1 --2- 172.21.15.133:0/1672253382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf512d6b0 0x564bf52d8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.594+0000 7f147bfff700 1 -- 172.21.15.133:0/1672253382 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf52d2b10 msgr2=0x564bf52dac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.594+0000 7f147bfff700 1 --2- 172.21.15.133:0/1672253382 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf52d2b10 0x564bf52dac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.594+0000 7f147bfff700 1 -- 172.21.15.133:0/1672253382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf512d6b0 msgr2=0x564bf52d8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.594+0000 7f147bfff700 1 --2- 172.21.15.133:0/1672253382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf512d6b0 0x564bf52d8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.594+0000 7f147bfff700 1 -- 172.21.15.133:0/1672253382 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bf51e8e80 con 0x564bf52d1db0 2023-11-27T15:28:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.594+0000 7f1480d9b700 1 --2- 172.21.15.133:0/1672253382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf512d6b0 0x564bf52d8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.595+0000 7f147bfff700 1 --2- 172.21.15.133:0/1672253382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf52d1db0 0x564bf52e21a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1468099bf0 tx=0x7f146800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7d17af6b496d1e server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.595+0000 7f14677fe700 1 -- 172.21.15.133:0/1672253382 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468099ef0 con 0x564bf52d1db0 2023-11-27T15:28:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.595+0000 7f14677fe700 1 -- 172.21.15.133:0/1672253382 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f146809df20 con 0x564bf52d1db0 2023-11-27T15:28:51.599 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.595+0000 7f14677fe700 1 -- 172.21.15.133:0/1672253382 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468094d20 con 0x564bf52d1db0 2023-11-27T15:28:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.595+0000 7f149690bb80 1 -- 172.21.15.133:0/1672253382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf52d1db0 msgr2=0x564bf52e21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.595+0000 7f149690bb80 1 --2- 172.21.15.133:0/1672253382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf52d1db0 0x564bf52e21a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1468099bf0 tx=0x7f146800eef0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.596+0000 7f149690bb80 1 -- 172.21.15.133:0/1672253382 shutdown_connections 2023-11-27T15:28:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.596+0000 7f149690bb80 1 --2- 172.21.15.133:0/1672253382 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf52d2b10 0x564bf52dac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.596+0000 7f149690bb80 1 --2- 172.21.15.133:0/1672253382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf512d6b0 0x564bf52d8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.596+0000 7f149690bb80 1 --2- 172.21.15.133:0/1672253382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf52d1db0 0x564bf52e21a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.602 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.596+0000 7f149690bb80 1 -- 172.21.15.133:0/1672253382 >> 172.21.15.133:0/1672253382 conn(0x564bf52c6f50 msgr2=0x564bf52ed7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.596+0000 7f149690bb80 1 -- 172.21.15.133:0/1672253382 shutdown_connections 2023-11-27T15:28:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.596+0000 7f149690bb80 1 -- 172.21.15.133:0/1672253382 wait complete. 2023-11-27T15:28:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.596+0000 7f149690bb80 1 -- start start 2023-11-27T15:28:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.596+0000 7f149690bb80 1 Processor -- start 2023-11-27T15:28:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.597+0000 7f149690bb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf512d6b0 0x564bf52de3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.597+0000 7f149690bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf52d1db0 0x564bf52de910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.597+0000 7f149690bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf52d2b10 0x564bf52e12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.597+0000 7f149690bb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf52e1810 con 0x564bf512d6b0 2023-11-27T15:28:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.597+0000 7f149690bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf52e1980 con 0x564bf52d1db0 2023-11-27T15:28:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.597+0000 7f149690bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bf52e1b40 con 0x564bf52d2b10 2023-11-27T15:28:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.597+0000 7f149690bb80 10 client.0 _ll_register_callbacks cb 0x7f1468096230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.597+0000 7f1480d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf512d6b0 0x564bf52de3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.597+0000 7f147bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf52d1db0 0x564bf52de910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:51.603 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.597+0000 7f1480d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf512d6b0 0x564bf52de3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:52722/0 (socket says 172.21.15.133:52722) 2023-11-27T15:28:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.597+0000 7f1480d9b700 1 -- 172.21.15.133:0/4169963906 learned_addr learned my addr 172.21.15.133:0/4169963906 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.597+0000 7f148159c700 1 --2- 172.21.15.133:0/4169963906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf52d2b10 0x564bf52e12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.597+0000 7f1480d9b700 1 -- 172.21.15.133:0/4169963906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf52d2b10 msgr2=0x564bf52e12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.597+0000 7f1480d9b700 1 --2- 172.21.15.133:0/4169963906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf52d2b10 0x564bf52e12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.597+0000 7f1480d9b700 1 -- 172.21.15.133:0/4169963906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf52d1db0 msgr2=0x564bf52de910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.597+0000 7f1480d9b700 1 --2- 172.21.15.133:0/4169963906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf52d1db0 0x564bf52de910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.597+0000 7f1480d9b700 1 -- 172.21.15.133:0/4169963906 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564bf51e9130 con 0x564bf512d6b0 2023-11-27T15:28:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.597+0000 7f148159c700 1 --2- 172.21.15.133:0/4169963906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf52d2b10 0x564bf52e12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:28:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.597+0000 7f147bfff700 1 --2- 172.21.15.133:0/4169963906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf52d1db0 0x564bf52de910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:28:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.598+0000 7f1480d9b700 1 --2- 172.21.15.133:0/4169963906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf512d6b0 0x564bf52de3d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f14700098e0 tx=0x7f1470009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e3a92f1d63544f server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.598+0000 7f145bfff700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:28:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.598+0000 7f145bfff700 1 -- 172.21.15.133:0/4169963906 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1470037560 con 0x564bf512d6b0 2023-11-27T15:28:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.598+0000 7f145bfff700 1 -- 172.21.15.133:0/4169963906 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f147000f460 con 0x564bf512d6b0 2023-11-27T15:28:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.598+0000 7f149690bb80 1 -- 172.21.15.133:0/4169963906 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f147400f050 con 0x564bf512d6b0 2023-11-27T15:28:51.604 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.598+0000 7f145bfff700 1 -- 172.21.15.133:0/4169963906 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f147000f620 con 0x564bf512d6b0 2023-11-27T15:28:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.598+0000 7f149690bb80 10 client.16389 fetch_fsmap learned FSMap version 30 2023-11-27T15:28:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.598+0000 7f149690bb80 1 -- 172.21.15.133:0/4169963906 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x564bf5039c00 con 0x564bf512d6b0 2023-11-27T15:28:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.598+0000 7f145bfff700 1 -- 172.21.15.133:0/4169963906 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f147400f050 con 0x564bf512d6b0 2023-11-27T15:28:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.599+0000 7f145bfff700 1 -- 172.21.15.133:0/4169963906 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f1470026790 con 0x564bf512d6b0 2023-11-27T15:28:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.599+0000 7f149690bb80 10 client.16389 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:28:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.599+0000 7f149690bb80 10 client.16389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-11-27T15:28:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.599+0000 7f149690bb80 20 client.16389 populate_metadata read hostname 'smithi133' 2023-11-27T15:28:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.599+0000 7f149690bb80 1 --2- 172.21.15.133:0/4169963906 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564bf533b570 0x564bf533d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.599+0000 7f149690bb80 4 client.16389 mds_command: new command op to 24560 tid=0[{"prefix": "flush journal"}] 2023-11-27T15:28:51.605 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.599+0000 7f149690bb80 1 -- 172.21.15.133:0/4169963906 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564bf51f1600 con 0x564bf533b570 2023-11-27T15:28:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.599+0000 7f147bfff700 1 --2- 172.21.15.133:0/4169963906 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564bf533b570 0x564bf533d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.600+0000 7f147bfff700 1 --2- 172.21.15.133:0/4169963906 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564bf533b570 0x564bf533d980 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8e60371e643c3ed server_cookie=4edb4ed584bd61b3 in_seq=0 out_seq=0 2023-11-27T15:28:51.606 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.600+0000 7f145bfff700 10 client.16389 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.616+0000 7f145bfff700 1 -- 172.21.15.133:0/4169963906 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564bf51f1600 con 0x564bf533b570 2023-11-27T15:28:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.616+0000 7f145bfff700 10 client.16389 handle_command_reply: tid=0 2023-11-27T15:28:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.616+0000 7f149690bb80 1 client.16389 shutdown 2023-11-27T15:28:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.616+0000 7f149690bb80 1 -- 172.21.15.133:0/4169963906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf512d6b0 msgr2=0x564bf52de3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.616+0000 7f149690bb80 1 --2- 172.21.15.133:0/4169963906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf512d6b0 0x564bf52de3d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f14700098e0 tx=0x7f1470009310 comp rx=0 tx=0).stop 2023-11-27T15:28:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.616+0000 7f149690bb80 1 -- 172.21.15.133:0/4169963906 shutdown_connections 2023-11-27T15:28:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.616+0000 7f149690bb80 1 --2- 172.21.15.133:0/4169963906 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x564bf533b570 0x564bf533d980 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.616+0000 7f149690bb80 1 --2- 172.21.15.133:0/4169963906 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x564bf52d2b10 0x564bf52e12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.616+0000 7f149690bb80 1 --2- 172.21.15.133:0/4169963906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564bf512d6b0 0x564bf52de3d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.616+0000 7f145bfff700 0 client.16389 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.616+0000 7f149690bb80 1 --2- 172.21.15.133:0/4169963906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564bf52d1db0 0x564bf52de910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.620 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.616+0000 7f149690bb80 1 -- 172.21.15.133:0/4169963906 >> 172.21.15.133:0/4169963906 conn(0x564bf52c6f50 msgr2=0x564bf52d8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.616+0000 7f149690bb80 1 -- 172.21.15.133:0/4169963906 shutdown_connections 2023-11-27T15:28:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.616+0000 7f149690bb80 1 -- 172.21.15.133:0/4169963906 wait complete. 2023-11-27T15:28:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.616+0000 7f149690bb80 20 client.16389 trim_cache size 0 max 16384 2023-11-27T15:28:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:{ 2023-11-27T15:28:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "message": "", 2023-11-27T15:28:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout: "return_code": 0 2023-11-27T15:28:51.621 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stdout:} 2023-11-27T15:28:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.617+0000 7f1482fff700 1 -- 172.21.15.133:0/2462223919 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f146c0a11d0 msgr2=0x7f146c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.617+0000 7f1482fff700 1 --2- 172.21.15.133:0/2462223919 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f146c0a11d0 0x7f146c0a3680 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1470009990 tx=0x7f1470005fb0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.617+0000 7f1482fff700 1 -- 172.21.15.133:0/2462223919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f147c1014b0 msgr2=0x7f147c198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.617+0000 7f1482fff700 1 --2- 172.21.15.133:0/2462223919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f147c1014b0 0x7f147c198750 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f146800e9a0 tx=0x7f146800ecb0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.617+0000 7f1482fff700 1 -- 172.21.15.133:0/2462223919 shutdown_connections 2023-11-27T15:28:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.617+0000 7f1482fff700 1 --2- 172.21.15.133:0/2462223919 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f146c0a11d0 0x7f146c0a3680 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.617+0000 7f1482fff700 1 --2- 172.21.15.133:0/2462223919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f147c101e40 0x7f147c19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.617+0000 7f1482fff700 1 --2- 172.21.15.133:0/2462223919 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f147c1002c0 0x7f147c198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.622 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.617+0000 7f1482fff700 1 --2- 172.21.15.133:0/2462223919 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f147c1014b0 0x7f147c198750 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.617+0000 7f1482fff700 1 -- 172.21.15.133:0/2462223919 >> 172.21.15.133:0/2462223919 conn(0x7f147c0fb8e0 msgr2=0x7f147c106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.617+0000 7f1482fff700 1 -- 172.21.15.133:0/2462223919 shutdown_connections 2023-11-27T15:28:51.623 INFO:teuthology.task.background_exec.ubuntu@smithi133.front.sepia.ceph.com.smithi133.stderr:2023-11-27T15:28:51.617+0000 7f1482fff700 1 -- 172.21.15.133:0/2462223919 wait complete. 2023-11-27T15:28:52.894 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-11-27T15:28:53.100 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.097+0000 7fd3e0dff700 1 -- 172.21.15.133:0/1970375239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3dc1081a0 msgr2=0x7fd3dc10a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:53.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.097+0000 7fd3e0dff700 1 --2- 172.21.15.133:0/1970375239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3dc1081a0 0x7fd3dc10a620 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400b0a0 tx=0x7fd3d400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.097+0000 7fd3e0dff700 1 -- 172.21.15.133:0/1970375239 shutdown_connections 2023-11-27T15:28:53.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.097+0000 7fd3e0dff700 1 --2- 172.21.15.133:0/1970375239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3dc1081a0 0x7fd3dc10a620 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.097+0000 7fd3e0dff700 1 --2- 172.21.15.133:0/1970375239 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3dc074ad0 0x7fd3dc072f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.097+0000 7fd3e0dff700 1 --2- 172.21.15.133:0/1970375239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3dc0734f0 0x7fd3dc073960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.097+0000 7fd3e0dff700 1 -- 172.21.15.133:0/1970375239 >> 172.21.15.133:0/1970375239 conn(0x7fd3dc0fb930 msgr2=0x7fd3dc0fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:53.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.097+0000 7fd3e0dff700 1 -- 172.21.15.133:0/1970375239 shutdown_connections 2023-11-27T15:28:53.101 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.097+0000 7fd3e0dff700 1 -- 172.21.15.133:0/1970375239 wait complete. 2023-11-27T15:28:53.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.098+0000 7fd3e0dff700 1 Processor -- start 2023-11-27T15:28:53.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.098+0000 7fd3e0dff700 1 -- start start 2023-11-27T15:28:53.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.098+0000 7fd3e0dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3dc0734f0 0x7fd3dc198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.098+0000 7fd3e0dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3dc074ad0 0x7fd3dc198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.103 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.098+0000 7fd3e0dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3dc1081a0 0x7fd3dc19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.098+0000 7fd3e0dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc19fe80 con 0x7fd3dc074ad0 2023-11-27T15:28:53.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.099+0000 7fd3e0dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc19ffc0 con 0x7fd3dc0734f0 2023-11-27T15:28:53.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.099+0000 7fd3e0dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc1a0150 con 0x7fd3dc1081a0 2023-11-27T15:28:53.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.099+0000 7fd3d9d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3dc074ad0 0x7fd3dc198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.099+0000 7fd3d9d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3dc074ad0 0x7fd3dc198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41812/0 (socket says 172.21.15.133:41812) 2023-11-27T15:28:53.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.099+0000 7fd3d9d9b700 1 -- 172.21.15.133:0/320127449 learned_addr learned my addr 172.21.15.133:0/320127449 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:53.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.099+0000 7fd3da59c700 1 --2- 172.21.15.133:0/320127449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3dc0734f0 0x7fd3dc198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.099+0000 7fd3dad9d700 1 --2- 172.21.15.133:0/320127449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3dc1081a0 0x7fd3dc19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.099+0000 7fd3d9d9b700 1 -- 172.21.15.133:0/320127449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3dc1081a0 msgr2=0x7fd3dc19f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:53.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.099+0000 7fd3d9d9b700 1 --2- 172.21.15.133:0/320127449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3dc1081a0 0x7fd3dc19f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.099+0000 7fd3d9d9b700 1 -- 172.21.15.133:0/320127449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3dc0734f0 msgr2=0x7fd3dc198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:53.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.099+0000 7fd3d9d9b700 1 --2- 172.21.15.133:0/320127449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3dc0734f0 0x7fd3dc198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.099+0000 7fd3d9d9b700 1 -- 172.21.15.133:0/320127449 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d4009d00 con 0x7fd3dc074ad0 2023-11-27T15:28:53.104 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.099+0000 7fd3d9d9b700 1 --2- 172.21.15.133:0/320127449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3dc074ad0 0x7fd3dc198840 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000d950 tx=0x7fd3d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:53.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.100+0000 7fd3cb7fe700 1 -- 172.21.15.133:0/320127449 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d000f890 con 0x7fd3dc074ad0 2023-11-27T15:28:53.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.100+0000 7fd3cb7fe700 1 -- 172.21.15.133:0/320127449 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3d0010460 con 0x7fd3dc074ad0 2023-11-27T15:28:53.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.100+0000 7fd3e0dff700 1 -- 172.21.15.133:0/320127449 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3dc10deb0 con 0x7fd3dc074ad0 2023-11-27T15:28:53.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.100+0000 7fd3cb7fe700 1 -- 172.21.15.133:0/320127449 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d000e610 con 0x7fd3dc074ad0 2023-11-27T15:28:53.105 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.100+0000 7fd3e0dff700 1 -- 172.21.15.133:0/320127449 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3dc10dd20 con 0x7fd3dc074ad0 2023-11-27T15:28:53.106 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.103+0000 7fd3cb7fe700 1 -- 172.21.15.133:0/320127449 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd3d00108d0 con 0x7fd3dc074ad0 2023-11-27T15:28:53.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.104+0000 7fd3f475fb80 1 Processor -- start 2023-11-27T15:28:53.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.104+0000 7fd3f475fb80 1 -- start start 2023-11-27T15:28:53.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.104+0000 7fd3f475fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c376da5c00 0x55c376da6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.104+0000 7fd3f475fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da6550 0x55c376da6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.108 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.104+0000 7fd3f475fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da7110 0x55c376da7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.104+0000 7fd3f475fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c376cab060 con 0x55c376da7110 2023-11-27T15:28:53.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.105+0000 7fd3f475fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c376cfd8a0 con 0x55c376da5c00 2023-11-27T15:28:53.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.105+0000 7fd3f475fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c376cf6d30 con 0x55c376da6550 2023-11-27T15:28:53.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.105+0000 7fd3d9d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da7110 0x55c376da7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.105+0000 7fd3dad9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c376da5c00 0x55c376da6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.109 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.105+0000 7fd3cb7fe700 1 --2- 172.21.15.133:0/320127449 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd3c40a1150 0x7fd3c40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.105+0000 7fd3d9d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da7110 0x55c376da7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41816/0 (socket says 172.21.15.133:41816) 2023-11-27T15:28:53.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.105+0000 7fd3dad9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c376da5c00 0x55c376da6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:50812/0 (socket says 172.21.15.133:50812) 2023-11-27T15:28:53.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.105+0000 7fd3d9d9b700 1 -- 172.21.15.133:0/2249870673 learned_addr learned my addr 172.21.15.133:0/2249870673 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:53.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.105+0000 7fd3da59c700 1 --2- 172.21.15.133:0/2249870673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da6550 0x55c376da6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.105+0000 7fd3cb7fe700 1 -- 172.21.15.133:0/320127449 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd3d00ccaa0 con 0x7fd3dc074ad0 2023-11-27T15:28:53.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.105+0000 7fd3da59c700 1 -- 172.21.15.133:0/2249870673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c376da5c00 msgr2=0x55c376da6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:53.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.105+0000 7fd3da59c700 1 --2- 172.21.15.133:0/2249870673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c376da5c00 0x55c376da6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.105+0000 7fd3da59c700 1 -- 172.21.15.133:0/2249870673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da7110 msgr2=0x55c376da7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:53.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.105+0000 7fd3da59c700 1 --2- 172.21.15.133:0/2249870673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da7110 0x55c376da7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.105+0000 7fd3dad9d700 1 --2- 172.21.15.133:0/320127449 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd3c40a1150 0x7fd3c40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.105+0000 7fd3da59c700 1 -- 172.21.15.133:0/2249870673 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c376b78630 con 0x55c376da6550 2023-11-27T15:28:53.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.106+0000 7fd3da59c700 1 --2- 172.21.15.133:0/2249870673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da6550 0x55c376da6b30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc007bf0 tx=0x7fd3cc007f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec8b2526943f3566 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:53.110 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.106+0000 7fd3c8ff9700 1 -- 172.21.15.133:0/2249870673 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc010b80 con 0x55c376da6550 2023-11-27T15:28:53.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.106+0000 7fd3c8ff9700 1 -- 172.21.15.133:0/2249870673 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3cc015070 con 0x55c376da6550 2023-11-27T15:28:53.111 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.106+0000 7fd3c8ff9700 1 -- 172.21.15.133:0/2249870673 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc004e60 con 0x55c376da6550 2023-11-27T15:28:53.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.107+0000 7fd3f475fb80 1 -- 172.21.15.133:0/2249870673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da6550 msgr2=0x55c376da6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:53.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.107+0000 7fd3dad9d700 1 --2- 172.21.15.133:0/320127449 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd3c40a1150 0x7fd3c40a3600 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400b830 tx=0x7fd3d400b3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:53.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.107+0000 7fd3f475fb80 1 --2- 172.21.15.133:0/2249870673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da6550 0x55c376da6b30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc007bf0 tx=0x7fd3cc007f00 comp rx=0 tx=0).stop 2023-11-27T15:28:53.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.107+0000 7fd3f475fb80 1 -- 172.21.15.133:0/2249870673 shutdown_connections 2023-11-27T15:28:53.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.107+0000 7fd3f475fb80 1 --2- 172.21.15.133:0/2249870673 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da7110 0x55c376da7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.107+0000 7fd3f475fb80 1 --2- 172.21.15.133:0/2249870673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c376da5c00 0x55c376da6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.113 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.107+0000 7fd3f475fb80 1 --2- 172.21.15.133:0/2249870673 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da6550 0x55c376da6b30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.107+0000 7fd3f475fb80 1 -- 172.21.15.133:0/2249870673 >> 172.21.15.133:0/2249870673 conn(0x55c376d9b360 msgr2=0x55c376da0670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:53.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.107+0000 7fd3f475fb80 1 -- 172.21.15.133:0/2249870673 shutdown_connections 2023-11-27T15:28:53.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.107+0000 7fd3f475fb80 1 -- 172.21.15.133:0/2249870673 wait complete. 2023-11-27T15:28:53.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.107+0000 7fd3f475fb80 1 -- start start 2023-11-27T15:28:53.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.108+0000 7fd3f475fb80 1 Processor -- start 2023-11-27T15:28:53.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.108+0000 7fd3f475fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da5c00 0x55c376daf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.108+0000 7fd3f475fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da6550 0x55c376dafb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.108+0000 7fd3f475fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c376da7110 0x55c376db6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.108+0000 7fd3f475fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c376c988e0 con 0x55c376da6550 2023-11-27T15:28:53.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.108+0000 7fd3f475fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c376c98580 con 0x55c376da7110 2023-11-27T15:28:53.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.108+0000 7fd3f475fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c376b7a920 con 0x55c376da5c00 2023-11-27T15:28:53.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.108+0000 7fd3f475fb80 10 client.0 _ll_register_callbacks cb 0x55c376cb6780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:53.114 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.108+0000 7fd3d9d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da6550 0x55c376dafb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.108+0000 7fd3da59c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da5c00 0x55c376daf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.108+0000 7fd3d9d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da6550 0x55c376dafb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41820/0 (socket says 172.21.15.133:41820) 2023-11-27T15:28:53.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.108+0000 7fd3d9d9b700 1 -- 172.21.15.133:0/30494912 learned_addr learned my addr 172.21.15.133:0/30494912 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:53.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.108+0000 7fd3dad9d700 1 --2- 172.21.15.133:0/30494912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c376da7110 0x55c376db6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.109+0000 7fd3da59c700 1 -- 172.21.15.133:0/30494912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c376da7110 msgr2=0x55c376db6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:53.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.109+0000 7fd3da59c700 1 --2- 172.21.15.133:0/30494912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c376da7110 0x55c376db6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.109+0000 7fd3da59c700 1 -- 172.21.15.133:0/30494912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da6550 msgr2=0x55c376dafb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:53.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.109+0000 7fd3da59c700 1 --2- 172.21.15.133:0/30494912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da6550 0x55c376dafb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.109+0000 7fd3da59c700 1 -- 172.21.15.133:0/30494912 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3cc0078a0 con 0x55c376da5c00 2023-11-27T15:28:53.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.109+0000 7fd3d9d9b700 1 --2- 172.21.15.133:0/30494912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da6550 0x55c376dafb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:28:53.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.109+0000 7fd3da59c700 1 --2- 172.21.15.133:0/30494912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da5c00 0x55c376daf5e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0106b0 tx=0x7fd3cc010b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad2f9d50ed46ab40 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:53.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.109+0000 7fd3c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:28:53.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.110+0000 7fd3c1ffb700 1 -- 172.21.15.133:0/30494912 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc01f070 con 0x55c376da5c00 2023-11-27T15:28:53.115 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.110+0000 7fd3c1ffb700 1 -- 172.21.15.133:0/30494912 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd3cc011440 con 0x55c376da5c00 2023-11-27T15:28:53.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.110+0000 7fd3f475fb80 1 -- 172.21.15.133:0/30494912 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c376d98b20 con 0x55c376da5c00 2023-11-27T15:28:53.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.110+0000 7fd3c1ffb700 1 -- 172.21.15.133:0/30494912 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3cc011600 con 0x55c376da5c00 2023-11-27T15:28:53.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.110+0000 7fd3f475fb80 10 client.26515 fetch_fsmap learned FSMap version 30 2023-11-27T15:28:53.116 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.110+0000 7fd3f475fb80 1 -- 172.21.15.133:0/30494912 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c376cbd190 con 0x55c376da5c00 2023-11-27T15:28:53.117 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.110+0000 7fd3c1ffb700 1 -- 172.21.15.133:0/30494912 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c376d98b20 con 0x55c376da5c00 2023-11-27T15:28:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.110+0000 7fd3c1ffb700 1 -- 172.21.15.133:0/30494912 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fd3cc019950 con 0x55c376da5c00 2023-11-27T15:28:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.110+0000 7fd3f475fb80 10 client.26515 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:28:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.110+0000 7fd3f475fb80 10 client.26515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:28:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.110+0000 7fd3f475fb80 20 client.26515 populate_metadata read hostname 'smithi133' 2023-11-27T15:28:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.110+0000 7fd3f475fb80 1 --2- 172.21.15.133:0/30494912 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c376dbb2c0 0x55c376dbd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.110+0000 7fd3f475fb80 4 client.26515 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:28:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.110+0000 7fd3f475fb80 1 -- 172.21.15.133:0/30494912 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c376cb90a0 con 0x55c376dbb2c0 2023-11-27T15:28:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.111+0000 7fd3d9d9b700 1 --2- 172.21.15.133:0/30494912 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c376dbb2c0 0x55c376dbd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.111+0000 7fd3d9d9b700 1 --2- 172.21.15.133:0/30494912 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c376dbb2c0 0x55c376dbd6d0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=247c3c539ec8d35c server_cookie=5880c1d1f867de15 in_seq=0 out_seq=0 2023-11-27T15:28:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.111+0000 7fd3c1ffb700 10 client.26515 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:53.118 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.112+0000 7fd3c1ffb700 1 -- 172.21.15.133:0/30494912 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c376cb90a0 con 0x55c376dbb2c0 2023-11-27T15:28:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.112+0000 7fd3c1ffb700 10 client.26515 handle_command_reply: tid=0 2023-11-27T15:28:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.112+0000 7fd3f475fb80 1 client.26515 shutdown 2023-11-27T15:28:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.112+0000 7fd3f475fb80 1 -- 172.21.15.133:0/30494912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da5c00 msgr2=0x55c376daf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.112+0000 7fd3f475fb80 1 --2- 172.21.15.133:0/30494912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da5c00 0x55c376daf5e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0106b0 tx=0x7fd3cc010b90 comp rx=0 tx=0).stop 2023-11-27T15:28:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.112+0000 7fd3f475fb80 1 -- 172.21.15.133:0/30494912 shutdown_connections 2023-11-27T15:28:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.112+0000 7fd3f475fb80 1 --2- 172.21.15.133:0/30494912 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c376dbb2c0 0x55c376dbd6d0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.112+0000 7fd3f475fb80 1 --2- 172.21.15.133:0/30494912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c376da7110 0x55c376db6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.112+0000 7fd3f475fb80 1 --2- 172.21.15.133:0/30494912 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da5c00 0x55c376daf5e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.112+0000 7fd3c1ffb700 0 client.26515 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.112+0000 7fd3f475fb80 1 --2- 172.21.15.133:0/30494912 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da6550 0x55c376dafb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.112+0000 7fd3f475fb80 1 -- 172.21.15.133:0/30494912 >> 172.21.15.133:0/30494912 conn(0x55c376d9b360 msgr2=0x55c376d9fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.112+0000 7fd3f475fb80 1 -- 172.21.15.133:0/30494912 shutdown_connections 2023-11-27T15:28:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.112+0000 7fd3f475fb80 1 -- 172.21.15.133:0/30494912 wait complete. 2023-11-27T15:28:53.119 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.113+0000 7fd3f475fb80 20 client.26515 trim_cache size 0 max 16384 2023-11-27T15:28:53.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.130+0000 7fd3f475fb80 1 Processor -- start 2023-11-27T15:28:53.133 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.130+0000 7fd3f475fb80 1 -- start start 2023-11-27T15:28:53.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.130+0000 7fd3f475fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da5c00 0x55c376da9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.130+0000 7fd3f475fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c376da6550 0x55c376db6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.130+0000 7fd3f475fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da7110 0x55c376daa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.131+0000 7fd3f475fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c376da7a00 con 0x55c376da7110 2023-11-27T15:28:53.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.131+0000 7fd3f475fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c376db0760 con 0x55c376da6550 2023-11-27T15:28:53.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.131+0000 7fd3f475fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c376da1aa0 con 0x55c376da5c00 2023-11-27T15:28:53.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.131+0000 7fd3dad9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da7110 0x55c376daa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.131+0000 7fd3dad9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da7110 0x55c376daa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41828/0 (socket says 172.21.15.133:41828) 2023-11-27T15:28:53.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.131+0000 7fd3dad9d700 1 -- 172.21.15.133:0/2782835032 learned_addr learned my addr 172.21.15.133:0/2782835032 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:53.136 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.131+0000 7fd3da59c700 1 --2- 172.21.15.133:0/2782835032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da5c00 0x55c376da9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.131+0000 7fd3d9d9b700 1 --2- 172.21.15.133:0/2782835032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c376da6550 0x55c376db6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.131+0000 7fd3dad9d700 1 -- 172.21.15.133:0/2782835032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da5c00 msgr2=0x55c376da9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:53.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.131+0000 7fd3dad9d700 1 --2- 172.21.15.133:0/2782835032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da5c00 0x55c376da9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.131+0000 7fd3dad9d700 1 -- 172.21.15.133:0/2782835032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c376da6550 msgr2=0x55c376db6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:53.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.131+0000 7fd3dad9d700 1 --2- 172.21.15.133:0/2782835032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c376da6550 0x55c376db6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.131+0000 7fd3dad9d700 1 -- 172.21.15.133:0/2782835032 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c376cbcee0 con 0x55c376da7110 2023-11-27T15:28:53.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.132+0000 7fd3dad9d700 1 --2- 172.21.15.133:0/2782835032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da7110 0x55c376daa520 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400b040 tx=0x7fd3d4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf2cff3549c2265 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:53.137 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.132+0000 7fd3c97fa700 1 -- 172.21.15.133:0/2782835032 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d400e040 con 0x55c376da7110 2023-11-27T15:28:53.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.132+0000 7fd3c97fa700 1 -- 172.21.15.133:0/2782835032 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3d4020480 con 0x55c376da7110 2023-11-27T15:28:53.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.132+0000 7fd3c97fa700 1 -- 172.21.15.133:0/2782835032 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d40052a0 con 0x55c376da7110 2023-11-27T15:28:53.139 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi133 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh' 2023-11-27T15:28:53.230 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=6d08514d206b4b3bafe35b4f1ce9f14b Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_5e4a743ad7fd1422ebd6c4dc950a21a303279207/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi133 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh' 2023-11-27T15:28:53.232 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-11-27T15:28:53.241 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi133.front.sepia.ceph.com 2023-11-27T15:28:53.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.133+0000 7fd3f475fb80 1 -- 172.21.15.133:0/2782835032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da7110 msgr2=0x55c376daa520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:53.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.133+0000 7fd3f475fb80 1 --2- 172.21.15.133:0/2782835032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da7110 0x55c376daa520 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400b040 tx=0x7fd3d4009b80 comp rx=0 tx=0).stop 2023-11-27T15:28:53.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.133+0000 7fd3f475fb80 1 -- 172.21.15.133:0/2782835032 shutdown_connections 2023-11-27T15:28:53.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.133+0000 7fd3f475fb80 1 --2- 172.21.15.133:0/2782835032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da7110 0x55c376daa520 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.133+0000 7fd3f475fb80 1 --2- 172.21.15.133:0/2782835032 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da5c00 0x55c376da9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.133+0000 7fd3f475fb80 1 --2- 172.21.15.133:0/2782835032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c376da6550 0x55c376db6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.133+0000 7fd3f475fb80 1 -- 172.21.15.133:0/2782835032 >> 172.21.15.133:0/2782835032 conn(0x55c376d9b360 msgr2=0x55c376dae2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:53.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.133+0000 7fd3f475fb80 1 -- 172.21.15.133:0/2782835032 shutdown_connections 2023-11-27T15:28:53.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.133+0000 7fd3f475fb80 1 -- 172.21.15.133:0/2782835032 wait complete. 2023-11-27T15:28:53.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.133+0000 7fd3f475fb80 1 -- start start 2023-11-27T15:28:53.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.133+0000 7fd3f475fb80 1 Processor -- start 2023-11-27T15:28:53.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.134+0000 7fd3f475fb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da5c00 0x55c376db25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.134+0000 7fd3f475fb80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da6550 0x55c376db2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.134+0000 7fd3f475fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c376da7110 0x55c376db5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.134+0000 7fd3f475fb80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c376db5f70 con 0x55c376da6550 2023-11-27T15:28:53.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.134+0000 7fd3f475fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c376db60e0 con 0x55c376da7110 2023-11-27T15:28:53.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.134+0000 7fd3f475fb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c376db62a0 con 0x55c376da5c00 2023-11-27T15:28:53.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.134+0000 7fd3f475fb80 10 client.0 _ll_register_callbacks cb 0x7fd3d4009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:53.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.134+0000 7fd3d9d9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da6550 0x55c376db2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.134+0000 7fd3da59c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da5c00 0x55c376db25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.134+0000 7fd3dad9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c376da7110 0x55c376db5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.134+0000 7fd3da59c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da5c00 0x55c376db25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:57994/0 (socket says 172.21.15.133:57994) 2023-11-27T15:28:53.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.134+0000 7fd3da59c700 1 -- 172.21.15.133:0/1461895954 learned_addr learned my addr 172.21.15.133:0/1461895954 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:53.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.135+0000 7fd3d9d9b700 1 -- 172.21.15.133:0/1461895954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da5c00 msgr2=0x55c376db25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:53.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.135+0000 7fd3d9d9b700 1 --2- 172.21.15.133:0/1461895954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da5c00 0x55c376db25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.135+0000 7fd3d9d9b700 1 -- 172.21.15.133:0/1461895954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c376da7110 msgr2=0x55c376db5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:53.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.135+0000 7fd3d9d9b700 1 --2- 172.21.15.133:0/1461895954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c376da7110 0x55c376db5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.135+0000 7fd3d9d9b700 1 -- 172.21.15.133:0/1461895954 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3dc10b240 con 0x55c376da6550 2023-11-27T15:28:53.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.135+0000 7fd3d9d9b700 1 --2- 172.21.15.133:0/1461895954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da6550 0x55c376db2af0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0010fd0 tx=0x7fd3d0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7632d8b5af998819 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:53.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.135+0000 7fd3c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:28:53.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.135+0000 7fd3c1ffb700 1 -- 172.21.15.133:0/1461895954 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d00b9040 con 0x55c376da6550 2023-11-27T15:28:53.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.135+0000 7fd3c1ffb700 1 -- 172.21.15.133:0/1461895954 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd3d0097f60 con 0x55c376da6550 2023-11-27T15:28:53.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.135+0000 7fd3f475fb80 1 -- 172.21.15.133:0/1461895954 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3cc015070 con 0x55c376da6550 2023-11-27T15:28:53.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.135+0000 7fd3c1ffb700 1 -- 172.21.15.133:0/1461895954 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3d0099d60 con 0x55c376da6550 2023-11-27T15:28:53.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.135+0000 7fd3f475fb80 10 client.16407 fetch_fsmap learned FSMap version 30 2023-11-27T15:28:53.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.135+0000 7fd3f475fb80 1 -- 172.21.15.133:0/1461895954 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c376b0dc60 con 0x55c376da6550 2023-11-27T15:28:53.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.136+0000 7fd3c1ffb700 1 -- 172.21.15.133:0/1461895954 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3d00b6080 con 0x55c376da6550 2023-11-27T15:28:53.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.136+0000 7fd3c1ffb700 1 -- 172.21.15.133:0/1461895954 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fd3d00a1bc0 con 0x55c376da6550 2023-11-27T15:28:53.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.136+0000 7fd3f475fb80 10 client.16407 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:28:53.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.136+0000 7fd3f475fb80 10 client.16407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:28:53.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.136+0000 7fd3f475fb80 20 client.16407 populate_metadata read hostname 'smithi133' 2023-11-27T15:28:53.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.136+0000 7fd3f475fb80 1 --2- 172.21.15.133:0/1461895954 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c376e0fbc0 0x55c376e11fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.136+0000 7fd3f475fb80 4 client.16407 mds_command: new command op to 24560 tid=0[{"prefix": "scrub status"}] 2023-11-27T15:28:53.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.136+0000 7fd3f475fb80 1 -- 172.21.15.133:0/1461895954 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c376cc5660 con 0x55c376e0fbc0 2023-11-27T15:28:53.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.136+0000 7fd3da59c700 1 --2- 172.21.15.133:0/1461895954 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c376e0fbc0 0x55c376e11fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.137+0000 7fd3c1ffb700 10 client.16407 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:53.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.137+0000 7fd3da59c700 1 --2- 172.21.15.133:0/1461895954 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c376e0fbc0 0x55c376e11fd0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=703aa78ef4b457b3 server_cookie=58ca4da8b0394f87 in_seq=0 out_seq=0 2023-11-27T15:28:53.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.137+0000 7fd3c1ffb700 1 -- 172.21.15.133:0/1461895954 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c376cc5660 con 0x55c376e0fbc0 2023-11-27T15:28:53.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.137+0000 7fd3c1ffb700 10 client.16407 handle_command_reply: tid=0 2023-11-27T15:28:53.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.137+0000 7fd3f475fb80 1 client.16407 shutdown 2023-11-27T15:28:53.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.137+0000 7fd3f475fb80 1 -- 172.21.15.133:0/1461895954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da6550 msgr2=0x55c376db2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:53.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.137+0000 7fd3f475fb80 1 --2- 172.21.15.133:0/1461895954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da6550 0x55c376db2af0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd3d0010fd0 tx=0x7fd3d0000bc0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.137+0000 7fd3f475fb80 1 -- 172.21.15.133:0/1461895954 shutdown_connections 2023-11-27T15:28:53.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.137+0000 7fd3f475fb80 1 --2- 172.21.15.133:0/1461895954 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55c376e0fbc0 0x55c376e11fd0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.137+0000 7fd3f475fb80 1 --2- 172.21.15.133:0/1461895954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c376da7110 0x55c376db5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.137+0000 7fd3f475fb80 1 --2- 172.21.15.133:0/1461895954 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c376da5c00 0x55c376db25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.137+0000 7fd3c1ffb700 0 client.16407 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:53.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.137+0000 7fd3f475fb80 1 --2- 172.21.15.133:0/1461895954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c376da6550 0x55c376db2af0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.137+0000 7fd3f475fb80 1 -- 172.21.15.133:0/1461895954 >> 172.21.15.133:0/1461895954 conn(0x55c376d9b360 msgr2=0x55c376dad4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:53.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.137+0000 7fd3f475fb80 1 -- 172.21.15.133:0/1461895954 shutdown_connections 2023-11-27T15:28:53.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.137+0000 7fd3f475fb80 1 -- 172.21.15.133:0/1461895954 wait complete. 2023-11-27T15:28:53.261 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.137+0000 7fd3f475fb80 20 client.16407 trim_cache size 0 max 16384 2023-11-27T15:28:53.262 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:28:53.262 INFO:teuthology.orchestra.run.smithi133.stdout: "status": "no active scrubs running", 2023-11-27T15:28:53.262 INFO:teuthology.orchestra.run.smithi133.stdout: "scrubs": {} 2023-11-27T15:28:53.262 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:28:53.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.138+0000 7fd3e0dff700 1 -- 172.21.15.133:0/320127449 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd3c40a1150 msgr2=0x7fd3c40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:53.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.138+0000 7fd3e0dff700 1 --2- 172.21.15.133:0/320127449 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd3c40a1150 0x7fd3c40a3600 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400b830 tx=0x7fd3d400b3b0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.138+0000 7fd3e0dff700 1 -- 172.21.15.133:0/320127449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3dc074ad0 msgr2=0x7fd3dc198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:53.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.138+0000 7fd3e0dff700 1 --2- 172.21.15.133:0/320127449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3dc074ad0 0x7fd3dc198840 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000d950 tx=0x7fd3d000dc60 comp rx=0 tx=0).stop 2023-11-27T15:28:53.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.138+0000 7fd3e0dff700 1 -- 172.21.15.133:0/320127449 shutdown_connections 2023-11-27T15:28:53.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.138+0000 7fd3e0dff700 1 --2- 172.21.15.133:0/320127449 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd3c40a1150 0x7fd3c40a3600 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.138+0000 7fd3e0dff700 1 --2- 172.21.15.133:0/320127449 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3dc1081a0 0x7fd3dc19f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.138+0000 7fd3e0dff700 1 --2- 172.21.15.133:0/320127449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3dc0734f0 0x7fd3dc198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.138+0000 7fd3e0dff700 1 --2- 172.21.15.133:0/320127449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3dc074ad0 0x7fd3dc198840 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.138+0000 7fd3e0dff700 1 -- 172.21.15.133:0/320127449 >> 172.21.15.133:0/320127449 conn(0x7fd3dc0fb930 msgr2=0x7fd3dc10a410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:53.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.138+0000 7fd3e0dff700 1 -- 172.21.15.133:0/320127449 shutdown_connections 2023-11-27T15:28:53.265 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.138+0000 7fd3e0dff700 1 -- 172.21.15.133:0/320127449 wait complete. 2023-11-27T15:28:53.265 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-27T15:28:53.265 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:28:53.266 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:52 smithi133 ceph-mon[97860]: pgmap v529: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2023-11-27T15:28:53.267 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-11-27T15:28:53.277 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-11-27T15:28:53.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:52 smithi177 ceph-mon[131504]: pgmap v529: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2023-11-27T15:28:53.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:52 smithi174 ceph-mon[131110]: pgmap v529: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.7 KiB/s wr, 16 op/s 2023-11-27T15:28:53.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.498+0000 7fec9c7ff700 1 -- 172.21.15.133:0/1469515070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec941002b0 msgr2=0x7fec941006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:53.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.498+0000 7fec9c7ff700 1 --2- 172.21.15.133:0/1469515070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec941002b0 0x7fec941006c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fec900098f0 tx=0x7fec90009c00 comp rx=0 tx=0).stop 2023-11-27T15:28:53.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.498+0000 7fec9c7ff700 1 -- 172.21.15.133:0/1469515070 shutdown_connections 2023-11-27T15:28:53.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.498+0000 7fec9c7ff700 1 --2- 172.21.15.133:0/1469515070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec94101e30 0x7fec941086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.498+0000 7fec9c7ff700 1 --2- 172.21.15.133:0/1469515070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec941002b0 0x7fec941006c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.498+0000 7fec9c7ff700 1 --2- 172.21.15.133:0/1469515070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec941014a0 0x7fec941018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.498+0000 7fec9c7ff700 1 -- 172.21.15.133:0/1469515070 >> 172.21.15.133:0/1469515070 conn(0x7fec940fb890 msgr2=0x7fec940fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:53.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.498+0000 7fec9c7ff700 1 -- 172.21.15.133:0/1469515070 shutdown_connections 2023-11-27T15:28:53.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.498+0000 7fec9c7ff700 1 -- 172.21.15.133:0/1469515070 wait complete. 2023-11-27T15:28:53.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.499+0000 7fec9c7ff700 1 Processor -- start 2023-11-27T15:28:53.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.500+0000 7fec9c7ff700 1 -- start start 2023-11-27T15:28:53.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.500+0000 7fec9c7ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec941002b0 0x7fec941981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.500+0000 7fec9c7ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec941014a0 0x7fec94198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.500+0000 7fec9c7ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec94101e30 0x7fec9419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.500+0000 7fec9c7ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9419fd50 con 0x7fec94101e30 2023-11-27T15:28:53.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.500+0000 7fec9c7ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9419fe90 con 0x7fec941002b0 2023-11-27T15:28:53.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.500+0000 7fec9c7ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec941a0020 con 0x7fec941014a0 2023-11-27T15:28:53.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.500+0000 7fec99d9a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec941014a0 0x7fec94198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.500+0000 7fec9ad9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec94101e30 0x7fec9419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.500+0000 7fec99d9a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec941014a0 0x7fec94198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:58012/0 (socket says 172.21.15.133:58012) 2023-11-27T15:28:53.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.500+0000 7fec99d9a700 1 -- 172.21.15.133:0/867958945 learned_addr learned my addr 172.21.15.133:0/867958945 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:53.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.500+0000 7fec9a59b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec941002b0 0x7fec941981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.501+0000 7fec99d9a700 1 -- 172.21.15.133:0/867958945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec941002b0 msgr2=0x7fec941981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:53.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.501+0000 7fec99d9a700 1 --2- 172.21.15.133:0/867958945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec941002b0 0x7fec941981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.501+0000 7fec99d9a700 1 -- 172.21.15.133:0/867958945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec94101e30 msgr2=0x7fec9419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:53.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.501+0000 7fec99d9a700 1 --2- 172.21.15.133:0/867958945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec94101e30 0x7fec9419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.501+0000 7fec99d9a700 1 -- 172.21.15.133:0/867958945 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec90009520 con 0x7fec941014a0 2023-11-27T15:28:53.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.501+0000 7fec9a59b700 1 --2- 172.21.15.133:0/867958945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec941002b0 0x7fec941981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:28:53.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.501+0000 7fec9ad9c700 1 --2- 172.21.15.133:0/867958945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec94101e30 0x7fec9419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:28:53.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.501+0000 7fec99d9a700 1 --2- 172.21.15.133:0/867958945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec941014a0 0x7fec94198710 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fec8400e9c0 tx=0x7fec8400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:53.508 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.501+0000 7fec8b7fe700 1 -- 172.21.15.133:0/867958945 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec84004a30 con 0x7fec941014a0 2023-11-27T15:28:53.508 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.501+0000 7fec8b7fe700 1 -- 172.21.15.133:0/867958945 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec8400cc80 con 0x7fec941014a0 2023-11-27T15:28:53.508 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.501+0000 7fec9c7ff700 1 -- 172.21.15.133:0/867958945 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9410bc00 con 0x7fec941014a0 2023-11-27T15:28:53.508 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.501+0000 7fec8b7fe700 1 -- 172.21.15.133:0/867958945 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec840054e0 con 0x7fec941014a0 2023-11-27T15:28:53.508 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.501+0000 7fec9c7ff700 1 -- 172.21.15.133:0/867958945 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9410ba70 con 0x7fec941014a0 2023-11-27T15:28:53.509 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.502+0000 7fec9c7ff700 1 -- 172.21.15.133:0/867958945 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec941a07c0 con 0x7fec941014a0 2023-11-27T15:28:53.509 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.504+0000 7fec8b7fe700 1 -- 172.21.15.133:0/867958945 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fec84004b90 con 0x7fec941014a0 2023-11-27T15:28:53.509 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.506+0000 7fec8b7fe700 1 --2- 172.21.15.133:0/867958945 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fec800a11d0 0x7fec800a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.509 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.506+0000 7fec8b7fe700 1 -- 172.21.15.133:0/867958945 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fec84014070 con 0x7fec941014a0 2023-11-27T15:28:53.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.506+0000 7fec9a59b700 1 --2- 172.21.15.133:0/867958945 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fec800a11d0 0x7fec800a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.510 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.507+0000 7fec9a59b700 1 --2- 172.21.15.133:0/867958945 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fec800a11d0 0x7fec800a3680 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fec90009fd0 tx=0x7fec90019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:53.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.508+0000 7fec8b7fe700 1 -- 172.21.15.133:0/867958945 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fec84096290 con 0x7fec941014a0 2023-11-27T15:28:53.760 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.757+0000 7fec9c7ff700 1 -- 172.21.15.133:0/867958945 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec941a0d70 con 0x7fec941014a0 2023-11-27T15:28:53.762 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.758+0000 7fec8b7fe700 1 -- 172.21.15.133:0/867958945 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10838 (secure 0 0 0) 0x7fec840998b0 con 0x7fec941014a0 2023-11-27T15:28:53.762 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:28:53.762 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14493,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2381290259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2381290259},{"type":"v1","addr":"172.21.15.133:6839","nonce":2381290259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1704875895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1704875895},{"type":"v1","addr":"172.21.15.133:6841","nonce":1704875895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:20:33.201023+0000","modified":"2023-11-27T15:28:26.922183+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24521,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6839/2781994447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2781994447},{"type":"v1","addr":"172.21.15.174:6839","nonce":2781994447}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_26467":{"gid":26467,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/1199749740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1199749740},{"type":"v1","addr":"172.21.15.177:6837","nonce":1199749740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:28:53.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.761+0000 7fec9c7ff700 1 -- 172.21.15.133:0/867958945 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fec800a11d0 msgr2=0x7fec800a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:53.764 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.761+0000 7fec9c7ff700 1 --2- 172.21.15.133:0/867958945 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fec800a11d0 0x7fec800a3680 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fec90009fd0 tx=0x7fec90019040 comp rx=0 tx=0).stop 2023-11-27T15:28:53.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.761+0000 7fec9c7ff700 1 -- 172.21.15.133:0/867958945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec941014a0 msgr2=0x7fec94198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:53.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.761+0000 7fec9c7ff700 1 --2- 172.21.15.133:0/867958945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec941014a0 0x7fec94198710 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fec8400e9c0 tx=0x7fec8400ecd0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.761+0000 7fec9c7ff700 1 -- 172.21.15.133:0/867958945 shutdown_connections 2023-11-27T15:28:53.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.761+0000 7fec9c7ff700 1 --2- 172.21.15.133:0/867958945 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fec800a11d0 0x7fec800a3680 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.761+0000 7fec9c7ff700 1 --2- 172.21.15.133:0/867958945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fec94101e30 0x7fec9419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.761+0000 7fec9c7ff700 1 --2- 172.21.15.133:0/867958945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec941002b0 0x7fec941981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.761+0000 7fec9c7ff700 1 --2- 172.21.15.133:0/867958945 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec941014a0 0x7fec94198710 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.765 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.761+0000 7fec9c7ff700 1 -- 172.21.15.133:0/867958945 >> 172.21.15.133:0/867958945 conn(0x7fec940fb890 msgr2=0x7fec940fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:53.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.761+0000 7fec9c7ff700 1 -- 172.21.15.133:0/867958945 shutdown_connections 2023-11-27T15:28:53.766 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.761+0000 7fec9c7ff700 1 -- 172.21.15.133:0/867958945 wait complete. 2023-11-27T15:28:53.766 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2023-11-27T15:28:53.783 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-11-27T15:28:53.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.982+0000 7efeec3ff700 1 -- 172.21.15.133:0/2243390287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efee41002c0 msgr2=0x7efee41006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:53.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.982+0000 7efeec3ff700 1 --2- 172.21.15.133:0/2243390287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efee41002c0 0x7efee41006d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efee00098f0 tx=0x7efee0009c00 comp rx=0 tx=0).stop 2023-11-27T15:28:53.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.982+0000 7efeec3ff700 1 -- 172.21.15.133:0/2243390287 shutdown_connections 2023-11-27T15:28:53.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.982+0000 7efeec3ff700 1 --2- 172.21.15.133:0/2243390287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efee4101e40 0x7efee4108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.982+0000 7efeec3ff700 1 --2- 172.21.15.133:0/2243390287 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efee41002c0 0x7efee41006d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.982+0000 7efeec3ff700 1 --2- 172.21.15.133:0/2243390287 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee41014b0 0x7efee4101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.982+0000 7efeec3ff700 1 -- 172.21.15.133:0/2243390287 >> 172.21.15.133:0/2243390287 conn(0x7efee40fb8a0 msgr2=0x7efee40fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.982+0000 7efeec3ff700 1 -- 172.21.15.133:0/2243390287 shutdown_connections 2023-11-27T15:28:53.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.982+0000 7efeec3ff700 1 -- 172.21.15.133:0/2243390287 wait complete. 2023-11-27T15:28:53.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.983+0000 7efeec3ff700 1 Processor -- start 2023-11-27T15:28:53.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.983+0000 7efeec3ff700 1 -- start start 2023-11-27T15:28:53.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.984+0000 7efeec3ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efee41002c0 0x7efee4198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.984+0000 7efeec3ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee41014b0 0x7efee41987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.984+0000 7efeec3ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efee4101e40 0x7efee419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.984+0000 7efeec3ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee419fe10 con 0x7efee41014b0 2023-11-27T15:28:53.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.984+0000 7efeec3ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee419ff50 con 0x7efee41002c0 2023-11-27T15:28:53.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.984+0000 7efeec3ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee41a00e0 con 0x7efee4101e40 2023-11-27T15:28:53.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.984+0000 7efee999a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee41014b0 0x7efee41987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.984+0000 7efeea99c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efee4101e40 0x7efee419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.984+0000 7efeea99c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efee4101e40 0x7efee419f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:58040/0 (socket says 172.21.15.133:58040) 2023-11-27T15:28:53.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.984+0000 7efeea99c700 1 -- 172.21.15.133:0/609376895 learned_addr learned my addr 172.21.15.133:0/609376895 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:53.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.984+0000 7efeea19b700 1 --2- 172.21.15.133:0/609376895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efee41002c0 0x7efee4198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.984+0000 7efee999a700 1 -- 172.21.15.133:0/609376895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efee4101e40 msgr2=0x7efee419f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:53.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.984+0000 7efee999a700 1 --2- 172.21.15.133:0/609376895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efee4101e40 0x7efee419f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.984+0000 7efee999a700 1 -- 172.21.15.133:0/609376895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efee41002c0 msgr2=0x7efee4198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:53.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.984+0000 7efee999a700 1 --2- 172.21.15.133:0/609376895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efee41002c0 0x7efee4198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.984+0000 7efee999a700 1 -- 172.21.15.133:0/609376895 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed40095f0 con 0x7efee41014b0 2023-11-27T15:28:53.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.985+0000 7efee999a700 1 --2- 172.21.15.133:0/609376895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee41014b0 0x7efee41987d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7efed400eb90 tx=0x7efed400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:53.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.985+0000 7efedb7fe700 1 -- 172.21.15.133:0/609376895 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed4009970 con 0x7efee41014b0 2023-11-27T15:28:53.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.985+0000 7efedb7fe700 1 -- 172.21.15.133:0/609376895 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efed40044e0 con 0x7efee41014b0 2023-11-27T15:28:53.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.985+0000 7efedb7fe700 1 -- 172.21.15.133:0/609376895 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed4003e50 con 0x7efee41014b0 2023-11-27T15:28:53.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.985+0000 7efeec3ff700 1 -- 172.21.15.133:0/609376895 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee0009520 con 0x7efee41014b0 2023-11-27T15:28:53.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.985+0000 7efeec3ff700 1 -- 172.21.15.133:0/609376895 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee410bc10 con 0x7efee41014b0 2023-11-27T15:28:53.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.988+0000 7efedb7fe700 1 -- 172.21.15.133:0/609376895 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7efed4004000 con 0x7efee41014b0 2023-11-27T15:28:53.992 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.989+0000 7efeffcd8b80 1 Processor -- start 2023-11-27T15:28:53.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.990+0000 7efeffcd8b80 1 -- start start 2023-11-27T15:28:53.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.990+0000 7efedb7fe700 1 --2- 172.21.15.133:0/609376895 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7efed00a1150 0x7efed00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.990+0000 7efeffcd8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c5c00 0x55e3b16c6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.990+0000 7efeffcd8b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c6550 0x55e3b16c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.990+0000 7efeffcd8b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3b16c7110 0x55e3b16c7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:53.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.990+0000 7efeffcd8b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b15cb060 con 0x55e3b16c6550 2023-11-27T15:28:53.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.990+0000 7efeffcd8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b161d8a0 con 0x55e3b16c5c00 2023-11-27T15:28:53.995 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.990+0000 7efeffcd8b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b1616d30 con 0x55e3b16c7110 2023-11-27T15:28:53.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.990+0000 7efedb7fe700 1 -- 172.21.15.133:0/609376895 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7efed4014070 con 0x7efee41014b0 2023-11-27T15:28:53.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.990+0000 7efeea99c700 1 --2- 172.21.15.133:0/609376895 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7efed00a1150 0x7efed00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.990+0000 7efee999a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c6550 0x55e3b16c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.990+0000 7efeea19b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c5c00 0x55e3b16c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.990+0000 7efeea99c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3b16c7110 0x55e3b16c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:53.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.990+0000 7efeea19b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c5c00 0x55e3b16c6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:50874/0 (socket says 172.21.15.133:50874) 2023-11-27T15:28:53.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.990+0000 7efeea19b700 1 -- 172.21.15.133:0/1235937703 learned_addr learned my addr 172.21.15.133:0/1235937703 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:53.996 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.991+0000 7efeea19b700 1 -- 172.21.15.133:0/1235937703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3b16c7110 msgr2=0x55e3b16c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:53.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.991+0000 7efeea19b700 1 --2- 172.21.15.133:0/1235937703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3b16c7110 0x55e3b16c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.991+0000 7efeea19b700 1 -- 172.21.15.133:0/1235937703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c6550 msgr2=0x55e3b16c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:53.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.991+0000 7efeea19b700 1 --2- 172.21.15.133:0/1235937703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c6550 0x55e3b16c6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:53.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.991+0000 7efeea19b700 1 -- 172.21.15.133:0/1235937703 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3b1498630 con 0x55e3b16c5c00 2023-11-27T15:28:53.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.991+0000 7efee999a700 1 --2- 172.21.15.133:0/1235937703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c6550 0x55e3b16c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:28:53.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.991+0000 7efeea99c700 1 --2- 172.21.15.133:0/609376895 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7efed00a1150 0x7efed00a3600 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efedc007c70 tx=0x7efedc0099e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:53.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.991+0000 7efeea19b700 1 --2- 172.21.15.133:0/1235937703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c5c00 0x55e3b16c6010 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efee00098c0 tx=0x7efee0005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aed156e7a800a4d server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:53.997 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.991+0000 7efed8bfe700 1 -- 172.21.15.133:0/1235937703 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee000f720 con 0x55e3b16c5c00 2023-11-27T15:28:53.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.991+0000 7efed8bfe700 1 -- 172.21.15.133:0/1235937703 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efee001d040 con 0x55e3b16c5c00 2023-11-27T15:28:53.998 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.991+0000 7efed8bfe700 1 -- 172.21.15.133:0/1235937703 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee00055b0 con 0x55e3b16c5c00 2023-11-27T15:28:54.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.992+0000 7efeffcd8b80 1 -- 172.21.15.133:0/1235937703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c5c00 msgr2=0x55e3b16c6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.992+0000 7efeffcd8b80 1 --2- 172.21.15.133:0/1235937703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c5c00 0x55e3b16c6010 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efee00098c0 tx=0x7efee0005dc0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.992+0000 7efeffcd8b80 1 -- 172.21.15.133:0/1235937703 shutdown_connections 2023-11-27T15:28:54.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.992+0000 7efeffcd8b80 1 --2- 172.21.15.133:0/1235937703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3b16c7110 0x55e3b16c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.992+0000 7efeffcd8b80 1 --2- 172.21.15.133:0/1235937703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c5c00 0x55e3b16c6010 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.992+0000 7efeffcd8b80 1 --2- 172.21.15.133:0/1235937703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c6550 0x55e3b16c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.992+0000 7efeffcd8b80 1 -- 172.21.15.133:0/1235937703 >> 172.21.15.133:0/1235937703 conn(0x55e3b16bb360 msgr2=0x55e3b16c0670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:54.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.992+0000 7efeffcd8b80 1 -- 172.21.15.133:0/1235937703 shutdown_connections 2023-11-27T15:28:54.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.992+0000 7efeffcd8b80 1 -- 172.21.15.133:0/1235937703 wait complete. 2023-11-27T15:28:54.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.993+0000 7efeffcd8b80 1 -- start start 2023-11-27T15:28:54.002 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.993+0000 7efeffcd8b80 1 Processor -- start 2023-11-27T15:28:54.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.993+0000 7efeffcd8b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3b16c5c00 0x55e3b16cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.993+0000 7efeffcd8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c6550 0x55e3b16cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.993+0000 7efeffcd8b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c7110 0x55e3b16d6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.993+0000 7efeffcd8b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b15b88e0 con 0x55e3b16c7110 2023-11-27T15:28:54.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.993+0000 7efeffcd8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b15b8580 con 0x55e3b16c6550 2023-11-27T15:28:54.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.993+0000 7efeffcd8b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b149a920 con 0x55e3b16c5c00 2023-11-27T15:28:54.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.993+0000 7efeffcd8b80 10 client.0 _ll_register_callbacks cb 0x55e3b15d6780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:54.003 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.994+0000 7efee999a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c6550 0x55e3b16cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.994+0000 7efeea99c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c7110 0x55e3b16d6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.994+0000 7efee999a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c6550 0x55e3b16cfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:50886/0 (socket says 172.21.15.133:50886) 2023-11-27T15:28:54.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.994+0000 7efee999a700 1 -- 172.21.15.133:0/3671831486 learned_addr learned my addr 172.21.15.133:0/3671831486 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:54.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.994+0000 7efeea19b700 1 --2- 172.21.15.133:0/3671831486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3b16c5c00 0x55e3b16cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.994+0000 7efee999a700 1 -- 172.21.15.133:0/3671831486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3b16c5c00 msgr2=0x55e3b16cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.994+0000 7efee999a700 1 --2- 172.21.15.133:0/3671831486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3b16c5c00 0x55e3b16cf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.994+0000 7efee999a700 1 -- 172.21.15.133:0/3671831486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c7110 msgr2=0x55e3b16d6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.994+0000 7efee999a700 1 --2- 172.21.15.133:0/3671831486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c7110 0x55e3b16d6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.004 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.994+0000 7efee999a700 1 -- 172.21.15.133:0/3671831486 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee4109030 con 0x55e3b16c6550 2023-11-27T15:28:54.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.994+0000 7efeea99c700 1 --2- 172.21.15.133:0/3671831486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c7110 0x55e3b16d6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:28:54.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.994+0000 7efee999a700 1 --2- 172.21.15.133:0/3671831486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c6550 0x55e3b16cfa80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efed4099ef0 tx=0x7efed4096640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5ac2adbfdb6b551 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:54.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.994+0000 7efecdffb700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:28:54.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.994+0000 7efecdffb700 1 -- 172.21.15.133:0/3671831486 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed400b2c0 con 0x55e3b16c6550 2023-11-27T15:28:54.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.995+0000 7efeffcd8b80 1 -- 172.21.15.133:0/3671831486 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3b16b8b20 con 0x55e3b16c6550 2023-11-27T15:28:54.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.995+0000 7efecdffb700 1 -- 172.21.15.133:0/3671831486 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7efed409a1f0 con 0x55e3b16c6550 2023-11-27T15:28:54.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.995+0000 7efecdffb700 1 -- 172.21.15.133:0/3671831486 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efed409e220 con 0x55e3b16c6550 2023-11-27T15:28:54.005 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.995+0000 7efeffcd8b80 10 client.26567 fetch_fsmap learned FSMap version 30 2023-11-27T15:28:54.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.995+0000 7efeffcd8b80 1 -- 172.21.15.133:0/3671831486 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e3b15dd190 con 0x55e3b16c6550 2023-11-27T15:28:54.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.995+0000 7efecdffb700 1 -- 172.21.15.133:0/3671831486 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e3b16b8b20 con 0x55e3b16c6550 2023-11-27T15:28:54.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.995+0000 7efecdffb700 1 -- 172.21.15.133:0/3671831486 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7efed40996e0 con 0x55e3b16c6550 2023-11-27T15:28:54.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.995+0000 7efeffcd8b80 10 client.26567 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:28:54.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.995+0000 7efeffcd8b80 10 client.26567 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-11-27T15:28:54.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.995+0000 7efeffcd8b80 20 client.26567 populate_metadata read hostname 'smithi133' 2023-11-27T15:28:54.006 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.995+0000 7efeffcd8b80 1 --2- 172.21.15.133:0/3671831486 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55e3b16db370 0x55e3b16dd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.995+0000 7efeffcd8b80 4 client.26567 mds_command: new command op to 24520 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:28:54.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.995+0000 7efeffcd8b80 1 -- 172.21.15.133:0/3671831486 --> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3b15d90a0 con 0x55e3b16db370 2023-11-27T15:28:54.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.996+0000 7efeea19b700 1 --2- 172.21.15.133:0/3671831486 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55e3b16db370 0x55e3b16dd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.996+0000 7efeea19b700 1 --2- 172.21.15.133:0/3671831486 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55e3b16db370 0x55e3b16dd780 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9477b72dad599f7c server_cookie=1165417fd7c9c27f in_seq=0 out_seq=0 2023-11-27T15:28:54.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.996+0000 7efecdffb700 10 client.26567 ms_handle_connect on v2:172.21.15.177:6838/1463687598 2023-11-27T15:28:54.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.997+0000 7efecdffb700 1 -- 172.21.15.133:0/3671831486 <== mds.2 v2:172.21.15.177:6838/1463687598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e3b15d90a0 con 0x55e3b16db370 2023-11-27T15:28:54.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.997+0000 7efecdffb700 10 client.26567 handle_command_reply: tid=0 2023-11-27T15:28:54.007 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.997+0000 7efeffcd8b80 1 client.26567 shutdown 2023-11-27T15:28:54.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.997+0000 7efeffcd8b80 1 -- 172.21.15.133:0/3671831486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c6550 msgr2=0x55e3b16cfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.997+0000 7efeffcd8b80 1 --2- 172.21.15.133:0/3671831486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c6550 0x55e3b16cfa80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efed4099ef0 tx=0x7efed4096640 comp rx=0 tx=0).stop 2023-11-27T15:28:54.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.997+0000 7efeffcd8b80 1 -- 172.21.15.133:0/3671831486 shutdown_connections 2023-11-27T15:28:54.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.997+0000 7efeffcd8b80 1 --2- 172.21.15.133:0/3671831486 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55e3b16db370 0x55e3b16dd780 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.997+0000 7efeffcd8b80 1 --2- 172.21.15.133:0/3671831486 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c7110 0x55e3b16d6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.997+0000 7efeffcd8b80 1 --2- 172.21.15.133:0/3671831486 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3b16c5c00 0x55e3b16cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.997+0000 7efeffcd8b80 1 --2- 172.21.15.133:0/3671831486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c6550 0x55e3b16cfa80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.997+0000 7efecdffb700 0 client.26567 ms_handle_reset on v2:172.21.15.177:6838/1463687598 2023-11-27T15:28:54.008 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.997+0000 7efeffcd8b80 1 -- 172.21.15.133:0/3671831486 >> 172.21.15.133:0/3671831486 conn(0x55e3b16bb360 msgr2=0x55e3b16bfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:54.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.997+0000 7efeffcd8b80 1 -- 172.21.15.133:0/3671831486 shutdown_connections 2023-11-27T15:28:54.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.998+0000 7efeffcd8b80 1 -- 172.21.15.133:0/3671831486 wait complete. 2023-11-27T15:28:54.009 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:53.998+0000 7efeffcd8b80 20 client.26567 trim_cache size 0 max 16384 2023-11-27T15:28:54.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.011+0000 7efeffcd8b80 1 Processor -- start 2023-11-27T15:28:54.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.011+0000 7efeffcd8b80 1 -- start start 2023-11-27T15:28:54.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.011+0000 7efeffcd8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c5c00 0x55e3b16e20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.011+0000 7efeffcd8b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c6550 0x55e3b16d6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.011+0000 7efeffcd8b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3b16c7110 0x55e3b16cf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.011+0000 7efeffcd8b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b16c6950 con 0x55e3b16c6550 2023-11-27T15:28:54.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.011+0000 7efeffcd8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b16cd240 con 0x55e3b16c5c00 2023-11-27T15:28:54.015 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.011+0000 7efeffcd8b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b16cab30 con 0x55e3b16c7110 2023-11-27T15:28:54.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.012+0000 7efee999a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c6550 0x55e3b16d6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.012+0000 7efee999a700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c6550 0x55e3b16d6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41884/0 (socket says 172.21.15.133:41884) 2023-11-27T15:28:54.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.012+0000 7efee999a700 1 -- 172.21.15.133:0/1579818640 learned_addr learned my addr 172.21.15.133:0/1579818640 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:54.017 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.012+0000 7efeea19b700 1 --2- 172.21.15.133:0/1579818640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c5c00 0x55e3b16e20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.012+0000 7efeea99c700 1 --2- 172.21.15.133:0/1579818640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3b16c7110 0x55e3b16cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.012+0000 7efee999a700 1 -- 172.21.15.133:0/1579818640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3b16c7110 msgr2=0x55e3b16cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.012+0000 7efee999a700 1 --2- 172.21.15.133:0/1579818640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3b16c7110 0x55e3b16cf540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.012+0000 7efee999a700 1 -- 172.21.15.133:0/1579818640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c5c00 msgr2=0x55e3b16e20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.012+0000 7efee999a700 1 --2- 172.21.15.133:0/1579818640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c5c00 0x55e3b16e20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.012+0000 7efee999a700 1 -- 172.21.15.133:0/1579818640 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3b15dcee0 con 0x55e3b16c6550 2023-11-27T15:28:54.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.012+0000 7efee999a700 1 --2- 172.21.15.133:0/1579818640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c6550 0x55e3b16d6a80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efed409a180 tx=0x7efed4009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28987d2ec423a7e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:54.018 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.013+0000 7efed93ff700 1 -- 172.21.15.133:0/1579818640 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed400b2c0 con 0x55e3b16c6550 2023-11-27T15:28:54.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.013+0000 7efed93ff700 1 -- 172.21.15.133:0/1579818640 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efed409a1f0 con 0x55e3b16c6550 2023-11-27T15:28:54.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.013+0000 7efed93ff700 1 -- 172.21.15.133:0/1579818640 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed40a2470 con 0x55e3b16c6550 2023-11-27T15:28:54.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.013+0000 7efeffcd8b80 1 -- 172.21.15.133:0/1579818640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c6550 msgr2=0x55e3b16d6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.013+0000 7efeffcd8b80 1 --2- 172.21.15.133:0/1579818640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c6550 0x55e3b16d6a80 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7efed409a180 tx=0x7efed4009820 comp rx=0 tx=0).stop 2023-11-27T15:28:54.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.013+0000 7efeffcd8b80 1 -- 172.21.15.133:0/1579818640 shutdown_connections 2023-11-27T15:28:54.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.013+0000 7efeffcd8b80 1 --2- 172.21.15.133:0/1579818640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3b16c7110 0x55e3b16cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.013+0000 7efeffcd8b80 1 --2- 172.21.15.133:0/1579818640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c5c00 0x55e3b16e20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.019 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.013+0000 7efeffcd8b80 1 --2- 172.21.15.133:0/1579818640 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c6550 0x55e3b16d6a80 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.013+0000 7efeffcd8b80 1 -- 172.21.15.133:0/1579818640 >> 172.21.15.133:0/1579818640 conn(0x55e3b16bb360 msgr2=0x55e3b16c0640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:54.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.013+0000 7efeffcd8b80 1 -- 172.21.15.133:0/1579818640 shutdown_connections 2023-11-27T15:28:54.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.013+0000 7efeffcd8b80 1 -- 172.21.15.133:0/1579818640 wait complete. 2023-11-27T15:28:54.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.014+0000 7efeffcd8b80 1 -- start start 2023-11-27T15:28:54.020 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.014+0000 7efeffcd8b80 1 Processor -- start 2023-11-27T15:28:54.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.014+0000 7efeffcd8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c5c00 0x55e3b16d2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.014+0000 7efeffcd8b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3b16c6550 0x55e3b16d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.023 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.014+0000 7efeffcd8b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c7110 0x55e3b16d5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.014+0000 7efeffcd8b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b16d5ab0 con 0x55e3b16c7110 2023-11-27T15:28:54.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.014+0000 7efeffcd8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b16d5c20 con 0x55e3b16c5c00 2023-11-27T15:28:54.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.014+0000 7efeffcd8b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3b16d5de0 con 0x55e3b16c6550 2023-11-27T15:28:54.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.015+0000 7efeffcd8b80 10 client.0 _ll_register_callbacks cb 0x7efed4099fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:54.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.015+0000 7efeea99c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c7110 0x55e3b16d5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.015+0000 7efeea19b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c5c00 0x55e3b16d2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.015+0000 7efee999a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3b16c6550 0x55e3b16d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.024 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.015+0000 7efeea19b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c5c00 0x55e3b16d2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:50906/0 (socket says 172.21.15.133:50906) 2023-11-27T15:28:54.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.015+0000 7efeea99c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c7110 0x55e3b16d5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41894/0 (socket says 172.21.15.133:41894) 2023-11-27T15:28:54.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.015+0000 7efeea19b700 1 -- 172.21.15.133:0/2962487681 learned_addr learned my addr 172.21.15.133:0/2962487681 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:54.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.015+0000 7efeea19b700 1 -- 172.21.15.133:0/2962487681 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3b16c6550 msgr2=0x55e3b16d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.015+0000 7efeea19b700 1 --2- 172.21.15.133:0/2962487681 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3b16c6550 0x55e3b16d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.015+0000 7efeea19b700 1 -- 172.21.15.133:0/2962487681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c7110 msgr2=0x55e3b16d5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.015+0000 7efeea19b700 1 --2- 172.21.15.133:0/2962487681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c7110 0x55e3b16d5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.015+0000 7efeea19b700 1 -- 172.21.15.133:0/2962487681 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3b15dd190 con 0x55e3b16c5c00 2023-11-27T15:28:54.025 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.015+0000 7efeea99c700 1 --2- 172.21.15.133:0/2962487681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c7110 0x55e3b16d5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:28:54.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.015+0000 7efeea19b700 1 --2- 172.21.15.133:0/2962487681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c5c00 0x55e3b16d2010 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efee00098c0 tx=0x7efee0009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e85b8fe60526816 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:54.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.015+0000 7efecdffb700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2023-11-27T15:28:54.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.016+0000 7efecdffb700 1 -- 172.21.15.133:0/2962487681 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee0030040 con 0x55e3b16c5c00 2023-11-27T15:28:54.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.016+0000 7efecdffb700 1 -- 172.21.15.133:0/2962487681 <== mon.2 v2:172.21.15.174:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7efee0005360 con 0x55e3b16c5c00 2023-11-27T15:28:54.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.016+0000 7efeffcd8b80 1 -- 172.21.15.133:0/2962487681 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efee001d040 con 0x55e3b16c5c00 2023-11-27T15:28:54.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.016+0000 7efecdffb700 1 -- 172.21.15.133:0/2962487681 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7efee0005920 con 0x55e3b16c5c00 2023-11-27T15:28:54.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.016+0000 7efeffcd8b80 10 client.26573 fetch_fsmap learned FSMap version 30 2023-11-27T15:28:54.026 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.016+0000 7efeffcd8b80 1 -- 172.21.15.133:0/2962487681 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e3b142dc60 con 0x55e3b16c5c00 2023-11-27T15:28:54.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.016+0000 7efecdffb700 1 -- 172.21.15.133:0/2962487681 <== mon.2 v2:172.21.15.174:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efee001d040 con 0x55e3b16c5c00 2023-11-27T15:28:54.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.016+0000 7efecdffb700 1 -- 172.21.15.133:0/2962487681 <== mon.2 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7efee0009c70 con 0x55e3b16c5c00 2023-11-27T15:28:54.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.016+0000 7efeffcd8b80 10 client.26573 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:28:54.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.016+0000 7efeffcd8b80 10 client.26573 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-11-27T15:28:54.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.016+0000 7efeffcd8b80 20 client.26573 populate_metadata read hostname 'smithi133' 2023-11-27T15:28:54.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.016+0000 7efeffcd8b80 1 --2- 172.21.15.133:0/2962487681 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55e3b172f600 0x55e3b1731a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.016+0000 7efeffcd8b80 4 client.26573 mds_command: new command op to 24520 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:28:54.027 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.016+0000 7efeffcd8b80 1 -- 172.21.15.133:0/2962487681 --> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e3b15e5660 con 0x55e3b172f600 2023-11-27T15:28:54.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.017+0000 7efee999a700 1 --2- 172.21.15.133:0/2962487681 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55e3b172f600 0x55e3b1731a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.017+0000 7efee999a700 1 --2- 172.21.15.133:0/2962487681 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55e3b172f600 0x55e3b1731a10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3bbcc805deeda44f server_cookie=96d7434941aa7604 in_seq=0 out_seq=0 2023-11-27T15:28:54.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.017+0000 7efecdffb700 10 client.26573 ms_handle_connect on v2:172.21.15.177:6838/1463687598 2023-11-27T15:28:54.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.017+0000 7efecdffb700 1 -- 172.21.15.133:0/2962487681 <== mds.2 v2:172.21.15.177:6838/1463687598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e3b15e5660 con 0x55e3b172f600 2023-11-27T15:28:54.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.017+0000 7efecdffb700 10 client.26573 handle_command_reply: tid=0 2023-11-27T15:28:54.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.017+0000 7efeffcd8b80 1 client.26573 shutdown 2023-11-27T15:28:54.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.017+0000 7efeffcd8b80 1 -- 172.21.15.133:0/2962487681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c5c00 msgr2=0x55e3b16d2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.028 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.017+0000 7efeffcd8b80 1 --2- 172.21.15.133:0/2962487681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c5c00 0x55e3b16d2010 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efee00098c0 tx=0x7efee0009fd0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.018+0000 7efeffcd8b80 1 -- 172.21.15.133:0/2962487681 shutdown_connections 2023-11-27T15:28:54.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.018+0000 7efeffcd8b80 1 --2- 172.21.15.133:0/2962487681 >> [v2:172.21.15.177:6838/1463687598,v1:172.21.15.177:6839/1463687598] conn(0x55e3b172f600 0x55e3b1731a10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.018+0000 7efeffcd8b80 1 --2- 172.21.15.133:0/2962487681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e3b16c7110 0x55e3b16d5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.018+0000 7efeffcd8b80 1 --2- 172.21.15.133:0/2962487681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b16c5c00 0x55e3b16d2010 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.018+0000 7efeffcd8b80 1 --2- 172.21.15.133:0/2962487681 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55e3b16c6550 0x55e3b16d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.018+0000 7efecdffb700 0 client.26573 ms_handle_reset on v2:172.21.15.177:6838/1463687598 2023-11-27T15:28:54.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.018+0000 7efeffcd8b80 1 -- 172.21.15.133:0/2962487681 >> 172.21.15.133:0/2962487681 conn(0x55e3b16bb360 msgr2=0x55e3b16cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:54.029 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.018+0000 7efeffcd8b80 1 -- 172.21.15.133:0/2962487681 shutdown_connections 2023-11-27T15:28:54.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.018+0000 7efeffcd8b80 1 -- 172.21.15.133:0/2962487681 wait complete. 2023-11-27T15:28:54.030 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.018+0000 7efeffcd8b80 20 client.26573 trim_cache size 0 max 16384 2023-11-27T15:28:54.030 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2023-11-27T15:28:54.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.018+0000 7efeec3ff700 1 -- 172.21.15.133:0/609376895 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7efed00a1150 msgr2=0x7efed00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.018+0000 7efeec3ff700 1 --2- 172.21.15.133:0/609376895 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7efed00a1150 0x7efed00a3600 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efedc007c70 tx=0x7efedc0099e0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.018+0000 7efeec3ff700 1 -- 172.21.15.133:0/609376895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee41014b0 msgr2=0x7efee41987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.018+0000 7efeec3ff700 1 --2- 172.21.15.133:0/609376895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee41014b0 0x7efee41987d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7efed400eb90 tx=0x7efed400eea0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.018+0000 7efeec3ff700 1 -- 172.21.15.133:0/609376895 shutdown_connections 2023-11-27T15:28:54.031 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.018+0000 7efeec3ff700 1 --2- 172.21.15.133:0/609376895 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7efed00a1150 0x7efed00a3600 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.018+0000 7efeec3ff700 1 --2- 172.21.15.133:0/609376895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efee4101e40 0x7efee419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.019+0000 7efeec3ff700 1 --2- 172.21.15.133:0/609376895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efee41002c0 0x7efee4198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.019+0000 7efeec3ff700 1 --2- 172.21.15.133:0/609376895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efee41014b0 0x7efee41987d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.019+0000 7efeec3ff700 1 -- 172.21.15.133:0/609376895 >> 172.21.15.133:0/609376895 conn(0x7efee40fb8a0 msgr2=0x7efee4106dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:54.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.019+0000 7efeec3ff700 1 -- 172.21.15.133:0/609376895 shutdown_connections 2023-11-27T15:28:54.032 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.019+0000 7efeec3ff700 1 -- 172.21.15.133:0/609376895 wait complete. 2023-11-27T15:28:54.038 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:53 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/867958945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:28:54.040 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-11-27T15:28:54.239 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.236+0000 7fd3b9dff700 1 -- 172.21.15.133:0/2444340683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3b4101e50 msgr2=0x7fd3b4108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.236+0000 7fd3b9dff700 1 --2- 172.21.15.133:0/2444340683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3b4101e50 0x7fd3b4108710 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00b0a0 tx=0x7fd3ac00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.236+0000 7fd3b9dff700 1 -- 172.21.15.133:0/2444340683 shutdown_connections 2023-11-27T15:28:54.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.236+0000 7fd3b9dff700 1 --2- 172.21.15.133:0/2444340683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3b4101e50 0x7fd3b4108710 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.236+0000 7fd3b9dff700 1 --2- 172.21.15.133:0/2444340683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3b41002d0 0x7fd3b41006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.236+0000 7fd3b9dff700 1 --2- 172.21.15.133:0/2444340683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3b41014c0 0x7fd3b4101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.236+0000 7fd3b9dff700 1 -- 172.21.15.133:0/2444340683 >> 172.21.15.133:0/2444340683 conn(0x7fd3b40fb8b0 msgr2=0x7fd3b40fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:54.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.236+0000 7fd3b9dff700 1 -- 172.21.15.133:0/2444340683 shutdown_connections 2023-11-27T15:28:54.240 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.236+0000 7fd3b9dff700 1 -- 172.21.15.133:0/2444340683 wait complete. 2023-11-27T15:28:54.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.237+0000 7fd3b9dff700 1 Processor -- start 2023-11-27T15:28:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.237+0000 7fd3b9dff700 1 -- start start 2023-11-27T15:28:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.238+0000 7fd3b9dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3b41002d0 0x7fd3b4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.238+0000 7fd3b9dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3b41014c0 0x7fd3b4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.238+0000 7fd3b9dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3b4101e50 0x7fd3b419f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.238+0000 7fd3b9dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b419ff90 con 0x7fd3b41014c0 2023-11-27T15:28:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.238+0000 7fd3b9dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b41a00d0 con 0x7fd3b41002d0 2023-11-27T15:28:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.238+0000 7fd3b9dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b41a0260 con 0x7fd3b4101e50 2023-11-27T15:28:54.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.238+0000 7fd3b2ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3b41014c0 0x7fd3b4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.238+0000 7fd3b2ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3b41014c0 0x7fd3b4198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41904/0 (socket says 172.21.15.133:41904) 2023-11-27T15:28:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.238+0000 7fd3b2ffd700 1 -- 172.21.15.133:0/248346301 learned_addr learned my addr 172.21.15.133:0/248346301 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.238+0000 7fd3b37fe700 1 --2- 172.21.15.133:0/248346301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3b41002d0 0x7fd3b4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.238+0000 7fd3b3fff700 1 --2- 172.21.15.133:0/248346301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3b4101e50 0x7fd3b419f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.238+0000 7fd3b2ffd700 1 -- 172.21.15.133:0/248346301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3b4101e50 msgr2=0x7fd3b419f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.238+0000 7fd3b2ffd700 1 --2- 172.21.15.133:0/248346301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3b4101e50 0x7fd3b419f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.238+0000 7fd3b2ffd700 1 -- 172.21.15.133:0/248346301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3b41002d0 msgr2=0x7fd3b4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.244 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.238+0000 7fd3b2ffd700 1 --2- 172.21.15.133:0/248346301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3b41002d0 0x7fd3b4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.238+0000 7fd3b2ffd700 1 -- 172.21.15.133:0/248346301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ac009d00 con 0x7fd3b41014c0 2023-11-27T15:28:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.238+0000 7fd3b37fe700 1 --2- 172.21.15.133:0/248346301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3b41002d0 0x7fd3b4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:28:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.238+0000 7fd3b3fff700 1 --2- 172.21.15.133:0/248346301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3b4101e50 0x7fd3b419f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:28:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.239+0000 7fd3b2ffd700 1 --2- 172.21.15.133:0/248346301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3b41014c0 0x7fd3b4198950 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800e9a0 tx=0x7fd3a800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.239+0000 7fd3b0ff9700 1 -- 172.21.15.133:0/248346301 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8009790 con 0x7fd3b41014c0 2023-11-27T15:28:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.239+0000 7fd3b0ff9700 1 -- 172.21.15.133:0/248346301 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3a8010430 con 0x7fd3b41014c0 2023-11-27T15:28:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.239+0000 7fd3b9dff700 1 -- 172.21.15.133:0/248346301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b410bca0 con 0x7fd3b41014c0 2023-11-27T15:28:54.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.239+0000 7fd3b0ff9700 1 -- 172.21.15.133:0/248346301 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8018680 con 0x7fd3b41014c0 2023-11-27T15:28:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.239+0000 7fd3b9dff700 1 -- 172.21.15.133:0/248346301 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b410bb10 con 0x7fd3b41014c0 2023-11-27T15:28:54.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.243+0000 7fd3b0ff9700 1 -- 172.21.15.133:0/248346301 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fd3a80105a0 con 0x7fd3b41014c0 2023-11-27T15:28:54.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.244+0000 7fd3cd7c5b80 1 Processor -- start 2023-11-27T15:28:54.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.244+0000 7fd3b0ff9700 1 --2- 172.21.15.133:0/248346301 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd39c0a11d0 0x7fd39c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.244+0000 7fd3cd7c5b80 1 -- start start 2023-11-27T15:28:54.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.244+0000 7fd3b0ff9700 1 -- 172.21.15.133:0/248346301 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd3a8014070 con 0x7fd3b41014c0 2023-11-27T15:28:54.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.244+0000 7fd3cd7c5b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcac00 0x559caddcb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.244+0000 7fd3cd7c5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559caddcb550 0x559caddcbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.244+0000 7fd3cd7c5b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcc110 0x559caddcc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.244+0000 7fd3cd7c5b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cadcd0060 con 0x559caddcc110 2023-11-27T15:28:54.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.244+0000 7fd3cd7c5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cadd228a0 con 0x559caddcb550 2023-11-27T15:28:54.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.244+0000 7fd3cd7c5b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cadd1bd30 con 0x559caddcac00 2023-11-27T15:28:54.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.244+0000 7fd3b3fff700 1 --2- 172.21.15.133:0/248346301 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd39c0a11d0 0x7fd39c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.245+0000 7fd3b37fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcac00 0x559caddcb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.245+0000 7fd3b2ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559caddcb550 0x559caddcbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.245+0000 7fd3b37fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcac00 0x559caddcb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:58118/0 (socket says 172.21.15.133:58118) 2023-11-27T15:28:54.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.245+0000 7fd3b37fe700 1 -- 172.21.15.133:0/946873815 learned_addr learned my addr 172.21.15.133:0/946873815 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:54.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.245+0000 7fd3b37fe700 1 -- 172.21.15.133:0/946873815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559caddcb550 msgr2=0x559caddcbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.253 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.245+0000 7fd3b37fe700 1 --2- 172.21.15.133:0/946873815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559caddcb550 0x559caddcbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.245+0000 7fd3b37fe700 1 -- 172.21.15.133:0/946873815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcc110 msgr2=0x559caddcc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.245+0000 7fd3b3fff700 1 --2- 172.21.15.133:0/946873815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcc110 0x559caddcc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.245+0000 7fd3b37fe700 1 --2- 172.21.15.133:0/946873815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcc110 0x559caddcc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.245+0000 7fd3b37fe700 1 -- 172.21.15.133:0/946873815 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cadb9d630 con 0x559caddcac00 2023-11-27T15:28:54.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.245+0000 7fd3b3fff700 1 --2- 172.21.15.133:0/946873815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcc110 0x559caddcc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-27T15:28:54.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.245+0000 7fd3b3fff700 1 --2- 172.21.15.133:0/248346301 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd39c0a11d0 0x7fd39c0a3680 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00b780 tx=0x7fd3ac009be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:54.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.245+0000 7fd3b37fe700 1 --2- 172.21.15.133:0/946873815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcac00 0x559caddcb010 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40115c0 tx=0x7fd3a40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67501316d6ebba38 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:54.254 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.246+0000 7fd399ffb700 1 -- 172.21.15.133:0/946873815 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4004220 con 0x559caddcac00 2023-11-27T15:28:54.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.246+0000 7fd399ffb700 1 -- 172.21.15.133:0/946873815 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3a4004cf0 con 0x559caddcac00 2023-11-27T15:28:54.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.246+0000 7fd399ffb700 1 -- 172.21.15.133:0/946873815 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4005650 con 0x559caddcac00 2023-11-27T15:28:54.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.246+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/946873815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcac00 msgr2=0x559caddcb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.246+0000 7fd3cd7c5b80 1 --2- 172.21.15.133:0/946873815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcac00 0x559caddcb010 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40115c0 tx=0x7fd3a40118d0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.246+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/946873815 shutdown_connections 2023-11-27T15:28:54.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.246+0000 7fd3cd7c5b80 1 --2- 172.21.15.133:0/946873815 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcc110 0x559caddcc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.246+0000 7fd3cd7c5b80 1 --2- 172.21.15.133:0/946873815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcac00 0x559caddcb010 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.255 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.246+0000 7fd3cd7c5b80 1 --2- 172.21.15.133:0/946873815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559caddcb550 0x559caddcbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.246+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/946873815 >> 172.21.15.133:0/946873815 conn(0x559caddc0360 msgr2=0x559caddc5670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:54.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.246+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/946873815 shutdown_connections 2023-11-27T15:28:54.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.246+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/946873815 wait complete. 2023-11-27T15:28:54.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.247+0000 7fd3cd7c5b80 1 -- start start 2023-11-27T15:28:54.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.247+0000 7fd3cd7c5b80 1 Processor -- start 2023-11-27T15:28:54.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.248+0000 7fd3cd7c5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559caddcac00 0x559caddd4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.248+0000 7fd3cd7c5b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcb550 0x559caddd4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.256 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.248+0000 7fd3cd7c5b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcc110 0x559cadddbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.248+0000 7fd3cd7c5b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cadcbd8e0 con 0x559caddcc110 2023-11-27T15:28:54.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.248+0000 7fd3cd7c5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cadcbd580 con 0x559caddcac00 2023-11-27T15:28:54.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.248+0000 7fd3cd7c5b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cadb9f920 con 0x559caddcb550 2023-11-27T15:28:54.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.248+0000 7fd3cd7c5b80 10 client.0 _ll_register_callbacks cb 0x559cadcdb780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:54.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.248+0000 7fd3b3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcc110 0x559cadddbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.248+0000 7fd3b2ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcb550 0x559caddd4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.248+0000 7fd3b3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcc110 0x559cadddbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41924/0 (socket says 172.21.15.133:41924) 2023-11-27T15:28:54.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.248+0000 7fd3b2ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcb550 0x559caddd4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:58122/0 (socket says 172.21.15.133:58122) 2023-11-27T15:28:54.257 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.248+0000 7fd3b3fff700 1 -- 172.21.15.133:0/3853228453 learned_addr learned my addr 172.21.15.133:0/3853228453 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:54.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.248+0000 7fd3b37fe700 1 --2- 172.21.15.133:0/3853228453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559caddcac00 0x559caddd4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.248+0000 7fd3b2ffd700 1 -- 172.21.15.133:0/3853228453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559caddcac00 msgr2=0x559caddd4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.248+0000 7fd3b2ffd700 1 --2- 172.21.15.133:0/3853228453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559caddcac00 0x559caddd4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.248+0000 7fd3b2ffd700 1 -- 172.21.15.133:0/3853228453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcc110 msgr2=0x559cadddbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.248+0000 7fd3b2ffd700 1 --2- 172.21.15.133:0/3853228453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcc110 0x559cadddbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.248+0000 7fd3b2ffd700 1 -- 172.21.15.133:0/3853228453 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a4009d50 con 0x559caddcb550 2023-11-27T15:28:54.258 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.249+0000 7fd3b2ffd700 1 --2- 172.21.15.133:0/3853228453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcb550 0x559caddd4aa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800c2d0 tx=0x7fd3a809a380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bc70eef842f8fef server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:54.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.249+0000 7fd392ffd700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:28:54.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.249+0000 7fd392ffd700 1 -- 172.21.15.133:0/3853228453 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a809de70 con 0x559caddcb550 2023-11-27T15:28:54.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.249+0000 7fd392ffd700 1 -- 172.21.15.133:0/3853228453 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd3a80a2290 con 0x559caddcb550 2023-11-27T15:28:54.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.249+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/3853228453 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559caddbdb20 con 0x559caddcb550 2023-11-27T15:28:54.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.249+0000 7fd392ffd700 1 -- 172.21.15.133:0/3853228453 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3a80b8250 con 0x559caddcb550 2023-11-27T15:28:54.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.249+0000 7fd3cd7c5b80 10 client.26557 fetch_fsmap learned FSMap version 30 2023-11-27T15:28:54.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.249+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/3853228453 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559cadce2190 con 0x559caddcb550 2023-11-27T15:28:54.259 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.249+0000 7fd392ffd700 1 -- 172.21.15.133:0/3853228453 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559caddbdb20 con 0x559caddcb550 2023-11-27T15:28:54.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.249+0000 7fd392ffd700 1 -- 172.21.15.133:0/3853228453 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fd3a8095c30 con 0x559caddcb550 2023-11-27T15:28:54.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.249+0000 7fd3cd7c5b80 10 client.26557 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:28:54.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.249+0000 7fd3cd7c5b80 10 client.26557 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:28:54.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.250+0000 7fd3cd7c5b80 20 client.26557 populate_metadata read hostname 'smithi133' 2023-11-27T15:28:54.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.250+0000 7fd3cd7c5b80 1 --2- 172.21.15.133:0/3853228453 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x559cadde02d0 0x559cadde26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.250+0000 7fd3cd7c5b80 4 client.26557 mds_command: new command op to 24521 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:28:54.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.250+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/3853228453 --> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559cadcde0a0 con 0x559cadde02d0 2023-11-27T15:28:54.260 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.250+0000 7fd3b37fe700 1 --2- 172.21.15.133:0/3853228453 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x559cadde02d0 0x559cadde26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.250+0000 7fd392ffd700 10 client.26557 ms_handle_connect on v2:172.21.15.174:6836/1955090548 2023-11-27T15:28:54.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.250+0000 7fd3b37fe700 1 --2- 172.21.15.133:0/3853228453 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x559cadde02d0 0x559cadde26e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=539e7df346455f42 server_cookie=62acc883f08eedf8 in_seq=0 out_seq=0 2023-11-27T15:28:54.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.251+0000 7fd392ffd700 1 -- 172.21.15.133:0/3853228453 <== mds.1 v2:172.21.15.174:6836/1955090548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559cadcde0a0 con 0x559cadde02d0 2023-11-27T15:28:54.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.251+0000 7fd392ffd700 10 client.26557 handle_command_reply: tid=0 2023-11-27T15:28:54.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.251+0000 7fd3cd7c5b80 1 client.26557 shutdown 2023-11-27T15:28:54.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.251+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/3853228453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcb550 msgr2=0x559caddd4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.262 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.251+0000 7fd3cd7c5b80 1 --2- 172.21.15.133:0/3853228453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcb550 0x559caddd4aa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800c2d0 tx=0x7fd3a809a380 comp rx=0 tx=0).stop 2023-11-27T15:28:54.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.251+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/3853228453 shutdown_connections 2023-11-27T15:28:54.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.251+0000 7fd3cd7c5b80 1 --2- 172.21.15.133:0/3853228453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcc110 0x559cadddbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.251+0000 7fd3cd7c5b80 1 --2- 172.21.15.133:0/3853228453 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x559cadde02d0 0x559cadde26e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.251+0000 7fd3cd7c5b80 1 --2- 172.21.15.133:0/3853228453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559caddcac00 0x559caddd4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.251+0000 7fd3cd7c5b80 1 --2- 172.21.15.133:0/3853228453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcb550 0x559caddd4aa0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.251+0000 7fd392ffd700 0 client.26557 ms_handle_reset on v2:172.21.15.174:6836/1955090548 2023-11-27T15:28:54.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.251+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/3853228453 >> 172.21.15.133:0/3853228453 conn(0x559caddc0360 msgr2=0x559caddc4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:54.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.251+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/3853228453 shutdown_connections 2023-11-27T15:28:54.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.251+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/3853228453 wait complete. 2023-11-27T15:28:54.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.252+0000 7fd3cd7c5b80 20 client.26557 trim_cache size 0 max 16384 2023-11-27T15:28:54.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.265+0000 7fd3cd7c5b80 1 Processor -- start 2023-11-27T15:28:54.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.265+0000 7fd3cd7c5b80 1 -- start start 2023-11-27T15:28:54.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.265+0000 7fd3cd7c5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559caddcac00 0x559cadde7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.265+0000 7fd3cd7c5b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcb550 0x559cadddbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.265+0000 7fd3cd7c5b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcc110 0x559caddcef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.265+0000 7fd3cd7c5b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559caddcca00 con 0x559caddcb550 2023-11-27T15:28:54.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.265+0000 7fd3cd7c5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x559caddd3a10 con 0x559caddcac00 2023-11-27T15:28:54.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.265+0000 7fd3cd7c5b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cadddd6c0 con 0x559caddcc110 2023-11-27T15:28:54.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.266+0000 7fd3b2ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcb550 0x559cadddbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.266+0000 7fd3b2ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcb550 0x559cadddbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41938/0 (socket says 172.21.15.133:41938) 2023-11-27T15:28:54.272 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.266+0000 7fd3b2ffd700 1 -- 172.21.15.133:0/2777567956 learned_addr learned my addr 172.21.15.133:0/2777567956 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:54.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.266+0000 7fd3b37fe700 1 --2- 172.21.15.133:0/2777567956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559caddcac00 0x559cadde7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.266+0000 7fd3b3fff700 1 --2- 172.21.15.133:0/2777567956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcc110 0x559caddcef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.266+0000 7fd3b2ffd700 1 -- 172.21.15.133:0/2777567956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcc110 msgr2=0x559caddcef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.266+0000 7fd3b2ffd700 1 --2- 172.21.15.133:0/2777567956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcc110 0x559caddcef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.266+0000 7fd3b2ffd700 1 -- 172.21.15.133:0/2777567956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559caddcac00 msgr2=0x559cadde7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.266+0000 7fd3b2ffd700 1 --2- 172.21.15.133:0/2777567956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559caddcac00 0x559cadde7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.266+0000 7fd3b2ffd700 1 -- 172.21.15.133:0/2777567956 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cadce1ee0 con 0x559caddcb550 2023-11-27T15:28:54.273 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.266+0000 7fd3b2ffd700 1 --2- 172.21.15.133:0/2777567956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcb550 0x559cadddbaa0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80a1d60 tx=0x7fd3a80962b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7755c4b762a0949 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.266+0000 7fd39a7fc700 1 -- 172.21.15.133:0/2777567956 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8018ce0 con 0x559caddcb550 2023-11-27T15:28:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.266+0000 7fd39a7fc700 1 -- 172.21.15.133:0/2777567956 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3a80b8250 con 0x559caddcb550 2023-11-27T15:28:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.267+0000 7fd39a7fc700 1 -- 172.21.15.133:0/2777567956 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a80ca400 con 0x559caddcb550 2023-11-27T15:28:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.267+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/2777567956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcb550 msgr2=0x559cadddbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.267+0000 7fd3cd7c5b80 1 --2- 172.21.15.133:0/2777567956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcb550 0x559cadddbaa0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80a1d60 tx=0x7fd3a80962b0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.267+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/2777567956 shutdown_connections 2023-11-27T15:28:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.267+0000 7fd3cd7c5b80 1 --2- 172.21.15.133:0/2777567956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcc110 0x559caddcef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.267+0000 7fd3cd7c5b80 1 --2- 172.21.15.133:0/2777567956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559caddcac00 0x559cadde7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.267+0000 7fd3cd7c5b80 1 --2- 172.21.15.133:0/2777567956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcb550 0x559cadddbaa0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.267+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/2777567956 >> 172.21.15.133:0/2777567956 conn(0x559caddc0360 msgr2=0x559caddc5640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:54.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.267+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/2777567956 shutdown_connections 2023-11-27T15:28:54.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.267+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/2777567956 wait complete. 2023-11-27T15:28:54.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.267+0000 7fd3cd7c5b80 1 -- start start 2023-11-27T15:28:54.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.268+0000 7fd3cd7c5b80 1 Processor -- start 2023-11-27T15:28:54.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.268+0000 7fd3cd7c5b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcac00 0x559caddd70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.275 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.268+0000 7fd3cd7c5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559caddcb550 0x559caddd7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.268+0000 7fd3cd7c5b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcc110 0x559caddda640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.268+0000 7fd3cd7c5b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cadddab80 con 0x559caddcc110 2023-11-27T15:28:54.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.268+0000 7fd3cd7c5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cadddacf0 con 0x559caddcb550 2023-11-27T15:28:54.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.268+0000 7fd3cd7c5b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cadddaeb0 con 0x559caddcac00 2023-11-27T15:28:54.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.268+0000 7fd3cd7c5b80 10 client.0 _ll_register_callbacks cb 0x7fd3a8099980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:54.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.268+0000 7fd3b3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcc110 0x559caddda640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.268+0000 7fd3b2ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559caddcb550 0x559caddd7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.276 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.268+0000 7fd3b3fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcc110 0x559caddda640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41940/0 (socket says 172.21.15.133:41940) 2023-11-27T15:28:54.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.268+0000 7fd3b37fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcac00 0x559caddd70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.268+0000 7fd3b3fff700 1 -- 172.21.15.133:0/2196323059 learned_addr learned my addr 172.21.15.133:0/2196323059 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:54.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.269+0000 7fd3b37fe700 1 -- 172.21.15.133:0/2196323059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559caddcb550 msgr2=0x559caddd7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.269+0000 7fd3b37fe700 1 --2- 172.21.15.133:0/2196323059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559caddcb550 0x559caddd7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.269+0000 7fd3b37fe700 1 -- 172.21.15.133:0/2196323059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcc110 msgr2=0x559caddda640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.269+0000 7fd3b37fe700 1 --2- 172.21.15.133:0/2196323059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcc110 0x559caddda640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.269+0000 7fd3b37fe700 1 -- 172.21.15.133:0/2196323059 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559cadce2190 con 0x559caddcac00 2023-11-27T15:28:54.277 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.269+0000 7fd3b37fe700 1 --2- 172.21.15.133:0/2196323059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcac00 0x559caddd70e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400a010 tx=0x7fd3a4005530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd70e41d8760df21 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:54.278 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.269+0000 7fd392ffd700 10 client.0 ms_handle_connect on v2:172.21.15.177:3300/0 2023-11-27T15:28:54.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.269+0000 7fd392ffd700 1 -- 172.21.15.133:0/2196323059 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4029d80 con 0x559caddcac00 2023-11-27T15:28:54.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.269+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/2196323059 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3a4004cf0 con 0x559caddcac00 2023-11-27T15:28:54.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.269+0000 7fd392ffd700 1 -- 172.21.15.133:0/2196323059 <== mon.1 v2:172.21.15.177:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fd3a40290b0 con 0x559caddcac00 2023-11-27T15:28:54.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.270+0000 7fd392ffd700 1 -- 172.21.15.133:0/2196323059 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3a4029290 con 0x559caddcac00 2023-11-27T15:28:54.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.270+0000 7fd3cd7c5b80 10 client.26563 fetch_fsmap learned FSMap version 30 2023-11-27T15:28:54.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.270+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/2196323059 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559cadb32c60 con 0x559caddcac00 2023-11-27T15:28:54.280 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.270+0000 7fd392ffd700 1 -- 172.21.15.133:0/2196323059 <== mon.1 v2:172.21.15.177:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3a4004cf0 con 0x559caddcac00 2023-11-27T15:28:54.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.270+0000 7fd392ffd700 1 -- 172.21.15.133:0/2196323059 <== mon.1 v2:172.21.15.177:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fd3a4003cd0 con 0x559caddcac00 2023-11-27T15:28:54.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.270+0000 7fd3cd7c5b80 10 client.26563 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:28:54.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.270+0000 7fd3cd7c5b80 10 client.26563 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-11-27T15:28:54.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.270+0000 7fd3cd7c5b80 20 client.26563 populate_metadata read hostname 'smithi133' 2023-11-27T15:28:54.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.270+0000 7fd3cd7c5b80 1 --2- 172.21.15.133:0/2196323059 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x559cade34a40 0x559cade36e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.270+0000 7fd3cd7c5b80 4 client.26563 mds_command: new command op to 24521 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:28:54.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.270+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/2196323059 --> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559cadcea660 con 0x559cade34a40 2023-11-27T15:28:54.281 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.271+0000 7fd3b2ffd700 1 --2- 172.21.15.133:0/2196323059 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x559cade34a40 0x559cade36e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.282 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.271+0000 7fd3b2ffd700 1 --2- 172.21.15.133:0/2196323059 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x559cade34a40 0x559cade36e50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=407c616ec9d0fe2f server_cookie=5d89303e3d59541c in_seq=0 out_seq=0 2023-11-27T15:28:54.282 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.271+0000 7fd392ffd700 10 client.26563 ms_handle_connect on v2:172.21.15.174:6836/1955090548 2023-11-27T15:28:54.282 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.271+0000 7fd392ffd700 1 -- 172.21.15.133:0/2196323059 <== mds.1 v2:172.21.15.174:6836/1955090548 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559cadcea660 con 0x559cade34a40 2023-11-27T15:28:54.282 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.271+0000 7fd392ffd700 10 client.26563 handle_command_reply: tid=0 2023-11-27T15:28:54.282 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.271+0000 7fd3cd7c5b80 1 client.26563 shutdown 2023-11-27T15:28:54.282 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.271+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/2196323059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcac00 msgr2=0x559caddd70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.282 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.271+0000 7fd3cd7c5b80 1 --2- 172.21.15.133:0/2196323059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcac00 0x559caddd70e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400a010 tx=0x7fd3a4005530 comp rx=0 tx=0).stop 2023-11-27T15:28:54.282 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.271+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/2196323059 shutdown_connections 2023-11-27T15:28:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.271+0000 7fd3cd7c5b80 1 --2- 172.21.15.133:0/2196323059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559caddcc110 0x559caddda640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.271+0000 7fd3cd7c5b80 1 --2- 172.21.15.133:0/2196323059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559caddcac00 0x559caddd70e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.271+0000 7fd3cd7c5b80 1 --2- 172.21.15.133:0/2196323059 >> [v2:172.21.15.174:6836/1955090548,v1:172.21.15.174:6837/1955090548] conn(0x559cade34a40 0x559cade36e50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.271+0000 7fd3cd7c5b80 1 --2- 172.21.15.133:0/2196323059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559caddcb550 0x559caddd7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.271+0000 7fd392ffd700 0 client.26563 ms_handle_reset on v2:172.21.15.174:6836/1955090548 2023-11-27T15:28:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.271+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/2196323059 >> 172.21.15.133:0/2196323059 conn(0x559caddc0360 msgr2=0x559caddcef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.271+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/2196323059 shutdown_connections 2023-11-27T15:28:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.271+0000 7fd3cd7c5b80 1 -- 172.21.15.133:0/2196323059 wait complete. 2023-11-27T15:28:54.283 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.272+0000 7fd3cd7c5b80 20 client.26563 trim_cache size 0 max 16384 2023-11-27T15:28:54.284 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2023-11-27T15:28:54.284 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.272+0000 7fd3b9dff700 1 -- 172.21.15.133:0/248346301 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd39c0a11d0 msgr2=0x7fd39c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.272+0000 7fd3b9dff700 1 --2- 172.21.15.133:0/248346301 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd39c0a11d0 0x7fd39c0a3680 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00b780 tx=0x7fd3ac009be0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.272+0000 7fd3b9dff700 1 -- 172.21.15.133:0/248346301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3b41014c0 msgr2=0x7fd3b4198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.272+0000 7fd3b9dff700 1 --2- 172.21.15.133:0/248346301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3b41014c0 0x7fd3b4198950 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800e9a0 tx=0x7fd3a800ecb0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.272+0000 7fd3b9dff700 1 -- 172.21.15.133:0/248346301 shutdown_connections 2023-11-27T15:28:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.272+0000 7fd3b9dff700 1 --2- 172.21.15.133:0/248346301 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fd39c0a11d0 0x7fd39c0a3680 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.272+0000 7fd3b9dff700 1 --2- 172.21.15.133:0/248346301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd3b4101e50 0x7fd3b419f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.285 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.272+0000 7fd3b9dff700 1 --2- 172.21.15.133:0/248346301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3b41002d0 0x7fd3b4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.272+0000 7fd3b9dff700 1 --2- 172.21.15.133:0/248346301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd3b41014c0 0x7fd3b4198950 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.272+0000 7fd3b9dff700 1 -- 172.21.15.133:0/248346301 >> 172.21.15.133:0/248346301 conn(0x7fd3b40fb8b0 msgr2=0x7fd3b4106d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:54.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.272+0000 7fd3b9dff700 1 -- 172.21.15.133:0/248346301 shutdown_connections 2023-11-27T15:28:54.286 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.272+0000 7fd3b9dff700 1 -- 172.21.15.133:0/248346301 wait complete. 2023-11-27T15:28:54.294 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-11-27T15:28:54.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:53 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/867958945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:28:54.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:53 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/867958945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:28:54.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.491+0000 7f71c13ff700 1 -- 172.21.15.133:0/3772100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71bc1002c0 msgr2=0x7f71bc1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.491+0000 7f71c13ff700 1 --2- 172.21.15.133:0/3772100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71bc1002c0 0x7f71bc1006d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f71ac0098f0 tx=0x7f71ac009c00 comp rx=0 tx=0).stop 2023-11-27T15:28:54.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.491+0000 7f71c13ff700 1 -- 172.21.15.133:0/3772100 shutdown_connections 2023-11-27T15:28:54.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.491+0000 7f71c13ff700 1 --2- 172.21.15.133:0/3772100 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71bc101e40 0x7f71bc108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.491+0000 7f71c13ff700 1 --2- 172.21.15.133:0/3772100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71bc1002c0 0x7f71bc1006d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.491+0000 7f71c13ff700 1 --2- 172.21.15.133:0/3772100 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71bc1014b0 0x7f71bc101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.495 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.491+0000 7f71c13ff700 1 -- 172.21.15.133:0/3772100 >> 172.21.15.133:0/3772100 conn(0x7f71bc0fb8a0 msgr2=0x7f71bc0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:54.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.491+0000 7f71c13ff700 1 -- 172.21.15.133:0/3772100 shutdown_connections 2023-11-27T15:28:54.496 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.491+0000 7f71c13ff700 1 -- 172.21.15.133:0/3772100 wait complete. 2023-11-27T15:28:54.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.492+0000 7f71c13ff700 1 Processor -- start 2023-11-27T15:28:54.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.492+0000 7f71c13ff700 1 -- start start 2023-11-27T15:28:54.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.493+0000 7f71c13ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71bc1002c0 0x7f71bc1960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.493+0000 7f71c13ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71bc1014b0 0x7f71bc196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.493+0000 7f71c13ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71bc101e40 0x7f71bc19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.498 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.493+0000 7f71c13ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc19dc70 con 0x7f71bc1002c0 2023-11-27T15:28:54.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.493+0000 7f71c13ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc19ddb0 con 0x7f71bc1014b0 2023-11-27T15:28:54.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.493+0000 7f71c13ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc19df40 con 0x7f71bc101e40 2023-11-27T15:28:54.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.493+0000 7f71baffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71bc1002c0 0x7f71bc1960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.493+0000 7f71ba7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71bc1014b0 0x7f71bc196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.493+0000 7f71baffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71bc1002c0 0x7f71bc1960f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41954/0 (socket says 172.21.15.133:41954) 2023-11-27T15:28:54.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.493+0000 7f71baffd700 1 -- 172.21.15.133:0/2748346708 learned_addr learned my addr 172.21.15.133:0/2748346708 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:54.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.493+0000 7f71bb7fe700 1 --2- 172.21.15.133:0/2748346708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71bc101e40 0x7f71bc19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.499 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.493+0000 7f71baffd700 1 -- 172.21.15.133:0/2748346708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71bc101e40 msgr2=0x7f71bc19d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.493+0000 7f71baffd700 1 --2- 172.21.15.133:0/2748346708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71bc101e40 0x7f71bc19d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.493+0000 7f71baffd700 1 -- 172.21.15.133:0/2748346708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71bc1014b0 msgr2=0x7f71bc196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.493+0000 7f71baffd700 1 --2- 172.21.15.133:0/2748346708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71bc1014b0 0x7f71bc196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.493+0000 7f71baffd700 1 -- 172.21.15.133:0/2748346708 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac009520 con 0x7f71bc1002c0 2023-11-27T15:28:54.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.493+0000 7f71ba7fc700 1 --2- 172.21.15.133:0/2748346708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71bc1014b0 0x7f71bc196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:28:54.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.494+0000 7f71baffd700 1 --2- 172.21.15.133:0/2748346708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71bc1002c0 0x7f71bc1960f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f71ac005e90 tx=0x7f71ac00b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:54.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.494+0000 7f71a3fff700 1 -- 172.21.15.133:0/2748346708 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac01c070 con 0x7f71bc1002c0 2023-11-27T15:28:54.500 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.494+0000 7f71a3fff700 1 -- 172.21.15.133:0/2748346708 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71ac00f7f0 con 0x7f71bc1002c0 2023-11-27T15:28:54.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.494+0000 7f71c13ff700 1 -- 172.21.15.133:0/2748346708 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71bc10bc90 con 0x7f71bc1002c0 2023-11-27T15:28:54.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.494+0000 7f71a3fff700 1 -- 172.21.15.133:0/2748346708 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac003ac0 con 0x7f71bc1002c0 2023-11-27T15:28:54.501 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.494+0000 7f71c13ff700 1 -- 172.21.15.133:0/2748346708 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71bc10bb00 con 0x7f71bc1002c0 2023-11-27T15:28:54.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.497+0000 7f71a3fff700 1 -- 172.21.15.133:0/2748346708 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f71ac020030 con 0x7f71bc1002c0 2023-11-27T15:28:54.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.498+0000 7f71d4e12b80 1 Processor -- start 2023-11-27T15:28:54.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.498+0000 7f71a3fff700 1 --2- 172.21.15.133:0/2748346708 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f71a40a1150 0x7f71a40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.498+0000 7f71d4e12b80 1 -- start start 2023-11-27T15:28:54.502 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.498+0000 7f71a3fff700 1 -- 172.21.15.133:0/2748346708 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f71ac0cdcb0 con 0x7f71bc1002c0 2023-11-27T15:28:54.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.498+0000 7f71bb7fe700 1 --2- 172.21.15.133:0/2748346708 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f71a40a1150 0x7f71a40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.499+0000 7f71d4e12b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451000 0x55a236451410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.503 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.499+0000 7f71d4e12b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236451950 0x55a236451fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.504 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.499+0000 7f71d4e12b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236452530 0x55a236452980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.499+0000 7f71d4e12b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a236352060 con 0x55a236452530 2023-11-27T15:28:54.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.499+0000 7f71d4e12b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2363a48a0 con 0x55a236451000 2023-11-27T15:28:54.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.499+0000 7f71d4e12b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a23639dd30 con 0x55a236451950 2023-11-27T15:28:54.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.499+0000 7f71bb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236452530 0x55a236452980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.499+0000 7f71baffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451000 0x55a236451410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.499+0000 7f71ba7fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236451950 0x55a236451fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.499+0000 7f71baffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451000 0x55a236451410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:50986/0 (socket says 172.21.15.133:50986) 2023-11-27T15:28:54.505 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.499+0000 7f71baffd700 1 -- 172.21.15.133:0/1475935055 learned_addr learned my addr 172.21.15.133:0/1475935055 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:54.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.499+0000 7f71baffd700 1 -- 172.21.15.133:0/1475935055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236451950 msgr2=0x55a236451fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.499+0000 7f71baffd700 1 --2- 172.21.15.133:0/1475935055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236451950 0x55a236451fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.499+0000 7f71baffd700 1 -- 172.21.15.133:0/1475935055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236452530 msgr2=0x55a236452980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.499+0000 7f71baffd700 1 --2- 172.21.15.133:0/1475935055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236452530 0x55a236452980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.500+0000 7f71baffd700 1 -- 172.21.15.133:0/1475935055 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a23621f630 con 0x55a236451000 2023-11-27T15:28:54.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.500+0000 7f71bb7fe700 1 --2- 172.21.15.133:0/2748346708 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f71a40a1150 0x7f71a40a3600 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f71b40076f0 tx=0x7f71b40094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:54.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.500+0000 7f71baffd700 1 --2- 172.21.15.133:0/1475935055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451000 0x55a236451410 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f71ac09ad30 tx=0x7f71ac0056e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f5f9412fa67eeeb server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:54.506 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.500+0000 7f71a17fa700 1 -- 172.21.15.133:0/1475935055 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac09ada0 con 0x55a236451000 2023-11-27T15:28:54.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.500+0000 7f71a17fa700 1 -- 172.21.15.133:0/1475935055 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71ac09edd0 con 0x55a236451000 2023-11-27T15:28:54.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.500+0000 7f71a17fa700 1 -- 172.21.15.133:0/1475935055 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac021e50 con 0x55a236451000 2023-11-27T15:28:54.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.501+0000 7f71d4e12b80 1 -- 172.21.15.133:0/1475935055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451000 msgr2=0x55a236451410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.501+0000 7f71d4e12b80 1 --2- 172.21.15.133:0/1475935055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451000 0x55a236451410 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f71ac09ad30 tx=0x7f71ac0056e0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.501+0000 7f71d4e12b80 1 -- 172.21.15.133:0/1475935055 shutdown_connections 2023-11-27T15:28:54.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.501+0000 7f71d4e12b80 1 --2- 172.21.15.133:0/1475935055 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236452530 0x55a236452980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.501+0000 7f71d4e12b80 1 --2- 172.21.15.133:0/1475935055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451000 0x55a236451410 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.507 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.501+0000 7f71d4e12b80 1 --2- 172.21.15.133:0/1475935055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236451950 0x55a236451fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.508 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.501+0000 7f71d4e12b80 1 -- 172.21.15.133:0/1475935055 >> 172.21.15.133:0/1475935055 conn(0x55a236442360 msgr2=0x55a236447670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:54.508 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.501+0000 7f71d4e12b80 1 -- 172.21.15.133:0/1475935055 shutdown_connections 2023-11-27T15:28:54.508 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.501+0000 7f71d4e12b80 1 -- 172.21.15.133:0/1475935055 wait complete. 2023-11-27T15:28:54.508 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.501+0000 7f71d4e12b80 1 -- start start 2023-11-27T15:28:54.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.502+0000 7f71d4e12b80 1 Processor -- start 2023-11-27T15:28:54.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.502+0000 7f71d4e12b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236451000 0x55a23644db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.502+0000 7f71d4e12b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451950 0x55a23644e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.502+0000 7f71d4e12b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236452530 0x55a23644f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.502+0000 7f71d4e12b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a23633f8e0 con 0x55a236451000 2023-11-27T15:28:54.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.502+0000 7f71d4e12b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a23633f580 con 0x55a236451950 2023-11-27T15:28:54.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.502+0000 7f71d4e12b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a236221920 con 0x55a236452530 2023-11-27T15:28:54.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.502+0000 7f71d4e12b80 10 client.0 _ll_register_callbacks cb 0x55a23635d780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:54.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.502+0000 7f71baffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236451000 0x55a23644db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.502+0000 7f71baffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236451000 0x55a23644db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41972/0 (socket says 172.21.15.133:41972) 2023-11-27T15:28:54.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.502+0000 7f71baffd700 1 -- 172.21.15.133:0/844791539 learned_addr learned my addr 172.21.15.133:0/844791539 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:54.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.502+0000 7f71bb7fe700 1 --2- 172.21.15.133:0/844791539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236452530 0x55a23644f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.502+0000 7f71ba7fc700 1 --2- 172.21.15.133:0/844791539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451950 0x55a23644e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.503+0000 7f71baffd700 1 -- 172.21.15.133:0/844791539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236452530 msgr2=0x55a23644f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.503+0000 7f71baffd700 1 --2- 172.21.15.133:0/844791539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236452530 0x55a23644f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.503+0000 7f71baffd700 1 -- 172.21.15.133:0/844791539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451950 msgr2=0x55a23644e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.503+0000 7f71baffd700 1 --2- 172.21.15.133:0/844791539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451950 0x55a23644e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.513 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.503+0000 7f71baffd700 1 -- 172.21.15.133:0/844791539 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71bc10bb00 con 0x55a236451000 2023-11-27T15:28:54.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.503+0000 7f71baffd700 1 --2- 172.21.15.133:0/844791539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236451000 0x55a23644db70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00bfd0 tx=0x7f71ac097580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8025964746c61483 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:54.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.503+0000 7f719eff5700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:28:54.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.503+0000 7f719eff5700 1 -- 172.21.15.133:0/844791539 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac021c50 con 0x55a236451000 2023-11-27T15:28:54.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.503+0000 7f719eff5700 1 -- 172.21.15.133:0/844791539 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f71ac021db0 con 0x55a236451000 2023-11-27T15:28:54.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.503+0000 7f71d4e12b80 1 -- 172.21.15.133:0/844791539 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a23643fb20 con 0x55a236451000 2023-11-27T15:28:54.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.503+0000 7f719eff5700 1 -- 172.21.15.133:0/844791539 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71ac003c20 con 0x55a236451000 2023-11-27T15:28:54.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.503+0000 7f71d4e12b80 10 client.16458 fetch_fsmap learned FSMap version 30 2023-11-27T15:28:54.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.503+0000 7f71d4e12b80 1 -- 172.21.15.133:0/844791539 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a236364190 con 0x55a236451000 2023-11-27T15:28:54.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.504+0000 7f719eff5700 1 -- 172.21.15.133:0/844791539 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a23643fb20 con 0x55a236451000 2023-11-27T15:28:54.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.504+0000 7f719eff5700 1 -- 172.21.15.133:0/844791539 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f71ac09a290 con 0x55a236451000 2023-11-27T15:28:54.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.504+0000 7f71d4e12b80 10 client.16458 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:28:54.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.504+0000 7f71d4e12b80 10 client.16458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:28:54.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.504+0000 7f71d4e12b80 20 client.16458 populate_metadata read hostname 'smithi133' 2023-11-27T15:28:54.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.504+0000 7f71d4e12b80 1 --2- 172.21.15.133:0/844791539 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55a236461bf0 0x55a236464000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.504+0000 7f71d4e12b80 4 client.16458 mds_command: new command op to 24560 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-27T15:28:54.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.504+0000 7f71d4e12b80 1 -- 172.21.15.133:0/844791539 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2363600a0 con 0x55a236461bf0 2023-11-27T15:28:54.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.504+0000 7f71ba7fc700 1 --2- 172.21.15.133:0/844791539 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55a236461bf0 0x55a236464000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.505+0000 7f71ba7fc700 1 --2- 172.21.15.133:0/844791539 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55a236461bf0 0x55a236464000 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6a8a14fb74dabbe server_cookie=57fedd525aee9235 in_seq=0 out_seq=0 2023-11-27T15:28:54.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.505+0000 7f719eff5700 10 client.16458 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:54.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.505+0000 7f719eff5700 1 -- 172.21.15.133:0/844791539 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a2363600a0 con 0x55a236461bf0 2023-11-27T15:28:54.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.506+0000 7f719eff5700 10 client.16458 handle_command_reply: tid=0 2023-11-27T15:28:54.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.506+0000 7f71d4e12b80 1 client.16458 shutdown 2023-11-27T15:28:54.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.506+0000 7f71d4e12b80 1 -- 172.21.15.133:0/844791539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236451000 msgr2=0x55a23644db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.506+0000 7f71d4e12b80 1 --2- 172.21.15.133:0/844791539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236451000 0x55a23644db70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00bfd0 tx=0x7f71ac097580 comp rx=0 tx=0).stop 2023-11-27T15:28:54.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.506+0000 7f71d4e12b80 1 -- 172.21.15.133:0/844791539 shutdown_connections 2023-11-27T15:28:54.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.506+0000 7f71d4e12b80 1 --2- 172.21.15.133:0/844791539 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55a236461bf0 0x55a236464000 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.506+0000 7f71d4e12b80 1 --2- 172.21.15.133:0/844791539 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236452530 0x55a23644f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.506+0000 7f71d4e12b80 1 --2- 172.21.15.133:0/844791539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236451000 0x55a23644db70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.506+0000 7f719eff5700 0 client.16458 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:54.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.506+0000 7f71d4e12b80 1 --2- 172.21.15.133:0/844791539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451950 0x55a23644e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.506+0000 7f71d4e12b80 1 -- 172.21.15.133:0/844791539 >> 172.21.15.133:0/844791539 conn(0x55a236442360 msgr2=0x55a236446de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:54.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.506+0000 7f71d4e12b80 1 -- 172.21.15.133:0/844791539 shutdown_connections 2023-11-27T15:28:54.518 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.506+0000 7f71d4e12b80 1 -- 172.21.15.133:0/844791539 wait complete. 2023-11-27T15:28:54.518 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.506+0000 7f71d4e12b80 20 client.16458 trim_cache size 0 max 16384 2023-11-27T15:28:54.523 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.519+0000 7f71d4e12b80 1 Processor -- start 2023-11-27T15:28:54.523 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.520+0000 7f71d4e12b80 1 -- start start 2023-11-27T15:28:54.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.520+0000 7f71d4e12b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451000 0x55a236448710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.520+0000 7f71d4e12b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236451950 0x55a23645ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.520+0000 7f71d4e12b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236452530 0x55a23644d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.520+0000 7f71d4e12b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a236441f60 con 0x55a236452530 2023-11-27T15:28:54.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.520+0000 7f71d4e12b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a23644bd60 con 0x55a236451000 2023-11-27T15:28:54.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.520+0000 7f71d4e12b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a23643f660 con 0x55a236451950 2023-11-27T15:28:54.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.520+0000 7f71bb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236452530 0x55a23644d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.520+0000 7f71ba7fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236451950 0x55a23645ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.520+0000 7f71bb7fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236452530 0x55a23644d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41988/0 (socket says 172.21.15.133:41988) 2023-11-27T15:28:54.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.520+0000 7f71baffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451000 0x55a236448710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.520+0000 7f71ba7fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236451950 0x55a23645ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:58192/0 (socket says 172.21.15.133:58192) 2023-11-27T15:28:54.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.520+0000 7f71bb7fe700 1 -- 172.21.15.133:0/3703326370 learned_addr learned my addr 172.21.15.133:0/3703326370 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:54.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.520+0000 7f71ba7fc700 1 -- 172.21.15.133:0/3703326370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451000 msgr2=0x55a236448710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.520+0000 7f71ba7fc700 1 --2- 172.21.15.133:0/3703326370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451000 0x55a236448710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.520+0000 7f71ba7fc700 1 -- 172.21.15.133:0/3703326370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236452530 msgr2=0x55a23644d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.520+0000 7f71ba7fc700 1 --2- 172.21.15.133:0/3703326370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236452530 0x55a23644d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.520+0000 7f71ba7fc700 1 -- 172.21.15.133:0/3703326370 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a236363ee0 con 0x55a236451950 2023-11-27T15:28:54.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.520+0000 7f71baffd700 1 --2- 172.21.15.133:0/3703326370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451000 0x55a236448710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:28:54.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.521+0000 7f71ba7fc700 1 --2- 172.21.15.133:0/3703326370 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236451950 0x55a23645ef90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55a236461fc0 tx=0x7f71b0021e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=436f4fecf414ad47 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:54.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.521+0000 7f71a1ffb700 1 -- 172.21.15.133:0/3703326370 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b0017040 con 0x55a236451950 2023-11-27T15:28:54.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.521+0000 7f71a1ffb700 1 -- 172.21.15.133:0/3703326370 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71b0025070 con 0x55a236451950 2023-11-27T15:28:54.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.521+0000 7f71a1ffb700 1 -- 172.21.15.133:0/3703326370 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b0004490 con 0x55a236451950 2023-11-27T15:28:54.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.522+0000 7f71d4e12b80 1 -- 172.21.15.133:0/3703326370 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236451950 msgr2=0x55a23645ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.522+0000 7f71d4e12b80 1 --2- 172.21.15.133:0/3703326370 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236451950 0x55a23645ef90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55a236461fc0 tx=0x7f71b0021e00 comp rx=0 tx=0).stop 2023-11-27T15:28:54.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.522+0000 7f71d4e12b80 1 -- 172.21.15.133:0/3703326370 shutdown_connections 2023-11-27T15:28:54.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.522+0000 7f71d4e12b80 1 --2- 172.21.15.133:0/3703326370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236452530 0x55a23644d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.522+0000 7f71d4e12b80 1 --2- 172.21.15.133:0/3703326370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451000 0x55a236448710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.530 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.522+0000 7f71d4e12b80 1 --2- 172.21.15.133:0/3703326370 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236451950 0x55a23645ef90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.522+0000 7f71d4e12b80 1 -- 172.21.15.133:0/3703326370 >> 172.21.15.133:0/3703326370 conn(0x55a236442360 msgr2=0x55a236447640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:54.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.522+0000 7f71d4e12b80 1 -- 172.21.15.133:0/3703326370 shutdown_connections 2023-11-27T15:28:54.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.522+0000 7f71d4e12b80 1 -- 172.21.15.133:0/3703326370 wait complete. 2023-11-27T15:28:54.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.522+0000 7f71d4e12b80 1 -- start start 2023-11-27T15:28:54.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.522+0000 7f71d4e12b80 1 Processor -- start 2023-11-27T15:28:54.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.523+0000 7f71d4e12b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451000 0x55a236457260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.523+0000 7f71d4e12b80 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236451950 0x55a2364577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.531 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.523+0000 7f71d4e12b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236452530 0x55a2364b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.523+0000 7f71d4e12b80 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2364b4680 con 0x55a236451950 2023-11-27T15:28:54.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.523+0000 7f71d4e12b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2364b47f0 con 0x55a236451000 2023-11-27T15:28:54.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.523+0000 7f71d4e12b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2364b49b0 con 0x55a236452530 2023-11-27T15:28:54.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.523+0000 7f71d4e12b80 10 client.0 _ll_register_callbacks cb 0x7f71b4009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-27T15:28:54.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.523+0000 7f71ba7fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236451950 0x55a2364577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.523+0000 7f71ba7fc700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236451950 0x55a2364577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:41994/0 (socket says 172.21.15.133:41994) 2023-11-27T15:28:54.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.523+0000 7f71ba7fc700 1 -- 172.21.15.133:0/995532407 learned_addr learned my addr 172.21.15.133:0/995532407 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:54.532 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.523+0000 7f71bb7fe700 1 --2- 172.21.15.133:0/995532407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236452530 0x55a2364b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.523+0000 7f71baffd700 1 --2- 172.21.15.133:0/995532407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451000 0x55a236457260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.523+0000 7f71ba7fc700 1 -- 172.21.15.133:0/995532407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236452530 msgr2=0x55a2364b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.523+0000 7f71ba7fc700 1 --2- 172.21.15.133:0/995532407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236452530 0x55a2364b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.523+0000 7f71ba7fc700 1 -- 172.21.15.133:0/995532407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451000 msgr2=0x55a236457260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-27T15:28:54.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.523+0000 7f71ba7fc700 1 --2- 172.21.15.133:0/995532407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451000 0x55a236457260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.523+0000 7f71ba7fc700 1 -- 172.21.15.133:0/995532407 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a236364190 con 0x55a236451950 2023-11-27T15:28:54.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.524+0000 7f71ba7fc700 1 --2- 172.21.15.133:0/995532407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236451950 0x55a2364577a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f71b0005f50 tx=0x7f71b0012390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad32a3d2fa8bffb server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:54.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.524+0000 7f719eff5700 10 client.0 ms_handle_connect on v2:172.21.15.133:3300/0 2023-11-27T15:28:54.533 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.524+0000 7f719eff5700 1 -- 172.21.15.133:0/995532407 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b00043d0 con 0x55a236451950 2023-11-27T15:28:54.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.524+0000 7f71d4e12b80 1 -- 172.21.15.133:0/995532407 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71ac09edd0 con 0x55a236451950 2023-11-27T15:28:54.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.524+0000 7f719eff5700 1 -- 172.21.15.133:0/995532407 <== mon.0 v2:172.21.15.133:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f71b0004530 con 0x55a236451950 2023-11-27T15:28:54.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.524+0000 7f719eff5700 1 -- 172.21.15.133:0/995532407 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71b00210f0 con 0x55a236451950 2023-11-27T15:28:54.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.524+0000 7f71d4e12b80 10 client.16464 fetch_fsmap learned FSMap version 30 2023-11-27T15:28:54.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.524+0000 7f71d4e12b80 1 -- 172.21.15.133:0/995532407 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f71b0009e60 con 0x55a236451950 2023-11-27T15:28:54.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.525+0000 7f719eff5700 1 -- 172.21.15.133:0/995532407 <== mon.0 v2:172.21.15.133:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71ac09edd0 con 0x55a236451950 2023-11-27T15:28:54.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.525+0000 7f719eff5700 1 -- 172.21.15.133:0/995532407 <== mon.0 v2:172.21.15.133:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f71b0010430 con 0x55a236451950 2023-11-27T15:28:54.534 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.525+0000 7f71d4e12b80 10 client.16464 fetch_fsmap finished waiting for FSMap version 30 2023-11-27T15:28:54.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.525+0000 7f71d4e12b80 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-11-27T15:28:54.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.525+0000 7f71d4e12b80 20 client.16464 populate_metadata read hostname 'smithi133' 2023-11-27T15:28:54.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.525+0000 7f71d4e12b80 1 --2- 172.21.15.133:0/995532407 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55a2364b6310 0x55a2364b8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:54.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.525+0000 7f71d4e12b80 4 client.16464 mds_command: new command op to 24560 tid=0[{"prefix": "damage ls"}] 2023-11-27T15:28:54.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.525+0000 7f71d4e12b80 1 -- 172.21.15.133:0/995532407 --> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a23636c660 con 0x55a2364b6310 2023-11-27T15:28:54.535 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.525+0000 7f71baffd700 1 --2- 172.21.15.133:0/995532407 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55a2364b6310 0x55a2364b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:54.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.526+0000 7f71baffd700 1 --2- 172.21.15.133:0/995532407 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55a2364b6310 0x55a2364b8720 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37323aa4a8881e9d server_cookie=fda34dcce2083a90 in_seq=0 out_seq=0 2023-11-27T15:28:54.536 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.526+0000 7f719eff5700 10 client.16464 ms_handle_connect on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:54.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.526+0000 7f719eff5700 1 -- 172.21.15.133:0/995532407 <== mds.0 v2:172.21.15.174:6838/2781994447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a23636c660 con 0x55a2364b6310 2023-11-27T15:28:54.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.526+0000 7f719eff5700 10 client.16464 handle_command_reply: tid=0 2023-11-27T15:28:54.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.526+0000 7f71d4e12b80 1 client.16464 shutdown 2023-11-27T15:28:54.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.526+0000 7f71d4e12b80 1 -- 172.21.15.133:0/995532407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236451950 msgr2=0x55a2364577a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.526+0000 7f71d4e12b80 1 --2- 172.21.15.133:0/995532407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236451950 0x55a2364577a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f71b0005f50 tx=0x7f71b0012390 comp rx=0 tx=0).stop 2023-11-27T15:28:54.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.526+0000 7f71d4e12b80 1 -- 172.21.15.133:0/995532407 shutdown_connections 2023-11-27T15:28:54.537 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.526+0000 7f71d4e12b80 1 --2- 172.21.15.133:0/995532407 >> [v2:172.21.15.174:6838/2781994447,v1:172.21.15.174:6839/2781994447] conn(0x55a2364b6310 0x55a2364b8720 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.526+0000 7f71d4e12b80 1 --2- 172.21.15.133:0/995532407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a236452530 0x55a2364b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.526+0000 7f71d4e12b80 1 --2- 172.21.15.133:0/995532407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a236451000 0x55a236457260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.526+0000 7f71d4e12b80 1 --2- 172.21.15.133:0/995532407 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a236451950 0x55a2364577a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.526+0000 7f719eff5700 0 client.16464 ms_handle_reset on v2:172.21.15.174:6838/2781994447 2023-11-27T15:28:54.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.526+0000 7f71d4e12b80 1 -- 172.21.15.133:0/995532407 >> 172.21.15.133:0/995532407 conn(0x55a236442360 msgr2=0x55a236468960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:54.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.526+0000 7f71d4e12b80 1 -- 172.21.15.133:0/995532407 shutdown_connections 2023-11-27T15:28:54.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.526+0000 7f71d4e12b80 1 -- 172.21.15.133:0/995532407 wait complete. 2023-11-27T15:28:54.538 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.526+0000 7f71d4e12b80 20 client.16464 trim_cache size 0 max 16384 2023-11-27T15:28:54.539 INFO:teuthology.orchestra.run.smithi133.stdout:[] 2023-11-27T15:28:54.539 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.527+0000 7f71c13ff700 1 -- 172.21.15.133:0/2748346708 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f71a40a1150 msgr2=0x7f71a40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.527+0000 7f71c13ff700 1 --2- 172.21.15.133:0/2748346708 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f71a40a1150 0x7f71a40a3600 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f71b40076f0 tx=0x7f71b40094d0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.527+0000 7f71c13ff700 1 -- 172.21.15.133:0/2748346708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71bc1002c0 msgr2=0x7f71bc1960f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:54.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.527+0000 7f71c13ff700 1 --2- 172.21.15.133:0/2748346708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71bc1002c0 0x7f71bc1960f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f71ac005e90 tx=0x7f71ac00b7e0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.527+0000 7f71c13ff700 1 -- 172.21.15.133:0/2748346708 shutdown_connections 2023-11-27T15:28:54.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.527+0000 7f71c13ff700 1 --2- 172.21.15.133:0/2748346708 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f71a40a1150 0x7f71a40a3600 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.540 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.527+0000 7f71c13ff700 1 --2- 172.21.15.133:0/2748346708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71bc101e40 0x7f71bc19d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.527+0000 7f71c13ff700 1 --2- 172.21.15.133:0/2748346708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71bc1002c0 0x7f71bc1960f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.527+0000 7f71c13ff700 1 --2- 172.21.15.133:0/2748346708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71bc1014b0 0x7f71bc196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:54.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.527+0000 7f71c13ff700 1 -- 172.21.15.133:0/2748346708 >> 172.21.15.133:0/2748346708 conn(0x7f71bc0fb8a0 msgr2=0x7f71bc0fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:54.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.527+0000 7f71c13ff700 1 -- 172.21.15.133:0/2748346708 shutdown_connections 2023-11-27T15:28:54.541 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:54.527+0000 7f71c13ff700 1 -- 172.21.15.133:0/2748346708 wait complete. 2023-11-27T15:28:55.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:54 smithi133 ceph-mon[97860]: pgmap v530: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 13 op/s 2023-11-27T15:28:55.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:54 smithi177 ceph-mon[131504]: pgmap v530: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 13 op/s 2023-11-27T15:28:55.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:54 smithi174 ceph-mon[131110]: pgmap v530: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 10 KiB/s wr, 13 op/s 2023-11-27T15:28:55.549 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2023-11-27T15:28:55.549 INFO:tasks.fwd_scrub:done joining 2023-11-27T15:28:55.550 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-11-27T15:28:55.559 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:28:55.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.764+0000 7f7f455ff700 1 -- 172.21.15.133:0/446573112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f40100290 msgr2=0x7f7f401006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:55.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.764+0000 7f7f455ff700 1 --2- 172.21.15.133:0/446573112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f40100290 0x7f7f401006a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009990 tx=0x7f7f30009ca0 comp rx=0 tx=0).stop 2023-11-27T15:28:55.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.764+0000 7f7f455ff700 1 -- 172.21.15.133:0/446573112 shutdown_connections 2023-11-27T15:28:55.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.764+0000 7f7f455ff700 1 --2- 172.21.15.133:0/446573112 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f40101e10 0x7f7f401086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:55.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.764+0000 7f7f455ff700 1 --2- 172.21.15.133:0/446573112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f40100290 0x7f7f401006a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:55.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.764+0000 7f7f455ff700 1 --2- 172.21.15.133:0/446573112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f40101480 0x7f7f401018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:55.768 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.764+0000 7f7f455ff700 1 -- 172.21.15.133:0/446573112 >> 172.21.15.133:0/446573112 conn(0x7f7f400fb870 msgr2=0x7f7f400fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:55.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.764+0000 7f7f455ff700 1 -- 172.21.15.133:0/446573112 shutdown_connections 2023-11-27T15:28:55.769 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.764+0000 7f7f455ff700 1 -- 172.21.15.133:0/446573112 wait complete. 2023-11-27T15:28:55.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.765+0000 7f7f455ff700 1 Processor -- start 2023-11-27T15:28:55.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.765+0000 7f7f455ff700 1 -- start start 2023-11-27T15:28:55.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.766+0000 7f7f455ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f40100290 0x7f7f400748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:55.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.766+0000 7f7f455ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f40101480 0x7f7f40072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:55.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.766+0000 7f7f455ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f40101e10 0x7f7f40073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:55.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.766+0000 7f7f455ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f40073bc0 con 0x7f7f40101480 2023-11-27T15:28:55.771 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.766+0000 7f7f455ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f40073d30 con 0x7f7f40100290 2023-11-27T15:28:55.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.766+0000 7f7f455ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f401a0070 con 0x7f7f40101e10 2023-11-27T15:28:55.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.766+0000 7f7f37fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f40101480 0x7f7f40072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:55.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.766+0000 7f7f37fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f40101480 0x7f7f40072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42012/0 (socket says 172.21.15.133:42012) 2023-11-27T15:28:55.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.766+0000 7f7f37fff700 1 -- 172.21.15.133:0/551519229 learned_addr learned my addr 172.21.15.133:0/551519229 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:55.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.766+0000 7f7f3effd700 1 --2- 172.21.15.133:0/551519229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f40100290 0x7f7f400748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:55.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.766+0000 7f7f3f7fe700 1 --2- 172.21.15.133:0/551519229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f40101e10 0x7f7f40073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:55.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.766+0000 7f7f37fff700 1 -- 172.21.15.133:0/551519229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f40101e10 msgr2=0x7f7f40073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:55.772 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.766+0000 7f7f37fff700 1 --2- 172.21.15.133:0/551519229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f40101e10 0x7f7f40073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.766+0000 7f7f37fff700 1 -- 172.21.15.133:0/551519229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f40100290 msgr2=0x7f7f400748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.766+0000 7f7f37fff700 1 --2- 172.21.15.133:0/551519229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f40100290 0x7f7f400748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.766+0000 7f7f37fff700 1 -- 172.21.15.133:0/551519229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f300095f0 con 0x7f7f40101480 2023-11-27T15:28:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.767+0000 7f7f37fff700 1 --2- 172.21.15.133:0/551519229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f40101480 0x7f7f40072ee0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800e9a0 tx=0x7f7f2800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.767+0000 7f7f3cff9700 1 -- 172.21.15.133:0/551519229 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28009790 con 0x7f7f40101480 2023-11-27T15:28:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.767+0000 7f7f3cff9700 1 -- 172.21.15.133:0/551519229 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f28010430 con 0x7f7f40101480 2023-11-27T15:28:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.767+0000 7f7f3cff9700 1 -- 172.21.15.133:0/551519229 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28018680 con 0x7f7f40101480 2023-11-27T15:28:55.773 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.767+0000 7f7f455ff700 1 -- 172.21.15.133:0/551519229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f4010bc40 con 0x7f7f40101480 2023-11-27T15:28:55.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.767+0000 7f7f455ff700 1 -- 172.21.15.133:0/551519229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4010bab0 con 0x7f7f40101480 2023-11-27T15:28:55.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.768+0000 7f7f455ff700 1 -- 172.21.15.133:0/551519229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f40107870 con 0x7f7f40101480 2023-11-27T15:28:55.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.770+0000 7f7f3cff9700 1 -- 172.21.15.133:0/551519229 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f7f280105a0 con 0x7f7f40101480 2023-11-27T15:28:55.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.771+0000 7f7f3cff9700 1 --2- 172.21.15.133:0/551519229 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7f2c0a1150 0x7f7f2c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:55.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.771+0000 7f7f3cff9700 1 -- 172.21.15.133:0/551519229 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f7f28014070 con 0x7f7f40101480 2023-11-27T15:28:55.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.772+0000 7f7f3effd700 1 --2- 172.21.15.133:0/551519229 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7f2c0a1150 0x7f7f2c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:55.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.772+0000 7f7f3effd700 1 --2- 172.21.15.133:0/551519229 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7f2c0a1150 0x7f7f2c0a3600 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000b2e0 tx=0x7f7f30005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:55.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.776+0000 7f7f3cff9700 1 -- 172.21.15.133:0/551519229 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7f28051730 con 0x7f7f40101480 2023-11-27T15:28:55.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.979+0000 7f7f455ff700 1 -- 172.21.15.133:0/551519229 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f4002cfe0 con 0x7f7f40101480 2023-11-27T15:28:55.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.981+0000 7f7f3cff9700 1 -- 172.21.15.133:0/551519229 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24616 (secure 0 0 0) 0x7f7f280d3080 con 0x7f7f40101480 2023-11-27T15:28:55.984 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:28:55.985 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":94,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:28:26.730963+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:28:55.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.983+0000 7f7f455ff700 1 -- 172.21.15.133:0/551519229 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7f2c0a1150 msgr2=0x7f7f2c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:55.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.983+0000 7f7f455ff700 1 --2- 172.21.15.133:0/551519229 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7f2c0a1150 0x7f7f2c0a3600 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000b2e0 tx=0x7f7f30005fb0 comp rx=0 tx=0).stop 2023-11-27T15:28:55.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.983+0000 7f7f455ff700 1 -- 172.21.15.133:0/551519229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f40101480 msgr2=0x7f7f40072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:55.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.983+0000 7f7f455ff700 1 --2- 172.21.15.133:0/551519229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f40101480 0x7f7f40072ee0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800e9a0 tx=0x7f7f2800ecb0 comp rx=0 tx=0).stop 2023-11-27T15:28:55.987 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.984+0000 7f7f455ff700 1 -- 172.21.15.133:0/551519229 shutdown_connections 2023-11-27T15:28:55.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.984+0000 7f7f455ff700 1 --2- 172.21.15.133:0/551519229 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f7f2c0a1150 0x7f7f2c0a3600 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:55.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.984+0000 7f7f455ff700 1 --2- 172.21.15.133:0/551519229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7f40101e10 0x7f7f40073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:55.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.984+0000 7f7f455ff700 1 --2- 172.21.15.133:0/551519229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f40100290 0x7f7f400748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:55.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.984+0000 7f7f455ff700 1 --2- 172.21.15.133:0/551519229 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f40101480 0x7f7f40072ee0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:55.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.984+0000 7f7f455ff700 1 -- 172.21.15.133:0/551519229 >> 172.21.15.133:0/551519229 conn(0x7f7f400fb870 msgr2=0x7f7f401049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:55.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.984+0000 7f7f455ff700 1 -- 172.21.15.133:0/551519229 shutdown_connections 2023-11-27T15:28:55.988 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:55.984+0000 7f7f455ff700 1 -- 172.21.15.133:0/551519229 wait complete. 2023-11-27T15:28:56.006 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:16:20.216150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:20:31.107146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:20:32.614089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-27T15:20:36.126040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-27T15:28:56.006 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:28:56.206 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.203+0000 7f16427ff700 1 -- 172.21.15.133:0/2957698325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f163c074b30 msgr2=0x7f163c072f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:56.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.203+0000 7f16427ff700 1 --2- 172.21.15.133:0/2957698325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f163c074b30 0x7f163c072f80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f162c009990 tx=0x7f162c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:28:56.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.203+0000 7f16427ff700 1 -- 172.21.15.133:0/2957698325 shutdown_connections 2023-11-27T15:28:56.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.203+0000 7f16427ff700 1 --2- 172.21.15.133:0/2957698325 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f163c10c890 0x7f163c10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:56.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.203+0000 7f16427ff700 1 --2- 172.21.15.133:0/2957698325 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f163c074b30 0x7f163c072f80 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:56.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.203+0000 7f16427ff700 1 --2- 172.21.15.133:0/2957698325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f163c073550 0x7f163c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:56.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.203+0000 7f16427ff700 1 -- 172.21.15.133:0/2957698325 >> 172.21.15.133:0/2957698325 conn(0x7f163c0fb8a0 msgr2=0x7f163c0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:56.207 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.203+0000 7f16427ff700 1 -- 172.21.15.133:0/2957698325 shutdown_connections 2023-11-27T15:28:56.208 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.203+0000 7f16427ff700 1 -- 172.21.15.133:0/2957698325 wait complete. 2023-11-27T15:28:56.209 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.204+0000 7f16427ff700 1 Processor -- start 2023-11-27T15:28:56.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.204+0000 7f16427ff700 1 -- start start 2023-11-27T15:28:56.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.204+0000 7f16427ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f163c073550 0x7f163c19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:56.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.204+0000 7f16427ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f163c074b30 0x7f163c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:56.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.204+0000 7f16427ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f163c10c890 0x7f163c1a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:56.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.204+0000 7f16427ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c1a4210 con 0x7f163c10c890 2023-11-27T15:28:56.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.204+0000 7f16427ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c1a4350 con 0x7f163c074b30 2023-11-27T15:28:56.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.204+0000 7f16427ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c1a44e0 con 0x7f163c073550 2023-11-27T15:28:56.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.205+0000 7f1640d9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f163c10c890 0x7f163c1a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:56.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.205+0000 7f163b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f163c074b30 0x7f163c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:56.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.205+0000 7f1640d9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f163c10c890 0x7f163c1a3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42022/0 (socket says 172.21.15.133:42022) 2023-11-27T15:28:56.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.205+0000 7f163b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f163c074b30 0x7f163c19cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:51038/0 (socket says 172.21.15.133:51038) 2023-11-27T15:28:56.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.205+0000 7f1640d9c700 1 -- 172.21.15.133:0/1085932148 learned_addr learned my addr 172.21.15.133:0/1085932148 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:56.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.205+0000 7f163bfff700 1 --2- 172.21.15.133:0/1085932148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f163c073550 0x7f163c19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:56.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.205+0000 7f163b7fe700 1 -- 172.21.15.133:0/1085932148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f163c073550 msgr2=0x7f163c19c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:56.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.205+0000 7f163b7fe700 1 --2- 172.21.15.133:0/1085932148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f163c073550 0x7f163c19c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:56.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.205+0000 7f163b7fe700 1 -- 172.21.15.133:0/1085932148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f163c10c890 msgr2=0x7f163c1a3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:56.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.205+0000 7f163b7fe700 1 --2- 172.21.15.133:0/1085932148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f163c10c890 0x7f163c1a3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:56.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.205+0000 7f163b7fe700 1 -- 172.21.15.133:0/1085932148 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162c0095f0 con 0x7f163c074b30 2023-11-27T15:28:56.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.205+0000 7f1640d9c700 1 --2- 172.21.15.133:0/1085932148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f163c10c890 0x7f163c1a3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:28:56.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.205+0000 7f163b7fe700 1 --2- 172.21.15.133:0/1085932148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f163c074b30 0x7f163c19cbd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f163000e9c0 tx=0x7f163000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:56.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.206+0000 7f16397fa700 1 -- 172.21.15.133:0/1085932148 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1630004220 con 0x7f163c074b30 2023-11-27T15:28:56.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.206+0000 7f16427ff700 1 -- 172.21.15.133:0/1085932148 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163c112280 con 0x7f163c074b30 2023-11-27T15:28:56.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.206+0000 7f16397fa700 1 -- 172.21.15.133:0/1085932148 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1630004380 con 0x7f163c074b30 2023-11-27T15:28:56.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.206+0000 7f16397fa700 1 -- 172.21.15.133:0/1085932148 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1630003be0 con 0x7f163c074b30 2023-11-27T15:28:56.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.206+0000 7f16427ff700 1 -- 172.21.15.133:0/1085932148 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163c1120f0 con 0x7f163c074b30 2023-11-27T15:28:56.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.207+0000 7f16427ff700 1 -- 172.21.15.133:0/1085932148 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f163c1023e0 con 0x7f163c074b30 2023-11-27T15:28:56.213 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.209+0000 7f16397fa700 1 -- 172.21.15.133:0/1085932148 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1630003d40 con 0x7f163c074b30 2023-11-27T15:28:56.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.210+0000 7f16397fa700 1 --2- 172.21.15.133:0/1085932148 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f16240a1150 0x7f16240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:56.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.210+0000 7f16397fa700 1 -- 172.21.15.133:0/1085932148 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1630014070 con 0x7f163c074b30 2023-11-27T15:28:56.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.210+0000 7f163bfff700 1 --2- 172.21.15.133:0/1085932148 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f16240a1150 0x7f16240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:56.214 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.211+0000 7f163bfff700 1 --2- 172.21.15.133:0/1085932148 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f16240a1150 0x7f16240a3600 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f162c00b2e0 tx=0x7f162c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:56.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.212+0000 7f16397fa700 1 -- 172.21.15.133:0/1085932148 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f16300968d0 con 0x7f163c074b30 2023-11-27T15:28:56.414 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.411+0000 7f16427ff700 1 -- 172.21.15.133:0/1085932148 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f163c02cfe0 con 0x7f163c074b30 2023-11-27T15:28:56.415 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.411+0000 7f16397fa700 1 -- 172.21.15.133:0/1085932148 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f163001f340 con 0x7f163c074b30 2023-11-27T15:28:56.415 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-11-27T15:28:56.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.414+0000 7f16427ff700 1 -- 172.21.15.133:0/1085932148 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f16240a1150 msgr2=0x7f16240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:56.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.414+0000 7f16427ff700 1 --2- 172.21.15.133:0/1085932148 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f16240a1150 0x7f16240a3600 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f162c00b2e0 tx=0x7f162c005fb0 comp rx=0 tx=0).stop 2023-11-27T15:28:56.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.414+0000 7f16427ff700 1 -- 172.21.15.133:0/1085932148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f163c074b30 msgr2=0x7f163c19cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:56.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.414+0000 7f16427ff700 1 --2- 172.21.15.133:0/1085932148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f163c074b30 0x7f163c19cbd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f163000e9c0 tx=0x7f163000ecd0 comp rx=0 tx=0).stop 2023-11-27T15:28:56.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.414+0000 7f16427ff700 1 -- 172.21.15.133:0/1085932148 shutdown_connections 2023-11-27T15:28:56.418 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.414+0000 7f16427ff700 1 --2- 172.21.15.133:0/1085932148 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f16240a1150 0x7f16240a3600 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:56.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.414+0000 7f16427ff700 1 --2- 172.21.15.133:0/1085932148 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f163c10c890 0x7f163c1a3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:56.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.414+0000 7f16427ff700 1 --2- 172.21.15.133:0/1085932148 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f163c073550 0x7f163c19c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:56.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.414+0000 7f16427ff700 1 --2- 172.21.15.133:0/1085932148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f163c074b30 0x7f163c19cbd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:56.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.414+0000 7f16427ff700 1 -- 172.21.15.133:0/1085932148 >> 172.21.15.133:0/1085932148 conn(0x7f163c0fb8a0 msgr2=0x7f163c0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:56.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.414+0000 7f16427ff700 1 -- 172.21.15.133:0/1085932148 shutdown_connections 2023-11-27T15:28:56.419 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.414+0000 7f16427ff700 1 -- 172.21.15.133:0/1085932148 wait complete. 2023-11-27T15:28:56.436 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:28:56.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.634+0000 7f893f9ff700 1 -- 172.21.15.133:0/3314266843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8938101e40 msgr2=0x7f8938108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:56.637 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.634+0000 7f893f9ff700 1 --2- 172.21.15.133:0/3314266843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8938101e40 0x7f8938108700 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8930009f90 tx=0x7f893000b330 comp rx=0 tx=0).stop 2023-11-27T15:28:56.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.634+0000 7f893f9ff700 1 -- 172.21.15.133:0/3314266843 shutdown_connections 2023-11-27T15:28:56.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.634+0000 7f893f9ff700 1 --2- 172.21.15.133:0/3314266843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8938101e40 0x7f8938108700 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:56.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.634+0000 7f893f9ff700 1 --2- 172.21.15.133:0/3314266843 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89381002c0 0x7f89381006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:56.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.634+0000 7f893f9ff700 1 --2- 172.21.15.133:0/3314266843 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f89381014b0 0x7f8938101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:56.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.634+0000 7f893f9ff700 1 -- 172.21.15.133:0/3314266843 >> 172.21.15.133:0/3314266843 conn(0x7f89380fb8a0 msgr2=0x7f89380fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:56.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.634+0000 7f893f9ff700 1 -- 172.21.15.133:0/3314266843 shutdown_connections 2023-11-27T15:28:56.638 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.634+0000 7f893f9ff700 1 -- 172.21.15.133:0/3314266843 wait complete. 2023-11-27T15:28:56.640 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.635+0000 7f893f9ff700 1 Processor -- start 2023-11-27T15:28:56.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.635+0000 7f893f9ff700 1 -- start start 2023-11-27T15:28:56.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.635+0000 7f893f9ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89381002c0 0x7f8938198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:56.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.635+0000 7f893f9ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89381014b0 0x7f89381988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:56.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.635+0000 7f893f9ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8938101e40 0x7f893819f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:56.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.635+0000 7f893f9ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893819fee0 con 0x7f8938101e40 2023-11-27T15:28:56.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.635+0000 7f893f9ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89381a0020 con 0x7f89381002c0 2023-11-27T15:28:56.641 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.635+0000 7f893f9ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89381a01b0 con 0x7f89381014b0 2023-11-27T15:28:56.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.636+0000 7f893df9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8938101e40 0x7f893819f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:56.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.636+0000 7f893df9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8938101e40 0x7f893819f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42044/0 (socket says 172.21.15.133:42044) 2023-11-27T15:28:56.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.636+0000 7f893df9c700 1 -- 172.21.15.133:0/3126936665 learned_addr learned my addr 172.21.15.133:0/3126936665 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:56.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.636+0000 7f893d79b700 1 --2- 172.21.15.133:0/3126936665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89381002c0 0x7f8938198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:56.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.636+0000 7f893cf9a700 1 --2- 172.21.15.133:0/3126936665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89381014b0 0x7f89381988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:56.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.636+0000 7f893df9c700 1 -- 172.21.15.133:0/3126936665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89381014b0 msgr2=0x7f89381988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:56.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.636+0000 7f893df9c700 1 --2- 172.21.15.133:0/3126936665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89381014b0 0x7f89381988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:56.642 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.636+0000 7f893df9c700 1 -- 172.21.15.133:0/3126936665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89381002c0 msgr2=0x7f8938198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:56.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.636+0000 7f893df9c700 1 --2- 172.21.15.133:0/3126936665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89381002c0 0x7f8938198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:56.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.636+0000 7f893df9c700 1 -- 172.21.15.133:0/3126936665 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89280095f0 con 0x7f8938101e40 2023-11-27T15:28:56.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.636+0000 7f893df9c700 1 --2- 172.21.15.133:0/3126936665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8938101e40 0x7f893819f910 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8930007930 tx=0x7f8930004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:56.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.636+0000 7f892e7fc700 1 -- 172.21.15.133:0/3126936665 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893000e040 con 0x7f8938101e40 2023-11-27T15:28:56.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.637+0000 7f893f9ff700 1 -- 172.21.15.133:0/3126936665 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8930009c40 con 0x7f8938101e40 2023-11-27T15:28:56.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.637+0000 7f892e7fc700 1 -- 172.21.15.133:0/3126936665 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8930004e80 con 0x7f8938101e40 2023-11-27T15:28:56.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.637+0000 7f893f9ff700 1 -- 172.21.15.133:0/3126936665 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893810bc90 con 0x7f8938101e40 2023-11-27T15:28:56.643 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.637+0000 7f892e7fc700 1 -- 172.21.15.133:0/3126936665 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930005240 con 0x7f8938101e40 2023-11-27T15:28:56.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.638+0000 7f893f9ff700 1 -- 172.21.15.133:0/3126936665 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893810be20 con 0x7f8938101e40 2023-11-27T15:28:56.644 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.640+0000 7f892e7fc700 1 -- 172.21.15.133:0/3126936665 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f893001d470 con 0x7f8938101e40 2023-11-27T15:28:56.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.641+0000 7f892e7fc700 1 --2- 172.21.15.133:0/3126936665 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f89240a1150 0x7f89240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:56.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.641+0000 7f892e7fc700 1 -- 172.21.15.133:0/3126936665 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f89300ce580 con 0x7f8938101e40 2023-11-27T15:28:56.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.641+0000 7f893d79b700 1 --2- 172.21.15.133:0/3126936665 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f89240a1150 0x7f89240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:56.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.642+0000 7f893d79b700 1 --2- 172.21.15.133:0/3126936665 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f89240a1150 0x7f89240a3600 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8938101310 tx=0x7f8934009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:56.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.644+0000 7f892e7fc700 1 -- 172.21.15.133:0/3126936665 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8930097a10 con 0x7f8938101e40 2023-11-27T15:28:56.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.844+0000 7f893f9ff700 1 -- 172.21.15.133:0/3126936665 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f89381a0c90 con 0x7f8938101e40 2023-11-27T15:28:56.848 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.845+0000 7f892e7fc700 1 -- 172.21.15.133:0/3126936665 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f893009b030 con 0x7f8938101e40 2023-11-27T15:28:56.848 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:28:56.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.847+0000 7f893f9ff700 1 -- 172.21.15.133:0/3126936665 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f89240a1150 msgr2=0x7f89240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:56.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.847+0000 7f893f9ff700 1 --2- 172.21.15.133:0/3126936665 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f89240a1150 0x7f89240a3600 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8938101310 tx=0x7f8934009260 comp rx=0 tx=0).stop 2023-11-27T15:28:56.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.847+0000 7f893f9ff700 1 -- 172.21.15.133:0/3126936665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8938101e40 msgr2=0x7f893819f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:56.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.847+0000 7f893f9ff700 1 --2- 172.21.15.133:0/3126936665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8938101e40 0x7f893819f910 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8930007930 tx=0x7f8930004c60 comp rx=0 tx=0).stop 2023-11-27T15:28:56.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.847+0000 7f893f9ff700 1 -- 172.21.15.133:0/3126936665 shutdown_connections 2023-11-27T15:28:56.851 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.847+0000 7f893f9ff700 1 --2- 172.21.15.133:0/3126936665 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f89240a1150 0x7f89240a3600 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:56.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.847+0000 7f893f9ff700 1 --2- 172.21.15.133:0/3126936665 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8938101e40 0x7f893819f910 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:56.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.847+0000 7f893f9ff700 1 --2- 172.21.15.133:0/3126936665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89381002c0 0x7f8938198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:56.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.847+0000 7f893f9ff700 1 --2- 172.21.15.133:0/3126936665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89381014b0 0x7f89381988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:56.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.847+0000 7f893f9ff700 1 -- 172.21.15.133:0/3126936665 >> 172.21.15.133:0/3126936665 conn(0x7f89380fb8a0 msgr2=0x7f8938106d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:56.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.847+0000 7f893f9ff700 1 -- 172.21.15.133:0/3126936665 shutdown_connections 2023-11-27T15:28:56.852 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:56.847+0000 7f893f9ff700 1 -- 172.21.15.133:0/3126936665 wait complete. 2023-11-27T15:28:56.870 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:28:57.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.068+0000 7f8033dff700 1 -- 172.21.15.133:0/3867505573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f802c1014c0 msgr2=0x7f802c101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:57.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.068+0000 7f8033dff700 1 --2- 172.21.15.133:0/3867505573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f802c1014c0 0x7f802c101910 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f801c009990 tx=0x7f801c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.072 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.068+0000 7f8033dff700 1 -- 172.21.15.133:0/3867505573 shutdown_connections 2023-11-27T15:28:57.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.068+0000 7f8033dff700 1 --2- 172.21.15.133:0/3867505573 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f802c101e50 0x7f802c108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.068+0000 7f8033dff700 1 --2- 172.21.15.133:0/3867505573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f802c1002d0 0x7f802c1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.068+0000 7f8033dff700 1 --2- 172.21.15.133:0/3867505573 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f802c1014c0 0x7f802c101910 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.068+0000 7f8033dff700 1 -- 172.21.15.133:0/3867505573 >> 172.21.15.133:0/3867505573 conn(0x7f802c0fb8b0 msgr2=0x7f802c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:57.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.068+0000 7f8033dff700 1 -- 172.21.15.133:0/3867505573 shutdown_connections 2023-11-27T15:28:57.073 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.068+0000 7f8033dff700 1 -- 172.21.15.133:0/3867505573 wait complete. 2023-11-27T15:28:57.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.069+0000 7f8033dff700 1 Processor -- start 2023-11-27T15:28:57.075 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.070+0000 7f8033dff700 1 -- start start 2023-11-27T15:28:57.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.070+0000 7f8033dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f802c1002d0 0x7f802c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:57.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.070+0000 7f8033dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f802c1014c0 0x7f802c1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:57.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.070+0000 7f8033dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f802c101e50 0x7f802c19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:57.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.070+0000 7f8033dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c19fe20 con 0x7f802c1002d0 2023-11-27T15:28:57.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.070+0000 7f8033dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c19ff60 con 0x7f802c101e50 2023-11-27T15:28:57.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.070+0000 7f8033dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c1a00f0 con 0x7f802c1014c0 2023-11-27T15:28:57.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.070+0000 7f8031b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f802c1002d0 0x7f802c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:57.076 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.070+0000 7f803139a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f802c1014c0 0x7f802c1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:57.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.070+0000 7f8031b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f802c1002d0 0x7f802c1982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42054/0 (socket says 172.21.15.133:42054) 2023-11-27T15:28:57.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.070+0000 7f8031b9b700 1 -- 172.21.15.133:0/46596916 learned_addr learned my addr 172.21.15.133:0/46596916 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:57.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.070+0000 7f803239c700 1 --2- 172.21.15.133:0/46596916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f802c101e50 0x7f802c19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:57.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.070+0000 7f8031b9b700 1 -- 172.21.15.133:0/46596916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f802c1014c0 msgr2=0x7f802c1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:57.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.070+0000 7f8031b9b700 1 --2- 172.21.15.133:0/46596916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f802c1014c0 0x7f802c1987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.070+0000 7f8031b9b700 1 -- 172.21.15.133:0/46596916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f802c101e50 msgr2=0x7f802c19f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:57.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.070+0000 7f8031b9b700 1 --2- 172.21.15.133:0/46596916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f802c101e50 0x7f802c19f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.077 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.070+0000 7f8031b9b700 1 -- 172.21.15.133:0/46596916 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c0095f0 con 0x7f802c1002d0 2023-11-27T15:28:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.071+0000 7f8031b9b700 1 --2- 172.21.15.133:0/46596916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f802c1002d0 0x7f802c1982a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f802800e970 tx=0x7f802800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.071+0000 7f8022ffd700 1 -- 172.21.15.133:0/46596916 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028009790 con 0x7f802c1002d0 2023-11-27T15:28:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.071+0000 7f8022ffd700 1 -- 172.21.15.133:0/46596916 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f802800eec0 con 0x7f802c1002d0 2023-11-27T15:28:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.071+0000 7f8022ffd700 1 -- 172.21.15.133:0/46596916 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028003cf0 con 0x7f802c1002d0 2023-11-27T15:28:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.071+0000 7f8033dff700 1 -- 172.21.15.133:0/46596916 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802c10bc80 con 0x7f802c1002d0 2023-11-27T15:28:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.071+0000 7f8033dff700 1 -- 172.21.15.133:0/46596916 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802c10baf0 con 0x7f802c1002d0 2023-11-27T15:28:57.078 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.072+0000 7f8033dff700 1 -- 172.21.15.133:0/46596916 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802c1a07f0 con 0x7f802c1002d0 2023-11-27T15:28:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.074+0000 7f8022ffd700 1 -- 172.21.15.133:0/46596916 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f802800ccd0 con 0x7f802c1002d0 2023-11-27T15:28:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.075+0000 7f8022ffd700 1 --2- 172.21.15.133:0/46596916 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f80180a11d0 0x7f80180a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:57.079 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.075+0000 7f8022ffd700 1 -- 172.21.15.133:0/46596916 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f8028014070 con 0x7f802c1002d0 2023-11-27T15:28:57.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.075+0000 7f803139a700 1 --2- 172.21.15.133:0/46596916 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f80180a11d0 0x7f80180a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:57.080 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.076+0000 7f803139a700 1 --2- 172.21.15.133:0/46596916 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f80180a11d0 0x7f80180a3680 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f801c00b2e0 tx=0x7f801c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:57.082 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.079+0000 7f8022ffd700 1 -- 172.21.15.133:0/46596916 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8028096210 con 0x7f802c1002d0 2023-11-27T15:28:57.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:56 smithi133 ceph-mon[97860]: pgmap v531: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2023-11-27T15:28:57.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:56 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/551519229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:28:57.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:56 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1085932148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:28:57.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:56 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3126936665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:28:57.287 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.284+0000 7f8033dff700 1 -- 172.21.15.133:0/46596916 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f802c1a0ed0 con 0x7f802c1002d0 2023-11-27T15:28:57.288 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.284+0000 7f8022ffd700 1 -- 172.21.15.133:0/46596916 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8028099830 con 0x7f802c1002d0 2023-11-27T15:28:57.288 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-11-27T15:28:57.290 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.287+0000 7f8033dff700 1 -- 172.21.15.133:0/46596916 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f80180a11d0 msgr2=0x7f80180a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:57.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.287+0000 7f8033dff700 1 --2- 172.21.15.133:0/46596916 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f80180a11d0 0x7f80180a3680 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f801c00b2e0 tx=0x7f801c005fb0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.287+0000 7f8033dff700 1 -- 172.21.15.133:0/46596916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f802c1002d0 msgr2=0x7f802c1982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:57.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.287+0000 7f8033dff700 1 --2- 172.21.15.133:0/46596916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f802c1002d0 0x7f802c1982a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f802800e970 tx=0x7f802800ec80 comp rx=0 tx=0).stop 2023-11-27T15:28:57.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.287+0000 7f8033dff700 1 -- 172.21.15.133:0/46596916 shutdown_connections 2023-11-27T15:28:57.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.287+0000 7f8033dff700 1 --2- 172.21.15.133:0/46596916 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f80180a11d0 0x7f80180a3680 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.287+0000 7f8033dff700 1 --2- 172.21.15.133:0/46596916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f802c101e50 0x7f802c19f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.291 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.287+0000 7f8033dff700 1 --2- 172.21.15.133:0/46596916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f802c1002d0 0x7f802c1982a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.287+0000 7f8033dff700 1 --2- 172.21.15.133:0/46596916 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f802c1014c0 0x7f802c1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.287+0000 7f8033dff700 1 -- 172.21.15.133:0/46596916 >> 172.21.15.133:0/46596916 conn(0x7f802c0fb8b0 msgr2=0x7f802c0fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:57.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.287+0000 7f8033dff700 1 -- 172.21.15.133:0/46596916 shutdown_connections 2023-11-27T15:28:57.292 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.287+0000 7f8033dff700 1 -- 172.21.15.133:0/46596916 wait complete. 2023-11-27T15:28:57.310 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-27T15:28:57.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:56 smithi177 ceph-mon[131504]: pgmap v531: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2023-11-27T15:28:57.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:56 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/551519229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:28:57.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:56 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1085932148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:28:57.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:56 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3126936665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:28:57.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:56 smithi174 ceph-mon[131110]: pgmap v531: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2023-11-27T15:28:57.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:56 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/551519229' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:28:57.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:56 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1085932148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:28:57.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:56 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3126936665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:28:57.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.507+0000 7ff2891ff700 1 -- 172.21.15.133:0/60369600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2840feb30 msgr2=0x7ff284100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:57.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.507+0000 7ff2891ff700 1 --2- 172.21.15.133:0/60369600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2840feb30 0x7ff284100f50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff274009990 tx=0x7ff274009ca0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.507+0000 7ff2891ff700 1 -- 172.21.15.133:0/60369600 shutdown_connections 2023-11-27T15:28:57.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.507+0000 7ff2891ff700 1 --2- 172.21.15.133:0/60369600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff284103e50 0x7ff2841042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.507+0000 7ff2891ff700 1 --2- 172.21.15.133:0/60369600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2840feb30 0x7ff284100f50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.507+0000 7ff2891ff700 1 --2- 172.21.15.133:0/60369600 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff284101490 0x7ff284103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.511 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.507+0000 7ff2891ff700 1 -- 172.21.15.133:0/60369600 >> 172.21.15.133:0/60369600 conn(0x7ff2840fa7c0 msgr2=0x7ff2840fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:57.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.507+0000 7ff2891ff700 1 -- 172.21.15.133:0/60369600 shutdown_connections 2023-11-27T15:28:57.512 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.507+0000 7ff2891ff700 1 -- 172.21.15.133:0/60369600 wait complete. 2023-11-27T15:28:57.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.508+0000 7ff2891ff700 1 Processor -- start 2023-11-27T15:28:57.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.509+0000 7ff2891ff700 1 -- start start 2023-11-27T15:28:57.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.509+0000 7ff2891ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2840feb30 0x7ff28419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:57.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.509+0000 7ff2891ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff284101490 0x7ff28419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:57.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.509+0000 7ff2891ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff284103e50 0x7ff2841a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:57.514 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.509+0000 7ff2891ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2841a4250 con 0x7ff2840feb30 2023-11-27T15:28:57.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.509+0000 7ff2891ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2841a4390 con 0x7ff284101490 2023-11-27T15:28:57.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.509+0000 7ff2891ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2841a4520 con 0x7ff284103e50 2023-11-27T15:28:57.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.509+0000 7ff282d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2840feb30 0x7ff28419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:57.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.509+0000 7ff282d9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2840feb30 0x7ff28419c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:42080/0 (socket says 172.21.15.133:42080) 2023-11-27T15:28:57.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.509+0000 7ff282d9d700 1 -- 172.21.15.133:0/1477480169 learned_addr learned my addr 172.21.15.133:0/1477480169 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:57.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.509+0000 7ff28259c700 1 --2- 172.21.15.133:0/1477480169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff284101490 0x7ff28419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:57.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.510+0000 7ff28359e700 1 --2- 172.21.15.133:0/1477480169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff284103e50 0x7ff2841a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:57.515 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.510+0000 7ff282d9d700 1 -- 172.21.15.133:0/1477480169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff284103e50 msgr2=0x7ff2841a3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:57.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.510+0000 7ff282d9d700 1 --2- 172.21.15.133:0/1477480169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff284103e50 0x7ff2841a3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.510+0000 7ff282d9d700 1 -- 172.21.15.133:0/1477480169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff284101490 msgr2=0x7ff28419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:57.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.510+0000 7ff282d9d700 1 --2- 172.21.15.133:0/1477480169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff284101490 0x7ff28419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.510+0000 7ff282d9d700 1 -- 172.21.15.133:0/1477480169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2740095f0 con 0x7ff2840feb30 2023-11-27T15:28:57.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.510+0000 7ff282d9d700 1 --2- 172.21.15.133:0/1477480169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2840feb30 0x7ff28419c6d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff27400b680 tx=0x7ff274005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:57.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.510+0000 7ff26bfff700 1 -- 172.21.15.133:0/1477480169 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27401d070 con 0x7ff2840feb30 2023-11-27T15:28:57.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.510+0000 7ff26bfff700 1 -- 172.21.15.133:0/1477480169 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff274021410 con 0x7ff2840feb30 2023-11-27T15:28:57.516 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.510+0000 7ff26bfff700 1 -- 172.21.15.133:0/1477480169 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff274017650 con 0x7ff2840feb30 2023-11-27T15:28:57.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.510+0000 7ff2891ff700 1 -- 172.21.15.133:0/1477480169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff284112260 con 0x7ff2840feb30 2023-11-27T15:28:57.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.510+0000 7ff2891ff700 1 -- 172.21.15.133:0/1477480169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2841120d0 con 0x7ff2840feb30 2023-11-27T15:28:57.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.512+0000 7ff2891ff700 1 -- 172.21.15.133:0/1477480169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2840fc330 con 0x7ff2840feb30 2023-11-27T15:28:57.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.513+0000 7ff26bfff700 1 -- 172.21.15.133:0/1477480169 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff2740217f0 con 0x7ff2840feb30 2023-11-27T15:28:57.518 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.515+0000 7ff26bfff700 1 --2- 172.21.15.133:0/1477480169 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff26c0a11d0 0x7ff26c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:57.518 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.515+0000 7ff26bfff700 1 -- 172.21.15.133:0/1477480169 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ff2740ce090 con 0x7ff2840feb30 2023-11-27T15:28:57.518 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.515+0000 7ff28259c700 1 --2- 172.21.15.133:0/1477480169 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff26c0a11d0 0x7ff26c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:57.519 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.515+0000 7ff28259c700 1 --2- 172.21.15.133:0/1477480169 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff26c0a11d0 0x7ff26c0a3680 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff278009c60 tx=0x7ff278009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:57.522 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.518+0000 7ff26bfff700 1 -- 172.21.15.133:0/1477480169 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff2740974c0 con 0x7ff2840feb30 2023-11-27T15:28:57.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.715+0000 7ff2891ff700 1 -- 172.21.15.133:0/1477480169 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff284061700 con 0x7ff2840feb30 2023-11-27T15:28:57.719 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.716+0000 7ff26bfff700 1 -- 172.21.15.133:0/1477480169 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff27409aae0 con 0x7ff2840feb30 2023-11-27T15:28:57.719 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:28:57.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.718+0000 7ff2891ff700 1 -- 172.21.15.133:0/1477480169 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff26c0a11d0 msgr2=0x7ff26c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:57.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.718+0000 7ff2891ff700 1 --2- 172.21.15.133:0/1477480169 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff26c0a11d0 0x7ff26c0a3680 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff278009c60 tx=0x7ff278009260 comp rx=0 tx=0).stop 2023-11-27T15:28:57.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.718+0000 7ff2891ff700 1 -- 172.21.15.133:0/1477480169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2840feb30 msgr2=0x7ff28419c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:57.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.718+0000 7ff2891ff700 1 --2- 172.21.15.133:0/1477480169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2840feb30 0x7ff28419c6d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff27400b680 tx=0x7ff274005740 comp rx=0 tx=0).stop 2023-11-27T15:28:57.722 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.718+0000 7ff2891ff700 1 -- 172.21.15.133:0/1477480169 shutdown_connections 2023-11-27T15:28:57.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.718+0000 7ff2891ff700 1 --2- 172.21.15.133:0/1477480169 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff26c0a11d0 0x7ff26c0a3680 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.718+0000 7ff2891ff700 1 --2- 172.21.15.133:0/1477480169 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff284103e50 0x7ff2841a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.718+0000 7ff2891ff700 1 --2- 172.21.15.133:0/1477480169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2840feb30 0x7ff28419c6d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.718+0000 7ff2891ff700 1 --2- 172.21.15.133:0/1477480169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff284101490 0x7ff28419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.718+0000 7ff2891ff700 1 -- 172.21.15.133:0/1477480169 >> 172.21.15.133:0/1477480169 conn(0x7ff2840fa7c0 msgr2=0x7ff284102190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:57.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.719+0000 7ff2891ff700 1 -- 172.21.15.133:0/1477480169 shutdown_connections 2023-11-27T15:28:57.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.719+0000 7ff2891ff700 1 -- 172.21.15.133:0/1477480169 wait complete. 2023-11-27T15:28:57.741 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-11-27T15:28:57.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.939+0000 7f1b9dfff700 1 -- 172.21.15.133:0/2368715882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b98101e10 msgr2=0x7f1b981086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:57.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.939+0000 7f1b9dfff700 1 --2- 172.21.15.133:0/2368715882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b98101e10 0x7f1b981086d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000b0a0 tx=0x7f1b9000b3b0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.943 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.939+0000 7f1b9dfff700 1 -- 172.21.15.133:0/2368715882 shutdown_connections 2023-11-27T15:28:57.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.939+0000 7f1b9dfff700 1 --2- 172.21.15.133:0/2368715882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b98101e10 0x7f1b981086d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.939+0000 7f1b9dfff700 1 --2- 172.21.15.133:0/2368715882 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b98100290 0x7f1b981006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.939+0000 7f1b9dfff700 1 --2- 172.21.15.133:0/2368715882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b98101480 0x7f1b981018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.939+0000 7f1b9dfff700 1 -- 172.21.15.133:0/2368715882 >> 172.21.15.133:0/2368715882 conn(0x7f1b980fb870 msgr2=0x7f1b980fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:28:57.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.939+0000 7f1b9dfff700 1 -- 172.21.15.133:0/2368715882 shutdown_connections 2023-11-27T15:28:57.944 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.939+0000 7f1b9dfff700 1 -- 172.21.15.133:0/2368715882 wait complete. 2023-11-27T15:28:57.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.940+0000 7f1b9dfff700 1 Processor -- start 2023-11-27T15:28:57.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.940+0000 7f1b9dfff700 1 -- start start 2023-11-27T15:28:57.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.941+0000 7f1b9dfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b98100290 0x7f1b98198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:57.946 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.941+0000 7f1b9dfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b98101480 0x7f1b981987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:57.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.941+0000 7f1b9dfff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b98101e10 0x7f1b9819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:57.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.941+0000 7f1b9dfff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9819fe10 con 0x7f1b98101e10 2023-11-27T15:28:57.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.941+0000 7f1b9dfff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9819ff50 con 0x7f1b98101480 2023-11-27T15:28:57.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.941+0000 7f1b9dfff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b981a00e0 con 0x7f1b98100290 2023-11-27T15:28:57.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.941+0000 7f1b97fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b98101e10 0x7f1b9819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:57.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.941+0000 7f1b96ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b98101480 0x7f1b981987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:57.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.941+0000 7f1b96ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b98101480 0x7f1b981987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:51128/0 (socket says 172.21.15.133:51128) 2023-11-27T15:28:57.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.941+0000 7f1b977fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b98100290 0x7f1b98198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:57.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.941+0000 7f1b96ffd700 1 -- 172.21.15.133:0/2544083123 learned_addr learned my addr 172.21.15.133:0/2544083123 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:28:57.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.941+0000 7f1b97fff700 1 -- 172.21.15.133:0/2544083123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b98100290 msgr2=0x7f1b98198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:57.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.941+0000 7f1b97fff700 1 --2- 172.21.15.133:0/2544083123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b98100290 0x7f1b98198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.941+0000 7f1b97fff700 1 -- 172.21.15.133:0/2544083123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b98101480 msgr2=0x7f1b981987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:28:57.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.941+0000 7f1b97fff700 1 --2- 172.21.15.133:0/2544083123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b98101480 0x7f1b981987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:28:57.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.941+0000 7f1b97fff700 1 -- 172.21.15.133:0/2544083123 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b90009d00 con 0x7f1b98101e10 2023-11-27T15:28:57.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.942+0000 7f1b97fff700 1 --2- 172.21.15.133:0/2544083123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b98101e10 0x7f1b9819f840 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000b070 tx=0x7f1b9000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:57.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.942+0000 7f1b94ff9700 1 -- 172.21.15.133:0/2544083123 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9000e040 con 0x7f1b98101e10 2023-11-27T15:28:57.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.942+0000 7f1b94ff9700 1 -- 172.21.15.133:0/2544083123 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b90003e10 con 0x7f1b98101e10 2023-11-27T15:28:57.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.942+0000 7f1b94ff9700 1 -- 172.21.15.133:0/2544083123 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b900052a0 con 0x7f1b98101e10 2023-11-27T15:28:57.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.942+0000 7f1b9dfff700 1 -- 172.21.15.133:0/2544083123 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b9810bc70 con 0x7f1b98101e10 2023-11-27T15:28:57.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.942+0000 7f1b9dfff700 1 -- 172.21.15.133:0/2544083123 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9810bae0 con 0x7f1b98101e10 2023-11-27T15:28:57.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.943+0000 7f1b9dfff700 1 -- 172.21.15.133:0/2544083123 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b981a0600 con 0x7f1b98101e10 2023-11-27T15:28:57.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.945+0000 7f1b94ff9700 1 -- 172.21.15.133:0/2544083123 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f1b9001d480 con 0x7f1b98101e10 2023-11-27T15:28:57.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.946+0000 7f1b94ff9700 1 --2- 172.21.15.133:0/2544083123 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1b800a1150 0x7f1b800a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:28:57.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.947+0000 7f1b94ff9700 1 -- 172.21.15.133:0/2544083123 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f1b900ce0d0 con 0x7f1b98101e10 2023-11-27T15:28:57.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.947+0000 7f1b977fe700 1 --2- 172.21.15.133:0/2544083123 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1b800a1150 0x7f1b800a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:28:57.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.947+0000 7f1b977fe700 1 --2- 172.21.15.133:0/2544083123 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1b800a1150 0x7f1b800a3600 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1b981012e0 tx=0x7f1b8800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:28:57.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:57.950+0000 7f1b94ff9700 1 -- 172.21.15.133:0/2544083123 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1b90097500 con 0x7f1b98101e10 2023-11-27T15:28:58.026 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:57 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/46596916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:28:58.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:57 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1477480169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:28:58.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:58.208+0000 7f1b9dfff700 1 -- 172.21.15.133:0/2544083123 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f1b981a0ca0 con 0x7f1b98101e10 2023-11-27T15:28:58.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:58.219+0000 7f1b94ff9700 1 -- 172.21.15.133:0/2544083123 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+733249 (secure 0 0 0) 0x7f1b9009ab20 con 0x7f1b98101e10 2023-11-27T15:28:58.232 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:28:58.232 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:28:58.232 INFO:teuthology.orchestra.run.smithi133.stdout: "epoch": 18, 2023-11-27T15:28:58.232 INFO:teuthology.orchestra.run.smithi133.stdout: "active_gid": 14150, 2023-11-27T15:28:58.232 INFO:teuthology.orchestra.run.smithi133.stdout: "active_name": "x", 2023-11-27T15:28:58.233 INFO:teuthology.orchestra.run.smithi133.stdout: "active_addrs": { 2023-11-27T15:28:58.233 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2023-11-27T15:28:58.233 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.233 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2023-11-27T15:28:58.233 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:6800", 2023-11-27T15:28:58.233 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 4209056668 2023-11-27T15:28:58.233 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.234 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.234 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v1", 2023-11-27T15:28:58.234 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:6801", 2023-11-27T15:28:58.234 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 4209056668 2023-11-27T15:28:58.234 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.234 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-11-27T15:28:58.234 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.234 INFO:teuthology.orchestra.run.smithi133.stdout: "active_addr": "172.21.15.133:6801/4209056668", 2023-11-27T15:28:58.234 INFO:teuthology.orchestra.run.smithi133.stdout: "active_change": "2023-11-27T15:14:01.628299+0000", 2023-11-27T15:28:58.235 INFO:teuthology.orchestra.run.smithi133.stdout: "active_mgr_features": 4540138322906710015, 2023-11-27T15:28:58.235 INFO:teuthology.orchestra.run.smithi133.stdout: "available": true, 2023-11-27T15:28:58.235 INFO:teuthology.orchestra.run.smithi133.stdout: "standbys": [ 2023-11-27T15:28:58.235 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.235 INFO:teuthology.orchestra.run.smithi133.stdout: "gid": 24109, 2023-11-27T15:28:58.235 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "z", 2023-11-27T15:28:58.235 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr_features": 4540138322906710015, 2023-11-27T15:28:58.235 INFO:teuthology.orchestra.run.smithi133.stdout: "available_modules": [ 2023-11-27T15:28:58.236 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.236 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alerts", 2023-11-27T15:28:58.236 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.236 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.236 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.236 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-11-27T15:28:58.236 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-11-27T15:28:58.236 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.237 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.237 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.237 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-11-27T15:28:58.237 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.237 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.237 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.237 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How frequently to reexamine health status", 2023-11-27T15:28:58.237 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.238 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.239 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.239 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.239 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.239 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.239 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.239 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.239 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.239 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.239 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.240 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.240 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.240 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.240 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.240 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.240 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.240 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.240 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.241 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.241 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.241 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.241 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.241 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.241 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.241 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.241 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.242 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.242 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.242 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.242 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.242 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.242 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.242 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.243 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.243 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.243 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.243 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.243 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.243 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.243 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.243 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.243 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.244 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.244 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.244 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.244 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.244 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.244 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.244 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.244 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.245 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.245 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.245 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.245 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.245 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.245 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.245 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.246 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.246 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.246 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.246 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.246 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.246 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.246 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.246 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.247 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.247 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_destination": { 2023-11-27T15:28:58.247 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_destination", 2023-11-27T15:28:58.247 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.247 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.247 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.247 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.247 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.248 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.248 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.248 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email address to send alerts to", 2023-11-27T15:28:58.248 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.248 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.248 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.248 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.248 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_from_name": { 2023-11-27T15:28:58.249 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_from_name", 2023-11-27T15:28:58.249 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.249 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.249 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.249 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "Ceph", 2023-11-27T15:28:58.249 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.249 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.249 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.250 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email From: name", 2023-11-27T15:28:58.250 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.250 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.250 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.250 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.250 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_host": { 2023-11-27T15:28:58.250 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_host", 2023-11-27T15:28:58.250 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.251 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.251 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.251 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.251 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.251 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.251 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.251 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP server", 2023-11-27T15:28:58.251 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.252 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.252 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.252 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.252 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_password": { 2023-11-27T15:28:58.252 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_password", 2023-11-27T15:28:58.252 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.252 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.252 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.253 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.253 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.253 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.253 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.253 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Password to authenticate with", 2023-11-27T15:28:58.253 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.253 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.253 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.254 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.254 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_port": { 2023-11-27T15:28:58.254 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_port", 2023-11-27T15:28:58.254 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.254 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.254 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.254 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "465", 2023-11-27T15:28:58.254 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.255 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.255 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.255 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP port", 2023-11-27T15:28:58.255 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.255 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.255 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.255 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.255 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_sender": { 2023-11-27T15:28:58.256 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_sender", 2023-11-27T15:28:58.256 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.256 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.256 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.256 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.256 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.256 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.256 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.257 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP envelope sender", 2023-11-27T15:28:58.257 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.257 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.257 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.257 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.257 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_ssl": { 2023-11-27T15:28:58.257 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_ssl", 2023-11-27T15:28:58.258 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.258 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.258 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.258 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.258 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.258 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.258 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.259 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-27T15:28:58.259 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.259 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.259 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.259 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.259 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_user": { 2023-11-27T15:28:58.259 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_user", 2023-11-27T15:28:58.259 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.260 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.260 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.260 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.260 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.260 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.260 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.260 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "User to authenticate as", 2023-11-27T15:28:58.260 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.260 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.261 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.261 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.261 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.261 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.261 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.261 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "balancer", 2023-11-27T15:28:58.261 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.261 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.261 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.262 INFO:teuthology.orchestra.run.smithi133.stdout: "active": { 2023-11-27T15:28:58.262 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "active", 2023-11-27T15:28:58.262 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.262 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.262 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.262 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.262 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.262 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.263 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.263 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically balance PGs across cluster", 2023-11-27T15:28:58.263 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.263 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.263 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.263 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.263 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_time": { 2023-11-27T15:28:58.263 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_time", 2023-11-27T15:28:58.263 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.264 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.264 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.264 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0000", 2023-11-27T15:28:58.264 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.264 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.264 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.264 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "beginning time of day to automatically balance", 2023-11-27T15:28:58.264 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-27T15:28:58.265 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.265 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.265 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.265 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_weekday": { 2023-11-27T15:28:58.265 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_weekday", 2023-11-27T15:28:58.265 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-11-27T15:28:58.265 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.265 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.265 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-11-27T15:28:58.266 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2023-11-27T15:28:58.266 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2023-11-27T15:28:58.266 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.266 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-27T15:28:58.266 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-27T15:28:58.266 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.266 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.266 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.267 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_max_iterations": { 2023-11-27T15:28:58.267 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_max_iterations", 2023-11-27T15:28:58.267 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-11-27T15:28:58.267 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.267 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.267 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "25", 2023-11-27T15:28:58.267 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-11-27T15:28:58.267 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "250", 2023-11-27T15:28:58.267 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.268 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-27T15:28:58.268 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.268 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.268 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.268 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.268 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_metrics": { 2023-11-27T15:28:58.268 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_metrics", 2023-11-27T15:28:58.268 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.269 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.269 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.269 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "pgs,objects,bytes", 2023-11-27T15:28:58.269 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.269 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.269 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.269 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-27T15:28:58.269 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-27T15:28:58.270 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.270 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.270 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.270 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_step": { 2023-11-27T15:28:58.270 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_step", 2023-11-27T15:28:58.270 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:28:58.270 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.270 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.271 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.5", 2023-11-27T15:28:58.271 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0.001", 2023-11-27T15:28:58.271 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "0.999", 2023-11-27T15:28:58.271 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.271 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "aggressiveness of optimization", 2023-11-27T15:28:58.271 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-27T15:28:58.271 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.271 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.271 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.272 INFO:teuthology.orchestra.run.smithi133.stdout: "end_time": { 2023-11-27T15:28:58.272 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_time", 2023-11-27T15:28:58.272 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.272 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.272 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.272 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2359", 2023-11-27T15:28:58.272 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.272 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.273 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.273 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ending time of day to automatically balance", 2023-11-27T15:28:58.273 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-27T15:28:58.273 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.273 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.273 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.273 INFO:teuthology.orchestra.run.smithi133.stdout: "end_weekday": { 2023-11-27T15:28:58.273 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_weekday", 2023-11-27T15:28:58.274 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-11-27T15:28:58.274 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.274 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.274 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-11-27T15:28:58.274 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2023-11-27T15:28:58.274 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2023-11-27T15:28:58.274 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.274 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-27T15:28:58.274 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-27T15:28:58.275 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.275 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.275 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.275 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.275 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.275 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.275 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.275 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.276 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.276 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.276 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.276 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.276 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.276 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.276 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.276 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.277 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.277 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.277 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.277 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.277 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.277 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.277 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.278 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.278 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.278 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.278 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.278 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.278 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.278 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.278 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.279 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.279 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.279 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.279 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.279 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.279 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.279 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.279 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.280 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.280 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.280 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.280 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.280 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.280 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.280 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.280 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.281 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.281 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.281 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.281 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.281 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.281 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.281 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.281 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.282 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.282 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.282 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.282 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.282 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.282 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.282 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.282 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.283 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.283 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.283 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.283 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.283 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.283 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.283 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.283 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.284 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.284 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.284 INFO:teuthology.orchestra.run.smithi133.stdout: "min_score": { 2023-11-27T15:28:58.284 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_score", 2023-11-27T15:28:58.284 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:28:58.284 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.284 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.284 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-11-27T15:28:58.285 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.285 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.285 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.285 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-27T15:28:58.285 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.285 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.285 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.285 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.286 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2023-11-27T15:28:58.286 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2023-11-27T15:28:58.286 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.286 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.286 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.286 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "upmap", 2023-11-27T15:28:58.286 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.287 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.287 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.287 INFO:teuthology.orchestra.run.smithi133.stdout: "crush-compat", 2023-11-27T15:28:58.287 INFO:teuthology.orchestra.run.smithi133.stdout: "none", 2023-11-27T15:28:58.287 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap" 2023-11-27T15:28:58.287 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.287 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Balancer mode", 2023-11-27T15:28:58.287 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.288 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.288 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.288 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.288 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_ids": { 2023-11-27T15:28:58.288 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_ids", 2023-11-27T15:28:58.288 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.288 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.288 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.288 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.289 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.289 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.289 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.289 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-27T15:28:58.289 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.289 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.289 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.289 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.290 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-11-27T15:28:58.290 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-11-27T15:28:58.290 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.290 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.290 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.290 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-11-27T15:28:58.290 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.290 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.291 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.291 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-27T15:28:58.291 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.291 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.291 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.291 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.291 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_deviation": { 2023-11-27T15:28:58.291 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_deviation", 2023-11-27T15:28:58.291 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.292 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.292 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.292 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-11-27T15:28:58.292 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-11-27T15:28:58.292 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.292 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.292 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-27T15:28:58.292 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-27T15:28:58.293 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.293 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.293 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.293 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_optimizations": { 2023-11-27T15:28:58.293 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_optimizations", 2023-11-27T15:28:58.293 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-11-27T15:28:58.293 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.293 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.293 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:28:58.294 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.294 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.294 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.294 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-27T15:28:58.294 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.294 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.294 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.295 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.295 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.295 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.295 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.295 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephadm", 2023-11-27T15:28:58.295 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.295 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.295 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.296 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_down_multiplier": { 2023-11-27T15:28:58.296 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_down_multiplier", 2023-11-27T15:28:58.296 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:28:58.296 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.296 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.296 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2023-11-27T15:28:58.296 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.296 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.297 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.297 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-27T15:28:58.297 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.297 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.297 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.297 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.297 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_refresh_rate": { 2023-11-27T15:28:58.297 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_refresh_rate", 2023-11-27T15:28:58.297 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.298 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.298 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.298 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "20", 2023-11-27T15:28:58.298 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.298 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.298 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.298 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-27T15:28:58.298 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.299 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.299 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.299 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.299 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_starting_port": { 2023-11-27T15:28:58.299 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_starting_port", 2023-11-27T15:28:58.299 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.299 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.299 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.299 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4721", 2023-11-27T15:28:58.300 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.300 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.300 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.300 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-27T15:28:58.300 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.300 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.300 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.300 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.301 INFO:teuthology.orchestra.run.smithi133.stdout: "alertmanager_web_password": { 2023-11-27T15:28:58.301 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alertmanager_web_password", 2023-11-27T15:28:58.301 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.301 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.301 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.301 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-11-27T15:28:58.301 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.301 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.301 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.302 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Alertmanager web password", 2023-11-27T15:28:58.302 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.302 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.302 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.302 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.302 INFO:teuthology.orchestra.run.smithi133.stdout: "alertmanager_web_user": { 2023-11-27T15:28:58.302 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alertmanager_web_user", 2023-11-27T15:28:58.302 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.302 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.303 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.303 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-11-27T15:28:58.303 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.303 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.303 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.303 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Alertmanager web user", 2023-11-27T15:28:58.303 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.303 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.303 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.304 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.304 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_ptrace": { 2023-11-27T15:28:58.304 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_ptrace", 2023-11-27T15:28:58.304 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.304 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.304 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.304 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.304 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.305 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.305 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.305 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-27T15:28:58.305 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-27T15:28:58.305 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.305 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.305 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.305 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_interval": { 2023-11-27T15:28:58.306 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_interval", 2023-11-27T15:28:58.306 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.306 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.306 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.306 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-11-27T15:28:58.306 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.306 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.306 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.307 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-27T15:28:58.307 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.307 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.307 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.307 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.307 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_memory_target_ratio": { 2023-11-27T15:28:58.307 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_memory_target_ratio", 2023-11-27T15:28:58.307 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:28:58.308 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.308 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.308 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.7", 2023-11-27T15:28:58.308 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.308 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.308 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.308 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-27T15:28:58.308 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.309 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.309 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.309 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.309 INFO:teuthology.orchestra.run.smithi133.stdout: "cgroups_split": { 2023-11-27T15:28:58.309 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cgroups_split", 2023-11-27T15:28:58.309 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.309 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.309 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.310 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.310 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.310 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.310 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.310 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-27T15:28:58.310 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.310 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.310 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.311 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.311 INFO:teuthology.orchestra.run.smithi133.stdout: "config_checks_enabled": { 2023-11-27T15:28:58.311 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_checks_enabled", 2023-11-27T15:28:58.311 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.311 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.311 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.311 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.311 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.312 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.312 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.312 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-27T15:28:58.312 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.312 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.312 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.312 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.312 INFO:teuthology.orchestra.run.smithi133.stdout: "config_dashboard": { 2023-11-27T15:28:58.313 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_dashboard", 2023-11-27T15:28:58.313 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.313 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.313 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.313 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.313 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.313 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.313 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.313 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-27T15:28:58.314 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.314 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.314 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.314 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.314 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_alertmanager": { 2023-11-27T15:28:58.314 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_alertmanager", 2023-11-27T15:28:58.314 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.314 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.315 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.315 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-27T15:28:58.315 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.315 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.315 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.315 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:58.315 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.315 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.316 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.316 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.316 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_base": { 2023-11-27T15:28:58.316 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_base", 2023-11-27T15:28:58.316 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.316 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.316 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.316 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-27T15:28:58.317 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.317 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.317 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.317 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Container image name, without the tag", 2023-11-27T15:28:58.317 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.317 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.317 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.317 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.318 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_elasticsearch": { 2023-11-27T15:28:58.318 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_elasticsearch", 2023-11-27T15:28:58.318 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.318 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.318 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.318 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-27T15:28:58.318 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.318 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.319 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.319 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "elasticsearch container image", 2023-11-27T15:28:58.319 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.319 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.319 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.319 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.319 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_grafana": { 2023-11-27T15:28:58.319 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_grafana", 2023-11-27T15:28:58.320 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.320 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.320 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.320 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-27T15:28:58.320 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.320 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.320 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.320 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:58.321 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.321 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.321 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.321 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.321 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_haproxy": { 2023-11-27T15:28:58.321 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_haproxy", 2023-11-27T15:28:58.321 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.322 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.322 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.322 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-27T15:28:58.322 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.322 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.322 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.322 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "HAproxy container image", 2023-11-27T15:28:58.322 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.323 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.323 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.323 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.323 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_agent": { 2023-11-27T15:28:58.323 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_agent", 2023-11-27T15:28:58.323 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.323 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.323 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.323 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-27T15:28:58.324 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.324 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.324 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.324 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger agent container image", 2023-11-27T15:28:58.324 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.324 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.324 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.324 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.325 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_collector": { 2023-11-27T15:28:58.325 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_collector", 2023-11-27T15:28:58.325 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.325 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.325 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.325 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-27T15:28:58.325 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.326 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.326 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.326 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger collector container image", 2023-11-27T15:28:58.326 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.326 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.326 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.326 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.326 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_query": { 2023-11-27T15:28:58.327 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_query", 2023-11-27T15:28:58.327 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.327 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.327 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.327 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-27T15:28:58.327 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.327 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.328 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.328 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger query container image", 2023-11-27T15:28:58.328 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.328 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.328 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.328 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.328 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_keepalived": { 2023-11-27T15:28:58.328 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_keepalived", 2023-11-27T15:28:58.329 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.329 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.329 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.329 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-27T15:28:58.329 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.329 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.329 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.329 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Keepalived container image", 2023-11-27T15:28:58.330 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.330 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.330 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.330 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.330 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_loki": { 2023-11-27T15:28:58.330 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_loki", 2023-11-27T15:28:58.330 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.330 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.331 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.331 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-27T15:28:58.331 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.331 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.331 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.331 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Loki container image", 2023-11-27T15:28:58.331 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.331 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.332 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.332 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.332 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_node_exporter": { 2023-11-27T15:28:58.332 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_node_exporter", 2023-11-27T15:28:58.332 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.332 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.332 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.332 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-27T15:28:58.333 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.333 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.333 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.333 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:58.333 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.333 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.333 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.333 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.334 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_prometheus": { 2023-11-27T15:28:58.334 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_prometheus", 2023-11-27T15:28:58.334 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.334 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.334 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.334 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-27T15:28:58.334 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.335 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.335 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.335 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:58.335 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.335 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.335 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.335 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.335 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_promtail": { 2023-11-27T15:28:58.336 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_promtail", 2023-11-27T15:28:58.336 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.336 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.336 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.336 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-27T15:28:58.336 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.336 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.336 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.337 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Promtail container image", 2023-11-27T15:28:58.337 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.337 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.337 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.337 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.337 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_snmp_gateway": { 2023-11-27T15:28:58.337 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_snmp_gateway", 2023-11-27T15:28:58.337 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.338 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.338 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.338 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-27T15:28:58.338 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.338 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.338 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.338 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SNMP Gateway container image", 2023-11-27T15:28:58.338 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.339 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.339 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.339 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.339 INFO:teuthology.orchestra.run.smithi133.stdout: "container_init": { 2023-11-27T15:28:58.339 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_init", 2023-11-27T15:28:58.339 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.339 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.339 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.340 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.340 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.340 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.340 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.340 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Run podman/docker with `--init`", 2023-11-27T15:28:58.340 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.340 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.340 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.341 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.341 INFO:teuthology.orchestra.run.smithi133.stdout: "daemon_cache_timeout": { 2023-11-27T15:28:58.341 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "daemon_cache_timeout", 2023-11-27T15:28:58.341 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.341 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.341 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.341 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-11-27T15:28:58.341 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.342 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.342 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.342 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-27T15:28:58.342 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.342 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.342 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.342 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.343 INFO:teuthology.orchestra.run.smithi133.stdout: "default_cephadm_command_timeout": { 2023-11-27T15:28:58.343 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_cephadm_command_timeout", 2023-11-27T15:28:58.343 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.343 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.343 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.343 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "900", 2023-11-27T15:28:58.343 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.343 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.344 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.344 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-27T15:28:58.344 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.344 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.344 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.344 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.344 INFO:teuthology.orchestra.run.smithi133.stdout: "default_registry": { 2023-11-27T15:28:58.344 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_registry", 2023-11-27T15:28:58.345 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.345 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.345 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.345 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io", 2023-11-27T15:28:58.345 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.345 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.345 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.345 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-27T15:28:58.346 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.346 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.346 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.346 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.346 INFO:teuthology.orchestra.run.smithi133.stdout: "device_cache_timeout": { 2023-11-27T15:28:58.346 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_cache_timeout", 2023-11-27T15:28:58.346 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.346 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.347 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.347 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1800", 2023-11-27T15:28:58.347 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.347 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.347 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.347 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache device inventory", 2023-11-27T15:28:58.347 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.348 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.348 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.348 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.348 INFO:teuthology.orchestra.run.smithi133.stdout: "device_enhanced_scan": { 2023-11-27T15:28:58.348 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_enhanced_scan", 2023-11-27T15:28:58.348 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.348 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.348 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.349 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.349 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.349 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.349 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.349 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-27T15:28:58.349 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.349 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.349 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.350 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.350 INFO:teuthology.orchestra.run.smithi133.stdout: "facts_cache_timeout": { 2023-11-27T15:28:58.350 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "facts_cache_timeout", 2023-11-27T15:28:58.350 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.350 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.350 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.350 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-11-27T15:28:58.350 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.351 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.351 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.351 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache host facts data", 2023-11-27T15:28:58.351 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.351 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.351 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.351 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.351 INFO:teuthology.orchestra.run.smithi133.stdout: "host_check_interval": { 2023-11-27T15:28:58.352 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "host_check_interval", 2023-11-27T15:28:58.352 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.352 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.352 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.352 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-11-27T15:28:58.352 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.352 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.352 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.353 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to perform a host check", 2023-11-27T15:28:58.353 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.353 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.353 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.353 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.353 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.353 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.353 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.354 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.354 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.354 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.354 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.354 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.354 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.354 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.354 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.355 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.355 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.355 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.355 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.355 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.355 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.355 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.355 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.355 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.356 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.356 INFO:teuthology.orchestra.run.smithi133.stdout: "log_refresh_metadata": { 2023-11-27T15:28:58.356 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_refresh_metadata", 2023-11-27T15:28:58.356 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.356 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.356 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.356 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.356 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.357 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.357 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.357 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-27T15:28:58.357 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.357 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.357 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.357 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.358 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.358 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.358 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.358 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.358 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.358 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.358 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.358 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.359 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.359 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-27T15:28:58.359 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.359 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.359 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.359 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.359 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.359 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.360 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.360 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.360 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.360 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.360 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.360 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.360 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.360 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.361 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.361 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.361 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.361 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.361 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.361 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.361 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.361 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.362 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.362 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.362 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.362 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.362 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.362 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.362 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.362 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.363 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.363 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.363 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.363 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.363 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.363 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.363 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.363 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.364 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.364 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-27T15:28:58.364 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-27T15:28:58.364 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.364 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.364 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.364 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.364 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.365 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.365 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.365 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-27T15:28:58.365 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.365 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.365 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.365 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.366 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-27T15:28:58.366 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-27T15:28:58.366 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.366 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.366 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.366 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "*", 2023-11-27T15:28:58.366 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.366 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.367 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.367 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-27T15:28:58.367 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.367 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.367 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.367 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.367 INFO:teuthology.orchestra.run.smithi133.stdout: "max_count_per_host": { 2023-11-27T15:28:58.367 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_count_per_host", 2023-11-27T15:28:58.368 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.368 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.368 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.368 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:28:58.368 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.368 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.368 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.368 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of daemons per service per host", 2023-11-27T15:28:58.369 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.369 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.369 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.369 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.369 INFO:teuthology.orchestra.run.smithi133.stdout: "max_osd_draining_count": { 2023-11-27T15:28:58.369 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_osd_draining_count", 2023-11-27T15:28:58.369 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.369 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.370 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.370 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:28:58.370 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.370 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.370 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.370 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-27T15:28:58.370 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.370 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.371 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.371 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.371 INFO:teuthology.orchestra.run.smithi133.stdout: "migration_current": { 2023-11-27T15:28:58.371 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "migration_current", 2023-11-27T15:28:58.371 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.371 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.371 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.371 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.372 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.372 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.372 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.372 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "internal - do not modify", 2023-11-27T15:28:58.372 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.372 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.372 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.372 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.373 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2023-11-27T15:28:58.373 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2023-11-27T15:28:58.373 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.373 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.373 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.373 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "root", 2023-11-27T15:28:58.373 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.373 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.374 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.374 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm-package", 2023-11-27T15:28:58.374 INFO:teuthology.orchestra.run.smithi133.stdout: "root" 2023-11-27T15:28:58.374 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.374 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "mode for remote execution of cephadm", 2023-11-27T15:28:58.374 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.374 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.374 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.375 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.375 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_alerts_path": { 2023-11-27T15:28:58.375 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_alerts_path", 2023-11-27T15:28:58.375 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.375 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.375 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.375 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-27T15:28:58.375 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.376 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.376 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.376 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-27T15:28:58.376 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.376 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.376 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.376 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.376 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_web_password": { 2023-11-27T15:28:58.377 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_web_password", 2023-11-27T15:28:58.377 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.377 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.377 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.377 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-11-27T15:28:58.377 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.377 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.378 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.378 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus web password", 2023-11-27T15:28:58.378 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.378 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.378 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.378 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.378 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_web_user": { 2023-11-27T15:28:58.378 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_web_user", 2023-11-27T15:28:58.379 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.379 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.379 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.379 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-11-27T15:28:58.379 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.379 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.379 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.379 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus web user", 2023-11-27T15:28:58.380 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.380 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.380 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.380 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.380 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_insecure": { 2023-11-27T15:28:58.380 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_insecure", 2023-11-27T15:28:58.380 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.380 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.381 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.381 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.381 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.381 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.381 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.381 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-27T15:28:58.381 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.381 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.382 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.382 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.382 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_password": { 2023-11-27T15:28:58.382 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_password", 2023-11-27T15:28:58.382 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.382 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.382 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.382 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.383 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.383 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.383 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.383 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-27T15:28:58.383 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.383 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.383 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.383 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.384 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_url": { 2023-11-27T15:28:58.384 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_url", 2023-11-27T15:28:58.384 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.384 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.384 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.384 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.384 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.384 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.385 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.385 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-27T15:28:58.385 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.385 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.385 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.385 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.385 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_username": { 2023-11-27T15:28:58.385 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_username", 2023-11-27T15:28:58.386 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.386 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.386 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.386 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.386 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.386 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.386 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.386 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-27T15:28:58.387 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.387 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.387 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.387 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.387 INFO:teuthology.orchestra.run.smithi133.stdout: "secure_monitoring_stack": { 2023-11-27T15:28:58.387 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "secure_monitoring_stack", 2023-11-27T15:28:58.387 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.388 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.388 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.388 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.388 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.388 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.388 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.388 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-27T15:28:58.388 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.389 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.389 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.389 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.389 INFO:teuthology.orchestra.run.smithi133.stdout: "service_discovery_port": { 2023-11-27T15:28:58.389 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "service_discovery_port", 2023-11-27T15:28:58.389 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.389 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.389 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.390 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8765", 2023-11-27T15:28:58.390 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.390 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.390 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.390 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "cephadm service discovery port", 2023-11-27T15:28:58.390 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.390 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.390 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.391 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.391 INFO:teuthology.orchestra.run.smithi133.stdout: "ssh_config_file": { 2023-11-27T15:28:58.391 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssh_config_file", 2023-11-27T15:28:58.391 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.391 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.391 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.391 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.391 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.392 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.392 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.392 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-27T15:28:58.392 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.392 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.392 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.392 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.392 INFO:teuthology.orchestra.run.smithi133.stdout: "use_agent": { 2023-11-27T15:28:58.393 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_agent", 2023-11-27T15:28:58.393 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.393 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.393 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.393 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.393 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.393 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.393 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.394 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-27T15:28:58.394 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.394 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.394 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.394 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.394 INFO:teuthology.orchestra.run.smithi133.stdout: "use_repo_digest": { 2023-11-27T15:28:58.394 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_repo_digest", 2023-11-27T15:28:58.395 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.395 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.395 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.395 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.395 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.397 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.397 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.397 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-27T15:28:58.397 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.397 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.397 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.397 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.398 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_failed_host_check": { 2023-11-27T15:28:58.398 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_failed_host_check", 2023-11-27T15:28:58.398 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.398 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.398 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.398 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.398 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.399 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.399 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.399 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if the host check fails", 2023-11-27T15:28:58.399 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.399 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.399 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.399 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.399 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_daemons": { 2023-11-27T15:28:58.400 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_daemons", 2023-11-27T15:28:58.400 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.400 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.400 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.400 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.400 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.400 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.400 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.401 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-27T15:28:58.401 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.401 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.401 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.401 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.401 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_hosts": { 2023-11-27T15:28:58.401 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_hosts", 2023-11-27T15:28:58.401 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.402 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.402 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.402 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.402 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.402 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.402 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.402 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-27T15:28:58.402 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.403 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.403 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.403 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.403 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.403 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.403 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.403 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crash", 2023-11-27T15:28:58.403 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.404 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.404 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.404 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.404 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.404 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.404 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.404 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.404 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.404 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.405 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.405 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.405 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.405 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.405 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.405 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.405 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.405 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.406 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.406 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.406 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.406 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.406 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.406 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.406 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.406 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.407 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.407 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.407 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.407 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.407 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.407 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.407 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.408 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.408 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.408 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.408 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.408 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.408 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.408 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.408 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.409 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.409 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.409 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.409 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.409 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.409 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.409 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.409 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.410 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.410 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.410 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.410 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.410 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.410 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.410 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.410 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.411 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.411 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.411 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.411 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.411 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.411 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.411 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.411 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.412 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.412 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.412 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.412 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.412 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.412 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.412 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.412 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.413 INFO:teuthology.orchestra.run.smithi133.stdout: "retain_interval": { 2023-11-27T15:28:58.413 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retain_interval", 2023-11-27T15:28:58.413 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.413 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.413 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.413 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "31536000", 2023-11-27T15:28:58.413 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.413 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.414 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.414 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-27T15:28:58.414 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.414 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.414 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.414 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.414 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_recent_interval": { 2023-11-27T15:28:58.415 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_recent_interval", 2023-11-27T15:28:58.415 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.415 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.415 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.415 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1209600", 2023-11-27T15:28:58.415 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.415 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.415 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.415 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-27T15:28:58.416 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.416 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.416 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.416 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.416 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.416 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.416 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.416 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dashboard", 2023-11-27T15:28:58.417 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.417 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.417 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.417 INFO:teuthology.orchestra.run.smithi133.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-27T15:28:58.417 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-27T15:28:58.417 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.417 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.418 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.418 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:28:58.418 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.418 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.418 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.418 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.418 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.418 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.419 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.419 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.419 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_HOST": { 2023-11-27T15:28:58.419 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-27T15:28:58.419 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.419 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.419 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.419 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.420 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.420 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.420 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.420 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.420 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.420 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.420 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.421 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.421 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-27T15:28:58.421 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-27T15:28:58.421 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.421 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.421 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.421 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.421 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.422 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.422 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.422 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.422 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.422 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.422 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.422 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.422 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_ENABLED": { 2023-11-27T15:28:58.423 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_ENABLED", 2023-11-27T15:28:58.423 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.423 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.423 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.423 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.423 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.423 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.423 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.424 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.424 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.424 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.424 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.424 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.424 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-27T15:28:58.424 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-27T15:28:58.424 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.425 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.425 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.425 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.425 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.425 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.425 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.425 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.425 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.425 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.426 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.426 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.426 INFO:teuthology.orchestra.run.smithi133.stdout: "ENABLE_BROWSABLE_API": { 2023-11-27T15:28:58.426 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-27T15:28:58.426 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.426 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.426 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.427 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.427 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.427 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.427 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.427 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.427 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.427 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.427 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.428 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.428 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-27T15:28:58.428 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-27T15:28:58.428 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.428 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.428 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.428 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.428 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.429 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.429 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.429 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.429 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.429 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.429 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.429 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.429 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-27T15:28:58.430 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-27T15:28:58.430 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.430 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.430 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.430 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.430 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.430 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.431 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.431 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.431 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.431 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.431 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.431 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.431 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-27T15:28:58.431 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-27T15:28:58.432 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.432 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.432 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.432 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.432 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.432 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.432 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.433 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.433 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.433 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.433 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.433 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.433 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-27T15:28:58.433 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-27T15:28:58.433 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.434 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.434 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.434 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.434 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.434 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.434 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.434 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.435 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.435 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.435 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.435 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.435 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-27T15:28:58.435 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-27T15:28:58.435 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.435 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.436 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.436 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.436 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.436 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.436 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.436 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.436 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.436 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.437 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.437 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.437 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-27T15:28:58.437 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-27T15:28:58.437 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.437 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.437 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.437 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.438 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.438 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.438 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.438 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.438 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.438 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.438 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.438 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.439 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-27T15:28:58.439 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-27T15:28:58.439 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.439 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.439 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.439 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.439 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.439 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.440 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.440 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.440 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.440 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.440 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.440 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.440 INFO:teuthology.orchestra.run.smithi133.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-27T15:28:58.441 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-27T15:28:58.441 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.441 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.441 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.441 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.441 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.441 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.441 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.442 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.442 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.442 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.442 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.442 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.442 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_PASSWORD": { 2023-11-27T15:28:58.442 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-27T15:28:58.442 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.443 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.443 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.443 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-11-27T15:28:58.443 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.443 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.443 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.443 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.443 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.444 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.444 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.444 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.444 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-27T15:28:58.444 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-27T15:28:58.444 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.444 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.444 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.445 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.445 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.445 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.445 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.445 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.445 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.445 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.445 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.446 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.446 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_URL": { 2023-11-27T15:28:58.446 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_URL", 2023-11-27T15:28:58.446 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.446 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.446 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.446 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.446 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.447 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.447 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.447 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.447 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.447 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.447 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.447 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.447 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_USERNAME": { 2023-11-27T15:28:58.448 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-27T15:28:58.448 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.448 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.448 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.448 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-11-27T15:28:58.448 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.448 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.448 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.449 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.449 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.449 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.449 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.449 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.449 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-27T15:28:58.449 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-27T15:28:58.449 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.450 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.450 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.450 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.450 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.450 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.450 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.450 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.450 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.451 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.451 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.451 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.451 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-27T15:28:58.451 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-27T15:28:58.451 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.451 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.452 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.452 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.452 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.452 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.452 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.452 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.452 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.452 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.452 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.453 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.453 INFO:teuthology.orchestra.run.smithi133.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-27T15:28:58.453 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-27T15:28:58.453 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.453 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.453 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.453 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.453 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.454 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.454 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.454 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.454 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.454 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.454 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.454 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.454 INFO:teuthology.orchestra.run.smithi133.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-27T15:28:58.455 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-27T15:28:58.455 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.455 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.455 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.455 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.455 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.455 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.455 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.456 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.456 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.456 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.456 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.456 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.456 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_HOST": { 2023-11-27T15:28:58.456 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-27T15:28:58.456 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.457 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.457 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.457 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.457 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.457 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.457 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.457 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.457 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.458 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.458 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.458 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.458 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-27T15:28:58.458 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-27T15:28:58.458 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.458 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.459 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.459 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.459 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.459 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.459 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.459 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.459 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.459 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.460 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.460 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.460 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-27T15:28:58.460 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-27T15:28:58.460 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.460 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.460 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.460 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.461 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.461 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.461 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.461 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.461 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.461 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.461 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.461 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.462 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-27T15:28:58.462 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-27T15:28:58.462 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.462 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.462 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.462 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.462 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.462 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.463 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.463 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.463 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.463 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.463 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.463 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.463 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-27T15:28:58.463 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-27T15:28:58.464 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.464 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.464 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.464 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.464 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.464 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.464 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.465 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.465 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.465 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.465 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.465 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.465 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-27T15:28:58.465 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-27T15:28:58.465 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.466 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.466 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.466 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.466 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.466 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.466 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.466 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.466 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.467 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.467 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.467 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.467 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-27T15:28:58.467 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-27T15:28:58.467 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.467 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.468 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.468 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.468 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.468 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.468 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.468 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.468 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.468 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.469 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.469 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.469 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-27T15:28:58.469 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-27T15:28:58.469 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.469 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.469 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.469 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.470 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.470 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.470 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.470 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.470 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.470 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.470 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.470 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.471 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-27T15:28:58.471 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-27T15:28:58.471 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.471 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.471 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.471 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.471 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.471 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.472 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.472 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.472 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.472 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.472 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.472 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.472 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_ENABLED": { 2023-11-27T15:28:58.472 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-27T15:28:58.473 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.473 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.473 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.473 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.473 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.473 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.473 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.473 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.474 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.474 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.474 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.474 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.474 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-27T15:28:58.474 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-27T15:28:58.474 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.474 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.475 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.475 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-27T15:28:58.475 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.475 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.475 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.475 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.475 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.475 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.476 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.476 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.476 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-27T15:28:58.476 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-27T15:28:58.476 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.476 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.476 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.476 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:28:58.477 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.477 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.477 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.477 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.477 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.477 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.477 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.477 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.478 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-27T15:28:58.478 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-27T15:28:58.478 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.478 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.478 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.478 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8", 2023-11-27T15:28:58.478 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.478 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.479 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.479 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.479 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.479 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.479 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.479 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.479 INFO:teuthology.orchestra.run.smithi133.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-27T15:28:58.479 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-27T15:28:58.480 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.480 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.480 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.480 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "45", 2023-11-27T15:28:58.480 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.480 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.480 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.480 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.481 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.481 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.481 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.481 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.481 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ACCESS_KEY": { 2023-11-27T15:28:58.481 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-27T15:28:58.481 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.482 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.482 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.482 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.482 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.482 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.482 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.482 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.482 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.483 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.483 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.483 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.483 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-27T15:28:58.483 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-27T15:28:58.483 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.483 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.483 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.484 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-11-27T15:28:58.484 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.484 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.484 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.484 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.484 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.484 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.484 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.485 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.485 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SECRET_KEY": { 2023-11-27T15:28:58.485 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-27T15:28:58.485 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.485 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.485 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.485 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.485 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.486 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.486 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.486 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.486 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.486 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.486 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.486 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.487 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SSL_VERIFY": { 2023-11-27T15:28:58.487 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-27T15:28:58.487 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.487 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.487 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.487 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.487 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.487 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.488 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.488 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.488 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.488 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.488 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.488 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.488 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-27T15:28:58.488 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-27T15:28:58.489 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.489 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.489 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.489 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-11-27T15:28:58.489 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.489 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.489 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.489 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.489 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.490 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.490 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.490 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.490 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-27T15:28:58.490 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-27T15:28:58.490 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.490 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.490 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.491 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:28:58.491 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.491 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.491 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.491 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.491 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.491 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.491 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.492 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.492 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-27T15:28:58.492 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-27T15:28:58.492 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.492 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.492 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.492 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-11-27T15:28:58.492 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.493 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.493 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.493 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.493 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.493 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.493 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.493 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.493 INFO:teuthology.orchestra.run.smithi133.stdout: "cross_origin_url": { 2023-11-27T15:28:58.494 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cross_origin_url", 2023-11-27T15:28:58.494 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.494 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.494 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.494 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.494 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.494 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.495 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.495 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.495 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.495 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.495 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.495 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.495 INFO:teuthology.orchestra.run.smithi133.stdout: "crt_file": { 2023-11-27T15:28:58.495 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crt_file", 2023-11-27T15:28:58.496 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.496 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.496 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.496 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.496 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.496 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.496 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.496 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.497 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.497 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.497 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.497 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.497 INFO:teuthology.orchestra.run.smithi133.stdout: "debug": { 2023-11-27T15:28:58.497 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "debug", 2023-11-27T15:28:58.497 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.497 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.498 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.498 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.498 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.498 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.498 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.498 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable/disable debug options", 2023-11-27T15:28:58.498 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.498 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.499 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.499 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.499 INFO:teuthology.orchestra.run.smithi133.stdout: "jwt_token_ttl": { 2023-11-27T15:28:58.499 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "jwt_token_ttl", 2023-11-27T15:28:58.499 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.499 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.499 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.500 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "28800", 2023-11-27T15:28:58.500 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.500 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.500 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.500 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.500 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.500 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.500 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.501 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.501 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2023-11-27T15:28:58.501 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2023-11-27T15:28:58.501 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.501 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.501 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.501 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.501 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.502 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.503 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.504 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.505 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.506 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.507 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.507 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.507 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.507 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.507 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.507 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.507 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.508 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.509 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.509 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.509 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.509 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.509 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.509 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.509 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.509 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.510 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.510 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.510 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.510 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.510 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.510 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.510 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.510 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: "motd": { 2023-11-27T15:28:58.511 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "motd", 2023-11-27T15:28:58.512 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.512 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.512 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.512 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.512 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.512 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.512 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.512 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "The message of the day", 2023-11-27T15:28:58.513 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.513 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.513 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.513 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.513 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect_resolve_ip_addr": { 2023-11-27T15:28:58.513 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "redirect_resolve_ip_addr", 2023-11-27T15:28:58.513 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.513 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.514 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.514 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.514 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.514 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.514 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.514 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.514 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.514 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.515 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.515 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.515 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2023-11-27T15:28:58.515 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2023-11-27T15:28:58.515 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.515 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.515 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.515 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2023-11-27T15:28:58.516 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.516 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.516 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.516 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.516 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.516 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.516 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.516 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.517 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2023-11-27T15:28:58.517 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2023-11-27T15:28:58.517 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.517 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.517 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.517 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8080", 2023-11-27T15:28:58.517 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.517 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.518 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.518 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.518 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.518 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.518 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.518 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.518 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2023-11-27T15:28:58.518 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2023-11-27T15:28:58.519 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.519 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.519 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.519 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.519 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.519 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.519 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.519 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.520 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.520 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.520 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.520 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.520 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl_server_port": { 2023-11-27T15:28:58.520 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl_server_port", 2023-11-27T15:28:58.520 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.521 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.521 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.521 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8443", 2023-11-27T15:28:58.521 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.521 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.521 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.521 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.521 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.522 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.522 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.522 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.522 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2023-11-27T15:28:58.522 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2023-11-27T15:28:58.522 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.522 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.522 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.523 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "redirect", 2023-11-27T15:28:58.523 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.523 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.523 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.523 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.523 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect" 2023-11-27T15:28:58.523 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.523 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.524 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.524 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.524 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.524 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.524 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2023-11-27T15:28:58.524 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2023-11-27T15:28:58.524 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.524 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.525 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.525 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2023-11-27T15:28:58.525 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2023-11-27T15:28:58.525 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2023-11-27T15:28:58.525 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.525 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.525 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.525 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.526 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.526 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.526 INFO:teuthology.orchestra.run.smithi133.stdout: "url_prefix": { 2023-11-27T15:28:58.526 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url_prefix", 2023-11-27T15:28:58.526 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.526 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.526 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.526 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.527 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.527 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.527 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.527 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.527 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.527 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.527 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.527 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.528 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.528 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.528 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.528 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "devicehealth", 2023-11-27T15:28:58.528 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.528 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.528 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.529 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_monitoring": { 2023-11-27T15:28:58.529 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_monitoring", 2023-11-27T15:28:58.529 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.529 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.529 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.529 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.529 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.529 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.530 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.530 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "monitor device health metrics", 2023-11-27T15:28:58.530 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.530 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.530 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.530 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.530 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.531 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.531 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.531 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.531 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.531 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.531 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.531 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.531 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.532 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.532 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.532 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.532 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.532 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.532 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.532 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.532 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.533 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.533 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.533 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.533 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.533 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.533 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.533 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.533 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.534 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.534 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.534 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.534 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.534 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.534 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.534 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.534 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.535 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.535 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.535 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.535 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.535 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.535 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.535 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.535 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.536 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.536 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.536 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.536 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.536 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.536 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.536 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.537 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.537 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.537 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.537 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.537 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.537 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.537 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.537 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.538 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.538 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.538 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.538 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.538 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.538 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.538 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.538 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.539 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.539 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.539 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.539 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.539 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.539 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.539 INFO:teuthology.orchestra.run.smithi133.stdout: "mark_out_threshold": { 2023-11-27T15:28:58.540 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mark_out_threshold", 2023-11-27T15:28:58.540 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.540 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.540 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.540 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2419200", 2023-11-27T15:28:58.540 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.540 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.540 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.541 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-27T15:28:58.541 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.541 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.541 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.541 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.541 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_name": { 2023-11-27T15:28:58.541 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_name", 2023-11-27T15:28:58.541 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.542 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.542 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.542 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "device_health_metrics", 2023-11-27T15:28:58.542 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.542 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.542 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.542 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-27T15:28:58.542 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.543 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.543 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.543 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.543 INFO:teuthology.orchestra.run.smithi133.stdout: "retention_period": { 2023-11-27T15:28:58.543 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retention_period", 2023-11-27T15:28:58.543 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.543 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.543 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.544 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15552000", 2023-11-27T15:28:58.544 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.544 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.544 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.544 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain device health metrics", 2023-11-27T15:28:58.544 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.544 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.544 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.545 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.545 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_frequency": { 2023-11-27T15:28:58.545 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_frequency", 2023-11-27T15:28:58.545 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.545 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.545 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.545 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2023-11-27T15:28:58.545 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.546 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.546 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.546 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-27T15:28:58.546 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.546 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.546 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.546 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.547 INFO:teuthology.orchestra.run.smithi133.stdout: "self_heal": { 2023-11-27T15:28:58.547 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "self_heal", 2023-11-27T15:28:58.547 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.547 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.547 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.547 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.547 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.547 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.548 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.548 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-27T15:28:58.548 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.548 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.548 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.548 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.548 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-11-27T15:28:58.548 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-11-27T15:28:58.549 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.549 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.549 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.549 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-11-27T15:28:58.549 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.549 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.549 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.549 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and check device health", 2023-11-27T15:28:58.550 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.550 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.550 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.550 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.550 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_threshold": { 2023-11-27T15:28:58.550 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_threshold", 2023-11-27T15:28:58.550 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.550 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.551 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.551 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7257600", 2023-11-27T15:28:58.551 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.551 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.551 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.551 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-27T15:28:58.551 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.551 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.552 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.552 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.552 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.552 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.552 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.552 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "diskprediction_local", 2023-11-27T15:28:58.552 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.552 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.553 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.553 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.553 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.553 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.553 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.553 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.553 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.554 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.554 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.554 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.554 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.554 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.554 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.554 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.554 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.555 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.555 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.555 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.555 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.555 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.555 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.555 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.555 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.556 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.556 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.556 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.556 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.556 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.556 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.556 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.556 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.557 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.557 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.557 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.557 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.557 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.557 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.557 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.558 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.558 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.558 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.558 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.558 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.558 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.558 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.559 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.559 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.559 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.559 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.559 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.559 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.559 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.559 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.560 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.560 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.560 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.560 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.560 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.560 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.560 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.560 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.561 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.561 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.561 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.561 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.561 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.561 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.561 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.561 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.562 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.562 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.562 INFO:teuthology.orchestra.run.smithi133.stdout: "predict_interval": { 2023-11-27T15:28:58.562 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predict_interval", 2023-11-27T15:28:58.562 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.562 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.562 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.562 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2023-11-27T15:28:58.563 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.563 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.563 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.563 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.563 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.563 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.563 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.563 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.564 INFO:teuthology.orchestra.run.smithi133.stdout: "predictor_model": { 2023-11-27T15:28:58.564 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predictor_model", 2023-11-27T15:28:58.564 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.564 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.564 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.564 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "prophetstor", 2023-11-27T15:28:58.564 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.564 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.565 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.565 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.565 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.565 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.565 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.565 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.565 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-11-27T15:28:58.565 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-11-27T15:28:58.566 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.566 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.566 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.566 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-11-27T15:28:58.566 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.566 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.566 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.567 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.567 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.567 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.567 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.567 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.567 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.567 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.567 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.568 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "influx", 2023-11-27T15:28:58.568 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": false, 2023-11-27T15:28:58.568 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "influxdb python module not found", 2023-11-27T15:28:58.568 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.568 INFO:teuthology.orchestra.run.smithi133.stdout: "batch_size": { 2023-11-27T15:28:58.568 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "batch_size", 2023-11-27T15:28:58.568 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.568 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.569 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.569 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5000", 2023-11-27T15:28:58.569 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.569 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.569 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.569 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-27T15:28:58.569 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.569 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.570 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.570 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.570 INFO:teuthology.orchestra.run.smithi133.stdout: "database": { 2023-11-27T15:28:58.570 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "database", 2023-11-27T15:28:58.570 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.570 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.570 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.571 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "ceph", 2023-11-27T15:28:58.571 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.571 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.571 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.571 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-27T15:28:58.571 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.571 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.571 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.572 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.572 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": { 2023-11-27T15:28:58.572 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "hostname", 2023-11-27T15:28:58.572 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.572 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.572 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.572 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.572 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.573 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.573 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.573 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server hostname", 2023-11-27T15:28:58.573 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.573 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.573 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.573 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.573 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-11-27T15:28:58.574 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-11-27T15:28:58.574 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.574 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.574 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.574 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "30", 2023-11-27T15:28:58.574 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "5", 2023-11-27T15:28:58.574 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.574 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.575 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-27T15:28:58.575 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.575 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.575 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.575 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.575 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.575 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.576 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.576 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.576 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.576 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.576 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.576 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.576 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.576 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.577 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.577 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.577 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.577 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.577 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.577 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.577 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.578 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.578 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.578 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.578 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.578 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.578 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.578 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.578 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.579 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.579 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.579 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.579 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.579 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.579 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.579 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.579 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.580 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.580 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.580 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.580 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.580 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.580 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.580 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.580 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.581 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.581 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.581 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.581 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.581 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.581 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.581 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.581 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.582 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.582 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.582 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.582 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.582 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.582 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.582 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.582 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.583 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.583 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.583 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.583 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.583 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.583 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.583 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.583 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.584 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.584 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.584 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.584 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.584 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.584 INFO:teuthology.orchestra.run.smithi133.stdout: "password": { 2023-11-27T15:28:58.584 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "password", 2023-11-27T15:28:58.584 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.585 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.585 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.585 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.585 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.585 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.585 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.585 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "password of InfluxDB server user", 2023-11-27T15:28:58.585 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.586 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.586 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.586 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.586 INFO:teuthology.orchestra.run.smithi133.stdout: "port": { 2023-11-27T15:28:58.586 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "port", 2023-11-27T15:28:58.586 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.586 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.587 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.587 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8086", 2023-11-27T15:28:58.587 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.587 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.587 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.587 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server port", 2023-11-27T15:28:58.587 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.587 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.588 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.588 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.588 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2023-11-27T15:28:58.588 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2023-11-27T15:28:58.588 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.588 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.588 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.588 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "false", 2023-11-27T15:28:58.589 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.589 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.589 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.589 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-27T15:28:58.589 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.589 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.589 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.589 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.590 INFO:teuthology.orchestra.run.smithi133.stdout: "threads": { 2023-11-27T15:28:58.590 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threads", 2023-11-27T15:28:58.590 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.590 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.590 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.590 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-11-27T15:28:58.591 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-11-27T15:28:58.591 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "32", 2023-11-27T15:28:58.591 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.591 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-27T15:28:58.591 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.591 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.591 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.591 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.592 INFO:teuthology.orchestra.run.smithi133.stdout: "username": { 2023-11-27T15:28:58.592 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "username", 2023-11-27T15:28:58.592 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.592 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.592 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.592 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.592 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.592 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.593 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.593 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "username of InfluxDB server user", 2023-11-27T15:28:58.593 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.593 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.593 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.593 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.593 INFO:teuthology.orchestra.run.smithi133.stdout: "verify_ssl": { 2023-11-27T15:28:58.593 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "verify_ssl", 2023-11-27T15:28:58.594 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.594 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.594 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.594 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "true", 2023-11-27T15:28:58.594 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.594 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.594 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.595 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-27T15:28:58.595 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.595 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.595 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.595 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.595 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.595 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.595 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.596 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "insights", 2023-11-27T15:28:58.596 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.596 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.596 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.596 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.596 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.596 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.596 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.597 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.597 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.597 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.597 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.597 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.597 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.597 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.598 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.598 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.598 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.598 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.598 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.598 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.598 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.598 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.599 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.599 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.599 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.599 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.599 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.599 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.599 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.600 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.600 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.600 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.600 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.600 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.600 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.600 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.600 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.601 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.601 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.601 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.601 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.601 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.601 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.601 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.602 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.602 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.602 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.602 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.602 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.602 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.602 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.602 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.603 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.603 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.603 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.603 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.603 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.603 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.603 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.603 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.604 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.604 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.604 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.604 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.604 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.604 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.604 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.605 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.605 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.605 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.605 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.605 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.605 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.605 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.605 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.606 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.606 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "iostat", 2023-11-27T15:28:58.606 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.606 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.606 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.606 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.606 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.607 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.607 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.607 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.607 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.607 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.607 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.607 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.607 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.608 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.608 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.608 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.608 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.608 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.608 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.608 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.608 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.609 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.609 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.609 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.609 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.609 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.609 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.609 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.609 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.610 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.610 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.610 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.610 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.610 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.610 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.610 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.611 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.611 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.611 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.611 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.611 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.611 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.611 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.611 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.612 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.612 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.612 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.612 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.612 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.612 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.612 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.612 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.613 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.613 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.613 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.613 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.613 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.613 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.613 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.613 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.614 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.614 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.614 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.614 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.614 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.614 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.614 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.614 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.615 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.615 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.615 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.615 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.615 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.615 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.615 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.615 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.616 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "k8sevents", 2023-11-27T15:28:58.616 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.616 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.616 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.616 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_event_retention_days": { 2023-11-27T15:28:58.616 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ceph_event_retention_days", 2023-11-27T15:28:58.616 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.617 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.617 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.617 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7", 2023-11-27T15:28:58.617 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.617 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.617 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.617 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Days to hold ceph event information within local cache", 2023-11-27T15:28:58.617 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.618 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.618 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.618 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.618 INFO:teuthology.orchestra.run.smithi133.stdout: "config_check_secs": { 2023-11-27T15:28:58.618 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_check_secs", 2023-11-27T15:28:58.618 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.618 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.619 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.619 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:28:58.619 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "10", 2023-11-27T15:28:58.619 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.619 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.619 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-11-27T15:28:58.619 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.619 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.620 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.620 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.620 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.620 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.620 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.620 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.620 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.620 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.621 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.621 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.621 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.621 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.621 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.621 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.621 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.621 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.622 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.622 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.622 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.622 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.622 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.622 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.622 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.622 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.623 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.623 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.623 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.623 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.623 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.623 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.623 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.624 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.624 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.624 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.624 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.624 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.624 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.624 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.624 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.625 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.625 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.625 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.625 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.625 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.625 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.625 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.625 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.626 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.626 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.626 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.626 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.626 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.626 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.626 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.627 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.627 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.627 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.627 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.627 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.627 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.627 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.627 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.628 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.628 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.628 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.628 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.628 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.628 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.628 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.628 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.629 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.629 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.629 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.629 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.629 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.629 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "localpool", 2023-11-27T15:28:58.629 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.630 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.630 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.630 INFO:teuthology.orchestra.run.smithi133.stdout: "failure_domain": { 2023-11-27T15:28:58.630 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "failure_domain", 2023-11-27T15:28:58.630 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.630 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.630 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.630 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "host", 2023-11-27T15:28:58.631 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.631 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.631 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.631 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "failure domain for any created local pool", 2023-11-27T15:28:58.631 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-27T15:28:58.631 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.631 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.631 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.632 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.632 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.632 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.632 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.632 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.632 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.632 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.632 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.633 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.633 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.633 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.633 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.633 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.633 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.633 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.633 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.634 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.634 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.634 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.634 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.634 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.634 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.634 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.635 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.635 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.635 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.635 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.635 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.635 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.635 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.635 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.636 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.636 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.636 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.636 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.636 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.636 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.636 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.636 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.637 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.637 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.637 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.637 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.637 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.637 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.637 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.637 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.638 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.638 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.638 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.638 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.638 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.638 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.638 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.638 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.639 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.639 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.639 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.639 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.639 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.639 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.639 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.640 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.640 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.640 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.640 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.640 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.640 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.640 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.640 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.641 INFO:teuthology.orchestra.run.smithi133.stdout: "min_size": { 2023-11-27T15:28:58.641 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_size", 2023-11-27T15:28:58.641 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.641 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.641 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.641 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.641 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.641 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.642 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.642 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default min_size for any created local pool", 2023-11-27T15:28:58.642 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-27T15:28:58.642 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.642 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.642 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.642 INFO:teuthology.orchestra.run.smithi133.stdout: "num_rep": { 2023-11-27T15:28:58.643 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "num_rep", 2023-11-27T15:28:58.643 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.643 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.643 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.643 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3", 2023-11-27T15:28:58.643 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.643 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.643 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.644 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default replica count for any created local pool", 2023-11-27T15:28:58.644 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.644 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.644 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.644 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.644 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_num": { 2023-11-27T15:28:58.644 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_num", 2023-11-27T15:28:58.644 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.645 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.645 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.645 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "128", 2023-11-27T15:28:58.645 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.645 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.646 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.646 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default pg_num for any created local pool", 2023-11-27T15:28:58.646 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.646 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.646 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.646 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.647 INFO:teuthology.orchestra.run.smithi133.stdout: "prefix": { 2023-11-27T15:28:58.647 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prefix", 2023-11-27T15:28:58.647 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.647 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.647 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.647 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.647 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.647 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.648 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.648 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name prefix for any created local pool", 2023-11-27T15:28:58.648 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.648 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.648 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.648 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.648 INFO:teuthology.orchestra.run.smithi133.stdout: "subtree": { 2023-11-27T15:28:58.648 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "subtree", 2023-11-27T15:28:58.649 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.649 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.649 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.649 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "rack", 2023-11-27T15:28:58.649 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.649 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.649 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.649 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-27T15:28:58.650 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-27T15:28:58.650 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.650 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.650 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.650 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.650 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.650 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.651 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mds_autoscaler", 2023-11-27T15:28:58.651 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.651 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.651 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.651 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.651 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.651 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.651 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.652 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.652 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.652 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.652 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.652 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.652 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.652 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.653 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.653 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.653 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.653 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.653 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.653 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.653 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.653 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.654 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.654 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.654 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.654 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.654 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.654 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.654 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.655 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.655 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.655 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.655 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.655 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.655 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.655 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.655 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.656 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.656 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.656 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.656 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.656 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.656 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.656 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.656 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.657 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.657 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.657 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.657 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.657 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.657 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.657 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.658 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.658 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.658 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.658 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.658 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.658 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.658 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.658 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.659 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.659 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.659 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.659 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.659 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.659 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.659 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.660 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.660 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.660 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.660 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.660 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.660 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.660 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.660 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.661 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.661 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirroring", 2023-11-27T15:28:58.661 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.661 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.661 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.661 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.661 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.661 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.662 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.662 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.662 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.662 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.662 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.662 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.662 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.663 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.663 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.663 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.663 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.663 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.663 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.663 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.664 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.664 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.664 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.664 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.664 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.664 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.664 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.664 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.665 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.665 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.665 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.665 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.665 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.665 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.665 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.666 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.666 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.666 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.666 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.666 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.666 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.666 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.666 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.667 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.667 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.667 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.667 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.667 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.667 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.667 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.668 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.668 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.668 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.668 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.668 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.668 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.668 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.668 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.669 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.669 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.669 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.669 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.669 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.669 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.669 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.669 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.670 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.670 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.670 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.670 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.670 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.670 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.670 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.671 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.671 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.671 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.671 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "nfs", 2023-11-27T15:28:58.671 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.671 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.671 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.671 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.672 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.672 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.672 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.672 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.672 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.672 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.672 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.672 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.673 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.673 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.673 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.673 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.673 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.673 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.673 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.673 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.674 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.674 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.674 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.674 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.674 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.674 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.674 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.675 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.675 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.675 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.675 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.675 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.675 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.675 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.675 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.676 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.676 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.676 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.676 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.676 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.676 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.676 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.676 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.677 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.677 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.677 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.677 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.677 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.677 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.677 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.678 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.678 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.678 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.678 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.678 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.678 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.678 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.679 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.679 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.679 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.679 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.679 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.679 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.679 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.679 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.680 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.680 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.680 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.680 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.680 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.680 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.680 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.680 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.681 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.681 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.681 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.681 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2023-11-27T15:28:58.681 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.681 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.681 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.681 INFO:teuthology.orchestra.run.smithi133.stdout: "fail_fs": { 2023-11-27T15:28:58.682 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "fail_fs", 2023-11-27T15:28:58.682 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.682 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.682 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.682 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.682 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.682 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.683 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.683 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-27T15:28:58.683 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.683 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.683 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.683 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.683 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.683 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.684 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.684 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.684 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.684 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.684 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.684 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.684 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.684 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.685 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.685 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.685 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.685 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.685 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.685 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.685 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.685 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.686 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.686 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.686 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.686 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.686 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.686 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.686 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.687 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.687 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.687 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.687 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.687 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.687 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.687 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.687 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.688 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.688 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.688 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.688 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.688 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.688 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.688 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.688 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.689 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.689 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.689 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.689 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.689 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.689 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.689 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.689 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.690 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.690 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.690 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.690 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.690 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.690 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.690 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.690 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.691 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.691 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.691 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.691 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.691 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.691 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.691 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.691 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.692 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.692 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.692 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.692 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.692 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.692 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator": { 2023-11-27T15:28:58.692 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2023-11-27T15:28:58.693 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.693 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.693 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.693 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.693 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.693 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.693 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.693 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm", 2023-11-27T15:28:58.694 INFO:teuthology.orchestra.run.smithi133.stdout: "rook", 2023-11-27T15:28:58.694 INFO:teuthology.orchestra.run.smithi133.stdout: "test_orchestrator" 2023-11-27T15:28:58.694 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.694 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Orchestrator backend", 2023-11-27T15:28:58.694 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.694 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.694 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.695 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.695 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.695 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.695 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.695 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_perf_query", 2023-11-27T15:28:58.695 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.695 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.695 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.696 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.696 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.696 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.696 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.696 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.696 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.696 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.696 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.697 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.697 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.697 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.697 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.697 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.697 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.697 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.697 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.698 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.698 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.698 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.698 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.698 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.698 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.698 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.699 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.699 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.699 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.699 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.699 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.699 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.699 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.699 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.700 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.700 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.700 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.700 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.700 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.700 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.700 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.701 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.701 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.701 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.701 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.701 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.701 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.701 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.701 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.702 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.702 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.702 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.702 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.702 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.702 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.702 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.703 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.703 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.703 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.703 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.703 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.703 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.703 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.703 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.704 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.704 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.704 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.704 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.704 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.704 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.704 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.704 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.705 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.705 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.705 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.705 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.705 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_support", 2023-11-27T15:28:58.705 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.705 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.706 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.706 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.706 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.706 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.706 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.706 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.706 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.706 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.707 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.707 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.707 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.707 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.707 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.707 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.707 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.707 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.708 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.708 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.708 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.708 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.708 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.708 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.708 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.709 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.709 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.709 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.709 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.709 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.709 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.709 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.709 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.710 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.710 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.710 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.710 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.710 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.710 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.710 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.711 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.711 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.711 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.711 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.711 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.711 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.711 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.711 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.712 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.712 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.712 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.712 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.712 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.712 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.712 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.712 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.713 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.713 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.713 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.713 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.713 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.713 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.713 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.713 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.714 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.714 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.714 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.714 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.714 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.714 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.714 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.714 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.715 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.715 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.715 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.715 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.715 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_autoscaler", 2023-11-27T15:28:58.715 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.715 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.716 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.716 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.716 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.716 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.716 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.716 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.716 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.716 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.717 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.717 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.717 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.717 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.717 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.717 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.717 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.718 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.718 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.718 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.718 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.718 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.718 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.718 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.718 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.719 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.719 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.719 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.719 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.719 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.719 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.719 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.720 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.720 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.720 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.720 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.720 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.720 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.720 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.720 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.721 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.721 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.721 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.721 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.721 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.721 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.721 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.721 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.722 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.722 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.722 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.722 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.722 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.722 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.722 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.723 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.723 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.723 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.723 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.723 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.723 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.723 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.723 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.724 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.724 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.724 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.724 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.724 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.724 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.724 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.724 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.725 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.725 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.725 INFO:teuthology.orchestra.run.smithi133.stdout: "noautoscale": { 2023-11-27T15:28:58.725 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "noautoscale", 2023-11-27T15:28:58.725 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.725 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.725 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.725 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.726 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.726 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.726 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.726 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "global autoscale flag", 2023-11-27T15:28:58.726 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-11-27T15:28:58.726 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.726 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.726 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.727 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-11-27T15:28:58.727 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-11-27T15:28:58.727 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.727 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.727 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.727 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-11-27T15:28:58.727 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.728 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.728 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.728 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.728 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.728 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.728 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.728 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.729 INFO:teuthology.orchestra.run.smithi133.stdout: "threshold": { 2023-11-27T15:28:58.729 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threshold", 2023-11-27T15:28:58.729 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:28:58.729 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.729 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.729 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2023-11-27T15:28:58.729 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1.0", 2023-11-27T15:28:58.729 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.730 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.730 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "scaling threshold", 2023-11-27T15:28:58.730 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-27T15:28:58.730 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.730 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.730 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.730 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.731 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.731 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.731 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "progress", 2023-11-27T15:28:58.731 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.731 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.731 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.731 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_pg_recovery_event": { 2023-11-27T15:28:58.732 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_pg_recovery_event", 2023-11-27T15:28:58.732 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.732 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.732 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.732 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.732 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.732 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.732 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.733 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-27T15:28:58.733 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.733 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.733 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.733 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.733 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2023-11-27T15:28:58.733 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2023-11-27T15:28:58.733 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.734 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.734 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.734 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.734 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.734 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.734 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.734 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.735 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.735 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.735 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.735 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.735 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.735 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.735 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.735 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.736 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.736 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.736 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.736 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.736 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.736 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.736 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.737 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.737 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.737 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.737 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.737 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.737 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.737 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.738 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.738 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.738 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.738 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.738 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.738 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.738 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.738 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.739 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.739 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.739 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.739 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.739 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.739 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.739 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.739 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.740 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.740 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.740 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.740 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.740 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.740 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.740 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.740 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.741 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.741 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.741 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.741 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.741 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.741 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.741 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.742 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.742 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.742 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.742 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.742 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.742 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.742 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.742 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.743 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.743 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.743 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.743 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.743 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.743 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.743 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.743 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.744 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.744 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.744 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.744 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.744 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.744 INFO:teuthology.orchestra.run.smithi133.stdout: "max_completed_events": { 2023-11-27T15:28:58.744 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_completed_events", 2023-11-27T15:28:58.744 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.745 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.745 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.745 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "50", 2023-11-27T15:28:58.745 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.745 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.745 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.745 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "number of past completed events to remember", 2023-11-27T15:28:58.745 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.746 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.746 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.746 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.746 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-11-27T15:28:58.746 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-11-27T15:28:58.746 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.746 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.746 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.747 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-11-27T15:28:58.747 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.747 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.747 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.747 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long the module is going to sleep", 2023-11-27T15:28:58.747 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.747 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.748 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.748 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.748 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.748 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.748 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.748 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus", 2023-11-27T15:28:58.748 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.748 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.749 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.749 INFO:teuthology.orchestra.run.smithi133.stdout: "cache": { 2023-11-27T15:28:58.749 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cache", 2023-11-27T15:28:58.749 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.749 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.749 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.749 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.749 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.750 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.750 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.750 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.750 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.750 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.750 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.750 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.750 INFO:teuthology.orchestra.run.smithi133.stdout: "exclude_perf_counters": { 2023-11-27T15:28:58.751 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "exclude_perf_counters", 2023-11-27T15:28:58.751 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.751 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.751 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.751 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.751 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.751 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.752 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.752 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-27T15:28:58.752 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-27T15:28:58.752 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.752 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.752 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.752 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.752 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.753 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.753 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.753 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.753 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.753 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.753 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.753 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.753 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.754 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.754 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.754 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.754 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.754 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.754 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.754 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.755 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.755 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.755 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.755 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.755 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.755 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.755 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.755 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.755 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.756 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.756 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.756 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.756 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.756 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.756 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.756 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.756 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.757 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.757 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.757 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.757 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.757 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.757 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.757 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.758 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.758 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.758 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.758 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.758 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.758 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.758 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.758 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.759 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.759 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.759 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.759 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.759 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.759 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.759 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.759 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.760 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.760 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.760 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.760 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.760 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.760 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.760 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.760 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.761 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.761 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.761 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.761 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.761 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.761 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools": { 2023-11-27T15:28:58.761 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools", 2023-11-27T15:28:58.761 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.762 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.762 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.762 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.762 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.762 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.762 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.762 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.763 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.763 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.763 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.763 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.763 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-27T15:28:58.763 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-27T15:28:58.763 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.763 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.764 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.764 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "300", 2023-11-27T15:28:58.764 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.764 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.764 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.764 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.764 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.764 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.765 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.765 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.765 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_interval": { 2023-11-27T15:28:58.765 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_interval", 2023-11-27T15:28:58.765 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:28:58.765 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.765 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.765 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15.0", 2023-11-27T15:28:58.766 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.766 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.766 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.766 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.766 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.766 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.766 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.766 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.767 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2023-11-27T15:28:58.767 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2023-11-27T15:28:58.767 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.767 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.767 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.767 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2023-11-27T15:28:58.767 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.768 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.768 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.768 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-27T15:28:58.768 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.768 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.768 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.768 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.769 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2023-11-27T15:28:58.769 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2023-11-27T15:28:58.769 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.769 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.769 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.769 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "9283", 2023-11-27T15:28:58.769 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.769 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.770 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.770 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-27T15:28:58.770 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.770 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.770 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.770 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.770 INFO:teuthology.orchestra.run.smithi133.stdout: "stale_cache_strategy": { 2023-11-27T15:28:58.770 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stale_cache_strategy", 2023-11-27T15:28:58.771 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.771 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.771 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.771 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "log", 2023-11-27T15:28:58.771 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.771 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.771 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.771 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.772 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.772 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.772 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.772 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.772 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2023-11-27T15:28:58.772 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2023-11-27T15:28:58.772 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.772 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.773 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.773 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "default", 2023-11-27T15:28:58.773 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.773 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.773 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.773 INFO:teuthology.orchestra.run.smithi133.stdout: "default", 2023-11-27T15:28:58.773 INFO:teuthology.orchestra.run.smithi133.stdout: "error" 2023-11-27T15:28:58.773 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.774 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.774 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.774 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.774 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.774 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.774 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2023-11-27T15:28:58.774 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2023-11-27T15:28:58.774 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.775 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.775 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.775 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2023-11-27T15:28:58.775 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2023-11-27T15:28:58.775 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2023-11-27T15:28:58.775 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.775 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.775 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.776 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.776 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.776 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.776 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.776 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.776 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.776 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_support", 2023-11-27T15:28:58.777 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.777 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.777 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.777 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.777 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.777 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.777 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.777 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.778 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.778 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.778 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.778 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.778 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.778 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.778 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.778 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.779 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.779 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.779 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.779 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.779 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.779 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.779 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.779 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.780 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.780 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.780 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.780 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.780 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.780 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.780 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.780 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.781 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.781 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.781 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.781 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.781 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.781 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.781 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.781 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.782 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.782 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.782 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.782 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.782 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.782 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.782 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.783 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.783 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.783 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.783 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.783 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.783 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.783 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.783 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.784 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.784 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.784 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.784 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.784 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.784 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.784 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.784 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.785 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.785 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.785 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.785 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.785 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.785 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.785 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.785 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.786 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.786 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.786 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_snap_create": { 2023-11-27T15:28:58.786 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_snap_create", 2023-11-27T15:28:58.786 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.786 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.786 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.786 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:28:58.787 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.787 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.787 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.787 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.787 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.787 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.787 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.788 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.788 INFO:teuthology.orchestra.run.smithi133.stdout: "mirror_snapshot_schedule": { 2023-11-27T15:28:58.788 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirror_snapshot_schedule", 2023-11-27T15:28:58.788 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.788 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.788 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.788 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.788 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.789 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.789 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.789 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.789 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.789 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.789 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.789 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.789 INFO:teuthology.orchestra.run.smithi133.stdout: "trash_purge_schedule": { 2023-11-27T15:28:58.789 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "trash_purge_schedule", 2023-11-27T15:28:58.790 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.790 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.790 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.790 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.790 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.790 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.790 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.791 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.791 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.791 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.791 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.791 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.791 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.791 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.791 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.792 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "restful", 2023-11-27T15:28:58.792 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.792 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.792 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.792 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_auth": { 2023-11-27T15:28:58.792 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_auth", 2023-11-27T15:28:58.792 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.792 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.793 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.793 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.793 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.793 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.793 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.793 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.793 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.793 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.794 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.794 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.794 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2023-11-27T15:28:58.794 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2023-11-27T15:28:58.794 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.794 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.794 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.794 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.795 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.795 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.795 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.795 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.795 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.795 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.795 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.796 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.796 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.796 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.796 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.796 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.796 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.796 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.796 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.797 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.797 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.797 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.797 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.797 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.797 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.797 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.797 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.798 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.798 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.798 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.798 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.798 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.798 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.798 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.798 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.799 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.799 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.799 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.799 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.799 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.799 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.799 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.799 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.800 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.800 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.800 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.800 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.800 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.800 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.800 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.801 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.801 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.801 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.801 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.801 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.801 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.802 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.802 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.802 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.802 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.802 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.802 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.802 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.803 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.803 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.803 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.803 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.803 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.803 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.803 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.803 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.805 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.805 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.805 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.805 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.805 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.805 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.806 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.806 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.806 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.806 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.806 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.806 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2023-11-27T15:28:58.806 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2023-11-27T15:28:58.807 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.807 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.807 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.807 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.807 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.807 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.807 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.807 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.808 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.808 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.808 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.808 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.808 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2023-11-27T15:28:58.808 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2023-11-27T15:28:58.808 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.808 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.809 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.809 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.809 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.809 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.809 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.809 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.809 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.809 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.810 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.810 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.810 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.810 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.810 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.810 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rgw", 2023-11-27T15:28:58.810 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.811 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.811 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.811 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.811 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.811 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.811 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.811 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.811 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.812 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.812 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.812 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.812 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.812 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.812 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.812 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.812 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.813 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.813 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.813 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.813 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.813 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.813 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.813 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.813 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.814 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.814 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.814 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.814 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.814 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.814 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.814 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.814 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.815 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.815 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.815 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.815 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.815 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.815 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.815 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.815 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.816 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.816 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.816 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.816 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.816 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.816 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.816 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.817 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.817 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.817 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.817 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.817 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.817 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.817 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.817 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.818 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.818 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.818 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.818 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.818 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.818 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.818 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.818 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.819 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.819 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.819 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.819 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.819 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.819 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.819 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.819 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.820 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.820 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.820 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.820 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.820 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rook", 2023-11-27T15:28:58.820 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.820 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.820 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.821 INFO:teuthology.orchestra.run.smithi133.stdout: "drive_group_interval": { 2023-11-27T15:28:58.821 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "drive_group_interval", 2023-11-27T15:28:58.821 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:28:58.821 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.821 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.821 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "300.0", 2023-11-27T15:28:58.821 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.821 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.822 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.822 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-27T15:28:58.822 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.822 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.822 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.822 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.822 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.823 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.823 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.823 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.823 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.823 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.823 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.823 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.823 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.824 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.824 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.824 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.824 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.824 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.824 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.824 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.824 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.824 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.825 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.825 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.825 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.825 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.825 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.825 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.825 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.825 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.826 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.826 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.826 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.826 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.826 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.826 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.826 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.827 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.827 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.827 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.827 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.827 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.827 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.827 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.828 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.828 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.828 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.828 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.828 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.828 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.828 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.828 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.829 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.829 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.829 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.829 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.829 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.829 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.829 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.829 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.830 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.830 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.830 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.830 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.830 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.830 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.830 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.831 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.831 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.831 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.831 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.831 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.831 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.831 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.831 INFO:teuthology.orchestra.run.smithi133.stdout: "storage_class": { 2023-11-27T15:28:58.832 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "storage_class", 2023-11-27T15:28:58.832 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.832 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.832 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.832 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "local", 2023-11-27T15:28:58.832 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.832 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.833 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.833 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-27T15:28:58.833 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.833 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.833 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.833 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.833 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.833 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.834 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.834 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "selftest", 2023-11-27T15:28:58.834 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.834 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.834 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.834 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.834 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.835 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.835 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.835 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.835 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.835 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.835 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.835 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.835 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.836 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.836 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.836 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.836 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.836 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.836 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.836 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.836 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.837 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.837 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.837 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.837 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.837 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.837 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.837 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.838 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.838 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.838 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.838 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.838 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.838 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.838 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.838 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.839 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.839 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.839 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.839 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.839 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.839 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.839 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.839 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.840 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.840 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.840 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.840 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.840 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.840 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.840 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.841 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.841 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.841 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.841 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.841 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.841 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.841 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.841 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.842 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.842 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.842 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.842 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.842 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.842 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.842 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.843 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.843 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.843 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.843 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.843 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.843 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.843 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.843 INFO:teuthology.orchestra.run.smithi133.stdout: "roption1": { 2023-11-27T15:28:58.844 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption1", 2023-11-27T15:28:58.844 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.844 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.844 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.844 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.844 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.844 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.844 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.845 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.845 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.845 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.845 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.845 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.845 INFO:teuthology.orchestra.run.smithi133.stdout: "roption2": { 2023-11-27T15:28:58.845 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption2", 2023-11-27T15:28:58.845 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.846 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.846 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.846 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "xyz", 2023-11-27T15:28:58.846 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.846 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.846 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.846 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.846 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.847 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.847 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.847 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.847 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption1": { 2023-11-27T15:28:58.847 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption1", 2023-11-27T15:28:58.847 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.847 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.848 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.848 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.848 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.848 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.848 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.848 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.848 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.848 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.849 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.849 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.849 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption2": { 2023-11-27T15:28:58.849 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption2", 2023-11-27T15:28:58.849 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.849 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.849 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.849 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.850 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.850 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.850 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.850 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.850 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.850 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.850 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.850 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.851 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption3": { 2023-11-27T15:28:58.851 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption3", 2023-11-27T15:28:58.851 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:28:58.851 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.851 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.851 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.851 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.851 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.852 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.852 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.852 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.852 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.852 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.852 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.852 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption4": { 2023-11-27T15:28:58.852 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption4", 2023-11-27T15:28:58.853 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.853 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.853 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.853 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.853 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.853 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.853 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.854 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.854 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.854 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.854 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.854 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.854 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption5": { 2023-11-27T15:28:58.854 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption5", 2023-11-27T15:28:58.854 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.855 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.855 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.855 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.855 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.855 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.855 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.855 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.855 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.856 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.856 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.856 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.856 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption6": { 2023-11-27T15:28:58.856 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption6", 2023-11-27T15:28:58.856 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.856 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.856 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.857 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.857 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.857 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.857 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.857 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.857 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.857 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.857 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.858 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.858 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption7": { 2023-11-27T15:28:58.858 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption7", 2023-11-27T15:28:58.858 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.858 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.858 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.858 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.858 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-11-27T15:28:58.859 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "42", 2023-11-27T15:28:58.859 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.859 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.859 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.859 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.859 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.859 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.859 INFO:teuthology.orchestra.run.smithi133.stdout: "testkey": { 2023-11-27T15:28:58.860 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testkey", 2023-11-27T15:28:58.860 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.860 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.860 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.860 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.860 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.860 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.861 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.861 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.861 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.861 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.861 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.861 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.861 INFO:teuthology.orchestra.run.smithi133.stdout: "testlkey": { 2023-11-27T15:28:58.861 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testlkey", 2023-11-27T15:28:58.862 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.862 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.862 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.862 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.862 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.862 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.862 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.862 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.863 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.863 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.863 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.863 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.863 INFO:teuthology.orchestra.run.smithi133.stdout: "testnewline": { 2023-11-27T15:28:58.863 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testnewline", 2023-11-27T15:28:58.863 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.863 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.864 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.864 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.864 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.864 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.864 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.864 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.864 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.864 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.865 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.865 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.865 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.865 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.865 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.865 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snap_schedule", 2023-11-27T15:28:58.865 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.865 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.866 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.866 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_m_granularity": { 2023-11-27T15:28:58.866 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_m_granularity", 2023-11-27T15:28:58.866 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.866 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.866 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.866 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.867 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.867 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.867 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.867 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow minute scheduled snapshots", 2023-11-27T15:28:58.867 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.867 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.867 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.867 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.868 INFO:teuthology.orchestra.run.smithi133.stdout: "dump_on_update": { 2023-11-27T15:28:58.868 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dump_on_update", 2023-11-27T15:28:58.868 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.868 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.868 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.868 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.868 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.868 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.869 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.869 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "dump database to debug log on update", 2023-11-27T15:28:58.869 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.869 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.869 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.869 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.869 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.869 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.870 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.870 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.870 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.870 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.870 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.870 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.870 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.870 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.871 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.871 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.871 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.871 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.871 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.871 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.871 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.871 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.872 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.872 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.872 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.872 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.872 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.872 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.872 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.873 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.873 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.873 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.873 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.873 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.873 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.873 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.873 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.873 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.874 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.874 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.874 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.874 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.874 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.874 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.874 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.875 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.875 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.875 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.875 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.875 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.875 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.875 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.875 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.876 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.876 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.876 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.876 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.876 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.876 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.876 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.876 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.877 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.877 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.877 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.877 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.877 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.877 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.877 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.878 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.878 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.878 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.878 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.878 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.878 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.878 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.878 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.879 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.879 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stats", 2023-11-27T15:28:58.879 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.879 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.879 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.879 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.879 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.879 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.880 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.880 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.880 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.880 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.880 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.880 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.880 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.880 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.881 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.881 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.881 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.881 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.881 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.881 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.881 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.881 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.882 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.882 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.882 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.882 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.882 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.882 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.882 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.883 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.883 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.883 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.883 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.883 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.883 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.883 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.883 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.883 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.884 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.884 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.884 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.884 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.884 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.884 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.884 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.884 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.885 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.885 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.885 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.885 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.885 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.885 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.885 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.885 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.886 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.886 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.886 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.886 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.886 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.886 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.886 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.887 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.887 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.887 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.887 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.887 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.887 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.887 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.887 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.888 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.888 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.888 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.888 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.888 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.888 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.888 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.888 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "status", 2023-11-27T15:28:58.889 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.889 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.889 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.889 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.889 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.889 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.889 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.889 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.890 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.890 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.890 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.890 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.890 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.890 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.890 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.890 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.891 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.891 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.891 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.891 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.891 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.891 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.891 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.891 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.892 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.892 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.892 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.892 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.892 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.892 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.892 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.892 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.893 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.893 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.893 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.893 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.893 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.893 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.893 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.894 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.894 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.894 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.894 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.894 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.894 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.894 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.894 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.895 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.895 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.895 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.895 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.895 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.895 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.895 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.895 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.896 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.896 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.896 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.896 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.896 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.896 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.896 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.896 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.897 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.897 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.897 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.897 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.897 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.897 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.897 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.897 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.898 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.898 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.898 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.898 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.898 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.898 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telegraf", 2023-11-27T15:28:58.898 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.899 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.899 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.899 INFO:teuthology.orchestra.run.smithi133.stdout: "address": { 2023-11-27T15:28:58.899 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "address", 2023-11-27T15:28:58.899 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.899 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.899 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.899 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-27T15:28:58.900 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.900 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.900 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.900 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.900 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.900 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.900 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.900 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.901 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-11-27T15:28:58.901 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-11-27T15:28:58.901 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.901 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.901 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.901 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15", 2023-11-27T15:28:58.901 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.901 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.902 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.902 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.902 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.902 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.902 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.902 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.902 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.903 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.903 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.903 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.903 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.903 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.903 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.903 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.903 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.904 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.904 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.904 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.904 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.904 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.904 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.904 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.904 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.905 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.905 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.905 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.905 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.905 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.905 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.905 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.906 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.906 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.906 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.906 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.906 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.906 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.906 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.906 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.907 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.907 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.907 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.907 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.907 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.907 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.907 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.907 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.908 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.908 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.908 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.908 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.908 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.908 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.908 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.909 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.909 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.909 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.909 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.909 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.909 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.909 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.909 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.910 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.910 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.910 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.910 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.910 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.910 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.910 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.910 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.911 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.911 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.911 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.911 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.911 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.911 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.911 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.911 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.912 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.912 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.912 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telemetry", 2023-11-27T15:28:58.912 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.912 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.912 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.912 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_basic": { 2023-11-27T15:28:58.912 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_basic", 2023-11-27T15:28:58.913 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.913 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.913 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.913 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.913 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.913 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.913 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.914 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-27T15:28:58.914 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.914 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.914 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.914 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.914 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_crash": { 2023-11-27T15:28:58.914 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_crash", 2023-11-27T15:28:58.914 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.915 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.915 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.915 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.915 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.915 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.915 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.915 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-27T15:28:58.915 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.916 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.916 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.916 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.916 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_device": { 2023-11-27T15:28:58.916 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_device", 2023-11-27T15:28:58.916 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.916 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.917 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.917 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:58.917 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.917 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.917 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.917 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-27T15:28:58.917 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.917 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.918 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.918 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.918 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_ident": { 2023-11-27T15:28:58.918 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_ident", 2023-11-27T15:28:58.918 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.918 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.918 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.918 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.919 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.919 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.919 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.919 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-27T15:28:58.919 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.919 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.919 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.919 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.920 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_perf": { 2023-11-27T15:28:58.920 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_perf", 2023-11-27T15:28:58.920 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.920 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.920 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.920 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.920 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.920 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.920 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.921 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-27T15:28:58.921 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.921 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.921 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.921 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.921 INFO:teuthology.orchestra.run.smithi133.stdout: "contact": { 2023-11-27T15:28:58.921 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "contact", 2023-11-27T15:28:58.922 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.922 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.922 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.922 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.922 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.922 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.922 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.922 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.923 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.923 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.923 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.923 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.923 INFO:teuthology.orchestra.run.smithi133.stdout: "description": { 2023-11-27T15:28:58.923 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "description", 2023-11-27T15:28:58.923 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.923 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.924 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.924 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.924 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.924 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.924 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.924 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.924 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.924 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.925 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.925 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.925 INFO:teuthology.orchestra.run.smithi133.stdout: "device_url": { 2023-11-27T15:28:58.925 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_url", 2023-11-27T15:28:58.925 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.925 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.925 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.925 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-27T15:28:58.926 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.926 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.926 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.926 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.926 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.926 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.926 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.926 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.927 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2023-11-27T15:28:58.927 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2023-11-27T15:28:58.927 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.927 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.927 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.927 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.927 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.928 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.928 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.928 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.928 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.928 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.928 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.928 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.928 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-11-27T15:28:58.929 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-11-27T15:28:58.929 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.929 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.929 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.929 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "24", 2023-11-27T15:28:58.929 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "8", 2023-11-27T15:28:58.929 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.930 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.930 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.930 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.930 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.930 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.930 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.930 INFO:teuthology.orchestra.run.smithi133.stdout: "last_opt_revision": { 2023-11-27T15:28:58.930 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "last_opt_revision", 2023-11-27T15:28:58.931 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.931 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.931 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.931 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1", 2023-11-27T15:28:58.931 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.931 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.931 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.931 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.932 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.932 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.932 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.932 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.932 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard": { 2023-11-27T15:28:58.932 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard", 2023-11-27T15:28:58.932 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.933 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.933 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.933 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.933 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.933 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.933 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.933 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.933 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.934 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.934 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.934 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.934 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard_description": { 2023-11-27T15:28:58.934 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard_description", 2023-11-27T15:28:58.934 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.934 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.934 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.935 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.935 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.935 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.935 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.935 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.935 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.935 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.936 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.936 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.936 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.936 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.936 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.936 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.936 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.936 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.937 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.937 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.937 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.937 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.937 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.937 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.937 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.937 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.938 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.938 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.938 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.938 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.938 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.938 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.938 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.939 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.939 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.939 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.939 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.939 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.939 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.939 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.939 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.940 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.940 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.940 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.940 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.940 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.940 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.940 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.940 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.941 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.941 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.941 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.941 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.941 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.941 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.941 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.941 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.942 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.942 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.942 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.942 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.942 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.942 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.942 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.943 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.943 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.943 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.943 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.943 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.943 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.943 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.943 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.944 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.944 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.944 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.944 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.944 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.944 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.944 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.944 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.945 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.945 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.945 INFO:teuthology.orchestra.run.smithi133.stdout: "organization": { 2023-11-27T15:28:58.945 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "organization", 2023-11-27T15:28:58.945 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.945 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.945 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.946 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.946 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.946 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.946 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.946 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.946 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.946 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.946 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.947 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.947 INFO:teuthology.orchestra.run.smithi133.stdout: "proxy": { 2023-11-27T15:28:58.947 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "proxy", 2023-11-27T15:28:58.947 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.947 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.947 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.947 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.948 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.948 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.948 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.948 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.948 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.948 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.948 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.948 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.948 INFO:teuthology.orchestra.run.smithi133.stdout: "url": { 2023-11-27T15:28:58.949 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url", 2023-11-27T15:28:58.949 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.949 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.949 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.949 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-27T15:28:58.949 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.949 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.949 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.950 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.950 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.950 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.950 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.950 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.950 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.950 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.951 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.951 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "test_orchestrator", 2023-11-27T15:28:58.951 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.951 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.951 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.951 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.951 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.951 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.952 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.952 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.952 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.952 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.952 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.952 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.952 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.952 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.953 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.953 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.953 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.953 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.953 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.953 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.953 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.953 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.954 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.954 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.954 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.954 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.954 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.954 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.954 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.954 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.955 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.955 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.955 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.955 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.955 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.955 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.955 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.955 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.956 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.956 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.956 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.956 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.956 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.956 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.956 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.957 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.957 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.957 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.957 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.957 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.957 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.957 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.957 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.958 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.958 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.958 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.958 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.958 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.958 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.958 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.958 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.959 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.959 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.959 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.959 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.959 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.959 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.959 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.959 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.960 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.960 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.960 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.960 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.960 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.960 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.960 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.961 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "volumes", 2023-11-27T15:28:58.961 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.961 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.961 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.961 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.961 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.961 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.961 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.962 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.962 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.962 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.962 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.962 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.962 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.962 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.962 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.963 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.963 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.963 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.963 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.963 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.963 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.963 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.963 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.964 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.964 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.964 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.964 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.964 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.964 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.964 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.964 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.965 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.965 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.965 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.965 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.965 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.965 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.965 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.965 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.966 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.966 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.966 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.966 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.966 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.966 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.966 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.966 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.967 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.967 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.967 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.967 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.967 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.967 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.967 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.968 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.968 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.968 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.968 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.968 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.968 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.968 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.968 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.969 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.969 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.969 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.969 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.969 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.969 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.969 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.970 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.970 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.970 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.970 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.970 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_clones": { 2023-11-27T15:28:58.970 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_clones", 2023-11-27T15:28:58.970 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.970 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.971 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.971 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4", 2023-11-27T15:28:58.971 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.971 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.971 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.971 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-27T15:28:58.971 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.971 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.972 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.972 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.972 INFO:teuthology.orchestra.run.smithi133.stdout: "snapshot_clone_delay": { 2023-11-27T15:28:58.972 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snapshot_clone_delay", 2023-11-27T15:28:58.972 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.972 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.972 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.972 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-11-27T15:28:58.973 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.973 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.973 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.973 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-27T15:28:58.973 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.973 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.973 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.973 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.974 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.974 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.974 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.974 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix", 2023-11-27T15:28:58.974 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.974 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.974 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.975 INFO:teuthology.orchestra.run.smithi133.stdout: "discovery_interval": { 2023-11-27T15:28:58.975 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "discovery_interval", 2023-11-27T15:28:58.975 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-11-27T15:28:58.975 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.975 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.975 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "100", 2023-11-27T15:28:58.975 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.975 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.976 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.976 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.976 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.976 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.976 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.976 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.976 INFO:teuthology.orchestra.run.smithi133.stdout: "identifier": { 2023-11-27T15:28:58.976 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "identifier", 2023-11-27T15:28:58.977 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.977 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.977 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.977 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.977 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.977 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.977 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.978 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.978 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.978 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.978 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.978 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.978 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-11-27T15:28:58.978 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-11-27T15:28:58.979 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.979 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.979 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.979 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-11-27T15:28:58.979 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.979 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.979 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.979 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.980 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.980 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.980 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.980 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.980 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.980 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.980 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.980 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.981 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.981 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.981 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.981 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.981 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.981 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.981 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.981 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.982 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.982 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.982 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.982 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.982 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.982 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.982 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.982 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.983 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.983 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:58.983 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:58.983 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.983 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.983 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.983 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.984 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.984 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.984 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.984 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.984 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.984 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.984 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.984 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.985 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:58.985 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:58.985 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.985 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.985 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.985 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:58.985 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.985 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.986 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.986 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.986 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.986 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.986 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:58.986 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:58.986 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:58.986 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:58.987 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.987 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.987 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.987 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.987 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.987 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:58.987 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:58.987 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:58.988 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.988 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.988 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:58.988 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.988 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.988 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.988 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.988 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.989 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.989 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.989 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.989 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_host": { 2023-11-27T15:28:58.989 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_host", 2023-11-27T15:28:58.989 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.989 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.989 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.990 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.990 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.990 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.990 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.990 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.990 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.990 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.991 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.991 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.991 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_port": { 2023-11-27T15:28:58.991 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_port", 2023-11-27T15:28:58.991 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:58.991 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.991 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.991 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10051", 2023-11-27T15:28:58.992 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.992 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.992 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.992 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.992 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.992 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.992 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.992 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.993 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_sender": { 2023-11-27T15:28:58.993 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_sender", 2023-11-27T15:28:58.993 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.993 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.993 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:58.993 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-27T15:28:58.993 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.993 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.994 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.994 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:58.994 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.994 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.994 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.994 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.994 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.994 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:58.995 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-11-27T15:28:58.995 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.995 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.995 INFO:teuthology.orchestra.run.smithi133.stdout: "gid": 24110, 2023-11-27T15:28:58.995 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "y", 2023-11-27T15:28:58.995 INFO:teuthology.orchestra.run.smithi133.stdout: "mgr_features": 4540138322906710015, 2023-11-27T15:28:58.995 INFO:teuthology.orchestra.run.smithi133.stdout: "available_modules": [ 2023-11-27T15:28:58.995 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:58.996 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alerts", 2023-11-27T15:28:58.996 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:58.996 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:58.996 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:58.996 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-11-27T15:28:58.996 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-11-27T15:28:58.996 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:58.996 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.997 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.997 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-11-27T15:28:58.997 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.997 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.997 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:58.997 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How frequently to reexamine health status", 2023-11-27T15:28:58.997 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:58.998 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:58.998 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:58.998 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:58.998 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:58.998 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:58.998 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:58.998 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:58.999 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:58.999 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:58.999 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:58.999 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:58.999 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:58.999 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:58.999 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:58.999 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:58.999 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.000 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.000 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.000 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.000 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.000 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.000 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.000 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.001 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.001 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.001 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.001 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.001 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.001 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.001 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.001 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.002 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.002 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.002 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.002 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.002 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.002 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.002 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.003 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.003 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.003 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.003 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.003 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.003 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.003 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.003 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.004 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.004 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.004 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.004 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.004 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.004 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.004 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.004 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.005 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.005 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.005 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.005 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.005 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.005 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.005 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.005 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.006 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.006 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.006 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.006 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.006 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.006 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.006 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.006 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.007 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.007 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.007 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.007 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_destination": { 2023-11-27T15:28:59.007 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_destination", 2023-11-27T15:28:59.007 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.007 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.008 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.008 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.008 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.008 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.008 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.008 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email address to send alerts to", 2023-11-27T15:28:59.008 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.008 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.009 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.009 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.009 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_from_name": { 2023-11-27T15:28:59.009 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_from_name", 2023-11-27T15:28:59.009 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.009 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.009 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.009 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "Ceph", 2023-11-27T15:28:59.010 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.010 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.010 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.010 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email From: name", 2023-11-27T15:28:59.010 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.010 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.010 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.010 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.011 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_host": { 2023-11-27T15:28:59.011 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_host", 2023-11-27T15:28:59.011 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.011 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.011 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.011 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.011 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.011 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.012 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.012 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP server", 2023-11-27T15:28:59.012 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.012 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.012 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.012 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.012 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_password": { 2023-11-27T15:28:59.012 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_password", 2023-11-27T15:28:59.013 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.013 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.013 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.013 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.013 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.013 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.013 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.014 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Password to authenticate with", 2023-11-27T15:28:59.014 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.014 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.014 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.014 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.014 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_port": { 2023-11-27T15:28:59.014 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_port", 2023-11-27T15:28:59.014 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.015 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.015 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.015 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "465", 2023-11-27T15:28:59.015 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.015 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.015 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.015 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP port", 2023-11-27T15:28:59.015 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.016 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.016 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.016 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.016 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_sender": { 2023-11-27T15:28:59.016 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_sender", 2023-11-27T15:28:59.016 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.016 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.016 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.017 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.017 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.017 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.017 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.017 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP envelope sender", 2023-11-27T15:28:59.017 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.017 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.017 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.018 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.018 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_ssl": { 2023-11-27T15:28:59.018 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_ssl", 2023-11-27T15:28:59.018 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.018 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.018 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.018 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.018 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.019 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.019 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.019 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-27T15:28:59.019 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.019 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.019 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.019 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.019 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_user": { 2023-11-27T15:28:59.020 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_user", 2023-11-27T15:28:59.020 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.020 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.020 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.020 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.020 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.020 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.020 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.021 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "User to authenticate as", 2023-11-27T15:28:59.021 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.021 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.021 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.021 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.021 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.021 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.022 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.022 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "balancer", 2023-11-27T15:28:59.022 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.022 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.022 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.022 INFO:teuthology.orchestra.run.smithi133.stdout: "active": { 2023-11-27T15:28:59.022 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "active", 2023-11-27T15:28:59.022 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.023 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.023 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.023 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.023 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.023 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.023 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.023 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically balance PGs across cluster", 2023-11-27T15:28:59.023 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.024 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.024 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.024 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.024 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_time": { 2023-11-27T15:28:59.024 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_time", 2023-11-27T15:28:59.024 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.024 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.024 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.025 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0000", 2023-11-27T15:28:59.025 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.025 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.025 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.025 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "beginning time of day to automatically balance", 2023-11-27T15:28:59.025 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-27T15:28:59.025 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.025 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.026 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.026 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_weekday": { 2023-11-27T15:28:59.026 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_weekday", 2023-11-27T15:28:59.026 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-11-27T15:28:59.026 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.026 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.026 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-11-27T15:28:59.027 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2023-11-27T15:28:59.027 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2023-11-27T15:28:59.027 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.027 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-27T15:28:59.027 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-27T15:28:59.027 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.027 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.027 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.028 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_max_iterations": { 2023-11-27T15:28:59.028 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_max_iterations", 2023-11-27T15:28:59.028 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-11-27T15:28:59.028 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.028 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.028 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "25", 2023-11-27T15:28:59.028 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-11-27T15:28:59.028 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "250", 2023-11-27T15:28:59.029 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.029 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-27T15:28:59.029 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.029 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.029 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.029 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.029 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_metrics": { 2023-11-27T15:28:59.029 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_metrics", 2023-11-27T15:28:59.030 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.030 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.030 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.030 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "pgs,objects,bytes", 2023-11-27T15:28:59.030 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.030 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.030 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.030 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-27T15:28:59.031 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-27T15:28:59.031 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.031 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.031 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.031 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_step": { 2023-11-27T15:28:59.031 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_step", 2023-11-27T15:28:59.031 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:28:59.031 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.032 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.032 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.5", 2023-11-27T15:28:59.032 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0.001", 2023-11-27T15:28:59.032 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "0.999", 2023-11-27T15:28:59.032 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.032 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "aggressiveness of optimization", 2023-11-27T15:28:59.032 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-27T15:28:59.032 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.033 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.033 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.033 INFO:teuthology.orchestra.run.smithi133.stdout: "end_time": { 2023-11-27T15:28:59.033 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_time", 2023-11-27T15:28:59.033 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.033 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.033 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.034 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2359", 2023-11-27T15:28:59.034 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.034 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.034 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.034 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ending time of day to automatically balance", 2023-11-27T15:28:59.034 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-27T15:28:59.034 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.034 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.035 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.035 INFO:teuthology.orchestra.run.smithi133.stdout: "end_weekday": { 2023-11-27T15:28:59.035 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_weekday", 2023-11-27T15:28:59.035 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-11-27T15:28:59.035 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.035 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.035 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-11-27T15:28:59.035 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2023-11-27T15:28:59.036 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2023-11-27T15:28:59.036 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.036 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-27T15:28:59.036 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-27T15:28:59.036 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.036 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.036 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.036 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.037 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.037 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.037 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.037 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.037 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.037 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.037 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.037 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.038 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.038 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.038 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.038 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.038 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.038 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.038 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.039 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.039 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.039 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.039 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.039 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.039 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.039 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.039 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.040 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.040 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.040 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.040 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.040 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.040 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.040 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.040 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.041 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.041 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.041 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.041 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.041 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.041 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.041 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.041 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.042 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.042 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.042 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.042 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.042 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.042 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.042 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.043 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.043 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.043 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.043 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.043 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.043 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.043 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.043 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.044 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.044 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.044 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.044 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.044 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.044 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.044 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.044 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.045 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.045 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.045 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.045 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.045 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.045 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.045 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.046 INFO:teuthology.orchestra.run.smithi133.stdout: "min_score": { 2023-11-27T15:28:59.046 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_score", 2023-11-27T15:28:59.046 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:28:59.046 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.046 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.046 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-11-27T15:28:59.046 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.046 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.047 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.047 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-27T15:28:59.047 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.047 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.047 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.047 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.047 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2023-11-27T15:28:59.048 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2023-11-27T15:28:59.048 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.048 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.048 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.048 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "upmap", 2023-11-27T15:28:59.048 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.048 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.048 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.049 INFO:teuthology.orchestra.run.smithi133.stdout: "crush-compat", 2023-11-27T15:28:59.049 INFO:teuthology.orchestra.run.smithi133.stdout: "none", 2023-11-27T15:28:59.049 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap" 2023-11-27T15:28:59.049 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.049 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Balancer mode", 2023-11-27T15:28:59.049 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.049 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.050 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.050 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.050 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_ids": { 2023-11-27T15:28:59.050 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_ids", 2023-11-27T15:28:59.050 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.050 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.050 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.050 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.051 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.051 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.051 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.051 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-27T15:28:59.051 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.051 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.051 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.051 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.052 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-11-27T15:28:59.052 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-11-27T15:28:59.052 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.052 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.052 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.052 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-11-27T15:28:59.052 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.052 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.053 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.053 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-27T15:28:59.053 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.053 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.053 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.053 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.053 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_deviation": { 2023-11-27T15:28:59.053 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_deviation", 2023-11-27T15:28:59.054 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.054 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.054 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.054 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-11-27T15:28:59.054 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-11-27T15:28:59.054 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.054 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.054 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-27T15:28:59.055 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-27T15:28:59.055 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.055 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.055 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.055 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_optimizations": { 2023-11-27T15:28:59.055 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_optimizations", 2023-11-27T15:28:59.055 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-11-27T15:28:59.055 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.056 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.056 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:28:59.056 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.056 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.056 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.056 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-27T15:28:59.056 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.056 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.057 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.057 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.057 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.057 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.057 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.057 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephadm", 2023-11-27T15:28:59.057 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.058 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.058 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.058 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_down_multiplier": { 2023-11-27T15:28:59.058 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_down_multiplier", 2023-11-27T15:28:59.058 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:28:59.058 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.058 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.058 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2023-11-27T15:28:59.059 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.059 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.059 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.059 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-27T15:28:59.059 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.059 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.059 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.059 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.060 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_refresh_rate": { 2023-11-27T15:28:59.060 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_refresh_rate", 2023-11-27T15:28:59.060 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.060 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.060 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.060 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "20", 2023-11-27T15:28:59.060 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.061 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.061 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.061 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-27T15:28:59.061 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.061 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.061 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.061 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.061 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_starting_port": { 2023-11-27T15:28:59.062 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_starting_port", 2023-11-27T15:28:59.062 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.062 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.062 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.062 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4721", 2023-11-27T15:28:59.062 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.062 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.062 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.063 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-27T15:28:59.063 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.063 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.063 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.063 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.063 INFO:teuthology.orchestra.run.smithi133.stdout: "alertmanager_web_password": { 2023-11-27T15:28:59.063 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alertmanager_web_password", 2023-11-27T15:28:59.063 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.064 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.064 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.064 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-11-27T15:28:59.064 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.064 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.064 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.064 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Alertmanager web password", 2023-11-27T15:28:59.064 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.065 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.065 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.065 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.065 INFO:teuthology.orchestra.run.smithi133.stdout: "alertmanager_web_user": { 2023-11-27T15:28:59.065 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alertmanager_web_user", 2023-11-27T15:28:59.065 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.065 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.065 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.066 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-11-27T15:28:59.066 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.066 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.066 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.066 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Alertmanager web user", 2023-11-27T15:28:59.066 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.066 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.066 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.067 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.067 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_ptrace": { 2023-11-27T15:28:59.067 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_ptrace", 2023-11-27T15:28:59.067 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.067 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.067 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.067 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.068 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.068 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.068 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.068 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-27T15:28:59.068 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-27T15:28:59.068 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.068 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.068 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.069 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_interval": { 2023-11-27T15:28:59.069 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_interval", 2023-11-27T15:28:59.069 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.069 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.069 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.069 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-11-27T15:28:59.069 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.069 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.070 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.070 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-27T15:28:59.070 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.070 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.070 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.070 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.070 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_memory_target_ratio": { 2023-11-27T15:28:59.070 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_memory_target_ratio", 2023-11-27T15:28:59.071 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:28:59.071 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.071 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.071 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.7", 2023-11-27T15:28:59.071 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.071 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.071 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.072 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-27T15:28:59.072 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.072 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.072 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.072 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.072 INFO:teuthology.orchestra.run.smithi133.stdout: "cgroups_split": { 2023-11-27T15:28:59.072 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cgroups_split", 2023-11-27T15:28:59.072 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.073 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.073 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.073 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.073 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.073 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.073 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.073 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-27T15:28:59.073 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.074 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.074 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.074 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.074 INFO:teuthology.orchestra.run.smithi133.stdout: "config_checks_enabled": { 2023-11-27T15:28:59.074 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_checks_enabled", 2023-11-27T15:28:59.074 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.074 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.074 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.075 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.075 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.075 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.075 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.075 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-27T15:28:59.075 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.075 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.076 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.076 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.076 INFO:teuthology.orchestra.run.smithi133.stdout: "config_dashboard": { 2023-11-27T15:28:59.076 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_dashboard", 2023-11-27T15:28:59.076 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.076 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.076 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.076 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.077 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.077 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.077 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.077 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-27T15:28:59.077 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.077 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.077 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.077 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.078 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_alertmanager": { 2023-11-27T15:28:59.078 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_alertmanager", 2023-11-27T15:28:59.078 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.078 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.078 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.078 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-27T15:28:59.078 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.079 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.079 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.079 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:59.079 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.079 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.079 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.079 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.079 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_base": { 2023-11-27T15:28:59.080 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_base", 2023-11-27T15:28:59.080 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.080 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.080 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.080 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-27T15:28:59.080 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.080 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.080 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.081 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Container image name, without the tag", 2023-11-27T15:28:59.081 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.081 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.081 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.081 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.081 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_elasticsearch": { 2023-11-27T15:28:59.081 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_elasticsearch", 2023-11-27T15:28:59.081 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.082 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.082 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.082 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-27T15:28:59.082 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.082 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.082 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.082 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "elasticsearch container image", 2023-11-27T15:28:59.082 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.083 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.083 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.083 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.083 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_grafana": { 2023-11-27T15:28:59.083 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_grafana", 2023-11-27T15:28:59.083 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.083 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.083 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.084 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-27T15:28:59.084 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.084 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.084 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.084 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:59.084 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.084 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.084 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.085 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.085 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_haproxy": { 2023-11-27T15:28:59.085 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_haproxy", 2023-11-27T15:28:59.085 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.085 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.085 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.085 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-27T15:28:59.085 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.086 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.086 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.086 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "HAproxy container image", 2023-11-27T15:28:59.086 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.086 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.086 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.086 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.086 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_agent": { 2023-11-27T15:28:59.087 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_agent", 2023-11-27T15:28:59.087 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.087 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.087 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.087 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-27T15:28:59.087 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.087 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.088 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.088 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger agent container image", 2023-11-27T15:28:59.088 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.088 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.088 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.088 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.088 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_collector": { 2023-11-27T15:28:59.088 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_collector", 2023-11-27T15:28:59.089 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.089 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.089 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.089 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-27T15:28:59.089 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.089 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.089 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.089 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger collector container image", 2023-11-27T15:28:59.090 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.090 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.090 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.090 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.090 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_query": { 2023-11-27T15:28:59.090 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_query", 2023-11-27T15:28:59.090 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.090 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.091 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.091 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-27T15:28:59.091 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.091 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.091 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.091 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger query container image", 2023-11-27T15:28:59.091 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.092 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.092 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.092 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.092 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_keepalived": { 2023-11-27T15:28:59.092 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_keepalived", 2023-11-27T15:28:59.092 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.092 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.092 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.093 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-27T15:28:59.093 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.093 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.093 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.093 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Keepalived container image", 2023-11-27T15:28:59.093 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.093 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.093 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.094 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.094 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_loki": { 2023-11-27T15:28:59.094 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_loki", 2023-11-27T15:28:59.094 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.094 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.094 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.094 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-27T15:28:59.095 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.095 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.095 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.095 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Loki container image", 2023-11-27T15:28:59.095 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.095 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.095 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.095 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.096 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_node_exporter": { 2023-11-27T15:28:59.096 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_node_exporter", 2023-11-27T15:28:59.096 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.096 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.096 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.096 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-27T15:28:59.096 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.096 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.097 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.097 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:59.097 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.097 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.097 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.097 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.097 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_prometheus": { 2023-11-27T15:28:59.097 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_prometheus", 2023-11-27T15:28:59.098 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.098 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.098 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.098 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-27T15:28:59.098 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.098 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.098 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.099 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:59.099 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.099 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.099 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.099 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.099 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_promtail": { 2023-11-27T15:28:59.099 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_promtail", 2023-11-27T15:28:59.099 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.100 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.100 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.100 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-27T15:28:59.100 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.100 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.100 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.100 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Promtail container image", 2023-11-27T15:28:59.100 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.101 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.101 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.101 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.101 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_snmp_gateway": { 2023-11-27T15:28:59.101 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_snmp_gateway", 2023-11-27T15:28:59.101 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.101 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.101 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.102 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-27T15:28:59.102 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.102 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.102 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.102 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SNMP Gateway container image", 2023-11-27T15:28:59.102 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.102 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.102 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.103 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.103 INFO:teuthology.orchestra.run.smithi133.stdout: "container_init": { 2023-11-27T15:28:59.103 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_init", 2023-11-27T15:28:59.103 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.103 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.103 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.103 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.103 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.104 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.104 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.104 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Run podman/docker with `--init`", 2023-11-27T15:28:59.104 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.104 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.104 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.104 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.104 INFO:teuthology.orchestra.run.smithi133.stdout: "daemon_cache_timeout": { 2023-11-27T15:28:59.105 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "daemon_cache_timeout", 2023-11-27T15:28:59.105 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.105 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.105 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.105 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-11-27T15:28:59.105 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.105 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.105 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.106 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-27T15:28:59.106 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.106 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.106 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.106 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.106 INFO:teuthology.orchestra.run.smithi133.stdout: "default_cephadm_command_timeout": { 2023-11-27T15:28:59.106 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_cephadm_command_timeout", 2023-11-27T15:28:59.107 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.107 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.107 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.107 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "900", 2023-11-27T15:28:59.107 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.107 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.107 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.107 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-27T15:28:59.108 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.108 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.108 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.108 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.108 INFO:teuthology.orchestra.run.smithi133.stdout: "default_registry": { 2023-11-27T15:28:59.108 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_registry", 2023-11-27T15:28:59.108 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.108 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.109 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.109 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io", 2023-11-27T15:28:59.109 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.109 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.109 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.109 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-27T15:28:59.109 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.109 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.110 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.110 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.110 INFO:teuthology.orchestra.run.smithi133.stdout: "device_cache_timeout": { 2023-11-27T15:28:59.110 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_cache_timeout", 2023-11-27T15:28:59.110 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.110 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.110 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.110 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1800", 2023-11-27T15:28:59.111 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.111 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.111 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.111 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache device inventory", 2023-11-27T15:28:59.111 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.111 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.111 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.112 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.112 INFO:teuthology.orchestra.run.smithi133.stdout: "device_enhanced_scan": { 2023-11-27T15:28:59.112 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_enhanced_scan", 2023-11-27T15:28:59.112 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.112 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.112 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.112 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.112 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.113 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.113 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.113 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-27T15:28:59.113 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.113 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.113 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.113 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.113 INFO:teuthology.orchestra.run.smithi133.stdout: "facts_cache_timeout": { 2023-11-27T15:28:59.114 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "facts_cache_timeout", 2023-11-27T15:28:59.114 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.114 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.114 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.114 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-11-27T15:28:59.114 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.114 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.115 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.115 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache host facts data", 2023-11-27T15:28:59.115 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.115 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.115 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.115 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.115 INFO:teuthology.orchestra.run.smithi133.stdout: "host_check_interval": { 2023-11-27T15:28:59.115 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "host_check_interval", 2023-11-27T15:28:59.116 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.116 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.116 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.116 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-11-27T15:28:59.116 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.116 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.116 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.116 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to perform a host check", 2023-11-27T15:28:59.117 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.117 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.117 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.117 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.117 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.117 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.117 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.117 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.118 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.118 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.118 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.118 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.118 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.118 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.118 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.119 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.119 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.119 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.119 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.119 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.119 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.119 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.119 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.120 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.120 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.120 INFO:teuthology.orchestra.run.smithi133.stdout: "log_refresh_metadata": { 2023-11-27T15:28:59.120 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_refresh_metadata", 2023-11-27T15:28:59.120 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.120 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.120 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.120 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.121 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.121 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.121 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.121 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-27T15:28:59.121 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.121 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.121 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.121 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.122 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.122 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.122 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.122 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.122 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.122 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.122 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.123 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.123 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.123 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-27T15:28:59.123 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.123 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.123 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.123 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.123 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.124 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.124 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.124 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.124 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.124 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.124 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.124 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.124 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.125 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.125 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.125 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.125 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.125 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.125 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.125 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.125 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.126 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.126 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.126 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.126 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.126 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.126 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.126 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.126 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.127 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.127 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.127 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.127 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.127 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.127 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.127 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.127 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.128 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.128 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.128 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-27T15:28:59.128 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-27T15:28:59.128 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.128 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.128 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.128 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.129 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.129 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.129 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.129 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-27T15:28:59.129 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.129 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.129 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.130 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.130 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-27T15:28:59.130 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-27T15:28:59.130 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.130 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.130 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.130 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "*", 2023-11-27T15:28:59.130 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.131 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.131 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.131 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-27T15:28:59.131 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.131 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.131 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.131 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.131 INFO:teuthology.orchestra.run.smithi133.stdout: "max_count_per_host": { 2023-11-27T15:28:59.132 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_count_per_host", 2023-11-27T15:28:59.132 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.132 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.132 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.132 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:28:59.132 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.132 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.132 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.133 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of daemons per service per host", 2023-11-27T15:28:59.133 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.133 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.133 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.133 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.133 INFO:teuthology.orchestra.run.smithi133.stdout: "max_osd_draining_count": { 2023-11-27T15:28:59.133 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_osd_draining_count", 2023-11-27T15:28:59.133 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.134 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.134 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.134 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:28:59.134 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.134 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.134 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.134 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-27T15:28:59.135 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.135 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.135 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.135 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.135 INFO:teuthology.orchestra.run.smithi133.stdout: "migration_current": { 2023-11-27T15:28:59.135 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "migration_current", 2023-11-27T15:28:59.135 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.135 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.136 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.136 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.136 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.136 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.136 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.136 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "internal - do not modify", 2023-11-27T15:28:59.136 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.136 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.137 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.137 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.137 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2023-11-27T15:28:59.137 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2023-11-27T15:28:59.137 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.137 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.137 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.138 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "root", 2023-11-27T15:28:59.138 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.138 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.138 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.138 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm-package", 2023-11-27T15:28:59.138 INFO:teuthology.orchestra.run.smithi133.stdout: "root" 2023-11-27T15:28:59.138 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.138 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "mode for remote execution of cephadm", 2023-11-27T15:28:59.139 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.139 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.139 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.139 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.139 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_alerts_path": { 2023-11-27T15:28:59.139 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_alerts_path", 2023-11-27T15:28:59.139 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.139 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.140 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.140 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-27T15:28:59.140 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.140 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.140 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.140 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-27T15:28:59.140 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.140 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.141 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.141 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.141 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_web_password": { 2023-11-27T15:28:59.141 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_web_password", 2023-11-27T15:28:59.141 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.141 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.141 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.142 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-11-27T15:28:59.142 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.142 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.142 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.142 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus web password", 2023-11-27T15:28:59.142 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.142 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.142 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.143 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.143 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_web_user": { 2023-11-27T15:28:59.143 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_web_user", 2023-11-27T15:28:59.143 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.143 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.143 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.143 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-11-27T15:28:59.144 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.144 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.144 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.144 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus web user", 2023-11-27T15:28:59.144 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.144 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.144 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.144 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.145 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_insecure": { 2023-11-27T15:28:59.145 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_insecure", 2023-11-27T15:28:59.145 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.145 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.145 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.145 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.145 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.145 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.146 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.146 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-27T15:28:59.146 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.146 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.146 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.146 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.146 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_password": { 2023-11-27T15:28:59.147 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_password", 2023-11-27T15:28:59.147 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.147 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.147 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.147 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.147 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.147 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.147 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.148 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-27T15:28:59.148 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.148 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.148 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.148 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.148 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_url": { 2023-11-27T15:28:59.148 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_url", 2023-11-27T15:28:59.149 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.149 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.149 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.149 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.149 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.149 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.149 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.149 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-27T15:28:59.150 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.150 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.150 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.150 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.150 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_username": { 2023-11-27T15:28:59.150 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_username", 2023-11-27T15:28:59.150 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.151 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.151 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.151 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.151 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.151 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.151 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.151 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-27T15:28:59.151 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.152 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.152 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.152 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.152 INFO:teuthology.orchestra.run.smithi133.stdout: "secure_monitoring_stack": { 2023-11-27T15:28:59.152 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "secure_monitoring_stack", 2023-11-27T15:28:59.152 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.152 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.152 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.153 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.153 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.153 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.153 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.153 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-27T15:28:59.153 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.153 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.153 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.154 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.154 INFO:teuthology.orchestra.run.smithi133.stdout: "service_discovery_port": { 2023-11-27T15:28:59.154 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "service_discovery_port", 2023-11-27T15:28:59.154 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.154 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.154 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.154 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8765", 2023-11-27T15:28:59.154 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.155 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.155 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.155 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "cephadm service discovery port", 2023-11-27T15:28:59.155 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.155 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.155 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.155 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.155 INFO:teuthology.orchestra.run.smithi133.stdout: "ssh_config_file": { 2023-11-27T15:28:59.156 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssh_config_file", 2023-11-27T15:28:59.156 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.156 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.156 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.156 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.156 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.156 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.156 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.157 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-27T15:28:59.157 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.157 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.157 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.157 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.157 INFO:teuthology.orchestra.run.smithi133.stdout: "use_agent": { 2023-11-27T15:28:59.157 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_agent", 2023-11-27T15:28:59.157 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.158 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.158 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.158 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.158 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.158 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.158 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.158 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-27T15:28:59.159 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.159 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.159 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.159 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.159 INFO:teuthology.orchestra.run.smithi133.stdout: "use_repo_digest": { 2023-11-27T15:28:59.159 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_repo_digest", 2023-11-27T15:28:59.159 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.159 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.160 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.160 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.160 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.160 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.160 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.160 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-27T15:28:59.160 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.161 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.161 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.161 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.161 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_failed_host_check": { 2023-11-27T15:28:59.161 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_failed_host_check", 2023-11-27T15:28:59.161 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.161 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.161 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.162 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.162 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.162 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.162 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.162 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if the host check fails", 2023-11-27T15:28:59.162 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.162 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.162 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.163 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.163 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_daemons": { 2023-11-27T15:28:59.163 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_daemons", 2023-11-27T15:28:59.163 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.163 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.163 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.163 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.163 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.164 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.164 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.164 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-27T15:28:59.164 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.164 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.164 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.164 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.164 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_hosts": { 2023-11-27T15:28:59.165 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_hosts", 2023-11-27T15:28:59.165 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.165 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.165 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.165 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.165 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.165 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.165 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.166 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-27T15:28:59.166 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.166 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.166 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.166 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.166 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.166 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.166 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.167 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crash", 2023-11-27T15:28:59.167 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.167 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.167 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.167 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.167 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.167 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.168 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.168 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.168 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.168 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.168 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.168 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.168 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.168 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.169 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.169 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.169 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.169 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.169 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.169 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.169 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.169 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.170 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.170 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.170 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.170 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.170 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.170 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.170 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.170 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.171 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.171 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.171 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.171 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.171 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.171 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.171 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.171 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.172 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.172 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.172 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.172 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.172 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.172 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.172 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.172 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.173 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.173 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.173 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.173 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.173 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.173 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.173 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.173 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.174 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.174 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.174 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.174 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.174 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.174 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.174 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.175 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.175 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.175 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.175 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.175 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.175 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.175 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.175 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.176 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.176 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.176 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.176 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.176 INFO:teuthology.orchestra.run.smithi133.stdout: "retain_interval": { 2023-11-27T15:28:59.176 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retain_interval", 2023-11-27T15:28:59.176 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.176 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.177 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.177 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "31536000", 2023-11-27T15:28:59.177 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.177 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.177 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.177 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-27T15:28:59.177 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.178 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.178 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.178 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.178 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_recent_interval": { 2023-11-27T15:28:59.178 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_recent_interval", 2023-11-27T15:28:59.178 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.178 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.178 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.179 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1209600", 2023-11-27T15:28:59.179 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.179 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.179 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.179 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-27T15:28:59.179 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.179 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.179 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.180 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.180 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.180 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.180 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.180 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dashboard", 2023-11-27T15:28:59.180 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.180 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.180 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.181 INFO:teuthology.orchestra.run.smithi133.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-27T15:28:59.181 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-27T15:28:59.181 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.181 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.181 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.181 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:28:59.181 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.181 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.182 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.182 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.182 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.182 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.182 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.182 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.182 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_HOST": { 2023-11-27T15:28:59.183 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-27T15:28:59.183 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.183 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.183 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.183 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.183 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.183 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.183 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.184 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.184 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.184 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.184 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.184 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.184 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-27T15:28:59.184 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-27T15:28:59.184 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.185 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.185 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.185 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.185 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.185 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.185 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.185 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.185 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.186 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.186 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.186 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.186 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_ENABLED": { 2023-11-27T15:28:59.186 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_ENABLED", 2023-11-27T15:28:59.186 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.186 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.187 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.187 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.187 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.187 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.187 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.187 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.187 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.187 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.188 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.188 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.188 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-27T15:28:59.188 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-27T15:28:59.188 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.188 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.188 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.188 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.189 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.189 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.189 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.189 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.189 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.189 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.189 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.189 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.190 INFO:teuthology.orchestra.run.smithi133.stdout: "ENABLE_BROWSABLE_API": { 2023-11-27T15:28:59.190 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-27T15:28:59.190 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.190 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.190 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.190 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.190 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.190 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.191 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.191 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.191 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.191 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.191 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.191 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.191 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-27T15:28:59.191 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-27T15:28:59.192 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.192 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.192 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.192 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.192 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.192 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.192 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.192 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.193 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.193 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.193 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.193 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.193 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-27T15:28:59.193 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-27T15:28:59.193 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.193 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.194 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.194 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.194 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.194 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.194 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.194 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.194 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.194 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.195 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.195 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.195 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-27T15:28:59.195 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-27T15:28:59.195 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.195 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.195 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.195 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.196 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.196 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.196 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.196 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.196 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.196 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.196 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.196 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.197 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-27T15:28:59.197 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-27T15:28:59.197 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.197 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.197 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.197 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.197 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.198 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.198 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.198 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.198 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.198 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.198 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.198 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.198 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-27T15:28:59.199 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-27T15:28:59.199 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.199 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.199 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.199 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.199 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.199 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.200 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.200 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.200 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.200 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.200 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.200 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.200 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-27T15:28:59.200 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-27T15:28:59.200 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.201 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.201 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.201 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.201 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.201 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.201 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.201 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.201 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.202 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.202 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.202 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.202 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-27T15:28:59.202 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-27T15:28:59.202 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.202 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.203 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.203 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.203 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.203 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.203 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.203 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.203 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.203 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.204 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.204 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.204 INFO:teuthology.orchestra.run.smithi133.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-27T15:28:59.204 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-27T15:28:59.204 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.204 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.204 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.204 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.205 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.205 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.205 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.205 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.205 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.205 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.205 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.205 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.206 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_PASSWORD": { 2023-11-27T15:28:59.206 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-27T15:28:59.206 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.206 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.206 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.206 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-11-27T15:28:59.206 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.206 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.207 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.207 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.207 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.207 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.207 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.207 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.207 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-27T15:28:59.208 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-27T15:28:59.208 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.208 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.208 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.208 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.208 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.208 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.208 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.209 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.209 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.209 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.209 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.209 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.209 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_URL": { 2023-11-27T15:28:59.209 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_URL", 2023-11-27T15:28:59.209 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.210 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.210 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.210 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.210 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.210 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.210 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.210 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.210 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.211 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.211 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.211 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.211 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_USERNAME": { 2023-11-27T15:28:59.211 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-27T15:28:59.211 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.211 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.211 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.212 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-11-27T15:28:59.212 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.212 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.212 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.212 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.212 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.212 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.213 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.213 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.213 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-27T15:28:59.213 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-27T15:28:59.213 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.213 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.213 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.213 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.214 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.214 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.214 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.214 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.214 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.214 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.214 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.214 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.215 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-27T15:28:59.215 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-27T15:28:59.215 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.215 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.215 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.215 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.215 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.216 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.216 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.216 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.216 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.216 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.216 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.216 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.216 INFO:teuthology.orchestra.run.smithi133.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-27T15:28:59.217 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-27T15:28:59.217 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.217 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.217 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.217 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.217 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.217 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.217 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.218 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.218 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.218 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.218 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.218 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.218 INFO:teuthology.orchestra.run.smithi133.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-27T15:28:59.218 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-27T15:28:59.218 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.219 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.219 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.219 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.219 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.219 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.219 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.219 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.219 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.220 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.220 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.220 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.220 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_HOST": { 2023-11-27T15:28:59.220 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-27T15:28:59.220 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.220 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.220 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.221 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.221 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.221 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.221 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.221 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.221 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.221 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.221 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.222 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.222 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-27T15:28:59.222 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-27T15:28:59.222 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.222 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.222 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.222 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.222 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.223 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.223 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.223 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.223 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.223 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.223 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.223 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.223 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-27T15:28:59.224 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-27T15:28:59.224 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.224 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.224 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.224 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.224 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.224 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.224 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.225 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.225 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.225 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.225 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.225 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.225 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-27T15:28:59.225 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-27T15:28:59.225 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.226 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.226 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.226 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.226 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.226 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.226 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.226 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.226 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.227 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.227 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.227 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.227 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-27T15:28:59.227 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-27T15:28:59.227 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.227 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.227 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.228 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.228 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.228 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.228 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.228 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.228 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.228 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.228 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.229 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.229 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-27T15:28:59.229 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-27T15:28:59.229 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.229 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.229 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.229 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.230 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.230 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.230 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.230 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.230 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.230 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.230 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.231 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.231 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-27T15:28:59.231 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-27T15:28:59.231 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.231 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.231 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.231 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.231 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.232 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.232 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.232 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.232 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.232 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.232 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.232 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.232 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-27T15:28:59.233 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-27T15:28:59.233 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.233 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.233 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.233 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.233 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.233 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.234 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.234 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.234 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.234 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.234 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.234 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.234 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-27T15:28:59.234 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-27T15:28:59.235 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.235 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.235 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.235 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.235 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.235 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.235 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.235 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.236 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.236 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.236 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.236 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.236 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_ENABLED": { 2023-11-27T15:28:59.236 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-27T15:28:59.236 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.237 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.237 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.237 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.237 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.237 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.237 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.237 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.237 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.238 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.238 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.238 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.238 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-27T15:28:59.238 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-27T15:28:59.238 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.238 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.238 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.239 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-27T15:28:59.239 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.239 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.239 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.239 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.239 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.239 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.239 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.240 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.240 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-27T15:28:59.240 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-27T15:28:59.240 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.240 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.240 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.240 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:28:59.240 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.241 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.241 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.241 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.241 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.241 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.241 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.241 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.241 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-27T15:28:59.242 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-27T15:28:59.242 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.242 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.242 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.242 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8", 2023-11-27T15:28:59.242 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.242 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.243 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.243 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.243 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.243 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.243 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.243 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.243 INFO:teuthology.orchestra.run.smithi133.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-27T15:28:59.243 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-27T15:28:59.244 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.244 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.244 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.244 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "45", 2023-11-27T15:28:59.244 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.244 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.244 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.244 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.245 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.245 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.245 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.245 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.245 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ACCESS_KEY": { 2023-11-27T15:28:59.245 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-27T15:28:59.245 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.245 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.246 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.246 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.246 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.246 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.246 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.246 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.246 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.247 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.247 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.247 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.247 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-27T15:28:59.247 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-27T15:28:59.247 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.247 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.247 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.248 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-11-27T15:28:59.248 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.248 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.248 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.248 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.248 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.248 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.248 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.249 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.249 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SECRET_KEY": { 2023-11-27T15:28:59.249 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-27T15:28:59.249 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.249 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.249 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.249 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.250 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.250 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.250 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.250 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.250 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.250 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.250 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.250 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.251 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SSL_VERIFY": { 2023-11-27T15:28:59.251 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-27T15:28:59.251 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.251 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.251 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.251 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.251 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.251 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.252 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.252 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.252 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.252 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.252 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.252 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.252 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-27T15:28:59.252 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-27T15:28:59.253 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.253 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.253 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.253 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-11-27T15:28:59.253 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.253 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.253 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.253 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.254 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.254 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.254 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.254 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.254 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-27T15:28:59.254 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-27T15:28:59.254 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.255 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.255 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.255 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:28:59.255 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.255 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.255 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.255 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.255 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.256 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.256 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.256 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.256 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-27T15:28:59.256 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-27T15:28:59.256 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.256 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.256 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.257 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-11-27T15:28:59.257 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.257 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.257 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.257 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.257 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.257 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.257 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.258 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.258 INFO:teuthology.orchestra.run.smithi133.stdout: "cross_origin_url": { 2023-11-27T15:28:59.258 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cross_origin_url", 2023-11-27T15:28:59.258 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.258 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.258 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.258 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.258 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.259 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.259 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.259 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.259 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.259 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.259 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.259 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.259 INFO:teuthology.orchestra.run.smithi133.stdout: "crt_file": { 2023-11-27T15:28:59.260 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crt_file", 2023-11-27T15:28:59.260 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.260 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.260 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.260 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.260 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.260 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.260 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.261 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.261 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.261 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.261 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.261 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.261 INFO:teuthology.orchestra.run.smithi133.stdout: "debug": { 2023-11-27T15:28:59.261 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "debug", 2023-11-27T15:28:59.261 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.262 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.262 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.262 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.262 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.262 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.262 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.262 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable/disable debug options", 2023-11-27T15:28:59.262 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.263 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.263 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.263 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.263 INFO:teuthology.orchestra.run.smithi133.stdout: "jwt_token_ttl": { 2023-11-27T15:28:59.263 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "jwt_token_ttl", 2023-11-27T15:28:59.263 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.263 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.263 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.264 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "28800", 2023-11-27T15:28:59.264 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.264 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.264 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.264 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.264 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.264 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.264 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.265 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.265 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2023-11-27T15:28:59.265 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2023-11-27T15:28:59.265 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.265 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.265 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.265 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.265 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.266 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.266 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.266 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.266 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.266 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.266 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.266 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.266 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.267 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.267 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.267 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.267 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.267 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.267 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.267 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.267 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.268 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.268 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.268 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.268 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.268 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.268 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.268 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.268 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.269 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.269 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.269 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.269 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.269 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.269 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.269 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.270 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.270 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.270 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.270 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.270 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.270 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.270 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.270 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.271 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.271 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.271 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.271 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.271 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.271 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.271 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.271 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.272 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.272 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.272 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.272 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.272 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.272 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.272 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.272 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.273 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.273 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.273 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.273 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.273 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.273 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.273 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.273 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.274 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.274 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.274 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.274 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.274 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.274 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.274 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.274 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.275 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.275 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.275 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.275 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.275 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.275 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.275 INFO:teuthology.orchestra.run.smithi133.stdout: "motd": { 2023-11-27T15:28:59.275 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "motd", 2023-11-27T15:28:59.276 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.276 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.276 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.276 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.276 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.276 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.276 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.276 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "The message of the day", 2023-11-27T15:28:59.277 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.277 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.277 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.277 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.277 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect_resolve_ip_addr": { 2023-11-27T15:28:59.277 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "redirect_resolve_ip_addr", 2023-11-27T15:28:59.277 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.278 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.278 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.278 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.278 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.278 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.278 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.278 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.278 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.279 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.279 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.279 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.279 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2023-11-27T15:28:59.279 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2023-11-27T15:28:59.279 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.279 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.279 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.280 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2023-11-27T15:28:59.280 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.280 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.280 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.280 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.280 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.280 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.280 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.281 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.281 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2023-11-27T15:28:59.281 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2023-11-27T15:28:59.281 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.281 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.281 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.281 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8080", 2023-11-27T15:28:59.282 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.282 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.282 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.282 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.282 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.282 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.282 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.282 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.283 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2023-11-27T15:28:59.283 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2023-11-27T15:28:59.283 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.283 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.283 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.283 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.283 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.283 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.284 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.284 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.284 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.284 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.284 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.284 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.284 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl_server_port": { 2023-11-27T15:28:59.285 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl_server_port", 2023-11-27T15:28:59.285 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.285 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.285 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.285 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8443", 2023-11-27T15:28:59.285 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.285 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.285 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.286 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.286 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.286 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.286 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.286 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.286 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2023-11-27T15:28:59.286 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2023-11-27T15:28:59.286 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.287 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.287 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.287 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "redirect", 2023-11-27T15:28:59.287 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.287 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.287 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.287 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.288 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect" 2023-11-27T15:28:59.288 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.288 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.288 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.288 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.288 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.288 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.288 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2023-11-27T15:28:59.289 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2023-11-27T15:28:59.289 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.289 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.289 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.289 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2023-11-27T15:28:59.289 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2023-11-27T15:28:59.289 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2023-11-27T15:28:59.289 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.289 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.290 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.290 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.290 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.290 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.290 INFO:teuthology.orchestra.run.smithi133.stdout: "url_prefix": { 2023-11-27T15:28:59.290 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url_prefix", 2023-11-27T15:28:59.290 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.291 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.291 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.291 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.291 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.291 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.291 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.291 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.291 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.292 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.292 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.292 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.292 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.292 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.292 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.292 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "devicehealth", 2023-11-27T15:28:59.292 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.293 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.293 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.293 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_monitoring": { 2023-11-27T15:28:59.293 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_monitoring", 2023-11-27T15:28:59.293 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.293 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.293 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.293 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.293 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.294 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.294 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.294 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "monitor device health metrics", 2023-11-27T15:28:59.294 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.294 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.294 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.294 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.295 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.295 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.295 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.295 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.295 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.295 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.295 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.295 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.296 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.296 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.296 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.296 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.296 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.296 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.296 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.296 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.297 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.297 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.297 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.297 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.297 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.297 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.297 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.297 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.298 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.298 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.298 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.298 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.298 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.298 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.298 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.299 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.299 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.299 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.299 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.299 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.299 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.299 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.299 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.300 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.300 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.300 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.300 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.300 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.300 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.300 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.300 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.301 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.301 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.301 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.301 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.301 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.301 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.301 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.301 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.302 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.302 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.302 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.302 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.302 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.302 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.302 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.303 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.303 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.303 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.303 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.303 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.303 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.303 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.303 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.304 INFO:teuthology.orchestra.run.smithi133.stdout: "mark_out_threshold": { 2023-11-27T15:28:59.304 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mark_out_threshold", 2023-11-27T15:28:59.304 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.304 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.304 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.304 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2419200", 2023-11-27T15:28:59.304 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.304 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.305 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.305 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-27T15:28:59.305 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.305 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.305 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.305 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.305 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_name": { 2023-11-27T15:28:59.305 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_name", 2023-11-27T15:28:59.306 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.306 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.306 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.306 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "device_health_metrics", 2023-11-27T15:28:59.306 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.306 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.306 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.306 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-27T15:28:59.307 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.307 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.307 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.307 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.307 INFO:teuthology.orchestra.run.smithi133.stdout: "retention_period": { 2023-11-27T15:28:59.307 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retention_period", 2023-11-27T15:28:59.307 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.308 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.308 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.308 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15552000", 2023-11-27T15:28:59.308 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.308 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.308 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.308 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain device health metrics", 2023-11-27T15:28:59.308 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.309 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.309 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.309 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.309 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_frequency": { 2023-11-27T15:28:59.309 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_frequency", 2023-11-27T15:28:59.309 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.309 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.309 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.310 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2023-11-27T15:28:59.310 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.310 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.310 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.310 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-27T15:28:59.310 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.310 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.310 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.311 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.311 INFO:teuthology.orchestra.run.smithi133.stdout: "self_heal": { 2023-11-27T15:28:59.311 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "self_heal", 2023-11-27T15:28:59.311 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.311 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.311 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.311 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.311 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.312 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.312 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.312 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-27T15:28:59.312 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.312 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.312 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.312 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.312 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-11-27T15:28:59.313 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-11-27T15:28:59.313 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.313 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.313 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.313 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-11-27T15:28:59.313 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.313 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.313 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.314 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and check device health", 2023-11-27T15:28:59.314 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.314 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.314 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.314 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.314 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_threshold": { 2023-11-27T15:28:59.314 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_threshold", 2023-11-27T15:28:59.314 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.315 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.315 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.315 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7257600", 2023-11-27T15:28:59.315 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.315 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.315 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.315 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-27T15:28:59.315 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.316 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.316 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.316 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.316 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.316 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.316 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.316 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "diskprediction_local", 2023-11-27T15:28:59.317 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.317 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.317 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.317 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.317 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.317 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.317 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.317 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.318 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.318 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.318 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.318 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.318 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.318 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.318 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.318 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.319 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.319 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.319 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.319 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.319 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.319 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.319 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.319 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.320 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.320 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.320 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.320 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.320 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.320 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.320 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.320 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.321 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.321 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.321 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.321 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.321 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.321 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.321 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.321 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.322 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.322 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.322 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.322 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.322 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.322 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.322 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.322 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.323 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.323 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.323 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.323 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.323 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.323 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.323 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.323 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.324 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.324 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.324 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.324 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.324 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.324 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.324 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.324 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.325 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.325 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.325 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.325 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.325 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.325 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.325 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.325 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.326 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.326 INFO:teuthology.orchestra.run.smithi133.stdout: "predict_interval": { 2023-11-27T15:28:59.326 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predict_interval", 2023-11-27T15:28:59.326 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.326 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.326 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.326 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2023-11-27T15:28:59.327 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.327 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.327 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.327 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.327 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.327 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.327 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.327 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.328 INFO:teuthology.orchestra.run.smithi133.stdout: "predictor_model": { 2023-11-27T15:28:59.328 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predictor_model", 2023-11-27T15:28:59.328 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.328 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.328 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.328 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "prophetstor", 2023-11-27T15:28:59.328 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.328 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.329 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.329 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.329 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.329 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.329 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.329 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.329 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-11-27T15:28:59.329 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-11-27T15:28:59.330 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.330 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.330 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.330 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-11-27T15:28:59.330 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.330 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.330 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.330 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.331 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.331 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.331 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.331 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.331 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.331 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.331 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.332 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "influx", 2023-11-27T15:28:59.332 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": false, 2023-11-27T15:28:59.332 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "influxdb python module not found", 2023-11-27T15:28:59.332 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.332 INFO:teuthology.orchestra.run.smithi133.stdout: "batch_size": { 2023-11-27T15:28:59.332 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "batch_size", 2023-11-27T15:28:59.332 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.332 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.333 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.333 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5000", 2023-11-27T15:28:59.333 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.333 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.333 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.333 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-27T15:28:59.333 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.333 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.334 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.334 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.334 INFO:teuthology.orchestra.run.smithi133.stdout: "database": { 2023-11-27T15:28:59.334 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "database", 2023-11-27T15:28:59.334 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.334 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.334 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.334 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "ceph", 2023-11-27T15:28:59.335 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.335 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.335 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.335 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-27T15:28:59.335 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.335 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.335 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.335 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.336 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": { 2023-11-27T15:28:59.336 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "hostname", 2023-11-27T15:28:59.336 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.336 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.336 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.336 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.336 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.337 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.337 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.337 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server hostname", 2023-11-27T15:28:59.337 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.337 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.337 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.337 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.337 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-11-27T15:28:59.338 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-11-27T15:28:59.338 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.338 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.338 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.338 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "30", 2023-11-27T15:28:59.338 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "5", 2023-11-27T15:28:59.338 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.338 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.339 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-27T15:28:59.339 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.339 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.339 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.339 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.339 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.339 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.339 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.340 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.340 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.340 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.340 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.340 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.340 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.340 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.340 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.341 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.341 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.341 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.341 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.341 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.341 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.341 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.341 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.342 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.342 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.342 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.342 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.342 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.342 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.342 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.343 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.343 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.343 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.343 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.343 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.343 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.343 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.343 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.344 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.344 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.344 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.344 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.344 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.344 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.344 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.344 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.345 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.345 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.345 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.345 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.345 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.345 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.345 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.345 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.346 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.346 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.346 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.346 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.346 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.346 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.346 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.347 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.347 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.347 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.347 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.347 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.347 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.347 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.347 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.348 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.348 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.348 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.348 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.348 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.348 INFO:teuthology.orchestra.run.smithi133.stdout: "password": { 2023-11-27T15:28:59.348 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "password", 2023-11-27T15:28:59.348 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.349 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.349 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.349 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.349 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.349 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.349 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.349 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "password of InfluxDB server user", 2023-11-27T15:28:59.350 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.350 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.350 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.350 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.350 INFO:teuthology.orchestra.run.smithi133.stdout: "port": { 2023-11-27T15:28:59.350 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "port", 2023-11-27T15:28:59.350 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8086", 2023-11-27T15:28:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server port", 2023-11-27T15:28:59.351 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.352 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.352 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.352 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.352 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2023-11-27T15:28:59.352 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2023-11-27T15:28:59.352 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.352 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.352 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "false", 2023-11-27T15:28:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-27T15:28:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.353 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.354 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.354 INFO:teuthology.orchestra.run.smithi133.stdout: "threads": { 2023-11-27T15:28:59.354 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threads", 2023-11-27T15:28:59.354 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.354 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.354 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.354 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-11-27T15:28:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-11-27T15:28:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "32", 2023-11-27T15:28:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-27T15:28:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.355 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.356 INFO:teuthology.orchestra.run.smithi133.stdout: "username": { 2023-11-27T15:28:59.356 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "username", 2023-11-27T15:28:59.356 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.356 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.356 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.356 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.356 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.357 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.357 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.357 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "username of InfluxDB server user", 2023-11-27T15:28:59.357 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.357 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.357 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.357 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.357 INFO:teuthology.orchestra.run.smithi133.stdout: "verify_ssl": { 2023-11-27T15:28:59.358 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "verify_ssl", 2023-11-27T15:28:59.358 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.358 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.358 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.358 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "true", 2023-11-27T15:28:59.358 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.358 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.358 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.359 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-27T15:28:59.359 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.359 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.359 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.359 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.359 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.359 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.359 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "insights", 2023-11-27T15:28:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.360 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.361 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.362 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.363 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.364 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.364 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.364 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.364 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.364 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.364 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.364 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.364 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.365 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.365 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.365 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.365 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.365 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.365 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.365 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.365 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.366 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.366 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.366 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.366 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.366 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.366 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.366 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.366 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.367 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.367 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.367 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.367 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.367 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.367 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.367 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.367 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.368 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.368 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.368 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.368 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.368 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.368 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.368 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.369 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.370 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "iostat", 2023-11-27T15:28:59.370 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.370 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.370 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.370 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.370 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.370 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.370 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.371 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.372 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.372 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.372 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.372 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.372 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.372 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.372 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.372 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.373 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.373 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.373 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.373 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.373 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.373 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.373 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.374 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.374 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.374 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.374 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.374 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.374 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.374 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.374 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.375 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.375 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.375 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.375 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.375 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.375 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.375 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.376 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.377 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.377 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.377 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.377 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.377 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.377 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.377 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.378 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.378 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.378 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.378 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.378 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.378 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.378 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.378 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.379 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "k8sevents", 2023-11-27T15:28:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_event_retention_days": { 2023-11-27T15:28:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ceph_event_retention_days", 2023-11-27T15:28:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.380 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7", 2023-11-27T15:28:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Days to hold ceph event information within local cache", 2023-11-27T15:28:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.381 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.382 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.382 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.382 INFO:teuthology.orchestra.run.smithi133.stdout: "config_check_secs": { 2023-11-27T15:28:59.382 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_check_secs", 2023-11-27T15:28:59.382 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.382 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.382 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.383 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:28:59.383 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "10", 2023-11-27T15:28:59.383 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.383 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.383 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-11-27T15:28:59.383 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.383 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.383 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.384 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.384 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.384 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.384 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.384 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.384 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.384 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.385 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.386 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.386 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.386 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.386 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.386 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.386 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.386 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.387 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.387 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.387 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.387 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.387 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.387 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.387 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.387 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.388 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.388 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.388 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.388 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.388 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.388 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.388 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.388 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.389 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.390 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.390 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.390 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.390 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.390 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.390 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.390 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.390 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.391 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.391 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.391 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.391 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.391 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.391 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.391 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.392 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "localpool", 2023-11-27T15:28:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.393 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.394 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.394 INFO:teuthology.orchestra.run.smithi133.stdout: "failure_domain": { 2023-11-27T15:28:59.394 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "failure_domain", 2023-11-27T15:28:59.394 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.394 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.394 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.394 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "host", 2023-11-27T15:28:59.394 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "failure domain for any created local pool", 2023-11-27T15:28:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-27T15:28:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.395 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.396 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.396 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.396 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.396 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.396 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.396 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.396 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.396 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.397 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.397 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.397 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.397 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.397 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.397 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.397 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.397 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.398 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.398 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.398 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.398 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.398 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.398 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.398 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.398 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.399 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.399 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.399 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.399 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.399 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.399 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.399 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.400 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.400 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.400 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.400 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.400 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.400 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.400 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.400 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.401 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.401 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.401 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.401 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.401 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.401 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.401 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.401 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.402 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.402 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.402 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.402 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.402 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.402 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.402 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.403 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.403 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.403 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.403 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.403 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.403 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.403 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.403 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.404 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.404 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.404 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.404 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.404 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.404 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.404 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.404 INFO:teuthology.orchestra.run.smithi133.stdout: "min_size": { 2023-11-27T15:28:59.405 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_size", 2023-11-27T15:28:59.405 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.405 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.405 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.405 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.405 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.405 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.406 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.406 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default min_size for any created local pool", 2023-11-27T15:28:59.406 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-27T15:28:59.406 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.406 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.406 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.406 INFO:teuthology.orchestra.run.smithi133.stdout: "num_rep": { 2023-11-27T15:28:59.406 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "num_rep", 2023-11-27T15:28:59.407 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.407 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.407 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.407 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3", 2023-11-27T15:28:59.407 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.407 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.407 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.407 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default replica count for any created local pool", 2023-11-27T15:28:59.408 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.408 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.408 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.408 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.408 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_num": { 2023-11-27T15:28:59.408 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_num", 2023-11-27T15:28:59.408 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.408 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.409 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.409 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "128", 2023-11-27T15:28:59.409 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.409 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.409 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.409 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default pg_num for any created local pool", 2023-11-27T15:28:59.409 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.410 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.410 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.410 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.410 INFO:teuthology.orchestra.run.smithi133.stdout: "prefix": { 2023-11-27T15:28:59.410 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prefix", 2023-11-27T15:28:59.410 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.410 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.410 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.411 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.411 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.411 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.411 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.411 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name prefix for any created local pool", 2023-11-27T15:28:59.411 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.411 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.411 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.412 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.412 INFO:teuthology.orchestra.run.smithi133.stdout: "subtree": { 2023-11-27T15:28:59.412 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "subtree", 2023-11-27T15:28:59.412 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.412 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.412 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.412 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "rack", 2023-11-27T15:28:59.412 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.413 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.413 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.413 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-27T15:28:59.413 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-27T15:28:59.413 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.413 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.413 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.414 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.414 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.414 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.414 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mds_autoscaler", 2023-11-27T15:28:59.414 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.414 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.414 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.414 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.415 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.415 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.415 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.415 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.415 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.415 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.415 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.415 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.416 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.416 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.416 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.416 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.416 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.416 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.416 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.417 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.417 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.417 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.417 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.417 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.417 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.417 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.417 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.418 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.418 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.418 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.418 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.418 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.418 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.418 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.419 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.419 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.419 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.419 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.419 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.419 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.419 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.419 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.420 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.420 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.420 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.420 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.420 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.420 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.420 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.420 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.421 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.421 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.421 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.421 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.421 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.421 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.421 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.422 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.422 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.422 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.422 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.422 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.422 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.422 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.422 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.423 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.423 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.423 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.423 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.423 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.423 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.424 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.424 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.424 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.424 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.424 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.424 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirroring", 2023-11-27T15:28:59.424 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.424 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.425 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.425 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.425 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.425 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.425 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.425 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.425 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.426 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.426 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.426 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.426 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.426 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.426 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.426 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.426 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.427 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.427 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.427 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.427 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.427 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.427 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.427 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.427 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.428 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.428 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.428 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.428 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.428 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.428 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.428 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.429 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.429 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.429 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.429 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.429 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.429 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.429 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.429 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.430 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.430 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.430 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.430 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.430 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.430 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.430 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.430 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.431 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.431 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.431 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.431 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.431 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.431 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.431 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.432 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.432 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.432 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.432 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.432 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.432 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.432 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.432 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.433 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.433 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.433 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.433 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.433 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.433 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.433 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.433 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.434 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.434 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.434 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.434 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.434 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.434 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "nfs", 2023-11-27T15:28:59.434 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.434 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.435 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.435 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.435 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.435 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.435 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.435 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.435 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.435 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.436 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.436 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.436 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.436 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.436 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.436 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.436 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.436 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.437 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.437 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.437 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.437 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.437 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.437 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.437 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.437 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.438 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.438 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.438 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.438 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.438 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.438 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.438 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.439 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.439 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.439 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.439 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.439 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.439 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.439 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.439 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.440 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.440 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.440 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.440 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.440 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.440 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.440 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.440 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.441 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.441 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.441 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.441 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.441 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.441 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.441 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.441 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.442 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.442 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.442 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.442 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.442 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.442 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.442 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.442 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.443 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.443 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.443 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.443 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.443 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.443 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.443 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.443 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.444 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.444 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.444 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.444 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2023-11-27T15:28:59.444 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.444 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.444 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.444 INFO:teuthology.orchestra.run.smithi133.stdout: "fail_fs": { 2023-11-27T15:28:59.445 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "fail_fs", 2023-11-27T15:28:59.445 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.445 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.445 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.445 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.445 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.445 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.445 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.446 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-27T15:28:59.446 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.446 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.446 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.446 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.446 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.446 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.447 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.447 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.447 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.447 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.447 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.447 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.447 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.447 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.448 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.448 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.448 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.448 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.448 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.448 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.448 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.448 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.449 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.449 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.449 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.449 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.449 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.449 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.449 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.450 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.450 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.450 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.450 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.450 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.450 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.450 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.450 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.451 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.451 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.451 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.451 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.451 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.451 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.451 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.451 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.452 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.452 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.452 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.452 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.452 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.452 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.452 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.452 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.453 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.453 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.453 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.453 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.453 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.453 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.453 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.454 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.454 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.454 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.454 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.454 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.454 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.454 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.454 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.455 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.455 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.455 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.455 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.455 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.455 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.455 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator": { 2023-11-27T15:28:59.455 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2023-11-27T15:28:59.456 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.456 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.456 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.456 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.456 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.456 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.456 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.456 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm", 2023-11-27T15:28:59.457 INFO:teuthology.orchestra.run.smithi133.stdout: "rook", 2023-11-27T15:28:59.457 INFO:teuthology.orchestra.run.smithi133.stdout: "test_orchestrator" 2023-11-27T15:28:59.457 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.457 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Orchestrator backend", 2023-11-27T15:28:59.457 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.457 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.457 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.457 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.458 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.458 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.458 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.458 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_perf_query", 2023-11-27T15:28:59.458 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.458 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.458 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.458 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.459 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.459 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.459 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.459 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.459 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.459 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.459 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.459 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.460 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.460 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.460 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.460 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.460 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.460 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.460 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.460 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.461 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.461 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.461 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.461 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.461 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.461 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.461 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.461 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.462 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.462 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.462 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.462 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.462 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.462 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.462 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.462 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.463 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.463 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.463 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.463 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.463 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.463 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.463 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.463 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.464 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.464 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.464 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.464 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.464 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.464 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.464 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.465 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.465 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.465 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.465 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.465 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.465 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.465 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.465 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.466 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.466 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.466 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.466 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.466 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.466 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.466 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.466 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.467 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.467 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.467 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.467 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.467 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.467 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.467 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.467 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.468 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.468 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_support", 2023-11-27T15:28:59.468 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.468 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.468 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.468 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.468 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.468 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.469 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.469 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.469 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.469 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.469 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.469 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.469 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.469 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.470 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.470 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.470 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.470 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.470 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.470 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.470 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.471 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.471 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.471 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.471 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.471 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.471 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.471 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.471 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.472 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.472 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.472 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.472 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.472 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.472 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.472 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.472 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.473 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.473 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.473 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.473 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.473 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.473 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.473 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.473 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.474 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.474 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.474 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.474 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.474 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.474 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.474 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.475 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.475 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.475 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.475 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.475 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.475 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.475 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.475 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.476 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.476 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.476 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.476 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.476 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.476 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.476 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.476 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.477 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.477 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.477 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.477 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.477 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.477 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.477 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.477 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.478 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_autoscaler", 2023-11-27T15:28:59.478 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.478 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.478 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.478 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.478 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.478 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.479 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.479 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.479 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.479 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.479 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.479 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.479 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.479 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.480 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.480 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.480 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.480 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.480 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.480 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.480 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.481 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.481 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.481 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.481 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.481 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.481 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.481 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.481 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.482 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.482 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.482 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.482 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.482 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.482 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.482 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.482 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.483 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.483 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.483 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.483 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.483 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.483 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.483 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.483 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.484 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.484 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.484 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.484 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.484 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.484 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.484 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.484 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.485 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.485 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.485 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.485 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.485 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.485 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.485 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.486 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.486 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.486 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.486 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.486 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.486 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.486 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.487 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.487 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.487 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.487 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.487 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.487 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.487 INFO:teuthology.orchestra.run.smithi133.stdout: "noautoscale": { 2023-11-27T15:28:59.487 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "noautoscale", 2023-11-27T15:28:59.488 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.488 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.488 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.488 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.488 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.488 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.488 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.488 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "global autoscale flag", 2023-11-27T15:28:59.488 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-11-27T15:28:59.489 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.489 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.489 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.489 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-11-27T15:28:59.489 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-11-27T15:28:59.489 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.489 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.489 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.490 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-11-27T15:28:59.490 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.490 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.490 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.490 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.490 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.490 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.490 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.491 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.491 INFO:teuthology.orchestra.run.smithi133.stdout: "threshold": { 2023-11-27T15:28:59.491 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threshold", 2023-11-27T15:28:59.491 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:28:59.491 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.491 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.491 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2023-11-27T15:28:59.491 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1.0", 2023-11-27T15:28:59.492 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.492 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.492 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "scaling threshold", 2023-11-27T15:28:59.492 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-27T15:28:59.492 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.492 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.492 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.492 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.493 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.493 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.493 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "progress", 2023-11-27T15:28:59.493 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.493 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.493 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.493 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_pg_recovery_event": { 2023-11-27T15:28:59.493 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_pg_recovery_event", 2023-11-27T15:28:59.494 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.494 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.494 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.494 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.494 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.494 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.494 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.494 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-27T15:28:59.495 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.495 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.495 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.495 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.495 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2023-11-27T15:28:59.495 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2023-11-27T15:28:59.495 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.495 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.496 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.496 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.496 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.496 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.496 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.496 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.496 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.496 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.497 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.497 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.497 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.497 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.497 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.497 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.497 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.497 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.498 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.498 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.498 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.498 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.498 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.498 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.498 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.499 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.499 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.499 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.499 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.499 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.499 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.499 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.499 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.500 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.500 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.500 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.500 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.500 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.500 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.500 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.500 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.501 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.501 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.501 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.501 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.501 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.501 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.501 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.501 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.502 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.502 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.502 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.502 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.502 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.502 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.502 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.502 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.503 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.503 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.503 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.503 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.503 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.503 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.503 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.503 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.504 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.504 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.504 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.504 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.504 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.504 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.504 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.504 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.505 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.505 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.505 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.505 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.505 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.505 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.505 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.505 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.506 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.506 INFO:teuthology.orchestra.run.smithi133.stdout: "max_completed_events": { 2023-11-27T15:28:59.506 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_completed_events", 2023-11-27T15:28:59.506 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.506 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.506 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.506 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "50", 2023-11-27T15:28:59.506 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.507 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.507 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.507 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "number of past completed events to remember", 2023-11-27T15:28:59.507 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.507 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.507 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.507 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.507 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-11-27T15:28:59.508 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-11-27T15:28:59.508 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.508 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.508 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.508 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-11-27T15:28:59.508 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.508 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.508 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.509 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long the module is going to sleep", 2023-11-27T15:28:59.509 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.509 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.509 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.509 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.509 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.509 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.510 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.510 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus", 2023-11-27T15:28:59.510 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.510 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.510 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.510 INFO:teuthology.orchestra.run.smithi133.stdout: "cache": { 2023-11-27T15:28:59.510 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cache", 2023-11-27T15:28:59.510 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.511 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.511 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.511 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.511 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.511 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.511 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.511 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.511 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.512 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.512 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.512 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.512 INFO:teuthology.orchestra.run.smithi133.stdout: "exclude_perf_counters": { 2023-11-27T15:28:59.512 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "exclude_perf_counters", 2023-11-27T15:28:59.512 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.512 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.512 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.513 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.513 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.513 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.513 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.513 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-27T15:28:59.513 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-27T15:28:59.513 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.513 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.514 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.514 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.514 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.514 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.514 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.514 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.514 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.514 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.515 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.515 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.515 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.515 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.515 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.515 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.515 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.515 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.516 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.516 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.516 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.516 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.516 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.516 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.516 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.517 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.517 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.517 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.517 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.517 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.517 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.517 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.517 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.518 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.518 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.518 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.518 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.518 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.518 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.518 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.519 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.519 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.519 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.519 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.519 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.519 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.519 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.519 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.520 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.520 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.520 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.520 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.520 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.520 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.520 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.520 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.521 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.521 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.521 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.521 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.521 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.521 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.521 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.521 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.522 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.522 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.522 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.522 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.522 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.522 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.522 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.523 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.523 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.523 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools": { 2023-11-27T15:28:59.523 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools", 2023-11-27T15:28:59.523 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.523 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.523 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.523 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.524 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.524 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.524 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.524 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.524 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.524 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.524 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.524 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.525 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-27T15:28:59.525 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-27T15:28:59.525 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.525 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.525 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.525 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "300", 2023-11-27T15:28:59.525 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.525 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.526 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.526 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.526 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.526 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.526 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.526 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.526 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_interval": { 2023-11-27T15:28:59.526 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_interval", 2023-11-27T15:28:59.527 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:28:59.527 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.527 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.527 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15.0", 2023-11-27T15:28:59.527 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.527 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.527 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.528 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.528 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.528 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.528 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.528 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.528 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2023-11-27T15:28:59.528 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2023-11-27T15:28:59.528 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.529 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.529 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.529 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2023-11-27T15:28:59.529 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.529 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.529 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.529 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-27T15:28:59.529 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.530 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.530 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.530 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.530 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2023-11-27T15:28:59.530 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2023-11-27T15:28:59.530 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.530 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.530 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.531 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "9283", 2023-11-27T15:28:59.531 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.531 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.531 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.531 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-27T15:28:59.531 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.531 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.531 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.532 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.532 INFO:teuthology.orchestra.run.smithi133.stdout: "stale_cache_strategy": { 2023-11-27T15:28:59.532 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stale_cache_strategy", 2023-11-27T15:28:59.532 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.532 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.532 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.532 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "log", 2023-11-27T15:28:59.532 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.533 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.533 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.533 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.533 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.533 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.533 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.533 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.534 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2023-11-27T15:28:59.534 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2023-11-27T15:28:59.534 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.534 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.534 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.534 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "default", 2023-11-27T15:28:59.534 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.534 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.535 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.535 INFO:teuthology.orchestra.run.smithi133.stdout: "default", 2023-11-27T15:28:59.535 INFO:teuthology.orchestra.run.smithi133.stdout: "error" 2023-11-27T15:28:59.535 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.535 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.535 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.535 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.535 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.536 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.536 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2023-11-27T15:28:59.536 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2023-11-27T15:28:59.536 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.536 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.536 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.536 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2023-11-27T15:28:59.536 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2023-11-27T15:28:59.537 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2023-11-27T15:28:59.537 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.537 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.537 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.537 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.537 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.537 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.537 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.538 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.538 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.538 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_support", 2023-11-27T15:28:59.538 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.538 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.538 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.538 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.538 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.539 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.539 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.539 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.539 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.539 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.539 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.539 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.539 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.540 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.540 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.540 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.540 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.540 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.540 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.540 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.540 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.541 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.541 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.541 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.541 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.541 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.541 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.541 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.541 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.542 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.542 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.542 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.542 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.542 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.542 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.542 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.543 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.543 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.543 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.543 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.543 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.543 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.543 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.543 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.544 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.544 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.544 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.544 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.544 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.544 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.544 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.544 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.545 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.545 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.545 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.545 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.545 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.545 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.545 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.545 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.546 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.546 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.546 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.546 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.546 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.546 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.546 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.547 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.547 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.547 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.547 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.547 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.548 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.548 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_snap_create": { 2023-11-27T15:28:59.548 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_snap_create", 2023-11-27T15:28:59.548 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.548 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.548 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.548 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:28:59.549 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.549 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.549 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.549 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.549 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.549 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.549 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.550 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.550 INFO:teuthology.orchestra.run.smithi133.stdout: "mirror_snapshot_schedule": { 2023-11-27T15:28:59.550 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirror_snapshot_schedule", 2023-11-27T15:28:59.550 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.550 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.550 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.550 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.551 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.551 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.551 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.551 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.551 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.551 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.551 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.552 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.552 INFO:teuthology.orchestra.run.smithi133.stdout: "trash_purge_schedule": { 2023-11-27T15:28:59.552 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "trash_purge_schedule", 2023-11-27T15:28:59.552 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.552 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.552 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.552 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.553 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.553 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.553 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.553 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.553 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.553 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.554 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.554 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.554 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.554 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.554 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.554 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "restful", 2023-11-27T15:28:59.554 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.555 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.555 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.555 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_auth": { 2023-11-27T15:28:59.555 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_auth", 2023-11-27T15:28:59.555 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.555 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.555 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.556 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.556 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.556 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.556 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.556 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.556 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.556 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.557 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.557 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.557 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2023-11-27T15:28:59.557 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2023-11-27T15:28:59.557 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.557 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.557 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.558 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.558 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.558 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.558 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.558 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.558 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.558 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.559 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.559 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.559 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.559 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.559 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.559 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.559 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.560 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.560 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.560 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.560 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.560 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.560 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.560 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.561 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.561 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.561 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.561 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.561 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.561 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.561 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.562 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.562 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.562 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.562 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.562 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.562 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.562 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.563 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.563 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.563 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.563 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.563 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.563 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.563 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.563 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.564 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.564 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.564 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.564 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.564 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.564 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.564 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.565 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.565 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.565 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.565 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.565 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.565 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.565 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.566 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.566 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.566 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.566 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.566 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.566 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.566 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.567 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.567 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.567 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.567 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.567 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.567 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.568 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.568 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.568 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.568 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.568 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.568 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.568 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.568 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.569 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.569 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2023-11-27T15:28:59.569 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2023-11-27T15:28:59.569 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.569 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.569 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.569 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.570 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.570 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.570 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.570 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.570 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.570 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.571 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.571 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.571 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2023-11-27T15:28:59.571 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2023-11-27T15:28:59.571 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.571 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.571 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.572 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.572 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.572 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.572 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.572 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.572 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.572 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.573 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.573 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.573 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.573 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.573 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.573 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rgw", 2023-11-27T15:28:59.573 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.574 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.574 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.574 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.574 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.574 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.574 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.574 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.575 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.575 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.575 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.575 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.575 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.575 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.575 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.576 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.576 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.576 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.576 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.576 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.576 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.576 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.577 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.577 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.577 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.577 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.577 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.577 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.577 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.578 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.578 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.578 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.578 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.578 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.578 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.578 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.579 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.579 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.579 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.579 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.579 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.579 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.579 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.580 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.580 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.580 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.580 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.580 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.580 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.580 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.581 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.581 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.581 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.581 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.581 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.581 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.581 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.582 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.582 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.582 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.582 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.582 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.582 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.582 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.583 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.583 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.583 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.583 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.583 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.583 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.583 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.584 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.584 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.584 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.584 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.584 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.584 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rook", 2023-11-27T15:28:59.584 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.585 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.585 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.585 INFO:teuthology.orchestra.run.smithi133.stdout: "drive_group_interval": { 2023-11-27T15:28:59.585 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "drive_group_interval", 2023-11-27T15:28:59.585 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:28:59.585 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.586 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.586 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "300.0", 2023-11-27T15:28:59.586 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.586 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.586 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.586 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-27T15:28:59.586 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.587 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.587 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.587 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.587 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.587 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.587 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.587 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.588 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.588 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.588 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.588 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.588 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.588 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.588 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.589 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.589 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.589 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.589 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.589 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.589 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.589 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.590 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.590 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.590 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.590 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.590 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.590 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.590 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.591 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.591 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.591 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.591 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.591 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.591 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.591 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.592 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.592 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.592 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.592 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.592 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.592 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.592 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.593 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.593 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.593 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.593 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.593 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.593 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.593 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.594 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.594 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.594 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.594 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.594 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.594 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.594 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.595 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.595 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.595 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.595 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.595 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.595 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.595 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.596 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.596 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.596 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.596 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.596 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.596 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.597 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.597 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.597 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.597 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.597 INFO:teuthology.orchestra.run.smithi133.stdout: "storage_class": { 2023-11-27T15:28:59.597 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "storage_class", 2023-11-27T15:28:59.597 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.598 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.598 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.598 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "local", 2023-11-27T15:28:59.598 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.598 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.598 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.598 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-27T15:28:59.599 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.599 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.599 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.599 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.599 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.599 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.599 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.600 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "selftest", 2023-11-27T15:28:59.600 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.600 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.600 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.600 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.600 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.600 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.601 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.601 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.601 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.601 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.601 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.601 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.601 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.602 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.602 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.602 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.602 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.602 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.602 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.602 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.603 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.603 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.603 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.603 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.603 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.603 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.603 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.604 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.604 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.604 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.604 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.604 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.604 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.604 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.605 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.605 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.605 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.605 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.605 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.605 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.605 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.606 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.606 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.606 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.606 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.606 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.606 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.606 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.607 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.607 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.607 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.607 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.607 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.607 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.607 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.608 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.608 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.608 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.608 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.608 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.608 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.608 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.609 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.609 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.609 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.609 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.609 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.609 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.609 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.609 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.610 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.610 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.610 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.610 INFO:teuthology.orchestra.run.smithi133.stdout: "roption1": { 2023-11-27T15:28:59.610 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption1", 2023-11-27T15:28:59.610 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.611 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.611 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.611 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.611 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.611 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.611 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.611 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.612 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.612 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.612 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.612 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.612 INFO:teuthology.orchestra.run.smithi133.stdout: "roption2": { 2023-11-27T15:28:59.612 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption2", 2023-11-27T15:28:59.612 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.613 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.613 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.613 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "xyz", 2023-11-27T15:28:59.613 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.613 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.613 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.613 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.614 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.614 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.614 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.614 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.614 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption1": { 2023-11-27T15:28:59.614 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption1", 2023-11-27T15:28:59.614 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.615 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.615 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.615 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.615 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.615 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.615 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.615 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.616 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.616 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.616 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.616 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.616 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption2": { 2023-11-27T15:28:59.616 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption2", 2023-11-27T15:28:59.616 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.617 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.617 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.617 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.617 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.617 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.617 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.617 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.618 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.618 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.618 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.618 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.618 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption3": { 2023-11-27T15:28:59.618 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption3", 2023-11-27T15:28:59.621 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:28:59.621 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.621 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.621 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.621 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.622 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.622 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.622 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.622 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.622 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.622 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.623 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.623 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption4": { 2023-11-27T15:28:59.623 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption4", 2023-11-27T15:28:59.623 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.623 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.623 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.623 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.624 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.624 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.624 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.624 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.624 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.624 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.624 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.624 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.625 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption5": { 2023-11-27T15:28:59.625 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption5", 2023-11-27T15:28:59.625 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.625 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.625 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.625 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.625 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.626 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.626 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.626 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.626 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.626 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.626 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.626 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.627 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption6": { 2023-11-27T15:28:59.627 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption6", 2023-11-27T15:28:59.627 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.627 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.627 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.627 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.627 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.628 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.628 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.628 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.628 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.628 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.628 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.629 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.629 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption7": { 2023-11-27T15:28:59.629 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption7", 2023-11-27T15:28:59.629 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.629 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.629 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.629 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.630 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-11-27T15:28:59.630 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "42", 2023-11-27T15:28:59.630 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.630 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.630 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.630 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.630 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.631 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.631 INFO:teuthology.orchestra.run.smithi133.stdout: "testkey": { 2023-11-27T15:28:59.631 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testkey", 2023-11-27T15:28:59.631 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.631 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.631 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.631 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.632 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.632 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.632 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.632 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.632 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.632 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.632 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.633 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.633 INFO:teuthology.orchestra.run.smithi133.stdout: "testlkey": { 2023-11-27T15:28:59.633 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testlkey", 2023-11-27T15:28:59.633 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.633 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.633 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.633 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.634 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.634 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.634 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.634 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.634 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.634 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.634 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.635 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.635 INFO:teuthology.orchestra.run.smithi133.stdout: "testnewline": { 2023-11-27T15:28:59.635 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testnewline", 2023-11-27T15:28:59.635 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.635 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.635 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.635 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.636 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.636 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.636 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.636 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.636 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.636 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.636 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.637 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.637 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.637 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.637 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.637 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snap_schedule", 2023-11-27T15:28:59.637 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.637 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.638 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.638 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_m_granularity": { 2023-11-27T15:28:59.638 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_m_granularity", 2023-11-27T15:28:59.638 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.638 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.638 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.638 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.639 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.639 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.639 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.639 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow minute scheduled snapshots", 2023-11-27T15:28:59.639 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.639 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.639 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.640 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.640 INFO:teuthology.orchestra.run.smithi133.stdout: "dump_on_update": { 2023-11-27T15:28:59.640 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dump_on_update", 2023-11-27T15:28:59.640 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.640 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.640 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.640 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.641 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.641 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.641 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.641 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "dump database to debug log on update", 2023-11-27T15:28:59.641 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.641 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.641 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.642 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.642 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.642 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.642 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.642 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.642 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.642 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.643 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.643 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.643 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.643 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.643 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.643 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.643 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.644 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.644 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.644 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.644 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.644 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.644 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.644 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.645 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.645 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.645 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.645 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.645 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.645 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.645 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.646 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.646 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.646 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.646 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.646 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.646 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.646 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.647 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.647 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.647 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.647 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.647 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.647 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.647 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.648 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.648 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.648 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.648 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.648 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.648 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.648 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.649 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.649 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.649 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.649 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.649 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.649 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.649 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.650 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.650 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.650 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.650 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.650 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.650 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.650 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.651 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.651 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.651 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.651 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.651 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.651 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.651 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.652 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.652 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.652 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.652 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.652 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stats", 2023-11-27T15:28:59.652 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.652 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.653 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.653 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.653 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.653 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.653 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.653 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.653 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.654 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.654 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.654 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.654 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.654 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.654 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.654 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.655 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.655 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.655 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.655 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.655 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.655 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.655 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.656 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.656 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.656 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.656 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.656 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.656 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.656 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.657 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.657 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.657 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.657 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.657 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.657 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.657 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.658 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.658 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.658 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.658 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.658 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.658 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.658 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.659 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.659 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.659 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.659 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.659 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.659 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.659 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.659 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.660 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.660 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.660 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.660 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.660 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.660 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.660 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.661 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.661 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.661 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.661 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.661 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.661 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.661 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.662 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.662 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.662 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.662 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.662 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.662 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.662 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.663 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.663 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.663 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.663 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "status", 2023-11-27T15:28:59.663 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.663 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.663 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.664 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.664 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.664 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.664 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.664 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.664 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.664 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.665 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.665 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.665 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.665 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.665 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.665 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.665 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.666 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.666 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.666 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.666 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.666 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.666 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.666 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.667 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.667 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.667 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.667 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.667 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.667 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.667 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.668 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.668 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.668 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.668 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.668 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.668 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.668 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.669 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.669 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.669 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.669 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.669 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.669 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.669 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.670 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.670 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.670 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.670 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.670 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.670 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.670 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.671 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.671 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.671 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.671 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.671 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.671 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.671 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.672 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.672 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.672 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.672 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.672 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.672 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.672 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.672 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.673 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.673 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.673 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.673 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.673 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.673 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.674 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.674 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.674 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.674 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telegraf", 2023-11-27T15:28:59.674 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.674 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.674 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.675 INFO:teuthology.orchestra.run.smithi133.stdout: "address": { 2023-11-27T15:28:59.675 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "address", 2023-11-27T15:28:59.675 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.675 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.675 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.675 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-27T15:28:59.675 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.676 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.676 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.676 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.676 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.676 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.676 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.676 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.677 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-11-27T15:28:59.677 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-11-27T15:28:59.677 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.677 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.677 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.677 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15", 2023-11-27T15:28:59.677 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.677 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.678 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.678 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.678 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.678 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.678 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.678 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.679 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.679 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.679 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.679 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.679 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.679 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.679 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.680 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.680 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.680 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.680 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.680 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.680 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.680 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.681 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.681 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.681 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.681 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.681 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.681 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.681 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.681 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.682 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.682 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.682 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.682 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.682 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.682 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.683 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.683 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.683 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.683 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.683 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.683 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.683 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.683 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.684 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.684 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.684 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.684 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.684 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.684 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.684 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.685 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.685 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.685 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.685 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.685 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.685 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.685 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.686 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.686 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.686 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.686 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.686 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.686 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.687 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.687 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.687 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.687 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.687 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.687 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.687 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.687 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.688 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.688 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.688 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.688 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.688 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.688 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.689 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.689 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.689 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.689 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telemetry", 2023-11-27T15:28:59.689 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.689 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.689 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.690 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_basic": { 2023-11-27T15:28:59.690 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_basic", 2023-11-27T15:28:59.690 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.690 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.690 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.690 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.690 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.691 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.691 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.691 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-27T15:28:59.691 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.691 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.691 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.691 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.692 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_crash": { 2023-11-27T15:28:59.692 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_crash", 2023-11-27T15:28:59.692 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.692 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.692 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.692 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.692 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.693 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.693 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.693 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-27T15:28:59.693 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.693 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.693 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.693 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.694 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_device": { 2023-11-27T15:28:59.694 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_device", 2023-11-27T15:28:59.694 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.694 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.694 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.694 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.694 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.695 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.695 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.695 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-27T15:28:59.695 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.695 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.695 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.695 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.696 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_ident": { 2023-11-27T15:28:59.696 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_ident", 2023-11-27T15:28:59.696 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.696 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.696 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.696 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.696 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.697 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.697 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.697 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-27T15:28:59.697 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.697 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.697 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.697 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.698 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_perf": { 2023-11-27T15:28:59.698 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_perf", 2023-11-27T15:28:59.698 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.698 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.698 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.698 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.698 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.699 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.699 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.699 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-27T15:28:59.699 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.699 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.699 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.699 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.700 INFO:teuthology.orchestra.run.smithi133.stdout: "contact": { 2023-11-27T15:28:59.700 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "contact", 2023-11-27T15:28:59.700 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.700 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.700 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.700 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.700 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.701 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.701 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.701 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.701 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.701 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.701 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.701 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.702 INFO:teuthology.orchestra.run.smithi133.stdout: "description": { 2023-11-27T15:28:59.702 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "description", 2023-11-27T15:28:59.702 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.702 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.702 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.702 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.702 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.703 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.703 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.703 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.703 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.703 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.703 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.703 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.704 INFO:teuthology.orchestra.run.smithi133.stdout: "device_url": { 2023-11-27T15:28:59.704 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_url", 2023-11-27T15:28:59.704 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.704 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.704 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.704 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-27T15:28:59.704 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.704 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.705 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.705 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.705 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.705 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.705 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.705 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.705 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2023-11-27T15:28:59.706 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2023-11-27T15:28:59.706 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.706 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.706 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.706 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.706 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.706 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.707 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.707 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.707 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.707 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.707 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.707 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.707 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-11-27T15:28:59.708 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-11-27T15:28:59.708 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.708 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.708 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.708 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "24", 2023-11-27T15:28:59.708 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "8", 2023-11-27T15:28:59.708 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.709 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.709 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.709 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.709 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.709 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.709 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.709 INFO:teuthology.orchestra.run.smithi133.stdout: "last_opt_revision": { 2023-11-27T15:28:59.710 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "last_opt_revision", 2023-11-27T15:28:59.710 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.710 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.710 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.710 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1", 2023-11-27T15:28:59.710 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.710 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.711 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.711 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.711 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.711 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.711 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.711 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.711 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard": { 2023-11-27T15:28:59.712 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard", 2023-11-27T15:28:59.712 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.712 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.712 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.712 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.712 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.712 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.713 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.713 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.713 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.713 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.713 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.713 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.713 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard_description": { 2023-11-27T15:28:59.714 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard_description", 2023-11-27T15:28:59.714 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.714 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.714 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.714 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.714 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.714 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.715 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.715 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.715 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.715 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.715 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.715 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.716 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.716 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.716 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.716 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.716 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.716 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.716 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.716 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.717 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.717 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.717 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.717 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.717 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.717 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.717 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.718 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.718 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.718 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.718 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.718 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.718 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.719 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.719 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.719 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.719 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.719 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.719 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.719 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.719 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.720 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.720 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.720 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.720 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.720 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.720 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.720 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.721 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.721 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.721 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.721 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.721 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.721 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.722 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.722 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.722 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.722 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.722 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.722 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.722 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.723 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.723 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.723 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.723 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.723 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.723 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.723 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.724 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.724 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.724 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.724 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.724 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.724 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.724 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.725 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.725 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.725 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.725 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.725 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.725 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.725 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.726 INFO:teuthology.orchestra.run.smithi133.stdout: "organization": { 2023-11-27T15:28:59.726 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "organization", 2023-11-27T15:28:59.726 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.726 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.726 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.726 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.726 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.727 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.727 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.727 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.727 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.727 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.727 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.727 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.728 INFO:teuthology.orchestra.run.smithi133.stdout: "proxy": { 2023-11-27T15:28:59.728 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "proxy", 2023-11-27T15:28:59.728 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.728 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.728 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.728 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.728 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.729 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.729 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.729 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.729 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.729 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.729 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.729 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.729 INFO:teuthology.orchestra.run.smithi133.stdout: "url": { 2023-11-27T15:28:59.730 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url", 2023-11-27T15:28:59.730 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.730 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.730 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.730 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-27T15:28:59.730 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.730 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.731 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.731 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.731 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.731 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.731 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.731 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.731 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.732 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.732 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.732 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "test_orchestrator", 2023-11-27T15:28:59.732 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.732 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.732 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.732 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.733 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.733 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.733 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.733 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.733 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.733 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.733 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.734 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.734 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.734 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.734 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.734 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.734 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.734 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.735 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.735 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.735 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.735 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.735 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.735 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.735 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.736 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.736 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.736 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.736 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.736 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.736 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.736 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.737 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.737 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.737 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.737 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.737 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.737 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.737 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.738 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.738 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.738 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.738 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.738 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.738 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.738 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.739 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.739 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.739 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.739 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.739 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.739 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.739 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.740 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.740 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.740 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.740 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.740 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.740 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.740 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.741 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.741 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.741 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.741 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.741 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.741 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.741 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.742 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.742 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.742 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.742 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.742 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.742 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.742 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.743 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.743 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.743 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "volumes", 2023-11-27T15:28:59.743 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.743 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.743 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.743 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.744 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.744 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.744 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.744 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.744 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.744 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.745 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.745 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.745 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.745 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.745 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.745 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.745 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.745 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.746 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.746 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.746 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.746 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.746 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.746 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.747 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.747 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.747 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.747 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.747 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.747 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.747 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.748 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.748 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.748 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.748 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.748 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.748 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.748 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.749 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.749 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.749 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.749 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.749 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.749 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.749 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.749 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.750 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.750 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.750 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.750 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.750 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.750 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.750 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.751 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.751 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.751 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.751 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.751 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.751 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.751 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.752 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.752 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.752 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.752 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.752 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.752 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.752 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.753 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.753 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.753 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.753 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.753 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.753 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.753 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_clones": { 2023-11-27T15:28:59.754 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_clones", 2023-11-27T15:28:59.754 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.754 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.754 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.754 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4", 2023-11-27T15:28:59.754 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.754 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.755 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.755 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-27T15:28:59.755 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.755 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.755 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.755 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.755 INFO:teuthology.orchestra.run.smithi133.stdout: "snapshot_clone_delay": { 2023-11-27T15:28:59.756 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snapshot_clone_delay", 2023-11-27T15:28:59.756 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.756 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.756 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.756 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-11-27T15:28:59.756 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.757 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.757 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.757 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-27T15:28:59.757 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.757 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.757 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.757 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.757 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.758 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.758 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.758 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix", 2023-11-27T15:28:59.758 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.758 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.758 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.758 INFO:teuthology.orchestra.run.smithi133.stdout: "discovery_interval": { 2023-11-27T15:28:59.759 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "discovery_interval", 2023-11-27T15:28:59.759 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-11-27T15:28:59.759 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.759 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.759 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "100", 2023-11-27T15:28:59.759 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.759 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.760 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.760 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.760 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.760 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.760 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.760 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.760 INFO:teuthology.orchestra.run.smithi133.stdout: "identifier": { 2023-11-27T15:28:59.761 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "identifier", 2023-11-27T15:28:59.761 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.761 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.761 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.761 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.761 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.761 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.762 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.762 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.762 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.762 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.762 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.762 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.762 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-11-27T15:28:59.763 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-11-27T15:28:59.763 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.763 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.763 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.763 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-11-27T15:28:59.763 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.763 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.764 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.764 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.764 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.764 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.764 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.764 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.764 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.765 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.765 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.765 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.765 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.765 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.765 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.765 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.766 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.766 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.766 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.766 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.766 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.766 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.766 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.767 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.767 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.767 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.767 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.767 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.767 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.767 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.767 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.768 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.768 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.768 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.768 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.768 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.768 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.768 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.769 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.769 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.769 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.769 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.769 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.769 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.769 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.770 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.770 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.770 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.770 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.770 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.770 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.770 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.771 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.771 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.771 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.771 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.771 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.771 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.772 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.772 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.772 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.772 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.772 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.772 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.772 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.772 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.773 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.773 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.773 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.773 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.773 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.773 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.773 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.774 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.774 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.774 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.774 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.774 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.774 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_host": { 2023-11-27T15:28:59.775 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_host", 2023-11-27T15:28:59.775 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.775 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.775 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.775 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.775 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.775 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.775 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.776 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.776 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.776 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.776 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.776 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.776 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_port": { 2023-11-27T15:28:59.777 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_port", 2023-11-27T15:28:59.777 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.777 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.777 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.777 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10051", 2023-11-27T15:28:59.777 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.777 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.778 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.778 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.778 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.778 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.778 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.778 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.778 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_sender": { 2023-11-27T15:28:59.779 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_sender", 2023-11-27T15:28:59.779 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.779 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.779 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.779 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-27T15:28:59.779 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.779 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.780 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.780 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.780 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.780 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.780 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.780 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.780 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.781 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.781 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-11-27T15:28:59.781 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.781 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.781 INFO:teuthology.orchestra.run.smithi133.stdout: "modules": [ 2023-11-27T15:28:59.781 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm", 2023-11-27T15:28:59.781 INFO:teuthology.orchestra.run.smithi133.stdout: "dashboard", 2023-11-27T15:28:59.782 INFO:teuthology.orchestra.run.smithi133.stdout: "iostat", 2023-11-27T15:28:59.782 INFO:teuthology.orchestra.run.smithi133.stdout: "nfs", 2023-11-27T15:28:59.782 INFO:teuthology.orchestra.run.smithi133.stdout: "restful" 2023-11-27T15:28:59.782 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.782 INFO:teuthology.orchestra.run.smithi133.stdout: "available_modules": [ 2023-11-27T15:28:59.782 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.782 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alerts", 2023-11-27T15:28:59.783 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.783 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.783 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.783 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-11-27T15:28:59.783 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-11-27T15:28:59.783 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.783 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.784 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.784 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-11-27T15:28:59.784 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.784 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.784 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.784 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How frequently to reexamine health status", 2023-11-27T15:28:59.784 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.785 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.785 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.785 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.785 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.785 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.785 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.785 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.786 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.786 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.786 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.786 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.786 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.786 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.786 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.787 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.787 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.787 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.787 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.787 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.787 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.787 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.788 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.788 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.788 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.788 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.788 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.788 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.788 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.789 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.789 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.789 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.789 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.789 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.789 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.789 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.790 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.790 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.790 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.790 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.790 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.790 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.790 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.791 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.791 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.791 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.791 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.791 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.791 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.791 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.792 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.792 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.792 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.792 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.792 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.792 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.792 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.793 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.793 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.793 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.793 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.793 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.793 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.793 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.794 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.794 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.794 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.794 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.794 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.794 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.794 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.795 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.795 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.795 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.795 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_destination": { 2023-11-27T15:28:59.795 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_destination", 2023-11-27T15:28:59.795 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.795 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.796 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.796 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.796 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.796 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.796 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.796 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email address to send alerts to", 2023-11-27T15:28:59.796 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.797 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.797 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.797 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.797 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_from_name": { 2023-11-27T15:28:59.797 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_from_name", 2023-11-27T15:28:59.797 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.797 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.798 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.798 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "Ceph", 2023-11-27T15:28:59.798 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.798 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.798 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.798 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Email From: name", 2023-11-27T15:28:59.798 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.799 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.799 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.799 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.799 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_host": { 2023-11-27T15:28:59.799 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_host", 2023-11-27T15:28:59.799 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.799 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.800 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.800 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.800 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.800 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.800 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.800 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP server", 2023-11-27T15:28:59.800 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.801 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.801 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.801 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.801 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_password": { 2023-11-27T15:28:59.801 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_password", 2023-11-27T15:28:59.801 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.801 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.802 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.802 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.802 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.802 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.802 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.802 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Password to authenticate with", 2023-11-27T15:28:59.802 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.803 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.803 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.803 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.803 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_port": { 2023-11-27T15:28:59.803 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_port", 2023-11-27T15:28:59.803 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.803 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.804 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.804 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "465", 2023-11-27T15:28:59.804 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.804 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.804 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.804 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP port", 2023-11-27T15:28:59.804 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.805 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.805 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.805 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.805 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_sender": { 2023-11-27T15:28:59.805 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_sender", 2023-11-27T15:28:59.805 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.805 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.805 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.806 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.806 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.806 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.806 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.806 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SMTP envelope sender", 2023-11-27T15:28:59.806 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.807 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.807 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.807 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.807 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_ssl": { 2023-11-27T15:28:59.807 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_ssl", 2023-11-27T15:28:59.807 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.807 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.808 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.808 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.808 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.808 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.808 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.808 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use SSL to connect to SMTP server", 2023-11-27T15:28:59.808 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.809 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.809 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.809 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.809 INFO:teuthology.orchestra.run.smithi133.stdout: "smtp_user": { 2023-11-27T15:28:59.809 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "smtp_user", 2023-11-27T15:28:59.809 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.809 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.810 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.810 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.810 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.810 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.810 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.810 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "User to authenticate as", 2023-11-27T15:28:59.810 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.811 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.811 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.811 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.811 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.811 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.811 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.811 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "balancer", 2023-11-27T15:28:59.812 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.812 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.812 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.812 INFO:teuthology.orchestra.run.smithi133.stdout: "active": { 2023-11-27T15:28:59.812 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "active", 2023-11-27T15:28:59.812 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.812 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.813 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.813 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.813 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.813 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.813 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.813 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically balance PGs across cluster", 2023-11-27T15:28:59.813 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.814 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.814 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.814 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.814 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_time": { 2023-11-27T15:28:59.814 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_time", 2023-11-27T15:28:59.814 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.814 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.815 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.815 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0000", 2023-11-27T15:28:59.815 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.815 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.815 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.815 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "beginning time of day to automatically balance", 2023-11-27T15:28:59.815 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-27T15:28:59.816 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.816 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.816 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.816 INFO:teuthology.orchestra.run.smithi133.stdout: "begin_weekday": { 2023-11-27T15:28:59.816 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "begin_weekday", 2023-11-27T15:28:59.816 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-11-27T15:28:59.816 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.817 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.817 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-11-27T15:28:59.817 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2023-11-27T15:28:59.817 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2023-11-27T15:28:59.817 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.817 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-11-27T15:28:59.817 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-27T15:28:59.818 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.818 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.818 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.818 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_max_iterations": { 2023-11-27T15:28:59.818 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_max_iterations", 2023-11-27T15:28:59.818 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-11-27T15:28:59.818 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.819 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.819 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "25", 2023-11-27T15:28:59.819 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-11-27T15:28:59.819 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "250", 2023-11-27T15:28:59.819 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.819 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-11-27T15:28:59.819 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.820 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.820 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.820 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.820 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_metrics": { 2023-11-27T15:28:59.820 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_metrics", 2023-11-27T15:28:59.820 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.820 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.821 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.821 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "pgs,objects,bytes", 2023-11-27T15:28:59.821 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.821 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.821 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.821 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-11-27T15:28:59.821 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-11-27T15:28:59.822 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.822 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.822 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.822 INFO:teuthology.orchestra.run.smithi133.stdout: "crush_compat_step": { 2023-11-27T15:28:59.822 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crush_compat_step", 2023-11-27T15:28:59.822 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:28:59.822 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.823 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.823 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.5", 2023-11-27T15:28:59.823 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0.001", 2023-11-27T15:28:59.823 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "0.999", 2023-11-27T15:28:59.823 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.823 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "aggressiveness of optimization", 2023-11-27T15:28:59.825 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-11-27T15:28:59.825 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.825 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.825 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.825 INFO:teuthology.orchestra.run.smithi133.stdout: "end_time": { 2023-11-27T15:28:59.826 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_time", 2023-11-27T15:28:59.826 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.826 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.826 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.826 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2359", 2023-11-27T15:28:59.826 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.826 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.827 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.827 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ending time of day to automatically balance", 2023-11-27T15:28:59.827 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-11-27T15:28:59.827 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.827 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.827 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.827 INFO:teuthology.orchestra.run.smithi133.stdout: "end_weekday": { 2023-11-27T15:28:59.828 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "end_weekday", 2023-11-27T15:28:59.828 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-11-27T15:28:59.828 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.828 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.828 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-11-27T15:28:59.828 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "0", 2023-11-27T15:28:59.828 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "6", 2023-11-27T15:28:59.829 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.829 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-11-27T15:28:59.829 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-11-27T15:28:59.829 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.829 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.829 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.829 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.830 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.830 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.830 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.830 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.830 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.830 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.830 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.831 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.831 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.831 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.831 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.831 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.831 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.831 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.832 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.832 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.832 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.832 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.832 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.832 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.832 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.832 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.833 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.833 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.833 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.833 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.833 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.833 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.833 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.834 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.834 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.834 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.834 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.834 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.834 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.834 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.835 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.835 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.835 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.835 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.835 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.835 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.835 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.836 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.836 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.836 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.836 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.836 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.836 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.836 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.837 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.837 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.837 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.837 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.837 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.837 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.837 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.838 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.838 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.838 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.838 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.838 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.838 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.838 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.839 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.839 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.839 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.839 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.839 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.839 INFO:teuthology.orchestra.run.smithi133.stdout: "min_score": { 2023-11-27T15:28:59.839 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_score", 2023-11-27T15:28:59.840 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:28:59.840 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.840 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.840 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-11-27T15:28:59.840 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.840 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.840 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.841 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-11-27T15:28:59.841 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.841 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.841 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.841 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.841 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2023-11-27T15:28:59.841 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2023-11-27T15:28:59.842 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.842 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.842 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.842 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "upmap", 2023-11-27T15:28:59.842 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.842 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.842 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.843 INFO:teuthology.orchestra.run.smithi133.stdout: "crush-compat", 2023-11-27T15:28:59.843 INFO:teuthology.orchestra.run.smithi133.stdout: "none", 2023-11-27T15:28:59.843 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap" 2023-11-27T15:28:59.843 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.843 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Balancer mode", 2023-11-27T15:28:59.843 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.843 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.844 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.844 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.844 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_ids": { 2023-11-27T15:28:59.844 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_ids", 2023-11-27T15:28:59.844 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.844 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.844 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.845 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.845 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.845 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.845 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.845 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-11-27T15:28:59.845 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.845 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.846 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.846 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.846 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-11-27T15:28:59.846 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-11-27T15:28:59.846 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.846 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.846 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.847 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-11-27T15:28:59.847 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.847 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.847 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.847 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-11-27T15:28:59.847 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.847 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.848 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.848 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.848 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_deviation": { 2023-11-27T15:28:59.848 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_deviation", 2023-11-27T15:28:59.848 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.848 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.848 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.849 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-11-27T15:28:59.849 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-11-27T15:28:59.849 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.849 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.849 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "deviation below which no optimization is attempted", 2023-11-27T15:28:59.849 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-11-27T15:28:59.849 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.850 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.850 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.850 INFO:teuthology.orchestra.run.smithi133.stdout: "upmap_max_optimizations": { 2023-11-27T15:28:59.850 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "upmap_max_optimizations", 2023-11-27T15:28:59.850 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-11-27T15:28:59.850 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.850 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.851 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:28:59.851 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.851 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.851 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.851 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-11-27T15:28:59.851 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.851 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.852 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.852 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.852 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.852 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.852 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.852 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cephadm", 2023-11-27T15:28:59.852 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.853 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.853 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.853 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_down_multiplier": { 2023-11-27T15:28:59.853 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_down_multiplier", 2023-11-27T15:28:59.853 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:28:59.853 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.853 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.854 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2023-11-27T15:28:59.854 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.854 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.854 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.854 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-11-27T15:28:59.854 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.854 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.855 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.855 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.855 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_refresh_rate": { 2023-11-27T15:28:59.855 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_refresh_rate", 2023-11-27T15:28:59.855 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.855 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.855 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.856 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "20", 2023-11-27T15:28:59.856 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.856 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.856 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.856 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-11-27T15:28:59.856 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.856 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.857 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.857 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.857 INFO:teuthology.orchestra.run.smithi133.stdout: "agent_starting_port": { 2023-11-27T15:28:59.857 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "agent_starting_port", 2023-11-27T15:28:59.857 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.857 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.857 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.858 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4721", 2023-11-27T15:28:59.858 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.858 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.858 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.858 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-11-27T15:28:59.858 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.859 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.859 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.859 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.859 INFO:teuthology.orchestra.run.smithi133.stdout: "alertmanager_web_password": { 2023-11-27T15:28:59.859 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alertmanager_web_password", 2023-11-27T15:28:59.859 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.859 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.860 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.860 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-11-27T15:28:59.860 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.860 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.860 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.860 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Alertmanager web password", 2023-11-27T15:28:59.860 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.860 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.861 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.861 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.861 INFO:teuthology.orchestra.run.smithi133.stdout: "alertmanager_web_user": { 2023-11-27T15:28:59.861 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "alertmanager_web_user", 2023-11-27T15:28:59.861 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.861 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.861 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.862 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-11-27T15:28:59.862 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.862 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.862 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.862 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Alertmanager web user", 2023-11-27T15:28:59.862 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.862 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.863 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.863 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.863 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_ptrace": { 2023-11-27T15:28:59.863 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_ptrace", 2023-11-27T15:28:59.863 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.863 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.863 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.864 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.864 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.864 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.864 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.864 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-11-27T15:28:59.864 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-11-27T15:28:59.864 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.864 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.865 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.865 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_interval": { 2023-11-27T15:28:59.865 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_interval", 2023-11-27T15:28:59.865 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.865 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.865 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.865 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-11-27T15:28:59.866 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.866 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.866 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.866 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to autotune daemon memory", 2023-11-27T15:28:59.866 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.866 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.867 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.867 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.867 INFO:teuthology.orchestra.run.smithi133.stdout: "autotune_memory_target_ratio": { 2023-11-27T15:28:59.867 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "autotune_memory_target_ratio", 2023-11-27T15:28:59.867 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:28:59.867 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.867 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.868 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0.7", 2023-11-27T15:28:59.868 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.868 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.868 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.868 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-11-27T15:28:59.868 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.868 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.868 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.869 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.869 INFO:teuthology.orchestra.run.smithi133.stdout: "cgroups_split": { 2023-11-27T15:28:59.869 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cgroups_split", 2023-11-27T15:28:59.869 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.869 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.869 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.869 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.870 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.870 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.870 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.870 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-11-27T15:28:59.870 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.870 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.870 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.871 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.871 INFO:teuthology.orchestra.run.smithi133.stdout: "config_checks_enabled": { 2023-11-27T15:28:59.871 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_checks_enabled", 2023-11-27T15:28:59.871 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.871 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.871 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.871 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.872 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.872 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.872 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.872 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-11-27T15:28:59.872 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.872 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.872 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.873 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.873 INFO:teuthology.orchestra.run.smithi133.stdout: "config_dashboard": { 2023-11-27T15:28:59.873 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_dashboard", 2023-11-27T15:28:59.873 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.873 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.873 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.873 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.874 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.874 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.874 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.874 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-11-27T15:28:59.874 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.874 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.874 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.875 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.875 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_alertmanager": { 2023-11-27T15:28:59.875 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_alertmanager", 2023-11-27T15:28:59.875 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.875 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.875 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.875 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-11-27T15:28:59.876 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.876 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.876 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.876 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:59.876 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.876 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.876 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.877 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.877 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_base": { 2023-11-27T15:28:59.877 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_base", 2023-11-27T15:28:59.877 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.877 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.877 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.877 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/ceph", 2023-11-27T15:28:59.877 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.878 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.878 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.878 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Container image name, without the tag", 2023-11-27T15:28:59.878 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.878 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.878 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.879 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.879 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_elasticsearch": { 2023-11-27T15:28:59.879 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_elasticsearch", 2023-11-27T15:28:59.879 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.879 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.879 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.879 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-11-27T15:28:59.879 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.880 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.880 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.880 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "elasticsearch container image", 2023-11-27T15:28:59.880 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.880 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.880 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.880 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.881 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_grafana": { 2023-11-27T15:28:59.881 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_grafana", 2023-11-27T15:28:59.881 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.881 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.881 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.881 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-11-27T15:28:59.881 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.882 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.882 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.882 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:59.882 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.882 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.882 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.882 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.883 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_haproxy": { 2023-11-27T15:28:59.883 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_haproxy", 2023-11-27T15:28:59.883 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.883 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.883 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.883 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-11-27T15:28:59.883 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.884 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.884 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.884 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "HAproxy container image", 2023-11-27T15:28:59.884 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.884 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.884 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.884 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.885 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_agent": { 2023-11-27T15:28:59.885 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_agent", 2023-11-27T15:28:59.885 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.885 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.885 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.885 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-11-27T15:28:59.885 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.886 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.886 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.886 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger agent container image", 2023-11-27T15:28:59.886 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.886 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.886 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.886 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.887 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_collector": { 2023-11-27T15:28:59.887 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_collector", 2023-11-27T15:28:59.887 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.887 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.887 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.887 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-11-27T15:28:59.887 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.888 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.888 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.888 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger collector container image", 2023-11-27T15:28:59.888 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.888 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.888 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.888 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.889 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_jaeger_query": { 2023-11-27T15:28:59.889 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_jaeger_query", 2023-11-27T15:28:59.889 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.889 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.889 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.889 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-11-27T15:28:59.889 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.890 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.890 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.890 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Jaeger query container image", 2023-11-27T15:28:59.890 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.890 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.890 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.890 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.891 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_keepalived": { 2023-11-27T15:28:59.891 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_keepalived", 2023-11-27T15:28:59.891 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.891 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.891 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.891 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-11-27T15:28:59.891 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.892 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.892 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.892 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Keepalived container image", 2023-11-27T15:28:59.892 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.892 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.892 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.892 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.893 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_loki": { 2023-11-27T15:28:59.893 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_loki", 2023-11-27T15:28:59.893 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.893 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.893 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.893 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-11-27T15:28:59.893 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.894 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.894 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.894 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Loki container image", 2023-11-27T15:28:59.894 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.894 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.894 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.894 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.895 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_node_exporter": { 2023-11-27T15:28:59.895 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_node_exporter", 2023-11-27T15:28:59.895 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.895 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.895 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.895 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-11-27T15:28:59.895 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.896 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.896 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.896 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:59.896 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.896 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.896 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.896 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.897 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_prometheus": { 2023-11-27T15:28:59.897 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_prometheus", 2023-11-27T15:28:59.897 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.897 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.897 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.897 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-11-27T15:28:59.897 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.898 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.898 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.898 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus container image", 2023-11-27T15:28:59.898 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.898 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.898 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.898 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.899 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_promtail": { 2023-11-27T15:28:59.899 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_promtail", 2023-11-27T15:28:59.899 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.899 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.899 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.899 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-11-27T15:28:59.899 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.900 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.900 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.900 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Promtail container image", 2023-11-27T15:28:59.900 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.900 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.900 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.900 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.901 INFO:teuthology.orchestra.run.smithi133.stdout: "container_image_snmp_gateway": { 2023-11-27T15:28:59.901 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_image_snmp_gateway", 2023-11-27T15:28:59.901 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.901 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.901 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.901 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-11-27T15:28:59.901 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.902 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.902 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.902 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "SNMP Gateway container image", 2023-11-27T15:28:59.902 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.902 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.902 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.902 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.903 INFO:teuthology.orchestra.run.smithi133.stdout: "container_init": { 2023-11-27T15:28:59.903 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "container_init", 2023-11-27T15:28:59.903 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.903 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.903 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.903 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.903 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.904 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.904 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.904 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Run podman/docker with `--init`", 2023-11-27T15:28:59.904 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.904 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.904 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.904 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.905 INFO:teuthology.orchestra.run.smithi133.stdout: "daemon_cache_timeout": { 2023-11-27T15:28:59.905 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "daemon_cache_timeout", 2023-11-27T15:28:59.905 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.905 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.905 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.905 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-11-27T15:28:59.905 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.906 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.906 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.906 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-11-27T15:28:59.906 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.906 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.906 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.906 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.907 INFO:teuthology.orchestra.run.smithi133.stdout: "default_cephadm_command_timeout": { 2023-11-27T15:28:59.907 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_cephadm_command_timeout", 2023-11-27T15:28:59.907 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.907 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.907 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.907 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "900", 2023-11-27T15:28:59.907 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.908 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.908 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.908 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-11-27T15:28:59.908 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.908 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.908 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.908 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.909 INFO:teuthology.orchestra.run.smithi133.stdout: "default_registry": { 2023-11-27T15:28:59.909 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "default_registry", 2023-11-27T15:28:59.909 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.909 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.909 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.909 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "docker.io", 2023-11-27T15:28:59.909 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.910 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.910 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.910 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-11-27T15:28:59.910 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.910 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.910 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.910 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.911 INFO:teuthology.orchestra.run.smithi133.stdout: "device_cache_timeout": { 2023-11-27T15:28:59.911 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_cache_timeout", 2023-11-27T15:28:59.911 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.911 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.911 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.911 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1800", 2023-11-27T15:28:59.911 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.912 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.912 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.912 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache device inventory", 2023-11-27T15:28:59.912 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.912 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.912 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.912 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.913 INFO:teuthology.orchestra.run.smithi133.stdout: "device_enhanced_scan": { 2023-11-27T15:28:59.913 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_enhanced_scan", 2023-11-27T15:28:59.913 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.913 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.913 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.913 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.913 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.914 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.914 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.914 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use libstoragemgmt during device scans", 2023-11-27T15:28:59.914 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.914 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.914 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.914 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.915 INFO:teuthology.orchestra.run.smithi133.stdout: "facts_cache_timeout": { 2023-11-27T15:28:59.915 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "facts_cache_timeout", 2023-11-27T15:28:59.915 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.915 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.915 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.915 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-11-27T15:28:59.915 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.916 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.916 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.916 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "seconds to cache host facts data", 2023-11-27T15:28:59.916 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.916 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.916 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.916 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.917 INFO:teuthology.orchestra.run.smithi133.stdout: "host_check_interval": { 2023-11-27T15:28:59.917 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "host_check_interval", 2023-11-27T15:28:59.917 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.917 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.917 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.917 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-11-27T15:28:59.917 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.918 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.918 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.918 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to perform a host check", 2023-11-27T15:28:59.918 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.918 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.918 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.918 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.919 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.919 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.919 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.919 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.919 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.919 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.919 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.920 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.920 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.920 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.920 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.920 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.920 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.920 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.921 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.921 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.921 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.921 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.921 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.921 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.921 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.922 INFO:teuthology.orchestra.run.smithi133.stdout: "log_refresh_metadata": { 2023-11-27T15:28:59.922 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_refresh_metadata", 2023-11-27T15:28:59.922 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.922 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.922 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.922 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.922 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.923 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.923 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.923 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-11-27T15:28:59.923 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.923 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.923 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.924 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.924 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.924 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.924 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.924 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.924 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.924 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.924 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.925 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.925 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.925 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-11-27T15:28:59.925 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.925 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.925 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.925 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.926 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.926 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.926 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.926 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.926 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.926 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.926 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.927 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.927 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.927 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.927 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.927 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.927 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.927 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.928 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.928 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.928 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.928 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.928 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.928 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.928 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.929 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.929 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.929 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.929 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.929 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.929 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.929 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.930 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.930 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.930 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.930 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.930 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.930 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.930 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.931 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf": { 2023-11-27T15:28:59.931 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-11-27T15:28:59.931 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.931 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.931 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.931 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.931 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.932 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.932 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.932 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-11-27T15:28:59.932 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.932 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.932 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.932 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.933 INFO:teuthology.orchestra.run.smithi133.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-11-27T15:28:59.933 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-11-27T15:28:59.933 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.933 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.933 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.933 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "*", 2023-11-27T15:28:59.933 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.934 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.934 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.934 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-11-27T15:28:59.934 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.934 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.934 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.934 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.935 INFO:teuthology.orchestra.run.smithi133.stdout: "max_count_per_host": { 2023-11-27T15:28:59.935 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_count_per_host", 2023-11-27T15:28:59.935 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.935 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.935 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.935 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:28:59.935 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.936 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.936 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.936 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of daemons per service per host", 2023-11-27T15:28:59.936 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.936 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.936 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.936 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.937 INFO:teuthology.orchestra.run.smithi133.stdout: "max_osd_draining_count": { 2023-11-27T15:28:59.937 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_osd_draining_count", 2023-11-27T15:28:59.937 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.937 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.937 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.937 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:28:59.937 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.938 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.938 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.938 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-11-27T15:28:59.938 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.938 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.938 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.938 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.939 INFO:teuthology.orchestra.run.smithi133.stdout: "migration_current": { 2023-11-27T15:28:59.939 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "migration_current", 2023-11-27T15:28:59.939 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.939 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.939 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.939 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.939 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.939 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.940 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.940 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "internal - do not modify", 2023-11-27T15:28:59.940 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.940 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.940 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.940 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.940 INFO:teuthology.orchestra.run.smithi133.stdout: "mode": { 2023-11-27T15:28:59.941 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mode", 2023-11-27T15:28:59.941 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.941 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.941 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.941 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "root", 2023-11-27T15:28:59.941 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.941 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.942 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.942 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm-package", 2023-11-27T15:28:59.942 INFO:teuthology.orchestra.run.smithi133.stdout: "root" 2023-11-27T15:28:59.942 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.942 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "mode for remote execution of cephadm", 2023-11-27T15:28:59.942 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.942 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.943 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.943 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.943 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_alerts_path": { 2023-11-27T15:28:59.943 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_alerts_path", 2023-11-27T15:28:59.943 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.945 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.945 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.945 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-11-27T15:28:59.945 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.945 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.946 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.946 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-11-27T15:28:59.946 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.946 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.946 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.946 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.947 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_web_password": { 2023-11-27T15:28:59.947 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_web_password", 2023-11-27T15:28:59.947 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.947 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.947 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.947 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-11-27T15:28:59.947 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.948 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.948 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.948 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus web password", 2023-11-27T15:28:59.948 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.949 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.949 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.949 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.950 INFO:teuthology.orchestra.run.smithi133.stdout: "prometheus_web_user": { 2023-11-27T15:28:59.950 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus_web_user", 2023-11-27T15:28:59.950 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.950 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.951 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.951 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-11-27T15:28:59.951 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.951 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.952 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.952 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Prometheus web user", 2023-11-27T15:28:59.952 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.953 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.953 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.953 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.953 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_insecure": { 2023-11-27T15:28:59.954 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_insecure", 2023-11-27T15:28:59.954 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.954 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.955 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.955 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.955 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.955 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.956 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.956 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-11-27T15:28:59.956 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.956 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.957 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.957 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.957 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_password": { 2023-11-27T15:28:59.958 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_password", 2023-11-27T15:28:59.958 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.958 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.958 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.959 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.959 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.959 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.960 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.960 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-11-27T15:28:59.960 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.960 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.961 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.961 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.961 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_url": { 2023-11-27T15:28:59.961 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_url", 2023-11-27T15:28:59.962 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.962 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.962 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.963 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.963 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.963 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.963 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.964 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-11-27T15:28:59.964 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.964 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.964 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.964 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.964 INFO:teuthology.orchestra.run.smithi133.stdout: "registry_username": { 2023-11-27T15:28:59.965 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "registry_username", 2023-11-27T15:28:59.965 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.965 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.965 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.965 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.965 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.965 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.966 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.966 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-11-27T15:28:59.966 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.966 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.966 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.966 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.966 INFO:teuthology.orchestra.run.smithi133.stdout: "secure_monitoring_stack": { 2023-11-27T15:28:59.967 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "secure_monitoring_stack", 2023-11-27T15:28:59.967 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.967 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.967 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.967 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.967 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.967 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.968 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.968 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-11-27T15:28:59.968 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.968 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.968 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.968 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.968 INFO:teuthology.orchestra.run.smithi133.stdout: "service_discovery_port": { 2023-11-27T15:28:59.969 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "service_discovery_port", 2023-11-27T15:28:59.969 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.969 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.969 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.969 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8765", 2023-11-27T15:28:59.969 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.969 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.970 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.970 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "cephadm service discovery port", 2023-11-27T15:28:59.970 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.970 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.970 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.970 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.971 INFO:teuthology.orchestra.run.smithi133.stdout: "ssh_config_file": { 2023-11-27T15:28:59.971 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssh_config_file", 2023-11-27T15:28:59.971 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.971 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.971 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.971 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.971 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.972 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.972 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.972 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-11-27T15:28:59.972 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.972 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.972 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.972 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.973 INFO:teuthology.orchestra.run.smithi133.stdout: "use_agent": { 2023-11-27T15:28:59.973 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_agent", 2023-11-27T15:28:59.973 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.973 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.973 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.973 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.973 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.974 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.974 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.974 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-11-27T15:28:59.974 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.974 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.974 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.974 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.975 INFO:teuthology.orchestra.run.smithi133.stdout: "use_repo_digest": { 2023-11-27T15:28:59.975 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "use_repo_digest", 2023-11-27T15:28:59.975 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.975 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.975 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.975 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.975 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.976 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.976 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.976 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-11-27T15:28:59.976 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.976 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.976 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.976 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.977 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_failed_host_check": { 2023-11-27T15:28:59.977 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_failed_host_check", 2023-11-27T15:28:59.977 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.977 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.977 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.977 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.977 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.978 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.978 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.978 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if the host check fails", 2023-11-27T15:28:59.978 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.978 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.978 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.978 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.979 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_daemons": { 2023-11-27T15:28:59.979 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_daemons", 2023-11-27T15:28:59.979 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.979 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.979 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.979 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.979 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.980 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.980 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.980 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-11-27T15:28:59.980 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.980 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.980 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.980 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.981 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_on_stray_hosts": { 2023-11-27T15:28:59.981 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_on_stray_hosts", 2023-11-27T15:28:59.981 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.981 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.981 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.981 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:28:59.982 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.982 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.982 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.982 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-11-27T15:28:59.982 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.982 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.982 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.983 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.983 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.983 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.983 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.983 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crash", 2023-11-27T15:28:59.983 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.983 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.984 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.984 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:28:59.984 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:28:59.984 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.984 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.984 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.984 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:28:59.985 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.985 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.985 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.985 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.985 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.985 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.985 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.986 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.986 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.986 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.986 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.986 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.986 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.986 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.987 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.987 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:28:59.987 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:28:59.987 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.987 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.987 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.987 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.988 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.988 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.988 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.988 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.988 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.988 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.988 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.989 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.989 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:28:59.989 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:28:59.989 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:28:59.989 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.989 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.989 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:28:59.990 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.990 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.990 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:28:59.990 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:28:59.990 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:28:59.990 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:28:59.990 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:28:59.991 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:28:59.991 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:28:59.991 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:28:59.991 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.991 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.991 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.992 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.992 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.992 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:28:59.992 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:28:59.992 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:28:59.992 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.992 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.992 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:28:59.993 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.993 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.993 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.993 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:28:59.993 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.993 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.993 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.994 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.994 INFO:teuthology.orchestra.run.smithi133.stdout: "retain_interval": { 2023-11-27T15:28:59.994 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retain_interval", 2023-11-27T15:28:59.994 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.994 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.994 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.994 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "31536000", 2023-11-27T15:28:59.995 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.995 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.995 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.995 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain crashes before pruning them", 2023-11-27T15:28:59.995 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.995 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.995 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.996 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.996 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_recent_interval": { 2023-11-27T15:28:59.996 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_recent_interval", 2023-11-27T15:28:59.996 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:28:59.996 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.996 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:28:59.996 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1209600", 2023-11-27T15:28:59.996 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:28:59.997 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:28:59.997 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:28:59.997 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "time interval in which to warn about recent crashes", 2023-11-27T15:28:59.997 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:28:59.997 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:28:59.997 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:28:59.998 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.998 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:28:59.998 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:28:59.998 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:28:59.998 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dashboard", 2023-11-27T15:28:59.998 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:28:59.998 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:28:59.999 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:28:59.999 INFO:teuthology.orchestra.run.smithi133.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-11-27T15:28:59.999 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-11-27T15:28:59.999 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:28:59.999 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:28:59.999 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:28:59.999 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:28:59.999 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.000 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.000 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.000 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.000 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.000 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.000 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.000 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.001 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_HOST": { 2023-11-27T15:29:00.001 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_HOST", 2023-11-27T15:29:00.001 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.001 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.001 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.001 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.001 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.002 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.002 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.002 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.002 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.002 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.002 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.002 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.003 INFO:teuthology.orchestra.run.smithi133.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-11-27T15:29:00.003 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-11-27T15:29:00.003 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.003 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.003 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.003 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.003 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.004 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.004 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.004 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.004 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.004 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.004 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.004 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.004 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_ENABLED": { 2023-11-27T15:29:00.005 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_ENABLED", 2023-11-27T15:29:00.005 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.005 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.005 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.005 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.005 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.005 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.006 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.006 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.006 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.006 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.006 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.006 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.006 INFO:teuthology.orchestra.run.smithi133.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-11-27T15:29:00.007 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-11-27T15:29:00.007 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.007 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.007 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.007 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.007 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.007 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.008 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.008 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.008 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.008 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.008 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.008 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.008 INFO:teuthology.orchestra.run.smithi133.stdout: "ENABLE_BROWSABLE_API": { 2023-11-27T15:29:00.009 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ENABLE_BROWSABLE_API", 2023-11-27T15:29:00.009 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.009 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.009 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.009 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.009 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.009 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.010 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.010 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.010 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.010 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.010 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.010 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.010 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-11-27T15:29:00.011 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-11-27T15:29:00.011 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.011 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.011 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.011 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.011 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.011 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.012 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.012 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.012 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.012 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.012 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.012 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.012 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-11-27T15:29:00.013 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-11-27T15:29:00.013 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.013 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.013 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.013 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.013 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.013 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.014 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.014 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.014 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.014 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.014 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.014 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.014 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-11-27T15:29:00.015 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-11-27T15:29:00.015 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.015 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.015 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.015 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.015 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.015 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.016 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.016 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.016 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.016 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.016 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.016 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.016 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-11-27T15:29:00.017 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-11-27T15:29:00.017 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.017 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.017 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.017 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.017 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.018 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.018 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.018 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.018 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.018 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.018 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.018 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.019 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_NFS": { 2023-11-27T15:29:00.019 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-11-27T15:29:00.019 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.019 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.019 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.019 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.019 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.020 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.020 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.020 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.020 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.020 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.020 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.020 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.021 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RBD": { 2023-11-27T15:29:00.021 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-11-27T15:29:00.021 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.021 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.021 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.021 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.021 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.022 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.022 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.022 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.022 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.022 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.022 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.023 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.023 INFO:teuthology.orchestra.run.smithi133.stdout: "FEATURE_TOGGLE_RGW": { 2023-11-27T15:29:00.023 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-11-27T15:29:00.023 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.023 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.023 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.023 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.024 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.024 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.024 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.024 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.024 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.024 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.024 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.025 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.025 INFO:teuthology.orchestra.run.smithi133.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-11-27T15:29:00.025 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-11-27T15:29:00.025 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.025 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.025 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.025 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.026 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.026 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.026 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.026 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.026 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.026 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.026 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.027 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.027 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_PASSWORD": { 2023-11-27T15:29:00.027 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_PASSWORD", 2023-11-27T15:29:00.027 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.027 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.027 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.028 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-11-27T15:29:00.028 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.028 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.028 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.028 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.028 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.028 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.029 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.029 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.029 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-11-27T15:29:00.029 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-11-27T15:29:00.029 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.029 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.029 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.030 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.030 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.030 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.030 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.030 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.030 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.030 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.031 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.031 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.031 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_URL": { 2023-11-27T15:29:00.031 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_URL", 2023-11-27T15:29:00.031 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.031 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.032 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.032 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.032 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.032 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.032 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.032 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.032 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.033 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.033 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.033 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.033 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_API_USERNAME": { 2023-11-27T15:29:00.033 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_API_USERNAME", 2023-11-27T15:29:00.033 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.033 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.034 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.034 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-11-27T15:29:00.034 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.034 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.034 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.034 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.034 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.035 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.035 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.035 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.035 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-11-27T15:29:00.035 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-11-27T15:29:00.035 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.035 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.036 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.036 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.036 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.036 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.036 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.036 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.036 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.037 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.037 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.037 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.037 INFO:teuthology.orchestra.run.smithi133.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-11-27T15:29:00.037 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-11-27T15:29:00.037 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.038 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.038 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.038 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.038 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.038 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.038 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.038 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.039 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.039 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.039 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.039 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.039 INFO:teuthology.orchestra.run.smithi133.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-11-27T15:29:00.039 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-11-27T15:29:00.039 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.040 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.040 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.040 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.040 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.040 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.040 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.040 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.041 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.041 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.041 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.041 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.041 INFO:teuthology.orchestra.run.smithi133.stdout: "ISSUE_TRACKER_API_KEY": { 2023-11-27T15:29:00.041 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-11-27T15:29:00.041 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.042 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.042 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.042 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.042 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.042 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.042 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.042 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.043 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.043 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.043 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.043 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.043 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_HOST": { 2023-11-27T15:29:00.043 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_HOST", 2023-11-27T15:29:00.044 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.044 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.044 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.044 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.044 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.044 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.044 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.045 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.045 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.045 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.045 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.045 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.045 INFO:teuthology.orchestra.run.smithi133.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-11-27T15:29:00.045 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-11-27T15:29:00.046 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.046 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.046 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.046 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.046 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.046 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.046 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.047 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.047 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.047 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.047 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.047 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.047 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-11-27T15:29:00.047 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-11-27T15:29:00.048 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.048 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.048 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.048 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.048 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.048 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.048 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.049 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.049 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.049 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.049 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.049 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.049 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-11-27T15:29:00.049 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-11-27T15:29:00.050 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.050 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.050 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.050 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.050 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.050 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.050 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.051 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.051 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.051 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.051 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.051 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.051 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-11-27T15:29:00.051 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-11-27T15:29:00.052 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.052 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.052 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.052 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.052 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.052 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.052 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.053 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.053 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.053 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.053 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.053 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.053 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-11-27T15:29:00.053 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-11-27T15:29:00.054 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.054 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.054 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.054 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.054 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.054 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.054 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.055 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.055 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.055 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.055 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.055 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.055 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-11-27T15:29:00.055 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-11-27T15:29:00.056 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.056 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.056 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.056 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.056 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.056 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.056 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.057 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.057 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.057 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.057 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.057 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.057 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-11-27T15:29:00.057 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-11-27T15:29:00.058 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.058 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.058 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.058 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.058 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.058 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.058 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.059 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.059 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.059 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.059 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.059 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.059 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-11-27T15:29:00.060 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-11-27T15:29:00.060 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.060 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.060 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.060 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.060 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.060 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.060 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.061 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.061 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.061 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.061 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.061 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.061 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_ENABLED": { 2023-11-27T15:29:00.061 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_ENABLED", 2023-11-27T15:29:00.062 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.062 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.062 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.062 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.062 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.062 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.062 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.063 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.063 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.063 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.063 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.063 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.063 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-11-27T15:29:00.063 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-11-27T15:29:00.064 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.064 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.064 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.064 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-11-27T15:29:00.064 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.064 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.064 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.065 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.065 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.065 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.065 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.065 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.065 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-11-27T15:29:00.065 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-11-27T15:29:00.066 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.066 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.066 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.066 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:29:00.066 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.066 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.066 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.067 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.067 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.067 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.067 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.067 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.067 INFO:teuthology.orchestra.run.smithi133.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-11-27T15:29:00.067 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-11-27T15:29:00.068 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.068 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.068 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.068 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8", 2023-11-27T15:29:00.068 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.068 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.068 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.069 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.069 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.069 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.069 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.069 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.069 INFO:teuthology.orchestra.run.smithi133.stdout: "REST_REQUESTS_TIMEOUT": { 2023-11-27T15:29:00.069 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-11-27T15:29:00.070 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.070 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.070 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.070 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "45", 2023-11-27T15:29:00.070 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.070 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.070 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.071 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.071 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.071 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.071 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.071 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.071 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ACCESS_KEY": { 2023-11-27T15:29:00.071 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ACCESS_KEY", 2023-11-27T15:29:00.072 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.072 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.072 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.072 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.072 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.072 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.072 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.072 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.073 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.073 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.073 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.073 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.073 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-11-27T15:29:00.073 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-11-27T15:29:00.073 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.074 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.074 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.074 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "admin", 2023-11-27T15:29:00.074 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.074 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.074 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.074 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.075 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.075 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.075 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.075 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.075 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SECRET_KEY": { 2023-11-27T15:29:00.075 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SECRET_KEY", 2023-11-27T15:29:00.075 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.076 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.076 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.076 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.076 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.076 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.076 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.076 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.077 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.077 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.077 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.077 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.077 INFO:teuthology.orchestra.run.smithi133.stdout: "RGW_API_SSL_VERIFY": { 2023-11-27T15:29:00.077 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "RGW_API_SSL_VERIFY", 2023-11-27T15:29:00.077 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.078 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.078 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.078 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.078 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.078 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.078 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.078 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.079 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.079 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.079 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.079 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.079 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-11-27T15:29:00.079 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-11-27T15:29:00.080 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.080 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.080 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.080 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-11-27T15:29:00.080 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.080 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.081 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.081 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.081 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.081 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.081 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.081 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.081 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-11-27T15:29:00.082 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-11-27T15:29:00.082 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.082 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.082 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.082 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:29:00.082 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.082 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.083 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.083 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.083 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.083 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.083 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.083 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.083 INFO:teuthology.orchestra.run.smithi133.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-11-27T15:29:00.084 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-11-27T15:29:00.084 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.084 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.084 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.084 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-11-27T15:29:00.084 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.084 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.085 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.085 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.085 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.085 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.085 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.085 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.085 INFO:teuthology.orchestra.run.smithi133.stdout: "cross_origin_url": { 2023-11-27T15:29:00.086 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cross_origin_url", 2023-11-27T15:29:00.086 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.086 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.086 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.086 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.086 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.086 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.087 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.087 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.087 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.087 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.087 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.087 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.087 INFO:teuthology.orchestra.run.smithi133.stdout: "crt_file": { 2023-11-27T15:29:00.088 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "crt_file", 2023-11-27T15:29:00.088 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.088 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.088 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.088 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.088 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.089 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.089 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.089 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.089 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.089 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.089 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.089 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.090 INFO:teuthology.orchestra.run.smithi133.stdout: "debug": { 2023-11-27T15:29:00.090 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "debug", 2023-11-27T15:29:00.090 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.090 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.090 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.090 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.090 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.091 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.091 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.091 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Enable/disable debug options", 2023-11-27T15:29:00.091 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.091 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.091 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.091 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.092 INFO:teuthology.orchestra.run.smithi133.stdout: "jwt_token_ttl": { 2023-11-27T15:29:00.092 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "jwt_token_ttl", 2023-11-27T15:29:00.092 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.092 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.092 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.092 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "28800", 2023-11-27T15:29:00.092 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.093 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.093 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.093 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.093 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.093 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.093 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.093 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.094 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2023-11-27T15:29:00.094 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2023-11-27T15:29:00.094 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.094 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.094 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.094 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.094 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.095 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.095 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.095 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.095 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.095 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.095 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.095 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.096 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.096 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.096 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.096 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.096 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.096 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.096 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.097 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.097 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.097 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.097 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.097 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.097 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.097 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.098 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.098 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.098 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.098 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.098 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.098 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.098 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.099 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.099 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.099 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.099 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.099 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.099 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.099 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.100 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.100 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.100 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.100 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.100 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.100 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.100 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.101 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.101 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.101 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.101 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.101 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.101 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.101 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.102 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.102 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.102 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.102 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.102 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.102 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.102 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.103 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.103 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.103 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.103 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.103 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.103 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.103 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.104 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.104 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.104 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.104 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.104 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.104 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.104 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.105 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.105 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.105 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.105 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.105 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.105 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.105 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.106 INFO:teuthology.orchestra.run.smithi133.stdout: "motd": { 2023-11-27T15:29:00.106 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "motd", 2023-11-27T15:29:00.106 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.106 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.106 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.106 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.106 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.107 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.107 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.107 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "The message of the day", 2023-11-27T15:29:00.107 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.107 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.107 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.107 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.108 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect_resolve_ip_addr": { 2023-11-27T15:29:00.108 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "redirect_resolve_ip_addr", 2023-11-27T15:29:00.108 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.108 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.108 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.108 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.108 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.109 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.109 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.109 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.109 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.109 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.109 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.109 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.110 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2023-11-27T15:29:00.110 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2023-11-27T15:29:00.110 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.110 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.110 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.110 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2023-11-27T15:29:00.111 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.111 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.111 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.111 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.111 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.111 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.111 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.112 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.112 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2023-11-27T15:29:00.112 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2023-11-27T15:29:00.112 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.112 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.112 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.112 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8080", 2023-11-27T15:29:00.112 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.113 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.113 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.113 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.113 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.113 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.113 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.114 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.114 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2023-11-27T15:29:00.114 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2023-11-27T15:29:00.114 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.114 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.114 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.114 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.115 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.115 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.115 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.115 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.115 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.115 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.115 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.116 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.116 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl_server_port": { 2023-11-27T15:29:00.116 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl_server_port", 2023-11-27T15:29:00.116 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.116 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.116 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.116 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8443", 2023-11-27T15:29:00.116 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.117 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.117 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.117 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.117 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.117 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.117 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.117 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.118 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2023-11-27T15:29:00.118 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2023-11-27T15:29:00.118 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.118 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.118 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.118 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "redirect", 2023-11-27T15:29:00.119 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.119 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.119 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.119 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.119 INFO:teuthology.orchestra.run.smithi133.stdout: "redirect" 2023-11-27T15:29:00.119 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.119 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.120 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.120 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.120 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.120 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.120 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2023-11-27T15:29:00.120 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2023-11-27T15:29:00.120 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.121 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.121 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.121 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2023-11-27T15:29:00.121 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2023-11-27T15:29:00.121 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2023-11-27T15:29:00.121 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.121 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.122 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.122 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.122 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.122 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.122 INFO:teuthology.orchestra.run.smithi133.stdout: "url_prefix": { 2023-11-27T15:29:00.122 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url_prefix", 2023-11-27T15:29:00.122 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.123 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.123 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.123 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.123 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.123 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.123 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.123 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.124 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.124 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.124 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.124 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.124 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.124 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.124 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.125 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "devicehealth", 2023-11-27T15:29:00.125 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.125 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.125 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.125 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_monitoring": { 2023-11-27T15:29:00.125 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_monitoring", 2023-11-27T15:29:00.125 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.126 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.126 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.126 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.126 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.126 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.126 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.126 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "monitor device health metrics", 2023-11-27T15:29:00.127 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.127 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.127 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.127 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.127 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.127 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.127 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.128 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.128 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.128 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.128 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.128 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.128 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.128 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.129 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.129 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.129 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.129 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.129 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.129 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.129 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.130 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.130 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.130 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.130 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.130 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.130 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.131 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.131 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.131 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.131 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.131 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.131 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.131 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.132 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.132 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.132 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.132 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.132 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.132 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.132 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.132 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.133 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.133 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.133 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.133 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.133 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.133 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.134 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.134 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.134 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.134 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.134 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.134 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.134 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.135 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.135 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.135 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.135 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.135 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.135 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.135 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.136 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.136 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.136 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.136 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.136 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.136 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.136 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.137 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.137 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.137 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.137 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.137 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.137 INFO:teuthology.orchestra.run.smithi133.stdout: "mark_out_threshold": { 2023-11-27T15:29:00.137 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mark_out_threshold", 2023-11-27T15:29:00.138 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:29:00.138 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.138 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.138 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "2419200", 2023-11-27T15:29:00.138 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.138 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.138 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.139 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-11-27T15:29:00.139 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.139 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.139 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.139 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.139 INFO:teuthology.orchestra.run.smithi133.stdout: "pool_name": { 2023-11-27T15:29:00.139 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pool_name", 2023-11-27T15:29:00.140 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.140 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.140 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.140 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "device_health_metrics", 2023-11-27T15:29:00.140 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.140 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.140 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.141 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name of pool in which to store device health metrics", 2023-11-27T15:29:00.141 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.141 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.141 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.141 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.141 INFO:teuthology.orchestra.run.smithi133.stdout: "retention_period": { 2023-11-27T15:29:00.141 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "retention_period", 2023-11-27T15:29:00.142 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:29:00.142 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.142 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.142 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15552000", 2023-11-27T15:29:00.142 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.142 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.142 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.143 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long to retain device health metrics", 2023-11-27T15:29:00.143 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.143 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.143 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.143 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.143 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_frequency": { 2023-11-27T15:29:00.143 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_frequency", 2023-11-27T15:29:00.144 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:29:00.144 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.144 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.144 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2023-11-27T15:29:00.144 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.144 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.144 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.145 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to scrape device health metrics", 2023-11-27T15:29:00.145 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.145 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.145 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.145 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.145 INFO:teuthology.orchestra.run.smithi133.stdout: "self_heal": { 2023-11-27T15:29:00.145 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "self_heal", 2023-11-27T15:29:00.146 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.146 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.146 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.146 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.146 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.146 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.146 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.147 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-11-27T15:29:00.147 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.147 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.147 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.147 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.147 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-11-27T15:29:00.147 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-11-27T15:29:00.148 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:29:00.148 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.148 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.148 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-11-27T15:29:00.148 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.148 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.148 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.149 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how frequently to wake up and check device health", 2023-11-27T15:29:00.149 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.149 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.149 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.149 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.149 INFO:teuthology.orchestra.run.smithi133.stdout: "warn_threshold": { 2023-11-27T15:29:00.149 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "warn_threshold", 2023-11-27T15:29:00.150 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:29:00.150 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.150 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.150 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7257600", 2023-11-27T15:29:00.150 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.150 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.150 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.151 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-11-27T15:29:00.151 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.151 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.151 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.151 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.151 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.151 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.152 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.152 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "diskprediction_local", 2023-11-27T15:29:00.152 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.152 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.152 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.152 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.152 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.153 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.153 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.153 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.153 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.153 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.153 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.153 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.154 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.154 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.154 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.154 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.154 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.154 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.154 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.155 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.155 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.155 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.155 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.155 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.155 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.155 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.156 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.156 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.156 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.156 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.156 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.156 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.156 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.157 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.157 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.157 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.157 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.157 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.157 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.157 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.158 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.158 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.158 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.158 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.158 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.158 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.158 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.159 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.159 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.159 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.159 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.159 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.159 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.159 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.160 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.160 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.160 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.160 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.160 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.160 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.160 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.161 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.161 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.161 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.161 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.161 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.161 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.161 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.162 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.162 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.162 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.162 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.162 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.162 INFO:teuthology.orchestra.run.smithi133.stdout: "predict_interval": { 2023-11-27T15:29:00.162 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predict_interval", 2023-11-27T15:29:00.163 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.163 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.163 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.163 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "86400", 2023-11-27T15:29:00.163 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.163 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.163 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.164 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.164 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.164 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.164 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.164 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.164 INFO:teuthology.orchestra.run.smithi133.stdout: "predictor_model": { 2023-11-27T15:29:00.164 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "predictor_model", 2023-11-27T15:29:00.165 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.165 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.165 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.165 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "prophetstor", 2023-11-27T15:29:00.165 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.165 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.165 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.166 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.166 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.166 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.166 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.166 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.166 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-11-27T15:29:00.166 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-11-27T15:29:00.167 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.167 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.167 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.167 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "600", 2023-11-27T15:29:00.167 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.167 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.167 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.168 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.168 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.168 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.168 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.168 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.168 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.168 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.169 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.169 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "influx", 2023-11-27T15:29:00.169 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": false, 2023-11-27T15:29:00.169 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "influxdb python module not found", 2023-11-27T15:29:00.169 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.169 INFO:teuthology.orchestra.run.smithi133.stdout: "batch_size": { 2023-11-27T15:29:00.169 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "batch_size", 2023-11-27T15:29:00.170 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.170 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.170 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.170 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5000", 2023-11-27T15:29:00.170 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.170 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.170 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.171 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-11-27T15:29:00.171 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.171 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.171 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.171 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.171 INFO:teuthology.orchestra.run.smithi133.stdout: "database": { 2023-11-27T15:29:00.171 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "database", 2023-11-27T15:29:00.172 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.172 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.172 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.172 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "ceph", 2023-11-27T15:29:00.172 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.172 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.172 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.173 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-11-27T15:29:00.173 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.173 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.173 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.173 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.173 INFO:teuthology.orchestra.run.smithi133.stdout: "hostname": { 2023-11-27T15:29:00.173 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "hostname", 2023-11-27T15:29:00.174 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.174 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.174 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.174 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.174 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.174 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.174 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.175 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server hostname", 2023-11-27T15:29:00.175 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.175 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.175 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.175 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.175 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-11-27T15:29:00.175 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-11-27T15:29:00.176 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:29:00.176 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.176 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.176 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "30", 2023-11-27T15:29:00.176 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "5", 2023-11-27T15:29:00.176 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.176 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.177 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-11-27T15:29:00.177 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.177 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.177 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.177 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.177 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.177 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.178 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.178 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.178 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.178 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.178 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.178 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.178 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.179 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.179 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.179 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.179 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.179 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.179 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.179 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.180 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.180 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.180 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.180 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.180 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.180 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.180 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.181 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.181 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.181 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.181 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.181 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.181 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.181 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.182 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.182 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.182 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.182 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.182 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.182 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.182 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.183 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.183 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.183 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.183 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.183 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.183 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.183 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.184 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.184 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.184 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.184 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.184 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.184 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.184 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.184 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.185 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.185 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.185 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.185 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.185 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.185 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.186 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.186 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.186 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.186 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.186 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.186 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.186 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.187 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.187 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.187 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.187 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.187 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.187 INFO:teuthology.orchestra.run.smithi133.stdout: "password": { 2023-11-27T15:29:00.187 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "password", 2023-11-27T15:29:00.188 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.188 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.188 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.188 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.188 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.188 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.188 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.189 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "password of InfluxDB server user", 2023-11-27T15:29:00.189 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.189 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.189 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.189 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.189 INFO:teuthology.orchestra.run.smithi133.stdout: "port": { 2023-11-27T15:29:00.189 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "port", 2023-11-27T15:29:00.190 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.190 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.190 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.190 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "8086", 2023-11-27T15:29:00.190 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.191 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.191 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.191 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "InfluxDB server port", 2023-11-27T15:29:00.191 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.191 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.191 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.191 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.192 INFO:teuthology.orchestra.run.smithi133.stdout: "ssl": { 2023-11-27T15:29:00.192 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ssl", 2023-11-27T15:29:00.192 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.192 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.192 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.192 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "false", 2023-11-27T15:29:00.193 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.193 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.193 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.193 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-11-27T15:29:00.193 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.193 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.193 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.194 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.194 INFO:teuthology.orchestra.run.smithi133.stdout: "threads": { 2023-11-27T15:29:00.194 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threads", 2023-11-27T15:29:00.194 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.194 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.194 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.194 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-11-27T15:29:00.195 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-11-27T15:29:00.195 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "32", 2023-11-27T15:29:00.195 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.195 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-11-27T15:29:00.195 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.195 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.195 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.196 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.196 INFO:teuthology.orchestra.run.smithi133.stdout: "username": { 2023-11-27T15:29:00.196 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "username", 2023-11-27T15:29:00.196 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.196 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.196 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.197 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.197 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.197 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.197 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.197 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "username of InfluxDB server user", 2023-11-27T15:29:00.197 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.197 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.198 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.198 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.198 INFO:teuthology.orchestra.run.smithi133.stdout: "verify_ssl": { 2023-11-27T15:29:00.198 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "verify_ssl", 2023-11-27T15:29:00.198 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.198 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.198 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.199 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "true", 2023-11-27T15:29:00.199 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.199 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.199 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.199 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-11-27T15:29:00.199 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.199 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.200 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.200 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.200 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.200 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.200 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.200 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "insights", 2023-11-27T15:29:00.200 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.200 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.201 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.201 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.201 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.201 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.201 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.201 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.201 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.202 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.202 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.202 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.202 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.202 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.202 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.202 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.203 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.203 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.203 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.203 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.203 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.203 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.203 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.204 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.204 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.204 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.204 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.204 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.204 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.204 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.205 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.205 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.205 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.205 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.205 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.205 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.205 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.205 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.206 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.206 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.206 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.206 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.206 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.206 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.207 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.207 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.207 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.207 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.207 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.207 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.207 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.208 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.208 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.208 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.208 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.208 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.208 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.208 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.209 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.209 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.209 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.209 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.209 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.209 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.209 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.210 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.210 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.210 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.210 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.210 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.210 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.210 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.211 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.211 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.211 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.211 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.211 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "iostat", 2023-11-27T15:29:00.211 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.211 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.212 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.212 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.212 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.212 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.212 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.212 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.212 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.212 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.213 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.213 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.213 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.213 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.213 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.213 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.213 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.214 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.214 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.214 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.214 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.214 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.214 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.215 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.215 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.215 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.215 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.215 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.215 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.215 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.215 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.216 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.216 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.216 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.216 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.216 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.216 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.216 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.217 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.217 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.217 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.217 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.217 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.217 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.217 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.218 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.218 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.218 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.218 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.218 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.218 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.219 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.219 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.219 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.219 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.219 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.219 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.219 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.220 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.220 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.220 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.220 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.220 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.220 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.220 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.220 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.221 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.221 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.221 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.221 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.221 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.221 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.221 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.222 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.222 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.222 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.222 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "k8sevents", 2023-11-27T15:29:00.222 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.222 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.222 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.223 INFO:teuthology.orchestra.run.smithi133.stdout: "ceph_event_retention_days": { 2023-11-27T15:29:00.223 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "ceph_event_retention_days", 2023-11-27T15:29:00.223 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.223 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.223 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.223 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "7", 2023-11-27T15:29:00.223 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.224 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.224 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.224 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Days to hold ceph event information within local cache", 2023-11-27T15:29:00.224 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.224 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.224 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.224 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.225 INFO:teuthology.orchestra.run.smithi133.stdout: "config_check_secs": { 2023-11-27T15:29:00.225 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "config_check_secs", 2023-11-27T15:29:00.225 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.225 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.225 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.225 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:29:00.225 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "10", 2023-11-27T15:29:00.226 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.226 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.226 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-11-27T15:29:00.226 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.226 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.226 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.226 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.227 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.227 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.227 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.227 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.227 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.227 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.227 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.228 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.228 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.228 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.228 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.228 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.228 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.228 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.229 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.229 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.229 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.229 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.229 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.229 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.229 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.230 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.230 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.230 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.230 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.230 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.230 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.230 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.231 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.231 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.231 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.231 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.231 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.231 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.231 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.232 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.232 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.232 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.232 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.232 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.232 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.232 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.233 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.233 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.233 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.233 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.233 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.233 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.233 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.234 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.234 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.234 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.234 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.234 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.234 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.234 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.235 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.235 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.235 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.235 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.235 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.235 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.235 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.237 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.237 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.237 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.237 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.238 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.238 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.238 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.238 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.238 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.238 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.238 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "localpool", 2023-11-27T15:29:00.239 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.239 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.239 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.239 INFO:teuthology.orchestra.run.smithi133.stdout: "failure_domain": { 2023-11-27T15:29:00.239 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "failure_domain", 2023-11-27T15:29:00.239 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.239 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.240 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.240 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "host", 2023-11-27T15:29:00.240 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.240 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.240 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.240 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "failure domain for any created local pool", 2023-11-27T15:29:00.240 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-11-27T15:29:00.241 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.241 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.241 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.241 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.241 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.241 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.242 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.242 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.242 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.242 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.242 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.242 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.242 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.243 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.243 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.243 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.243 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.243 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.243 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.243 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.244 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.244 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.244 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.244 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.244 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.244 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.244 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.245 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.245 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.245 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.245 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.245 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.245 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.245 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.246 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.246 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.246 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.246 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.246 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.246 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.246 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.247 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.247 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.247 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.247 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.247 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.247 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.247 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.248 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.248 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.248 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.248 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.248 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.248 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.248 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.249 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.249 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.249 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.249 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.249 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.249 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.249 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.250 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.250 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.250 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.250 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.250 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.250 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.250 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.251 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.251 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.251 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.251 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.251 INFO:teuthology.orchestra.run.smithi133.stdout: "min_size": { 2023-11-27T15:29:00.251 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "min_size", 2023-11-27T15:29:00.251 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.252 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.252 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.252 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.252 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.252 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.252 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.252 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default min_size for any created local pool", 2023-11-27T15:29:00.253 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-11-27T15:29:00.253 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.253 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.253 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.253 INFO:teuthology.orchestra.run.smithi133.stdout: "num_rep": { 2023-11-27T15:29:00.253 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "num_rep", 2023-11-27T15:29:00.253 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.253 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.254 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.254 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3", 2023-11-27T15:29:00.254 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.254 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.254 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.254 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default replica count for any created local pool", 2023-11-27T15:29:00.255 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.255 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.255 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.255 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.255 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_num": { 2023-11-27T15:29:00.255 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_num", 2023-11-27T15:29:00.255 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.256 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.256 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.256 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "128", 2023-11-27T15:29:00.256 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.256 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.256 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.256 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "default pg_num for any created local pool", 2023-11-27T15:29:00.257 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.257 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.257 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.257 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.257 INFO:teuthology.orchestra.run.smithi133.stdout: "prefix": { 2023-11-27T15:29:00.257 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prefix", 2023-11-27T15:29:00.257 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.258 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.258 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.258 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.258 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.258 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.258 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.258 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "name prefix for any created local pool", 2023-11-27T15:29:00.259 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.259 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.259 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.259 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.259 INFO:teuthology.orchestra.run.smithi133.stdout: "subtree": { 2023-11-27T15:29:00.259 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "subtree", 2023-11-27T15:29:00.259 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.260 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.260 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.260 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "rack", 2023-11-27T15:29:00.260 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.260 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.260 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.260 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "CRUSH level for which to create a local pool", 2023-11-27T15:29:00.261 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-11-27T15:29:00.261 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.261 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.261 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.261 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.261 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.261 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.262 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mds_autoscaler", 2023-11-27T15:29:00.262 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.262 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.262 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.262 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.262 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.262 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.263 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.263 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.263 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.263 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.263 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.263 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.263 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.264 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.264 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.264 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.264 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.264 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.264 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.264 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.265 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.265 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.265 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.265 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.265 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.265 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.265 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.266 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.266 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.266 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.266 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.266 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.266 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.266 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.267 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.267 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.267 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.267 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.267 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.267 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.267 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.268 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.268 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.268 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.268 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.268 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.268 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.268 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.269 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.269 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.269 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.269 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.269 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.269 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.269 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.270 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.270 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.270 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.270 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.270 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.270 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.270 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.271 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.271 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.271 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.271 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.271 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.271 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.271 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.272 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.272 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.272 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.272 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.272 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.272 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.272 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.273 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirroring", 2023-11-27T15:29:00.273 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.273 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.273 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.273 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.273 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.273 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.274 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.274 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.274 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.274 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.274 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.274 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.274 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.275 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.275 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.275 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.275 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.275 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.275 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.275 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.275 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.276 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.276 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.276 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.276 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.276 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.276 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.276 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.277 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.277 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.277 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.277 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.277 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.277 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.277 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.278 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.278 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.278 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.278 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.278 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.278 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.279 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.279 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.279 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.279 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.279 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.279 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.279 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.279 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.280 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.280 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.280 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.280 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.280 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.280 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.280 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.281 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.281 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.281 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.281 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.281 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.281 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.281 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.282 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.282 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.282 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.282 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.282 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.282 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.282 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.283 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.283 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.283 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.283 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.283 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.283 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.283 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "nfs", 2023-11-27T15:29:00.284 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.284 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.284 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.284 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.284 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.284 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.284 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.285 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.285 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.285 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.285 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.285 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.285 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.285 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.286 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.286 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.286 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.286 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.286 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.286 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.286 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.287 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.287 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.287 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.287 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.287 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.287 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.287 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.288 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.288 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.288 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.288 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.288 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.288 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.288 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.289 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.289 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.289 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.289 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.289 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.289 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.289 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.290 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.290 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.290 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.290 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.290 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.290 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.290 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.291 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.291 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.291 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.291 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.291 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.291 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.291 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.292 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.292 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.292 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.292 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.292 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.292 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.292 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.293 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.293 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.293 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.293 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.293 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.293 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.293 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.294 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.294 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.294 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.294 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.294 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.294 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.294 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2023-11-27T15:29:00.295 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.295 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.295 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.295 INFO:teuthology.orchestra.run.smithi133.stdout: "fail_fs": { 2023-11-27T15:29:00.295 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "fail_fs", 2023-11-27T15:29:00.295 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.295 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.296 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.296 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.296 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.296 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.296 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.296 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-11-27T15:29:00.296 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.297 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.297 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.297 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.297 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.297 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.297 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.297 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.298 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.298 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.298 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.298 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.298 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.298 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.299 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.299 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.299 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.299 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.299 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.299 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.299 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.300 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.300 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.300 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.300 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.300 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.300 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.300 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.300 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.301 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.301 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.301 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.301 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.301 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.301 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.301 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.302 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.302 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.302 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.302 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.302 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.302 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.302 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.303 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.303 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.303 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.303 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.303 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.303 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.303 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.304 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.304 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.304 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.304 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.304 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.304 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.304 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.305 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.305 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.305 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.305 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.305 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.305 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.305 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.306 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.306 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.306 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.306 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.306 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.306 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.306 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.307 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.307 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.307 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.307 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator": { 2023-11-27T15:29:00.307 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "orchestrator", 2023-11-27T15:29:00.307 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.307 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.308 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.308 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.308 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.308 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.308 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.308 INFO:teuthology.orchestra.run.smithi133.stdout: "cephadm", 2023-11-27T15:29:00.308 INFO:teuthology.orchestra.run.smithi133.stdout: "rook", 2023-11-27T15:29:00.308 INFO:teuthology.orchestra.run.smithi133.stdout: "test_orchestrator" 2023-11-27T15:29:00.309 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.309 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Orchestrator backend", 2023-11-27T15:29:00.309 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.309 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.309 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.309 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.310 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.310 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.310 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.310 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_perf_query", 2023-11-27T15:29:00.310 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.310 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.310 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.311 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.311 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.311 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.311 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.311 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.311 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.311 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.311 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.312 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.312 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.312 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.312 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.312 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.312 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.313 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.313 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.313 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.313 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.313 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.313 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.313 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.314 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.314 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.314 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.314 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.314 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.314 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.314 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.315 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.315 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.315 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.315 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.315 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.315 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.315 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.316 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.316 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.316 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.316 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.316 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.316 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.316 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.317 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.317 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.317 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.317 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.317 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.317 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.317 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.317 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.318 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.318 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.318 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.318 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.318 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.318 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.319 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.319 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.319 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.319 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.319 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.319 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.319 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.319 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.320 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.320 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.320 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.320 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.320 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.320 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.320 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.321 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.321 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.321 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "osd_support", 2023-11-27T15:29:00.321 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.321 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.321 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.322 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.322 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.322 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.322 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.322 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.322 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.322 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.323 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.323 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.323 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.323 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.323 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.323 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.323 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.323 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.324 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.324 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.324 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.324 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.324 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.324 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.324 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.325 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.325 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.325 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.325 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.325 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.325 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.325 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.326 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.326 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.326 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.326 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.326 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.326 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.326 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.327 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.327 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.327 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.327 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.327 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.327 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.327 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.328 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.328 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.328 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.328 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.328 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.328 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.328 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.329 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.329 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.329 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.329 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.329 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.329 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.329 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.330 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.330 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.330 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.330 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.330 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.330 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.330 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.331 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.331 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.331 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.331 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.331 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.331 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.331 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.332 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.332 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.332 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "pg_autoscaler", 2023-11-27T15:29:00.332 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.332 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.332 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.332 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.333 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.333 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.333 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.333 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.333 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.333 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.333 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.334 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.334 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.334 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.334 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.334 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.334 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.334 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.335 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.335 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.335 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.335 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.335 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.335 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.335 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.335 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.336 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.336 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.336 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.336 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.336 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.336 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.336 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.337 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.337 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.337 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.337 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.337 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.337 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.337 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.338 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.338 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.338 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.338 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.338 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.338 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.338 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.339 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.339 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.339 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.339 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.339 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.339 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.339 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.340 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.340 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.340 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.340 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.340 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.340 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.340 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.341 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.341 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.341 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.341 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.341 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.341 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.341 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.342 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.342 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.342 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.342 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.342 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.342 INFO:teuthology.orchestra.run.smithi133.stdout: "noautoscale": { 2023-11-27T15:29:00.343 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "noautoscale", 2023-11-27T15:29:00.343 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.343 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.343 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.343 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.343 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.343 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.343 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.344 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "global autoscale flag", 2023-11-27T15:29:00.344 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-11-27T15:29:00.344 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.344 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.344 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.344 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-11-27T15:29:00.344 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-11-27T15:29:00.345 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:29:00.345 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.345 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.345 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-11-27T15:29:00.345 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.345 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.345 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.346 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.346 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.346 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.346 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.346 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.346 INFO:teuthology.orchestra.run.smithi133.stdout: "threshold": { 2023-11-27T15:29:00.346 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "threshold", 2023-11-27T15:29:00.347 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:29:00.347 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.347 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.347 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "3.0", 2023-11-27T15:29:00.347 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1.0", 2023-11-27T15:29:00.347 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.347 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.348 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "scaling threshold", 2023-11-27T15:29:00.348 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-11-27T15:29:00.348 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.348 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.348 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.348 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.348 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.349 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.349 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "progress", 2023-11-27T15:29:00.349 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.349 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.349 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.349 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_pg_recovery_event": { 2023-11-27T15:29:00.349 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_pg_recovery_event", 2023-11-27T15:29:00.350 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.350 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.350 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.350 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.350 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.350 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.350 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.351 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow the module to show pg recovery progress", 2023-11-27T15:29:00.351 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.351 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.351 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.351 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.351 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2023-11-27T15:29:00.351 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2023-11-27T15:29:00.352 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.352 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.352 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.352 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.352 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.352 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.353 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.353 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.353 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.353 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.353 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.353 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.353 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.353 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.354 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.354 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.354 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.354 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.354 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.354 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.354 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.355 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.355 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.355 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.355 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.355 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.355 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.355 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.356 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.356 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.356 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.356 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.356 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.356 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.356 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.357 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.357 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.357 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.357 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.357 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.357 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.357 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.358 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.358 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.358 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.358 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.358 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.358 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.358 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.359 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.359 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.359 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.359 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.359 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.359 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.359 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.360 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.360 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.360 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.360 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.360 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.360 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.360 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.360 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.361 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.361 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.361 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.361 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.361 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.361 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.361 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.362 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.362 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.362 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.362 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.362 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.362 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.362 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.363 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.363 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.363 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.363 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.363 INFO:teuthology.orchestra.run.smithi133.stdout: "max_completed_events": { 2023-11-27T15:29:00.363 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_completed_events", 2023-11-27T15:29:00.364 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.364 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.364 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.364 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "50", 2023-11-27T15:29:00.364 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.364 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.365 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.365 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "number of past completed events to remember", 2023-11-27T15:29:00.365 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.365 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.365 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.365 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.365 INFO:teuthology.orchestra.run.smithi133.stdout: "sleep_interval": { 2023-11-27T15:29:00.366 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "sleep_interval", 2023-11-27T15:29:00.366 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:29:00.366 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.366 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.366 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "5", 2023-11-27T15:29:00.366 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.367 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.367 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.367 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "how long the module is going to sleep", 2023-11-27T15:29:00.367 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.367 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.368 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.368 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.368 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.368 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.368 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.368 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "prometheus", 2023-11-27T15:29:00.368 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.369 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.369 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.369 INFO:teuthology.orchestra.run.smithi133.stdout: "cache": { 2023-11-27T15:29:00.369 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "cache", 2023-11-27T15:29:00.369 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.370 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.370 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.370 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.370 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.370 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.370 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.371 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.371 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.371 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.371 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.371 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.371 INFO:teuthology.orchestra.run.smithi133.stdout: "exclude_perf_counters": { 2023-11-27T15:29:00.372 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "exclude_perf_counters", 2023-11-27T15:29:00.372 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.372 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.372 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.372 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.372 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.373 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.373 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.373 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-11-27T15:29:00.373 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-11-27T15:29:00.373 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.373 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.374 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.374 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.374 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.374 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.374 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.374 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.375 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.375 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.375 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.375 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.375 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.375 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.375 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.376 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.376 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.376 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.376 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.376 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.376 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.376 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.377 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.377 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.377 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.377 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.377 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.377 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.377 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.378 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.378 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.378 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.378 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.378 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.378 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.378 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.379 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.379 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.379 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.379 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.379 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.379 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.379 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.380 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.380 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.380 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.380 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.380 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.380 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.380 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.381 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.381 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.381 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.381 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.381 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.381 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.381 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.382 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.382 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.382 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.382 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.382 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.382 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.382 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.383 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.383 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.383 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.383 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.383 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.383 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.384 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.384 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.384 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.384 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools": { 2023-11-27T15:29:00.384 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools", 2023-11-27T15:29:00.384 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.384 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.385 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.385 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.385 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.385 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.385 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.385 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.385 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.385 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.386 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.386 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.386 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_stats_pools_refresh_interval": { 2023-11-27T15:29:00.386 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-11-27T15:29:00.386 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.386 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.387 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.387 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "300", 2023-11-27T15:29:00.387 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.387 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.387 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.387 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.387 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.388 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.388 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.388 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.388 INFO:teuthology.orchestra.run.smithi133.stdout: "scrape_interval": { 2023-11-27T15:29:00.388 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "scrape_interval", 2023-11-27T15:29:00.388 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:29:00.388 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.389 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.389 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15.0", 2023-11-27T15:29:00.389 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.389 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.389 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.389 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.389 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.390 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.390 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.390 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.390 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2023-11-27T15:29:00.390 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2023-11-27T15:29:00.390 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.390 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.391 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.391 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "::", 2023-11-27T15:29:00.391 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.391 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.391 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.391 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-11-27T15:29:00.391 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.392 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.392 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.392 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.392 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2023-11-27T15:29:00.392 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2023-11-27T15:29:00.392 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.392 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.393 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.393 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "9283", 2023-11-27T15:29:00.393 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.393 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.393 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.393 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-11-27T15:29:00.393 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.394 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.394 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.394 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.394 INFO:teuthology.orchestra.run.smithi133.stdout: "stale_cache_strategy": { 2023-11-27T15:29:00.394 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stale_cache_strategy", 2023-11-27T15:29:00.394 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.394 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.395 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.395 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "log", 2023-11-27T15:29:00.395 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.395 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.395 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.395 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.395 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.396 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.396 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.396 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.396 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_behaviour": { 2023-11-27T15:29:00.396 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_behaviour", 2023-11-27T15:29:00.396 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.396 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.397 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.397 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "default", 2023-11-27T15:29:00.397 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.397 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.397 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.397 INFO:teuthology.orchestra.run.smithi133.stdout: "default", 2023-11-27T15:29:00.397 INFO:teuthology.orchestra.run.smithi133.stdout: "error" 2023-11-27T15:29:00.398 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.398 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.398 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.398 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.398 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.398 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.398 INFO:teuthology.orchestra.run.smithi133.stdout: "standby_error_status_code": { 2023-11-27T15:29:00.399 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "standby_error_status_code", 2023-11-27T15:29:00.399 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.399 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.399 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.399 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "500", 2023-11-27T15:29:00.399 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "400", 2023-11-27T15:29:00.399 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "599", 2023-11-27T15:29:00.400 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.400 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.400 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.400 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.400 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.400 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.400 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.401 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.401 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.401 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_support", 2023-11-27T15:29:00.401 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.401 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.401 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.401 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.401 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.402 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.402 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.402 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.402 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.402 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.402 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.403 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.403 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.403 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.403 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.403 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.403 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.403 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.404 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.404 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.404 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.404 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.404 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.404 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.404 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.404 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.405 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.405 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.405 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.405 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.405 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.405 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.405 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.406 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.406 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.406 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.406 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.406 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.406 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.406 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.407 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.407 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.407 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.407 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.407 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.407 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.407 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.408 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.408 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.408 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.408 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.408 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.408 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.408 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.409 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.409 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.409 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.409 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.409 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.409 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.409 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.410 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.410 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.410 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.410 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.410 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.410 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.410 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.411 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.411 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.411 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.411 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.411 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.411 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_snap_create": { 2023-11-27T15:29:00.411 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_snap_create", 2023-11-27T15:29:00.412 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.412 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.412 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.412 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10", 2023-11-27T15:29:00.412 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.412 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.412 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.413 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.413 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.413 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.413 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.413 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.413 INFO:teuthology.orchestra.run.smithi133.stdout: "mirror_snapshot_schedule": { 2023-11-27T15:29:00.413 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "mirror_snapshot_schedule", 2023-11-27T15:29:00.414 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.414 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.414 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.414 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.414 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.414 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.414 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.415 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.415 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.415 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.415 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.415 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.415 INFO:teuthology.orchestra.run.smithi133.stdout: "trash_purge_schedule": { 2023-11-27T15:29:00.416 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "trash_purge_schedule", 2023-11-27T15:29:00.416 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.416 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.416 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.416 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.416 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.416 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.417 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.417 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.417 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.417 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.417 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.417 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.417 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.418 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.418 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.418 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "restful", 2023-11-27T15:29:00.418 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.418 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.418 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.418 INFO:teuthology.orchestra.run.smithi133.stdout: "enable_auth": { 2023-11-27T15:29:00.419 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enable_auth", 2023-11-27T15:29:00.419 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.419 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.419 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.419 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.419 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.419 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.419 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.420 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.420 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.420 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.420 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.420 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.420 INFO:teuthology.orchestra.run.smithi133.stdout: "key_file": { 2023-11-27T15:29:00.420 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "key_file", 2023-11-27T15:29:00.421 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.421 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.421 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.421 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.421 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.421 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.421 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.422 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.422 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.422 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.422 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.422 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.422 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.422 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.423 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.423 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.423 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.423 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.423 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.423 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.423 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.424 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.424 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.424 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.424 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.424 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.424 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.424 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.424 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.425 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.425 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.425 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.425 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.425 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.425 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.425 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.426 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.426 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.426 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.426 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.426 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.426 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.426 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.427 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.427 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.427 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.427 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.427 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.427 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.427 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.428 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.428 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.428 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.428 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.428 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.428 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.428 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.429 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.429 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.429 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.429 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.429 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.429 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.430 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.430 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.430 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.430 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.430 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.430 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.430 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.431 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.431 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.431 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.431 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.431 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.431 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.431 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.432 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.432 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.432 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.432 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.432 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.432 INFO:teuthology.orchestra.run.smithi133.stdout: "server_addr": { 2023-11-27T15:29:00.432 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_addr", 2023-11-27T15:29:00.433 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.433 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.433 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.433 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.433 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.433 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.433 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.434 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.434 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.434 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.434 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.434 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.434 INFO:teuthology.orchestra.run.smithi133.stdout: "server_port": { 2023-11-27T15:29:00.434 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "server_port", 2023-11-27T15:29:00.435 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.435 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.435 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.435 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.435 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.435 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.435 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.436 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.436 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.436 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.436 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.436 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.436 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.436 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.437 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.437 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rgw", 2023-11-27T15:29:00.437 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.437 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.437 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.437 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.437 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.438 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.438 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.438 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.438 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.438 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.438 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.438 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.439 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.439 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.439 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.439 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.439 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.439 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.441 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.441 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.441 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.441 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.441 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.442 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.442 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.442 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.442 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.442 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.442 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.442 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.443 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.443 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.443 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.443 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.443 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.443 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.443 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.443 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.443 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.444 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.444 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.444 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.444 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.444 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.444 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.444 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.444 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.445 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.445 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.445 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.445 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.445 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.445 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.445 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.445 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.446 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.446 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.446 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.446 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.446 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.446 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.446 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.446 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.447 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.447 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.447 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.447 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.447 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.447 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.447 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.447 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.447 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.448 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.448 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.448 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.448 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.448 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rook", 2023-11-27T15:29:00.448 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.448 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.448 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.449 INFO:teuthology.orchestra.run.smithi133.stdout: "drive_group_interval": { 2023-11-27T15:29:00.449 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "drive_group_interval", 2023-11-27T15:29:00.449 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:29:00.449 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.449 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.449 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "300.0", 2023-11-27T15:29:00.449 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.449 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.450 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.450 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-11-27T15:29:00.450 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.450 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.450 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.450 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.450 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.451 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.451 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.451 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.451 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.451 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.451 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.451 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.451 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.451 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.452 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.452 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.452 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.452 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.452 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.452 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.452 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.452 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.453 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.453 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.453 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.453 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.453 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.453 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.453 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.453 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.454 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.454 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.454 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.454 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.454 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.454 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.454 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.454 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.454 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.455 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.455 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.455 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.455 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.455 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.455 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.455 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.455 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.456 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.456 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.456 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.456 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.456 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.456 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.456 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.456 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.456 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.457 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.457 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.457 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.457 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.457 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.457 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.457 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.457 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.458 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.458 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.458 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.458 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.458 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.458 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.458 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.458 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.459 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.459 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.459 INFO:teuthology.orchestra.run.smithi133.stdout: "storage_class": { 2023-11-27T15:29:00.459 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "storage_class", 2023-11-27T15:29:00.459 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.459 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.459 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.459 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "local", 2023-11-27T15:29:00.460 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.460 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.460 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.460 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-11-27T15:29:00.460 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.460 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.460 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.460 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.460 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.461 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.461 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.461 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "selftest", 2023-11-27T15:29:00.461 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.461 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.461 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.461 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.461 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.462 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.462 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.462 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.462 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.462 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.462 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.462 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.462 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.462 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.463 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.463 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.463 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.463 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.463 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.463 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.463 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.463 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.464 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.464 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.464 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.464 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.464 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.464 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.464 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.464 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.465 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.465 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.465 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.465 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.465 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.465 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.465 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.465 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.465 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.466 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.466 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.466 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.466 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.466 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.466 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.466 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.466 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.467 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.467 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.467 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.467 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.467 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.467 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.467 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.467 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.468 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.468 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.468 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.468 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.468 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.468 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.468 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.468 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.468 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.469 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.469 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.469 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.469 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.469 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.469 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.469 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.469 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.469 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.470 INFO:teuthology.orchestra.run.smithi133.stdout: "roption1": { 2023-11-27T15:29:00.470 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption1", 2023-11-27T15:29:00.470 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.470 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.470 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.470 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.470 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.471 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.471 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.471 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.471 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.471 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.471 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.471 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.471 INFO:teuthology.orchestra.run.smithi133.stdout: "roption2": { 2023-11-27T15:29:00.471 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "roption2", 2023-11-27T15:29:00.472 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.472 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.472 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.472 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "xyz", 2023-11-27T15:29:00.472 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.472 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.472 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.472 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.473 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.473 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.473 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.473 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.473 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption1": { 2023-11-27T15:29:00.473 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption1", 2023-11-27T15:29:00.473 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.473 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.473 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.474 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.474 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.474 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.474 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.474 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.474 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.474 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.474 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.475 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.475 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption2": { 2023-11-27T15:29:00.475 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption2", 2023-11-27T15:29:00.475 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.475 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.475 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.475 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.475 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.476 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.476 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.476 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.476 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.476 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.476 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.476 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.476 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption3": { 2023-11-27T15:29:00.476 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption3", 2023-11-27T15:29:00.477 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "float", 2023-11-27T15:29:00.477 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.477 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.477 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.477 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.477 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.477 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.477 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.478 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.478 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.478 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.478 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.478 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption4": { 2023-11-27T15:29:00.478 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption4", 2023-11-27T15:29:00.478 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.478 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.479 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.479 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.479 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.479 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.479 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.479 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.479 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.479 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.480 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.480 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.480 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption5": { 2023-11-27T15:29:00.480 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption5", 2023-11-27T15:29:00.480 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.480 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.480 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.480 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.480 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.481 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.481 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.481 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.481 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.481 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.481 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.481 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.481 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption6": { 2023-11-27T15:29:00.482 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption6", 2023-11-27T15:29:00.482 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.482 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.482 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.482 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.482 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.482 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.482 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.483 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.483 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.483 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.483 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.483 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.483 INFO:teuthology.orchestra.run.smithi133.stdout: "rwoption7": { 2023-11-27T15:29:00.483 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rwoption7", 2023-11-27T15:29:00.483 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.484 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.484 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.484 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.484 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "1", 2023-11-27T15:29:00.484 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "42", 2023-11-27T15:29:00.484 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.484 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.484 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.484 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.485 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.485 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.485 INFO:teuthology.orchestra.run.smithi133.stdout: "testkey": { 2023-11-27T15:29:00.485 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testkey", 2023-11-27T15:29:00.485 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.485 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.485 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.485 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.486 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.486 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.486 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.486 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.486 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.486 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.486 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.486 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.487 INFO:teuthology.orchestra.run.smithi133.stdout: "testlkey": { 2023-11-27T15:29:00.487 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testlkey", 2023-11-27T15:29:00.487 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.487 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.487 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.487 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.487 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.487 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.487 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.488 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.488 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.488 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.488 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.488 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.488 INFO:teuthology.orchestra.run.smithi133.stdout: "testnewline": { 2023-11-27T15:29:00.488 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "testnewline", 2023-11-27T15:29:00.488 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.489 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.489 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.489 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.489 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.489 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.489 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.489 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.489 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.489 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.490 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.490 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.490 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.490 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.490 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.490 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snap_schedule", 2023-11-27T15:29:00.490 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.490 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.491 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.491 INFO:teuthology.orchestra.run.smithi133.stdout: "allow_m_granularity": { 2023-11-27T15:29:00.491 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "allow_m_granularity", 2023-11-27T15:29:00.491 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.491 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.491 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.491 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.491 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.492 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.492 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.492 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "allow minute scheduled snapshots", 2023-11-27T15:29:00.492 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.492 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.492 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.492 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.492 INFO:teuthology.orchestra.run.smithi133.stdout: "dump_on_update": { 2023-11-27T15:29:00.492 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "dump_on_update", 2023-11-27T15:29:00.493 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.493 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.493 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.493 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.493 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.493 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.493 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.493 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "dump database to debug log on update", 2023-11-27T15:29:00.494 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.494 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.494 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.494 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.494 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.494 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.494 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.494 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.494 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.495 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.495 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.495 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.495 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.495 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.495 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.495 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.495 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.496 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.496 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.496 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.496 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.496 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.496 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.496 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.496 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.496 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.497 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.497 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.497 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.497 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.497 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.497 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.497 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.497 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.498 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.498 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.498 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.498 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.498 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.498 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.498 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.498 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.499 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.499 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.499 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.499 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.499 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.499 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.499 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.499 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.499 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.500 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.500 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.500 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.500 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.500 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.500 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.500 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.500 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.501 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.501 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.501 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.501 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.501 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.501 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.501 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.501 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.502 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.502 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.502 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.502 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.502 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.502 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.502 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.502 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.503 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.503 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.503 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "stats", 2023-11-27T15:29:00.503 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.503 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.503 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.503 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.503 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.503 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.504 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.504 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.504 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.504 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.504 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.504 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.504 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.504 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.505 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.505 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.505 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.505 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.505 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.505 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.505 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.505 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.505 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.506 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.506 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.506 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.506 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.506 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.506 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.506 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.506 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.507 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.507 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.507 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.507 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.507 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.507 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.507 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.507 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.508 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.508 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.508 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.508 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.508 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.508 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.508 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.508 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.508 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.509 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.509 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.509 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.509 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.509 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.509 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.509 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.509 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.510 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.510 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.510 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.510 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.510 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.510 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.510 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.510 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.511 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.511 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.511 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.511 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.511 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.511 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.511 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.511 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.511 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.512 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.512 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.512 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.512 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "status", 2023-11-27T15:29:00.512 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.512 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.512 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.512 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.512 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.513 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.513 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.513 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.513 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.513 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.513 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.513 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.513 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.513 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.514 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.514 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.514 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.514 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.514 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.514 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.514 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.514 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.515 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.515 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.515 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.515 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.515 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.515 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.515 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.515 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.516 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.516 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.516 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.516 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.516 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.516 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.516 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.516 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.516 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.517 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.517 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.517 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.517 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.517 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.517 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.517 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.517 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.518 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.518 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.518 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.518 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.518 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.518 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.518 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.518 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.519 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.519 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.519 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.519 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.519 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.519 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.519 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.519 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.520 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.520 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.520 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.520 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.520 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.520 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.520 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.520 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.521 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.521 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.521 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.521 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.521 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.521 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telegraf", 2023-11-27T15:29:00.521 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.521 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.521 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.522 INFO:teuthology.orchestra.run.smithi133.stdout: "address": { 2023-11-27T15:29:00.522 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "address", 2023-11-27T15:29:00.522 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.522 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.522 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.522 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-11-27T15:29:00.522 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.523 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.523 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.523 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.523 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.523 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.523 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.523 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.523 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-11-27T15:29:00.523 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-11-27T15:29:00.524 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:29:00.524 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.524 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.524 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "15", 2023-11-27T15:29:00.524 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.524 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.524 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.524 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.524 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.525 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.525 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.525 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.525 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.525 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.525 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.525 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.525 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.526 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.526 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.526 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.526 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.526 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.526 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.526 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.526 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.527 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.527 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.527 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.527 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.527 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.527 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.527 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.527 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.528 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.528 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.528 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.528 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.528 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.528 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.528 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.528 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.528 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.529 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.529 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.529 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.529 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.529 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.529 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.529 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.529 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.530 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.530 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.530 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.530 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.530 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.530 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.530 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.530 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.531 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.531 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.531 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.531 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.531 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.531 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.531 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.531 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.531 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.532 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.532 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.532 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.532 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.532 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.532 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.532 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.532 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.533 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.533 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.533 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.533 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.533 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.533 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.533 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.533 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.534 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.534 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.534 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "telemetry", 2023-11-27T15:29:00.534 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.534 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.534 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.534 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_basic": { 2023-11-27T15:29:00.534 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_basic", 2023-11-27T15:29:00.535 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.535 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.535 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.535 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.535 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.535 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.535 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.535 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share basic cluster information (size, version)", 2023-11-27T15:29:00.536 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.536 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.536 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.536 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.536 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_crash": { 2023-11-27T15:29:00.536 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_crash", 2023-11-27T15:29:00.536 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.536 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.536 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.537 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.537 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.537 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.537 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.537 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-11-27T15:29:00.537 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.537 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.537 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.538 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.538 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_device": { 2023-11-27T15:29:00.538 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_device", 2023-11-27T15:29:00.538 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.538 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.538 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.538 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "True", 2023-11-27T15:29:00.538 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.539 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.539 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.539 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-11-27T15:29:00.539 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.539 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.539 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.539 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.539 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_ident": { 2023-11-27T15:29:00.540 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_ident", 2023-11-27T15:29:00.540 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.540 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.540 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.540 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.540 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.540 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.540 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.540 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-11-27T15:29:00.541 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.541 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.541 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.541 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.541 INFO:teuthology.orchestra.run.smithi133.stdout: "channel_perf": { 2023-11-27T15:29:00.541 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "channel_perf", 2023-11-27T15:29:00.541 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.541 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.542 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.542 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.542 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.542 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.542 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.542 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Share various performance metrics of a cluster", 2023-11-27T15:29:00.542 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.542 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.543 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.543 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.543 INFO:teuthology.orchestra.run.smithi133.stdout: "contact": { 2023-11-27T15:29:00.543 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "contact", 2023-11-27T15:29:00.543 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.543 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.543 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.543 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.544 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.544 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.544 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.544 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.544 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.544 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.544 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.544 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.544 INFO:teuthology.orchestra.run.smithi133.stdout: "description": { 2023-11-27T15:29:00.545 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "description", 2023-11-27T15:29:00.545 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.545 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.545 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.545 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.545 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.545 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.545 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.546 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.546 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.546 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.546 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.546 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.546 INFO:teuthology.orchestra.run.smithi133.stdout: "device_url": { 2023-11-27T15:29:00.546 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "device_url", 2023-11-27T15:29:00.546 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.547 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.547 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.547 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-11-27T15:29:00.547 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.547 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.547 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.547 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.547 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.548 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.548 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.548 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.548 INFO:teuthology.orchestra.run.smithi133.stdout: "enabled": { 2023-11-27T15:29:00.548 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "enabled", 2023-11-27T15:29:00.548 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.548 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.548 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.549 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.549 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.549 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.549 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.549 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.549 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.549 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.549 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.550 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.550 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-11-27T15:29:00.550 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-11-27T15:29:00.550 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.550 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.550 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.550 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "24", 2023-11-27T15:29:00.550 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "8", 2023-11-27T15:29:00.551 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.551 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.551 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.551 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.551 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.551 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.551 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.551 INFO:teuthology.orchestra.run.smithi133.stdout: "last_opt_revision": { 2023-11-27T15:29:00.552 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "last_opt_revision", 2023-11-27T15:29:00.552 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.552 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.552 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.552 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "1", 2023-11-27T15:29:00.552 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.552 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.552 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.553 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.553 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.553 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.553 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.553 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.553 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard": { 2023-11-27T15:29:00.553 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard", 2023-11-27T15:29:00.553 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.554 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.554 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.554 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.554 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.554 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.554 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.554 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.554 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.555 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.555 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.555 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.555 INFO:teuthology.orchestra.run.smithi133.stdout: "leaderboard_description": { 2023-11-27T15:29:00.555 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "leaderboard_description", 2023-11-27T15:29:00.555 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.555 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.555 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.555 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.556 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.556 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.556 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.556 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.556 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.556 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.556 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.556 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.557 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.557 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.557 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.557 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.557 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.557 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.557 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.557 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.558 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.558 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.558 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.558 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.558 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.558 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.558 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.558 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.559 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.559 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.559 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.559 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.559 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.559 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.559 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.559 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.560 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.560 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.560 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.560 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.560 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.560 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.560 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.560 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.561 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.561 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.561 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.561 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.561 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.561 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.561 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.561 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.561 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.562 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.562 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.562 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.562 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.562 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.562 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.562 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.562 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.563 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.563 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.563 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.563 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.563 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.563 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.563 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.563 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.564 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.564 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.564 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.564 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.564 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.564 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.564 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.564 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.564 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.565 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.565 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.565 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.565 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.565 INFO:teuthology.orchestra.run.smithi133.stdout: "organization": { 2023-11-27T15:29:00.565 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "organization", 2023-11-27T15:29:00.565 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.565 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.566 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.566 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.566 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.566 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.566 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.566 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.566 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.566 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.567 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.567 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.567 INFO:teuthology.orchestra.run.smithi133.stdout: "proxy": { 2023-11-27T15:29:00.567 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "proxy", 2023-11-27T15:29:00.567 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.567 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.567 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.567 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.568 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.568 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.568 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.568 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.568 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.568 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.568 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.568 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.568 INFO:teuthology.orchestra.run.smithi133.stdout: "url": { 2023-11-27T15:29:00.569 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "url", 2023-11-27T15:29:00.569 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.569 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.569 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.569 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-11-27T15:29:00.569 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.569 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.569 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.570 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.570 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.570 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.570 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.570 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.570 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.570 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.570 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.571 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "test_orchestrator", 2023-11-27T15:29:00.571 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.571 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.571 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.571 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.571 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.571 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.571 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.572 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.572 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.572 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.572 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.572 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.572 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.572 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.572 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.573 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.573 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.573 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.573 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.573 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.573 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.573 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.573 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.573 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.574 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.574 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.574 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.574 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.574 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.574 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.574 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.575 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.575 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.575 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.575 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.575 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.575 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.575 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.575 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.576 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.576 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.576 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.576 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.576 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.576 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.576 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.576 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.576 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.577 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.577 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.577 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.577 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.577 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.577 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.577 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.578 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.578 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.578 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.578 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.578 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.578 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.578 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.578 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.579 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.579 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.579 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.579 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.579 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.579 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.579 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.579 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.580 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.580 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.580 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.580 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.580 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.580 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "volumes", 2023-11-27T15:29:00.580 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.580 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.580 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.581 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.581 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.581 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.581 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.581 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.581 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.581 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.581 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.582 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.582 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.582 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.582 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.582 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.582 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.582 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.582 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.583 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.583 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.583 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.583 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.583 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.583 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.583 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.583 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.583 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.584 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.584 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.584 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.584 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.584 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.584 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.584 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.584 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.585 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.585 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.585 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.585 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.585 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.585 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.585 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.585 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.586 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.586 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.586 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.586 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.586 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.586 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.586 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.586 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.587 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.587 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.587 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.587 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.587 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.587 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.587 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.587 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.588 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.588 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.588 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.588 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.588 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.588 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.588 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.588 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.588 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.589 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.589 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.589 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.589 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.589 INFO:teuthology.orchestra.run.smithi133.stdout: "max_concurrent_clones": { 2023-11-27T15:29:00.589 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "max_concurrent_clones", 2023-11-27T15:29:00.589 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.589 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.590 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.590 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "4", 2023-11-27T15:29:00.590 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.590 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.590 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.590 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Number of asynchronous cloner threads", 2023-11-27T15:29:00.590 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.590 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.591 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.591 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.591 INFO:teuthology.orchestra.run.smithi133.stdout: "snapshot_clone_delay": { 2023-11-27T15:29:00.591 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "snapshot_clone_delay", 2023-11-27T15:29:00.591 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.591 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.591 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.591 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "0", 2023-11-27T15:29:00.591 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.592 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.592 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.592 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-11-27T15:29:00.592 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.592 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.592 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.592 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.592 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.593 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.593 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.593 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix", 2023-11-27T15:29:00.593 INFO:teuthology.orchestra.run.smithi133.stdout: "can_run": true, 2023-11-27T15:29:00.593 INFO:teuthology.orchestra.run.smithi133.stdout: "error_string": "", 2023-11-27T15:29:00.593 INFO:teuthology.orchestra.run.smithi133.stdout: "module_options": { 2023-11-27T15:29:00.593 INFO:teuthology.orchestra.run.smithi133.stdout: "discovery_interval": { 2023-11-27T15:29:00.593 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "discovery_interval", 2023-11-27T15:29:00.593 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "uint", 2023-11-27T15:29:00.594 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.594 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.594 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "100", 2023-11-27T15:29:00.594 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.594 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.594 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.594 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.594 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.595 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.595 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.595 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.595 INFO:teuthology.orchestra.run.smithi133.stdout: "identifier": { 2023-11-27T15:29:00.595 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "identifier", 2023-11-27T15:29:00.595 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.595 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.595 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.596 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.596 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.596 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.596 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.596 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.596 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.596 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.596 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.597 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.597 INFO:teuthology.orchestra.run.smithi133.stdout: "interval": { 2023-11-27T15:29:00.597 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "interval", 2023-11-27T15:29:00.597 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "secs", 2023-11-27T15:29:00.597 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.597 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.597 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "60", 2023-11-27T15:29:00.597 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.597 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.598 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.598 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.598 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.598 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.598 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.598 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.598 INFO:teuthology.orchestra.run.smithi133.stdout: "log_level": { 2023-11-27T15:29:00.598 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_level", 2023-11-27T15:29:00.598 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.599 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.599 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.599 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.599 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.599 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.599 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.599 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.599 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.600 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.600 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.600 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.600 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.600 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.600 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.600 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.600 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.601 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.601 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.601 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster": { 2023-11-27T15:29:00.601 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster", 2023-11-27T15:29:00.601 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.601 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.601 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.601 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.601 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.602 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.602 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.602 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.602 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.602 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.602 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.602 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.602 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_cluster_level": { 2023-11-27T15:29:00.603 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_cluster_level", 2023-11-27T15:29:00.603 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.603 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.603 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.603 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "info", 2023-11-27T15:29:00.603 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.603 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.603 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [ 2023-11-27T15:29:00.604 INFO:teuthology.orchestra.run.smithi133.stdout: "", 2023-11-27T15:29:00.604 INFO:teuthology.orchestra.run.smithi133.stdout: "critical", 2023-11-27T15:29:00.604 INFO:teuthology.orchestra.run.smithi133.stdout: "debug", 2023-11-27T15:29:00.604 INFO:teuthology.orchestra.run.smithi133.stdout: "error", 2023-11-27T15:29:00.604 INFO:teuthology.orchestra.run.smithi133.stdout: "info", 2023-11-27T15:29:00.604 INFO:teuthology.orchestra.run.smithi133.stdout: "warning" 2023-11-27T15:29:00.604 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.604 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.604 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.605 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.605 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.605 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.605 INFO:teuthology.orchestra.run.smithi133.stdout: "log_to_file": { 2023-11-27T15:29:00.605 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "log_to_file", 2023-11-27T15:29:00.605 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "bool", 2023-11-27T15:29:00.605 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.605 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 1, 2023-11-27T15:29:00.606 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "False", 2023-11-27T15:29:00.606 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.606 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.606 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.606 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.606 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.606 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.606 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.607 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.607 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_host": { 2023-11-27T15:29:00.607 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_host", 2023-11-27T15:29:00.607 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.607 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.607 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.607 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "", 2023-11-27T15:29:00.607 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.607 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.608 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.608 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.608 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.608 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.608 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.608 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.608 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_port": { 2023-11-27T15:29:00.608 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_port", 2023-11-27T15:29:00.609 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "int", 2023-11-27T15:29:00.609 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.609 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.609 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "10051", 2023-11-27T15:29:00.609 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.609 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.609 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.609 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.610 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.610 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.610 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.610 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.610 INFO:teuthology.orchestra.run.smithi133.stdout: "zabbix_sender": { 2023-11-27T15:29:00.610 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "zabbix_sender", 2023-11-27T15:29:00.610 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "str", 2023-11-27T15:29:00.610 INFO:teuthology.orchestra.run.smithi133.stdout: "level": "advanced", 2023-11-27T15:29:00.611 INFO:teuthology.orchestra.run.smithi133.stdout: "flags": 0, 2023-11-27T15:29:00.611 INFO:teuthology.orchestra.run.smithi133.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-11-27T15:29:00.611 INFO:teuthology.orchestra.run.smithi133.stdout: "min": "", 2023-11-27T15:29:00.611 INFO:teuthology.orchestra.run.smithi133.stdout: "max": "", 2023-11-27T15:29:00.611 INFO:teuthology.orchestra.run.smithi133.stdout: "enum_allowed": [], 2023-11-27T15:29:00.611 INFO:teuthology.orchestra.run.smithi133.stdout: "desc": "", 2023-11-27T15:29:00.611 INFO:teuthology.orchestra.run.smithi133.stdout: "long_desc": "", 2023-11-27T15:29:00.611 INFO:teuthology.orchestra.run.smithi133.stdout: "tags": [], 2023-11-27T15:29:00.611 INFO:teuthology.orchestra.run.smithi133.stdout: "see_also": [] 2023-11-27T15:29:00.612 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.612 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.612 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.612 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.612 INFO:teuthology.orchestra.run.smithi133.stdout: "services": { 2023-11-27T15:29:00.612 INFO:teuthology.orchestra.run.smithi133.stdout: "dashboard": "https://172.21.15.133:8443/" 2023-11-27T15:29:00.612 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.612 INFO:teuthology.orchestra.run.smithi133.stdout: "always_on_modules": { 2023-11-27T15:29:00.613 INFO:teuthology.orchestra.run.smithi133.stdout: "octopus": [ 2023-11-27T15:29:00.613 INFO:teuthology.orchestra.run.smithi133.stdout: "balancer", 2023-11-27T15:29:00.613 INFO:teuthology.orchestra.run.smithi133.stdout: "crash", 2023-11-27T15:29:00.613 INFO:teuthology.orchestra.run.smithi133.stdout: "devicehealth", 2023-11-27T15:29:00.613 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator", 2023-11-27T15:29:00.613 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_autoscaler", 2023-11-27T15:29:00.613 INFO:teuthology.orchestra.run.smithi133.stdout: "progress", 2023-11-27T15:29:00.613 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_support", 2023-11-27T15:29:00.613 INFO:teuthology.orchestra.run.smithi133.stdout: "status", 2023-11-27T15:29:00.614 INFO:teuthology.orchestra.run.smithi133.stdout: "telemetry", 2023-11-27T15:29:00.614 INFO:teuthology.orchestra.run.smithi133.stdout: "volumes" 2023-11-27T15:29:00.614 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.614 INFO:teuthology.orchestra.run.smithi133.stdout: "pacific": [ 2023-11-27T15:29:00.614 INFO:teuthology.orchestra.run.smithi133.stdout: "balancer", 2023-11-27T15:29:00.614 INFO:teuthology.orchestra.run.smithi133.stdout: "crash", 2023-11-27T15:29:00.614 INFO:teuthology.orchestra.run.smithi133.stdout: "devicehealth", 2023-11-27T15:29:00.614 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator", 2023-11-27T15:29:00.615 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_autoscaler", 2023-11-27T15:29:00.615 INFO:teuthology.orchestra.run.smithi133.stdout: "progress", 2023-11-27T15:29:00.615 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_support", 2023-11-27T15:29:00.615 INFO:teuthology.orchestra.run.smithi133.stdout: "status", 2023-11-27T15:29:00.615 INFO:teuthology.orchestra.run.smithi133.stdout: "telemetry", 2023-11-27T15:29:00.615 INFO:teuthology.orchestra.run.smithi133.stdout: "volumes" 2023-11-27T15:29:00.615 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.615 INFO:teuthology.orchestra.run.smithi133.stdout: "quincy": [ 2023-11-27T15:29:00.615 INFO:teuthology.orchestra.run.smithi133.stdout: "balancer", 2023-11-27T15:29:00.616 INFO:teuthology.orchestra.run.smithi133.stdout: "crash", 2023-11-27T15:29:00.616 INFO:teuthology.orchestra.run.smithi133.stdout: "devicehealth", 2023-11-27T15:29:00.616 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator", 2023-11-27T15:29:00.616 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_autoscaler", 2023-11-27T15:29:00.616 INFO:teuthology.orchestra.run.smithi133.stdout: "progress", 2023-11-27T15:29:00.616 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_support", 2023-11-27T15:29:00.616 INFO:teuthology.orchestra.run.smithi133.stdout: "status", 2023-11-27T15:29:00.616 INFO:teuthology.orchestra.run.smithi133.stdout: "telemetry", 2023-11-27T15:29:00.617 INFO:teuthology.orchestra.run.smithi133.stdout: "volumes" 2023-11-27T15:29:00.617 INFO:teuthology.orchestra.run.smithi133.stdout: ], 2023-11-27T15:29:00.617 INFO:teuthology.orchestra.run.smithi133.stdout: "reef": [ 2023-11-27T15:29:00.617 INFO:teuthology.orchestra.run.smithi133.stdout: "balancer", 2023-11-27T15:29:00.617 INFO:teuthology.orchestra.run.smithi133.stdout: "crash", 2023-11-27T15:29:00.617 INFO:teuthology.orchestra.run.smithi133.stdout: "devicehealth", 2023-11-27T15:29:00.617 INFO:teuthology.orchestra.run.smithi133.stdout: "orchestrator", 2023-11-27T15:29:00.617 INFO:teuthology.orchestra.run.smithi133.stdout: "pg_autoscaler", 2023-11-27T15:29:00.617 INFO:teuthology.orchestra.run.smithi133.stdout: "progress", 2023-11-27T15:29:00.618 INFO:teuthology.orchestra.run.smithi133.stdout: "rbd_support", 2023-11-27T15:29:00.618 INFO:teuthology.orchestra.run.smithi133.stdout: "status", 2023-11-27T15:29:00.618 INFO:teuthology.orchestra.run.smithi133.stdout: "telemetry", 2023-11-27T15:29:00.618 INFO:teuthology.orchestra.run.smithi133.stdout: "volumes" 2023-11-27T15:29:00.618 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-11-27T15:29:00.618 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.618 INFO:teuthology.orchestra.run.smithi133.stdout: "last_failure_osd_epoch": 3, 2023-11-27T15:29:00.618 INFO:teuthology.orchestra.run.smithi133.stdout: "active_clients": [ 2023-11-27T15:29:00.619 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.619 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "devicehealth", 2023-11-27T15:29:00.619 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2023-11-27T15:29:00.619 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.619 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2023-11-27T15:29:00.619 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2023-11-27T15:29:00.619 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 2764902438 2023-11-27T15:29:00.619 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.619 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-11-27T15:29:00.620 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.620 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.620 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "libcephsqlite", 2023-11-27T15:29:00.620 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2023-11-27T15:29:00.620 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.620 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2023-11-27T15:29:00.620 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2023-11-27T15:29:00.620 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 3716770505 2023-11-27T15:29:00.621 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.621 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-11-27T15:29:00.621 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.621 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.621 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "rbd_support", 2023-11-27T15:29:00.621 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2023-11-27T15:29:00.621 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.621 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2023-11-27T15:29:00.622 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2023-11-27T15:29:00.622 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 22185990 2023-11-27T15:29:00.622 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.622 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-11-27T15:29:00.622 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:00.622 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.622 INFO:teuthology.orchestra.run.smithi133.stdout: "name": "volumes", 2023-11-27T15:29:00.622 INFO:teuthology.orchestra.run.smithi133.stdout: "addrvec": [ 2023-11-27T15:29:00.623 INFO:teuthology.orchestra.run.smithi133.stdout: { 2023-11-27T15:29:00.623 INFO:teuthology.orchestra.run.smithi133.stdout: "type": "v2", 2023-11-27T15:29:00.623 INFO:teuthology.orchestra.run.smithi133.stdout: "addr": "172.21.15.133:0", 2023-11-27T15:29:00.623 INFO:teuthology.orchestra.run.smithi133.stdout: "nonce": 992274730 2023-11-27T15:29:00.623 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.623 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-11-27T15:29:00.623 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:00.623 INFO:teuthology.orchestra.run.smithi133.stdout: ] 2023-11-27T15:29:00.623 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:29:00.625 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:57 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/46596916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:29:00.625 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:57 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1477480169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:29:00.625 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:58 smithi174 ceph-mon[131110]: pgmap v532: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2023-11-27T15:29:00.625 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:28:58 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2544083123' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-11-27T15:29:00.626 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:57 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/46596916' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:29:00.626 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:57 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1477480169' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:29:00.626 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:58 smithi177 ceph-mon[131504]: pgmap v532: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2023-11-27T15:29:00.626 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:28:58 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2544083123' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-11-27T15:29:00.626 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:58.226+0000 7f1b9dfff700 1 -- 172.21.15.133:0/2544083123 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1b800a1150 msgr2=0x7f1b800a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:00.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:58.226+0000 7f1b9dfff700 1 --2- 172.21.15.133:0/2544083123 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1b800a1150 0x7f1b800a3600 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1b981012e0 tx=0x7f1b8800b410 comp rx=0 tx=0).stop 2023-11-27T15:29:00.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:58.226+0000 7f1b9dfff700 1 -- 172.21.15.133:0/2544083123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b98101e10 msgr2=0x7f1b9819f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:00.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:58.226+0000 7f1b9dfff700 1 --2- 172.21.15.133:0/2544083123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b98101e10 0x7f1b9819f840 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f1b9000b070 tx=0x7f1b9000bc80 comp rx=0 tx=0).stop 2023-11-27T15:29:00.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:58.226+0000 7f1b9dfff700 1 -- 172.21.15.133:0/2544083123 shutdown_connections 2023-11-27T15:29:00.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:58.226+0000 7f1b9dfff700 1 --2- 172.21.15.133:0/2544083123 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f1b800a1150 0x7f1b800a3600 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:00.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:58.226+0000 7f1b9dfff700 1 --2- 172.21.15.133:0/2544083123 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1b98101e10 0x7f1b9819f840 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:00.627 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:58.226+0000 7f1b9dfff700 1 --2- 172.21.15.133:0/2544083123 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b98100290 0x7f1b98198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:00.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:58.226+0000 7f1b9dfff700 1 --2- 172.21.15.133:0/2544083123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b98101480 0x7f1b981987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:00.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:58.226+0000 7f1b9dfff700 1 -- 172.21.15.133:0/2544083123 >> 172.21.15.133:0/2544083123 conn(0x7f1b980fb870 msgr2=0x7f1b980fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:00.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:58.226+0000 7f1b9dfff700 1 -- 172.21.15.133:0/2544083123 shutdown_connections 2023-11-27T15:29:00.628 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:28:58.226+0000 7f1b9dfff700 1 -- 172.21.15.133:0/2544083123 wait complete. 2023-11-27T15:29:00.635 DEBUG:tasks.check_counter:Getting stats from a 2023-11-27T15:29:00.635 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.a perf dump 2023-11-27T15:29:00.635 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:58 smithi133 ceph-mon[97860]: pgmap v532: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.2 KiB/s wr, 15 op/s 2023-11-27T15:29:00.635 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:28:58 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2544083123' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-11-27T15:29:01.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:00 smithi133 ceph-mon[97860]: pgmap v533: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2023-11-27T15:29:01.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:00 smithi177 ceph-mon[131504]: pgmap v533: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2023-11-27T15:29:01.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:00 smithi174 ceph-mon[131110]: pgmap v533: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2023-11-27T15:29:01.512 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:29:03.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:02 smithi133 ceph-mon[97860]: pgmap v534: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2023-11-27T15:29:03.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:02 smithi177 ceph-mon[131504]: pgmap v534: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2023-11-27T15:29:03.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:02 smithi174 ceph-mon[131110]: pgmap v534: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 6.1 KiB/s wr, 15 op/s 2023-11-27T15:29:04.502 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:29:04.502 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:29:04.503 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 0, 2023-11-27T15:29:04.503 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 2, 2023-11-27T15:29:04.503 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 0, 2023-11-27T15:29:04.503 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 0, 2023-11-27T15:29:04.503 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2023-11-27T15:29:04.503 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 0, 2023-11-27T15:29:04.503 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.003555450, 2023-11-27T15:29:04.503 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.000000000, 2023-11-27T15:29:04.504 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.000724138, 2023-11-27T15:29:04.504 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:29:04.504 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:04.504 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-11-27T15:29:04.504 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-11-27T15:29:04.504 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-11-27T15:29:04.504 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:04.504 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:04.504 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-11-27T15:29:04.505 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-11-27T15:29:04.505 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-11-27T15:29:04.505 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:04.505 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:29:04.505 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:29:04.505 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:04.505 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:29:04.505 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 1, 2023-11-27T15:29:04.506 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 107, 2023-11-27T15:29:04.506 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 160, 2023-11-27T15:29:04.506 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 94608, 2023-11-27T15:29:04.506 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 3, 2023-11-27T15:29:04.506 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 2, 2023-11-27T15:29:04.506 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.012991896, 2023-11-27T15:29:04.506 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.009449725, 2023-11-27T15:29:04.506 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.001983476, 2023-11-27T15:29:04.507 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:29:04.507 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:04.507 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 105, 2023-11-27T15:29:04.507 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.006924695, 2023-11-27T15:29:04.507 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000065949 2023-11-27T15:29:04.507 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:04.507 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:04.507 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-11-27T15:29:04.507 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-11-27T15:29:04.508 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-11-27T15:29:04.508 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:04.508 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-27T15:29:04.508 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 94608 2023-11-27T15:29:04.508 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:04.508 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:29:04.508 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 149, 2023-11-27T15:29:04.508 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 144, 2023-11-27T15:29:04.509 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 453312, 2023-11-27T15:29:04.509 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 80896, 2023-11-27T15:29:04.509 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 2, 2023-11-27T15:29:04.509 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 1, 2023-11-27T15:29:04.509 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.030892007, 2023-11-27T15:29:04.509 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.013374665, 2023-11-27T15:29:04.509 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.015342427, 2023-11-27T15:29:04.509 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.003875970, 2023-11-27T15:29:04.509 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:04.510 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 144, 2023-11-27T15:29:04.510 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.010337325, 2023-11-27T15:29:04.510 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000071786 2023-11-27T15:29:04.510 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:04.510 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:04.510 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-11-27T15:29:04.510 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-11-27T15:29:04.510 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-11-27T15:29:04.510 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:04.511 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 453312, 2023-11-27T15:29:04.511 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 80896 2023-11-27T15:29:04.511 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:04.511 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2023-11-27T15:29:04.511 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 0, 2023-11-27T15:29:04.511 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2023-11-27T15:29:04.511 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:04.511 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2023-11-27T15:29:04.512 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:29:04.512 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2023-11-27T15:29:04.512 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:29:04.512 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:29:04.512 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:29:04.512 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:29:04.512 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:29:04.512 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:29:04.512 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:29:04.513 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:29:04.513 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:29:04.513 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:29:04.513 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:29:04.513 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:29:04.513 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:29:04.513 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2023-11-27T15:29:04.513 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:29:04.513 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2023-11-27T15:29:04.514 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:29:04.514 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:29:04.514 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:29:04.514 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:29:04.514 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:29:04.514 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:29:04.514 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:29:04.514 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2023-11-27T15:29:04.515 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:29:04.515 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:29:04.515 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:29:04.515 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2023-11-27T15:29:04.515 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2023-11-27T15:29:04.515 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2023-11-27T15:29:04.515 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:29:04.515 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:29:04.515 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:29:04.516 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:29:04.516 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 149007, 2023-11-27T15:29:04.516 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 30, 2023-11-27T15:29:04.516 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:29:04.516 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2023-11-27T15:29:04.516 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2023-11-27T15:29:04.516 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2023-11-27T15:29:04.516 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:29:04.517 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2023-11-27T15:29:04.517 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:29:04.517 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2023-11-27T15:29:04.517 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 0, 2023-11-27T15:29:04.517 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 0, 2023-11-27T15:29:04.517 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:29:04.517 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:29:04.517 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2023-11-27T15:29:04.517 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2023-11-27T15:29:04.518 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 928, 2023-11-27T15:29:04.518 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 2, 2023-11-27T15:29:04.518 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2023-11-27T15:29:04.518 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2023-11-27T15:29:04.518 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2023-11-27T15:29:04.518 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2023-11-27T15:29:04.518 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:04.518 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:29:04.519 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-11-27T15:29:04.519 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2023-11-27T15:29:04.519 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-11-27T15:29:04.519 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 146, 2023-11-27T15:29:04.519 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 429839, 2023-11-27T15:29:04.519 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:04.519 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 146, 2023-11-27T15:29:04.519 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2023-11-27T15:29:04.519 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2023-11-27T15:29:04.520 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 146, 2023-11-27T15:29:04.520 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 429839, 2023-11-27T15:29:04.520 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-11-27T15:29:04.520 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-11-27T15:29:04.520 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-11-27T15:29:04.520 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-11-27T15:29:04.520 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:04.520 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:04.520 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:29:05.112 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-11-27T15:29:05.112 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-11-27T15:29:05.112 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2023-11-27T15:29:05.113 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2023-11-27T15:29:05.113 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2023-11-27T15:29:05.113 DEBUG:tasks.check_counter:Getting stats from d 2023-11-27T15:29:05.113 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.d perf dump 2023-11-27T15:29:05.276 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:04 smithi133 ceph-mon[97860]: pgmap v535: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.1 KiB/s wr, 12 op/s 2023-11-27T15:29:05.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:04 smithi177 ceph-mon[131504]: pgmap v535: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.1 KiB/s wr, 12 op/s 2023-11-27T15:29:05.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:04 smithi174 ceph-mon[131110]: pgmap v535: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 6.1 KiB/s wr, 12 op/s 2023-11-27T15:29:05.964 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:29:07.225 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:06 smithi133 ceph-mon[97860]: pgmap v536: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-11-27T15:29:07.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:06 smithi177 ceph-mon[131504]: pgmap v536: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-11-27T15:29:07.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:06 smithi174 ceph-mon[131110]: pgmap v536: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2023-11-27T15:29:07.502 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:29:07.503 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:29:07.503 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 4, 2023-11-27T15:29:07.503 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 4, 2023-11-27T15:29:07.503 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 2992, 2023-11-27T15:29:07.504 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 448, 2023-11-27T15:29:07.504 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2023-11-27T15:29:07.504 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 0, 2023-11-27T15:29:07.504 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.003611267, 2023-11-27T15:29:07.504 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.000072058, 2023-11-27T15:29:07.504 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.001023648, 2023-11-27T15:29:07.505 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:29:07.505 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:07.505 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 3, 2023-11-27T15:29:07.505 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.001146910, 2023-11-27T15:29:07.505 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000382303 2023-11-27T15:29:07.505 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:07.506 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:07.506 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-11-27T15:29:07.506 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-11-27T15:29:07.506 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-11-27T15:29:07.506 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:07.507 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-11-27T15:29:07.507 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 448 2023-11-27T15:29:07.507 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:07.507 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:29:07.507 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 1, 2023-11-27T15:29:07.507 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 111, 2023-11-27T15:29:07.508 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 160, 2023-11-27T15:29:07.508 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 95312, 2023-11-27T15:29:07.508 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 3, 2023-11-27T15:29:07.508 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 2, 2023-11-27T15:29:07.508 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.013443474, 2023-11-27T15:29:07.509 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.009616395, 2023-11-27T15:29:07.509 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.001540722, 2023-11-27T15:29:07.509 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:29:07.509 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:07.509 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 109, 2023-11-27T15:29:07.509 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.008269355, 2023-11-27T15:29:07.510 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000075865 2023-11-27T15:29:07.510 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:07.510 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:07.510 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-11-27T15:29:07.510 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-11-27T15:29:07.510 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-11-27T15:29:07.511 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:07.511 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-27T15:29:07.511 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 95312 2023-11-27T15:29:07.511 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:07.511 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:29:07.512 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 151, 2023-11-27T15:29:07.512 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 148, 2023-11-27T15:29:07.512 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 452576, 2023-11-27T15:29:07.512 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 83744, 2023-11-27T15:29:07.512 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 2, 2023-11-27T15:29:07.512 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 1, 2023-11-27T15:29:07.513 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.030458962, 2023-11-27T15:29:07.513 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.012637972, 2023-11-27T15:29:07.513 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.015871637, 2023-11-27T15:29:07.513 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.003462870, 2023-11-27T15:29:07.513 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:07.514 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 147, 2023-11-27T15:29:07.514 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.013042883, 2023-11-27T15:29:07.514 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000088727 2023-11-27T15:29:07.514 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:07.514 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:07.514 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-11-27T15:29:07.515 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-11-27T15:29:07.515 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-11-27T15:29:07.515 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:07.515 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 452576, 2023-11-27T15:29:07.515 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 83744 2023-11-27T15:29:07.515 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:07.516 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2023-11-27T15:29:07.516 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 0, 2023-11-27T15:29:07.516 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2023-11-27T15:29:07.516 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:07.516 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2023-11-27T15:29:07.517 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:29:07.517 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2023-11-27T15:29:07.517 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:29:07.517 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:29:07.517 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:29:07.517 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:29:07.518 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:29:07.518 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:29:07.518 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:29:07.518 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:29:07.518 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:29:07.519 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:29:07.519 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:29:07.519 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:29:07.519 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:29:07.519 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2023-11-27T15:29:07.519 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:29:07.520 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2023-11-27T15:29:07.520 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:29:07.520 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:29:07.520 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:29:07.520 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:29:07.520 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:29:07.521 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:29:07.521 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:29:07.521 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2023-11-27T15:29:07.521 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:29:07.521 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:29:07.521 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:29:07.522 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2023-11-27T15:29:07.522 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2023-11-27T15:29:07.522 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2023-11-27T15:29:07.522 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:29:07.522 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:29:07.523 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:29:07.523 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:29:07.523 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 148967, 2023-11-27T15:29:07.523 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 29, 2023-11-27T15:29:07.523 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:29:07.523 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2023-11-27T15:29:07.523 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2023-11-27T15:29:07.524 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2023-11-27T15:29:07.524 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:29:07.524 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2023-11-27T15:29:07.524 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:29:07.524 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2023-11-27T15:29:07.524 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 0, 2023-11-27T15:29:07.525 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 0, 2023-11-27T15:29:07.525 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:29:07.525 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:29:07.525 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2023-11-27T15:29:07.525 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2023-11-27T15:29:07.525 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 928, 2023-11-27T15:29:07.526 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 2, 2023-11-27T15:29:07.526 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2023-11-27T15:29:07.526 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2023-11-27T15:29:07.526 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2023-11-27T15:29:07.526 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2023-11-27T15:29:07.526 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:07.527 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:29:07.527 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-11-27T15:29:07.527 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2023-11-27T15:29:07.527 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-11-27T15:29:07.527 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 152, 2023-11-27T15:29:07.527 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 431256, 2023-11-27T15:29:07.528 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:07.528 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 152, 2023-11-27T15:29:07.528 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2023-11-27T15:29:07.528 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2023-11-27T15:29:07.528 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 152, 2023-11-27T15:29:07.528 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 431256, 2023-11-27T15:29:07.529 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-11-27T15:29:07.529 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-11-27T15:29:07.529 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-11-27T15:29:07.529 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-11-27T15:29:07.529 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:07.529 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:07.530 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:29:08.059 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-11-27T15:29:08.060 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-11-27T15:29:08.060 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-11-27T15:29:08.060 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-11-27T15:29:08.060 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2023-11-27T15:29:08.061 DEBUG:tasks.check_counter:Getting stats from g 2023-11-27T15:29:08.061 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.g perf dump 2023-11-27T15:29:08.909 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:29:09.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:08 smithi177 ceph-mon[131504]: pgmap v537: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2023-11-27T15:29:09.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:08 smithi174 ceph-mon[131110]: pgmap v537: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2023-11-27T15:29:09.209 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:08 smithi133 ceph-mon[97860]: pgmap v537: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2023-11-27T15:29:10.466 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:29:10.467 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:29:10.467 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 147, 2023-11-27T15:29:10.467 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 144, 2023-11-27T15:29:10.467 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 450864, 2023-11-27T15:29:10.467 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 81632, 2023-11-27T15:29:10.468 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2023-11-27T15:29:10.468 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 0, 2023-11-27T15:29:10.468 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.033834539, 2023-11-27T15:29:10.468 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.012804860, 2023-11-27T15:29:10.468 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.019437352, 2023-11-27T15:29:10.468 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.003178151, 2023-11-27T15:29:10.468 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:10.468 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 143, 2023-11-27T15:29:10.469 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.012580377, 2023-11-27T15:29:10.469 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000087974 2023-11-27T15:29:10.469 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:10.469 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:10.469 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-11-27T15:29:10.469 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-11-27T15:29:10.469 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-11-27T15:29:10.469 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:10.470 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 450864, 2023-11-27T15:29:10.470 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 81632 2023-11-27T15:29:10.470 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:10.470 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:29:10.470 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 5, 2023-11-27T15:29:10.470 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 111, 2023-11-27T15:29:10.470 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 3152, 2023-11-27T15:29:10.470 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 95408, 2023-11-27T15:29:10.471 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 3, 2023-11-27T15:29:10.471 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 2, 2023-11-27T15:29:10.471 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.013332069, 2023-11-27T15:29:10.471 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.009277810, 2023-11-27T15:29:10.471 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.002001089, 2023-11-27T15:29:10.471 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:29:10.471 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:10.472 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 110, 2023-11-27T15:29:10.472 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.011379281, 2023-11-27T15:29:10.472 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000103448 2023-11-27T15:29:10.472 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:10.472 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:10.472 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-11-27T15:29:10.472 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-11-27T15:29:10.472 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-11-27T15:29:10.473 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:10.473 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-11-27T15:29:10.473 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 95408 2023-11-27T15:29:10.473 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:10.473 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:29:10.473 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 0, 2023-11-27T15:29:10.473 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 2, 2023-11-27T15:29:10.473 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 0, 2023-11-27T15:29:10.474 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 0, 2023-11-27T15:29:10.474 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 2, 2023-11-27T15:29:10.474 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 1, 2023-11-27T15:29:10.474 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.000827311, 2023-11-27T15:29:10.474 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.000000000, 2023-11-27T15:29:10.474 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.000694597, 2023-11-27T15:29:10.474 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:29:10.475 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:10.475 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-11-27T15:29:10.475 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-11-27T15:29:10.475 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-11-27T15:29:10.475 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:10.475 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:10.475 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-11-27T15:29:10.475 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-11-27T15:29:10.476 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-11-27T15:29:10.476 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:10.476 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:29:10.476 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:29:10.476 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:10.476 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2023-11-27T15:29:10.477 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 0, 2023-11-27T15:29:10.477 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2023-11-27T15:29:10.477 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:10.477 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2023-11-27T15:29:10.477 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:29:10.477 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2023-11-27T15:29:10.477 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:29:10.477 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:29:10.478 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:29:10.478 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:29:10.478 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:29:10.478 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:29:10.479 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:29:10.479 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:29:10.479 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:29:10.479 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:29:10.479 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:29:10.479 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:29:10.479 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:29:10.479 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2023-11-27T15:29:10.480 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:29:10.480 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2023-11-27T15:29:10.480 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:29:10.480 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:29:10.480 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:29:10.480 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:29:10.480 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:29:10.481 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:29:10.481 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:29:10.481 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2023-11-27T15:29:10.481 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:29:10.481 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:29:10.481 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:29:10.481 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2023-11-27T15:29:10.481 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2023-11-27T15:29:10.482 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2023-11-27T15:29:10.482 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:29:10.482 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:29:10.482 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:29:10.482 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:29:10.482 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 148967, 2023-11-27T15:29:10.482 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 29, 2023-11-27T15:29:10.483 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:29:10.483 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2023-11-27T15:29:10.483 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2023-11-27T15:29:10.483 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2023-11-27T15:29:10.483 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:29:10.483 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2023-11-27T15:29:10.483 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:29:10.483 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2023-11-27T15:29:10.484 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 0, 2023-11-27T15:29:10.484 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 0, 2023-11-27T15:29:10.484 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:29:10.484 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:29:10.484 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2023-11-27T15:29:10.484 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2023-11-27T15:29:10.484 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 928, 2023-11-27T15:29:10.484 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 2, 2023-11-27T15:29:10.485 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2023-11-27T15:29:10.485 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2023-11-27T15:29:10.485 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2023-11-27T15:29:10.485 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2023-11-27T15:29:10.485 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:10.485 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:29:10.485 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-11-27T15:29:10.485 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2023-11-27T15:29:10.486 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-11-27T15:29:10.486 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 148, 2023-11-27T15:29:10.486 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 430099, 2023-11-27T15:29:10.486 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:10.486 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 148, 2023-11-27T15:29:10.486 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2023-11-27T15:29:10.486 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2023-11-27T15:29:10.486 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 148, 2023-11-27T15:29:10.487 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 430099, 2023-11-27T15:29:10.487 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-11-27T15:29:10.487 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-11-27T15:29:10.487 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-11-27T15:29:10.487 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-11-27T15:29:10.487 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:10.487 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:10.487 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:29:10.903 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-11-27T15:29:10.903 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-11-27T15:29:10.903 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2023-11-27T15:29:10.903 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2023-11-27T15:29:10.903 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2023-11-27T15:29:10.904 DEBUG:tasks.check_counter:Getting stats from j 2023-11-27T15:29:10.904 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.j perf dump 2023-11-27T15:29:11.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:10 smithi133 ceph-mon[97860]: pgmap v538: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 8 op/s 2023-11-27T15:29:11.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:10 smithi177 ceph-mon[131504]: pgmap v538: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 8 op/s 2023-11-27T15:29:11.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:10 smithi174 ceph-mon[131110]: pgmap v538: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 8 op/s 2023-11-27T15:29:11.756 INFO:teuthology.orchestra.run.smithi133.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.a/config 2023-11-27T15:29:13.017 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:12 smithi133 ceph-mon[97860]: pgmap v539: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 9 op/s 2023-11-27T15:29:13.323 INFO:teuthology.orchestra.run.smithi133.stdout:{ 2023-11-27T15:29:13.323 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:29:13.323 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 0, 2023-11-27T15:29:13.323 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 2, 2023-11-27T15:29:13.323 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 0, 2023-11-27T15:29:13.324 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 0, 2023-11-27T15:29:13.324 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 4, 2023-11-27T15:29:13.324 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 0, 2023-11-27T15:29:13.324 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.003144463, 2023-11-27T15:29:13.324 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.000000000, 2023-11-27T15:29:13.324 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.000715329, 2023-11-27T15:29:13.324 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:29:13.325 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:13.325 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-11-27T15:29:13.325 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-11-27T15:29:13.325 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-11-27T15:29:13.325 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:13.325 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:13.325 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-11-27T15:29:13.325 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-11-27T15:29:13.326 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-11-27T15:29:13.326 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:13.326 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:29:13.326 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:29:13.326 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:13.326 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:29:13.326 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 153, 2023-11-27T15:29:13.327 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 150, 2023-11-27T15:29:13.327 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 453120, 2023-11-27T15:29:13.327 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 84928, 2023-11-27T15:29:13.327 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 2, 2023-11-27T15:29:13.327 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 1, 2023-11-27T15:29:13.327 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.025917572, 2023-11-27T15:29:13.327 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.008318197, 2023-11-27T15:29:13.327 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.015621305, 2023-11-27T15:29:13.328 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.003469207, 2023-11-27T15:29:13.328 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:13.328 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 149, 2023-11-27T15:29:13.328 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.011577399, 2023-11-27T15:29:13.328 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000077700 2023-11-27T15:29:13.328 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:13.328 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:13.329 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-11-27T15:29:13.329 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-11-27T15:29:13.329 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-11-27T15:29:13.329 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:13.329 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 453120, 2023-11-27T15:29:13.329 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 84928 2023-11-27T15:29:13.329 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:13.330 INFO:teuthology.orchestra.run.smithi133.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:29:13.330 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_messages": 5, 2023-11-27T15:29:13.330 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages": 115, 2023-11-27T15:29:13.330 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_bytes": 3152, 2023-11-27T15:29:13.330 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_bytes": 96112, 2023-11-27T15:29:13.330 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_created_connections": 3, 2023-11-27T15:29:13.330 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_active_connections": 2, 2023-11-27T15:29:13.331 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_total_time": 0.014369873, 2023-11-27T15:29:13.331 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_send_time": 0.010492840, 2023-11-27T15:29:13.331 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_recv_time": 0.002350317, 2023-11-27T15:29:13.331 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:29:13.331 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:13.331 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 114, 2023-11-27T15:29:13.331 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.009259870, 2023-11-27T15:29:13.331 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000081226 2023-11-27T15:29:13.332 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:13.332 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:13.332 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-11-27T15:29:13.332 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-11-27T15:29:13.332 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-11-27T15:29:13.332 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:13.332 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-11-27T15:29:13.333 INFO:teuthology.orchestra.run.smithi133.stdout: "msgr_send_encrypted_bytes": 96112 2023-11-27T15:29:13.333 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:13.333 INFO:teuthology.orchestra.run.smithi133.stdout: "cct": { 2023-11-27T15:29:13.333 INFO:teuthology.orchestra.run.smithi133.stdout: "total_workers": 0, 2023-11-27T15:29:13.333 INFO:teuthology.orchestra.run.smithi133.stdout: "unhealthy_workers": 0 2023-11-27T15:29:13.333 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:13.333 INFO:teuthology.orchestra.run.smithi133.stdout: "mempool": { 2023-11-27T15:29:13.333 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:29:13.334 INFO:teuthology.orchestra.run.smithi133.stdout: "bloom_filter_items": 0, 2023-11-27T15:29:13.334 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:29:13.334 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:29:13.334 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:29:13.334 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:29:13.334 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:29:13.334 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:29:13.335 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:29:13.335 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:29:13.335 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:29:13.335 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:29:13.335 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:29:13.335 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:29:13.335 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:29:13.336 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_extent_items": 0, 2023-11-27T15:29:13.336 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:29:13.336 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_blob_items": 0, 2023-11-27T15:29:13.336 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:29:13.336 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:29:13.336 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:29:13.336 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:29:13.336 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:29:13.337 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:29:13.337 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:29:13.337 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_txc_items": 0, 2023-11-27T15:29:13.337 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:29:13.337 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:29:13.337 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:29:13.337 INFO:teuthology.orchestra.run.smithi133.stdout: "bluestore_writing_items": 0, 2023-11-27T15:29:13.338 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_bytes": 0, 2023-11-27T15:29:13.338 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_items": 0, 2023-11-27T15:29:13.338 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:29:13.338 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:29:13.338 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:29:13.338 INFO:teuthology.orchestra.run.smithi133.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:29:13.338 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_bytes": 148967, 2023-11-27T15:29:13.339 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_anon_items": 29, 2023-11-27T15:29:13.339 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:29:13.339 INFO:teuthology.orchestra.run.smithi133.stdout: "buffer_meta_items": 0, 2023-11-27T15:29:13.339 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_bytes": 0, 2023-11-27T15:29:13.339 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_items": 0, 2023-11-27T15:29:13.339 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:29:13.339 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_mapbl_items": 0, 2023-11-27T15:29:13.340 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:29:13.340 INFO:teuthology.orchestra.run.smithi133.stdout: "osd_pglog_items": 0, 2023-11-27T15:29:13.340 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_bytes": 0, 2023-11-27T15:29:13.340 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_items": 0, 2023-11-27T15:29:13.340 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:29:13.340 INFO:teuthology.orchestra.run.smithi133.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:29:13.340 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_bytes": 0, 2023-11-27T15:29:13.340 INFO:teuthology.orchestra.run.smithi133.stdout: "pgmap_items": 0, 2023-11-27T15:29:13.341 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_bytes": 928, 2023-11-27T15:29:13.341 INFO:teuthology.orchestra.run.smithi133.stdout: "mds_co_items": 2, 2023-11-27T15:29:13.341 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_bytes": 0, 2023-11-27T15:29:13.341 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_1_items": 0, 2023-11-27T15:29:13.341 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_bytes": 0, 2023-11-27T15:29:13.341 INFO:teuthology.orchestra.run.smithi133.stdout: "unittest_2_items": 0 2023-11-27T15:29:13.341 INFO:teuthology.orchestra.run.smithi133.stdout: }, 2023-11-27T15:29:13.341 INFO:teuthology.orchestra.run.smithi133.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:29:13.342 INFO:teuthology.orchestra.run.smithi133.stdout: "val": 0, 2023-11-27T15:29:13.342 INFO:teuthology.orchestra.run.smithi133.stdout: "max": 104857600, 2023-11-27T15:29:13.342 INFO:teuthology.orchestra.run.smithi133.stdout: "get_started": 0, 2023-11-27T15:29:13.342 INFO:teuthology.orchestra.run.smithi133.stdout: "get": 154, 2023-11-27T15:29:13.342 INFO:teuthology.orchestra.run.smithi133.stdout: "get_sum": 431516, 2023-11-27T15:29:13.342 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:13.342 INFO:teuthology.orchestra.run.smithi133.stdout: "get_or_fail_success": 154, 2023-11-27T15:29:13.343 INFO:teuthology.orchestra.run.smithi133.stdout: "take": 0, 2023-11-27T15:29:13.343 INFO:teuthology.orchestra.run.smithi133.stdout: "take_sum": 0, 2023-11-27T15:29:13.343 INFO:teuthology.orchestra.run.smithi133.stdout: "put": 154, 2023-11-27T15:29:13.343 INFO:teuthology.orchestra.run.smithi133.stdout: "put_sum": 431516, 2023-11-27T15:29:13.343 INFO:teuthology.orchestra.run.smithi133.stdout: "wait": { 2023-11-27T15:29:13.343 INFO:teuthology.orchestra.run.smithi133.stdout: "avgcount": 0, 2023-11-27T15:29:13.343 INFO:teuthology.orchestra.run.smithi133.stdout: "sum": 0.000000000, 2023-11-27T15:29:13.343 INFO:teuthology.orchestra.run.smithi133.stdout: "avgtime": 0.000000000 2023-11-27T15:29:13.344 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:13.344 INFO:teuthology.orchestra.run.smithi133.stdout: } 2023-11-27T15:29:13.344 INFO:teuthology.orchestra.run.smithi133.stdout:} 2023-11-27T15:29:13.344 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:12 smithi177 ceph-mon[131504]: pgmap v539: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 9 op/s 2023-11-27T15:29:13.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:12 smithi174 ceph-mon[131110]: pgmap v539: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 9 op/s 2023-11-27T15:29:13.856 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-11-27T15:29:13.856 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-11-27T15:29:13.856 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-11-27T15:29:13.857 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-11-27T15:29:13.857 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2023-11-27T15:29:13.857 DEBUG:tasks.check_counter:Getting stats from b 2023-11-27T15:29:13.857 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.b perf dump 2023-11-27T15:29:14.521 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:29:15.097 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:14 smithi174 ceph-mon[131110]: pgmap v540: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-11-27T15:29:15.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:14 smithi133 ceph-mon[97860]: pgmap v540: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-11-27T15:29:15.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:14 smithi177 ceph-mon[131504]: pgmap v540: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2023-11-27T15:29:15.775 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2023-11-27T15:29:15.775 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:29:15.776 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 38669, 2023-11-27T15:29:15.776 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 38208, 2023-11-27T15:29:15.776 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 48338136, 2023-11-27T15:29:15.776 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 112264748, 2023-11-27T15:29:15.776 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 112, 2023-11-27T15:29:15.777 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 3, 2023-11-27T15:29:15.777 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 2.454301610, 2023-11-27T15:29:15.777 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.818353114, 2023-11-27T15:29:15.777 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.708638205, 2023-11-27T15:29:15.778 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.946352694, 2023-11-27T15:29:15.778 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:15.778 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 38208, 2023-11-27T15:29:15.778 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 54.796573843, 2023-11-27T15:29:15.779 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001434164 2023-11-27T15:29:15.779 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.779 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:15.779 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 18882, 2023-11-27T15:29:15.780 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.003426705, 2023-11-27T15:29:15.780 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000181 2023-11-27T15:29:15.780 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.780 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:29:15.780 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:29:15.781 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.781 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:29:15.781 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 105735, 2023-11-27T15:29:15.781 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 180946, 2023-11-27T15:29:15.782 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 42925892, 2023-11-27T15:29:15.782 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 157593540, 2023-11-27T15:29:15.782 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 11, 2023-11-27T15:29:15.782 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 6, 2023-11-27T15:29:15.782 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 6.932625053, 2023-11-27T15:29:15.783 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 2.905972809, 2023-11-27T15:29:15.783 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 7.193366763, 2023-11-27T15:29:15.783 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.952109490, 2023-11-27T15:29:15.783 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:15.783 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 180946, 2023-11-27T15:29:15.784 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 865.907651338, 2023-11-27T15:29:15.784 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.004785447 2023-11-27T15:29:15.784 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.784 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:15.785 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 182325, 2023-11-27T15:29:15.785 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.031494011, 2023-11-27T15:29:15.785 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000172 2023-11-27T15:29:15.785 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.785 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 30374144, 2023-11-27T15:29:15.786 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 88033984 2023-11-27T15:29:15.786 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.786 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:29:15.786 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 167408, 2023-11-27T15:29:15.787 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 168248, 2023-11-27T15:29:15.787 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 36243015, 2023-11-27T15:29:15.787 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 95166295, 2023-11-27T15:29:15.787 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 14, 2023-11-27T15:29:15.787 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 7, 2023-11-27T15:29:15.788 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 6.853003397, 2023-11-27T15:29:15.788 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 2.502237429, 2023-11-27T15:29:15.788 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 3.798336349, 2023-11-27T15:29:15.788 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 1.026981982, 2023-11-27T15:29:15.789 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:15.789 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 168244, 2023-11-27T15:29:15.789 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 63.212137613, 2023-11-27T15:29:15.789 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000375717 2023-11-27T15:29:15.789 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.790 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:15.790 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 266157, 2023-11-27T15:29:15.790 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.032122085, 2023-11-27T15:29:15.790 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000120 2023-11-27T15:29:15.791 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.791 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-27T15:29:15.791 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 269808 2023-11-27T15:29:15.791 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.791 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2023-11-27T15:29:15.792 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2023-11-27T15:29:15.792 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2023-11-27T15:29:15.792 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.792 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2023-11-27T15:29:15.792 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2023-11-27T15:29:15.793 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2023-11-27T15:29:15.793 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 11943, 2023-11-27T15:29:15.793 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 63.087321411, 2023-11-27T15:29:15.793 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.005282368 2023-11-27T15:29:15.794 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:15.794 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.794 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2023-11-27T15:29:15.794 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2023-11-27T15:29:15.794 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2023-11-27T15:29:15.795 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 21350, 2023-11-27T15:29:15.795 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 7.207637255, 2023-11-27T15:29:15.795 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000337594 2023-11-27T15:29:15.795 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:15.795 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.796 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2023-11-27T15:29:15.796 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 57521, 2023-11-27T15:29:15.796 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 46499, 2023-11-27T15:29:15.796 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2023-11-27T15:29:15.796 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 46499, 2023-11-27T15:29:15.797 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1012.778957271, 2023-11-27T15:29:15.797 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.021780661 2023-11-27T15:29:15.797 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.797 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2023-11-27T15:29:15.798 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 11022, 2023-11-27T15:29:15.798 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 372, 2023-11-27T15:29:15.798 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 3, 2023-11-27T15:29:15.798 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 6643, 2023-11-27T15:29:15.799 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 130, 2023-11-27T15:29:15.799 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 46, 2023-11-27T15:29:15.799 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 27, 2023-11-27T15:29:15.799 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 4, 2023-11-27T15:29:15.799 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 8, 2023-11-27T15:29:15.800 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 15, 2023-11-27T15:29:15.800 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 20, 2023-11-27T15:29:15.800 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 29635, 2023-11-27T15:29:15.800 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 5, 2023-11-27T15:29:15.801 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 5, 2023-11-27T15:29:15.801 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 4, 2023-11-27T15:29:15.801 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 70187, 2023-11-27T15:29:15.801 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 58490, 2023-11-27T15:29:15.801 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 9878, 2023-11-27T15:29:15.802 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 0, 2023-11-27T15:29:15.802 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 252, 2023-11-27T15:29:15.802 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2023-11-27T15:29:15.802 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2023-11-27T15:29:15.802 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 348, 2023-11-27T15:29:15.803 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2023-11-27T15:29:15.803 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 397, 2023-11-27T15:29:15.803 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 23934, 2023-11-27T15:29:15.803 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 452, 2023-11-27T15:29:15.803 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2023-11-27T15:29:15.804 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2023-11-27T15:29:15.804 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 12185, 2023-11-27T15:29:15.804 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2023-11-27T15:29:15.804 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 5, 2023-11-27T15:29:15.804 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 246, 2023-11-27T15:29:15.805 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2023-11-27T15:29:15.805 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 12185, 2023-11-27T15:29:15.805 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2023-11-27T15:29:15.805 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2023-11-27T15:29:15.805 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2023-11-27T15:29:15.806 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 58, 2023-11-27T15:29:15.806 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 6, 2023-11-27T15:29:15.806 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 52, 2023-11-27T15:29:15.806 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 12127, 2023-11-27T15:29:15.806 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 4, 2023-11-27T15:29:15.807 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 105, 2023-11-27T15:29:15.807 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 33338, 2023-11-27T15:29:15.807 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 1, 2023-11-27T15:29:15.807 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-27T15:29:15.807 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-27T15:29:15.807 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 16997, 2023-11-27T15:29:15.808 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 16855, 2023-11-27T15:29:15.808 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 299, 2023-11-27T15:29:15.808 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 25249 2023-11-27T15:29:15.808 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.809 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2023-11-27T15:29:15.809 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 1471, 2023-11-27T15:29:15.809 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2023-11-27T15:29:15.809 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2023-11-27T15:29:15.809 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 0, 2023-11-27T15:29:15.809 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 1863, 2023-11-27T15:29:15.810 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2023-11-27T15:29:15.810 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2023-11-27T15:29:15.810 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2023-11-27T15:29:15.810 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 12273, 2023-11-27T15:29:15.810 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 12273, 2023-11-27T15:29:15.811 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2023-11-27T15:29:15.811 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2023-11-27T15:29:15.811 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2023-11-27T15:29:15.811 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2023-11-27T15:29:15.811 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2023-11-27T15:29:15.812 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2023-11-27T15:29:15.812 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2023-11-27T15:29:15.812 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 6, 2023-11-27T15:29:15.812 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 419, 2023-11-27T15:29:15.812 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2023-11-27T15:29:15.812 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 189, 2023-11-27T15:29:15.813 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2023-11-27T15:29:15.813 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2023-11-27T15:29:15.813 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.813 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2023-11-27T15:29:15.813 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 67006, 2023-11-27T15:29:15.814 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 66994, 2023-11-27T15:29:15.814 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 66994, 2023-11-27T15:29:15.814 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 12, 2023-11-27T15:29:15.814 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2023-11-27T15:29:15.814 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 41, 2023-11-27T15:29:15.815 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 107, 2023-11-27T15:29:15.815 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 106, 2023-11-27T15:29:15.815 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 106, 2023-11-27T15:29:15.815 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 2, 2023-11-27T15:29:15.815 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2023-11-27T15:29:15.816 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 1, 2023-11-27T15:29:15.816 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 173644428, 2023-11-27T15:29:15.816 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 173666818, 2023-11-27T15:29:15.816 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2023-11-27T15:29:15.816 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2023-11-27T15:29:15.816 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1856, 2023-11-27T15:29:15.817 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 105.093241246, 2023-11-27T15:29:15.817 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.056623513 2023-11-27T15:29:15.817 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.817 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2023-11-27T15:29:15.817 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.818 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2023-11-27T15:29:15.818 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 30, 2023-11-27T15:29:15.818 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 29908, 2023-11-27T15:29:15.818 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 29878, 2023-11-27T15:29:15.818 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 25, 2023-11-27T15:29:15.819 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 3238, 2023-11-27T15:29:15.819 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 3213, 2023-11-27T15:29:15.819 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 27, 2023-11-27T15:29:15.819 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 46889, 2023-11-27T15:29:15.819 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 46862, 2023-11-27T15:29:15.820 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 5, 2023-11-27T15:29:15.820 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 43517, 2023-11-27T15:29:15.820 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 43512, 2023-11-27T15:29:15.820 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 158972, 2023-11-27T15:29:15.820 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 207156 2023-11-27T15:29:15.820 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.821 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2023-11-27T15:29:15.821 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 58244, 2023-11-27T15:29:15.821 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 90, 2023-11-27T15:29:15.821 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 57521, 2023-11-27T15:29:15.821 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 151, 2023-11-27T15:29:15.821 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 217, 2023-11-27T15:29:15.822 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2023-11-27T15:29:15.822 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 21061, 2023-11-27T15:29:15.822 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 935.862078265, 2023-11-27T15:29:15.822 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.044435785 2023-11-27T15:29:15.822 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.823 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2023-11-27T15:29:15.823 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 50, 2023-11-27T15:29:15.823 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.014377584, 2023-11-27T15:29:15.823 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000287551 2023-11-27T15:29:15.823 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.823 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2023-11-27T15:29:15.824 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.824 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.824 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.824 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.824 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2023-11-27T15:29:15.824 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.825 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.825 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.825 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.825 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2023-11-27T15:29:15.825 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 12142, 2023-11-27T15:29:15.826 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 22.391600724, 2023-11-27T15:29:15.826 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001844144 2023-11-27T15:29:15.826 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.826 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2023-11-27T15:29:15.826 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.826 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.827 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.827 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.827 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2023-11-27T15:29:15.827 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1, 2023-11-27T15:29:15.827 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000338377, 2023-11-27T15:29:15.828 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000338377 2023-11-27T15:29:15.828 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.828 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2023-11-27T15:29:15.828 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.828 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.829 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.829 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.829 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2023-11-27T15:29:15.829 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.829 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.829 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.830 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.830 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2023-11-27T15:29:15.830 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.830 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.830 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.830 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.831 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2023-11-27T15:29:15.831 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.831 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.831 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.831 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.832 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2023-11-27T15:29:15.832 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 125, 2023-11-27T15:29:15.832 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.132404315, 2023-11-27T15:29:15.832 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001059234 2023-11-27T15:29:15.832 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.832 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2023-11-27T15:29:15.833 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.833 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.833 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.833 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.833 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2023-11-27T15:29:15.833 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.834 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.834 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.834 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.834 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2023-11-27T15:29:15.834 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.834 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.835 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.835 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.835 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2023-11-27T15:29:15.835 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 800, 2023-11-27T15:29:15.835 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 2.127093621, 2023-11-27T15:29:15.835 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.002658867 2023-11-27T15:29:15.836 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.836 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2023-11-27T15:29:15.836 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2, 2023-11-27T15:29:15.836 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.003170056, 2023-11-27T15:29:15.836 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001585028 2023-11-27T15:29:15.836 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.837 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2023-11-27T15:29:15.837 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.837 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.837 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.837 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.838 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2023-11-27T15:29:15.838 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.838 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.838 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.838 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.838 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2023-11-27T15:29:15.839 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 112, 2023-11-27T15:29:15.839 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.312270067, 2023-11-27T15:29:15.839 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.002788125 2023-11-27T15:29:15.839 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.839 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2023-11-27T15:29:15.840 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.840 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.840 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.840 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.840 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2023-11-27T15:29:15.840 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3, 2023-11-27T15:29:15.841 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.001957536, 2023-11-27T15:29:15.841 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000652512 2023-11-27T15:29:15.841 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.841 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2023-11-27T15:29:15.841 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4, 2023-11-27T15:29:15.841 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.013492247, 2023-11-27T15:29:15.842 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003373061 2023-11-27T15:29:15.842 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.842 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2023-11-27T15:29:15.842 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.842 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.842 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.843 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.843 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2023-11-27T15:29:15.843 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 22, 2023-11-27T15:29:15.843 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 10.016746202, 2023-11-27T15:29:15.843 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.455306645 2023-11-27T15:29:15.843 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.844 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2023-11-27T15:29:15.844 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.844 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.844 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.844 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.845 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2023-11-27T15:29:15.845 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 16, 2023-11-27T15:29:15.845 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.030574391, 2023-11-27T15:29:15.845 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001910899 2023-11-27T15:29:15.845 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.845 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2023-11-27T15:29:15.846 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.846 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.846 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.846 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.846 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2023-11-27T15:29:15.846 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 12161, 2023-11-27T15:29:15.847 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 41.872853886, 2023-11-27T15:29:15.847 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003443208 2023-11-27T15:29:15.847 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.847 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2023-11-27T15:29:15.847 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2023-11-27T15:29:15.847 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2023-11-27T15:29:15.848 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.848 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.848 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.848 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:15.848 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.849 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2023-11-27T15:29:15.849 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2023-11-27T15:29:15.849 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 3, 2023-11-27T15:29:15.849 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 2, 2023-11-27T15:29:15.849 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 1, 2023-11-27T15:29:15.849 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2023-11-27T15:29:15.850 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 11057, 2023-11-27T15:29:15.850 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 11057, 2023-11-27T15:29:15.850 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 391 2023-11-27T15:29:15.850 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.850 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2023-11-27T15:29:15.851 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:29:15.851 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2023-11-27T15:29:15.851 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:29:15.851 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:29:15.851 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:29:15.851 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:29:15.852 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:29:15.852 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:29:15.852 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:29:15.852 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:29:15.852 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:29:15.852 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:29:15.853 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:29:15.853 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:29:15.853 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:29:15.853 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2023-11-27T15:29:15.853 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:29:15.853 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2023-11-27T15:29:15.854 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:29:15.854 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:29:15.854 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:29:15.854 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:29:15.854 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:29:15.854 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:29:15.855 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:29:15.855 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2023-11-27T15:29:15.855 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:29:15.855 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:29:15.855 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:29:15.855 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2023-11-27T15:29:15.856 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2023-11-27T15:29:15.856 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2023-11-27T15:29:15.856 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:29:15.856 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:29:15.856 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:29:15.856 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:29:15.857 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 291693, 2023-11-27T15:29:15.857 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 120, 2023-11-27T15:29:15.857 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 616, 2023-11-27T15:29:15.857 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 7, 2023-11-27T15:29:15.857 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2023-11-27T15:29:15.857 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2023-11-27T15:29:15.858 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:29:15.858 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2023-11-27T15:29:15.858 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:29:15.858 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2023-11-27T15:29:15.858 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8888, 2023-11-27T15:29:15.858 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 170, 2023-11-27T15:29:15.859 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:29:15.859 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:29:15.859 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2023-11-27T15:29:15.859 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2023-11-27T15:29:15.859 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 122853, 2023-11-27T15:29:15.859 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 844, 2023-11-27T15:29:15.860 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2023-11-27T15:29:15.860 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2023-11-27T15:29:15.860 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2023-11-27T15:29:15.860 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2023-11-27T15:29:15.860 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.860 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2023-11-27T15:29:15.861 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2023-11-27T15:29:15.861 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2023-11-27T15:29:15.861 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 92326, 2023-11-27T15:29:15.861 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 221599178, 2023-11-27T15:29:15.861 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 37, 2023-11-27T15:29:15.861 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 92326, 2023-11-27T15:29:15.862 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2023-11-27T15:29:15.862 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 92326, 2023-11-27T15:29:15.862 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 42362.105773194, 2023-11-27T15:29:15.862 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.458831810 2023-11-27T15:29:15.862 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.862 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2023-11-27T15:29:15.863 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2023-11-27T15:29:15.863 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 92326, 2023-11-27T15:29:15.863 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 170458 2023-11-27T15:29:15.863 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.863 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 92326, 2023-11-27T15:29:15.863 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 12808, 2023-11-27T15:29:15.863 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 79518, 2023-11-27T15:29:15.864 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2023-11-27T15:29:15.864 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2023-11-27T15:29:15.864 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 5522, 2023-11-27T15:29:15.864 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 44519, 2023-11-27T15:29:15.864 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 242, 2023-11-27T15:29:15.864 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 2092, 2023-11-27T15:29:15.865 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 200, 2023-11-27T15:29:15.865 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2023-11-27T15:29:15.865 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2023-11-27T15:29:15.865 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2023-11-27T15:29:15.865 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 5, 2023-11-27T15:29:15.865 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 25945, 2023-11-27T15:29:15.866 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2023-11-27T15:29:15.866 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2023-11-27T15:29:15.866 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2023-11-27T15:29:15.866 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 12203, 2023-11-27T15:29:15.866 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 67183, 2023-11-27T15:29:15.866 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2023-11-27T15:29:15.866 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2023-11-27T15:29:15.867 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2023-11-27T15:29:15.867 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2023-11-27T15:29:15.867 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2023-11-27T15:29:15.867 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2023-11-27T15:29:15.867 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2023-11-27T15:29:15.867 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2023-11-27T15:29:15.868 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2023-11-27T15:29:15.868 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 1, 2023-11-27T15:29:15.868 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2023-11-27T15:29:15.868 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2023-11-27T15:29:15.868 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2023-11-27T15:29:15.868 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2023-11-27T15:29:15.869 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2023-11-27T15:29:15.869 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2023-11-27T15:29:15.869 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2023-11-27T15:29:15.869 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2023-11-27T15:29:15.869 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2023-11-27T15:29:15.869 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2023-11-27T15:29:15.870 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2023-11-27T15:29:15.870 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2023-11-27T15:29:15.870 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2023-11-27T15:29:15.870 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2023-11-27T15:29:15.870 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2023-11-27T15:29:15.870 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2023-11-27T15:29:15.871 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 94, 2023-11-27T15:29:15.871 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2023-11-27T15:29:15.871 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 11, 2023-11-27T15:29:15.871 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 12, 2023-11-27T15:29:15.871 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 12, 2023-11-27T15:29:15.871 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2023-11-27T15:29:15.872 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2023-11-27T15:29:15.872 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 11573, 2023-11-27T15:29:15.872 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 750, 2023-11-27T15:29:15.872 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 223 2023-11-27T15:29:15.872 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.872 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2023-11-27T15:29:15.872 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2023-11-27T15:29:15.873 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2023-11-27T15:29:15.873 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2023-11-27T15:29:15.873 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2023-11-27T15:29:15.873 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.873 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2023-11-27T15:29:15.873 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 58235 2023-11-27T15:29:15.874 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.874 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2023-11-27T15:29:15.874 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2023-11-27T15:29:15.874 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 16, 2023-11-27T15:29:15.874 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2023-11-27T15:29:15.874 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 12, 2023-11-27T15:29:15.875 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 12273, 2023-11-27T15:29:15.875 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2023-11-27T15:29:15.875 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.875 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:29:15.875 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2023-11-27T15:29:15.875 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2023-11-27T15:29:15.875 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2023-11-27T15:29:15.876 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 311808, 2023-11-27T15:29:15.876 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 95764015, 2023-11-27T15:29:15.876 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:15.876 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 311808, 2023-11-27T15:29:15.876 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2023-11-27T15:29:15.876 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2023-11-27T15:29:15.877 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 311808, 2023-11-27T15:29:15.877 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 95764015, 2023-11-27T15:29:15.877 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2023-11-27T15:29:15.877 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.877 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.877 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.877 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:15.878 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.878 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2023-11-27T15:29:15.878 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2023-11-27T15:29:15.878 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2023-11-27T15:29:15.878 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2023-11-27T15:29:15.878 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2023-11-27T15:29:15.879 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2023-11-27T15:29:15.879 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:15.879 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2023-11-27T15:29:15.879 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 92326, 2023-11-27T15:29:15.879 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 222930083, 2023-11-27T15:29:15.879 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 92326, 2023-11-27T15:29:15.880 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 222930083, 2023-11-27T15:29:15.880 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2023-11-27T15:29:15.880 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.880 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.880 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.880 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:15.881 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.881 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2023-11-27T15:29:15.881 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2023-11-27T15:29:15.881 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2023-11-27T15:29:15.881 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2023-11-27T15:29:15.881 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2023-11-27T15:29:15.881 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2023-11-27T15:29:15.882 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:15.882 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2023-11-27T15:29:15.882 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 92326, 2023-11-27T15:29:15.882 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 92326, 2023-11-27T15:29:15.882 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 92326, 2023-11-27T15:29:15.883 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 92326, 2023-11-27T15:29:15.883 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2023-11-27T15:29:15.883 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.883 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.883 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.883 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:15.883 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.884 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2023-11-27T15:29:15.884 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2023-11-27T15:29:15.884 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2023-11-27T15:29:15.884 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2023-11-27T15:29:15.884 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 12273, 2023-11-27T15:29:15.884 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 1336869, 2023-11-27T15:29:15.885 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:15.885 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 12273, 2023-11-27T15:29:15.885 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2023-11-27T15:29:15.885 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2023-11-27T15:29:15.885 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 236, 2023-11-27T15:29:15.885 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 1336869, 2023-11-27T15:29:15.886 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2023-11-27T15:29:15.886 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.886 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.886 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.886 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:15.886 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:15.887 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x56477cfe40a0": { 2023-11-27T15:29:15.887 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2023-11-27T15:29:15.887 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2023-11-27T15:29:15.887 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2023-11-27T15:29:15.887 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 67006, 2023-11-27T15:29:15.887 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 169472514, 2023-11-27T15:29:15.887 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:15.888 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 67006, 2023-11-27T15:29:15.888 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2023-11-27T15:29:15.888 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2023-11-27T15:29:15.888 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 1856, 2023-11-27T15:29:15.888 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 169472514, 2023-11-27T15:29:15.888 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2023-11-27T15:29:15.889 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:15.889 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:15.889 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:15.889 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:15.889 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:15.889 INFO:teuthology.orchestra.run.smithi174.stdout:} 2023-11-27T15:29:16.177 INFO:tasks.check_counter:Daemon mds.b mds.exported=397 2023-11-27T15:29:16.177 INFO:tasks.check_counter:Daemon mds.b mds.imported=452 2023-11-27T15:29:16.178 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=1471 2023-11-27T15:29:16.178 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2023-11-27T15:29:16.178 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=130 2023-11-27T15:29:16.178 DEBUG:tasks.check_counter:Getting stats from e 2023-11-27T15:29:16.179 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.e perf dump 2023-11-27T15:29:16.820 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:29:17.195 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:16 smithi174 ceph-mon[131110]: pgmap v541: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 8 op/s 2023-11-27T15:29:17.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:16 smithi133 ceph-mon[97860]: pgmap v541: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 8 op/s 2023-11-27T15:29:17.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:16 smithi177 ceph-mon[131504]: pgmap v541: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 8 op/s 2023-11-27T15:29:18.030 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2023-11-27T15:29:18.030 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:29:18.031 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 386, 2023-11-27T15:29:18.031 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 374, 2023-11-27T15:29:18.031 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 55095038, 2023-11-27T15:29:18.031 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 153145, 2023-11-27T15:29:18.031 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 7, 2023-11-27T15:29:18.031 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 3, 2023-11-27T15:29:18.032 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.083521689, 2023-11-27T15:29:18.032 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.013685505, 2023-11-27T15:29:18.032 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.043014680, 2023-11-27T15:29:18.032 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.025106566, 2023-11-27T15:29:18.032 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:18.032 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 374, 2023-11-27T15:29:18.033 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.021937839, 2023-11-27T15:29:18.033 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000058657 2023-11-27T15:29:18.033 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.033 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:18.033 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.033 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.034 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.034 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.034 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 499616, 2023-11-27T15:29:18.034 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 86896 2023-11-27T15:29:18.034 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.034 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:29:18.035 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 516, 2023-11-27T15:29:18.035 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 628, 2023-11-27T15:29:18.035 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 88132541, 2023-11-27T15:29:18.035 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 418282, 2023-11-27T15:29:18.035 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 6, 2023-11-27T15:29:18.035 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2023-11-27T15:29:18.035 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.099564972, 2023-11-27T15:29:18.036 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.015447263, 2023-11-27T15:29:18.036 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.062650162, 2023-11-27T15:29:18.036 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.015826694, 2023-11-27T15:29:18.036 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:18.036 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 626, 2023-11-27T15:29:18.036 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.027412062, 2023-11-27T15:29:18.037 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000043789 2023-11-27T15:29:18.037 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.037 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:18.037 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.037 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.037 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.037 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.038 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-27T15:29:18.038 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 258112 2023-11-27T15:29:18.038 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.038 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:29:18.038 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1062, 2023-11-27T15:29:18.038 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1064, 2023-11-27T15:29:18.039 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 95223680, 2023-11-27T15:29:18.039 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 330340, 2023-11-27T15:29:18.039 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 6, 2023-11-27T15:29:18.039 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2023-11-27T15:29:18.039 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.145921996, 2023-11-27T15:29:18.039 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.029694958, 2023-11-27T15:29:18.039 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.072178546, 2023-11-27T15:29:18.040 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.034997875, 2023-11-27T15:29:18.040 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:18.040 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1062, 2023-11-27T15:29:18.040 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.049985026, 2023-11-27T15:29:18.040 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000047066 2023-11-27T15:29:18.040 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.040 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:18.041 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.041 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.041 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.041 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.041 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:29:18.041 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:29:18.041 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.042 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2023-11-27T15:29:18.042 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2023-11-27T15:29:18.042 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2023-11-27T15:29:18.042 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.042 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2023-11-27T15:29:18.042 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2023-11-27T15:29:18.043 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2023-11-27T15:29:18.043 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2857, 2023-11-27T15:29:18.043 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.084836443, 2023-11-27T15:29:18.043 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000029694 2023-11-27T15:29:18.043 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:18.043 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.044 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2023-11-27T15:29:18.044 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2023-11-27T15:29:18.044 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2023-11-27T15:29:18.044 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.044 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.044 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.044 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:18.044 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.045 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2023-11-27T15:29:18.045 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 0, 2023-11-27T15:29:18.045 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 0, 2023-11-27T15:29:18.045 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2023-11-27T15:29:18.045 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.045 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.045 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.046 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.046 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2023-11-27T15:29:18.046 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 0, 2023-11-27T15:29:18.046 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2023-11-27T15:29:18.046 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2023-11-27T15:29:18.046 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 0, 2023-11-27T15:29:18.047 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2023-11-27T15:29:18.047 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2023-11-27T15:29:18.047 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 25381, 2023-11-27T15:29:18.047 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 0, 2023-11-27T15:29:18.047 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 0, 2023-11-27T15:29:18.047 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 25381, 2023-11-27T15:29:18.048 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 25381, 2023-11-27T15:29:18.048 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 0, 2023-11-27T15:29:18.048 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2023-11-27T15:29:18.048 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2023-11-27T15:29:18.048 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 1, 2023-11-27T15:29:18.048 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 0, 2023-11-27T15:29:18.049 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 0, 2023-11-27T15:29:18.049 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2023-11-27T15:29:18.049 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 0, 2023-11-27T15:29:18.049 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2023-11-27T15:29:18.049 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2023-11-27T15:29:18.049 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2023-11-27T15:29:18.050 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2023-11-27T15:29:18.050 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2023-11-27T15:29:18.050 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2023-11-27T15:29:18.050 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2023-11-27T15:29:18.050 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2023-11-27T15:29:18.051 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2023-11-27T15:29:18.051 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2023-11-27T15:29:18.051 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2023-11-27T15:29:18.051 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2023-11-27T15:29:18.051 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 0, 2023-11-27T15:29:18.051 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 0, 2023-11-27T15:29:18.052 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2023-11-27T15:29:18.052 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2023-11-27T15:29:18.052 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2023-11-27T15:29:18.052 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2023-11-27T15:29:18.052 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2023-11-27T15:29:18.053 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2023-11-27T15:29:18.053 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2023-11-27T15:29:18.053 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2023-11-27T15:29:18.053 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2023-11-27T15:29:18.053 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2023-11-27T15:29:18.054 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 0, 2023-11-27T15:29:18.054 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2023-11-27T15:29:18.054 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2023-11-27T15:29:18.054 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-27T15:29:18.054 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-27T15:29:18.055 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 0, 2023-11-27T15:29:18.055 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2023-11-27T15:29:18.055 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2023-11-27T15:29:18.055 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 0 2023-11-27T15:29:18.055 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.055 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2023-11-27T15:29:18.056 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2023-11-27T15:29:18.056 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2023-11-27T15:29:18.056 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2023-11-27T15:29:18.056 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 0, 2023-11-27T15:29:18.056 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2023-11-27T15:29:18.056 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2023-11-27T15:29:18.057 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2023-11-27T15:29:18.057 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2023-11-27T15:29:18.057 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2023-11-27T15:29:18.057 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2023-11-27T15:29:18.057 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2023-11-27T15:29:18.058 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2023-11-27T15:29:18.058 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2023-11-27T15:29:18.058 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2023-11-27T15:29:18.058 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2023-11-27T15:29:18.058 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2023-11-27T15:29:18.058 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2023-11-27T15:29:18.059 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2023-11-27T15:29:18.059 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2023-11-27T15:29:18.059 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2023-11-27T15:29:18.059 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2023-11-27T15:29:18.059 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2023-11-27T15:29:18.059 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2023-11-27T15:29:18.059 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.060 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2023-11-27T15:29:18.060 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 0, 2023-11-27T15:29:18.060 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2023-11-27T15:29:18.060 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2023-11-27T15:29:18.060 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 0, 2023-11-27T15:29:18.060 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2023-11-27T15:29:18.061 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2023-11-27T15:29:18.061 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 0, 2023-11-27T15:29:18.061 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2023-11-27T15:29:18.061 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2023-11-27T15:29:18.061 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 125, 2023-11-27T15:29:18.061 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2023-11-27T15:29:18.061 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2023-11-27T15:29:18.062 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2023-11-27T15:29:18.062 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 0, 2023-11-27T15:29:18.062 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 237240181, 2023-11-27T15:29:18.062 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2023-11-27T15:29:18.062 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.062 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.063 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.063 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.063 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 91419 2023-11-27T15:29:18.063 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.063 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2023-11-27T15:29:18.063 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 13, 2023-11-27T15:29:18.064 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 28783, 2023-11-27T15:29:18.064 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 28770, 2023-11-27T15:29:18.064 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 11, 2023-11-27T15:29:18.064 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 1826, 2023-11-27T15:29:18.064 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 1815, 2023-11-27T15:29:18.064 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 25381, 2023-11-27T15:29:18.064 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 78244, 2023-11-27T15:29:18.065 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 52863, 2023-11-27T15:29:18.065 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2023-11-27T15:29:18.065 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 0, 2023-11-27T15:29:18.065 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 0, 2023-11-27T15:29:18.065 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 100720, 2023-11-27T15:29:18.065 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 207156 2023-11-27T15:29:18.065 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.066 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2023-11-27T15:29:18.066 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 0, 2023-11-27T15:29:18.066 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2023-11-27T15:29:18.066 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 0, 2023-11-27T15:29:18.066 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 0, 2023-11-27T15:29:18.067 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 0, 2023-11-27T15:29:18.067 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2023-11-27T15:29:18.067 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.067 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.067 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.067 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.067 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2023-11-27T15:29:18.068 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.068 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.068 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.068 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.068 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2023-11-27T15:29:18.068 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.069 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.069 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.069 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.069 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2023-11-27T15:29:18.069 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.069 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.069 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.070 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.070 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2023-11-27T15:29:18.070 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.070 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.070 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.071 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.071 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2023-11-27T15:29:18.071 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.071 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.071 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.071 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.071 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2023-11-27T15:29:18.072 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.072 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.072 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.072 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.072 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2023-11-27T15:29:18.072 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.072 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.073 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.073 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.073 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2023-11-27T15:29:18.073 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.073 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.073 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.073 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.074 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2023-11-27T15:29:18.074 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.074 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.074 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.074 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.074 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2023-11-27T15:29:18.075 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.075 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.075 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.075 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.075 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2023-11-27T15:29:18.075 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.075 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.076 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.076 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.076 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2023-11-27T15:29:18.077 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.077 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.077 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.077 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.077 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2023-11-27T15:29:18.078 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.078 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.078 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.078 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.078 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2023-11-27T15:29:18.078 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.079 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.079 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.079 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.079 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2023-11-27T15:29:18.079 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.079 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.080 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.080 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.080 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2023-11-27T15:29:18.080 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.080 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.080 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.081 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.081 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2023-11-27T15:29:18.081 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.081 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.081 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.081 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.082 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2023-11-27T15:29:18.082 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.082 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.082 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.082 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.082 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2023-11-27T15:29:18.082 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.083 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.083 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.083 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.083 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2023-11-27T15:29:18.083 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.083 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.083 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.084 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.084 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2023-11-27T15:29:18.084 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.084 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.084 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.084 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.085 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2023-11-27T15:29:18.085 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.085 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.085 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.085 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.085 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2023-11-27T15:29:18.085 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.086 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.086 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.086 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.086 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2023-11-27T15:29:18.086 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.086 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.087 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.087 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.087 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2023-11-27T15:29:18.087 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.087 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.087 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.087 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.088 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2023-11-27T15:29:18.088 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.088 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.088 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.088 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.088 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2023-11-27T15:29:18.089 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.089 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.089 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.089 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.089 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2023-11-27T15:29:18.089 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.090 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.090 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.090 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.090 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2023-11-27T15:29:18.090 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2023-11-27T15:29:18.090 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2023-11-27T15:29:18.090 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.091 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.091 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.091 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:18.091 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.091 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2023-11-27T15:29:18.091 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2023-11-27T15:29:18.092 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 2, 2023-11-27T15:29:18.092 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 1, 2023-11-27T15:29:18.092 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 1, 2023-11-27T15:29:18.092 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2023-11-27T15:29:18.092 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 0, 2023-11-27T15:29:18.092 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 0, 2023-11-27T15:29:18.092 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 2317 2023-11-27T15:29:18.093 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.093 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2023-11-27T15:29:18.093 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:29:18.093 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2023-11-27T15:29:18.093 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:29:18.093 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:29:18.094 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:29:18.094 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:29:18.094 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:29:18.094 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:29:18.094 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:29:18.094 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:29:18.095 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:29:18.095 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:29:18.095 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:29:18.095 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:29:18.095 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:29:18.095 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2023-11-27T15:29:18.095 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:29:18.096 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2023-11-27T15:29:18.096 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:29:18.096 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:29:18.096 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:29:18.096 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:29:18.096 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:29:18.097 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:29:18.097 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:29:18.097 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2023-11-27T15:29:18.097 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:29:18.097 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:29:18.097 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:29:18.097 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2023-11-27T15:29:18.098 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2023-11-27T15:29:18.098 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2023-11-27T15:29:18.098 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:29:18.098 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:29:18.098 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:29:18.098 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:29:18.098 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 231346, 2023-11-27T15:29:18.099 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 75, 2023-11-27T15:29:18.099 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:29:18.099 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2023-11-27T15:29:18.099 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2023-11-27T15:29:18.099 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2023-11-27T15:29:18.100 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:29:18.100 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2023-11-27T15:29:18.100 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:29:18.100 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2023-11-27T15:29:18.100 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8888, 2023-11-27T15:29:18.100 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 170, 2023-11-27T15:29:18.101 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:29:18.101 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:29:18.101 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2023-11-27T15:29:18.101 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2023-11-27T15:29:18.101 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 17910984, 2023-11-27T15:29:18.101 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 101652, 2023-11-27T15:29:18.102 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2023-11-27T15:29:18.102 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2023-11-27T15:29:18.102 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2023-11-27T15:29:18.102 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2023-11-27T15:29:18.102 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.102 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2023-11-27T15:29:18.102 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2023-11-27T15:29:18.103 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2023-11-27T15:29:18.103 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 1790, 2023-11-27T15:29:18.103 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 21, 2023-11-27T15:29:18.103 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 3, 2023-11-27T15:29:18.103 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 1790, 2023-11-27T15:29:18.103 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2023-11-27T15:29:18.104 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1790, 2023-11-27T15:29:18.104 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 3.513953838, 2023-11-27T15:29:18.104 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001963102 2023-11-27T15:29:18.104 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.104 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2023-11-27T15:29:18.104 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2023-11-27T15:29:18.104 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1790, 2023-11-27T15:29:18.105 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1791 2023-11-27T15:29:18.105 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.105 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 1790, 2023-11-27T15:29:18.105 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 1790, 2023-11-27T15:29:18.105 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 0, 2023-11-27T15:29:18.105 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2023-11-27T15:29:18.105 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2023-11-27T15:29:18.106 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 1104, 2023-11-27T15:29:18.106 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 0, 2023-11-27T15:29:18.106 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 684, 2023-11-27T15:29:18.106 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 0, 2023-11-27T15:29:18.106 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 0, 2023-11-27T15:29:18.106 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2023-11-27T15:29:18.107 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2023-11-27T15:29:18.107 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2023-11-27T15:29:18.107 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2023-11-27T15:29:18.107 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 0, 2023-11-27T15:29:18.107 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2023-11-27T15:29:18.107 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2023-11-27T15:29:18.107 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2023-11-27T15:29:18.108 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 1, 2023-11-27T15:29:18.108 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 0, 2023-11-27T15:29:18.108 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2023-11-27T15:29:18.108 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2023-11-27T15:29:18.108 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2023-11-27T15:29:18.108 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2023-11-27T15:29:18.109 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2023-11-27T15:29:18.109 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2023-11-27T15:29:18.109 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2023-11-27T15:29:18.109 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2023-11-27T15:29:18.109 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2023-11-27T15:29:18.109 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2023-11-27T15:29:18.109 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2023-11-27T15:29:18.110 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2023-11-27T15:29:18.110 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2023-11-27T15:29:18.110 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2023-11-27T15:29:18.110 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2023-11-27T15:29:18.110 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2023-11-27T15:29:18.110 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2023-11-27T15:29:18.110 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2023-11-27T15:29:18.111 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2023-11-27T15:29:18.111 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2023-11-27T15:29:18.111 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2023-11-27T15:29:18.111 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2023-11-27T15:29:18.111 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2023-11-27T15:29:18.111 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2023-11-27T15:29:18.112 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2023-11-27T15:29:18.112 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2023-11-27T15:29:18.112 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 94, 2023-11-27T15:29:18.112 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2023-11-27T15:29:18.112 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 4, 2023-11-27T15:29:18.112 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 10, 2023-11-27T15:29:18.112 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 10, 2023-11-27T15:29:18.112 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2023-11-27T15:29:18.113 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2023-11-27T15:29:18.113 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 0, 2023-11-27T15:29:18.113 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 2, 2023-11-27T15:29:18.113 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 0 2023-11-27T15:29:18.113 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.113 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2023-11-27T15:29:18.114 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 0, 2023-11-27T15:29:18.114 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2023-11-27T15:29:18.114 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2023-11-27T15:29:18.114 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2023-11-27T15:29:18.114 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.114 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2023-11-27T15:29:18.114 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2023-11-27T15:29:18.115 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.115 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2023-11-27T15:29:18.115 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2023-11-27T15:29:18.115 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2023-11-27T15:29:18.115 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2023-11-27T15:29:18.115 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2023-11-27T15:29:18.116 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2023-11-27T15:29:18.116 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2023-11-27T15:29:18.116 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.116 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:29:18.116 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2023-11-27T15:29:18.116 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2023-11-27T15:29:18.116 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2023-11-27T15:29:18.117 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 1960, 2023-11-27T15:29:18.117 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 238263242, 2023-11-27T15:29:18.117 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:18.117 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 1960, 2023-11-27T15:29:18.117 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2023-11-27T15:29:18.117 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2023-11-27T15:29:18.117 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 1960, 2023-11-27T15:29:18.118 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 238263242, 2023-11-27T15:29:18.118 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2023-11-27T15:29:18.118 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.118 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.118 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.118 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:18.118 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.119 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2023-11-27T15:29:18.119 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2023-11-27T15:29:18.119 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2023-11-27T15:29:18.119 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2023-11-27T15:29:18.119 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2023-11-27T15:29:18.119 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2023-11-27T15:29:18.120 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:18.120 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2023-11-27T15:29:18.120 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 1790, 2023-11-27T15:29:18.120 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 237453593, 2023-11-27T15:29:18.120 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 1790, 2023-11-27T15:29:18.120 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 237453593, 2023-11-27T15:29:18.120 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2023-11-27T15:29:18.121 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.121 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.121 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.121 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:18.121 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.121 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2023-11-27T15:29:18.121 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2023-11-27T15:29:18.122 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2023-11-27T15:29:18.122 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2023-11-27T15:29:18.122 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2023-11-27T15:29:18.122 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2023-11-27T15:29:18.122 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:18.122 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2023-11-27T15:29:18.122 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 1790, 2023-11-27T15:29:18.123 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 1790, 2023-11-27T15:29:18.123 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 1790, 2023-11-27T15:29:18.123 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 1790, 2023-11-27T15:29:18.123 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2023-11-27T15:29:18.123 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.123 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.124 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.124 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:18.124 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.124 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2023-11-27T15:29:18.124 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2023-11-27T15:29:18.124 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2023-11-27T15:29:18.124 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2023-11-27T15:29:18.125 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2023-11-27T15:29:18.125 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2023-11-27T15:29:18.125 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:18.125 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2023-11-27T15:29:18.125 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2023-11-27T15:29:18.125 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2023-11-27T15:29:18.125 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2023-11-27T15:29:18.126 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2023-11-27T15:29:18.126 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2023-11-27T15:29:18.126 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.126 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.126 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.126 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:18.126 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:18.127 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x556c725cc460": { 2023-11-27T15:29:18.127 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2023-11-27T15:29:18.127 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2023-11-27T15:29:18.127 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2023-11-27T15:29:18.127 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2023-11-27T15:29:18.127 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2023-11-27T15:29:18.127 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:18.128 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2023-11-27T15:29:18.128 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2023-11-27T15:29:18.128 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2023-11-27T15:29:18.128 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2023-11-27T15:29:18.128 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2023-11-27T15:29:18.128 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2023-11-27T15:29:18.128 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:18.129 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:18.129 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:18.129 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:18.129 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:18.129 INFO:teuthology.orchestra.run.smithi174.stdout:} 2023-11-27T15:29:18.471 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-11-27T15:29:18.471 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-11-27T15:29:18.472 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2023-11-27T15:29:18.472 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2023-11-27T15:29:18.472 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2023-11-27T15:29:18.472 DEBUG:tasks.check_counter:Getting stats from h 2023-11-27T15:29:18.472 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.h perf dump 2023-11-27T15:29:19.087 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:18 smithi177 ceph-mon[131504]: pgmap v542: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 8 op/s 2023-11-27T15:29:19.113 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:29:19.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:18 smithi174 ceph-mon[131110]: pgmap v542: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 8 op/s 2023-11-27T15:29:19.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:18 smithi133 ceph-mon[97860]: pgmap v542: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 8 op/s 2023-11-27T15:29:20.131 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:19 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:29:20.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:19 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:29:20.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:19 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-11-27T15:29:20.368 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2023-11-27T15:29:20.368 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:29:20.368 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 25399, 2023-11-27T15:29:20.369 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 25864, 2023-11-27T15:29:20.369 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 18917720, 2023-11-27T15:29:20.369 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 106674103, 2023-11-27T15:29:20.369 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 19, 2023-11-27T15:29:20.370 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 4, 2023-11-27T15:29:20.370 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 1.768593514, 2023-11-27T15:29:20.370 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.674022428, 2023-11-27T15:29:20.370 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.584870988, 2023-11-27T15:29:20.370 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.525689147, 2023-11-27T15:29:20.371 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:20.371 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 25862, 2023-11-27T15:29:20.371 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 2.687579326, 2023-11-27T15:29:20.371 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000103920 2023-11-27T15:29:20.371 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.372 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:20.372 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 18883, 2023-11-27T15:29:20.372 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.003016127, 2023-11-27T15:29:20.372 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000159 2023-11-27T15:29:20.372 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.372 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:29:20.373 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:29:20.373 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.373 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:29:20.373 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 95447, 2023-11-27T15:29:20.373 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 182198, 2023-11-27T15:29:20.374 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 38772048, 2023-11-27T15:29:20.374 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 178023034, 2023-11-27T15:29:20.374 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 16, 2023-11-27T15:29:20.374 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 4, 2023-11-27T15:29:20.374 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 6.745309084, 2023-11-27T15:29:20.375 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 3.114645766, 2023-11-27T15:29:20.375 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 6.480150963, 2023-11-27T15:29:20.375 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.439873643, 2023-11-27T15:29:20.375 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:20.375 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 182196, 2023-11-27T15:29:20.376 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 11.379877437, 2023-11-27T15:29:20.376 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000062459 2023-11-27T15:29:20.376 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.376 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:20.376 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 204189, 2023-11-27T15:29:20.376 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.035612168, 2023-11-27T15:29:20.377 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000174 2023-11-27T15:29:20.377 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.377 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 31942848, 2023-11-27T15:29:20.377 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 95420112 2023-11-27T15:29:20.377 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.378 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:29:20.378 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 66580, 2023-11-27T15:29:20.378 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 57502, 2023-11-27T15:29:20.378 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 24420188, 2023-11-27T15:29:20.378 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 123494472, 2023-11-27T15:29:20.379 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 7, 2023-11-27T15:29:20.379 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 6, 2023-11-27T15:29:20.379 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 3.244978302, 2023-11-27T15:29:20.379 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 1.156849522, 2023-11-27T15:29:20.379 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 1.708414120, 2023-11-27T15:29:20.380 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.645462569, 2023-11-27T15:29:20.380 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:20.380 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 57502, 2023-11-27T15:29:20.380 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 4.698834058, 2023-11-27T15:29:20.380 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000081716 2023-11-27T15:29:20.380 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.381 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:20.381 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 77113, 2023-11-27T15:29:20.381 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.013927615, 2023-11-27T15:29:20.381 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000180 2023-11-27T15:29:20.381 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.382 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 514128, 2023-11-27T15:29:20.382 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 93616 2023-11-27T15:29:20.382 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.382 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2023-11-27T15:29:20.382 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2023-11-27T15:29:20.382 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2023-11-27T15:29:20.383 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.383 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2023-11-27T15:29:20.383 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2023-11-27T15:29:20.383 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2023-11-27T15:29:20.383 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 14692, 2023-11-27T15:29:20.384 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 67.871545510, 2023-11-27T15:29:20.384 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.004619626 2023-11-27T15:29:20.384 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:20.384 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.384 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2023-11-27T15:29:20.384 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2023-11-27T15:29:20.385 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2023-11-27T15:29:20.385 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 22178, 2023-11-27T15:29:20.385 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 5.093544119, 2023-11-27T15:29:20.385 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000229666 2023-11-27T15:29:20.385 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:20.386 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.386 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2023-11-27T15:29:20.386 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 57450, 2023-11-27T15:29:20.386 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 47654, 2023-11-27T15:29:20.386 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2023-11-27T15:29:20.387 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 47654, 2023-11-27T15:29:20.387 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 169.962773717, 2023-11-27T15:29:20.387 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003566600 2023-11-27T15:29:20.387 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.387 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2023-11-27T15:29:20.387 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 9796, 2023-11-27T15:29:20.388 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 861, 2023-11-27T15:29:20.388 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2023-11-27T15:29:20.388 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 1634, 2023-11-27T15:29:20.388 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 193, 2023-11-27T15:29:20.388 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 79, 2023-11-27T15:29:20.389 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 61, 2023-11-27T15:29:20.389 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 30, 2023-11-27T15:29:20.389 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 19, 2023-11-27T15:29:20.389 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 12, 2023-11-27T15:29:20.389 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 17, 2023-11-27T15:29:20.389 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 32416, 2023-11-27T15:29:20.390 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2023-11-27T15:29:20.390 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2023-11-27T15:29:20.390 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 4, 2023-11-27T15:29:20.390 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 69513, 2023-11-27T15:29:20.390 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 59459, 2023-11-27T15:29:20.391 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 9052, 2023-11-27T15:29:20.391 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 108, 2023-11-27T15:29:20.391 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 164, 2023-11-27T15:29:20.391 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2023-11-27T15:29:20.391 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2023-11-27T15:29:20.391 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2023-11-27T15:29:20.392 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2023-11-27T15:29:20.392 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 986, 2023-11-27T15:29:20.392 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 29182, 2023-11-27T15:29:20.392 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 820, 2023-11-27T15:29:20.392 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2023-11-27T15:29:20.392 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2023-11-27T15:29:20.393 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 19984, 2023-11-27T15:29:20.393 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2023-11-27T15:29:20.393 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 20148, 2023-11-27T15:29:20.393 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 246, 2023-11-27T15:29:20.393 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2023-11-27T15:29:20.394 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 19984, 2023-11-27T15:29:20.394 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2023-11-27T15:29:20.394 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2023-11-27T15:29:20.394 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2023-11-27T15:29:20.394 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 5, 2023-11-27T15:29:20.394 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2023-11-27T15:29:20.395 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 5, 2023-11-27T15:29:20.395 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 19979, 2023-11-27T15:29:20.395 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 8, 2023-11-27T15:29:20.395 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 90, 2023-11-27T15:29:20.395 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 36188, 2023-11-27T15:29:20.395 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2023-11-27T15:29:20.396 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-27T15:29:20.396 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-27T15:29:20.396 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 23017, 2023-11-27T15:29:20.396 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 22924, 2023-11-27T15:29:20.396 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 244, 2023-11-27T15:29:20.397 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 24191 2023-11-27T15:29:20.397 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.397 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2023-11-27T15:29:20.397 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 790, 2023-11-27T15:29:20.397 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 2039, 2023-11-27T15:29:20.398 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 1022, 2023-11-27T15:29:20.398 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 1400, 2023-11-27T15:29:20.398 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 1303, 2023-11-27T15:29:20.398 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 2, 2023-11-27T15:29:20.398 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2023-11-27T15:29:20.398 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2023-11-27T15:29:20.399 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 12480, 2023-11-27T15:29:20.399 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 12478, 2023-11-27T15:29:20.399 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2023-11-27T15:29:20.399 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2023-11-27T15:29:20.399 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2023-11-27T15:29:20.399 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2023-11-27T15:29:20.400 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2023-11-27T15:29:20.400 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2023-11-27T15:29:20.400 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2023-11-27T15:29:20.400 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2023-11-27T15:29:20.400 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 989, 2023-11-27T15:29:20.400 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2023-11-27T15:29:20.401 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 305, 2023-11-27T15:29:20.401 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2023-11-27T15:29:20.401 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2023-11-27T15:29:20.401 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.401 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2023-11-27T15:29:20.401 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 75332, 2023-11-27T15:29:20.402 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2023-11-27T15:29:20.402 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2023-11-27T15:29:20.402 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 75332, 2023-11-27T15:29:20.402 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2023-11-27T15:29:20.402 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2023-11-27T15:29:20.403 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 94, 2023-11-27T15:29:20.403 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2023-11-27T15:29:20.403 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2023-11-27T15:29:20.403 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 94, 2023-11-27T15:29:20.403 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2023-11-27T15:29:20.403 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2023-11-27T15:29:20.404 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2023-11-27T15:29:20.404 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 210709649, 2023-11-27T15:29:20.404 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2023-11-27T15:29:20.404 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2023-11-27T15:29:20.404 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2860, 2023-11-27T15:29:20.404 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 120.878306143, 2023-11-27T15:29:20.405 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.042265142 2023-11-27T15:29:20.405 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.405 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2023-11-27T15:29:20.405 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.405 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2023-11-27T15:29:20.405 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 26, 2023-11-27T15:29:20.406 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 31460, 2023-11-27T15:29:20.406 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 31434, 2023-11-27T15:29:20.406 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 28, 2023-11-27T15:29:20.406 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 2972, 2023-11-27T15:29:20.406 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 2944, 2023-11-27T15:29:20.406 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 61, 2023-11-27T15:29:20.407 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 59610, 2023-11-27T15:29:20.407 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 59549, 2023-11-27T15:29:20.407 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2023-11-27T15:29:20.407 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 40766, 2023-11-27T15:29:20.407 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 40766, 2023-11-27T15:29:20.407 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 197536, 2023-11-27T15:29:20.408 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 198964 2023-11-27T15:29:20.408 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.408 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2023-11-27T15:29:20.408 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 59008, 2023-11-27T15:29:20.408 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 330, 2023-11-27T15:29:20.408 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 57450, 2023-11-27T15:29:20.409 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 755, 2023-11-27T15:29:20.409 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 671, 2023-11-27T15:29:20.409 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2023-11-27T15:29:20.409 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 23729, 2023-11-27T15:29:20.409 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 102.687694956, 2023-11-27T15:29:20.409 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.004327518 2023-11-27T15:29:20.410 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.410 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2023-11-27T15:29:20.410 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 6, 2023-11-27T15:29:20.410 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.270008279, 2023-11-27T15:29:20.410 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.045001379 2023-11-27T15:29:20.411 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.411 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2023-11-27T15:29:20.411 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.411 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.411 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.411 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.412 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2023-11-27T15:29:20.412 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.412 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.412 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.412 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.412 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2023-11-27T15:29:20.412 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 10591, 2023-11-27T15:29:20.413 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 14.829973392, 2023-11-27T15:29:20.413 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001400242 2023-11-27T15:29:20.413 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.413 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2023-11-27T15:29:20.413 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.413 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.414 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.414 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.414 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2023-11-27T15:29:20.414 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.414 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.414 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.415 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.415 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2023-11-27T15:29:20.415 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.415 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.415 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.415 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.416 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2023-11-27T15:29:20.416 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.416 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.416 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.416 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.416 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2023-11-27T15:29:20.417 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.417 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.417 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.417 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.417 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2023-11-27T15:29:20.417 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.418 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.418 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.418 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.418 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2023-11-27T15:29:20.418 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.418 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.419 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.419 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.419 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2023-11-27T15:29:20.419 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.419 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.419 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.420 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.420 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2023-11-27T15:29:20.420 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.420 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.420 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.420 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.420 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2023-11-27T15:29:20.421 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.421 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.421 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.421 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.421 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2023-11-27T15:29:20.421 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 848, 2023-11-27T15:29:20.422 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 11.329014132, 2023-11-27T15:29:20.422 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.013359686 2023-11-27T15:29:20.422 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.422 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2023-11-27T15:29:20.422 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.422 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.423 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.423 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.423 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2023-11-27T15:29:20.423 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.423 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.423 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.423 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.424 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2023-11-27T15:29:20.424 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.424 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.424 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.424 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.424 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2023-11-27T15:29:20.425 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2, 2023-11-27T15:29:20.425 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.055680602, 2023-11-27T15:29:20.425 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.027840301 2023-11-27T15:29:20.425 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.425 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2023-11-27T15:29:20.425 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.426 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.426 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.426 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.426 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2023-11-27T15:29:20.426 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.426 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.427 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.427 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.427 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2023-11-27T15:29:20.427 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.427 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.427 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.428 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.428 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2023-11-27T15:29:20.428 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.428 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.428 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.428 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.429 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2023-11-27T15:29:20.429 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.429 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.429 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.429 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.429 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2023-11-27T15:29:20.429 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.430 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.430 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.430 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.430 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2023-11-27T15:29:20.430 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.431 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.431 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.431 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.431 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2023-11-27T15:29:20.431 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.431 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.431 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.432 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.432 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2023-11-27T15:29:20.432 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 12478, 2023-11-27T15:29:20.432 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 40.790402356, 2023-11-27T15:29:20.432 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003268985 2023-11-27T15:29:20.432 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.432 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2023-11-27T15:29:20.433 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2023-11-27T15:29:20.433 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2023-11-27T15:29:20.433 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.433 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.433 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.433 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:20.434 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.434 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2023-11-27T15:29:20.434 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2023-11-27T15:29:20.434 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 1, 2023-11-27T15:29:20.434 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 0, 2023-11-27T15:29:20.434 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 1, 2023-11-27T15:29:20.435 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2023-11-27T15:29:20.435 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 16830, 2023-11-27T15:29:20.435 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 16830, 2023-11-27T15:29:20.435 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 395 2023-11-27T15:29:20.435 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.435 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2023-11-27T15:29:20.435 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:29:20.436 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2023-11-27T15:29:20.436 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:29:20.436 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:29:20.436 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:29:20.436 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:29:20.436 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:29:20.437 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:29:20.437 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:29:20.437 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:29:20.437 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:29:20.437 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:29:20.437 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:29:20.437 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:29:20.438 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:29:20.438 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2023-11-27T15:29:20.438 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:29:20.438 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2023-11-27T15:29:20.438 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:29:20.438 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:29:20.439 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:29:20.439 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:29:20.439 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:29:20.439 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:29:20.439 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:29:20.439 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2023-11-27T15:29:20.440 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:29:20.440 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:29:20.440 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:29:20.440 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2023-11-27T15:29:20.440 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2023-11-27T15:29:20.440 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2023-11-27T15:29:20.440 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:29:20.441 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:29:20.441 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:29:20.441 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:29:20.441 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 298655, 2023-11-27T15:29:20.441 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 122, 2023-11-27T15:29:20.441 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 704, 2023-11-27T15:29:20.441 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 8, 2023-11-27T15:29:20.442 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2023-11-27T15:29:20.442 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2023-11-27T15:29:20.442 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:29:20.442 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2023-11-27T15:29:20.442 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:29:20.442 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2023-11-27T15:29:20.443 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8888, 2023-11-27T15:29:20.443 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 170, 2023-11-27T15:29:20.443 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:29:20.443 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:29:20.443 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2023-11-27T15:29:20.443 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2023-11-27T15:29:20.443 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 139278, 2023-11-27T15:29:20.444 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 1070, 2023-11-27T15:29:20.444 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2023-11-27T15:29:20.444 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2023-11-27T15:29:20.444 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2023-11-27T15:29:20.444 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2023-11-27T15:29:20.444 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.444 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2023-11-27T15:29:20.445 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2023-11-27T15:29:20.445 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2023-11-27T15:29:20.445 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 52231, 2023-11-27T15:29:20.445 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 224334134, 2023-11-27T15:29:20.445 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 33, 2023-11-27T15:29:20.445 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 52231, 2023-11-27T15:29:20.446 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2023-11-27T15:29:20.446 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 52231, 2023-11-27T15:29:20.446 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 300.931044371, 2023-11-27T15:29:20.446 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.005761540 2023-11-27T15:29:20.446 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.446 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2023-11-27T15:29:20.446 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2023-11-27T15:29:20.447 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 52231, 2023-11-27T15:29:20.447 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 54733 2023-11-27T15:29:20.447 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.447 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 52231, 2023-11-27T15:29:20.447 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 21023, 2023-11-27T15:29:20.447 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 31208, 2023-11-27T15:29:20.448 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2023-11-27T15:29:20.448 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2023-11-27T15:29:20.448 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 0, 2023-11-27T15:29:20.448 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 10, 2023-11-27T15:29:20.448 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 178, 2023-11-27T15:29:20.448 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 3038, 2023-11-27T15:29:20.448 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 93, 2023-11-27T15:29:20.449 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2023-11-27T15:29:20.449 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2023-11-27T15:29:20.449 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2023-11-27T15:29:20.449 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2023-11-27T15:29:20.449 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 26392, 2023-11-27T15:29:20.449 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2023-11-27T15:29:20.449 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2023-11-27T15:29:20.450 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2023-11-27T15:29:20.450 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 19984, 2023-11-27T15:29:20.450 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 20, 2023-11-27T15:29:20.450 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2023-11-27T15:29:20.450 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2023-11-27T15:29:20.450 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2023-11-27T15:29:20.451 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2023-11-27T15:29:20.451 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2023-11-27T15:29:20.451 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2023-11-27T15:29:20.451 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2023-11-27T15:29:20.451 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2023-11-27T15:29:20.451 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2023-11-27T15:29:20.451 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2023-11-27T15:29:20.452 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2023-11-27T15:29:20.452 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2023-11-27T15:29:20.452 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2023-11-27T15:29:20.452 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2023-11-27T15:29:20.452 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2023-11-27T15:29:20.452 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2023-11-27T15:29:20.452 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2023-11-27T15:29:20.453 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2023-11-27T15:29:20.453 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2023-11-27T15:29:20.453 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2023-11-27T15:29:20.453 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2023-11-27T15:29:20.453 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2023-11-27T15:29:20.453 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2023-11-27T15:29:20.453 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2023-11-27T15:29:20.454 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2023-11-27T15:29:20.454 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2023-11-27T15:29:20.454 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 94, 2023-11-27T15:29:20.454 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2023-11-27T15:29:20.454 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 5, 2023-11-27T15:29:20.454 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 12, 2023-11-27T15:29:20.454 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 12, 2023-11-27T15:29:20.455 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2023-11-27T15:29:20.455 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2023-11-27T15:29:20.455 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 3261, 2023-11-27T15:29:20.455 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 1722, 2023-11-27T15:29:20.455 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 35 2023-11-27T15:29:20.455 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.455 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2023-11-27T15:29:20.456 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2023-11-27T15:29:20.456 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2023-11-27T15:29:20.456 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2023-11-27T15:29:20.456 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2023-11-27T15:29:20.456 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.456 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2023-11-27T15:29:20.457 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 59078 2023-11-27T15:29:20.457 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.457 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2023-11-27T15:29:20.457 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2023-11-27T15:29:20.457 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 16, 2023-11-27T15:29:20.457 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2023-11-27T15:29:20.457 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 8, 2023-11-27T15:29:20.458 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 12478, 2023-11-27T15:29:20.458 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2023-11-27T15:29:20.458 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.458 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:29:20.458 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2023-11-27T15:29:20.458 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2023-11-27T15:29:20.458 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2023-11-27T15:29:20.459 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 187422, 2023-11-27T15:29:20.459 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 61339591, 2023-11-27T15:29:20.459 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:20.459 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 187422, 2023-11-27T15:29:20.459 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2023-11-27T15:29:20.459 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2023-11-27T15:29:20.459 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 187422, 2023-11-27T15:29:20.460 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 61339591, 2023-11-27T15:29:20.460 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2023-11-27T15:29:20.460 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.460 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.460 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.460 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:20.460 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.461 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2023-11-27T15:29:20.461 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2023-11-27T15:29:20.461 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2023-11-27T15:29:20.461 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2023-11-27T15:29:20.461 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2023-11-27T15:29:20.461 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2023-11-27T15:29:20.461 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:20.462 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2023-11-27T15:29:20.462 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 52231, 2023-11-27T15:29:20.462 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 225680460, 2023-11-27T15:29:20.462 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 52231, 2023-11-27T15:29:20.462 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 225680460, 2023-11-27T15:29:20.462 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2023-11-27T15:29:20.462 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.463 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.463 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.463 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:20.463 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.463 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2023-11-27T15:29:20.463 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2023-11-27T15:29:20.463 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2023-11-27T15:29:20.464 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2023-11-27T15:29:20.464 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2023-11-27T15:29:20.464 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2023-11-27T15:29:20.464 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:20.464 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2023-11-27T15:29:20.464 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 52231, 2023-11-27T15:29:20.464 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 52231, 2023-11-27T15:29:20.465 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 52231, 2023-11-27T15:29:20.465 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 52231, 2023-11-27T15:29:20.465 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2023-11-27T15:29:20.465 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.465 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.465 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.465 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:20.466 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.466 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2023-11-27T15:29:20.466 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2023-11-27T15:29:20.466 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2023-11-27T15:29:20.466 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2023-11-27T15:29:20.466 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 12478, 2023-11-27T15:29:20.466 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 1360102, 2023-11-27T15:29:20.467 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:20.467 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 12478, 2023-11-27T15:29:20.467 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2023-11-27T15:29:20.467 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2023-11-27T15:29:20.467 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 178, 2023-11-27T15:29:20.467 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 1360102, 2023-11-27T15:29:20.467 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2023-11-27T15:29:20.468 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.468 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.468 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.468 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:20.468 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:20.468 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x55a1856b40a0": { 2023-11-27T15:29:20.468 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2023-11-27T15:29:20.469 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2023-11-27T15:29:20.469 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2023-11-27T15:29:20.469 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 75332, 2023-11-27T15:29:20.469 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 206515345, 2023-11-27T15:29:20.469 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:20.469 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 75332, 2023-11-27T15:29:20.469 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2023-11-27T15:29:20.470 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2023-11-27T15:29:20.470 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 2860, 2023-11-27T15:29:20.470 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 206515345, 2023-11-27T15:29:20.470 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2023-11-27T15:29:20.470 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:20.470 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:20.470 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:20.471 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:20.471 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:20.471 INFO:teuthology.orchestra.run.smithi174.stdout:} 2023-11-27T15:29:20.769 INFO:tasks.check_counter:Daemon mds.h mds.exported=986 2023-11-27T15:29:20.769 INFO:tasks.check_counter:Daemon mds.h mds.imported=820 2023-11-27T15:29:20.770 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=790 2023-11-27T15:29:20.770 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=2039 2023-11-27T15:29:20.770 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=193 2023-11-27T15:29:20.770 DEBUG:tasks.check_counter:Getting stats from k 2023-11-27T15:29:20.771 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.k perf dump 2023-11-27T15:29:21.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:20 smithi174 ceph-mon[131110]: pgmap v543: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 7 op/s 2023-11-27T15:29:21.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:20 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:29:21.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:20 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:29:21.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:20 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:29:21.153 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:20 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:29:21.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:20 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:29:21.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:20 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:29:21.154 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:20 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:29:21.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:20 smithi133 ceph-mon[97860]: pgmap v543: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 7 op/s 2023-11-27T15:29:21.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:20 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:29:21.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:20 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:29:21.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:20 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:29:21.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:20 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:29:21.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:20 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:29:21.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:20 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:29:21.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:20 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:29:21.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:20 smithi177 ceph-mon[131504]: pgmap v543: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 7 op/s 2023-11-27T15:29:21.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:20 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:29:21.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:20 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:29:21.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:20 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:29:21.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:20 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-11-27T15:29:21.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:20 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-11-27T15:29:21.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:20 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-11-27T15:29:21.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:20 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' 2023-11-27T15:29:21.414 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.b/config 2023-11-27T15:29:22.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:21 smithi174 ceph-mon[131110]: Adjusting osd_memory_target on smithi133 to 221.9M 2023-11-27T15:29:22.241 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:21 smithi174 ceph-mon[131110]: Unable to set osd_memory_target on smithi133 to 232752128: error parsing value: Value '232752128' is below minimum 939524096 2023-11-27T15:29:22.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:21 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:29:22.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:21 smithi174 ceph-mon[131110]: Service mgr has no related daemon already placed. 2023-11-27T15:29:22.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:21 smithi174 ceph-mon[131110]: Service mon has no related daemon already placed. 2023-11-27T15:29:22.242 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:21 smithi174 ceph-mon[131110]: Service mds.all has no related daemon already placed. 2023-11-27T15:29:22.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:21 smithi133 ceph-mon[97860]: Adjusting osd_memory_target on smithi133 to 221.9M 2023-11-27T15:29:22.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:21 smithi133 ceph-mon[97860]: Unable to set osd_memory_target on smithi133 to 232752128: error parsing value: Value '232752128' is below minimum 939524096 2023-11-27T15:29:22.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:21 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:29:22.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:21 smithi133 ceph-mon[97860]: Service mgr has no related daemon already placed. 2023-11-27T15:29:22.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:21 smithi133 ceph-mon[97860]: Service mon has no related daemon already placed. 2023-11-27T15:29:22.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:21 smithi133 ceph-mon[97860]: Service mds.all has no related daemon already placed. 2023-11-27T15:29:22.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:21 smithi177 ceph-mon[131504]: Adjusting osd_memory_target on smithi133 to 221.9M 2023-11-27T15:29:22.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:21 smithi177 ceph-mon[131504]: Unable to set osd_memory_target on smithi133 to 232752128: error parsing value: Value '232752128' is below minimum 939524096 2023-11-27T15:29:22.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:21 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:29:22.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:21 smithi177 ceph-mon[131504]: Service mgr has no related daemon already placed. 2023-11-27T15:29:22.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:21 smithi177 ceph-mon[131504]: Service mon has no related daemon already placed. 2023-11-27T15:29:22.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:21 smithi177 ceph-mon[131504]: Service mds.all has no related daemon already placed. 2023-11-27T15:29:22.695 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2023-11-27T15:29:22.695 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:29:22.695 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 160, 2023-11-27T15:29:22.696 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 155, 2023-11-27T15:29:22.696 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 456928, 2023-11-27T15:29:22.696 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 87136, 2023-11-27T15:29:22.696 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 4, 2023-11-27T15:29:22.696 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 0, 2023-11-27T15:29:22.696 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.025964694, 2023-11-27T15:29:22.696 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.008598379, 2023-11-27T15:29:22.696 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.013324067, 2023-11-27T15:29:22.697 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.004307167, 2023-11-27T15:29:22.697 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:22.697 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 155, 2023-11-27T15:29:22.697 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.009495478, 2023-11-27T15:29:22.697 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000061261 2023-11-27T15:29:22.697 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:22.697 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:22.698 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:22.698 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:22.698 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:22.698 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:22.698 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 456928, 2023-11-27T15:29:22.698 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 87136 2023-11-27T15:29:22.698 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:22.698 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:29:22.699 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1, 2023-11-27T15:29:22.699 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 116, 2023-11-27T15:29:22.699 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 160, 2023-11-27T15:29:22.699 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 96192, 2023-11-27T15:29:22.699 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 3, 2023-11-27T15:29:22.699 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 2, 2023-11-27T15:29:22.699 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.006447199, 2023-11-27T15:29:22.699 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.004368525, 2023-11-27T15:29:22.700 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.000861718, 2023-11-27T15:29:22.700 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:29:22.700 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:22.700 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 114, 2023-11-27T15:29:22.700 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.009031647, 2023-11-27T15:29:22.700 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000079224 2023-11-27T15:29:22.700 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:22.700 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:22.701 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:22.701 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:22.701 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:22.701 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:22.701 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-27T15:29:22.701 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 96192 2023-11-27T15:29:22.701 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:22.701 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:29:22.702 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 0, 2023-11-27T15:29:22.702 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 2, 2023-11-27T15:29:22.702 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 0, 2023-11-27T15:29:22.702 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 0, 2023-11-27T15:29:22.702 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 2, 2023-11-27T15:29:22.702 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 1, 2023-11-27T15:29:22.702 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.000440401, 2023-11-27T15:29:22.702 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.000000000, 2023-11-27T15:29:22.703 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.000371909, 2023-11-27T15:29:22.703 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:29:22.703 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:22.703 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:22.703 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:22.703 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:22.703 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:22.703 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:22.704 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:22.704 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:22.704 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:22.704 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:22.704 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:29:22.704 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:29:22.704 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:22.704 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2023-11-27T15:29:22.705 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 0, 2023-11-27T15:29:22.705 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2023-11-27T15:29:22.705 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:22.705 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2023-11-27T15:29:22.705 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:29:22.705 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2023-11-27T15:29:22.705 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:29:22.705 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:29:22.706 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:29:22.706 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:29:22.706 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:29:22.706 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:29:22.706 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:29:22.706 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:29:22.706 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:29:22.706 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:29:22.707 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:29:22.707 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:29:22.707 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:29:22.707 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2023-11-27T15:29:22.707 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:29:22.707 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2023-11-27T15:29:22.707 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:29:22.709 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:29:22.709 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:29:22.709 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:29:22.709 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:29:22.709 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:29:22.709 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:29:22.709 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2023-11-27T15:29:22.709 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:29:22.710 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:29:22.710 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:29:22.710 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2023-11-27T15:29:22.710 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2023-11-27T15:29:22.710 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2023-11-27T15:29:22.710 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:29:22.710 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:29:22.711 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:29:22.711 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:29:22.711 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 149024, 2023-11-27T15:29:22.711 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 31, 2023-11-27T15:29:22.711 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:29:22.711 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2023-11-27T15:29:22.711 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2023-11-27T15:29:22.711 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2023-11-27T15:29:22.712 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:29:22.712 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2023-11-27T15:29:22.712 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:29:22.712 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2023-11-27T15:29:22.712 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 0, 2023-11-27T15:29:22.712 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 0, 2023-11-27T15:29:22.712 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:29:22.712 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:29:22.713 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2023-11-27T15:29:22.713 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2023-11-27T15:29:22.713 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 928, 2023-11-27T15:29:22.713 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 2, 2023-11-27T15:29:22.713 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2023-11-27T15:29:22.713 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2023-11-27T15:29:22.713 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2023-11-27T15:29:22.713 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2023-11-27T15:29:22.714 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2023-11-27T15:29:22.714 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:29:22.714 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2023-11-27T15:29:22.714 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2023-11-27T15:29:22.714 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2023-11-27T15:29:22.714 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 157, 2023-11-27T15:29:22.714 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 431906, 2023-11-27T15:29:22.714 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:22.715 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 157, 2023-11-27T15:29:22.715 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2023-11-27T15:29:22.715 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2023-11-27T15:29:22.715 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 157, 2023-11-27T15:29:22.715 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 431906, 2023-11-27T15:29:22.715 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2023-11-27T15:29:22.715 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2023-11-27T15:29:22.715 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2023-11-27T15:29:22.716 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2023-11-27T15:29:22.716 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:22.716 INFO:teuthology.orchestra.run.smithi174.stdout: } 2023-11-27T15:29:22.716 INFO:teuthology.orchestra.run.smithi174.stdout:} 2023-11-27T15:29:23.068 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-11-27T15:29:23.068 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-11-27T15:29:23.068 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2023-11-27T15:29:23.068 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2023-11-27T15:29:23.069 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2023-11-27T15:29:23.069 DEBUG:tasks.check_counter:Getting stats from c 2023-11-27T15:29:23.069 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.c perf dump 2023-11-27T15:29:23.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:22 smithi174 ceph-mon[131110]: pgmap v544: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2023-11-27T15:29:23.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:22 smithi133 ceph-mon[97860]: pgmap v544: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2023-11-27T15:29:23.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:22 smithi177 ceph-mon[131504]: pgmap v544: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2023-11-27T15:29:23.730 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:29:25.012 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2023-11-27T15:29:25.012 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:29:25.012 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 87, 2023-11-27T15:29:25.012 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 86, 2023-11-27T15:29:25.012 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 178272, 2023-11-27T15:29:25.012 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 29809, 2023-11-27T15:29:25.013 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 5, 2023-11-27T15:29:25.013 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 1, 2023-11-27T15:29:25.013 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.013002652, 2023-11-27T15:29:25.013 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.003089109, 2023-11-27T15:29:25.013 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.005720033, 2023-11-27T15:29:25.013 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.003231910, 2023-11-27T15:29:25.013 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:25.014 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 85, 2023-11-27T15:29:25.014 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.004963973, 2023-11-27T15:29:25.014 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000058399 2023-11-27T15:29:25.014 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.014 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:25.014 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.014 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.014 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.015 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 157952, 2023-11-27T15:29:25.015 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 11456 2023-11-27T15:29:25.015 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.015 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:29:25.015 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 80, 2023-11-27T15:29:25.015 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 92, 2023-11-27T15:29:25.015 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 204216, 2023-11-27T15:29:25.016 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 129679, 2023-11-27T15:29:25.016 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 5, 2023-11-27T15:29:25.016 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 4, 2023-11-27T15:29:25.016 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.008909138, 2023-11-27T15:29:25.016 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.003091351, 2023-11-27T15:29:25.016 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.003184634, 2023-11-27T15:29:25.016 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.001887257, 2023-11-27T15:29:25.016 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:25.017 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 91, 2023-11-27T15:29:25.017 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.007033492, 2023-11-27T15:29:25.017 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000077291 2023-11-27T15:29:25.017 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.017 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:25.017 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.017 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.017 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.018 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-11-27T15:29:25.018 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 106288 2023-11-27T15:29:25.018 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.018 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:29:25.018 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 61, 2023-11-27T15:29:25.018 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 63, 2023-11-27T15:29:25.018 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 15112, 2023-11-27T15:29:25.019 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 18978, 2023-11-27T15:29:25.019 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 4, 2023-11-27T15:29:25.019 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 3, 2023-11-27T15:29:25.019 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.007145789, 2023-11-27T15:29:25.019 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.002027708, 2023-11-27T15:29:25.019 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.002277811, 2023-11-27T15:29:25.019 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.002229317, 2023-11-27T15:29:25.019 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:25.020 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 61, 2023-11-27T15:29:25.020 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.004398808, 2023-11-27T15:29:25.020 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000072111 2023-11-27T15:29:25.020 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.020 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:25.020 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.020 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.020 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.021 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:29:25.021 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:29:25.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.021 INFO:teuthology.orchestra.run.smithi177.stdout: "cct": { 2023-11-27T15:29:25.021 INFO:teuthology.orchestra.run.smithi177.stdout: "total_workers": 1, 2023-11-27T15:29:25.021 INFO:teuthology.orchestra.run.smithi177.stdout: "unhealthy_workers": 0 2023-11-27T15:29:25.021 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.022 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-MDSRank": { 2023-11-27T15:29:25.022 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-11-27T15:29:25.022 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-11-27T15:29:25.022 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 317, 2023-11-27T15:29:25.022 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.011061649, 2023-11-27T15:29:25.022 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000034894 2023-11-27T15:29:25.022 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:25.022 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.023 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-PurgeQueue": { 2023-11-27T15:29:25.023 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-11-27T15:29:25.023 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-11-27T15:29:25.023 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.023 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.023 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.023 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:25.023 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.024 INFO:teuthology.orchestra.run.smithi177.stdout: "mds": { 2023-11-27T15:29:25.024 INFO:teuthology.orchestra.run.smithi177.stdout: "request": 0, 2023-11-27T15:29:25.024 INFO:teuthology.orchestra.run.smithi177.stdout: "reply": 0, 2023-11-27T15:29:25.024 INFO:teuthology.orchestra.run.smithi177.stdout: "reply_latency": { 2023-11-27T15:29:25.024 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.024 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.024 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.024 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.025 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_reply": 0, 2023-11-27T15:29:25.025 INFO:teuthology.orchestra.run.smithi177.stdout: "forward": 0, 2023-11-27T15:29:25.025 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_complete": 0, 2023-11-27T15:29:25.025 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_keys": 0, 2023-11-27T15:29:25.025 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_commit": 0, 2023-11-27T15:29:25.025 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_split": 0, 2023-11-27T15:29:25.025 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_merge": 0, 2023-11-27T15:29:25.025 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes": 17, 2023-11-27T15:29:25.025 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_top": 4, 2023-11-27T15:29:25.026 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_bottom": 5, 2023-11-27T15:29:25.026 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pin_tail": 8, 2023-11-27T15:29:25.026 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pinned": 10, 2023-11-27T15:29:25.026 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_expired": 2, 2023-11-27T15:29:25.026 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_with_caps": 0, 2023-11-27T15:29:25.026 INFO:teuthology.orchestra.run.smithi177.stdout: "caps": 0, 2023-11-27T15:29:25.026 INFO:teuthology.orchestra.run.smithi177.stdout: "subtrees": 2, 2023-11-27T15:29:25.026 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse": 0, 2023-11-27T15:29:25.027 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_hit": 0, 2023-11-27T15:29:25.027 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_forward": 0, 2023-11-27T15:29:25.027 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_discover": 0, 2023-11-27T15:29:25.027 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_dir_fetch": 0, 2023-11-27T15:29:25.027 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_remote_ino": 0, 2023-11-27T15:29:25.027 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_lock": 0, 2023-11-27T15:29:25.027 INFO:teuthology.orchestra.run.smithi177.stdout: "load_cent": 0, 2023-11-27T15:29:25.027 INFO:teuthology.orchestra.run.smithi177.stdout: "q": 0, 2023-11-27T15:29:25.028 INFO:teuthology.orchestra.run.smithi177.stdout: "exported": 0, 2023-11-27T15:29:25.028 INFO:teuthology.orchestra.run.smithi177.stdout: "exported_inodes": 0, 2023-11-27T15:29:25.028 INFO:teuthology.orchestra.run.smithi177.stdout: "imported": 0, 2023-11-27T15:29:25.028 INFO:teuthology.orchestra.run.smithi177.stdout: "imported_inodes": 0, 2023-11-27T15:29:25.028 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_dir_fetch": 0, 2023-11-27T15:29:25.028 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_backtrace_fetch": 0, 2023-11-27T15:29:25.028 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_peer_discover": 0, 2023-11-27T15:29:25.028 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rfiles": 4, 2023-11-27T15:29:25.029 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rbytes": 246, 2023-11-27T15:29:25.029 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rsnaps": 0, 2023-11-27T15:29:25.029 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_fetch": 0, 2023-11-27T15:29:25.029 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_set_tag": 0, 2023-11-27T15:29:25.029 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_repaired": 0, 2023-11-27T15:29:25.029 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_inotable_repaired": 0, 2023-11-27T15:29:25.029 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_inodes": 0, 2023-11-27T15:29:25.029 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_base_inodes": 0, 2023-11-27T15:29:25.029 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dirfrag_rstats": 0, 2023-11-27T15:29:25.030 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_file_inodes": 0, 2023-11-27T15:29:25.030 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_inode_file_caps": 0, 2023-11-27T15:29:25.030 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_revoke": 0, 2023-11-27T15:29:25.030 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_grant": 0, 2023-11-27T15:29:25.030 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_trunc": 0, 2023-11-27T15:29:25.030 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-27T15:29:25.030 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-27T15:29:25.030 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps": 0, 2023-11-27T15:29:25.031 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps_dirty": 0, 2023-11-27T15:29:25.031 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_cap_release": 0, 2023-11-27T15:29:25.031 INFO:teuthology.orchestra.run.smithi177.stdout: "process_request_cap_release": 0 2023-11-27T15:29:25.031 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.031 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache": { 2023-11-27T15:29:25.031 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update": 0, 2023-11-27T15:29:25.031 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update_receipt": 0, 2023-11-27T15:29:25.031 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_try_discover": 0, 2023-11-27T15:29:25.032 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_send_discover": 0, 2023-11-27T15:29:25.032 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_handle_discover": 0, 2023-11-27T15:29:25.032 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays": 0, 2023-11-27T15:29:25.032 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_delayed": 0, 2023-11-27T15:29:25.032 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_enqueuing": 0, 2023-11-27T15:29:25.032 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_created": 0, 2023-11-27T15:29:25.032 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_enqueued": 0, 2023-11-27T15:29:25.032 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_reintegrated": 0, 2023-11-27T15:29:25.033 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_migrated": 0, 2023-11-27T15:29:25.033 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_processing": 0, 2023-11-27T15:29:25.033 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_enqueued": 0, 2023-11-27T15:29:25.033 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_prioritized": 0, 2023-11-27T15:29:25.033 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_started": 0, 2023-11-27T15:29:25.033 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_completed": 0, 2023-11-27T15:29:25.033 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_enqueue_scrub": 0, 2023-11-27T15:29:25.033 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_exportdir": 0, 2023-11-27T15:29:25.034 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_flush": 0, 2023-11-27T15:29:25.034 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragmentdir": 0, 2023-11-27T15:29:25.034 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragstats": 0, 2023-11-27T15:29:25.034 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_inodestats": 0 2023-11-27T15:29:25.034 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.034 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log": { 2023-11-27T15:29:25.034 INFO:teuthology.orchestra.run.smithi177.stdout: "evadd": 0, 2023-11-27T15:29:25.034 INFO:teuthology.orchestra.run.smithi177.stdout: "evex": 0, 2023-11-27T15:29:25.035 INFO:teuthology.orchestra.run.smithi177.stdout: "evtrm": 0, 2023-11-27T15:29:25.035 INFO:teuthology.orchestra.run.smithi177.stdout: "ev": 0, 2023-11-27T15:29:25.035 INFO:teuthology.orchestra.run.smithi177.stdout: "evexg": 0, 2023-11-27T15:29:25.035 INFO:teuthology.orchestra.run.smithi177.stdout: "evexd": 0, 2023-11-27T15:29:25.035 INFO:teuthology.orchestra.run.smithi177.stdout: "segadd": 0, 2023-11-27T15:29:25.035 INFO:teuthology.orchestra.run.smithi177.stdout: "segex": 0, 2023-11-27T15:29:25.035 INFO:teuthology.orchestra.run.smithi177.stdout: "segtrm": 0, 2023-11-27T15:29:25.035 INFO:teuthology.orchestra.run.smithi177.stdout: "seg": 2, 2023-11-27T15:29:25.036 INFO:teuthology.orchestra.run.smithi177.stdout: "segexg": 0, 2023-11-27T15:29:25.036 INFO:teuthology.orchestra.run.smithi177.stdout: "segexd": 0, 2023-11-27T15:29:25.036 INFO:teuthology.orchestra.run.smithi177.stdout: "expos": 173644428, 2023-11-27T15:29:25.036 INFO:teuthology.orchestra.run.smithi177.stdout: "wrpos": 0, 2023-11-27T15:29:25.036 INFO:teuthology.orchestra.run.smithi177.stdout: "rdpos": 173666818, 2023-11-27T15:29:25.036 INFO:teuthology.orchestra.run.smithi177.stdout: "jlat": { 2023-11-27T15:29:25.036 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.036 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.036 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.037 INFO:teuthology.orchestra.run.smithi177.stdout: "replayed": 76 2023-11-27T15:29:25.037 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.037 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_mem": { 2023-11-27T15:29:25.037 INFO:teuthology.orchestra.run.smithi177.stdout: "ino": 12, 2023-11-27T15:29:25.037 INFO:teuthology.orchestra.run.smithi177.stdout: "ino+": 21, 2023-11-27T15:29:25.037 INFO:teuthology.orchestra.run.smithi177.stdout: "ino-": 9, 2023-11-27T15:29:25.037 INFO:teuthology.orchestra.run.smithi177.stdout: "dir": 11, 2023-11-27T15:29:25.038 INFO:teuthology.orchestra.run.smithi177.stdout: "dir+": 17, 2023-11-27T15:29:25.038 INFO:teuthology.orchestra.run.smithi177.stdout: "dir-": 6, 2023-11-27T15:29:25.038 INFO:teuthology.orchestra.run.smithi177.stdout: "dn": 17, 2023-11-27T15:29:25.038 INFO:teuthology.orchestra.run.smithi177.stdout: "dn+": 25, 2023-11-27T15:29:25.038 INFO:teuthology.orchestra.run.smithi177.stdout: "dn-": 8, 2023-11-27T15:29:25.038 INFO:teuthology.orchestra.run.smithi177.stdout: "cap": 0, 2023-11-27T15:29:25.038 INFO:teuthology.orchestra.run.smithi177.stdout: "cap+": 0, 2023-11-27T15:29:25.038 INFO:teuthology.orchestra.run.smithi177.stdout: "cap-": 0, 2023-11-27T15:29:25.038 INFO:teuthology.orchestra.run.smithi177.stdout: "rss": 41644, 2023-11-27T15:29:25.039 INFO:teuthology.orchestra.run.smithi177.stdout: "heap": 207156 2023-11-27T15:29:25.039 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.039 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_server": { 2023-11-27T15:29:25.039 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_client_request": 0, 2023-11-27T15:29:25.039 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_server_request": 0, 2023-11-27T15:29:25.039 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_request": 0, 2023-11-27T15:29:25.039 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_session": 0, 2023-11-27T15:29:25.039 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_peer_request": 0, 2023-11-27T15:29:25.040 INFO:teuthology.orchestra.run.smithi177.stdout: "req_create_latency": { 2023-11-27T15:29:25.040 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.040 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.040 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.040 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.040 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getattr_latency": { 2023-11-27T15:29:25.040 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.040 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.040 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.041 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getfilelock_latency": { 2023-11-27T15:29:25.041 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.041 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.041 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.041 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.041 INFO:teuthology.orchestra.run.smithi177.stdout: "req_link_latency": { 2023-11-27T15:29:25.041 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.041 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.042 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.042 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookup_latency": { 2023-11-27T15:29:25.042 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.042 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.042 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.042 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.042 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookuphash_latency": { 2023-11-27T15:29:25.043 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.043 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.043 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.043 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupino_latency": { 2023-11-27T15:29:25.043 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.043 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.043 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.043 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.044 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupname_latency": { 2023-11-27T15:29:25.044 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.044 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.044 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.044 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.044 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupparent_latency": { 2023-11-27T15:29:25.044 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.044 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.045 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.045 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupsnap_latency": { 2023-11-27T15:29:25.045 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.045 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.045 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.045 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.045 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lssnap_latency": { 2023-11-27T15:29:25.045 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.046 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.046 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.046 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mkdir_latency": { 2023-11-27T15:29:25.046 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.046 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.046 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.046 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.047 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mknod_latency": { 2023-11-27T15:29:25.047 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.047 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.047 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.047 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.047 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mksnap_latency": { 2023-11-27T15:29:25.047 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.047 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.047 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.048 INFO:teuthology.orchestra.run.smithi177.stdout: "req_open_latency": { 2023-11-27T15:29:25.048 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.048 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.048 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.048 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.048 INFO:teuthology.orchestra.run.smithi177.stdout: "req_readdir_latency": { 2023-11-27T15:29:25.048 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.048 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.049 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.049 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rename_latency": { 2023-11-27T15:29:25.049 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.049 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.049 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.049 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.049 INFO:teuthology.orchestra.run.smithi177.stdout: "req_renamesnap_latency": { 2023-11-27T15:29:25.050 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.050 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.050 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.050 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.050 INFO:teuthology.orchestra.run.smithi177.stdout: "req_snapdiff_latency": { 2023-11-27T15:29:25.050 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.050 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.050 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.051 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmdir_latency": { 2023-11-27T15:29:25.051 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.051 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.051 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.051 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.051 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmsnap_latency": { 2023-11-27T15:29:25.051 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.052 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.052 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.052 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmxattr_latency": { 2023-11-27T15:29:25.052 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.052 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.052 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.052 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.053 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setattr_latency": { 2023-11-27T15:29:25.053 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.053 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.053 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.053 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.053 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setdirlayout_latency": { 2023-11-27T15:29:25.053 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.053 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.054 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.054 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setfilelock_latency": { 2023-11-27T15:29:25.054 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.054 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.054 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.054 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.054 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setlayout_latency": { 2023-11-27T15:29:25.055 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.055 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.055 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.055 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.055 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setxattr_latency": { 2023-11-27T15:29:25.055 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.055 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.055 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.056 INFO:teuthology.orchestra.run.smithi177.stdout: "req_symlink_latency": { 2023-11-27T15:29:25.056 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.056 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.056 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.056 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.056 INFO:teuthology.orchestra.run.smithi177.stdout: "req_unlink_latency": { 2023-11-27T15:29:25.056 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.056 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.057 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.057 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.057 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_revoke_eviction": 0, 2023-11-27T15:29:25.057 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_acquisition_throttle": 0, 2023-11-27T15:29:25.057 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getvxattr_latency": { 2023-11-27T15:29:25.057 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.057 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.057 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.058 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:25.058 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.058 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_sessions": { 2023-11-27T15:29:25.058 INFO:teuthology.orchestra.run.smithi177.stdout: "session_count": 1, 2023-11-27T15:29:25.058 INFO:teuthology.orchestra.run.smithi177.stdout: "session_add": 1, 2023-11-27T15:29:25.058 INFO:teuthology.orchestra.run.smithi177.stdout: "session_remove": 0, 2023-11-27T15:29:25.058 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_open": 0, 2023-11-27T15:29:25.058 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_stale": 0, 2023-11-27T15:29:25.059 INFO:teuthology.orchestra.run.smithi177.stdout: "total_load": 0, 2023-11-27T15:29:25.059 INFO:teuthology.orchestra.run.smithi177.stdout: "average_load": 0, 2023-11-27T15:29:25.059 INFO:teuthology.orchestra.run.smithi177.stdout: "avg_session_uptime": 2327 2023-11-27T15:29:25.059 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.059 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool": { 2023-11-27T15:29:25.059 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:29:25.059 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_items": 0, 2023-11-27T15:29:25.059 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:29:25.059 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:29:25.060 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:29:25.060 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:29:25.060 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:29:25.060 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:29:25.060 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:29:25.060 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:29:25.060 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:29:25.060 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:29:25.061 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:29:25.061 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:29:25.061 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:29:25.061 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_items": 0, 2023-11-27T15:29:25.061 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:29:25.061 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_items": 0, 2023-11-27T15:29:25.061 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:29:25.061 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:29:25.061 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:29:25.062 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:29:25.062 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:29:25.062 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:29:25.062 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:29:25.062 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_items": 0, 2023-11-27T15:29:25.062 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:29:25.062 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:29:25.062 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:29:25.063 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_items": 0, 2023-11-27T15:29:25.063 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_bytes": 0, 2023-11-27T15:29:25.063 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_items": 0, 2023-11-27T15:29:25.063 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:29:25.063 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:29:25.063 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:29:25.063 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:29:25.063 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_bytes": 188541, 2023-11-27T15:29:25.064 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_items": 48, 2023-11-27T15:29:25.064 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:29:25.064 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_items": 0, 2023-11-27T15:29:25.064 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bytes": 0, 2023-11-27T15:29:25.064 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_items": 0, 2023-11-27T15:29:25.064 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:29:25.064 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_items": 0, 2023-11-27T15:29:25.064 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:29:25.065 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_items": 0, 2023-11-27T15:29:25.065 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_bytes": 8888, 2023-11-27T15:29:25.065 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_items": 170, 2023-11-27T15:29:25.065 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:29:25.065 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:29:25.065 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_bytes": 0, 2023-11-27T15:29:25.065 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_items": 0, 2023-11-27T15:29:25.065 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_bytes": 53272, 2023-11-27T15:29:25.065 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_items": 272, 2023-11-27T15:29:25.066 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_bytes": 0, 2023-11-27T15:29:25.066 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_items": 0, 2023-11-27T15:29:25.066 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_bytes": 0, 2023-11-27T15:29:25.066 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_items": 0 2023-11-27T15:29:25.066 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.066 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter": { 2023-11-27T15:29:25.066 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2023-11-27T15:29:25.066 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2023-11-27T15:29:25.067 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 195, 2023-11-27T15:29:25.067 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 26, 2023-11-27T15:29:25.067 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 4, 2023-11-27T15:29:25.067 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 195, 2023-11-27T15:29:25.067 INFO:teuthology.orchestra.run.smithi177.stdout: "op_latency": { 2023-11-27T15:29:25.067 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 195, 2023-11-27T15:29:25.067 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.111996888, 2023-11-27T15:29:25.067 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000574343 2023-11-27T15:29:25.067 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.068 INFO:teuthology.orchestra.run.smithi177.stdout: "op_inflight": 0, 2023-11-27T15:29:25.068 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2023-11-27T15:29:25.068 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 195, 2023-11-27T15:29:25.068 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 196 2023-11-27T15:29:25.068 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.068 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 195, 2023-11-27T15:29:25.068 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 195, 2023-11-27T15:29:25.068 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 0, 2023-11-27T15:29:25.068 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2023-11-27T15:29:25.069 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2023-11-27T15:29:25.069 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 116, 2023-11-27T15:29:25.069 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 0, 2023-11-27T15:29:25.069 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 76, 2023-11-27T15:29:25.069 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 0, 2023-11-27T15:29:25.069 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 0, 2023-11-27T15:29:25.069 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2023-11-27T15:29:25.069 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2023-11-27T15:29:25.070 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2023-11-27T15:29:25.070 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2023-11-27T15:29:25.070 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 0, 2023-11-27T15:29:25.070 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2023-11-27T15:29:25.070 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2023-11-27T15:29:25.070 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2023-11-27T15:29:25.070 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 2, 2023-11-27T15:29:25.070 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 0, 2023-11-27T15:29:25.070 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2023-11-27T15:29:25.071 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2023-11-27T15:29:25.071 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2023-11-27T15:29:25.071 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2023-11-27T15:29:25.071 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2023-11-27T15:29:25.071 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2023-11-27T15:29:25.071 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2023-11-27T15:29:25.071 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2023-11-27T15:29:25.071 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2023-11-27T15:29:25.072 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 0, 2023-11-27T15:29:25.072 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2023-11-27T15:29:25.072 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2023-11-27T15:29:25.072 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2023-11-27T15:29:25.072 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2023-11-27T15:29:25.072 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2023-11-27T15:29:25.072 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2023-11-27T15:29:25.072 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2023-11-27T15:29:25.072 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2023-11-27T15:29:25.073 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2023-11-27T15:29:25.073 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2023-11-27T15:29:25.073 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2023-11-27T15:29:25.073 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2023-11-27T15:29:25.073 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2023-11-27T15:29:25.073 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2023-11-27T15:29:25.073 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2023-11-27T15:29:25.073 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2023-11-27T15:29:25.074 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 0, 2023-11-27T15:29:25.074 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2023-11-27T15:29:25.074 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 0, 2023-11-27T15:29:25.074 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 5, 2023-11-27T15:29:25.074 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 5, 2023-11-27T15:29:25.074 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 0, 2023-11-27T15:29:25.074 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2023-11-27T15:29:25.074 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 0, 2023-11-27T15:29:25.075 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 2, 2023-11-27T15:29:25.075 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 0 2023-11-27T15:29:25.075 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.075 INFO:teuthology.orchestra.run.smithi177.stdout: "oft": { 2023-11-27T15:29:25.075 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_objs": 0, 2023-11-27T15:29:25.075 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_kv_pairs": 0, 2023-11-27T15:29:25.075 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_updates": 0, 2023-11-27T15:29:25.075 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_removes": 0 2023-11-27T15:29:25.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.076 INFO:teuthology.orchestra.run.smithi177.stdout: "osd-slow-ops": { 2023-11-27T15:29:25.076 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_ops_count": 0 2023-11-27T15:29:25.076 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.076 INFO:teuthology.orchestra.run.smithi177.stdout: "purge_queue": { 2023-11-27T15:29:25.076 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops": 0, 2023-11-27T15:29:25.076 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops_high_water": 0, 2023-11-27T15:29:25.076 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing": 0, 2023-11-27T15:29:25.077 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_high_water": 0, 2023-11-27T15:29:25.077 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed": 0, 2023-11-27T15:29:25.077 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_item_in_journal": 0 2023-11-27T15:29:25.077 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.077 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:29:25.077 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-11-27T15:29:25.077 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-11-27T15:29:25.077 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-11-27T15:29:25.078 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 224, 2023-11-27T15:29:25.078 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 373026, 2023-11-27T15:29:25.078 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:25.078 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 224, 2023-11-27T15:29:25.078 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-11-27T15:29:25.078 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-11-27T15:29:25.078 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 224, 2023-11-27T15:29:25.078 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 373026, 2023-11-27T15:29:25.079 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-11-27T15:29:25.079 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.079 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.079 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.079 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:25.079 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.079 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes": { 2023-11-27T15:29:25.079 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-11-27T15:29:25.080 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-11-27T15:29:25.080 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-11-27T15:29:25.080 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-11-27T15:29:25.080 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-11-27T15:29:25.080 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:25.080 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-11-27T15:29:25.080 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 195, 2023-11-27T15:29:25.080 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 179414, 2023-11-27T15:29:25.081 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 195, 2023-11-27T15:29:25.081 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 179414, 2023-11-27T15:29:25.081 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-11-27T15:29:25.081 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.081 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.081 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.081 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:25.081 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.082 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops": { 2023-11-27T15:29:25.082 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-11-27T15:29:25.082 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2023-11-27T15:29:25.082 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-11-27T15:29:25.082 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-11-27T15:29:25.082 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-11-27T15:29:25.082 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:25.083 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-11-27T15:29:25.083 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 195, 2023-11-27T15:29:25.083 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 195, 2023-11-27T15:29:25.083 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 195, 2023-11-27T15:29:25.083 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 195, 2023-11-27T15:29:25.083 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-11-27T15:29:25.083 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.083 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.084 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.084 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:25.084 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.084 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle": { 2023-11-27T15:29:25.084 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-11-27T15:29:25.084 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2023-11-27T15:29:25.084 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-11-27T15:29:25.084 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-11-27T15:29:25.085 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-11-27T15:29:25.085 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:25.085 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-11-27T15:29:25.085 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-11-27T15:29:25.085 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-11-27T15:29:25.085 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2023-11-27T15:29:25.085 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2023-11-27T15:29:25.085 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-11-27T15:29:25.085 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.086 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.086 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.086 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:25.086 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:25.086 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle-0x561b195fa460": { 2023-11-27T15:29:25.086 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-11-27T15:29:25.086 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2023-11-27T15:29:25.086 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-11-27T15:29:25.087 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-11-27T15:29:25.087 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-11-27T15:29:25.087 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:25.087 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-11-27T15:29:25.087 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-11-27T15:29:25.087 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-11-27T15:29:25.087 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2023-11-27T15:29:25.087 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2023-11-27T15:29:25.088 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-11-27T15:29:25.088 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:25.088 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:25.088 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:25.088 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:25.088 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:25.088 INFO:teuthology.orchestra.run.smithi177.stdout:} 2023-11-27T15:29:25.089 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:24 smithi177 ceph-mon[131504]: pgmap v545: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 7 op/s 2023-11-27T15:29:25.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:24 smithi133 ceph-mon[97860]: pgmap v545: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 7 op/s 2023-11-27T15:29:25.385 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-11-27T15:29:25.385 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-11-27T15:29:25.386 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2023-11-27T15:29:25.386 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2023-11-27T15:29:25.386 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2023-11-27T15:29:25.386 DEBUG:tasks.check_counter:Getting stats from f 2023-11-27T15:29:25.386 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.f perf dump 2023-11-27T15:29:25.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:24 smithi174 ceph-mon[131110]: pgmap v545: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 767 B/s wr, 7 op/s 2023-11-27T15:29:26.026 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:29:27.240 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:26 smithi177 ceph-mon[131504]: pgmap v546: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:29:27.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:26 smithi133 ceph-mon[97860]: pgmap v546: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:29:27.392 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2023-11-27T15:29:27.392 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:29:27.392 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 828, 2023-11-27T15:29:27.392 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 818, 2023-11-27T15:29:27.393 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 35254969, 2023-11-27T15:29:27.393 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 291678, 2023-11-27T15:29:27.393 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 7, 2023-11-27T15:29:27.393 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 3, 2023-11-27T15:29:27.393 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.108703044, 2023-11-27T15:29:27.393 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.026929250, 2023-11-27T15:29:27.393 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.046705474, 2023-11-27T15:29:27.393 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.029006323, 2023-11-27T15:29:27.394 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:27.394 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 817, 2023-11-27T15:29:27.394 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.042436843, 2023-11-27T15:29:27.394 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000051942 2023-11-27T15:29:27.394 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.394 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:27.394 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.394 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.395 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.395 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 496896, 2023-11-27T15:29:27.395 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 87040 2023-11-27T15:29:27.395 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.395 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:29:27.395 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 291, 2023-11-27T15:29:27.395 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 402, 2023-11-27T15:29:27.396 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 71386997, 2023-11-27T15:29:27.396 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 349296, 2023-11-27T15:29:27.396 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 7, 2023-11-27T15:29:27.396 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 6, 2023-11-27T15:29:27.396 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.059637944, 2023-11-27T15:29:27.396 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.008821319, 2023-11-27T15:29:27.396 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.041330994, 2023-11-27T15:29:27.396 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.006632149, 2023-11-27T15:29:27.397 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:27.397 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 401, 2023-11-27T15:29:27.397 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.022424678, 2023-11-27T15:29:27.397 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000055921 2023-11-27T15:29:27.397 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.397 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:27.397 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.397 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.398 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.398 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-11-27T15:29:27.398 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 260288 2023-11-27T15:29:27.398 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.398 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:29:27.398 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 877, 2023-11-27T15:29:27.398 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 879, 2023-11-27T15:29:27.398 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 100899592, 2023-11-27T15:29:27.399 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 272754, 2023-11-27T15:29:27.399 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 6, 2023-11-27T15:29:27.399 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 5, 2023-11-27T15:29:27.399 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.106173571, 2023-11-27T15:29:27.399 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.020499713, 2023-11-27T15:29:27.399 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.057439265, 2023-11-27T15:29:27.399 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.022756016, 2023-11-27T15:29:27.399 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:27.400 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 877, 2023-11-27T15:29:27.400 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.036188517, 2023-11-27T15:29:27.400 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000041263 2023-11-27T15:29:27.400 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.400 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:27.400 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.400 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.400 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.401 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:29:27.401 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:29:27.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.401 INFO:teuthology.orchestra.run.smithi177.stdout: "cct": { 2023-11-27T15:29:27.401 INFO:teuthology.orchestra.run.smithi177.stdout: "total_workers": 1, 2023-11-27T15:29:27.401 INFO:teuthology.orchestra.run.smithi177.stdout: "unhealthy_workers": 0 2023-11-27T15:29:27.401 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.401 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-MDSRank": { 2023-11-27T15:29:27.402 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-11-27T15:29:27.402 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-11-27T15:29:27.402 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2952, 2023-11-27T15:29:27.402 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.110031094, 2023-11-27T15:29:27.402 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000037273 2023-11-27T15:29:27.402 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:27.402 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.402 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-PurgeQueue": { 2023-11-27T15:29:27.403 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-11-27T15:29:27.403 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-11-27T15:29:27.403 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.403 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.403 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.403 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:27.403 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.403 INFO:teuthology.orchestra.run.smithi177.stdout: "mds": { 2023-11-27T15:29:27.404 INFO:teuthology.orchestra.run.smithi177.stdout: "request": 0, 2023-11-27T15:29:27.404 INFO:teuthology.orchestra.run.smithi177.stdout: "reply": 0, 2023-11-27T15:29:27.404 INFO:teuthology.orchestra.run.smithi177.stdout: "reply_latency": { 2023-11-27T15:29:27.404 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.404 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.404 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.404 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.404 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_reply": 0, 2023-11-27T15:29:27.405 INFO:teuthology.orchestra.run.smithi177.stdout: "forward": 0, 2023-11-27T15:29:27.405 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_complete": 0, 2023-11-27T15:29:27.405 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_keys": 0, 2023-11-27T15:29:27.405 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_commit": 0, 2023-11-27T15:29:27.405 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_split": 0, 2023-11-27T15:29:27.405 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_merge": 0, 2023-11-27T15:29:27.405 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes": 27092, 2023-11-27T15:29:27.405 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_top": 0, 2023-11-27T15:29:27.405 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_bottom": 0, 2023-11-27T15:29:27.406 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pin_tail": 27092, 2023-11-27T15:29:27.406 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pinned": 27092, 2023-11-27T15:29:27.406 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_expired": 4738, 2023-11-27T15:29:27.406 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_with_caps": 0, 2023-11-27T15:29:27.406 INFO:teuthology.orchestra.run.smithi177.stdout: "caps": 0, 2023-11-27T15:29:27.406 INFO:teuthology.orchestra.run.smithi177.stdout: "subtrees": 1, 2023-11-27T15:29:27.406 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse": 0, 2023-11-27T15:29:27.406 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_hit": 0, 2023-11-27T15:29:27.407 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_forward": 0, 2023-11-27T15:29:27.407 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_discover": 0, 2023-11-27T15:29:27.407 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_dir_fetch": 0, 2023-11-27T15:29:27.407 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_remote_ino": 0, 2023-11-27T15:29:27.407 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_lock": 0, 2023-11-27T15:29:27.407 INFO:teuthology.orchestra.run.smithi177.stdout: "load_cent": 0, 2023-11-27T15:29:27.407 INFO:teuthology.orchestra.run.smithi177.stdout: "q": 0, 2023-11-27T15:29:27.407 INFO:teuthology.orchestra.run.smithi177.stdout: "exported": 0, 2023-11-27T15:29:27.408 INFO:teuthology.orchestra.run.smithi177.stdout: "exported_inodes": 0, 2023-11-27T15:29:27.408 INFO:teuthology.orchestra.run.smithi177.stdout: "imported": 0, 2023-11-27T15:29:27.408 INFO:teuthology.orchestra.run.smithi177.stdout: "imported_inodes": 0, 2023-11-27T15:29:27.408 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_dir_fetch": 0, 2023-11-27T15:29:27.408 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_backtrace_fetch": 0, 2023-11-27T15:29:27.408 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_peer_discover": 0, 2023-11-27T15:29:27.408 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rfiles": 0, 2023-11-27T15:29:27.408 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rbytes": 0, 2023-11-27T15:29:27.409 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rsnaps": 0, 2023-11-27T15:29:27.409 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_fetch": 0, 2023-11-27T15:29:27.409 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_set_tag": 0, 2023-11-27T15:29:27.409 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_repaired": 0, 2023-11-27T15:29:27.409 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_inotable_repaired": 0, 2023-11-27T15:29:27.409 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_inodes": 0, 2023-11-27T15:29:27.409 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_base_inodes": 0, 2023-11-27T15:29:27.409 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dirfrag_rstats": 0, 2023-11-27T15:29:27.410 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_file_inodes": 0, 2023-11-27T15:29:27.410 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_inode_file_caps": 0, 2023-11-27T15:29:27.410 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_revoke": 0, 2023-11-27T15:29:27.410 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_grant": 0, 2023-11-27T15:29:27.410 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_trunc": 0, 2023-11-27T15:29:27.410 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-27T15:29:27.410 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-27T15:29:27.410 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps": 0, 2023-11-27T15:29:27.411 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps_dirty": 0, 2023-11-27T15:29:27.411 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_cap_release": 0, 2023-11-27T15:29:27.411 INFO:teuthology.orchestra.run.smithi177.stdout: "process_request_cap_release": 0 2023-11-27T15:29:27.411 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.411 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache": { 2023-11-27T15:29:27.411 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update": 0, 2023-11-27T15:29:27.411 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update_receipt": 0, 2023-11-27T15:29:27.411 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_try_discover": 0, 2023-11-27T15:29:27.411 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_send_discover": 0, 2023-11-27T15:29:27.412 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_handle_discover": 0, 2023-11-27T15:29:27.412 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays": 0, 2023-11-27T15:29:27.412 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_delayed": 0, 2023-11-27T15:29:27.412 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_enqueuing": 0, 2023-11-27T15:29:27.412 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_created": 0, 2023-11-27T15:29:27.412 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_enqueued": 0, 2023-11-27T15:29:27.412 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_reintegrated": 0, 2023-11-27T15:29:27.412 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_migrated": 0, 2023-11-27T15:29:27.413 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_processing": 0, 2023-11-27T15:29:27.413 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_enqueued": 0, 2023-11-27T15:29:27.413 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_prioritized": 0, 2023-11-27T15:29:27.413 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_started": 0, 2023-11-27T15:29:27.413 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_completed": 0, 2023-11-27T15:29:27.413 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_enqueue_scrub": 0, 2023-11-27T15:29:27.413 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_exportdir": 0, 2023-11-27T15:29:27.413 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_flush": 0, 2023-11-27T15:29:27.414 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragmentdir": 0, 2023-11-27T15:29:27.414 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragstats": 0, 2023-11-27T15:29:27.414 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_inodestats": 0 2023-11-27T15:29:27.414 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.414 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log": { 2023-11-27T15:29:27.414 INFO:teuthology.orchestra.run.smithi177.stdout: "evadd": 0, 2023-11-27T15:29:27.414 INFO:teuthology.orchestra.run.smithi177.stdout: "evex": 0, 2023-11-27T15:29:27.414 INFO:teuthology.orchestra.run.smithi177.stdout: "evtrm": 0, 2023-11-27T15:29:27.415 INFO:teuthology.orchestra.run.smithi177.stdout: "ev": 0, 2023-11-27T15:29:27.415 INFO:teuthology.orchestra.run.smithi177.stdout: "evexg": 0, 2023-11-27T15:29:27.415 INFO:teuthology.orchestra.run.smithi177.stdout: "evexd": 0, 2023-11-27T15:29:27.415 INFO:teuthology.orchestra.run.smithi177.stdout: "segadd": 0, 2023-11-27T15:29:27.415 INFO:teuthology.orchestra.run.smithi177.stdout: "segex": 0, 2023-11-27T15:29:27.415 INFO:teuthology.orchestra.run.smithi177.stdout: "segtrm": 0, 2023-11-27T15:29:27.415 INFO:teuthology.orchestra.run.smithi177.stdout: "seg": 94, 2023-11-27T15:29:27.415 INFO:teuthology.orchestra.run.smithi177.stdout: "segexg": 0, 2023-11-27T15:29:27.415 INFO:teuthology.orchestra.run.smithi177.stdout: "segexd": 0, 2023-11-27T15:29:27.416 INFO:teuthology.orchestra.run.smithi177.stdout: "expos": 4194304, 2023-11-27T15:29:27.416 INFO:teuthology.orchestra.run.smithi177.stdout: "wrpos": 0, 2023-11-27T15:29:27.416 INFO:teuthology.orchestra.run.smithi177.stdout: "rdpos": 210707320, 2023-11-27T15:29:27.416 INFO:teuthology.orchestra.run.smithi177.stdout: "jlat": { 2023-11-27T15:29:27.416 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.416 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.416 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.416 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.417 INFO:teuthology.orchestra.run.smithi177.stdout: "replayed": 75332 2023-11-27T15:29:27.417 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.417 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_mem": { 2023-11-27T15:29:27.417 INFO:teuthology.orchestra.run.smithi177.stdout: "ino": 15, 2023-11-27T15:29:27.417 INFO:teuthology.orchestra.run.smithi177.stdout: "ino+": 29986, 2023-11-27T15:29:27.417 INFO:teuthology.orchestra.run.smithi177.stdout: "ino-": 29971, 2023-11-27T15:29:27.417 INFO:teuthology.orchestra.run.smithi177.stdout: "dir": 17, 2023-11-27T15:29:27.417 INFO:teuthology.orchestra.run.smithi177.stdout: "dir+": 1848, 2023-11-27T15:29:27.418 INFO:teuthology.orchestra.run.smithi177.stdout: "dir-": 1831, 2023-11-27T15:29:27.418 INFO:teuthology.orchestra.run.smithi177.stdout: "dn": 27092, 2023-11-27T15:29:27.418 INFO:teuthology.orchestra.run.smithi177.stdout: "dn+": 61361, 2023-11-27T15:29:27.418 INFO:teuthology.orchestra.run.smithi177.stdout: "dn-": 34269, 2023-11-27T15:29:27.418 INFO:teuthology.orchestra.run.smithi177.stdout: "cap": 0, 2023-11-27T15:29:27.418 INFO:teuthology.orchestra.run.smithi177.stdout: "cap+": 0, 2023-11-27T15:29:27.418 INFO:teuthology.orchestra.run.smithi177.stdout: "cap-": 0, 2023-11-27T15:29:27.418 INFO:teuthology.orchestra.run.smithi177.stdout: "rss": 117268, 2023-11-27T15:29:27.419 INFO:teuthology.orchestra.run.smithi177.stdout: "heap": 207156 2023-11-27T15:29:27.419 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.419 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_server": { 2023-11-27T15:29:27.419 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_client_request": 0, 2023-11-27T15:29:27.419 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_server_request": 0, 2023-11-27T15:29:27.419 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_request": 0, 2023-11-27T15:29:27.419 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_session": 0, 2023-11-27T15:29:27.419 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_peer_request": 0, 2023-11-27T15:29:27.419 INFO:teuthology.orchestra.run.smithi177.stdout: "req_create_latency": { 2023-11-27T15:29:27.420 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.420 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.420 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.420 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.420 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getattr_latency": { 2023-11-27T15:29:27.420 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.420 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.420 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.421 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.421 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getfilelock_latency": { 2023-11-27T15:29:27.421 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.421 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.421 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.421 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.421 INFO:teuthology.orchestra.run.smithi177.stdout: "req_link_latency": { 2023-11-27T15:29:27.421 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.422 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.422 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.422 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.422 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookup_latency": { 2023-11-27T15:29:27.422 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.422 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.422 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.422 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.423 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookuphash_latency": { 2023-11-27T15:29:27.423 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.423 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.423 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.423 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.423 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupino_latency": { 2023-11-27T15:29:27.423 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.423 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.423 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.424 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupname_latency": { 2023-11-27T15:29:27.424 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.424 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.424 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.424 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.424 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupparent_latency": { 2023-11-27T15:29:27.424 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.425 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.425 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.425 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupsnap_latency": { 2023-11-27T15:29:27.425 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.425 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.425 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.425 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.426 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lssnap_latency": { 2023-11-27T15:29:27.426 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.426 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.426 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.426 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.426 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mkdir_latency": { 2023-11-27T15:29:27.426 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.426 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.427 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.427 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mknod_latency": { 2023-11-27T15:29:27.427 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.427 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.427 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.427 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.427 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mksnap_latency": { 2023-11-27T15:29:27.428 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.428 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.428 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.428 INFO:teuthology.orchestra.run.smithi177.stdout: "req_open_latency": { 2023-11-27T15:29:27.428 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.428 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.428 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.428 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.429 INFO:teuthology.orchestra.run.smithi177.stdout: "req_readdir_latency": { 2023-11-27T15:29:27.429 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.429 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.429 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.429 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.429 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rename_latency": { 2023-11-27T15:29:27.429 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.429 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.430 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.430 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.430 INFO:teuthology.orchestra.run.smithi177.stdout: "req_renamesnap_latency": { 2023-11-27T15:29:27.430 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.430 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.430 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.430 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.430 INFO:teuthology.orchestra.run.smithi177.stdout: "req_snapdiff_latency": { 2023-11-27T15:29:27.431 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.431 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.431 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.431 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.431 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmdir_latency": { 2023-11-27T15:29:27.431 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.431 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.431 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.432 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.432 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmsnap_latency": { 2023-11-27T15:29:27.432 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.432 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.432 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.432 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.432 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmxattr_latency": { 2023-11-27T15:29:27.432 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.432 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.433 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.433 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.433 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setattr_latency": { 2023-11-27T15:29:27.433 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.433 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.433 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.433 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.433 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setdirlayout_latency": { 2023-11-27T15:29:27.434 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.434 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.434 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.434 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.434 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setfilelock_latency": { 2023-11-27T15:29:27.434 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.434 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.434 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.435 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.435 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setlayout_latency": { 2023-11-27T15:29:27.435 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.435 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.435 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.435 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.435 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setxattr_latency": { 2023-11-27T15:29:27.435 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.436 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.436 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.436 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.436 INFO:teuthology.orchestra.run.smithi177.stdout: "req_symlink_latency": { 2023-11-27T15:29:27.436 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.436 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.436 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.436 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.437 INFO:teuthology.orchestra.run.smithi177.stdout: "req_unlink_latency": { 2023-11-27T15:29:27.437 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.437 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.437 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.437 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.437 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_revoke_eviction": 0, 2023-11-27T15:29:27.437 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_acquisition_throttle": 0, 2023-11-27T15:29:27.437 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getvxattr_latency": { 2023-11-27T15:29:27.437 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.438 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.438 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.438 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:27.438 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.438 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_sessions": { 2023-11-27T15:29:27.438 INFO:teuthology.orchestra.run.smithi177.stdout: "session_count": 1, 2023-11-27T15:29:27.438 INFO:teuthology.orchestra.run.smithi177.stdout: "session_add": 1, 2023-11-27T15:29:27.438 INFO:teuthology.orchestra.run.smithi177.stdout: "session_remove": 0, 2023-11-27T15:29:27.439 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_open": 1, 2023-11-27T15:29:27.439 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_stale": 0, 2023-11-27T15:29:27.439 INFO:teuthology.orchestra.run.smithi177.stdout: "total_load": 0, 2023-11-27T15:29:27.439 INFO:teuthology.orchestra.run.smithi177.stdout: "average_load": 0, 2023-11-27T15:29:27.439 INFO:teuthology.orchestra.run.smithi177.stdout: "avg_session_uptime": 2332 2023-11-27T15:29:27.439 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.439 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool": { 2023-11-27T15:29:27.439 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:29:27.440 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_items": 0, 2023-11-27T15:29:27.440 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:29:27.440 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:29:27.440 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:29:27.440 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:29:27.440 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:29:27.440 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:29:27.440 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:29:27.441 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:29:27.441 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:29:27.441 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:29:27.441 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:29:27.441 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:29:27.441 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:29:27.441 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_items": 0, 2023-11-27T15:29:27.441 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:29:27.442 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_items": 0, 2023-11-27T15:29:27.442 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:29:27.442 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:29:27.442 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:29:27.442 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:29:27.442 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:29:27.442 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:29:27.442 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:29:27.442 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_items": 0, 2023-11-27T15:29:27.443 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:29:27.443 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:29:27.443 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:29:27.443 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_items": 0, 2023-11-27T15:29:27.443 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_bytes": 0, 2023-11-27T15:29:27.443 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_items": 0, 2023-11-27T15:29:27.443 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:29:27.443 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:29:27.444 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:29:27.444 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:29:27.444 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_bytes": 239556, 2023-11-27T15:29:27.444 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_items": 82, 2023-11-27T15:29:27.444 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:29:27.444 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_items": 0, 2023-11-27T15:29:27.444 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bytes": 0, 2023-11-27T15:29:27.444 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_items": 0, 2023-11-27T15:29:27.445 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:29:27.445 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_items": 0, 2023-11-27T15:29:27.445 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:29:27.445 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_items": 0, 2023-11-27T15:29:27.445 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_bytes": 8888, 2023-11-27T15:29:27.445 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_items": 170, 2023-11-27T15:29:27.445 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:29:27.445 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:29:27.445 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_bytes": 0, 2023-11-27T15:29:27.446 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_items": 0, 2023-11-27T15:29:27.446 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_bytes": 19071815, 2023-11-27T15:29:27.446 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_items": 101664, 2023-11-27T15:29:27.446 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_bytes": 0, 2023-11-27T15:29:27.446 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_items": 0, 2023-11-27T15:29:27.446 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_bytes": 0, 2023-11-27T15:29:27.446 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_items": 0 2023-11-27T15:29:27.446 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.447 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter": { 2023-11-27T15:29:27.447 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2023-11-27T15:29:27.447 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2023-11-27T15:29:27.447 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 1821, 2023-11-27T15:29:27.447 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 21, 2023-11-27T15:29:27.447 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 3, 2023-11-27T15:29:27.447 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 1821, 2023-11-27T15:29:27.447 INFO:teuthology.orchestra.run.smithi177.stdout: "op_latency": { 2023-11-27T15:29:27.448 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1821, 2023-11-27T15:29:27.448 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 2.375934058, 2023-11-27T15:29:27.448 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.001304741 2023-11-27T15:29:27.448 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.448 INFO:teuthology.orchestra.run.smithi177.stdout: "op_inflight": 0, 2023-11-27T15:29:27.448 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2023-11-27T15:29:27.448 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1821, 2023-11-27T15:29:27.448 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 1822 2023-11-27T15:29:27.449 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.449 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 1821, 2023-11-27T15:29:27.449 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 1821, 2023-11-27T15:29:27.449 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 0, 2023-11-27T15:29:27.449 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2023-11-27T15:29:27.449 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2023-11-27T15:29:27.449 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 1108, 2023-11-27T15:29:27.449 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 0, 2023-11-27T15:29:27.450 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 711, 2023-11-27T15:29:27.450 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 0, 2023-11-27T15:29:27.450 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 0, 2023-11-27T15:29:27.450 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2023-11-27T15:29:27.450 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2023-11-27T15:29:27.450 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2023-11-27T15:29:27.450 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2023-11-27T15:29:27.450 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 0, 2023-11-27T15:29:27.450 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2023-11-27T15:29:27.451 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2023-11-27T15:29:27.451 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2023-11-27T15:29:27.451 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 1, 2023-11-27T15:29:27.451 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 0, 2023-11-27T15:29:27.451 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2023-11-27T15:29:27.451 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2023-11-27T15:29:27.451 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2023-11-27T15:29:27.451 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2023-11-27T15:29:27.452 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2023-11-27T15:29:27.452 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2023-11-27T15:29:27.452 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2023-11-27T15:29:27.452 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2023-11-27T15:29:27.452 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2023-11-27T15:29:27.452 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 0, 2023-11-27T15:29:27.452 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2023-11-27T15:29:27.452 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2023-11-27T15:29:27.453 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2023-11-27T15:29:27.453 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2023-11-27T15:29:27.453 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2023-11-27T15:29:27.453 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2023-11-27T15:29:27.453 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2023-11-27T15:29:27.453 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2023-11-27T15:29:27.453 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2023-11-27T15:29:27.453 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2023-11-27T15:29:27.454 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2023-11-27T15:29:27.454 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2023-11-27T15:29:27.454 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2023-11-27T15:29:27.454 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2023-11-27T15:29:27.454 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2023-11-27T15:29:27.454 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2023-11-27T15:29:27.454 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 94, 2023-11-27T15:29:27.454 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2023-11-27T15:29:27.455 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 4, 2023-11-27T15:29:27.455 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 11, 2023-11-27T15:29:27.455 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 11, 2023-11-27T15:29:27.455 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 0, 2023-11-27T15:29:27.455 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2023-11-27T15:29:27.455 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 0, 2023-11-27T15:29:27.455 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 2, 2023-11-27T15:29:27.455 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 0 2023-11-27T15:29:27.455 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.456 INFO:teuthology.orchestra.run.smithi177.stdout: "oft": { 2023-11-27T15:29:27.456 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_objs": 0, 2023-11-27T15:29:27.456 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_kv_pairs": 0, 2023-11-27T15:29:27.456 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_updates": 0, 2023-11-27T15:29:27.456 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_removes": 0 2023-11-27T15:29:27.456 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.456 INFO:teuthology.orchestra.run.smithi177.stdout: "osd-slow-ops": { 2023-11-27T15:29:27.456 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_ops_count": 0 2023-11-27T15:29:27.457 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.457 INFO:teuthology.orchestra.run.smithi177.stdout: "purge_queue": { 2023-11-27T15:29:27.457 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops": 0, 2023-11-27T15:29:27.457 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops_high_water": 0, 2023-11-27T15:29:27.457 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing": 0, 2023-11-27T15:29:27.457 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_high_water": 0, 2023-11-27T15:29:27.457 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed": 0, 2023-11-27T15:29:27.457 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_item_in_journal": 0 2023-11-27T15:29:27.458 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.458 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:29:27.458 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-11-27T15:29:27.458 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-11-27T15:29:27.458 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-11-27T15:29:27.458 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 1992, 2023-11-27T15:29:27.458 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 207350609, 2023-11-27T15:29:27.458 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:27.459 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 1992, 2023-11-27T15:29:27.459 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-11-27T15:29:27.459 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-11-27T15:29:27.459 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 1992, 2023-11-27T15:29:27.459 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 207350609, 2023-11-27T15:29:27.459 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-11-27T15:29:27.459 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.459 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.459 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.460 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:27.460 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.460 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes": { 2023-11-27T15:29:27.460 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-11-27T15:29:27.460 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-11-27T15:29:27.460 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-11-27T15:29:27.460 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-11-27T15:29:27.460 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-11-27T15:29:27.461 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:27.461 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-11-27T15:29:27.461 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 1821, 2023-11-27T15:29:27.461 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 206534180, 2023-11-27T15:29:27.461 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 1821, 2023-11-27T15:29:27.461 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 206534180, 2023-11-27T15:29:27.461 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-11-27T15:29:27.461 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.462 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.462 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.462 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:27.462 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.462 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops": { 2023-11-27T15:29:27.462 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-11-27T15:29:27.462 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2023-11-27T15:29:27.462 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-11-27T15:29:27.463 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-11-27T15:29:27.463 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-11-27T15:29:27.463 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:27.463 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-11-27T15:29:27.463 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 1821, 2023-11-27T15:29:27.463 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 1821, 2023-11-27T15:29:27.463 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 1821, 2023-11-27T15:29:27.463 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 1821, 2023-11-27T15:29:27.463 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-11-27T15:29:27.464 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.464 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.464 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.464 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:27.464 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.464 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle": { 2023-11-27T15:29:27.464 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-11-27T15:29:27.464 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2023-11-27T15:29:27.465 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-11-27T15:29:27.465 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-11-27T15:29:27.465 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-11-27T15:29:27.465 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:27.465 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-11-27T15:29:27.465 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-11-27T15:29:27.465 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-11-27T15:29:27.465 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2023-11-27T15:29:27.466 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2023-11-27T15:29:27.466 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-11-27T15:29:27.466 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.466 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.466 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.466 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:27.466 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:27.466 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle-0x559d821b4460": { 2023-11-27T15:29:27.466 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-11-27T15:29:27.467 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2023-11-27T15:29:27.467 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-11-27T15:29:27.467 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-11-27T15:29:27.467 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-11-27T15:29:27.467 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:27.467 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-11-27T15:29:27.467 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-11-27T15:29:27.467 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-11-27T15:29:27.468 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 0, 2023-11-27T15:29:27.468 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 0, 2023-11-27T15:29:27.468 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-11-27T15:29:27.468 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:27.468 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:27.468 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:27.468 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:27.468 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:27.469 INFO:teuthology.orchestra.run.smithi177.stdout:} 2023-11-27T15:29:27.469 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:26 smithi174 ceph-mon[131110]: pgmap v546: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:29:27.782 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-11-27T15:29:27.783 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-11-27T15:29:27.783 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2023-11-27T15:29:27.783 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2023-11-27T15:29:27.783 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2023-11-27T15:29:27.783 DEBUG:tasks.check_counter:Getting stats from i 2023-11-27T15:29:27.783 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.i perf dump 2023-11-27T15:29:28.423 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:29:29.059 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:28 smithi177 ceph-mon[131504]: pgmap v547: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 8 op/s 2023-11-27T15:29:29.152 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:28 smithi174 ceph-mon[131110]: pgmap v547: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 8 op/s 2023-11-27T15:29:29.276 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:28 smithi133 ceph-mon[97860]: pgmap v547: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 8 op/s 2023-11-27T15:29:29.701 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2023-11-27T15:29:29.701 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:29:29.701 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 103883, 2023-11-27T15:29:29.701 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 175493, 2023-11-27T15:29:29.702 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 42342133, 2023-11-27T15:29:29.702 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 202462795, 2023-11-27T15:29:29.702 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 16, 2023-11-27T15:29:29.702 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 4, 2023-11-27T15:29:29.702 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 6.960078545, 2023-11-27T15:29:29.702 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 2.992905700, 2023-11-27T15:29:29.702 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 3.736224412, 2023-11-27T15:29:29.703 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.930583922, 2023-11-27T15:29:29.703 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:29.703 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 175492, 2023-11-27T15:29:29.703 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 14.704209232, 2023-11-27T15:29:29.703 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000083788 2023-11-27T15:29:29.703 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.703 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:29.703 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 171643, 2023-11-27T15:29:29.704 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.026330337, 2023-11-27T15:29:29.704 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000153 2023-11-27T15:29:29.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.704 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 29862352, 2023-11-27T15:29:29.704 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 95585840 2023-11-27T15:29:29.704 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.704 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:29:29.704 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 27140, 2023-11-27T15:29:29.705 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 27131, 2023-11-27T15:29:29.705 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 11359673, 2023-11-27T15:29:29.705 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 95377429, 2023-11-27T15:29:29.705 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 16, 2023-11-27T15:29:29.705 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 7, 2023-11-27T15:29:29.705 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 1.976623560, 2023-11-27T15:29:29.705 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.650967184, 2023-11-27T15:29:29.706 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.408290106, 2023-11-27T15:29:29.706 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.811599180, 2023-11-27T15:29:29.706 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:29.706 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 27128, 2023-11-27T15:29:29.706 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 3.010770349, 2023-11-27T15:29:29.706 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000110983 2023-11-27T15:29:29.706 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.706 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:29.707 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 8, 2023-11-27T15:29:29.707 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000823, 2023-11-27T15:29:29.707 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000102 2023-11-27T15:29:29.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.707 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 508608, 2023-11-27T15:29:29.707 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 101184 2023-11-27T15:29:29.707 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.707 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:29:29.707 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 182701, 2023-11-27T15:29:29.708 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 191142, 2023-11-27T15:29:29.708 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 96319989, 2023-11-27T15:29:29.708 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 76616124, 2023-11-27T15:29:29.708 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 10, 2023-11-27T15:29:29.708 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 5, 2023-11-27T15:29:29.708 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 7.143077850, 2023-11-27T15:29:29.708 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 2.629998269, 2023-11-27T15:29:29.709 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 4.881492434, 2023-11-27T15:29:29.709 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.468426997, 2023-11-27T15:29:29.709 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:29.709 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 191142, 2023-11-27T15:29:29.709 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 8.827442791, 2023-11-27T15:29:29.709 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000046182 2023-11-27T15:29:29.709 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.709 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:29.710 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 337330, 2023-11-27T15:29:29.710 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.039152206, 2023-11-27T15:29:29.710 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000116 2023-11-27T15:29:29.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.710 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 160, 2023-11-27T15:29:29.710 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 262976 2023-11-27T15:29:29.710 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.710 INFO:teuthology.orchestra.run.smithi177.stdout: "cct": { 2023-11-27T15:29:29.711 INFO:teuthology.orchestra.run.smithi177.stdout: "total_workers": 1, 2023-11-27T15:29:29.711 INFO:teuthology.orchestra.run.smithi177.stdout: "unhealthy_workers": 0 2023-11-27T15:29:29.711 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.711 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-MDSRank": { 2023-11-27T15:29:29.711 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-11-27T15:29:29.711 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-11-27T15:29:29.711 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 9451, 2023-11-27T15:29:29.711 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 65.218098097, 2023-11-27T15:29:29.712 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.006900655 2023-11-27T15:29:29.712 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:29.712 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.712 INFO:teuthology.orchestra.run.smithi177.stdout: "finisher-PurgeQueue": { 2023-11-27T15:29:29.712 INFO:teuthology.orchestra.run.smithi177.stdout: "queue_len": 0, 2023-11-27T15:29:29.712 INFO:teuthology.orchestra.run.smithi177.stdout: "complete_latency": { 2023-11-27T15:29:29.712 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 43960, 2023-11-27T15:29:29.712 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 19.104909193, 2023-11-27T15:29:29.713 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000434597 2023-11-27T15:29:29.713 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:29.713 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.713 INFO:teuthology.orchestra.run.smithi177.stdout: "mds": { 2023-11-27T15:29:29.713 INFO:teuthology.orchestra.run.smithi177.stdout: "request": 61723, 2023-11-27T15:29:29.713 INFO:teuthology.orchestra.run.smithi177.stdout: "reply": 56589, 2023-11-27T15:29:29.713 INFO:teuthology.orchestra.run.smithi177.stdout: "reply_latency": { 2023-11-27T15:29:29.713 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 56589, 2023-11-27T15:29:29.713 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 186.532027754, 2023-11-27T15:29:29.714 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.003296259 2023-11-27T15:29:29.714 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.714 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_reply": 0, 2023-11-27T15:29:29.714 INFO:teuthology.orchestra.run.smithi177.stdout: "forward": 5134, 2023-11-27T15:29:29.714 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_complete": 922, 2023-11-27T15:29:29.714 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_fetch_keys": 0, 2023-11-27T15:29:29.714 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_commit": 381, 2023-11-27T15:29:29.714 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_split": 42, 2023-11-27T15:29:29.715 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_merge": 34, 2023-11-27T15:29:29.715 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes": 19, 2023-11-27T15:29:29.715 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_top": 2, 2023-11-27T15:29:29.715 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_bottom": 7, 2023-11-27T15:29:29.715 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pin_tail": 10, 2023-11-27T15:29:29.715 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_pinned": 15, 2023-11-27T15:29:29.715 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_expired": 27325, 2023-11-27T15:29:29.715 INFO:teuthology.orchestra.run.smithi177.stdout: "inodes_with_caps": 1, 2023-11-27T15:29:29.715 INFO:teuthology.orchestra.run.smithi177.stdout: "caps": 1, 2023-11-27T15:29:29.716 INFO:teuthology.orchestra.run.smithi177.stdout: "subtrees": 4, 2023-11-27T15:29:29.716 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse": 89749, 2023-11-27T15:29:29.716 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_hit": 83155, 2023-11-27T15:29:29.716 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_forward": 5130, 2023-11-27T15:29:29.716 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_discover": 412, 2023-11-27T15:29:29.716 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_dir_fetch": 549, 2023-11-27T15:29:29.716 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_remote_ino": 0, 2023-11-27T15:29:29.716 INFO:teuthology.orchestra.run.smithi177.stdout: "traverse_lock": 0, 2023-11-27T15:29:29.717 INFO:teuthology.orchestra.run.smithi177.stdout: "load_cent": 0, 2023-11-27T15:29:29.717 INFO:teuthology.orchestra.run.smithi177.stdout: "q": 0, 2023-11-27T15:29:29.717 INFO:teuthology.orchestra.run.smithi177.stdout: "exported": 1009, 2023-11-27T15:29:29.717 INFO:teuthology.orchestra.run.smithi177.stdout: "exported_inodes": 24231, 2023-11-27T15:29:29.717 INFO:teuthology.orchestra.run.smithi177.stdout: "imported": 1120, 2023-11-27T15:29:29.717 INFO:teuthology.orchestra.run.smithi177.stdout: "imported_inodes": 0, 2023-11-27T15:29:29.717 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_dir_fetch": 0, 2023-11-27T15:29:29.717 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_backtrace_fetch": 11901, 2023-11-27T15:29:29.718 INFO:teuthology.orchestra.run.smithi177.stdout: "openino_peer_discover": 0, 2023-11-27T15:29:29.718 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rfiles": 20148, 2023-11-27T15:29:29.718 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rbytes": 246, 2023-11-27T15:29:29.718 INFO:teuthology.orchestra.run.smithi177.stdout: "root_rsnaps": 0, 2023-11-27T15:29:29.718 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_fetch": 11901, 2023-11-27T15:29:29.718 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_set_tag": 0, 2023-11-27T15:29:29.718 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_backtrace_repaired": 0, 2023-11-27T15:29:29.718 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_inotable_repaired": 0, 2023-11-27T15:29:29.719 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_inodes": 3, 2023-11-27T15:29:29.719 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dir_base_inodes": 0, 2023-11-27T15:29:29.719 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_dirfrag_rstats": 3, 2023-11-27T15:29:29.719 INFO:teuthology.orchestra.run.smithi177.stdout: "scrub_file_inodes": 11898, 2023-11-27T15:29:29.719 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_inode_file_caps": 4, 2023-11-27T15:29:29.719 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_revoke": 843, 2023-11-27T15:29:29.719 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_grant": 30611, 2023-11-27T15:29:29.719 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_trunc": 0, 2023-11-27T15:29:29.720 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-11-27T15:29:29.720 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_cap_op_flush_ack": 0, 2023-11-27T15:29:29.720 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps": 13680, 2023-11-27T15:29:29.720 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_caps_dirty": 12834, 2023-11-27T15:29:29.720 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_cap_release": 364, 2023-11-27T15:29:29.720 INFO:teuthology.orchestra.run.smithi177.stdout: "process_request_cap_release": 51594 2023-11-27T15:29:29.720 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.720 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_cache": { 2023-11-27T15:29:29.721 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update": 336, 2023-11-27T15:29:29.721 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_update_receipt": 3165, 2023-11-27T15:29:29.721 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_try_discover": 1585, 2023-11-27T15:29:29.721 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_send_discover": 2101, 2023-11-27T15:29:29.721 INFO:teuthology.orchestra.run.smithi177.stdout: "dir_handle_discover": 336, 2023-11-27T15:29:29.721 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays": 0, 2023-11-27T15:29:29.721 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_delayed": 0, 2023-11-27T15:29:29.721 INFO:teuthology.orchestra.run.smithi177.stdout: "num_strays_enqueuing": 0, 2023-11-27T15:29:29.721 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_created": 25371, 2023-11-27T15:29:29.722 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_enqueued": 25371, 2023-11-27T15:29:29.722 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_reintegrated": 0, 2023-11-27T15:29:29.722 INFO:teuthology.orchestra.run.smithi177.stdout: "strays_migrated": 0, 2023-11-27T15:29:29.722 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_processing": 0, 2023-11-27T15:29:29.722 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_enqueued": 0, 2023-11-27T15:29:29.722 INFO:teuthology.orchestra.run.smithi177.stdout: "num_recovering_prioritized": 0, 2023-11-27T15:29:29.722 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_started": 0, 2023-11-27T15:29:29.722 INFO:teuthology.orchestra.run.smithi177.stdout: "recovery_completed": 0, 2023-11-27T15:29:29.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_enqueue_scrub": 0, 2023-11-27T15:29:29.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_exportdir": 1041, 2023-11-27T15:29:29.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_flush": 0, 2023-11-27T15:29:29.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragmentdir": 76, 2023-11-27T15:29:29.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_fragstats": 0, 2023-11-27T15:29:29.723 INFO:teuthology.orchestra.run.smithi177.stdout: "ireq_inodestats": 0 2023-11-27T15:29:29.723 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.723 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_log": { 2023-11-27T15:29:29.723 INFO:teuthology.orchestra.run.smithi177.stdout: "evadd": 91419, 2023-11-27T15:29:29.724 INFO:teuthology.orchestra.run.smithi177.stdout: "evex": 0, 2023-11-27T15:29:29.724 INFO:teuthology.orchestra.run.smithi177.stdout: "evtrm": 0, 2023-11-27T15:29:29.724 INFO:teuthology.orchestra.run.smithi177.stdout: "ev": 91419, 2023-11-27T15:29:29.724 INFO:teuthology.orchestra.run.smithi177.stdout: "evexg": 0, 2023-11-27T15:29:29.724 INFO:teuthology.orchestra.run.smithi177.stdout: "evexd": 0, 2023-11-27T15:29:29.724 INFO:teuthology.orchestra.run.smithi177.stdout: "segadd": 125, 2023-11-27T15:29:29.724 INFO:teuthology.orchestra.run.smithi177.stdout: "segex": 0, 2023-11-27T15:29:29.724 INFO:teuthology.orchestra.run.smithi177.stdout: "segtrm": 0, 2023-11-27T15:29:29.725 INFO:teuthology.orchestra.run.smithi177.stdout: "seg": 125, 2023-11-27T15:29:29.725 INFO:teuthology.orchestra.run.smithi177.stdout: "segexg": 0, 2023-11-27T15:29:29.725 INFO:teuthology.orchestra.run.smithi177.stdout: "segexd": 0, 2023-11-27T15:29:29.725 INFO:teuthology.orchestra.run.smithi177.stdout: "expos": 4194304, 2023-11-27T15:29:29.725 INFO:teuthology.orchestra.run.smithi177.stdout: "wrpos": 237241319, 2023-11-27T15:29:29.725 INFO:teuthology.orchestra.run.smithi177.stdout: "rdpos": 0, 2023-11-27T15:29:29.725 INFO:teuthology.orchestra.run.smithi177.stdout: "jlat": { 2023-11-27T15:29:29.725 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 3153, 2023-11-27T15:29:29.725 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 140.490664905, 2023-11-27T15:29:29.726 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.044557775 2023-11-27T15:29:29.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.726 INFO:teuthology.orchestra.run.smithi177.stdout: "replayed": 0 2023-11-27T15:29:29.726 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.726 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_mem": { 2023-11-27T15:29:29.726 INFO:teuthology.orchestra.run.smithi177.stdout: "ino": 24, 2023-11-27T15:29:29.726 INFO:teuthology.orchestra.run.smithi177.stdout: "ino+": 30063, 2023-11-27T15:29:29.726 INFO:teuthology.orchestra.run.smithi177.stdout: "ino-": 30039, 2023-11-27T15:29:29.727 INFO:teuthology.orchestra.run.smithi177.stdout: "dir": 22, 2023-11-27T15:29:29.727 INFO:teuthology.orchestra.run.smithi177.stdout: "dir+": 2803, 2023-11-27T15:29:29.727 INFO:teuthology.orchestra.run.smithi177.stdout: "dir-": 2781, 2023-11-27T15:29:29.727 INFO:teuthology.orchestra.run.smithi177.stdout: "dn": 19, 2023-11-27T15:29:29.727 INFO:teuthology.orchestra.run.smithi177.stdout: "dn+": 73838, 2023-11-27T15:29:29.727 INFO:teuthology.orchestra.run.smithi177.stdout: "dn-": 73819, 2023-11-27T15:29:29.727 INFO:teuthology.orchestra.run.smithi177.stdout: "cap": 1, 2023-11-27T15:29:29.727 INFO:teuthology.orchestra.run.smithi177.stdout: "cap+": 45840, 2023-11-27T15:29:29.727 INFO:teuthology.orchestra.run.smithi177.stdout: "cap-": 45839, 2023-11-27T15:29:29.728 INFO:teuthology.orchestra.run.smithi177.stdout: "rss": 134724, 2023-11-27T15:29:29.728 INFO:teuthology.orchestra.run.smithi177.stdout: "heap": 198964 2023-11-27T15:29:29.728 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.728 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_server": { 2023-11-27T15:29:29.728 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_client_request": 63779, 2023-11-27T15:29:29.728 INFO:teuthology.orchestra.run.smithi177.stdout: "dispatch_server_request": 29, 2023-11-27T15:29:29.728 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_request": 61723, 2023-11-27T15:29:29.729 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_client_session": 10, 2023-11-27T15:29:29.729 INFO:teuthology.orchestra.run.smithi177.stdout: "handle_peer_request": 464, 2023-11-27T15:29:29.729 INFO:teuthology.orchestra.run.smithi177.stdout: "req_create_latency": { 2023-11-27T15:29:29.729 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 5222, 2023-11-27T15:29:29.729 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 11.281652524, 2023-11-27T15:29:29.729 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.002160408 2023-11-27T15:29:29.729 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.729 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getattr_latency": { 2023-11-27T15:29:29.729 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 5, 2023-11-27T15:29:29.730 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.149931185, 2023-11-27T15:29:29.730 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.029986237 2023-11-27T15:29:29.730 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.730 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getfilelock_latency": { 2023-11-27T15:29:29.730 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.730 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.730 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.730 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.731 INFO:teuthology.orchestra.run.smithi177.stdout: "req_link_latency": { 2023-11-27T15:29:29.731 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.731 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.731 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.731 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.731 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookup_latency": { 2023-11-27T15:29:29.731 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 24978, 2023-11-27T15:29:29.731 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 72.044152529, 2023-11-27T15:29:29.732 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.002884304 2023-11-27T15:29:29.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.732 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookuphash_latency": { 2023-11-27T15:29:29.732 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.732 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.732 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.732 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.732 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupino_latency": { 2023-11-27T15:29:29.732 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1, 2023-11-27T15:29:29.733 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000148543, 2023-11-27T15:29:29.733 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000148543 2023-11-27T15:29:29.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.733 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupname_latency": { 2023-11-27T15:29:29.733 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.733 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.733 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.733 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.734 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupparent_latency": { 2023-11-27T15:29:29.734 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.734 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.734 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.734 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.734 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lookupsnap_latency": { 2023-11-27T15:29:29.734 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.734 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.735 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.735 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.735 INFO:teuthology.orchestra.run.smithi177.stdout: "req_lssnap_latency": { 2023-11-27T15:29:29.735 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.735 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.735 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.735 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.735 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mkdir_latency": { 2023-11-27T15:29:29.735 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.736 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.736 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.736 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mknod_latency": { 2023-11-27T15:29:29.736 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.736 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.736 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.736 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.737 INFO:teuthology.orchestra.run.smithi177.stdout: "req_mksnap_latency": { 2023-11-27T15:29:29.737 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.737 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.737 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.737 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.737 INFO:teuthology.orchestra.run.smithi177.stdout: "req_open_latency": { 2023-11-27T15:29:29.737 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.737 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.738 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.738 INFO:teuthology.orchestra.run.smithi177.stdout: "req_readdir_latency": { 2023-11-27T15:29:29.738 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 1012, 2023-11-27T15:29:29.738 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 1.475390621, 2023-11-27T15:29:29.738 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.001457895 2023-11-27T15:29:29.738 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.738 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rename_latency": { 2023-11-27T15:29:29.739 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.739 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.739 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.739 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.739 INFO:teuthology.orchestra.run.smithi177.stdout: "req_renamesnap_latency": { 2023-11-27T15:29:29.739 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.739 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.739 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.739 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.740 INFO:teuthology.orchestra.run.smithi177.stdout: "req_snapdiff_latency": { 2023-11-27T15:29:29.740 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.740 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.740 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.740 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.740 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmdir_latency": { 2023-11-27T15:29:29.740 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 2, 2023-11-27T15:29:29.740 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.028315126, 2023-11-27T15:29:29.741 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.014157563 2023-11-27T15:29:29.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.741 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmsnap_latency": { 2023-11-27T15:29:29.741 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.741 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.741 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.741 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.741 INFO:teuthology.orchestra.run.smithi177.stdout: "req_rmxattr_latency": { 2023-11-27T15:29:29.741 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.742 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.742 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.742 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setattr_latency": { 2023-11-27T15:29:29.742 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.742 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.742 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.742 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.743 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setdirlayout_latency": { 2023-11-27T15:29:29.743 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.743 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.743 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.743 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.743 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setfilelock_latency": { 2023-11-27T15:29:29.743 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.743 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.743 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.744 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setlayout_latency": { 2023-11-27T15:29:29.744 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.744 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.744 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.744 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.744 INFO:teuthology.orchestra.run.smithi177.stdout: "req_setxattr_latency": { 2023-11-27T15:29:29.745 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.745 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.745 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.745 INFO:teuthology.orchestra.run.smithi177.stdout: "req_symlink_latency": { 2023-11-27T15:29:29.745 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.745 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.745 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.745 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.746 INFO:teuthology.orchestra.run.smithi177.stdout: "req_unlink_latency": { 2023-11-27T15:29:29.746 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 25369, 2023-11-27T15:29:29.746 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 101.552437226, 2023-11-27T15:29:29.746 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.004003013 2023-11-27T15:29:29.746 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.746 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_revoke_eviction": 0, 2023-11-27T15:29:29.746 INFO:teuthology.orchestra.run.smithi177.stdout: "cap_acquisition_throttle": 0, 2023-11-27T15:29:29.746 INFO:teuthology.orchestra.run.smithi177.stdout: "req_getvxattr_latency": { 2023-11-27T15:29:29.747 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.747 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.747 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.747 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:29.747 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.747 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_sessions": { 2023-11-27T15:29:29.747 INFO:teuthology.orchestra.run.smithi177.stdout: "session_count": 1, 2023-11-27T15:29:29.747 INFO:teuthology.orchestra.run.smithi177.stdout: "session_add": 2, 2023-11-27T15:29:29.748 INFO:teuthology.orchestra.run.smithi177.stdout: "session_remove": 1, 2023-11-27T15:29:29.748 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_open": 1, 2023-11-27T15:29:29.748 INFO:teuthology.orchestra.run.smithi177.stdout: "sessions_stale": 0, 2023-11-27T15:29:29.748 INFO:teuthology.orchestra.run.smithi177.stdout: "total_load": 30438, 2023-11-27T15:29:29.748 INFO:teuthology.orchestra.run.smithi177.stdout: "average_load": 30438, 2023-11-27T15:29:29.748 INFO:teuthology.orchestra.run.smithi177.stdout: "avg_session_uptime": 397 2023-11-27T15:29:29.748 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.748 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool": { 2023-11-27T15:29:29.748 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:29:29.749 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_items": 0, 2023-11-27T15:29:29.749 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:29:29.749 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:29:29.749 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:29:29.749 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:29:29.749 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:29:29.749 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:29:29.749 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:29:29.750 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:29:29.750 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:29:29.750 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:29:29.750 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:29:29.750 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:29:29.750 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:29:29.750 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_items": 0, 2023-11-27T15:29:29.750 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:29:29.751 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_items": 0, 2023-11-27T15:29:29.751 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:29:29.751 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:29:29.751 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:29:29.751 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:29:29.751 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:29:29.751 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:29:29.752 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:29:29.752 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_items": 0, 2023-11-27T15:29:29.752 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:29:29.752 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:29:29.752 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:29:29.752 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_items": 0, 2023-11-27T15:29:29.752 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_bytes": 0, 2023-11-27T15:29:29.752 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_items": 0, 2023-11-27T15:29:29.753 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:29:29.753 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:29:29.753 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:29:29.753 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:29:29.753 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_bytes": 292587, 2023-11-27T15:29:29.753 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_items": 137, 2023-11-27T15:29:29.753 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_bytes": 1760, 2023-11-27T15:29:29.753 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_items": 20, 2023-11-27T15:29:29.754 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bytes": 0, 2023-11-27T15:29:29.754 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_items": 0, 2023-11-27T15:29:29.754 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:29:29.754 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_items": 0, 2023-11-27T15:29:29.754 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:29:29.754 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_items": 0, 2023-11-27T15:29:29.754 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_bytes": 8888, 2023-11-27T15:29:29.754 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_items": 170, 2023-11-27T15:29:29.755 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:29:29.755 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:29:29.755 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_bytes": 0, 2023-11-27T15:29:29.755 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_items": 0, 2023-11-27T15:29:29.755 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_bytes": 96231, 2023-11-27T15:29:29.755 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_items": 541, 2023-11-27T15:29:29.755 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_bytes": 0, 2023-11-27T15:29:29.755 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_items": 0, 2023-11-27T15:29:29.756 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_bytes": 0, 2023-11-27T15:29:29.756 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_items": 0 2023-11-27T15:29:29.756 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.756 INFO:teuthology.orchestra.run.smithi177.stdout: "objecter": { 2023-11-27T15:29:29.756 INFO:teuthology.orchestra.run.smithi177.stdout: "op_active": 0, 2023-11-27T15:29:29.756 INFO:teuthology.orchestra.run.smithi177.stdout: "op_laggy": 0, 2023-11-27T15:29:29.756 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send": 68868, 2023-11-27T15:29:29.756 INFO:teuthology.orchestra.run.smithi177.stdout: "op_send_bytes": 239948719, 2023-11-27T15:29:29.757 INFO:teuthology.orchestra.run.smithi177.stdout: "op_resend": 27, 2023-11-27T15:29:29.757 INFO:teuthology.orchestra.run.smithi177.stdout: "op_reply": 68868, 2023-11-27T15:29:29.757 INFO:teuthology.orchestra.run.smithi177.stdout: "op_latency": { 2023-11-27T15:29:29.757 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 68868, 2023-11-27T15:29:29.757 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 500.210107234, 2023-11-27T15:29:29.757 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.007263316 2023-11-27T15:29:29.757 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.758 INFO:teuthology.orchestra.run.smithi177.stdout: "op_inflight": 0, 2023-11-27T15:29:29.758 INFO:teuthology.orchestra.run.smithi177.stdout: "oplen_avg": { 2023-11-27T15:29:29.758 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 68868, 2023-11-27T15:29:29.758 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 70217 2023-11-27T15:29:29.758 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.758 INFO:teuthology.orchestra.run.smithi177.stdout: "op": 68868, 2023-11-27T15:29:29.758 INFO:teuthology.orchestra.run.smithi177.stdout: "op_r": 13193, 2023-11-27T15:29:29.758 INFO:teuthology.orchestra.run.smithi177.stdout: "op_w": 55675, 2023-11-27T15:29:29.759 INFO:teuthology.orchestra.run.smithi177.stdout: "op_rmw": 0, 2023-11-27T15:29:29.759 INFO:teuthology.orchestra.run.smithi177.stdout: "op_pg": 0, 2023-11-27T15:29:29.759 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_stat": 0, 2023-11-27T15:29:29.759 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_create": 10, 2023-11-27T15:29:29.759 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_read": 370, 2023-11-27T15:29:29.759 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_write": 3523, 2023-11-27T15:29:29.759 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writefull": 135, 2023-11-27T15:29:29.759 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_writesame": 0, 2023-11-27T15:29:29.760 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_append": 0, 2023-11-27T15:29:29.760 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_zero": 0, 2023-11-27T15:29:29.760 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_truncate": 0, 2023-11-27T15:29:29.760 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_delete": 51586, 2023-11-27T15:29:29.760 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_mapext": 0, 2023-11-27T15:29:29.760 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_sparse_read": 0, 2023-11-27T15:29:29.760 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_clonerange": 0, 2023-11-27T15:29:29.760 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_getxattr": 11901, 2023-11-27T15:29:29.760 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_setxattr": 20, 2023-11-27T15:29:29.761 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_cmpxattr": 0, 2023-11-27T15:29:29.761 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_rmxattr": 0, 2023-11-27T15:29:29.761 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_resetxattrs": 0, 2023-11-27T15:29:29.761 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_call": 0, 2023-11-27T15:29:29.761 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_watch": 0, 2023-11-27T15:29:29.761 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_notify": 0, 2023-11-27T15:29:29.761 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_src_cmpxattr": 0, 2023-11-27T15:29:29.761 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls": 0, 2023-11-27T15:29:29.762 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_pgls_filter": 0, 2023-11-27T15:29:29.762 INFO:teuthology.orchestra.run.smithi177.stdout: "osdop_other": 0, 2023-11-27T15:29:29.762 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_active": 0, 2023-11-27T15:29:29.762 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_send": 0, 2023-11-27T15:29:29.762 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_resend": 0, 2023-11-27T15:29:29.762 INFO:teuthology.orchestra.run.smithi177.stdout: "linger_ping": 0, 2023-11-27T15:29:29.762 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_active": 0, 2023-11-27T15:29:29.763 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_send": 0, 2023-11-27T15:29:29.763 INFO:teuthology.orchestra.run.smithi177.stdout: "poolop_resend": 0, 2023-11-27T15:29:29.763 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_active": 0, 2023-11-27T15:29:29.763 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_send": 0, 2023-11-27T15:29:29.763 INFO:teuthology.orchestra.run.smithi177.stdout: "poolstat_resend": 0, 2023-11-27T15:29:29.763 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_active": 0, 2023-11-27T15:29:29.763 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_send": 0, 2023-11-27T15:29:29.763 INFO:teuthology.orchestra.run.smithi177.stdout: "statfs_resend": 0, 2023-11-27T15:29:29.764 INFO:teuthology.orchestra.run.smithi177.stdout: "command_active": 0, 2023-11-27T15:29:29.764 INFO:teuthology.orchestra.run.smithi177.stdout: "command_send": 0, 2023-11-27T15:29:29.764 INFO:teuthology.orchestra.run.smithi177.stdout: "command_resend": 0, 2023-11-27T15:29:29.764 INFO:teuthology.orchestra.run.smithi177.stdout: "map_epoch": 94, 2023-11-27T15:29:29.764 INFO:teuthology.orchestra.run.smithi177.stdout: "map_full": 0, 2023-11-27T15:29:29.764 INFO:teuthology.orchestra.run.smithi177.stdout: "map_inc": 4, 2023-11-27T15:29:29.764 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_sessions": 12, 2023-11-27T15:29:29.764 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_open": 12, 2023-11-27T15:29:29.764 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_session_close": 0, 2023-11-27T15:29:29.765 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_laggy": 0, 2023-11-27T15:29:29.765 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_wr": 795, 2023-11-27T15:29:29.765 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_rd": 1844, 2023-11-27T15:29:29.765 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_del": 33 2023-11-27T15:29:29.765 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.765 INFO:teuthology.orchestra.run.smithi177.stdout: "oft": { 2023-11-27T15:29:29.765 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_objs": 1, 2023-11-27T15:29:29.765 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_kv_pairs": 0, 2023-11-27T15:29:29.766 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_updates": 0, 2023-11-27T15:29:29.766 INFO:teuthology.orchestra.run.smithi177.stdout: "omap_total_removes": 0 2023-11-27T15:29:29.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.766 INFO:teuthology.orchestra.run.smithi177.stdout: "osd-slow-ops": { 2023-11-27T15:29:29.766 INFO:teuthology.orchestra.run.smithi177.stdout: "slow_ops_count": 62871 2023-11-27T15:29:29.766 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.766 INFO:teuthology.orchestra.run.smithi177.stdout: "purge_queue": { 2023-11-27T15:29:29.767 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops": 0, 2023-11-27T15:29:29.767 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_ops_high_water": 37, 2023-11-27T15:29:29.767 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing": 0, 2023-11-27T15:29:29.767 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executing_high_water": 8, 2023-11-27T15:29:29.767 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_executed": 25371, 2023-11-27T15:29:29.767 INFO:teuthology.orchestra.run.smithi177.stdout: "pq_item_in_journal": 0 2023-11-27T15:29:29.767 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.767 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:29:29.768 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-11-27T15:29:29.768 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-11-27T15:29:29.768 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-11-27T15:29:29.768 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 313720, 2023-11-27T15:29:29.768 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 118125028, 2023-11-27T15:29:29.768 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:29.768 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 313720, 2023-11-27T15:29:29.768 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-11-27T15:29:29.769 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-11-27T15:29:29.769 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 313720, 2023-11-27T15:29:29.769 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 118125028, 2023-11-27T15:29:29.769 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-11-27T15:29:29.769 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.769 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.769 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.769 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:29.770 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.770 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_bytes": { 2023-11-27T15:29:29.770 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-11-27T15:29:29.770 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-11-27T15:29:29.770 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-11-27T15:29:29.770 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-11-27T15:29:29.770 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-11-27T15:29:29.770 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:29.771 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-11-27T15:29:29.771 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 68868, 2023-11-27T15:29:29.771 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 242699494, 2023-11-27T15:29:29.771 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 68868, 2023-11-27T15:29:29.771 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 242699494, 2023-11-27T15:29:29.771 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-11-27T15:29:29.771 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.772 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.772 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.772 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:29.772 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.772 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-objecter_ops": { 2023-11-27T15:29:29.772 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-11-27T15:29:29.772 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 1024, 2023-11-27T15:29:29.772 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-11-27T15:29:29.772 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 0, 2023-11-27T15:29:29.773 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 0, 2023-11-27T15:29:29.773 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:29.773 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 0, 2023-11-27T15:29:29.773 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 68868, 2023-11-27T15:29:29.773 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 68868, 2023-11-27T15:29:29.773 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 68868, 2023-11-27T15:29:29.773 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 68868, 2023-11-27T15:29:29.773 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-11-27T15:29:29.774 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.774 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.774 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.774 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:29.774 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.774 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle": { 2023-11-27T15:29:29.774 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-11-27T15:29:29.775 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2023-11-27T15:29:29.775 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-11-27T15:29:29.775 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 25371, 2023-11-27T15:29:29.775 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 2765527, 2023-11-27T15:29:29.775 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:29.775 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 25371, 2023-11-27T15:29:29.775 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-11-27T15:29:29.775 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-11-27T15:29:29.776 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 370, 2023-11-27T15:29:29.776 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 2765527, 2023-11-27T15:29:29.776 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-11-27T15:29:29.776 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.776 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.776 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.776 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:29.776 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:29.777 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-write_buf_throttle-0x558d771ba0a0": { 2023-11-27T15:29:29.777 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-11-27T15:29:29.777 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 3758096384, 2023-11-27T15:29:29.777 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-11-27T15:29:29.777 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 91419, 2023-11-27T15:29:29.777 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 233047015, 2023-11-27T15:29:29.777 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:29.777 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 91419, 2023-11-27T15:29:29.778 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-11-27T15:29:29.778 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-11-27T15:29:29.778 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 3153, 2023-11-27T15:29:29.778 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 233047015, 2023-11-27T15:29:29.778 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-11-27T15:29:29.778 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:29.778 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:29.779 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:29.779 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:29.779 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:29.779 INFO:teuthology.orchestra.run.smithi177.stdout:} 2023-11-27T15:29:30.064 INFO:tasks.check_counter:Daemon mds.i mds.exported=1009 2023-11-27T15:29:30.064 INFO:tasks.check_counter:Daemon mds.i mds.imported=1120 2023-11-27T15:29:30.065 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=336 2023-11-27T15:29:30.065 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=3165 2023-11-27T15:29:30.065 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=42 2023-11-27T15:29:30.065 DEBUG:tasks.check_counter:Getting stats from l 2023-11-27T15:29:30.065 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1c9565146c8647d4e00eb6c7ae626dc192680e7b shell --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 -- ceph daemon mds.l perf dump 2023-11-27T15:29:30.709 INFO:teuthology.orchestra.run.smithi177.stderr:Inferring config /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/mon.c/config 2023-11-27T15:29:31.200 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:30 smithi177 ceph-mon[131504]: pgmap v548: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:29:31.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:30 smithi133 ceph-mon[97860]: pgmap v548: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:29:31.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:30 smithi174 ceph-mon[131110]: pgmap v548: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:29:31.977 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2023-11-27T15:29:31.978 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-0": { 2023-11-27T15:29:31.978 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 0, 2023-11-27T15:29:31.978 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 2, 2023-11-27T15:29:31.978 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 0, 2023-11-27T15:29:31.978 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 0, 2023-11-27T15:29:31.978 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 4, 2023-11-27T15:29:31.978 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 0, 2023-11-27T15:29:31.979 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.001074127, 2023-11-27T15:29:31.979 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.000000000, 2023-11-27T15:29:31.979 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.000189344, 2023-11-27T15:29:31.979 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:29:31.979 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:31.979 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:31.979 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:31.979 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:31.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:31.980 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:31.980 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:31.980 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:31.980 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:31.980 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:31.980 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 0, 2023-11-27T15:29:31.981 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 0 2023-11-27T15:29:31.981 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:31.981 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-1": { 2023-11-27T15:29:31.981 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 5, 2023-11-27T15:29:31.981 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 120, 2023-11-27T15:29:31.981 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 3152, 2023-11-27T15:29:31.981 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 96992, 2023-11-27T15:29:31.981 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 3, 2023-11-27T15:29:31.982 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 2, 2023-11-27T15:29:31.982 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.006353508, 2023-11-27T15:29:31.982 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.004099676, 2023-11-27T15:29:31.982 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.001100630, 2023-11-27T15:29:31.982 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-11-27T15:29:31.982 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:31.982 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 119, 2023-11-27T15:29:31.982 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.007531651, 2023-11-27T15:29:31.983 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000063291 2023-11-27T15:29:31.983 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:31.983 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:31.983 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:31.983 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:31.983 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:31.983 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:31.983 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 3152, 2023-11-27T15:29:31.984 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 96992 2023-11-27T15:29:31.984 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:31.984 INFO:teuthology.orchestra.run.smithi177.stdout: "AsyncMessenger::Worker-2": { 2023-11-27T15:29:31.984 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_messages": 159, 2023-11-27T15:29:31.984 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages": 156, 2023-11-27T15:29:31.984 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_bytes": 454752, 2023-11-27T15:29:31.984 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_bytes": 88464, 2023-11-27T15:29:31.984 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_created_connections": 2, 2023-11-27T15:29:31.985 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_active_connections": 1, 2023-11-27T15:29:31.985 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_total_time": 0.024598301, 2023-11-27T15:29:31.985 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_send_time": 0.008877394, 2023-11-27T15:29:31.985 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_recv_time": 0.012240045, 2023-11-27T15:29:31.985 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_running_fast_dispatch_time": 0.004166915, 2023-11-27T15:29:31.985 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_messages_queue_lat": { 2023-11-27T15:29:31.985 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 155, 2023-11-27T15:29:31.985 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.009809204, 2023-11-27T15:29:31.986 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000063285 2023-11-27T15:29:31.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:31.986 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_handle_ack_lat": { 2023-11-27T15:29:31.986 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:31.986 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:31.986 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:31.986 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:31.986 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_recv_encrypted_bytes": 454752, 2023-11-27T15:29:31.987 INFO:teuthology.orchestra.run.smithi177.stdout: "msgr_send_encrypted_bytes": 88464 2023-11-27T15:29:31.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:31.987 INFO:teuthology.orchestra.run.smithi177.stdout: "cct": { 2023-11-27T15:29:31.987 INFO:teuthology.orchestra.run.smithi177.stdout: "total_workers": 0, 2023-11-27T15:29:31.987 INFO:teuthology.orchestra.run.smithi177.stdout: "unhealthy_workers": 0 2023-11-27T15:29:31.987 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:31.987 INFO:teuthology.orchestra.run.smithi177.stdout: "mempool": { 2023-11-27T15:29:31.987 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_bytes": 0, 2023-11-27T15:29:31.988 INFO:teuthology.orchestra.run.smithi177.stdout: "bloom_filter_items": 0, 2023-11-27T15:29:31.988 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_bytes": 0, 2023-11-27T15:29:31.988 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_alloc_items": 0, 2023-11-27T15:29:31.988 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_bytes": 0, 2023-11-27T15:29:31.988 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_data_items": 0, 2023-11-27T15:29:31.988 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_bytes": 0, 2023-11-27T15:29:31.988 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_onode_items": 0, 2023-11-27T15:29:31.988 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_bytes": 0, 2023-11-27T15:29:31.989 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_meta_items": 0, 2023-11-27T15:29:31.989 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_bytes": 0, 2023-11-27T15:29:31.989 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_other_items": 0, 2023-11-27T15:29:31.989 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_bytes": 0, 2023-11-27T15:29:31.989 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_cache_buffer_items": 0, 2023-11-27T15:29:31.989 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_bytes": 0, 2023-11-27T15:29:31.989 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_extent_items": 0, 2023-11-27T15:29:31.989 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_bytes": 0, 2023-11-27T15:29:31.990 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_blob_items": 0, 2023-11-27T15:29:31.990 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_bytes": 0, 2023-11-27T15:29:31.990 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_shared_blob_items": 0, 2023-11-27T15:29:31.990 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_bytes": 0, 2023-11-27T15:29:31.990 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_inline_bl_items": 0, 2023-11-27T15:29:31.990 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_bytes": 0, 2023-11-27T15:29:31.990 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_fsck_items": 0, 2023-11-27T15:29:31.991 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_bytes": 0, 2023-11-27T15:29:31.991 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_txc_items": 0, 2023-11-27T15:29:31.991 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_bytes": 0, 2023-11-27T15:29:31.991 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_deferred_items": 0, 2023-11-27T15:29:31.991 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_bytes": 0, 2023-11-27T15:29:31.991 INFO:teuthology.orchestra.run.smithi177.stdout: "bluestore_writing_items": 0, 2023-11-27T15:29:31.991 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_bytes": 0, 2023-11-27T15:29:31.991 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_items": 0, 2023-11-27T15:29:31.992 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_bytes": 0, 2023-11-27T15:29:31.992 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_reader_items": 0, 2023-11-27T15:29:31.992 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_bytes": 0, 2023-11-27T15:29:31.992 INFO:teuthology.orchestra.run.smithi177.stdout: "bluefs_file_writer_items": 0, 2023-11-27T15:29:31.992 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_bytes": 148967, 2023-11-27T15:29:31.992 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_anon_items": 29, 2023-11-27T15:29:31.992 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_bytes": 0, 2023-11-27T15:29:31.992 INFO:teuthology.orchestra.run.smithi177.stdout: "buffer_meta_items": 0, 2023-11-27T15:29:31.993 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_bytes": 0, 2023-11-27T15:29:31.993 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_items": 0, 2023-11-27T15:29:31.993 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_bytes": 0, 2023-11-27T15:29:31.993 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_mapbl_items": 0, 2023-11-27T15:29:31.993 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_bytes": 0, 2023-11-27T15:29:31.993 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_pglog_items": 0, 2023-11-27T15:29:31.993 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_bytes": 0, 2023-11-27T15:29:31.993 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_items": 0, 2023-11-27T15:29:31.994 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_bytes": 0, 2023-11-27T15:29:31.994 INFO:teuthology.orchestra.run.smithi177.stdout: "osdmap_mapping_items": 0, 2023-11-27T15:29:31.994 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_bytes": 0, 2023-11-27T15:29:31.994 INFO:teuthology.orchestra.run.smithi177.stdout: "pgmap_items": 0, 2023-11-27T15:29:31.994 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_bytes": 928, 2023-11-27T15:29:31.994 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_co_items": 2, 2023-11-27T15:29:31.994 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_bytes": 0, 2023-11-27T15:29:31.994 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_1_items": 0, 2023-11-27T15:29:31.995 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_bytes": 0, 2023-11-27T15:29:31.995 INFO:teuthology.orchestra.run.smithi177.stdout: "unittest_2_items": 0 2023-11-27T15:29:31.995 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-11-27T15:29:31.995 INFO:teuthology.orchestra.run.smithi177.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-11-27T15:29:31.995 INFO:teuthology.orchestra.run.smithi177.stdout: "val": 0, 2023-11-27T15:29:31.995 INFO:teuthology.orchestra.run.smithi177.stdout: "max": 104857600, 2023-11-27T15:29:31.995 INFO:teuthology.orchestra.run.smithi177.stdout: "get_started": 0, 2023-11-27T15:29:31.995 INFO:teuthology.orchestra.run.smithi177.stdout: "get": 160, 2023-11-27T15:29:31.996 INFO:teuthology.orchestra.run.smithi177.stdout: "get_sum": 432296, 2023-11-27T15:29:31.996 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_fail": 0, 2023-11-27T15:29:31.996 INFO:teuthology.orchestra.run.smithi177.stdout: "get_or_fail_success": 160, 2023-11-27T15:29:31.996 INFO:teuthology.orchestra.run.smithi177.stdout: "take": 0, 2023-11-27T15:29:31.996 INFO:teuthology.orchestra.run.smithi177.stdout: "take_sum": 0, 2023-11-27T15:29:31.996 INFO:teuthology.orchestra.run.smithi177.stdout: "put": 160, 2023-11-27T15:29:31.996 INFO:teuthology.orchestra.run.smithi177.stdout: "put_sum": 432296, 2023-11-27T15:29:31.996 INFO:teuthology.orchestra.run.smithi177.stdout: "wait": { 2023-11-27T15:29:31.997 INFO:teuthology.orchestra.run.smithi177.stdout: "avgcount": 0, 2023-11-27T15:29:31.997 INFO:teuthology.orchestra.run.smithi177.stdout: "sum": 0.000000000, 2023-11-27T15:29:31.997 INFO:teuthology.orchestra.run.smithi177.stdout: "avgtime": 0.000000000 2023-11-27T15:29:31.997 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:31.997 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-11-27T15:29:31.997 INFO:teuthology.orchestra.run.smithi177.stdout:} 2023-11-27T15:29:32.365 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-11-27T15:29:32.365 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-11-27T15:29:32.366 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2023-11-27T15:29:32.366 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2023-11-27T15:29:32.366 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2023-11-27T15:29:32.366 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-27T15:29:32.378 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-27T15:29:32.379 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-11-27T15:29:32.379 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-27T15:29:32.442 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2023-11-27T15:29:32.442 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-27T15:29:32.530 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:29:32.774 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.770+0000 7f28e93ff700 1 -- 172.21.15.133:0/2748564675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28e4101e10 msgr2=0x7f28e41086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:32.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.770+0000 7f28e93ff700 1 --2- 172.21.15.133:0/2748564675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28e4101e10 0x7f28e41086d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009f90 tx=0x7f28dc00b330 comp rx=0 tx=0).stop 2023-11-27T15:29:32.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.770+0000 7f28e93ff700 1 -- 172.21.15.133:0/2748564675 shutdown_connections 2023-11-27T15:29:32.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.770+0000 7f28e93ff700 1 --2- 172.21.15.133:0/2748564675 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28e4101e10 0x7f28e41086d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:32.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.770+0000 7f28e93ff700 1 --2- 172.21.15.133:0/2748564675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28e4100290 0x7f28e41006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:32.775 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.770+0000 7f28e93ff700 1 --2- 172.21.15.133:0/2748564675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e4101480 0x7f28e41018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:32.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.770+0000 7f28e93ff700 1 -- 172.21.15.133:0/2748564675 >> 172.21.15.133:0/2748564675 conn(0x7f28e40fb870 msgr2=0x7f28e40fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:32.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.770+0000 7f28e93ff700 1 -- 172.21.15.133:0/2748564675 shutdown_connections 2023-11-27T15:29:32.776 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.770+0000 7f28e93ff700 1 -- 172.21.15.133:0/2748564675 wait complete. 2023-11-27T15:29:32.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.771+0000 7f28e93ff700 1 Processor -- start 2023-11-27T15:29:32.778 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.771+0000 7f28e93ff700 1 -- start start 2023-11-27T15:29:32.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.771+0000 7f28e93ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28e4100290 0x7f28e4195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:32.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.771+0000 7f28e93ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28e4101480 0x7f28e4196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:32.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.771+0000 7f28e93ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e4101e10 0x7f28e419d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:32.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.771+0000 7f28e93ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e419d9c0 con 0x7f28e4101e10 2023-11-27T15:29:32.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.771+0000 7f28e93ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e419db00 con 0x7f28e4100290 2023-11-27T15:29:32.779 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.771+0000 7f28e93ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e419dc90 con 0x7f28e4101480 2023-11-27T15:29:32.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.771+0000 7f28e27fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28e4101480 0x7f28e4196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:32.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.772+0000 7f28e37fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e4101e10 0x7f28e419d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:32.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.772+0000 7f28e27fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28e4101480 0x7f28e4196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:50162/0 (socket says 172.21.15.133:50162) 2023-11-27T15:29:32.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.772+0000 7f28e27fc700 1 -- 172.21.15.133:0/1203548248 learned_addr learned my addr 172.21.15.133:0/1203548248 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:29:32.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.772+0000 7f28e2ffd700 1 --2- 172.21.15.133:0/1203548248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28e4100290 0x7f28e4195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:32.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.772+0000 7f28e27fc700 1 -- 172.21.15.133:0/1203548248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28e4100290 msgr2=0x7f28e4195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:32.780 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.772+0000 7f28e27fc700 1 --2- 172.21.15.133:0/1203548248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28e4100290 0x7f28e4195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:32.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.772+0000 7f28e27fc700 1 -- 172.21.15.133:0/1203548248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e4101e10 msgr2=0x7f28e419d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:32.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.772+0000 7f28e27fc700 1 --2- 172.21.15.133:0/1203548248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e4101e10 0x7f28e419d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:32.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.772+0000 7f28e27fc700 1 -- 172.21.15.133:0/1203548248 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc009c40 con 0x7f28e4101480 2023-11-27T15:29:32.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.772+0000 7f28e37fe700 1 --2- 172.21.15.133:0/1203548248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e4101e10 0x7f28e419d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:29:32.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.772+0000 7f28e27fc700 1 --2- 172.21.15.133:0/1203548248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28e4101480 0x7f28e4196380 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f28d800e9c0 tx=0x7f28d800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:32.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.772+0000 7f28cbfff700 1 -- 172.21.15.133:0/1203548248 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d8004a30 con 0x7f28e4101480 2023-11-27T15:29:32.781 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.772+0000 7f28cbfff700 1 -- 172.21.15.133:0/1203548248 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28d800cc80 con 0x7f28e4101480 2023-11-27T15:29:32.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.772+0000 7f28cbfff700 1 -- 172.21.15.133:0/1203548248 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d80054e0 con 0x7f28e4101480 2023-11-27T15:29:32.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.772+0000 7f28e93ff700 1 -- 172.21.15.133:0/1203548248 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e410bb90 con 0x7f28e4101480 2023-11-27T15:29:32.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.772+0000 7f28e93ff700 1 -- 172.21.15.133:0/1203548248 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e410ba00 con 0x7f28e4101480 2023-11-27T15:29:32.782 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.774+0000 7f28e93ff700 1 -- 172.21.15.133:0/1203548248 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e419e2d0 con 0x7f28e4101480 2023-11-27T15:29:32.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.775+0000 7f28cbfff700 1 -- 172.21.15.133:0/1203548248 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f28d8004b90 con 0x7f28e4101480 2023-11-27T15:29:32.783 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.777+0000 7f28cbfff700 1 --2- 172.21.15.133:0/1203548248 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f28cc0a1150 0x7f28cc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:32.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.777+0000 7f28cbfff700 1 -- 172.21.15.133:0/1203548248 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f28d8014070 con 0x7f28e4101480 2023-11-27T15:29:32.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.777+0000 7f28e2ffd700 1 --2- 172.21.15.133:0/1203548248 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f28cc0a1150 0x7f28cc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:32.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.778+0000 7f28e2ffd700 1 --2- 172.21.15.133:0/1203548248 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f28cc0a1150 0x7f28cc0a3600 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f28e41012e0 tx=0x7f28d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:32.784 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.779+0000 7f28cbfff700 1 -- 172.21.15.133:0/1203548248 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f28d8096720 con 0x7f28e4101480 2023-11-27T15:29:32.984 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.979+0000 7f28e93ff700 1 -- 172.21.15.133:0/1203548248 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28e4107140 con 0x7f28e4101480 2023-11-27T15:29:32.986 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.982+0000 7f28cbfff700 1 -- 172.21.15.133:0/1203548248 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24616 (secure 0 0 0) 0x7f28d8099d40 con 0x7f28e4101480 2023-11-27T15:29:32.987 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:29:32.987 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":94,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:28:26.730963+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:29:32.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.984+0000 7f28e93ff700 1 -- 172.21.15.133:0/1203548248 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f28cc0a1150 msgr2=0x7f28cc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:32.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.984+0000 7f28e93ff700 1 --2- 172.21.15.133:0/1203548248 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f28cc0a1150 0x7f28cc0a3600 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f28e41012e0 tx=0x7f28d4009260 comp rx=0 tx=0).stop 2023-11-27T15:29:32.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.984+0000 7f28e93ff700 1 -- 172.21.15.133:0/1203548248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28e4101480 msgr2=0x7f28e4196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:32.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.984+0000 7f28e93ff700 1 --2- 172.21.15.133:0/1203548248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28e4101480 0x7f28e4196380 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f28d800e9c0 tx=0x7f28d800ecd0 comp rx=0 tx=0).stop 2023-11-27T15:29:32.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.984+0000 7f28e93ff700 1 -- 172.21.15.133:0/1203548248 shutdown_connections 2023-11-27T15:29:32.989 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.984+0000 7f28e93ff700 1 --2- 172.21.15.133:0/1203548248 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f28cc0a1150 0x7f28cc0a3600 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:32.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.984+0000 7f28e93ff700 1 --2- 172.21.15.133:0/1203548248 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28e4101e10 0x7f28e419d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:32.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.984+0000 7f28e93ff700 1 --2- 172.21.15.133:0/1203548248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28e4100290 0x7f28e4195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:32.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.984+0000 7f28e93ff700 1 --2- 172.21.15.133:0/1203548248 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28e4101480 0x7f28e4196380 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:32.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.984+0000 7f28e93ff700 1 -- 172.21.15.133:0/1203548248 >> 172.21.15.133:0/1203548248 conn(0x7f28e40fb870 msgr2=0x7f28e40fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:32.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.984+0000 7f28e93ff700 1 -- 172.21.15.133:0/1203548248 shutdown_connections 2023-11-27T15:29:32.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:32.984+0000 7f28e93ff700 1 -- 172.21.15.133:0/1203548248 wait complete. 2023-11-27T15:29:33.008 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-27T15:16:20.216150+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-27T15:20:31.107146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-27T15:20:32.614089+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-27T15:20:36.126040+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-27T15:29:33.008 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-27T15:29:33.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.205+0000 7f09c85ff700 1 -- 172.21.15.133:0/1814838282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09c01014b0 msgr2=0x7f09c0101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:33.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.205+0000 7f09c85ff700 1 --2- 172.21.15.133:0/1814838282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09c01014b0 0x7f09c0101900 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f09b00098f0 tx=0x7f09b0009c00 comp rx=0 tx=0).stop 2023-11-27T15:29:33.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.205+0000 7f09c85ff700 1 -- 172.21.15.133:0/1814838282 shutdown_connections 2023-11-27T15:29:33.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.205+0000 7f09c85ff700 1 --2- 172.21.15.133:0/1814838282 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09c0101e40 0x7f09c0108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.205+0000 7f09c85ff700 1 --2- 172.21.15.133:0/1814838282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c01002c0 0x7f09c01006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.205+0000 7f09c85ff700 1 --2- 172.21.15.133:0/1814838282 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09c01014b0 0x7f09c0101900 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.205+0000 7f09c85ff700 1 -- 172.21.15.133:0/1814838282 >> 172.21.15.133:0/1814838282 conn(0x7f09c00fb8a0 msgr2=0x7f09c00fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:33.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.205+0000 7f09c85ff700 1 -- 172.21.15.133:0/1814838282 shutdown_connections 2023-11-27T15:29:33.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.205+0000 7f09c85ff700 1 -- 172.21.15.133:0/1814838282 wait complete. 2023-11-27T15:29:33.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.206+0000 7f09c85ff700 1 Processor -- start 2023-11-27T15:29:33.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.207+0000 7f09c85ff700 1 -- start start 2023-11-27T15:29:33.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.207+0000 7f09c85ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c01002c0 0x7f09c00748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.215 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.207+0000 7f09c85ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09c01014b0 0x7f09c0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.207+0000 7f09c85ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09c0101e40 0x7f09c0073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.207+0000 7f09c85ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c0073bc0 con 0x7f09c0101e40 2023-11-27T15:29:33.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.207+0000 7f09c85ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c0073d30 con 0x7f09c01002c0 2023-11-27T15:29:33.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.207+0000 7f09c85ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c01a0090 con 0x7f09c01014b0 2023-11-27T15:29:33.216 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.207+0000 7f09c6b9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09c0101e40 0x7f09c0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.207+0000 7f09c6b9c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09c0101e40 0x7f09c0073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47952/0 (socket says 172.21.15.133:47952) 2023-11-27T15:29:33.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.207+0000 7f09c6b9c700 1 -- 172.21.15.133:0/52535964 learned_addr learned my addr 172.21.15.133:0/52535964 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:29:33.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.207+0000 7f09c5b9a700 1 --2- 172.21.15.133:0/52535964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09c01014b0 0x7f09c0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.207+0000 7f09c639b700 1 --2- 172.21.15.133:0/52535964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c01002c0 0x7f09c00748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.217 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.208+0000 7f09c6b9c700 1 -- 172.21.15.133:0/52535964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09c01014b0 msgr2=0x7f09c0072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:33.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.208+0000 7f09c6b9c700 1 --2- 172.21.15.133:0/52535964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09c01014b0 0x7f09c0072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.208+0000 7f09c6b9c700 1 -- 172.21.15.133:0/52535964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c01002c0 msgr2=0x7f09c00748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:33.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.208+0000 7f09c6b9c700 1 --2- 172.21.15.133:0/52535964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c01002c0 0x7f09c00748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.208+0000 7f09c6b9c700 1 -- 172.21.15.133:0/52535964 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b8009d00 con 0x7f09c0101e40 2023-11-27T15:29:33.218 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.208+0000 7f09c639b700 1 --2- 172.21.15.133:0/52535964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c01002c0 0x7f09c00748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:29:33.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.208+0000 7f09c5b9a700 1 --2- 172.21.15.133:0/52535964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09c01014b0 0x7f09c0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:29:33.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.208+0000 7f09c6b9c700 1 --2- 172.21.15.133:0/52535964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09c0101e40 0x7f09c0073420 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f09b800e3c0 tx=0x7f09b800e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:33.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.208+0000 7f09b77fe700 1 -- 172.21.15.133:0/52535964 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b80092d0 con 0x7f09c0101e40 2023-11-27T15:29:33.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.208+0000 7f09b77fe700 1 -- 172.21.15.133:0/52535964 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09b800f040 con 0x7f09c0101e40 2023-11-27T15:29:33.219 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.208+0000 7f09c85ff700 1 -- 172.21.15.133:0/52535964 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b0009520 con 0x7f09c0101e40 2023-11-27T15:29:33.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.208+0000 7f09b77fe700 1 -- 172.21.15.133:0/52535964 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b8007cc0 con 0x7f09c0101e40 2023-11-27T15:29:33.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.208+0000 7f09c85ff700 1 -- 172.21.15.133:0/52535964 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c010bbd0 con 0x7f09c0101e40 2023-11-27T15:29:33.220 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.209+0000 7f09c85ff700 1 -- 172.21.15.133:0/52535964 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c01078d0 con 0x7f09c0101e40 2023-11-27T15:29:33.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.211+0000 7f09b77fe700 1 -- 172.21.15.133:0/52535964 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f09b8003a00 con 0x7f09c0101e40 2023-11-27T15:29:33.221 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.212+0000 7f09b77fe700 1 --2- 172.21.15.133:0/52535964 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f09ac0a1150 0x7f09ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.213+0000 7f09b77fe700 1 -- 172.21.15.133:0/52535964 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f09b80ccaf0 con 0x7f09c0101e40 2023-11-27T15:29:33.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.213+0000 7f09c639b700 1 --2- 172.21.15.133:0/52535964 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f09ac0a1150 0x7f09ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.213+0000 7f09c639b700 1 --2- 172.21.15.133:0/52535964 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f09ac0a1150 0x7f09ac0a3600 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f09c0101310 tx=0x7f09bc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:33.222 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.216+0000 7f09b77fe700 1 -- 172.21.15.133:0/52535964 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f09b8095f20 con 0x7f09c0101e40 2023-11-27T15:29:33.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:32 smithi133 ceph-mon[97860]: pgmap v549: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2023-11-27T15:29:33.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:32 smithi177 ceph-mon[131504]: pgmap v549: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2023-11-27T15:29:33.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:32 smithi174 ceph-mon[131110]: pgmap v549: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 9 op/s 2023-11-27T15:29:33.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.415+0000 7f09c85ff700 1 -- 172.21.15.133:0/52535964 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f09c0067b00 con 0x7f09c0101e40 2023-11-27T15:29:33.420 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.416+0000 7f09b77fe700 1 -- 172.21.15.133:0/52535964 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09b8099540 con 0x7f09c0101e40 2023-11-27T15:29:33.421 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 1 2023-11-27T15:29:33.423 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.418+0000 7f09c85ff700 1 -- 172.21.15.133:0/52535964 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f09ac0a1150 msgr2=0x7f09ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:33.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.418+0000 7f09c85ff700 1 --2- 172.21.15.133:0/52535964 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f09ac0a1150 0x7f09ac0a3600 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f09c0101310 tx=0x7f09bc009260 comp rx=0 tx=0).stop 2023-11-27T15:29:33.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.418+0000 7f09c85ff700 1 -- 172.21.15.133:0/52535964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09c0101e40 msgr2=0x7f09c0073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:33.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.418+0000 7f09c85ff700 1 --2- 172.21.15.133:0/52535964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09c0101e40 0x7f09c0073420 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f09b800e3c0 tx=0x7f09b800e6d0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.418+0000 7f09c85ff700 1 -- 172.21.15.133:0/52535964 shutdown_connections 2023-11-27T15:29:33.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.418+0000 7f09c85ff700 1 --2- 172.21.15.133:0/52535964 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f09ac0a1150 0x7f09ac0a3600 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.424 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.418+0000 7f09c85ff700 1 --2- 172.21.15.133:0/52535964 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09c0101e40 0x7f09c0073420 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.418+0000 7f09c85ff700 1 --2- 172.21.15.133:0/52535964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c01002c0 0x7f09c00748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.418+0000 7f09c85ff700 1 --2- 172.21.15.133:0/52535964 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f09c01014b0 0x7f09c0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.418+0000 7f09c85ff700 1 -- 172.21.15.133:0/52535964 >> 172.21.15.133:0/52535964 conn(0x7f09c00fb8a0 msgr2=0x7f09c0104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:33.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.418+0000 7f09c85ff700 1 -- 172.21.15.133:0/52535964 shutdown_connections 2023-11-27T15:29:33.425 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.419+0000 7f09c85ff700 1 -- 172.21.15.133:0/52535964 wait complete. 2023-11-27T15:29:33.443 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-27T15:29:33.645 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.640+0000 7ff64f1ff700 1 -- 172.21.15.133:0/1456338577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6481002c0 msgr2=0x7ff6481006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:33.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.640+0000 7ff64f1ff700 1 --2- 172.21.15.133:0/1456338577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6481002c0 0x7ff6481006d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff63c009990 tx=0x7ff63c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.641+0000 7ff64f1ff700 1 -- 172.21.15.133:0/1456338577 shutdown_connections 2023-11-27T15:29:33.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.641+0000 7ff64f1ff700 1 --2- 172.21.15.133:0/1456338577 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff648101e40 0x7ff648108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.646 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.641+0000 7ff64f1ff700 1 --2- 172.21.15.133:0/1456338577 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6481002c0 0x7ff6481006d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.641+0000 7ff64f1ff700 1 --2- 172.21.15.133:0/1456338577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6481014b0 0x7ff648101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.641+0000 7ff64f1ff700 1 -- 172.21.15.133:0/1456338577 >> 172.21.15.133:0/1456338577 conn(0x7ff6480fb8a0 msgr2=0x7ff6480fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:33.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.641+0000 7ff64f1ff700 1 -- 172.21.15.133:0/1456338577 shutdown_connections 2023-11-27T15:29:33.647 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.641+0000 7ff64f1ff700 1 -- 172.21.15.133:0/1456338577 wait complete. 2023-11-27T15:29:33.649 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.642+0000 7ff64f1ff700 1 Processor -- start 2023-11-27T15:29:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.642+0000 7ff64f1ff700 1 -- start start 2023-11-27T15:29:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.642+0000 7ff64f1ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6481002c0 0x7ff6481982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.642+0000 7ff64f1ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6481014b0 0x7ff6481987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.650 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.642+0000 7ff64f1ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff648101e40 0x7ff64819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.642+0000 7ff64f1ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64819fe30 con 0x7ff648101e40 2023-11-27T15:29:33.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.642+0000 7ff64f1ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64819ff70 con 0x7ff6481002c0 2023-11-27T15:29:33.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.642+0000 7ff64f1ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6481a0100 con 0x7ff6481014b0 2023-11-27T15:29:33.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.642+0000 7ff64d79c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff648101e40 0x7ff64819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.651 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.642+0000 7ff647fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6481014b0 0x7ff6481987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.642+0000 7ff64d79c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff648101e40 0x7ff64819f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47974/0 (socket says 172.21.15.133:47974) 2023-11-27T15:29:33.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.642+0000 7ff64cf9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6481002c0 0x7ff6481982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.642+0000 7ff64d79c700 1 -- 172.21.15.133:0/166187991 learned_addr learned my addr 172.21.15.133:0/166187991 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:29:33.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.643+0000 7ff64d79c700 1 -- 172.21.15.133:0/166187991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6481014b0 msgr2=0x7ff6481987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:33.652 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.643+0000 7ff64d79c700 1 --2- 172.21.15.133:0/166187991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6481014b0 0x7ff6481987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.643+0000 7ff64d79c700 1 -- 172.21.15.133:0/166187991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6481002c0 msgr2=0x7ff6481982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.643+0000 7ff64d79c700 1 --2- 172.21.15.133:0/166187991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6481002c0 0x7ff6481982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.643+0000 7ff64d79c700 1 -- 172.21.15.133:0/166187991 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63c0095f0 con 0x7ff648101e40 2023-11-27T15:29:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.643+0000 7ff64d79c700 1 --2- 172.21.15.133:0/166187991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff648101e40 0x7ff64819f860 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff640007ee0 tx=0x7ff64000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:33.653 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.643+0000 7ff645ffb700 1 -- 172.21.15.133:0/166187991 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff640017070 con 0x7ff648101e40 2023-11-27T15:29:33.654 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.643+0000 7ff645ffb700 1 -- 172.21.15.133:0/166187991 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff64000f040 con 0x7ff648101e40 2023-11-27T15:29:33.654 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.643+0000 7ff64f1ff700 1 -- 172.21.15.133:0/166187991 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64810bc90 con 0x7ff648101e40 2023-11-27T15:29:33.654 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.643+0000 7ff645ffb700 1 -- 172.21.15.133:0/166187991 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff640005580 con 0x7ff648101e40 2023-11-27T15:29:33.654 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.643+0000 7ff64f1ff700 1 -- 172.21.15.133:0/166187991 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64810bb00 con 0x7ff648101e40 2023-11-27T15:29:33.654 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.644+0000 7ff64f1ff700 1 -- 172.21.15.133:0/166187991 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6481929f0 con 0x7ff648101e40 2023-11-27T15:29:33.655 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.646+0000 7ff645ffb700 1 -- 172.21.15.133:0/166187991 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff64000db20 con 0x7ff648101e40 2023-11-27T15:29:33.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.647+0000 7ff645ffb700 1 --2- 172.21.15.133:0/166187991 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff6380a11d0 0x7ff6380a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:33.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.648+0000 7ff645ffb700 1 -- 172.21.15.133:0/166187991 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ff6400cc300 con 0x7ff648101e40 2023-11-27T15:29:33.656 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.648+0000 7ff64cf9b700 1 --2- 172.21.15.133:0/166187991 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff6380a11d0 0x7ff6380a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:33.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.648+0000 7ff64cf9b700 1 --2- 172.21.15.133:0/166187991 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff6380a11d0 0x7ff6380a3680 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00b2e0 tx=0x7ff63c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:33.657 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.652+0000 7ff645ffb700 1 -- 172.21.15.133:0/166187991 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff640095700 con 0x7ff648101e40 2023-11-27T15:29:33.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.854+0000 7ff64f1ff700 1 -- 172.21.15.133:0/166187991 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff648061c40 con 0x7ff648101e40 2023-11-27T15:29:33.859 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.855+0000 7ff645ffb700 1 -- 172.21.15.133:0/166187991 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff640098d20 con 0x7ff648101e40 2023-11-27T15:29:33.860 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:29:33.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.857+0000 7ff64f1ff700 1 -- 172.21.15.133:0/166187991 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff6380a11d0 msgr2=0x7ff6380a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:33.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.857+0000 7ff64f1ff700 1 --2- 172.21.15.133:0/166187991 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff6380a11d0 0x7ff6380a3680 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00b2e0 tx=0x7ff63c005fb0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.862 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.857+0000 7ff64f1ff700 1 -- 172.21.15.133:0/166187991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff648101e40 msgr2=0x7ff64819f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:33.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.857+0000 7ff64f1ff700 1 --2- 172.21.15.133:0/166187991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff648101e40 0x7ff64819f860 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff640007ee0 tx=0x7ff64000d3b0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.857+0000 7ff64f1ff700 1 -- 172.21.15.133:0/166187991 shutdown_connections 2023-11-27T15:29:33.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.857+0000 7ff64f1ff700 1 --2- 172.21.15.133:0/166187991 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff6380a11d0 0x7ff6380a3680 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.857+0000 7ff64f1ff700 1 --2- 172.21.15.133:0/166187991 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff648101e40 0x7ff64819f860 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.863 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.857+0000 7ff64f1ff700 1 --2- 172.21.15.133:0/166187991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6481002c0 0x7ff6481982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.857+0000 7ff64f1ff700 1 --2- 172.21.15.133:0/166187991 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff6481014b0 0x7ff6481987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:33.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.857+0000 7ff64f1ff700 1 -- 172.21.15.133:0/166187991 >> 172.21.15.133:0/166187991 conn(0x7ff6480fb8a0 msgr2=0x7ff648106d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:33.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.857+0000 7ff64f1ff700 1 -- 172.21.15.133:0/166187991 shutdown_connections 2023-11-27T15:29:33.864 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:33.857+0000 7ff64f1ff700 1 -- 172.21.15.133:0/166187991 wait complete. 2023-11-27T15:29:33.881 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-27T15:29:34.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.078+0000 7fb2efdff700 1 -- 172.21.15.133:0/1247276666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2e81014c0 msgr2=0x7fb2e8101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:34.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.078+0000 7fb2efdff700 1 --2- 172.21.15.133:0/1247276666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2e81014c0 0x7fb2e8101910 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009990 tx=0x7fb2d8009ca0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.083 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.078+0000 7fb2efdff700 1 -- 172.21.15.133:0/1247276666 shutdown_connections 2023-11-27T15:29:34.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.078+0000 7fb2efdff700 1 --2- 172.21.15.133:0/1247276666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2e8101e50 0x7fb2e8108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.078+0000 7fb2efdff700 1 --2- 172.21.15.133:0/1247276666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2e81002d0 0x7fb2e81006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.078+0000 7fb2efdff700 1 --2- 172.21.15.133:0/1247276666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2e81014c0 0x7fb2e8101910 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.078+0000 7fb2efdff700 1 -- 172.21.15.133:0/1247276666 >> 172.21.15.133:0/1247276666 conn(0x7fb2e80fb8b0 msgr2=0x7fb2e80fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:34.084 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.078+0000 7fb2efdff700 1 -- 172.21.15.133:0/1247276666 shutdown_connections 2023-11-27T15:29:34.085 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.078+0000 7fb2efdff700 1 -- 172.21.15.133:0/1247276666 wait complete. 2023-11-27T15:29:34.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.079+0000 7fb2efdff700 1 Processor -- start 2023-11-27T15:29:34.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.079+0000 7fb2efdff700 1 -- start start 2023-11-27T15:29:34.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.079+0000 7fb2efdff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2e81002d0 0x7fb2e8196080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:34.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.079+0000 7fb2efdff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2e81014c0 0x7fb2e81965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:34.088 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.079+0000 7fb2efdff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2e8101e50 0x7fb2e819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:34.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.079+0000 7fb2efdff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e819dc00 con 0x7fb2e8101e50 2023-11-27T15:29:34.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.079+0000 7fb2efdff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e819dd40 con 0x7fb2e81002d0 2023-11-27T15:29:34.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.079+0000 7fb2efdff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e819ded0 con 0x7fb2e81014c0 2023-11-27T15:29:34.089 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.080+0000 7fb2ee39c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2e8101e50 0x7fb2e819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:34.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.080+0000 7fb2edb9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2e81002d0 0x7fb2e8196080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:34.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.080+0000 7fb2ee39c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2e8101e50 0x7fb2e819d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:47994/0 (socket says 172.21.15.133:47994) 2023-11-27T15:29:34.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.080+0000 7fb2ee39c700 1 -- 172.21.15.133:0/1123614177 learned_addr learned my addr 172.21.15.133:0/1123614177 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:29:34.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.080+0000 7fb2ed39a700 1 --2- 172.21.15.133:0/1123614177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2e81014c0 0x7fb2e81965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:34.090 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.080+0000 7fb2ee39c700 1 -- 172.21.15.133:0/1123614177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2e81014c0 msgr2=0x7fb2e81965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:34.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.080+0000 7fb2ee39c700 1 --2- 172.21.15.133:0/1123614177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2e81014c0 0x7fb2e81965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.080+0000 7fb2ee39c700 1 -- 172.21.15.133:0/1123614177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2e81002d0 msgr2=0x7fb2e8196080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:34.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.080+0000 7fb2ee39c700 1 --2- 172.21.15.133:0/1123614177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2e81002d0 0x7fb2e8196080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.080+0000 7fb2ee39c700 1 -- 172.21.15.133:0/1123614177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d80095f0 con 0x7fb2e8101e50 2023-11-27T15:29:34.091 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.080+0000 7fb2edb9b700 1 --2- 172.21.15.133:0/1123614177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2e81002d0 0x7fb2e8196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:29:34.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.080+0000 7fb2ee39c700 1 --2- 172.21.15.133:0/1123614177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2e8101e50 0x7fb2e819d630 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000ef30 tx=0x7fb2e000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:34.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.081+0000 7fb2deffd700 1 -- 172.21.15.133:0/1123614177 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0018070 con 0x7fb2e8101e50 2023-11-27T15:29:34.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.081+0000 7fb2efdff700 1 -- 172.21.15.133:0/1123614177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e810bc80 con 0x7fb2e8101e50 2023-11-27T15:29:34.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.081+0000 7fb2deffd700 1 -- 172.21.15.133:0/1123614177 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2e000f040 con 0x7fb2e8101e50 2023-11-27T15:29:34.092 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.081+0000 7fb2deffd700 1 -- 172.21.15.133:0/1123614177 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e000cd00 con 0x7fb2e8101e50 2023-11-27T15:29:34.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.081+0000 7fb2efdff700 1 -- 172.21.15.133:0/1123614177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e810baf0 con 0x7fb2e8101e50 2023-11-27T15:29:34.093 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.082+0000 7fb2efdff700 1 -- 172.21.15.133:0/1123614177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e8107200 con 0x7fb2e8101e50 2023-11-27T15:29:34.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.084+0000 7fb2deffd700 1 -- 172.21.15.133:0/1123614177 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fb2e0003a00 con 0x7fb2e8101e50 2023-11-27T15:29:34.094 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.085+0000 7fb2deffd700 1 --2- 172.21.15.133:0/1123614177 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb2d40a1150 0x7fb2d40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:34.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.085+0000 7fb2deffd700 1 -- 172.21.15.133:0/1123614177 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fb2e00cc7e0 con 0x7fb2e8101e50 2023-11-27T15:29:34.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.085+0000 7fb2edb9b700 1 --2- 172.21.15.133:0/1123614177 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb2d40a1150 0x7fb2d40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:34.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.086+0000 7fb2edb9b700 1 --2- 172.21.15.133:0/1123614177 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb2d40a1150 0x7fb2d40a3600 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8101320 tx=0x7fb2e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:34.095 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.088+0000 7fb2deffd700 1 -- 172.21.15.133:0/1123614177 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb2e0095c10 con 0x7fb2e8101e50 2023-11-27T15:29:34.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:33 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1203548248' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:29:34.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:33 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/52535964' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:29:34.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:33 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/166187991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:29:34.293 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.289+0000 7fb2efdff700 1 -- 172.21.15.133:0/1123614177 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb2e8067b00 con 0x7fb2e8101e50 2023-11-27T15:29:34.294 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.289+0000 7fb2deffd700 1 -- 172.21.15.133:0/1123614177 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb2e0099230 con 0x7fb2e8101e50 2023-11-27T15:29:34.294 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 64 2023-11-27T15:29:34.296 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.292+0000 7fb2efdff700 1 -- 172.21.15.133:0/1123614177 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb2d40a1150 msgr2=0x7fb2d40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:34.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.292+0000 7fb2efdff700 1 --2- 172.21.15.133:0/1123614177 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb2d40a1150 0x7fb2d40a3600 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8101320 tx=0x7fb2e4009260 comp rx=0 tx=0).stop 2023-11-27T15:29:34.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.292+0000 7fb2efdff700 1 -- 172.21.15.133:0/1123614177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2e8101e50 msgr2=0x7fb2e819d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:34.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.292+0000 7fb2efdff700 1 --2- 172.21.15.133:0/1123614177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2e8101e50 0x7fb2e819d630 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fb2e000ef30 tx=0x7fb2e000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.297 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.292+0000 7fb2efdff700 1 -- 172.21.15.133:0/1123614177 shutdown_connections 2023-11-27T15:29:34.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.292+0000 7fb2efdff700 1 --2- 172.21.15.133:0/1123614177 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fb2d40a1150 0x7fb2d40a3600 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.292+0000 7fb2efdff700 1 --2- 172.21.15.133:0/1123614177 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb2e8101e50 0x7fb2e819d630 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.292+0000 7fb2efdff700 1 --2- 172.21.15.133:0/1123614177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2e81002d0 0x7fb2e8196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.298 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.292+0000 7fb2efdff700 1 --2- 172.21.15.133:0/1123614177 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb2e81014c0 0x7fb2e81965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.292+0000 7fb2efdff700 1 -- 172.21.15.133:0/1123614177 >> 172.21.15.133:0/1123614177 conn(0x7fb2e80fb8b0 msgr2=0x7fb2e80fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:34.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.292+0000 7fb2efdff700 1 -- 172.21.15.133:0/1123614177 shutdown_connections 2023-11-27T15:29:34.299 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.292+0000 7fb2efdff700 1 -- 172.21.15.133:0/1123614177 wait complete. 2023-11-27T15:29:34.316 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-27T15:29:34.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:33 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1203548248' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:29:34.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:33 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/52535964' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:29:34.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:33 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/166187991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:29:34.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:33 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1203548248' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:29:34.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:33 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/52535964' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-11-27T15:29:34.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:33 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/166187991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-11-27T15:29:34.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.512+0000 7fcf29fff700 1 -- 172.21.15.133:0/3821091321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf24100230 msgr2=0x7fcf24100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:34.517 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.512+0000 7fcf29fff700 1 --2- 172.21.15.133:0/3821091321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf24100230 0x7fcf24100640 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcf140098f0 tx=0x7fcf14009c00 comp rx=0 tx=0).stop 2023-11-27T15:29:34.518 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.512+0000 7fcf29fff700 1 -- 172.21.15.133:0/3821091321 shutdown_connections 2023-11-27T15:29:34.518 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.512+0000 7fcf29fff700 1 --2- 172.21.15.133:0/3821091321 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf24101db0 0x7fcf24108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.518 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.512+0000 7fcf29fff700 1 --2- 172.21.15.133:0/3821091321 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf24100230 0x7fcf24100640 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.518 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.512+0000 7fcf29fff700 1 --2- 172.21.15.133:0/3821091321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf24101420 0x7fcf24101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.518 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.512+0000 7fcf29fff700 1 -- 172.21.15.133:0/3821091321 >> 172.21.15.133:0/3821091321 conn(0x7fcf240fb810 msgr2=0x7fcf240fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:34.519 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.512+0000 7fcf29fff700 1 -- 172.21.15.133:0/3821091321 shutdown_connections 2023-11-27T15:29:34.519 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.512+0000 7fcf29fff700 1 -- 172.21.15.133:0/3821091321 wait complete. 2023-11-27T15:29:34.522 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.513+0000 7fcf29fff700 1 Processor -- start 2023-11-27T15:29:34.522 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.513+0000 7fcf29fff700 1 -- start start 2023-11-27T15:29:34.522 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.514+0000 7fcf29fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf24100230 0x7fcf241981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:34.522 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.514+0000 7fcf29fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf24101420 0x7fcf241986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:34.522 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.514+0000 7fcf29fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf24101db0 0x7fcf2419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:34.523 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.514+0000 7fcf29fff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2419fd30 con 0x7fcf24101420 2023-11-27T15:29:34.523 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.514+0000 7fcf29fff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2419fe70 con 0x7fcf24100230 2023-11-27T15:29:34.523 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.514+0000 7fcf29fff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf241a0000 con 0x7fcf24101db0 2023-11-27T15:29:34.523 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.514+0000 7fcf22ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf24101420 0x7fcf241986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:34.523 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.514+0000 7fcf237fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf24100230 0x7fcf241981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:34.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.514+0000 7fcf22ffd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf24101420 0x7fcf241986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48022/0 (socket says 172.21.15.133:48022) 2023-11-27T15:29:34.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.514+0000 7fcf22ffd700 1 -- 172.21.15.133:0/3707949230 learned_addr learned my addr 172.21.15.133:0/3707949230 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:29:34.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.514+0000 7fcf23fff700 1 --2- 172.21.15.133:0/3707949230 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf24101db0 0x7fcf2419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:34.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.514+0000 7fcf237fe700 1 -- 172.21.15.133:0/3707949230 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf24101db0 msgr2=0x7fcf2419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:34.524 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.514+0000 7fcf237fe700 1 --2- 172.21.15.133:0/3707949230 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf24101db0 0x7fcf2419f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.514+0000 7fcf237fe700 1 -- 172.21.15.133:0/3707949230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf24101420 msgr2=0x7fcf241986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:34.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.514+0000 7fcf237fe700 1 --2- 172.21.15.133:0/3707949230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf24101420 0x7fcf241986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.514+0000 7fcf237fe700 1 -- 172.21.15.133:0/3707949230 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf14009520 con 0x7fcf24100230 2023-11-27T15:29:34.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.514+0000 7fcf23fff700 1 --2- 172.21.15.133:0/3707949230 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf24101db0 0x7fcf2419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:29:34.525 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.514+0000 7fcf22ffd700 1 --2- 172.21.15.133:0/3707949230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf24101420 0x7fcf241986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:29:34.526 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.515+0000 7fcf237fe700 1 --2- 172.21.15.133:0/3707949230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf24100230 0x7fcf241981b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcf14000c00 tx=0x7fcf1400b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:34.526 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.515+0000 7fcf20ff9700 1 -- 172.21.15.133:0/3707949230 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1401c070 con 0x7fcf24100230 2023-11-27T15:29:34.526 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.515+0000 7fcf29fff700 1 -- 172.21.15.133:0/3707949230 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf2410bb90 con 0x7fcf24100230 2023-11-27T15:29:34.526 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.515+0000 7fcf20ff9700 1 -- 172.21.15.133:0/3707949230 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf1400f7d0 con 0x7fcf24100230 2023-11-27T15:29:34.526 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.515+0000 7fcf20ff9700 1 -- 172.21.15.133:0/3707949230 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf140039b0 con 0x7fcf24100230 2023-11-27T15:29:34.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.515+0000 7fcf29fff700 1 -- 172.21.15.133:0/3707949230 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2410ba00 con 0x7fcf24100230 2023-11-27T15:29:34.527 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.516+0000 7fcf29fff700 1 -- 172.21.15.133:0/3707949230 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf24067a60 con 0x7fcf24100230 2023-11-27T15:29:34.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.518+0000 7fcf20ff9700 1 -- 172.21.15.133:0/3707949230 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7fcf14020030 con 0x7fcf24100230 2023-11-27T15:29:34.528 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.519+0000 7fcf20ff9700 1 --2- 172.21.15.133:0/3707949230 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcf0c0a1150 0x7fcf0c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:34.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.519+0000 7fcf20ff9700 1 -- 172.21.15.133:0/3707949230 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7fcf140cdfb0 con 0x7fcf24100230 2023-11-27T15:29:34.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.520+0000 7fcf22ffd700 1 --2- 172.21.15.133:0/3707949230 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcf0c0a1150 0x7fcf0c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:34.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.520+0000 7fcf22ffd700 1 --2- 172.21.15.133:0/3707949230 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcf0c0a1150 0x7fcf0c0a3600 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcf18011550 tx=0x7fcf18009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:34.529 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.522+0000 7fcf20ff9700 1 -- 172.21.15.133:0/3707949230 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcf140973e0 con 0x7fcf24100230 2023-11-27T15:29:34.723 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.719+0000 7fcf29fff700 1 -- 172.21.15.133:0/3707949230 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcf241a0b20 con 0x7fcf24100230 2023-11-27T15:29:34.724 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.720+0000 7fcf20ff9700 1 -- 172.21.15.133:0/3707949230 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcf1409aa00 con 0x7fcf24100230 2023-11-27T15:29:34.724 INFO:teuthology.orchestra.run.smithi133.stdout:pg_num: 32 2023-11-27T15:29:34.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.722+0000 7fcf29fff700 1 -- 172.21.15.133:0/3707949230 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcf0c0a1150 msgr2=0x7fcf0c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:34.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.722+0000 7fcf29fff700 1 --2- 172.21.15.133:0/3707949230 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcf0c0a1150 0x7fcf0c0a3600 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcf18011550 tx=0x7fcf18009260 comp rx=0 tx=0).stop 2023-11-27T15:29:34.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.722+0000 7fcf29fff700 1 -- 172.21.15.133:0/3707949230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf24100230 msgr2=0x7fcf241981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:34.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.722+0000 7fcf29fff700 1 --2- 172.21.15.133:0/3707949230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf24100230 0x7fcf241981b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fcf14000c00 tx=0x7fcf1400b790 comp rx=0 tx=0).stop 2023-11-27T15:29:34.727 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.722+0000 7fcf29fff700 1 -- 172.21.15.133:0/3707949230 shutdown_connections 2023-11-27T15:29:34.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.722+0000 7fcf29fff700 1 --2- 172.21.15.133:0/3707949230 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7fcf0c0a1150 0x7fcf0c0a3600 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.722+0000 7fcf29fff700 1 --2- 172.21.15.133:0/3707949230 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf24101db0 0x7fcf2419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.722+0000 7fcf29fff700 1 --2- 172.21.15.133:0/3707949230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf24100230 0x7fcf241981b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.722+0000 7fcf29fff700 1 --2- 172.21.15.133:0/3707949230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcf24101420 0x7fcf241986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.722+0000 7fcf29fff700 1 -- 172.21.15.133:0/3707949230 >> 172.21.15.133:0/3707949230 conn(0x7fcf240fb810 msgr2=0x7fcf240fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:34.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.722+0000 7fcf29fff700 1 -- 172.21.15.133:0/3707949230 shutdown_connections 2023-11-27T15:29:34.728 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.722+0000 7fcf29fff700 1 -- 172.21.15.133:0/3707949230 wait complete. 2023-11-27T15:29:34.747 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-27T15:29:34.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.944+0000 7f0b809ff700 1 -- 172.21.15.133:0/3497589456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b7c100230 msgr2=0x7f0b7c100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:34.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.944+0000 7f0b809ff700 1 --2- 172.21.15.133:0/3497589456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b7c100230 0x7f0b7c100640 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c009990 tx=0x7f0b6c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.944+0000 7f0b809ff700 1 -- 172.21.15.133:0/3497589456 shutdown_connections 2023-11-27T15:29:34.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.944+0000 7f0b809ff700 1 --2- 172.21.15.133:0/3497589456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b7c101db0 0x7f0b7c108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.950 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.944+0000 7f0b809ff700 1 --2- 172.21.15.133:0/3497589456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b7c100230 0x7f0b7c100640 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.944+0000 7f0b809ff700 1 --2- 172.21.15.133:0/3497589456 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b7c101420 0x7f0b7c101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.944+0000 7f0b809ff700 1 -- 172.21.15.133:0/3497589456 >> 172.21.15.133:0/3497589456 conn(0x7f0b7c0fb810 msgr2=0x7f0b7c0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:34.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.945+0000 7f0b809ff700 1 -- 172.21.15.133:0/3497589456 shutdown_connections 2023-11-27T15:29:34.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.945+0000 7f0b809ff700 1 -- 172.21.15.133:0/3497589456 wait complete. 2023-11-27T15:29:34.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.946+0000 7f0b809ff700 1 Processor -- start 2023-11-27T15:29:34.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.946+0000 7f0b809ff700 1 -- start start 2023-11-27T15:29:34.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.946+0000 7f0b809ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b7c100230 0x7f0b7c1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:34.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.946+0000 7f0b809ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b7c101420 0x7f0b7c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:34.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.946+0000 7f0b809ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b7c101db0 0x7f0b7c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:34.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.946+0000 7f0b809ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c19fd60 con 0x7f0b7c101db0 2023-11-27T15:29:34.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.946+0000 7f0b809ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c19fea0 con 0x7f0b7c101420 2023-11-27T15:29:34.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.946+0000 7f0b809ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c1a0030 con 0x7f0b7c100230 2023-11-27T15:29:34.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.946+0000 7f0b7a59c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b7c100230 0x7f0b7c1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:34.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.946+0000 7f0b7ad9d700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b7c101db0 0x7f0b7c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:34.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.946+0000 7f0b73fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b7c101420 0x7f0b7c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:34.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.946+0000 7f0b7a59c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b7c100230 0x7f0b7c1981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:38892/0 (socket says 172.21.15.133:38892) 2023-11-27T15:29:34.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.946+0000 7f0b7a59c700 1 -- 172.21.15.133:0/4033937158 learned_addr learned my addr 172.21.15.133:0/4033937158 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:29:34.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.947+0000 7f0b7a59c700 1 -- 172.21.15.133:0/4033937158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b7c101420 msgr2=0x7f0b7c198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:34.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.947+0000 7f0b7a59c700 1 --2- 172.21.15.133:0/4033937158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b7c101420 0x7f0b7c198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.947+0000 7f0b7a59c700 1 -- 172.21.15.133:0/4033937158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b7c101db0 msgr2=0x7f0b7c19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:34.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.947+0000 7f0b7a59c700 1 --2- 172.21.15.133:0/4033937158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b7c101db0 0x7f0b7c19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:34.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.947+0000 7f0b7a59c700 1 -- 172.21.15.133:0/4033937158 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6c0095f0 con 0x7f0b7c100230 2023-11-27T15:29:34.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.947+0000 7f0b73fff700 1 --2- 172.21.15.133:0/4033937158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b7c101420 0x7f0b7c198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:29:34.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.947+0000 7f0b7a59c700 1 --2- 172.21.15.133:0/4033937158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b7c100230 0x7f0b7c1981e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c005740 tx=0x7f0b6c005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:34.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.947+0000 7f0b737fe700 1 -- 172.21.15.133:0/4033937158 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c01d070 con 0x7f0b7c100230 2023-11-27T15:29:34.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.947+0000 7f0b737fe700 1 -- 172.21.15.133:0/4033937158 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b6c021410 con 0x7f0b7c100230 2023-11-27T15:29:34.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.947+0000 7f0b809ff700 1 -- 172.21.15.133:0/4033937158 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b7c10bbe0 con 0x7f0b7c100230 2023-11-27T15:29:34.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.947+0000 7f0b737fe700 1 -- 172.21.15.133:0/4033937158 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c017650 con 0x7f0b7c100230 2023-11-27T15:29:34.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.947+0000 7f0b809ff700 1 -- 172.21.15.133:0/4033937158 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b7c10ba50 con 0x7f0b7c100230 2023-11-27T15:29:34.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.948+0000 7f0b809ff700 1 -- 172.21.15.133:0/4033937158 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b7c192940 con 0x7f0b7c100230 2023-11-27T15:29:34.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.950+0000 7f0b737fe700 1 -- 172.21.15.133:0/4033937158 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f0b6c021580 con 0x7f0b7c100230 2023-11-27T15:29:34.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.951+0000 7f0b737fe700 1 --2- 172.21.15.133:0/4033937158 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0b680a1150 0x7f0b680a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:34.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.952+0000 7f0b737fe700 1 -- 172.21.15.133:0/4033937158 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f0b6c0cde10 con 0x7f0b7c100230 2023-11-27T15:29:34.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.952+0000 7f0b73fff700 1 --2- 172.21.15.133:0/4033937158 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0b680a1150 0x7f0b680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:34.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.952+0000 7f0b73fff700 1 --2- 172.21.15.133:0/4033937158 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0b680a1150 0x7f0b680a3600 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0b64011490 tx=0x7f0b64009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:34.960 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:34.954+0000 7f0b737fe700 1 -- 172.21.15.133:0/4033937158 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0b6c097240 con 0x7f0b7c100230 2023-11-27T15:29:35.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:34 smithi133 ceph-mon[97860]: pgmap v550: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:29:35.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:34 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/1123614177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:29:35.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:34 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/3707949230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:29:35.179 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.174+0000 7f0b809ff700 1 -- 172.21.15.133:0/4033937158 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0b7c061700 con 0x7f0b7c100230 2023-11-27T15:29:35.180 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.175+0000 7f0b737fe700 1 -- 172.21.15.133:0/4033937158 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v94) v1 ==== 81+0+602 (secure 0 0 0) 0x7f0b6c09a860 con 0x7f0b7c100230 2023-11-27T15:29:35.180 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6800/2374615220 2023-11-28T15:14:01.628185+0000 2023-11-27T15:29:35.180 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/1594111147 2023-11-28T15:14:01.628185+0000 2023-11-27T15:29:35.180 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6801/3149426454 2023-11-28T15:13:35.034149+0000 2023-11-27T15:29:35.181 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6800/3149426454 2023-11-28T15:13:35.034149+0000 2023-11-27T15:29:35.181 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/2898896137 2023-11-28T15:14:01.628185+0000 2023-11-27T15:29:35.181 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/1816614695 2023-11-28T15:13:35.034149+0000 2023-11-27T15:29:35.181 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/2624811615 2023-11-28T15:13:35.034149+0000 2023-11-27T15:29:35.181 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:6801/2374615220 2023-11-28T15:14:01.628185+0000 2023-11-27T15:29:35.181 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/4208867747 2023-11-28T15:14:01.628185+0000 2023-11-27T15:29:35.182 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.15.133:0/4192331415 2023-11-28T15:13:35.034149+0000 2023-11-27T15:29:35.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.178+0000 7f0b809ff700 1 -- 172.21.15.133:0/4033937158 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0b680a1150 msgr2=0x7f0b680a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:35.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.178+0000 7f0b809ff700 1 --2- 172.21.15.133:0/4033937158 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0b680a1150 0x7f0b680a3600 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f0b64011490 tx=0x7f0b64009260 comp rx=0 tx=0).stop 2023-11-27T15:29:35.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.178+0000 7f0b809ff700 1 -- 172.21.15.133:0/4033937158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b7c100230 msgr2=0x7f0b7c1981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:35.183 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.178+0000 7f0b809ff700 1 --2- 172.21.15.133:0/4033937158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b7c100230 0x7f0b7c1981e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c005740 tx=0x7f0b6c005770 comp rx=0 tx=0).stop 2023-11-27T15:29:35.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.178+0000 7f0b809ff700 1 -- 172.21.15.133:0/4033937158 shutdown_connections 2023-11-27T15:29:35.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.178+0000 7f0b809ff700 1 --2- 172.21.15.133:0/4033937158 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f0b680a1150 0x7f0b680a3600 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:35.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.178+0000 7f0b809ff700 1 --2- 172.21.15.133:0/4033937158 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b7c101db0 0x7f0b7c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:35.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.178+0000 7f0b809ff700 1 --2- 172.21.15.133:0/4033937158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b7c100230 0x7f0b7c1981e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:35.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.178+0000 7f0b809ff700 1 --2- 172.21.15.133:0/4033937158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b7c101420 0x7f0b7c198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:35.184 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.178+0000 7f0b809ff700 1 -- 172.21.15.133:0/4033937158 >> 172.21.15.133:0/4033937158 conn(0x7f0b7c0fb810 msgr2=0x7f0b7c106d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:35.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.178+0000 7f0b809ff700 1 -- 172.21.15.133:0/4033937158 shutdown_connections 2023-11-27T15:29:35.185 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.178+0000 7f0b809ff700 1 -- 172.21.15.133:0/4033937158 wait complete. 2023-11-27T15:29:35.185 INFO:teuthology.orchestra.run.smithi133.stderr:listed 10 entries 2023-11-27T15:29:35.202 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-27T15:29:35.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:29:35.202 DEBUG:teuthology.orchestra.run.smithi133:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-27T15:29:35.319 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2023-11-27T15:29:35.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:29:35.319 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-27T15:29:35.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:34 smithi177 ceph-mon[131504]: pgmap v550: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:29:35.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:34 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/1123614177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:29:35.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:34 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/3707949230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:29:35.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:34 smithi174 ceph-mon[131110]: pgmap v550: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 7 op/s 2023-11-27T15:29:35.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:34 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/1123614177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-11-27T15:29:35.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:34 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/3707949230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-11-27T15:29:35.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:29:35.439 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2023-11-27T15:29:35.553 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-27T15:29:35.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:29:35.554 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-27T15:29:35.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-27T15:29:35.625 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2023-11-27T15:29:35.727 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-11-27T15:29:35.739 INFO:tasks.cephadm:Teardown begin 2023-11-27T15:29:35.740 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-27T15:29:35.740 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:29:35.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.942+0000 7f2dde3ff700 1 -- 172.21.15.133:0/3676592983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dd8101480 msgr2=0x7f2dd81018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:35.947 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.942+0000 7f2dde3ff700 1 --2- 172.21.15.133:0/3676592983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dd8101480 0x7f2dd81018d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009990 tx=0x7f2dcc009ca0 comp rx=0 tx=0).stop 2023-11-27T15:29:35.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.942+0000 7f2dde3ff700 1 -- 172.21.15.133:0/3676592983 shutdown_connections 2023-11-27T15:29:35.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.942+0000 7f2dde3ff700 1 --2- 172.21.15.133:0/3676592983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dd8101e10 0x7f2dd81086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:35.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.942+0000 7f2dde3ff700 1 --2- 172.21.15.133:0/3676592983 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dd8100290 0x7f2dd81006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:35.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.942+0000 7f2dde3ff700 1 --2- 172.21.15.133:0/3676592983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dd8101480 0x7f2dd81018d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:35.948 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.942+0000 7f2dde3ff700 1 -- 172.21.15.133:0/3676592983 >> 172.21.15.133:0/3676592983 conn(0x7f2dd80fb870 msgr2=0x7f2dd80fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:35.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.943+0000 7f2dde3ff700 1 -- 172.21.15.133:0/3676592983 shutdown_connections 2023-11-27T15:29:35.949 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.943+0000 7f2dde3ff700 1 -- 172.21.15.133:0/3676592983 wait complete. 2023-11-27T15:29:35.951 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.944+0000 7f2dde3ff700 1 Processor -- start 2023-11-27T15:29:35.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.944+0000 7f2dde3ff700 1 -- start start 2023-11-27T15:29:35.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.945+0000 7f2dde3ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dd8100290 0x7f2dd80748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:35.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.945+0000 7f2dde3ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dd8101480 0x7f2dd8072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:35.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.945+0000 7f2dde3ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dd8101e10 0x7f2dd8073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:35.952 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.945+0000 7f2dde3ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd8073bc0 con 0x7f2dd8101480 2023-11-27T15:29:35.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.945+0000 7f2dde3ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd8073d30 con 0x7f2dd8100290 2023-11-27T15:29:35.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.945+0000 7f2dde3ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd819fff0 con 0x7f2dd8101e10 2023-11-27T15:29:35.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.945+0000 7f2dd7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dd8100290 0x7f2dd80748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:35.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.945+0000 7f2dd7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dd8100290 0x7f2dd80748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.133:35458/0 (socket says 172.21.15.133:35458) 2023-11-27T15:29:35.953 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.945+0000 7f2dd77fe700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dd8101480 0x7f2dd8072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:35.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.945+0000 7f2ddc99c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dd8101e10 0x7f2dd8073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:35.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.945+0000 7f2dd7fff700 1 -- 172.21.15.133:0/2400029101 learned_addr learned my addr 172.21.15.133:0/2400029101 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:29:35.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.945+0000 7f2dd7fff700 1 -- 172.21.15.133:0/2400029101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dd8101e10 msgr2=0x7f2dd8073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:35.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.945+0000 7f2dd7fff700 1 --2- 172.21.15.133:0/2400029101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dd8101e10 0x7f2dd8073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:35.954 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.945+0000 7f2dd7fff700 1 -- 172.21.15.133:0/2400029101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dd8101480 msgr2=0x7f2dd8072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:35.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.945+0000 7f2dd7fff700 1 --2- 172.21.15.133:0/2400029101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dd8101480 0x7f2dd8072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:35.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.946+0000 7f2dd7fff700 1 -- 172.21.15.133:0/2400029101 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc0095f0 con 0x7f2dd8100290 2023-11-27T15:29:35.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.946+0000 7f2dd7fff700 1 --2- 172.21.15.133:0/2400029101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dd8100290 0x7f2dd80748a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800e970 tx=0x7f2dc800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:35.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.946+0000 7f2dd57fa700 1 -- 172.21.15.133:0/2400029101 <== mon.2 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8009790 con 0x7f2dd8100290 2023-11-27T15:29:35.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.946+0000 7f2dde3ff700 1 -- 172.21.15.133:0/2400029101 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd810bc50 con 0x7f2dd8100290 2023-11-27T15:29:35.955 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.946+0000 7f2dd57fa700 1 -- 172.21.15.133:0/2400029101 <== mon.2 v2:172.21.15.174:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2dc800c7e0 con 0x7f2dd8100290 2023-11-27T15:29:35.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.946+0000 7f2dd57fa700 1 -- 172.21.15.133:0/2400029101 <== mon.2 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8003c70 con 0x7f2dd8100290 2023-11-27T15:29:35.956 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.946+0000 7f2dde3ff700 1 -- 172.21.15.133:0/2400029101 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd810bac0 con 0x7f2dd8100290 2023-11-27T15:29:35.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.947+0000 7f2dde3ff700 1 -- 172.21.15.133:0/2400029101 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd81a0790 con 0x7f2dd8100290 2023-11-27T15:29:35.957 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.949+0000 7f2dd57fa700 1 -- 172.21.15.133:0/2400029101 <== mon.2 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f2dc800cc50 con 0x7f2dd8100290 2023-11-27T15:29:35.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.950+0000 7f2dd57fa700 1 --2- 172.21.15.133:0/2400029101 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2dc00a11d0 0x7f2dc00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:35.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.951+0000 7f2dd57fa700 1 -- 172.21.15.133:0/2400029101 <== mon.2 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f2dc8014070 con 0x7f2dd8100290 2023-11-27T15:29:35.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.951+0000 7f2dd77fe700 1 --2- 172.21.15.133:0/2400029101 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2dc00a11d0 0x7f2dc00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:35.958 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.951+0000 7f2dd77fe700 1 --2- 172.21.15.133:0/2400029101 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2dc00a11d0 0x7f2dc00a3680 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00b2e0 tx=0x7f2dcc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:35.959 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:35.955+0000 7f2dd57fa700 1 -- 172.21.15.133:0/2400029101 <== mon.2 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2dc8096170 con 0x7f2dd8100290 2023-11-27T15:29:36.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.201+0000 7f2dde3ff700 1 -- 172.21.15.133:0/2400029101 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2dd81071d0 con 0x7f2dd8100290 2023-11-27T15:29:36.206 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.202+0000 7f2dd57fa700 1 -- 172.21.15.133:0/2400029101 <== mon.2 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10838 (secure 0 0 0) 0x7f2dc8003dd0 con 0x7f2dd8100290 2023-11-27T15:29:36.207 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:29:36.207 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14493,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2381290259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2381290259},{"type":"v1","addr":"172.21.15.133:6839","nonce":2381290259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1704875895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1704875895},{"type":"v1","addr":"172.21.15.133:6841","nonce":1704875895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:20:33.201023+0000","modified":"2023-11-27T15:28:26.922183+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24521,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6839/2781994447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2781994447},{"type":"v1","addr":"172.21.15.174:6839","nonce":2781994447}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_26467":{"gid":26467,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/1199749740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1199749740},{"type":"v1","addr":"172.21.15.177:6837","nonce":1199749740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:29:36.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.205+0000 7f2dde3ff700 1 -- 172.21.15.133:0/2400029101 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2dc00a11d0 msgr2=0x7f2dc00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:36.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.205+0000 7f2dde3ff700 1 --2- 172.21.15.133:0/2400029101 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2dc00a11d0 0x7f2dc00a3680 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00b2e0 tx=0x7f2dcc005fb0 comp rx=0 tx=0).stop 2023-11-27T15:29:36.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.205+0000 7f2dde3ff700 1 -- 172.21.15.133:0/2400029101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dd8100290 msgr2=0x7f2dd80748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:36.210 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.205+0000 7f2dde3ff700 1 --2- 172.21.15.133:0/2400029101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dd8100290 0x7f2dd80748a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800e970 tx=0x7f2dc800ec80 comp rx=0 tx=0).stop 2023-11-27T15:29:36.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.205+0000 7f2dde3ff700 1 -- 172.21.15.133:0/2400029101 shutdown_connections 2023-11-27T15:29:36.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.205+0000 7f2dde3ff700 1 --2- 172.21.15.133:0/2400029101 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f2dc00a11d0 0x7f2dc00a3680 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:36.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.205+0000 7f2dde3ff700 1 --2- 172.21.15.133:0/2400029101 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dd8101e10 0x7f2dd8073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:36.211 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.205+0000 7f2dde3ff700 1 --2- 172.21.15.133:0/2400029101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dd8100290 0x7f2dd80748a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:36.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.205+0000 7f2dde3ff700 1 --2- 172.21.15.133:0/2400029101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2dd8101480 0x7f2dd8072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:36.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.205+0000 7f2dde3ff700 1 -- 172.21.15.133:0/2400029101 >> 172.21.15.133:0/2400029101 conn(0x7f2dd80fb870 msgr2=0x7f2dd80fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:36.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.205+0000 7f2dde3ff700 1 -- 172.21.15.133:0/2400029101 shutdown_connections 2023-11-27T15:29:36.212 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.205+0000 7f2dde3ff700 1 -- 172.21.15.133:0/2400029101 wait complete. 2023-11-27T15:29:36.213 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2023-11-27T15:29:36.226 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:35 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/4033937158' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-27T15:29:36.230 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-27T15:29:36.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:35 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/4033937158' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-27T15:29:36.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:35 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/4033937158' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-11-27T15:29:36.432 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.427+0000 7ff57bbff700 1 -- 172.21.15.133:0/2219483929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57410c860 msgr2=0x7ff57410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:36.432 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.427+0000 7ff57bbff700 1 --2- 172.21.15.133:0/2219483929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57410c860 0x7ff57410ec40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00b0a0 tx=0x7ff56c00b3b0 comp rx=0 tx=0).stop 2023-11-27T15:29:36.433 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.427+0000 7ff57bbff700 1 -- 172.21.15.133:0/2219483929 shutdown_connections 2023-11-27T15:29:36.433 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.427+0000 7ff57bbff700 1 --2- 172.21.15.133:0/2219483929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57410c860 0x7ff57410ec40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:36.433 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.427+0000 7ff57bbff700 1 --2- 172.21.15.133:0/2219483929 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff574074b30 0x7ff574072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:36.433 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.427+0000 7ff57bbff700 1 --2- 172.21.15.133:0/2219483929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff574073550 0x7ff5740739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:36.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.427+0000 7ff57bbff700 1 -- 172.21.15.133:0/2219483929 >> 172.21.15.133:0/2219483929 conn(0x7ff5740fb870 msgr2=0x7ff5740fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:36.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.427+0000 7ff57bbff700 1 -- 172.21.15.133:0/2219483929 shutdown_connections 2023-11-27T15:29:36.434 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.427+0000 7ff57bbff700 1 -- 172.21.15.133:0/2219483929 wait complete. 2023-11-27T15:29:36.437 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.429+0000 7ff57bbff700 1 Processor -- start 2023-11-27T15:29:36.438 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.429+0000 7ff57bbff700 1 -- start start 2023-11-27T15:29:36.438 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.429+0000 7ff57bbff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff574073550 0x7ff57419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:36.438 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.429+0000 7ff57bbff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff574074b30 0x7ff57419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:36.438 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.429+0000 7ff57bbff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57410c860 0x7ff5741a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:36.438 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.429+0000 7ff57bbff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5741a4210 con 0x7ff574073550 2023-11-27T15:29:36.439 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.429+0000 7ff57bbff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5741a4350 con 0x7ff57410c860 2023-11-27T15:29:36.439 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.429+0000 7ff57bbff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5741a44e0 con 0x7ff574074b30 2023-11-27T15:29:36.439 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.430+0000 7ff57999b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff574073550 0x7ff57419c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:36.439 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.430+0000 7ff57999b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff574073550 0x7ff57419c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48084/0 (socket says 172.21.15.133:48084) 2023-11-27T15:29:36.439 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.430+0000 7ff57999b700 1 -- 172.21.15.133:0/4224282247 learned_addr learned my addr 172.21.15.133:0/4224282247 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:29:36.440 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.430+0000 7ff57a19c700 1 --2- 172.21.15.133:0/4224282247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57410c860 0x7ff5741a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:36.440 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.430+0000 7ff57919a700 1 --2- 172.21.15.133:0/4224282247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff574074b30 0x7ff57419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:36.440 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.430+0000 7ff57999b700 1 -- 172.21.15.133:0/4224282247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff574074b30 msgr2=0x7ff57419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:36.440 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.430+0000 7ff57999b700 1 --2- 172.21.15.133:0/4224282247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff574074b30 0x7ff57419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:36.440 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.430+0000 7ff57999b700 1 -- 172.21.15.133:0/4224282247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57410c860 msgr2=0x7ff5741a3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:36.441 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.430+0000 7ff57999b700 1 --2- 172.21.15.133:0/4224282247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57410c860 0x7ff5741a3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:36.441 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.430+0000 7ff57999b700 1 -- 172.21.15.133:0/4224282247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56c009d00 con 0x7ff574073550 2023-11-27T15:29:36.441 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.430+0000 7ff57919a700 1 --2- 172.21.15.133:0/4224282247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff574074b30 0x7ff57419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-27T15:29:36.441 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.430+0000 7ff57999b700 1 --2- 172.21.15.133:0/4224282247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff574073550 0x7ff57419c690 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff57000e970 tx=0x7ff57000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:36.441 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.430+0000 7ff56affd700 1 -- 172.21.15.133:0/4224282247 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570009790 con 0x7ff574073550 2023-11-27T15:29:36.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.431+0000 7ff56affd700 1 -- 172.21.15.133:0/4224282247 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff57000eec0 con 0x7ff574073550 2023-11-27T15:29:36.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.431+0000 7ff57bbff700 1 -- 172.21.15.133:0/4224282247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff574112270 con 0x7ff574073550 2023-11-27T15:29:36.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.431+0000 7ff56affd700 1 -- 172.21.15.133:0/4224282247 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570005590 con 0x7ff574073550 2023-11-27T15:29:36.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.431+0000 7ff57bbff700 1 -- 172.21.15.133:0/4224282247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5741120e0 con 0x7ff574073550 2023-11-27T15:29:36.442 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.432+0000 7ff57bbff700 1 -- 172.21.15.133:0/4224282247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5741023b0 con 0x7ff574073550 2023-11-27T15:29:36.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.434+0000 7ff56affd700 1 -- 172.21.15.133:0/4224282247 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff57000cbd0 con 0x7ff574073550 2023-11-27T15:29:36.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.436+0000 7ff56affd700 1 --2- 172.21.15.133:0/4224282247 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff5600a1150 0x7ff5600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:36.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.436+0000 7ff56affd700 1 -- 172.21.15.133:0/4224282247 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7ff570014070 con 0x7ff574073550 2023-11-27T15:29:36.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.436+0000 7ff57919a700 1 --2- 172.21.15.133:0/4224282247 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff5600a1150 0x7ff5600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:36.444 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.437+0000 7ff57919a700 1 --2- 172.21.15.133:0/4224282247 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff5600a1150 0x7ff5600a3600 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff574074850 tx=0x7ff564009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:36.445 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.440+0000 7ff56affd700 1 -- 172.21.15.133:0/4224282247 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff5700962a0 con 0x7ff574073550 2023-11-27T15:29:36.695 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.691+0000 7ff57bbff700 1 -- 172.21.15.133:0/4224282247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff57402cfe0 con 0x7ff574073550 2023-11-27T15:29:36.696 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.692+0000 7ff56affd700 1 -- 172.21.15.133:0/4224282247 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10838 (secure 0 0 0) 0x7ff570004ef0 con 0x7ff574073550 2023-11-27T15:29:36.696 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:29:36.697 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14442,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2249018853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6836","nonce":2249018853},{"type":"v1","addr":"172.21.15.133:6837","nonce":2249018853}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14493,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6839/2381290259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6838","nonce":2381290259},{"type":"v1","addr":"172.21.15.133:6839","nonce":2381290259}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14505,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6841/1704875895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6840","nonce":1704875895},{"type":"v1","addr":"172.21.15.133:6841","nonce":1704875895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1329176662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1329176662},{"type":"v1","addr":"172.21.15.177:6833","nonce":1329176662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24455,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/2946349291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2946349291},{"type":"v1","addr":"172.21.15.174:6833","nonce":2946349291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6835/1877742522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1877742522},{"type":"v1","addr":"172.21.15.133:6835","nonce":1877742522}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-11-27T15:20:33.201023+0000","modified":"2023-11-27T15:28:26.922183+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24560,"mds_1":24521,"mds_2":24520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/637171798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":637171798},{"type":"v1","addr":"172.21.15.174:6835","nonce":637171798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24496":{"gid":24496,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6835/3620466749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":3620466749},{"type":"v1","addr":"172.21.15.177:6835","nonce":3620466749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24520":{"gid":24520,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.177:6839/1463687598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6838","nonce":1463687598},{"type":"v1","addr":"172.21.15.177:6839","nonce":1463687598}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1955090548","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1955090548},{"type":"v1","addr":"172.21.15.174:6837","nonce":1955090548}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24560":{"gid":24560,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.174:6839/2781994447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2781994447},{"type":"v1","addr":"172.21.15.174:6839","nonce":2781994447}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_26467":{"gid":26467,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.177:6837/1199749740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6836","nonce":1199749740},{"type":"v1","addr":"172.21.15.177:6837","nonce":1199749740}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-27T15:29:36.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.694+0000 7ff57bbff700 1 -- 172.21.15.133:0/4224282247 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff5600a1150 msgr2=0x7ff5600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:36.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.694+0000 7ff57bbff700 1 --2- 172.21.15.133:0/4224282247 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff5600a1150 0x7ff5600a3600 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff574074850 tx=0x7ff564009260 comp rx=0 tx=0).stop 2023-11-27T15:29:36.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.694+0000 7ff57bbff700 1 -- 172.21.15.133:0/4224282247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff574073550 msgr2=0x7ff57419c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:36.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.694+0000 7ff57bbff700 1 --2- 172.21.15.133:0/4224282247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff574073550 0x7ff57419c690 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff57000e970 tx=0x7ff57000ec80 comp rx=0 tx=0).stop 2023-11-27T15:29:36.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.694+0000 7ff57bbff700 1 -- 172.21.15.133:0/4224282247 shutdown_connections 2023-11-27T15:29:36.699 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.694+0000 7ff57bbff700 1 --2- 172.21.15.133:0/4224282247 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff5600a1150 0x7ff5600a3600 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:36.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.694+0000 7ff57bbff700 1 --2- 172.21.15.133:0/4224282247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff57410c860 0x7ff5741a3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:36.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.694+0000 7ff57bbff700 1 --2- 172.21.15.133:0/4224282247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff574073550 0x7ff57419c690 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:36.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.694+0000 7ff57bbff700 1 --2- 172.21.15.133:0/4224282247 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff574074b30 0x7ff57419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:36.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.694+0000 7ff57bbff700 1 -- 172.21.15.133:0/4224282247 >> 172.21.15.133:0/4224282247 conn(0x7ff5740fb870 msgr2=0x7ff5740fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:36.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.694+0000 7ff57bbff700 1 -- 172.21.15.133:0/4224282247 shutdown_connections 2023-11-27T15:29:36.700 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.694+0000 7ff57bbff700 1 -- 172.21.15.133:0/4224282247 wait complete. 2023-11-27T15:29:36.701 INFO:teuthology.orchestra.run.smithi133.stderr:dumped fsmap epoch 30 2023-11-27T15:29:36.718 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-27T15:29:36.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.912+0000 7f81659ff700 1 -- 172.21.15.133:0/2527872467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81601014a0 msgr2=0x7f81601018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:36.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.912+0000 7f81659ff700 1 --2- 172.21.15.133:0/2527872467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81601014a0 0x7f81601018f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8148009990 tx=0x7f8148009ca0 comp rx=0 tx=0).stop 2023-11-27T15:29:36.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.912+0000 7f81659ff700 1 -- 172.21.15.133:0/2527872467 shutdown_connections 2023-11-27T15:29:36.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.912+0000 7f81659ff700 1 --2- 172.21.15.133:0/2527872467 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8160101e30 0x7f81601086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:36.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.912+0000 7f81659ff700 1 --2- 172.21.15.133:0/2527872467 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81601002b0 0x7f81601006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:36.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.912+0000 7f81659ff700 1 --2- 172.21.15.133:0/2527872467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81601014a0 0x7f81601018f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:36.917 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.912+0000 7f81659ff700 1 -- 172.21.15.133:0/2527872467 >> 172.21.15.133:0/2527872467 conn(0x7f81600fb890 msgr2=0x7f81600fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:36.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.912+0000 7f81659ff700 1 -- 172.21.15.133:0/2527872467 shutdown_connections 2023-11-27T15:29:36.918 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.912+0000 7f81659ff700 1 -- 172.21.15.133:0/2527872467 wait complete. 2023-11-27T15:29:36.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.913+0000 7f81659ff700 1 Processor -- start 2023-11-27T15:29:36.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.913+0000 7f81659ff700 1 -- start start 2023-11-27T15:29:36.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.914+0000 7f81659ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81601002b0 0x7f81601981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:36.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.914+0000 7f81659ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81601014a0 0x7f8160198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:36.920 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.914+0000 7f81659ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8160101e30 0x7f816019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:36.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.914+0000 7f81659ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816019fd40 con 0x7f81601002b0 2023-11-27T15:29:36.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.914+0000 7f81659ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816019fe80 con 0x7f8160101e30 2023-11-27T15:29:36.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.914+0000 7f81659ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81601a0010 con 0x7f81601014a0 2023-11-27T15:29:36.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.914+0000 7f815effd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81601002b0 0x7f81601981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:36.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.914+0000 7f815effd700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81601002b0 0x7f81601981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48104/0 (socket says 172.21.15.133:48104) 2023-11-27T15:29:36.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.914+0000 7f815effd700 1 -- 172.21.15.133:0/95572145 learned_addr learned my addr 172.21.15.133:0/95572145 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:29:36.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.914+0000 7f8157fff700 1 --2- 172.21.15.133:0/95572145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81601014a0 0x7f8160198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:36.921 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.914+0000 7f815f7fe700 1 --2- 172.21.15.133:0/95572145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8160101e30 0x7f816019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:36.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.914+0000 7f815effd700 1 -- 172.21.15.133:0/95572145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81601014a0 msgr2=0x7f8160198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:36.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.914+0000 7f815effd700 1 --2- 172.21.15.133:0/95572145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81601014a0 0x7f8160198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:36.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.914+0000 7f815effd700 1 -- 172.21.15.133:0/95572145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8160101e30 msgr2=0x7f816019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:36.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.914+0000 7f815effd700 1 --2- 172.21.15.133:0/95572145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8160101e30 0x7f816019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:36.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.914+0000 7f815effd700 1 -- 172.21.15.133:0/95572145 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81480095f0 con 0x7f81601002b0 2023-11-27T15:29:36.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.914+0000 7f8157fff700 1 --2- 172.21.15.133:0/95572145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81601014a0 0x7f8160198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:29:36.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.914+0000 7f815f7fe700 1 --2- 172.21.15.133:0/95572145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8160101e30 0x7f816019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:29:36.922 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.915+0000 7f815effd700 1 --2- 172.21.15.133:0/95572145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81601002b0 0x7f81601981c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f815000e970 tx=0x7f815000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:36.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.915+0000 7f81649fd700 1 -- 172.21.15.133:0/95572145 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150009790 con 0x7f81601002b0 2023-11-27T15:29:36.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.915+0000 7f81649fd700 1 -- 172.21.15.133:0/95572145 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f815000c7e0 con 0x7f81601002b0 2023-11-27T15:29:36.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.915+0000 7f81659ff700 1 -- 172.21.15.133:0/95572145 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816010bc60 con 0x7f81601002b0 2023-11-27T15:29:36.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.915+0000 7f81649fd700 1 -- 172.21.15.133:0/95572145 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150003c70 con 0x7f81601002b0 2023-11-27T15:29:36.923 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.915+0000 7f81659ff700 1 -- 172.21.15.133:0/95572145 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f816010bad0 con 0x7f81601002b0 2023-11-27T15:29:36.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.916+0000 7f81659ff700 1 -- 172.21.15.133:0/95572145 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81601928f0 con 0x7f81601002b0 2023-11-27T15:29:36.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.918+0000 7f81649fd700 1 -- 172.21.15.133:0/95572145 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f815000cc50 con 0x7f81601002b0 2023-11-27T15:29:36.924 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.920+0000 7f81649fd700 1 --2- 172.21.15.133:0/95572145 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f814c0a1150 0x7f814c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:36.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.920+0000 7f81649fd700 1 -- 172.21.15.133:0/95572145 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f8150014070 con 0x7f81601002b0 2023-11-27T15:29:36.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.920+0000 7f8157fff700 1 --2- 172.21.15.133:0/95572145 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f814c0a1150 0x7f814c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:36.925 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.921+0000 7f8157fff700 1 --2- 172.21.15.133:0/95572145 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f814c0a1150 0x7f814c0a3600 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f814800b2e0 tx=0x7f8148005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:36.927 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:36.923+0000 7f81649fd700 1 -- 172.21.15.133:0/95572145 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8150096210 con 0x7f81601002b0 2023-11-27T15:29:37.026 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:36 smithi133 ceph-mon[97860]: pgmap v551: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2023-11-27T15:29:37.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:36 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2400029101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:29:37.027 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:36 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/4224282247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:29:37.134 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.130+0000 7f81659ff700 1 -- 172.21.15.133:0/95572145 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f816002cfe0 con 0x7f81601002b0 2023-11-27T15:29:37.135 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.131+0000 7f81649fd700 1 -- 172.21.15.133:0/95572145 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+24616 (secure 0 0 0) 0x7f8150003dd0 con 0x7f81601002b0 2023-11-27T15:29:37.136 INFO:teuthology.orchestra.run.smithi133.stdout: 2023-11-27T15:29:37.136 INFO:teuthology.orchestra.run.smithi133.stdout:{"epoch":94,"fsid":"5227da4c-8d37-11ee-95a2-87774f69a715","created":"2023-11-27T15:13:01.701530+0000","modified":"2023-11-27T15:28:26.730963+0000","last_up_change":"2023-11-27T15:19:47.979002+0000","last_in_change":"2023-11-27T15:19:30.693355+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-27T15:16:20.216150+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-27T15:20:31.107146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-27T15:20:32.614089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-27T15:20:36.126040+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0ebdea23-8131-43b6-b3b1-06d1337bf063","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6803","nonce":3850463495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6805","nonce":3850463495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6809","nonce":3850463495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":3850463495},{"type":"v1","addr":"172.21.15.133:6807","nonce":3850463495}]},"public_addr":"172.21.15.133:6803/3850463495","cluster_addr":"172.21.15.133:6805/3850463495","heartbeat_back_addr":"172.21.15.133:6809/3850463495","heartbeat_front_addr":"172.21.15.133:6807/3850463495","state":["exists","up"]},{"osd":1,"uuid":"dc933082-b3e3-43d5-b110-c4d118df4418","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6801","nonce":3890663971}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6803","nonce":3890663971}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6807","nonce":3890663971}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":3890663971},{"type":"v1","addr":"172.21.15.174:6805","nonce":3890663971}]},"public_addr":"172.21.15.174:6801/3890663971","cluster_addr":"172.21.15.174:6803/3890663971","heartbeat_back_addr":"172.21.15.174:6807/3890663971","heartbeat_front_addr":"172.21.15.174:6805/3890663971","state":["exists","up"]},{"osd":2,"uuid":"4a8ba7ac-d927-4a9f-9079-105004d658a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6801","nonce":2043769283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6803","nonce":2043769283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6807","nonce":2043769283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":2043769283},{"type":"v1","addr":"172.21.15.177:6805","nonce":2043769283}]},"public_addr":"172.21.15.177:6801/2043769283","cluster_addr":"172.21.15.177:6803/2043769283","heartbeat_back_addr":"172.21.15.177:6807/2043769283","heartbeat_front_addr":"172.21.15.177:6805/2043769283","state":["exists","up"]},{"osd":3,"uuid":"452fbca8-3b28-4258-a86b-1042888fe294","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6811","nonce":673797009}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6813","nonce":673797009}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6817","nonce":673797009}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":673797009},{"type":"v1","addr":"172.21.15.133:6815","nonce":673797009}]},"public_addr":"172.21.15.133:6811/673797009","cluster_addr":"172.21.15.133:6813/673797009","heartbeat_back_addr":"172.21.15.133:6817/673797009","heartbeat_front_addr":"172.21.15.133:6815/673797009","state":["exists","up"]},{"osd":4,"uuid":"dd04b180-bbcc-4611-9c42-936c87827fe7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6809","nonce":959536963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6811","nonce":959536963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6815","nonce":959536963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":959536963},{"type":"v1","addr":"172.21.15.174:6813","nonce":959536963}]},"public_addr":"172.21.15.174:6809/959536963","cluster_addr":"172.21.15.174:6811/959536963","heartbeat_back_addr":"172.21.15.174:6815/959536963","heartbeat_front_addr":"172.21.15.174:6813/959536963","state":["exists","up"]},{"osd":5,"uuid":"ec77b94b-3eb1-41a2-a319-fd446ce04816","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6809","nonce":3863044388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6811","nonce":3863044388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6815","nonce":3863044388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":3863044388},{"type":"v1","addr":"172.21.15.177:6813","nonce":3863044388}]},"public_addr":"172.21.15.177:6809/3863044388","cluster_addr":"172.21.15.177:6811/3863044388","heartbeat_back_addr":"172.21.15.177:6815/3863044388","heartbeat_front_addr":"172.21.15.177:6813/3863044388","state":["exists","up"]},{"osd":6,"uuid":"2abbda56-2257-4a87-9f75-8491329c2df8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6819","nonce":3664128592}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6821","nonce":3664128592}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6825","nonce":3664128592}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3664128592},{"type":"v1","addr":"172.21.15.133:6823","nonce":3664128592}]},"public_addr":"172.21.15.133:6819/3664128592","cluster_addr":"172.21.15.133:6821/3664128592","heartbeat_back_addr":"172.21.15.133:6825/3664128592","heartbeat_front_addr":"172.21.15.133:6823/3664128592","state":["exists","up"]},{"osd":7,"uuid":"106a6b4b-99fc-44a8-a4a4-c635b25101d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6817","nonce":2275667196}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6819","nonce":2275667196}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6823","nonce":2275667196}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2275667196},{"type":"v1","addr":"172.21.15.174:6821","nonce":2275667196}]},"public_addr":"172.21.15.174:6817/2275667196","cluster_addr":"172.21.15.174:6819/2275667196","heartbeat_back_addr":"172.21.15.174:6823/2275667196","heartbeat_front_addr":"172.21.15.174:6821/2275667196","state":["exists","up"]},{"osd":8,"uuid":"d1541498-5236-447b-b194-d8da4226318e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6817","nonce":3783894564}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6819","nonce":3783894564}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6823","nonce":3783894564}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":3783894564},{"type":"v1","addr":"172.21.15.177:6821","nonce":3783894564}]},"public_addr":"172.21.15.177:6817/3783894564","cluster_addr":"172.21.15.177:6819/3783894564","heartbeat_back_addr":"172.21.15.177:6823/3783894564","heartbeat_front_addr":"172.21.15.177:6821/3783894564","state":["exists","up"]},{"osd":9,"uuid":"5e115ec1-93e3-41ea-a785-18b6c70b7b9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6827","nonce":34943790}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6829","nonce":34943790}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6833","nonce":34943790}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":34943790},{"type":"v1","addr":"172.21.15.133:6831","nonce":34943790}]},"public_addr":"172.21.15.133:6827/34943790","cluster_addr":"172.21.15.133:6829/34943790","heartbeat_back_addr":"172.21.15.133:6833/34943790","heartbeat_front_addr":"172.21.15.133:6831/34943790","state":["exists","up"]},{"osd":10,"uuid":"342cb818-eaf2-4961-81aa-fd849bfd7be5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6825","nonce":880237758}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6827","nonce":880237758}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6831","nonce":880237758}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":880237758},{"type":"v1","addr":"172.21.15.174:6829","nonce":880237758}]},"public_addr":"172.21.15.174:6825/880237758","cluster_addr":"172.21.15.174:6827/880237758","heartbeat_back_addr":"172.21.15.174:6831/880237758","heartbeat_front_addr":"172.21.15.174:6829/880237758","state":["exists","up"]},{"osd":11,"uuid":"d9a72525-14d3-4ff6-8a06-abcd6af6396d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":79,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6825","nonce":1870203795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6827","nonce":1870203795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6831","nonce":1870203795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":1870203795},{"type":"v1","addr":"172.21.15.177:6829","nonce":1870203795}]},"public_addr":"172.21.15.177:6825/1870203795","cluster_addr":"172.21.15.177:6827/1870203795","heartbeat_back_addr":"172.21.15.177:6831/1870203795","heartbeat_front_addr":"172.21.15.177:6829/1870203795","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:34.273913+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:15:56.512387+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:17.687403+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:16:44.686648+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:06.804770+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:27.382036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:17:53.336974+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:15.611191+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:18:36.979496+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:02.961185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:25.266530+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-27T15:19:46.846501+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6800/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1594111147":"2023-11-28T15:14:01.628185+0000","172.21.15.133:6801/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6800/3149426454":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2898896137":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/1816614695":"2023-11-28T15:13:35.034149+0000","172.21.15.133:0/2624811615":"2023-11-28T15:13:35.034149+0000","172.21.15.133:6801/2374615220":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4208867747":"2023-11-28T15:14:01.628185+0000","172.21.15.133:0/4192331415":"2023-11-28T15:13:35.034149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-27T15:29:37.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.133+0000 7f81659ff700 1 -- 172.21.15.133:0/95572145 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f814c0a1150 msgr2=0x7f814c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:37.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.133+0000 7f81659ff700 1 --2- 172.21.15.133:0/95572145 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f814c0a1150 0x7f814c0a3600 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f814800b2e0 tx=0x7f8148005fb0 comp rx=0 tx=0).stop 2023-11-27T15:29:37.138 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.134+0000 7f81659ff700 1 -- 172.21.15.133:0/95572145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81601002b0 msgr2=0x7f81601981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.134+0000 7f81659ff700 1 --2- 172.21.15.133:0/95572145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81601002b0 0x7f81601981c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f815000e970 tx=0x7f815000ec80 comp rx=0 tx=0).stop 2023-11-27T15:29:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.134+0000 7f81659ff700 1 -- 172.21.15.133:0/95572145 shutdown_connections 2023-11-27T15:29:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.134+0000 7f81659ff700 1 --2- 172.21.15.133:0/95572145 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f814c0a1150 0x7f814c0a3600 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.134+0000 7f81659ff700 1 --2- 172.21.15.133:0/95572145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8160101e30 0x7f816019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.134+0000 7f81659ff700 1 --2- 172.21.15.133:0/95572145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81601002b0 0x7f81601981c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.134+0000 7f81659ff700 1 --2- 172.21.15.133:0/95572145 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81601014a0 0x7f8160198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.134+0000 7f81659ff700 1 -- 172.21.15.133:0/95572145 >> 172.21.15.133:0/95572145 conn(0x7f81600fb890 msgr2=0x7f8160106d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:37.139 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.134+0000 7f81659ff700 1 -- 172.21.15.133:0/95572145 shutdown_connections 2023-11-27T15:29:37.140 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.134+0000 7f81659ff700 1 -- 172.21.15.133:0/95572145 wait complete. 2023-11-27T15:29:37.158 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-27T15:29:37.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:36 smithi177 ceph-mon[131504]: pgmap v551: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2023-11-27T15:29:37.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:36 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2400029101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:29:37.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:36 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/4224282247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:29:37.362 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.358+0000 7f95ebbff700 1 -- 172.21.15.133:0/1425478728 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95e4100240 msgr2=0x7f95e4100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:37.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.358+0000 7f95ebbff700 1 --2- 172.21.15.133:0/1425478728 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95e4100240 0x7f95e4100650 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f95e00098f0 tx=0x7f95e0009c00 comp rx=0 tx=0).stop 2023-11-27T15:29:37.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.358+0000 7f95ebbff700 1 -- 172.21.15.133:0/1425478728 shutdown_connections 2023-11-27T15:29:37.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.358+0000 7f95ebbff700 1 --2- 172.21.15.133:0/1425478728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95e4101dc0 0x7f95e4108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:37.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.358+0000 7f95ebbff700 1 --2- 172.21.15.133:0/1425478728 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95e4100240 0x7f95e4100650 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:37.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.358+0000 7f95ebbff700 1 --2- 172.21.15.133:0/1425478728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95e4101430 0x7f95e4101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:37.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.358+0000 7f95ebbff700 1 -- 172.21.15.133:0/1425478728 >> 172.21.15.133:0/1425478728 conn(0x7f95e40fb820 msgr2=0x7f95e40fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:37.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.358+0000 7f95ebbff700 1 -- 172.21.15.133:0/1425478728 shutdown_connections 2023-11-27T15:29:37.363 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.358+0000 7f95ebbff700 1 -- 172.21.15.133:0/1425478728 wait complete. 2023-11-27T15:29:37.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.359+0000 7f95ebbff700 1 Processor -- start 2023-11-27T15:29:37.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.359+0000 7f95ebbff700 1 -- start start 2023-11-27T15:29:37.365 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.359+0000 7f95ebbff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95e4100240 0x7f95e4196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:37.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.359+0000 7f95ebbff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95e4101430 0x7f95e41965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:37.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.359+0000 7f95ebbff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95e4101dc0 0x7f95e419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:37.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.359+0000 7f95ebbff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e419dbe0 con 0x7f95e4101dc0 2023-11-27T15:29:37.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.359+0000 7f95ebbff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e419dd20 con 0x7f95e4101430 2023-11-27T15:29:37.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.359+0000 7f95ebbff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e419deb0 con 0x7f95e4100240 2023-11-27T15:29:37.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.360+0000 7f95e999b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95e4100240 0x7f95e4196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:37.366 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.360+0000 7f95ea19c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95e4101dc0 0x7f95e419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:37.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.360+0000 7f95e919a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95e4101430 0x7f95e41965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:37.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.360+0000 7f95e999b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95e4100240 0x7f95e4196060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.133:38954/0 (socket says 172.21.15.133:38954) 2023-11-27T15:29:37.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.360+0000 7f95e999b700 1 -- 172.21.15.133:0/661578050 learned_addr learned my addr 172.21.15.133:0/661578050 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:29:37.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.360+0000 7f95e999b700 1 -- 172.21.15.133:0/661578050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95e4101430 msgr2=0x7f95e41965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:37.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.360+0000 7f95e999b700 1 --2- 172.21.15.133:0/661578050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95e4101430 0x7f95e41965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:37.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.360+0000 7f95e999b700 1 -- 172.21.15.133:0/661578050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95e4101dc0 msgr2=0x7f95e419d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:37.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.360+0000 7f95e999b700 1 --2- 172.21.15.133:0/661578050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95e4101dc0 0x7f95e419d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:37.367 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.360+0000 7f95e999b700 1 -- 172.21.15.133:0/661578050 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e0009520 con 0x7f95e4100240 2023-11-27T15:29:37.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.360+0000 7f95ea19c700 1 --2- 172.21.15.133:0/661578050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95e4101dc0 0x7f95e419d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:29:37.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.360+0000 7f95e999b700 1 --2- 172.21.15.133:0/661578050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95e4100240 0x7f95e4196060 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009fd0 tx=0x7f95e000b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:37.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.361+0000 7f95daffd700 1 -- 172.21.15.133:0/661578050 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e001c070 con 0x7f95e4100240 2023-11-27T15:29:37.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.361+0000 7f95daffd700 1 -- 172.21.15.133:0/661578050 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95e000f7d0 con 0x7f95e4100240 2023-11-27T15:29:37.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.361+0000 7f95ebbff700 1 -- 172.21.15.133:0/661578050 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e410bba0 con 0x7f95e4100240 2023-11-27T15:29:37.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.361+0000 7f95daffd700 1 -- 172.21.15.133:0/661578050 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e0003a20 con 0x7f95e4100240 2023-11-27T15:29:37.368 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.361+0000 7f95ebbff700 1 -- 172.21.15.133:0/661578050 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e410ba10 con 0x7f95e4100240 2023-11-27T15:29:37.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.362+0000 7f95ebbff700 1 -- 172.21.15.133:0/661578050 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e4107820 con 0x7f95e4100240 2023-11-27T15:29:37.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.364+0000 7f95daffd700 1 -- 172.21.15.133:0/661578050 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f95e0020030 con 0x7f95e4100240 2023-11-27T15:29:37.369 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.365+0000 7f95daffd700 1 --2- 172.21.15.133:0/661578050 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f95d00a1150 0x7f95d00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:37.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.365+0000 7f95daffd700 1 -- 172.21.15.133:0/661578050 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 9078+0+0 (secure 0 0 0) 0x7f95e00ce0b0 con 0x7f95e4100240 2023-11-27T15:29:37.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.365+0000 7f95e919a700 1 --2- 172.21.15.133:0/661578050 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f95d00a1150 0x7f95d00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:37.370 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.366+0000 7f95e919a700 1 --2- 172.21.15.133:0/661578050 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f95d00a1150 0x7f95d00a3600 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f95d4005fd0 tx=0x7f95d4009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:37.371 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.367+0000 7f95daffd700 1 -- 172.21.15.133:0/661578050 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f95e00974e0 con 0x7f95e4100240 2023-11-27T15:29:37.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:36 smithi174 ceph-mon[131110]: pgmap v551: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2023-11-27T15:29:37.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:36 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2400029101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:29:37.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:36 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/4224282247' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-11-27T15:29:37.615 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.611+0000 7f95ebbff700 1 -- 172.21.15.133:0/661578050 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f95e402cfe0 con 0x7f95e4100240 2023-11-27T15:29:37.965 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.961+0000 7f95daffd700 1 -- 172.21.15.133:0/661578050 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7f95e009ab00 con 0x7f95e4100240 2023-11-27T15:29:37.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.963+0000 7f95ebbff700 1 -- 172.21.15.133:0/661578050 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f95d00a1150 msgr2=0x7f95d00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:37.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.963+0000 7f95ebbff700 1 --2- 172.21.15.133:0/661578050 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f95d00a1150 0x7f95d00a3600 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f95d4005fd0 tx=0x7f95d4009310 comp rx=0 tx=0).stop 2023-11-27T15:29:37.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.963+0000 7f95ebbff700 1 -- 172.21.15.133:0/661578050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95e4100240 msgr2=0x7f95e4196060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:37.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.963+0000 7f95ebbff700 1 --2- 172.21.15.133:0/661578050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95e4100240 0x7f95e4196060 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009fd0 tx=0x7f95e000b790 comp rx=0 tx=0).stop 2023-11-27T15:29:37.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.963+0000 7f95ebbff700 1 -- 172.21.15.133:0/661578050 shutdown_connections 2023-11-27T15:29:37.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.963+0000 7f95ebbff700 1 --2- 172.21.15.133:0/661578050 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f95d00a1150 0x7f95d00a3600 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:37.968 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.963+0000 7f95ebbff700 1 --2- 172.21.15.133:0/661578050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95e4101dc0 0x7f95e419d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:37.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.963+0000 7f95ebbff700 1 --2- 172.21.15.133:0/661578050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f95e4100240 0x7f95e4196060 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:37.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.963+0000 7f95ebbff700 1 --2- 172.21.15.133:0/661578050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95e4101430 0x7f95e41965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:37.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.963+0000 7f95ebbff700 1 -- 172.21.15.133:0/661578050 >> 172.21.15.133:0/661578050 conn(0x7f95e40fb820 msgr2=0x7f95e4104960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:37.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.963+0000 7f95ebbff700 1 -- 172.21.15.133:0/661578050 shutdown_connections 2023-11-27T15:29:37.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:37.963+0000 7f95ebbff700 1 -- 172.21.15.133:0/661578050 wait complete. 2023-11-27T15:29:37.969 INFO:teuthology.orchestra.run.smithi133.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-27T15:29:37.987 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-27T15:29:38.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.186+0000 7f8b13dff700 1 -- 172.21.15.133:0/208619307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b0c1002c0 msgr2=0x7f8b0c1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:38.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.186+0000 7f8b13dff700 1 --2- 172.21.15.133:0/208619307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b0c1002c0 0x7f8b0c1006d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8b08009990 tx=0x7f8b08009ca0 comp rx=0 tx=0).stop 2023-11-27T15:29:38.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.187+0000 7f8b13dff700 1 -- 172.21.15.133:0/208619307 shutdown_connections 2023-11-27T15:29:38.191 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.187+0000 7f8b13dff700 1 --2- 172.21.15.133:0/208619307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b0c101e40 0x7f8b0c108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:38.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.187+0000 7f8b13dff700 1 --2- 172.21.15.133:0/208619307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b0c1002c0 0x7f8b0c1006d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:38.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.187+0000 7f8b13dff700 1 --2- 172.21.15.133:0/208619307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b0c1014b0 0x7f8b0c101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:38.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.187+0000 7f8b13dff700 1 -- 172.21.15.133:0/208619307 >> 172.21.15.133:0/208619307 conn(0x7f8b0c0fb8a0 msgr2=0x7f8b0c0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:38.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.187+0000 7f8b13dff700 1 -- 172.21.15.133:0/208619307 shutdown_connections 2023-11-27T15:29:38.192 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.187+0000 7f8b13dff700 1 -- 172.21.15.133:0/208619307 wait complete. 2023-11-27T15:29:38.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.188+0000 7f8b13dff700 1 Processor -- start 2023-11-27T15:29:38.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.188+0000 7f8b13dff700 1 -- start start 2023-11-27T15:29:38.194 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.189+0000 7f8b13dff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b0c1002c0 0x7f8b0c0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:38.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.189+0000 7f8b13dff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b0c1014b0 0x7f8b0c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:38.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.189+0000 7f8b13dff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b0c101e40 0x7f8b0c073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:38.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.189+0000 7f8b13dff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c073bc0 con 0x7f8b0c1002c0 2023-11-27T15:29:38.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.189+0000 7f8b13dff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c073d30 con 0x7f8b0c1014b0 2023-11-27T15:29:38.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.189+0000 7f8b13dff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c1a0100 con 0x7f8b0c101e40 2023-11-27T15:29:38.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.189+0000 7f8b11b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b0c1002c0 0x7f8b0c0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:38.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.189+0000 7f8b11b9b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b0c1002c0 0x7f8b0c0748a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48158/0 (socket says 172.21.15.133:48158) 2023-11-27T15:29:38.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.189+0000 7f8b11b9b700 1 -- 172.21.15.133:0/2369458387 learned_addr learned my addr 172.21.15.133:0/2369458387 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:29:38.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.189+0000 7f8b1139a700 1 --2- 172.21.15.133:0/2369458387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b0c1014b0 0x7f8b0c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:38.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.189+0000 7f8b1239c700 1 --2- 172.21.15.133:0/2369458387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b0c101e40 0x7f8b0c073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:38.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.189+0000 7f8b11b9b700 1 -- 172.21.15.133:0/2369458387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b0c101e40 msgr2=0x7f8b0c073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:38.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.189+0000 7f8b11b9b700 1 --2- 172.21.15.133:0/2369458387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b0c101e40 0x7f8b0c073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:38.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.189+0000 7f8b11b9b700 1 -- 172.21.15.133:0/2369458387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b0c1014b0 msgr2=0x7f8b0c072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:38.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.189+0000 7f8b11b9b700 1 --2- 172.21.15.133:0/2369458387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b0c1014b0 0x7f8b0c072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:38.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.189+0000 7f8b11b9b700 1 -- 172.21.15.133:0/2369458387 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b080095f0 con 0x7f8b0c1002c0 2023-11-27T15:29:38.197 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.190+0000 7f8b11b9b700 1 --2- 172.21.15.133:0/2369458387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b0c1002c0 0x7f8b0c0748a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8b08005740 tx=0x7f8b08005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:38.197 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.190+0000 7f8b02ffd700 1 -- 172.21.15.133:0/2369458387 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0801d070 con 0x7f8b0c1002c0 2023-11-27T15:29:38.197 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.190+0000 7f8b13dff700 1 -- 172.21.15.133:0/2369458387 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0c10bc90 con 0x7f8b0c1002c0 2023-11-27T15:29:38.197 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.190+0000 7f8b02ffd700 1 -- 172.21.15.133:0/2369458387 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b08021410 con 0x7f8b0c1002c0 2023-11-27T15:29:38.197 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.190+0000 7f8b02ffd700 1 -- 172.21.15.133:0/2369458387 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b08017650 con 0x7f8b0c1002c0 2023-11-27T15:29:38.197 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.190+0000 7f8b13dff700 1 -- 172.21.15.133:0/2369458387 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0c10bb00 con 0x7f8b0c1002c0 2023-11-27T15:29:38.198 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.191+0000 7f8b13dff700 1 -- 172.21.15.133:0/2369458387 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b0c107880 con 0x7f8b0c1002c0 2023-11-27T15:29:38.198 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.193+0000 7f8b02ffd700 1 -- 172.21.15.133:0/2369458387 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f8b08021580 con 0x7f8b0c1002c0 2023-11-27T15:29:38.198 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.194+0000 7f8b02ffd700 1 --2- 172.21.15.133:0/2369458387 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8af80a11d0 0x7f8af80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:38.198 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.194+0000 7f8b02ffd700 1 -- 172.21.15.133:0/2369458387 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f8b080ce7a0 con 0x7f8b0c1002c0 2023-11-27T15:29:38.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.194+0000 7f8b1139a700 1 --2- 172.21.15.133:0/2369458387 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8af80a11d0 0x7f8af80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:38.199 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.195+0000 7f8b1139a700 1 --2- 172.21.15.133:0/2369458387 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8af80a11d0 0x7f8af80a3680 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009c60 tx=0x7f8afc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:38.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.197+0000 7f8b02ffd700 1 -- 172.21.15.133:0/2369458387 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8b08097b60 con 0x7f8b0c1002c0 2023-11-27T15:29:38.276 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:37 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/95572145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:29:38.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:37 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/661578050' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-27T15:29:38.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:37 smithi133 ceph-mon[97860]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-27T15:29:38.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:37 smithi133 ceph-mon[97860]: osdmap e95: 12 total, 12 up, 12 in 2023-11-27T15:29:38.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:37 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mon-a[97836]: 2023-11-27T15:29:37.950+0000 7f00879fd700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-27T15:29:38.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:37 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/95572145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:29:38.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:37 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/661578050' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-27T15:29:38.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:37 smithi177 ceph-mon[131504]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-27T15:29:38.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:37 smithi177 ceph-mon[131504]: osdmap e95: 12 total, 12 up, 12 in 2023-11-27T15:29:38.338 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:29:38 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-c[160952]: ignoring --setuser ceph since I am not root 2023-11-27T15:29:38.338 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:29:38 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-c[160952]: ignoring --setgroup ceph since I am not root 2023-11-27T15:29:38.338 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:29:38 smithi177 ceph-mds[160976]: -- 172.21.15.177:0/3592431493 <== mon.1 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556183f9f520 con 0x556183f74c00 2023-11-27T15:29:38.339 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:29:38 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-c[160952]: starting mds.c at 2023-11-27T15:29:38.339 INFO:journalctl@ceph.mds.f.smithi177.stdout:Nov 27 15:29:38 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-f[156898]: ignoring --setuser ceph since I am not root 2023-11-27T15:29:38.339 INFO:journalctl@ceph.mds.f.smithi177.stdout:Nov 27 15:29:38 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-f[156898]: ignoring --setgroup ceph since I am not root 2023-11-27T15:29:38.339 INFO:journalctl@ceph.mds.f.smithi177.stdout:Nov 27 15:29:38 smithi177 ceph-mds[156922]: -- 172.21.15.177:0/659018153 <== mon.2 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55723f183520 con 0x55723f159000 2023-11-27T15:29:38.339 INFO:journalctl@ceph.mds.f.smithi177.stdout:Nov 27 15:29:38 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-f[156898]: starting mds.f at 2023-11-27T15:29:38.340 INFO:journalctl@ceph.mds.i.smithi177.stdout:Nov 27 15:29:38 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-i[157581]: ignoring --setuser ceph since I am not root 2023-11-27T15:29:38.340 INFO:journalctl@ceph.mds.i.smithi177.stdout:Nov 27 15:29:38 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-i[157581]: ignoring --setgroup ceph since I am not root 2023-11-27T15:29:38.340 INFO:journalctl@ceph.mds.i.smithi177.stdout:Nov 27 15:29:38 smithi177 ceph-mds[157605]: -- 172.21.15.177:0/3841887015 <== mon.1 v2:172.21.15.177:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d0390bf520 con 0x55d039095000 2023-11-27T15:29:38.340 INFO:journalctl@ceph.mds.i.smithi177.stdout:Nov 27 15:29:38 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-i[157581]: starting mds.i at 2023-11-27T15:29:38.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:37 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/95572145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-11-27T15:29:38.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:37 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/661578050' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-27T15:29:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:37 smithi174 ceph-mon[131110]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-11-27T15:29:38.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:37 smithi174 ceph-mon[131110]: osdmap e95: 12 total, 12 up, 12 in 2023-11-27T15:29:38.403 INFO:journalctl@ceph.mds.h.smithi174.stdout:Nov 27 15:29:38 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-h[156080]: ignoring --setuser ceph since I am not root 2023-11-27T15:29:38.403 INFO:journalctl@ceph.mds.h.smithi174.stdout:Nov 27 15:29:38 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-h[156080]: ignoring --setgroup ceph since I am not root 2023-11-27T15:29:38.403 INFO:journalctl@ceph.mds.h.smithi174.stdout:Nov 27 15:29:38 smithi174 ceph-mds[156104]: -- 172.21.15.174:0/31669505 <== mon.2 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5624fd625520 con 0x5624fd5fac00 2023-11-27T15:29:38.404 INFO:journalctl@ceph.mds.h.smithi174.stdout:Nov 27 15:29:38 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-h[156080]: starting mds.h at 2023-11-27T15:29:38.404 INFO:journalctl@ceph.mds.b.smithi174.stdout:Nov 27 15:29:38 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-b[156420]: ignoring --setuser ceph since I am not root 2023-11-27T15:29:38.404 INFO:journalctl@ceph.mds.b.smithi174.stdout:Nov 27 15:29:38 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-b[156420]: ignoring --setgroup ceph since I am not root 2023-11-27T15:29:38.404 INFO:journalctl@ceph.mds.b.smithi174.stdout:Nov 27 15:29:38 smithi174 ceph-mds[156444]: -- 172.21.15.174:0/1976788770 <== mon.2 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d4c385d520 con 0x55d4c3832800 2023-11-27T15:29:38.404 INFO:journalctl@ceph.mds.b.smithi174.stdout:Nov 27 15:29:38 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-b[156420]: starting mds.b at 2023-11-27T15:29:38.405 INFO:journalctl@ceph.mds.e.smithi174.stdout:Nov 27 15:29:38 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-e[155743]: ignoring --setuser ceph since I am not root 2023-11-27T15:29:38.405 INFO:journalctl@ceph.mds.e.smithi174.stdout:Nov 27 15:29:38 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-e[155743]: ignoring --setgroup ceph since I am not root 2023-11-27T15:29:38.405 INFO:journalctl@ceph.mds.e.smithi174.stdout:Nov 27 15:29:38 smithi174 ceph-mds[155767]: -- 172.21.15.174:0/2266286119 <== mon.2 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5644e80f1520 con 0x5644e80c6c00 2023-11-27T15:29:38.405 INFO:journalctl@ceph.mds.e.smithi174.stdout:Nov 27 15:29:38 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-e[155743]: starting mds.e at 2023-11-27T15:29:38.441 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.437+0000 7f8b13dff700 1 -- 172.21.15.133:0/2369458387 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8b0c02cfe0 con 0x7f8b0c1002c0 2023-11-27T15:29:38.969 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.965+0000 7f8b02ffd700 1 -- 172.21.15.133:0/2369458387 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8b0809b180 con 0x7f8b0c1002c0 2023-11-27T15:29:38.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.967+0000 7f8b13dff700 1 -- 172.21.15.133:0/2369458387 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8af80a11d0 msgr2=0x7f8af80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:38.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.967+0000 7f8b13dff700 1 --2- 172.21.15.133:0/2369458387 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8af80a11d0 0x7f8af80a3680 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009c60 tx=0x7f8afc009260 comp rx=0 tx=0).stop 2023-11-27T15:29:38.972 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.968+0000 7f8b13dff700 1 -- 172.21.15.133:0/2369458387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b0c1002c0 msgr2=0x7f8b0c0748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:38.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.968+0000 7f8b13dff700 1 --2- 172.21.15.133:0/2369458387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b0c1002c0 0x7f8b0c0748a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8b08005740 tx=0x7f8b08005770 comp rx=0 tx=0).stop 2023-11-27T15:29:38.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.968+0000 7f8b13dff700 1 -- 172.21.15.133:0/2369458387 shutdown_connections 2023-11-27T15:29:38.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.968+0000 7f8b13dff700 1 --2- 172.21.15.133:0/2369458387 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f8af80a11d0 0x7f8af80a3680 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:38.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.968+0000 7f8b13dff700 1 --2- 172.21.15.133:0/2369458387 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b0c101e40 0x7f8b0c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:38.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.968+0000 7f8b13dff700 1 --2- 172.21.15.133:0/2369458387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8b0c1002c0 0x7f8b0c0748a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:38.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.968+0000 7f8b13dff700 1 --2- 172.21.15.133:0/2369458387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b0c1014b0 0x7f8b0c072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:38.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.968+0000 7f8b13dff700 1 -- 172.21.15.133:0/2369458387 >> 172.21.15.133:0/2369458387 conn(0x7f8b0c0fb8a0 msgr2=0x7f8b0c1049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:38.973 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.968+0000 7f8b13dff700 1 -- 172.21.15.133:0/2369458387 shutdown_connections 2023-11-27T15:29:38.974 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:38.968+0000 7f8b13dff700 1 -- 172.21.15.133:0/2369458387 wait complete. 2023-11-27T15:29:38.993 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-27T15:29:39.195 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.191+0000 7f886b9ff700 1 -- 172.21.15.133:0/1025285703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88641014b0 msgr2=0x7f8864101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:39.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.191+0000 7f886b9ff700 1 --2- 172.21.15.133:0/1025285703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88641014b0 0x7f8864101900 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8854009990 tx=0x7f8854009ca0 comp rx=0 tx=0).stop 2023-11-27T15:29:39.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.191+0000 7f886b9ff700 1 -- 172.21.15.133:0/1025285703 shutdown_connections 2023-11-27T15:29:39.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.191+0000 7f886b9ff700 1 --2- 172.21.15.133:0/1025285703 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8864101e40 0x7f8864108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:39.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.191+0000 7f886b9ff700 1 --2- 172.21.15.133:0/1025285703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88641002c0 0x7f88641006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:39.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.191+0000 7f886b9ff700 1 --2- 172.21.15.133:0/1025285703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88641014b0 0x7f8864101900 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:39.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.191+0000 7f886b9ff700 1 -- 172.21.15.133:0/1025285703 >> 172.21.15.133:0/1025285703 conn(0x7f88640fb8a0 msgr2=0x7f88640fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:39.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.191+0000 7f886b9ff700 1 -- 172.21.15.133:0/1025285703 shutdown_connections 2023-11-27T15:29:39.196 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.191+0000 7f886b9ff700 1 -- 172.21.15.133:0/1025285703 wait complete. 2023-11-27T15:29:39.198 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.192+0000 7f886b9ff700 1 Processor -- start 2023-11-27T15:29:39.200 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.192+0000 7f886b9ff700 1 -- start start 2023-11-27T15:29:39.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.192+0000 7f886b9ff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88641002c0 0x7f8864198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:39.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.192+0000 7f886b9ff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88641014b0 0x7f88641987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:39.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.192+0000 7f886b9ff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8864101e40 0x7f886419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:39.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.192+0000 7f886b9ff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886419fde0 con 0x7f88641002c0 2023-11-27T15:29:39.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.192+0000 7f886b9ff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886419ff20 con 0x7f8864101e40 2023-11-27T15:29:39.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.192+0000 7f886b9ff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88641a00b0 con 0x7f88641014b0 2023-11-27T15:29:39.201 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.193+0000 7f886979b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88641002c0 0x7f8864198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:39.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.193+0000 7f886979b700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88641002c0 0x7f8864198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48190/0 (socket says 172.21.15.133:48190) 2023-11-27T15:29:39.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.193+0000 7f886979b700 1 -- 172.21.15.133:0/384935455 learned_addr learned my addr 172.21.15.133:0/384935455 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:29:39.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.193+0000 7f8868f9a700 1 --2- 172.21.15.133:0/384935455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88641014b0 0x7f88641987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:39.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.193+0000 7f8869f9c700 1 --2- 172.21.15.133:0/384935455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8864101e40 0x7f886419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:39.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.193+0000 7f886979b700 1 -- 172.21.15.133:0/384935455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88641014b0 msgr2=0x7f88641987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:39.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.193+0000 7f886979b700 1 --2- 172.21.15.133:0/384935455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88641014b0 0x7f88641987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:39.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.193+0000 7f886979b700 1 -- 172.21.15.133:0/384935455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8864101e40 msgr2=0x7f886419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:39.202 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.193+0000 7f886979b700 1 --2- 172.21.15.133:0/384935455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8864101e40 0x7f886419f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:39.203 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.193+0000 7f886979b700 1 -- 172.21.15.133:0/384935455 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88540095f0 con 0x7f88641002c0 2023-11-27T15:29:39.203 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.193+0000 7f886979b700 1 --2- 172.21.15.133:0/384935455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88641002c0 0x7f8864198260 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f886000e970 tx=0x7f886000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:39.203 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.194+0000 7f885a7fc700 1 -- 172.21.15.133:0/384935455 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8860009790 con 0x7f88641002c0 2023-11-27T15:29:39.203 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.194+0000 7f885a7fc700 1 -- 172.21.15.133:0/384935455 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f886000c7e0 con 0x7f88641002c0 2023-11-27T15:29:39.203 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.194+0000 7f886b9ff700 1 -- 172.21.15.133:0/384935455 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f886410bc70 con 0x7f88641002c0 2023-11-27T15:29:39.203 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.194+0000 7f885a7fc700 1 -- 172.21.15.133:0/384935455 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8860003c70 con 0x7f88641002c0 2023-11-27T15:29:39.203 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.194+0000 7f886b9ff700 1 -- 172.21.15.133:0/384935455 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f886410bae0 con 0x7f88641002c0 2023-11-27T15:29:39.204 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.195+0000 7f886b9ff700 1 -- 172.21.15.133:0/384935455 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8864067b00 con 0x7f88641002c0 2023-11-27T15:29:39.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.197+0000 7f885a7fc700 1 -- 172.21.15.133:0/384935455 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f886000cc50 con 0x7f88641002c0 2023-11-27T15:29:39.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.198+0000 7f885a7fc700 1 --2- 172.21.15.133:0/384935455 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f88500a1150 0x7f88500a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:39.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.198+0000 7f885a7fc700 1 -- 172.21.15.133:0/384935455 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9336+0+0 (secure 0 0 0) 0x7f8860014070 con 0x7f88641002c0 2023-11-27T15:29:39.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.198+0000 7f8868f9a700 1 --2- 172.21.15.133:0/384935455 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f88500a1150 0x7f88500a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:39.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.199+0000 7f8868f9a700 1 --2- 172.21.15.133:0/384935455 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f88500a1150 0x7f88500a3600 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f885400b2e0 tx=0x7f8854005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:39.205 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.200+0000 7f885a7fc700 1 -- 172.21.15.133:0/384935455 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8860096350 con 0x7f88641002c0 2023-11-27T15:29:39.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:38 smithi133 ceph-mon[97860]: pgmap v553: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 307 B/s wr, 8 op/s 2023-11-27T15:29:39.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:38 smithi133 ceph-mon[97860]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-11-27T15:29:39.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:38 smithi133 ceph-mon[97860]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-27T15:29:39.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:38 smithi133 ceph-mon[97860]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-11-27T15:29:39.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:38 smithi133 ceph-mon[97860]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-11-27T15:29:39.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:38 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2369458387' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:29:39.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:38 smithi177 ceph-mon[131504]: pgmap v553: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 307 B/s wr, 8 op/s 2023-11-27T15:29:39.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:38 smithi177 ceph-mon[131504]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-11-27T15:29:39.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:38 smithi177 ceph-mon[131504]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-27T15:29:39.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:38 smithi177 ceph-mon[131504]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-11-27T15:29:39.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:38 smithi177 ceph-mon[131504]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-11-27T15:29:39.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:38 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2369458387' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:29:39.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:38 smithi174 ceph-mon[131110]: pgmap v553: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 307 B/s wr, 8 op/s 2023-11-27T15:29:39.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:38 smithi174 ceph-mon[131110]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-11-27T15:29:39.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:38 smithi174 ceph-mon[131110]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-27T15:29:39.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:38 smithi174 ceph-mon[131110]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-11-27T15:29:39.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:38 smithi174 ceph-mon[131110]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-11-27T15:29:39.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:38 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2369458387' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-11-27T15:29:39.406 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.401+0000 7f886b9ff700 1 -- 172.21.15.133:0/384935455 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f88641a0c20 con 0x7f88641002c0 2023-11-27T15:29:39.976 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.972+0000 7f885a7fc700 1 -- 172.21.15.133:0/384935455 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8860003dd0 con 0x7f88641002c0 2023-11-27T15:29:39.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.975+0000 7f886b9ff700 1 -- 172.21.15.133:0/384935455 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f88500a1150 msgr2=0x7f88500a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:39.980 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.975+0000 7f886b9ff700 1 --2- 172.21.15.133:0/384935455 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f88500a1150 0x7f88500a3600 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f885400b2e0 tx=0x7f8854005fb0 comp rx=0 tx=0).stop 2023-11-27T15:29:39.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.975+0000 7f886b9ff700 1 -- 172.21.15.133:0/384935455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88641002c0 msgr2=0x7f8864198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:39.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.975+0000 7f886b9ff700 1 --2- 172.21.15.133:0/384935455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88641002c0 0x7f8864198260 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f886000e970 tx=0x7f886000ec80 comp rx=0 tx=0).stop 2023-11-27T15:29:39.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.976+0000 7f886b9ff700 1 -- 172.21.15.133:0/384935455 shutdown_connections 2023-11-27T15:29:39.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.976+0000 7f886b9ff700 1 --2- 172.21.15.133:0/384935455 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f88500a1150 0x7f88500a3600 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:39.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.976+0000 7f886b9ff700 1 --2- 172.21.15.133:0/384935455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8864101e40 0x7f886419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:39.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.976+0000 7f886b9ff700 1 --2- 172.21.15.133:0/384935455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88641002c0 0x7f8864198260 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:39.981 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.976+0000 7f886b9ff700 1 --2- 172.21.15.133:0/384935455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88641014b0 0x7f88641987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:39.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.976+0000 7f886b9ff700 1 -- 172.21.15.133:0/384935455 >> 172.21.15.133:0/384935455 conn(0x7f88640fb8a0 msgr2=0x7f88640fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:39.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.976+0000 7f886b9ff700 1 -- 172.21.15.133:0/384935455 shutdown_connections 2023-11-27T15:29:39.982 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:39.976+0000 7f886b9ff700 1 -- 172.21.15.133:0/384935455 wait complete. 2023-11-27T15:29:39.982 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_metadata' removed 2023-11-27T15:29:40.011 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-27T15:29:40.241 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.237+0000 7f443359e700 1 -- 172.21.15.133:0/1081350117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4434101310 msgr2=0x7f4434101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:40.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.237+0000 7f443359e700 1 --2- 172.21.15.133:0/1081350117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4434101310 0x7f4434101780 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f441c009990 tx=0x7f441c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:29:40.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.237+0000 7f443359e700 1 -- 172.21.15.133:0/1081350117 shutdown_connections 2023-11-27T15:29:40.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.237+0000 7f443359e700 1 --2- 172.21.15.133:0/1081350117 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4434101cc0 0x7f4434108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:40.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.237+0000 7f443359e700 1 --2- 172.21.15.133:0/1081350117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4434100080 0x7f4434100490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:40.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.237+0000 7f443359e700 1 --2- 172.21.15.133:0/1081350117 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4434101310 0x7f4434101780 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:40.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.237+0000 7f443359e700 1 -- 172.21.15.133:0/1081350117 >> 172.21.15.133:0/1081350117 conn(0x7f44340fb6a0 msgr2=0x7f44340fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:40.242 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.237+0000 7f443359e700 1 -- 172.21.15.133:0/1081350117 shutdown_connections 2023-11-27T15:29:40.243 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.237+0000 7f443359e700 1 -- 172.21.15.133:0/1081350117 wait complete. 2023-11-27T15:29:40.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.238+0000 7f443359e700 1 Processor -- start 2023-11-27T15:29:40.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.238+0000 7f443359e700 1 -- start start 2023-11-27T15:29:40.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.238+0000 7f443359e700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4434100080 0x7f4434195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:40.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.238+0000 7f443359e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4434101310 0x7f4434196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:40.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.238+0000 7f443359e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4434101cc0 0x7f443419d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:40.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.238+0000 7f443359e700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443419d960 con 0x7f4434100080 2023-11-27T15:29:40.245 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.238+0000 7f443359e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443419daa0 con 0x7f4434101310 2023-11-27T15:29:40.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.238+0000 7f443359e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443419dc30 con 0x7f4434101cc0 2023-11-27T15:29:40.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.239+0000 7f443259c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4434100080 0x7f4434195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:40.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.239+0000 7f442bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4434101310 0x7f4434196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:40.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.239+0000 7f4432d9d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4434101cc0 0x7f443419d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:40.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.239+0000 7f443259c700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4434100080 0x7f4434195de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48218/0 (socket says 172.21.15.133:48218) 2023-11-27T15:29:40.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.239+0000 7f443259c700 1 -- 172.21.15.133:0/2255311458 learned_addr learned my addr 172.21.15.133:0/2255311458 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:29:40.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.239+0000 7f4432d9d700 1 -- 172.21.15.133:0/2255311458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4434101310 msgr2=0x7f4434196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:40.246 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.239+0000 7f4432d9d700 1 --2- 172.21.15.133:0/2255311458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4434101310 0x7f4434196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:40.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.239+0000 7f4432d9d700 1 -- 172.21.15.133:0/2255311458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4434100080 msgr2=0x7f4434195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:40.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.239+0000 7f4432d9d700 1 --2- 172.21.15.133:0/2255311458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4434100080 0x7f4434195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:40.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.239+0000 7f4432d9d700 1 -- 172.21.15.133:0/2255311458 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441c0095f0 con 0x7f4434101cc0 2023-11-27T15:29:40.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.239+0000 7f442bfff700 1 --2- 172.21.15.133:0/2255311458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4434101310 0x7f4434196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:29:40.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.239+0000 7f443259c700 1 --2- 172.21.15.133:0/2255311458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4434100080 0x7f4434195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-27T15:29:40.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.239+0000 7f4432d9d700 1 --2- 172.21.15.133:0/2255311458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4434101cc0 0x7f443419d390 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f442c00efc0 tx=0x7f442c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:40.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.240+0000 7f442b7fe700 1 -- 172.21.15.133:0/2255311458 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c018070 con 0x7f4434101cc0 2023-11-27T15:29:40.247 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.240+0000 7f442b7fe700 1 -- 172.21.15.133:0/2255311458 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f442c00f040 con 0x7f4434101cc0 2023-11-27T15:29:40.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.240+0000 7f443359e700 1 -- 172.21.15.133:0/2255311458 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443410bad0 con 0x7f4434101cc0 2023-11-27T15:29:40.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.240+0000 7f442b7fe700 1 -- 172.21.15.133:0/2255311458 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c00cd40 con 0x7f4434101cc0 2023-11-27T15:29:40.248 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.240+0000 7f443359e700 1 -- 172.21.15.133:0/2255311458 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443410b940 con 0x7f4434101cc0 2023-11-27T15:29:40.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.241+0000 7f443359e700 1 -- 172.21.15.133:0/2255311458 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4434107610 con 0x7f4434101cc0 2023-11-27T15:29:40.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.243+0000 7f442b7fe700 1 -- 172.21.15.133:0/2255311458 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7f442c003a00 con 0x7f4434101cc0 2023-11-27T15:29:40.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.244+0000 7f442b7fe700 1 --2- 172.21.15.133:0/2255311458 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f44200a1150 0x7f44200a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:40.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.244+0000 7f442b7fe700 1 -- 172.21.15.133:0/2255311458 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8811+0+0 (secure 0 0 0) 0x7f442c0cc8b0 con 0x7f4434101cc0 2023-11-27T15:29:40.249 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.244+0000 7f443259c700 1 --2- 172.21.15.133:0/2255311458 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f44200a1150 0x7f44200a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:40.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.245+0000 7f443259c700 1 --2- 172.21.15.133:0/2255311458 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f44200a1150 0x7f44200a3600 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4424011550 tx=0x7f4424009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:40.250 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.247+0000 7f442b7fe700 1 -- 172.21.15.133:0/2255311458 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f442c095df0 con 0x7f4434101cc0 2023-11-27T15:29:40.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:39 smithi133 ceph-mon[97860]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-11-27T15:29:40.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:39 smithi133 ceph-mon[97860]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-11-27T15:29:40.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:39 smithi133 ceph-mon[97860]: Cluster is now healthy 2023-11-27T15:29:40.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:39 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.177:6834/1379036903,v1:172.21.15.177:6835/1379036903] up:boot 2023-11-27T15:29:40.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:39 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.177:6836/2687503193,v1:172.21.15.177:6837/2687503193] up:boot 2023-11-27T15:29:40.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:39 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.177:6838/3305349904,v1:172.21.15.177:6839/3305349904] up:boot 2023-11-27T15:29:40.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:39 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.174:6838/3386738899,v1:172.21.15.174:6839/3386738899] up:boot 2023-11-27T15:29:40.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:39 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.174:6834/4237477000,v1:172.21.15.174:6836/4237477000] up:boot 2023-11-27T15:29:40.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:39 smithi133 ceph-mon[97860]: mds.? [v2:172.21.15.174:6835/4200827608,v1:172.21.15.174:6837/4200827608] up:boot 2023-11-27T15:29:40.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:39 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2369458387' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-11-27T15:29:40.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:39 smithi133 ceph-mon[97860]: fsmap 12 up:standby 2023-11-27T15:29:40.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:39 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:29:40.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:39 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-27T15:29:40.278 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:39 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:29:40.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:39 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-27T15:29:40.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:39 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-27T15:29:40.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:39 smithi133 ceph-mon[97860]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:29:40.279 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:39 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/384935455' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:29:40.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:39 smithi177 ceph-mon[131504]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-11-27T15:29:40.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:39 smithi177 ceph-mon[131504]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-11-27T15:29:40.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:39 smithi177 ceph-mon[131504]: Cluster is now healthy 2023-11-27T15:29:40.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:39 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.177:6834/1379036903,v1:172.21.15.177:6835/1379036903] up:boot 2023-11-27T15:29:40.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:39 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.177:6836/2687503193,v1:172.21.15.177:6837/2687503193] up:boot 2023-11-27T15:29:40.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:39 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.177:6838/3305349904,v1:172.21.15.177:6839/3305349904] up:boot 2023-11-27T15:29:40.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:39 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.174:6838/3386738899,v1:172.21.15.174:6839/3386738899] up:boot 2023-11-27T15:29:40.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:39 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.174:6834/4237477000,v1:172.21.15.174:6836/4237477000] up:boot 2023-11-27T15:29:40.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:39 smithi177 ceph-mon[131504]: mds.? [v2:172.21.15.174:6835/4200827608,v1:172.21.15.174:6837/4200827608] up:boot 2023-11-27T15:29:40.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:39 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2369458387' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-11-27T15:29:40.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:39 smithi177 ceph-mon[131504]: fsmap 12 up:standby 2023-11-27T15:29:40.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:39 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:29:40.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:39 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-27T15:29:40.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:39 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:29:40.339 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:39 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-27T15:29:40.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:39 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-27T15:29:40.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:39 smithi177 ceph-mon[131504]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:29:40.340 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:39 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/384935455' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:29:40.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:39 smithi174 ceph-mon[131110]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-11-27T15:29:40.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:39 smithi174 ceph-mon[131110]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-11-27T15:29:40.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:39 smithi174 ceph-mon[131110]: Cluster is now healthy 2023-11-27T15:29:40.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:39 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.177:6834/1379036903,v1:172.21.15.177:6835/1379036903] up:boot 2023-11-27T15:29:40.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:39 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.177:6836/2687503193,v1:172.21.15.177:6837/2687503193] up:boot 2023-11-27T15:29:40.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:39 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.177:6838/3305349904,v1:172.21.15.177:6839/3305349904] up:boot 2023-11-27T15:29:40.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:39 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.174:6838/3386738899,v1:172.21.15.174:6839/3386738899] up:boot 2023-11-27T15:29:40.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:39 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.174:6834/4237477000,v1:172.21.15.174:6836/4237477000] up:boot 2023-11-27T15:29:40.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:39 smithi174 ceph-mon[131110]: mds.? [v2:172.21.15.174:6835/4200827608,v1:172.21.15.174:6837/4200827608] up:boot 2023-11-27T15:29:40.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:39 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2369458387' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-11-27T15:29:40.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:39 smithi174 ceph-mon[131110]: fsmap 12 up:standby 2023-11-27T15:29:40.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:39 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-11-27T15:29:40.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:39 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-11-27T15:29:40.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:39 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-11-27T15:29:40.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:39 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-11-27T15:29:40.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:39 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-11-27T15:29:40.404 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:39 smithi174 ceph-mon[131110]: from='mgr.14150 172.21.15.133:0/1546804433' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-11-27T15:29:40.405 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:39 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/384935455' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:29:40.453 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.448+0000 7f443359e700 1 -- 172.21.15.133:0/2255311458 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4434061900 con 0x7f4434101cc0 2023-11-27T15:29:40.985 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.980+0000 7f442b7fe700 1 -- 172.21.15.133:0/2255311458 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) v1 ==== 169+0+0 (secure 0 0 0) 0x7f442c099410 con 0x7f4434101cc0 2023-11-27T15:29:40.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.986+0000 7f443359e700 1 -- 172.21.15.133:0/2255311458 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f44200a1150 msgr2=0x7f44200a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:40.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.986+0000 7f443359e700 1 --2- 172.21.15.133:0/2255311458 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f44200a1150 0x7f44200a3600 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4424011550 tx=0x7f4424009260 comp rx=0 tx=0).stop 2023-11-27T15:29:40.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.986+0000 7f443359e700 1 -- 172.21.15.133:0/2255311458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4434101cc0 msgr2=0x7f443419d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:40.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.986+0000 7f443359e700 1 --2- 172.21.15.133:0/2255311458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4434101cc0 0x7f443419d390 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f442c00efc0 tx=0x7f442c00c3a0 comp rx=0 tx=0).stop 2023-11-27T15:29:40.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.986+0000 7f443359e700 1 -- 172.21.15.133:0/2255311458 shutdown_connections 2023-11-27T15:29:40.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.986+0000 7f443359e700 1 --2- 172.21.15.133:0/2255311458 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7f44200a1150 0x7f44200a3600 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:40.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.986+0000 7f443359e700 1 --2- 172.21.15.133:0/2255311458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4434101cc0 0x7f443419d390 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:40.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.986+0000 7f443359e700 1 --2- 172.21.15.133:0/2255311458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4434100080 0x7f4434195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:40.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.986+0000 7f443359e700 1 --2- 172.21.15.133:0/2255311458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4434101310 0x7f4434196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:40.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.986+0000 7f443359e700 1 -- 172.21.15.133:0/2255311458 >> 172.21.15.133:0/2255311458 conn(0x7f44340fb6a0 msgr2=0x7f4434104750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:40.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.986+0000 7f443359e700 1 -- 172.21.15.133:0/2255311458 shutdown_connections 2023-11-27T15:29:40.991 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:40.986+0000 7f443359e700 1 -- 172.21.15.133:0/2255311458 wait complete. 2023-11-27T15:29:40.994 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data' removed 2023-11-27T15:29:41.030 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-11-27T15:29:41.263 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.260+0000 7ff92ddff700 1 -- 172.21.15.133:0/3924665654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9281014b0 msgr2=0x7ff928101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:41.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.260+0000 7ff92ddff700 1 --2- 172.21.15.133:0/3924665654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9281014b0 0x7ff928101900 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009990 tx=0x7ff91c009ca0 comp rx=0 tx=0).stop 2023-11-27T15:29:41.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.260+0000 7ff92ddff700 1 -- 172.21.15.133:0/3924665654 shutdown_connections 2023-11-27T15:29:41.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.260+0000 7ff92ddff700 1 --2- 172.21.15.133:0/3924665654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff928101e40 0x7ff928108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:41.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.260+0000 7ff92ddff700 1 --2- 172.21.15.133:0/3924665654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9281002c0 0x7ff9281006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:41.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.260+0000 7ff92ddff700 1 --2- 172.21.15.133:0/3924665654 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9281014b0 0x7ff928101900 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:41.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.260+0000 7ff92ddff700 1 -- 172.21.15.133:0/3924665654 >> 172.21.15.133:0/3924665654 conn(0x7ff9280fb8a0 msgr2=0x7ff9280fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:41.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.260+0000 7ff92ddff700 1 -- 172.21.15.133:0/3924665654 shutdown_connections 2023-11-27T15:29:41.264 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.260+0000 7ff92ddff700 1 -- 172.21.15.133:0/3924665654 wait complete. 2023-11-27T15:29:41.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.261+0000 7ff92ddff700 1 Processor -- start 2023-11-27T15:29:41.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.261+0000 7ff92ddff700 1 -- start start 2023-11-27T15:29:41.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.261+0000 7ff92ddff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9281002c0 0x7ff9280748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:41.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.261+0000 7ff92ddff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9281014b0 0x7ff928072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:41.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.261+0000 7ff92ddff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff928101e40 0x7ff928073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:41.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.261+0000 7ff92ddff700 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff928073bc0 con 0x7ff928101e40 2023-11-27T15:29:41.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.261+0000 7ff92ddff700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff928073d30 con 0x7ff9281014b0 2023-11-27T15:29:41.267 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.261+0000 7ff92ddff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9281a00a0 con 0x7ff9281002c0 2023-11-27T15:29:41.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.262+0000 7ff927fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff928101e40 0x7ff928073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:41.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.262+0000 7ff927fff700 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff928101e40 0x7ff928073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:48232/0 (socket says 172.21.15.133:48232) 2023-11-27T15:29:41.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.262+0000 7ff927fff700 1 -- 172.21.15.133:0/2047858070 learned_addr learned my addr 172.21.15.133:0/2047858070 (peer_addr_for_me v2:172.21.15.133:0/0) 2023-11-27T15:29:41.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.262+0000 7ff9277fe700 1 --2- 172.21.15.133:0/2047858070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9281002c0 0x7ff9280748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:41.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.262+0000 7ff926ffd700 1 --2- 172.21.15.133:0/2047858070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9281014b0 0x7ff928072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:41.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.262+0000 7ff927fff700 1 -- 172.21.15.133:0/2047858070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9281002c0 msgr2=0x7ff9280748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:41.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.262+0000 7ff927fff700 1 --2- 172.21.15.133:0/2047858070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9281002c0 0x7ff9280748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:41.268 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.262+0000 7ff927fff700 1 -- 172.21.15.133:0/2047858070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9281014b0 msgr2=0x7ff928072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:41.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.262+0000 7ff927fff700 1 --2- 172.21.15.133:0/2047858070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9281014b0 0x7ff928072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:41.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.262+0000 7ff927fff700 1 -- 172.21.15.133:0/2047858070 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91c0095f0 con 0x7ff928101e40 2023-11-27T15:29:41.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.262+0000 7ff927fff700 1 --2- 172.21.15.133:0/2047858070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff928101e40 0x7ff928073420 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff92000ef30 tx=0x7ff92000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:41.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.263+0000 7ff924ff9700 1 -- 172.21.15.133:0/2047858070 <== mon.0 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff920018070 con 0x7ff928101e40 2023-11-27T15:29:41.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.263+0000 7ff924ff9700 1 -- 172.21.15.133:0/2047858070 <== mon.0 v2:172.21.15.133:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff92000f040 con 0x7ff928101e40 2023-11-27T15:29:41.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.263+0000 7ff92ddff700 1 -- 172.21.15.133:0/2047858070 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92810bc70 con 0x7ff928101e40 2023-11-27T15:29:41.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.263+0000 7ff924ff9700 1 -- 172.21.15.133:0/2047858070 <== mon.0 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92000cd00 con 0x7ff928101e40 2023-11-27T15:29:41.269 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.263+0000 7ff92ddff700 1 -- 172.21.15.133:0/2047858070 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92810bae0 con 0x7ff928101e40 2023-11-27T15:29:41.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.264+0000 7ff92ddff700 1 -- 172.21.15.133:0/2047858070 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9281071f0 con 0x7ff928101e40 2023-11-27T15:29:41.270 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.266+0000 7ff924ff9700 1 -- 172.21.15.133:0/2047858070 <== mon.0 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 18) v1 ==== 135716+0+0 (secure 0 0 0) 0x7ff920003a00 con 0x7ff928101e40 2023-11-27T15:29:41.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.267+0000 7ff924ff9700 1 --2- 172.21.15.133:0/2047858070 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff9100a11d0 0x7ff9100a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-27T15:29:41.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.267+0000 7ff924ff9700 1 -- 172.21.15.133:0/2047858070 <== mon.0 v2:172.21.15.133:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8252+0+0 (secure 0 0 0) 0x7ff9200cc3c0 con 0x7ff928101e40 2023-11-27T15:29:41.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.267+0000 7ff9277fe700 1 --2- 172.21.15.133:0/2047858070 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff9100a11d0 0x7ff9100a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-27T15:29:41.271 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.268+0000 7ff9277fe700 1 --2- 172.21.15.133:0/2047858070 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff9100a11d0 0x7ff9100a3680 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff928101310 tx=0x7ff918009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-27T15:29:41.274 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.271+0000 7ff924ff9700 1 -- 172.21.15.133:0/2047858070 <== mon.0 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff920095b90 con 0x7ff928101e40 2023-11-27T15:29:41.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:40 smithi133 ceph-mon[97860]: pgmap v554: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 716 B/s wr, 8 op/s 2023-11-27T15:29:41.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:40 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/384935455' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-11-27T15:29:41.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:40 smithi133 ceph-mon[97860]: osdmap e96: 12 total, 12 up, 12 in 2023-11-27T15:29:41.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:40 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2255311458' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:29:41.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:40 smithi133 ceph-mon[97860]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:29:41.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:40 smithi177 ceph-mon[131504]: pgmap v554: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 716 B/s wr, 8 op/s 2023-11-27T15:29:41.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:40 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/384935455' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-11-27T15:29:41.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:40 smithi177 ceph-mon[131504]: osdmap e96: 12 total, 12 up, 12 in 2023-11-27T15:29:41.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:40 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2255311458' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:29:41.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:40 smithi177 ceph-mon[131504]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:29:41.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:40 smithi174 ceph-mon[131110]: pgmap v554: 129 pgs: 129 active+clean; 427 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 716 B/s wr, 8 op/s 2023-11-27T15:29:41.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:40 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/384935455' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-11-27T15:29:41.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:40 smithi174 ceph-mon[131110]: osdmap e96: 12 total, 12 up, 12 in 2023-11-27T15:29:41.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:40 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2255311458' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:29:41.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:40 smithi174 ceph-mon[131110]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:29:41.473 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.470+0000 7ff92ddff700 1 -- 172.21.15.133:0/2047858070 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff92802cfe0 con 0x7ff928101e40 2023-11-27T15:29:41.990 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.986+0000 7ff924ff9700 1 -- 172.21.15.133:0/2047858070 <== mon.0 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v98) v1 ==== 178+0+0 (secure 0 0 0) 0x7ff9200991b0 con 0x7ff928101e40 2023-11-27T15:29:41.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.996+0000 7ff92ddff700 1 -- 172.21.15.133:0/2047858070 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff9100a11d0 msgr2=0x7ff9100a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:41.999 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.996+0000 7ff92ddff700 1 --2- 172.21.15.133:0/2047858070 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff9100a11d0 0x7ff9100a3680 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff928101310 tx=0x7ff918009260 comp rx=0 tx=0).stop 2023-11-27T15:29:42.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.996+0000 7ff92ddff700 1 -- 172.21.15.133:0/2047858070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff928101e40 msgr2=0x7ff928073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-27T15:29:42.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.996+0000 7ff92ddff700 1 --2- 172.21.15.133:0/2047858070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff928101e40 0x7ff928073420 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff92000ef30 tx=0x7ff92000c3a0 comp rx=0 tx=0).stop 2023-11-27T15:29:42.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.996+0000 7ff92ddff700 1 -- 172.21.15.133:0/2047858070 shutdown_connections 2023-11-27T15:29:42.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.996+0000 7ff92ddff700 1 --2- 172.21.15.133:0/2047858070 >> [v2:172.21.15.133:6800/4209056668,v1:172.21.15.133:6801/4209056668] conn(0x7ff9100a11d0 0x7ff9100a3680 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:42.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.996+0000 7ff92ddff700 1 --2- 172.21.15.133:0/2047858070 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff928101e40 0x7ff928073420 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:42.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.996+0000 7ff92ddff700 1 --2- 172.21.15.133:0/2047858070 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff9281002c0 0x7ff9280748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:42.000 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.996+0000 7ff92ddff700 1 --2- 172.21.15.133:0/2047858070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9281014b0 0x7ff928072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-27T15:29:42.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.996+0000 7ff92ddff700 1 -- 172.21.15.133:0/2047858070 >> 172.21.15.133:0/2047858070 conn(0x7ff9280fb8a0 msgr2=0x7ff9280fdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-27T15:29:42.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.996+0000 7ff92ddff700 1 -- 172.21.15.133:0/2047858070 shutdown_connections 2023-11-27T15:29:42.001 INFO:teuthology.orchestra.run.smithi133.stderr:2023-11-27T15:29:41.996+0000 7ff92ddff700 1 -- 172.21.15.133:0/2047858070 wait complete. 2023-11-27T15:29:42.004 INFO:teuthology.orchestra.run.smithi133.stderr:pool 'cephfs_data_ec' removed 2023-11-27T15:29:42.036 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:29:42.155 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:29:42.186 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:29:42.219 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-11-27T15:29:42.219 DEBUG:teuthology.orchestra.run.smithi133:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-11-27T15:29:42.263 DEBUG:teuthology.orchestra.run.smithi174:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-11-27T15:29:42.276 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:41 smithi133 ceph-mon[97860]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-11-27T15:29:42.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:41 smithi133 ceph-mon[97860]: osdmap e97: 12 total, 12 up, 12 in 2023-11-27T15:29:42.277 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:41 smithi133 ceph-mon[97860]: from='client.? 172.21.15.133:0/2047858070' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:29:42.279 DEBUG:teuthology.orchestra.run.smithi177:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-11-27T15:29:42.296 INFO:tasks.cephadm:Stopping all daemons... 2023-11-27T15:29:42.296 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-11-27T15:29:42.296 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mon.a 2023-11-27T15:29:42.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:41 smithi177 ceph-mon[131504]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-11-27T15:29:42.337 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:41 smithi177 ceph-mon[131504]: osdmap e97: 12 total, 12 up, 12 in 2023-11-27T15:29:42.338 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:41 smithi177 ceph-mon[131504]: from='client.? 172.21.15.133:0/2047858070' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:29:42.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:41 smithi174 ceph-mon[131110]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-11-27T15:29:42.402 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:41 smithi174 ceph-mon[131110]: osdmap e97: 12 total, 12 up, 12 in 2023-11-27T15:29:42.403 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:41 smithi174 ceph-mon[131110]: from='client.? 172.21.15.133:0/2047858070' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-11-27T15:29:42.777 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:42 smithi133 systemd[1]: Stopping Ceph mon.a for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:29:43.120 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:42 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mon-a[97836]: 2023-11-27T15:29:42.852+0000 7f008aa03700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:29:43.120 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:42 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mon-a[97836]: 2023-11-27T15:29:42.852+0000 7f008aa03700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-11-27T15:29:43.527 INFO:journalctl@ceph.mon.a.smithi133.stdout:Nov 27 15:29:43 smithi133 bash[258382]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mon-a 2023-11-27T15:29:43.614 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mon.a.service' 2023-11-27T15:29:43.710 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:29:43.710 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-11-27T15:29:43.710 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-11-27T15:29:43.710 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mon.b 2023-11-27T15:29:44.077 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:43 smithi174 systemd[1]: Stopping Ceph mon.b for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:29:44.343 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:44 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mon-b[131086]: 2023-11-27T15:29:44.072+0000 7fefd842e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:29:44.343 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:44 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mon-b[131086]: 2023-11-27T15:29:44.072+0000 7fefd842e700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-11-27T15:29:44.652 INFO:journalctl@ceph.mon.b.smithi174.stdout:Nov 27 15:29:44 smithi174 bash[160400]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mon-b 2023-11-27T15:29:44.743 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mon.b.service' 2023-11-27T15:29:44.820 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:29:44.821 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-11-27T15:29:44.821 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-11-27T15:29:44.821 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mon.c 2023-11-27T15:29:45.190 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:44 smithi177 systemd[1]: Stopping Ceph mon.c for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:29:45.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:45 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mon-c[131473]: 2023-11-27T15:29:45.186+0000 7ff221ae3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:29:45.588 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:45 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mon-c[131473]: 2023-11-27T15:29:45.186+0000 7ff221ae3700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-11-27T15:29:45.922 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mon.c.service' 2023-11-27T15:29:45.946 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:45 smithi177 bash[162126]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mon-c 2023-11-27T15:29:45.947 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:45 smithi177 systemd[1]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mon.c.service: Succeeded. 2023-11-27T15:29:45.947 INFO:journalctl@ceph.mon.c.smithi177.stdout:Nov 27 15:29:45 smithi177 systemd[1]: Stopped Ceph mon.c for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:29:45.980 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:29:45.980 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-11-27T15:29:45.980 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-11-27T15:29:45.981 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mgr.x 2023-11-27T15:29:46.277 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:29:46 smithi133 systemd[1]: Stopping Ceph mgr.x for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:29:46.766 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:29:46 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:29:46.510+0000 7f98c0735700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:29:46.766 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:29:46 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x[98106]: 2023-11-27T15:29:46.510+0000 7f98c0735700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-11-27T15:29:47.206 INFO:journalctl@ceph.mgr.x.smithi133.stdout:Nov 27 15:29:46 smithi133 bash[258537]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-x 2023-11-27T15:29:47.236 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mgr.x.service' 2023-11-27T15:29:47.326 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:29:47.326 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-11-27T15:29:47.326 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-11-27T15:29:47.326 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mgr.y 2023-11-27T15:29:47.652 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:29:47 smithi174 systemd[1]: Stopping Ceph mgr.y for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:29:48.035 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:29:47 smithi174 bash[160565]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-y 2023-11-27T15:29:48.035 INFO:journalctl@ceph.mgr.y.smithi174.stdout:Nov 27 15:29:47 smithi174 systemd[1]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-11-27T15:29:48.220 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mgr.y.service' 2023-11-27T15:29:48.256 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:29:48.256 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-11-27T15:29:48.256 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-11-27T15:29:48.257 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mgr.z 2023-11-27T15:29:48.588 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:29:48 smithi177 systemd[1]: Stopping Ceph mgr.z for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:29:48.964 INFO:journalctl@ceph.mgr.z.smithi177.stdout:Nov 27 15:29:48 smithi177 bash[162288]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mgr-z 2023-11-27T15:29:49.149 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mgr.z.service' 2023-11-27T15:29:49.186 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:29:49.187 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-11-27T15:29:49.187 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-11-27T15:29:49.187 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.0 2023-11-27T15:29:49.527 INFO:journalctl@ceph.osd.0.smithi133.stdout:Nov 27 15:29:49 smithi133 systemd[1]: Stopping Ceph osd.0 for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:29:50.027 INFO:journalctl@ceph.osd.0.smithi133.stdout:Nov 27 15:29:49 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-0[110842]: 2023-11-27T15:29:49.692+0000 7f02a450c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:29:50.027 INFO:journalctl@ceph.osd.0.smithi133.stdout:Nov 27 15:29:49 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-0[110842]: 2023-11-27T15:29:49.692+0000 7f02a450c700 -1 osd.0 98 *** Got signal Terminated *** 2023-11-27T15:29:50.028 INFO:journalctl@ceph.osd.0.smithi133.stdout:Nov 27 15:29:49 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-0[110842]: 2023-11-27T15:29:49.692+0000 7f02a450c700 -1 osd.0 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:29:55.779 INFO:journalctl@ceph.osd.0.smithi133.stdout:Nov 27 15:29:55 smithi133 bash[258684]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-0 2023-11-27T15:29:58.631 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.0.service' 2023-11-27T15:29:58.720 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:29:58.721 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-11-27T15:29:58.721 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-11-27T15:29:58.721 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.1 2023-11-27T15:29:59.152 INFO:journalctl@ceph.osd.1.smithi174.stdout:Nov 27 15:29:58 smithi174 systemd[1]: Stopping Ceph osd.1 for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:29:59.653 INFO:journalctl@ceph.osd.1.smithi174.stdout:Nov 27 15:29:59 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-1[136171]: 2023-11-27T15:29:59.290+0000 7f5020f24700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:29:59.653 INFO:journalctl@ceph.osd.1.smithi174.stdout:Nov 27 15:29:59 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-1[136171]: 2023-11-27T15:29:59.290+0000 7f5020f24700 -1 osd.1 98 *** Got signal Terminated *** 2023-11-27T15:29:59.653 INFO:journalctl@ceph.osd.1.smithi174.stdout:Nov 27 15:29:59 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-1[136171]: 2023-11-27T15:29:59.290+0000 7f5020f24700 -1 osd.1 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:30:05.152 INFO:journalctl@ceph.osd.1.smithi174.stdout:Nov 27 15:30:04 smithi174 bash[160721]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-1 2023-11-27T15:30:06.493 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.1.service' 2023-11-27T15:30:06.521 INFO:journalctl@ceph.osd.1.smithi174.stdout:Nov 27 15:30:06 smithi174 systemd[1]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.1.service: Succeeded. 2023-11-27T15:30:06.521 INFO:journalctl@ceph.osd.1.smithi174.stdout:Nov 27 15:30:06 smithi174 systemd[1]: Stopped Ceph osd.1 for 5227da4c-8d37-11ee-95a2-87774f69a715. 2023-11-27T15:30:06.536 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:30:06.536 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-11-27T15:30:06.536 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-11-27T15:30:06.537 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.2 2023-11-27T15:30:06.839 INFO:journalctl@ceph.osd.2.smithi177.stdout:Nov 27 15:30:06 smithi177 systemd[1]: Stopping Ceph osd.2 for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:30:07.337 INFO:journalctl@ceph.osd.2.smithi177.stdout:Nov 27 15:30:06 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-2[137294]: 2023-11-27T15:30:06.907+0000 7fd42bd0e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:30:07.338 INFO:journalctl@ceph.osd.2.smithi177.stdout:Nov 27 15:30:06 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-2[137294]: 2023-11-27T15:30:06.907+0000 7fd42bd0e700 -1 osd.2 98 *** Got signal Terminated *** 2023-11-27T15:30:07.338 INFO:journalctl@ceph.osd.2.smithi177.stdout:Nov 27 15:30:06 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-2[137294]: 2023-11-27T15:30:06.907+0000 7fd42bd0e700 -1 osd.2 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:30:12.837 INFO:journalctl@ceph.osd.2.smithi177.stdout:Nov 27 15:30:12 smithi177 bash[162438]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-2 2023-11-27T15:30:13.939 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.2.service' 2023-11-27T15:30:13.979 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:30:13.979 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-11-27T15:30:13.980 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-11-27T15:30:13.980 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.3 2023-11-27T15:30:14.500 INFO:journalctl@ceph.osd.3.smithi133.stdout:Nov 27 15:30:14 smithi133 systemd[1]: Stopping Ceph osd.3 for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:30:14.777 INFO:journalctl@ceph.osd.3.smithi133.stdout:Nov 27 15:30:14 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-3[116279]: 2023-11-27T15:30:14.496+0000 7fa57da42700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:30:14.777 INFO:journalctl@ceph.osd.3.smithi133.stdout:Nov 27 15:30:14 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-3[116279]: 2023-11-27T15:30:14.496+0000 7fa57da42700 -1 osd.3 98 *** Got signal Terminated *** 2023-11-27T15:30:14.777 INFO:journalctl@ceph.osd.3.smithi133.stdout:Nov 27 15:30:14 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-3[116279]: 2023-11-27T15:30:14.496+0000 7fa57da42700 -1 osd.3 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:30:16.837 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:16 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:16.410+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:51.957282+0000 front 2023-11-27T15:29:51.957264+0000 (oldest deadline 2023-11-27T15:30:16.057389+0000) 2023-11-27T15:30:17.337 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:17 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:17.066+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:17.837 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:17 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:17.419+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:51.957282+0000 front 2023-11-27T15:29:51.957264+0000 (oldest deadline 2023-11-27T15:30:16.057389+0000) 2023-11-27T15:30:18.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:17 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:17.575+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:18.437 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:18 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:18.099+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:18.837 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:18 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:18.434+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:51.957282+0000 front 2023-11-27T15:29:51.957264+0000 (oldest deadline 2023-11-27T15:30:16.057389+0000) 2023-11-27T15:30:19.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:18 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:18.602+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:19.152 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:18 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:18.794+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:19.434 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:19 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:19.096+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:19.837 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:19 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:19.431+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:51.957282+0000 front 2023-11-27T15:29:51.957264+0000 (oldest deadline 2023-11-27T15:30:16.057389+0000) 2023-11-27T15:30:20.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:19 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:19.610+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:20.152 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:19 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:19.750+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:20.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:20 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:20.060+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:20.566 INFO:journalctl@ceph.osd.3.smithi133.stdout:Nov 27 15:30:20 smithi133 bash[258958]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-3 2023-11-27T15:30:20.838 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:20 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:20.399+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:51.957282+0000 front 2023-11-27T15:29:51.957264+0000 (oldest deadline 2023-11-27T15:30:16.057389+0000) 2023-11-27T15:30:21.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:20 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:20.562+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:21.152 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:20 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:20.763+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:21.422 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:21 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:21.087+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:21.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:21 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:21.552+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:21.837 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:21 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:21.419+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:51.957282+0000 front 2023-11-27T15:29:51.957264+0000 (oldest deadline 2023-11-27T15:30:16.057389+0000) 2023-11-27T15:30:22.152 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:21 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:21.766+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:22.337 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:22 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:22.075+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:22.564 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.3.service' 2023-11-27T15:30:22.656 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:30:22.656 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-11-27T15:30:22.657 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-11-27T15:30:22.657 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.4 2023-11-27T15:30:22.838 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:22 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:22.383+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:51.957282+0000 front 2023-11-27T15:29:51.957264+0000 (oldest deadline 2023-11-27T15:30:16.057389+0000) 2023-11-27T15:30:23.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:22 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:22.596+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:23.036 INFO:journalctl@ceph.osd.4.smithi174.stdout:Nov 27 15:30:22 smithi174 systemd[1]: Stopping Ceph osd.4 for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:30:23.037 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:22 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:22.728+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:23.402 INFO:journalctl@ceph.osd.4.smithi174.stdout:Nov 27 15:30:23 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-4[141652]: 2023-11-27T15:30:23.033+0000 7fe40674f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:30:23.402 INFO:journalctl@ceph.osd.4.smithi174.stdout:Nov 27 15:30:23 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-4[141652]: 2023-11-27T15:30:23.033+0000 7fe40674f700 -1 osd.4 98 *** Got signal Terminated *** 2023-11-27T15:30:23.403 INFO:journalctl@ceph.osd.4.smithi174.stdout:Nov 27 15:30:23 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-4[141652]: 2023-11-27T15:30:23.033+0000 7fe40674f700 -1 osd.4 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:30:23.431 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:23 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:23.092+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:23.838 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:23 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:23.427+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:51.957282+0000 front 2023-11-27T15:29:51.957264+0000 (oldest deadline 2023-11-27T15:30:16.057389+0000) 2023-11-27T15:30:24.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:23 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:23.612+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:24.152 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:23 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:23.707+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:24.403 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:24 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:24.135+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:24.838 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:24 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:24.400+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:51.957282+0000 front 2023-11-27T15:29:51.957264+0000 (oldest deadline 2023-11-27T15:30:16.057389+0000) 2023-11-27T15:30:25.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:24 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:24.652+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:25.152 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:24 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:24.717+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:25.400 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:25 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:25.123+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:25.837 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:25 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:25.397+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:51.957282+0000 front 2023-11-27T15:29:51.957264+0000 (oldest deadline 2023-11-27T15:30:16.057389+0000) 2023-11-27T15:30:26.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:25 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:25.702+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:26.152 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:25 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:25.654+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:26.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:25 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:25.734+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:26.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:25 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:25.734+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:26.428 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:26 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:26.136+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:26.714 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:26 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:26.425+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:51.957282+0000 front 2023-11-27T15:29:51.957264+0000 (oldest deadline 2023-11-27T15:30:16.057389+0000) 2023-11-27T15:30:26.902 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:26 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:26.633+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:26.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:26 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:26.721+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:26.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:26 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:26.721+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:27.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:26 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:26.695+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:27.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:26 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:26.710+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:27.450 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:27 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:27.106+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:27.837 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:27 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:27.447+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:51.957282+0000 front 2023-11-27T15:29:51.957264+0000 (oldest deadline 2023-11-27T15:30:16.057389+0000) 2023-11-27T15:30:27.838 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:27 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:27.447+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:01.958748+0000 front 2023-11-27T15:30:01.958859+0000 (oldest deadline 2023-11-27T15:30:27.258747+0000) 2023-11-27T15:30:27.838 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:27 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:27.690+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:27.902 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:27 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:27.625+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:27.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:27 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:27.677+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:27.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:27 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:27.677+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:28.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:27 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:27.726+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:28.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:27 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:27.726+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:28.337 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:28 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:28.082+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:28.716 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:28 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:28.438+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:51.957282+0000 front 2023-11-27T15:29:51.957264+0000 (oldest deadline 2023-11-27T15:30:16.057389+0000) 2023-11-27T15:30:28.717 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:28 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:28.438+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:01.958748+0000 front 2023-11-27T15:30:01.958859+0000 (oldest deadline 2023-11-27T15:30:27.258747+0000) 2023-11-27T15:30:28.902 INFO:journalctl@ceph.osd.4.smithi174.stdout:Nov 27 15:30:28 smithi174 bash[161003]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-4 2023-11-27T15:30:28.903 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:28 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:28.671+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:28.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:28 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:28.704+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:28.904 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:28 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:28.704+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:29.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:28 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:28.691+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:29.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:28 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:28.691+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:29.084 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:28 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:28.713+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:29.337 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:29 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:29.081+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:29.694 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:29 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:29.415+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:51.957282+0000 front 2023-11-27T15:29:51.957264+0000 (oldest deadline 2023-11-27T15:30:16.057389+0000) 2023-11-27T15:30:29.694 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:29 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:29.415+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:01.958748+0000 front 2023-11-27T15:30:01.958859+0000 (oldest deadline 2023-11-27T15:30:27.258747+0000) 2023-11-27T15:30:29.777 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:29 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:29.482+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:00.979920+0000 front 2023-11-27T15:30:00.979999+0000 (oldest deadline 2023-11-27T15:30:29.180797+0000) 2023-11-27T15:30:29.778 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:29 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:29.680+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:29.778 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:29 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:29.680+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:30.058 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:29 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:29.706+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:30.058 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:29 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:29.693+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:30.059 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:29 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:29.693+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:30.065 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.4.service' 2023-11-27T15:30:30.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:29 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:29.691+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:30.104 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:30:30.104 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-11-27T15:30:30.104 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-11-27T15:30:30.105 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.5 2023-11-27T15:30:30.452 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:30 smithi177 systemd[1]: Stopping Ceph osd.5 for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:30:30.452 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:30 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:30.098+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:30.777 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:30 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:30.469+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:00.979920+0000 front 2023-11-27T15:30:00.979999+0000 (oldest deadline 2023-11-27T15:30:29.180797+0000) 2023-11-27T15:30:30.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:30 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:30.695+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:30.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:30 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:30.695+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:30.837 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:30 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:30.655+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:30.838 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:30 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:30.449+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:51.957282+0000 front 2023-11-27T15:29:51.957264+0000 (oldest deadline 2023-11-27T15:30:16.057389+0000) 2023-11-27T15:30:30.838 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:30 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:30.449+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:01.958748+0000 front 2023-11-27T15:30:01.958859+0000 (oldest deadline 2023-11-27T15:30:27.258747+0000) 2023-11-27T15:30:30.838 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:30 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:30.470+0000 7f85454c5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:30:30.838 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:30 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:30.470+0000 7f85454c5700 -1 osd.5 98 *** Got signal Terminated *** 2023-11-27T15:30:30.838 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:30 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:30.470+0000 7f85454c5700 -1 osd.5 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:30:31.152 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:30 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:30.697+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:31.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:30 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:30.703+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:31.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:30 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:30.703+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:31.337 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:31 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:31.069+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:31.777 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:31 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:31.497+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:00.979920+0000 front 2023-11-27T15:30:00.979999+0000 (oldest deadline 2023-11-27T15:30:29.180797+0000) 2023-11-27T15:30:31.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:31 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:31.688+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:31.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:31 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:31.688+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:31.837 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:31 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:31.629+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:31.838 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:31 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:31.482+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:51.957282+0000 front 2023-11-27T15:29:51.957264+0000 (oldest deadline 2023-11-27T15:30:16.057389+0000) 2023-11-27T15:30:31.838 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:31 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:31.482+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:01.958748+0000 front 2023-11-27T15:30:01.958859+0000 (oldest deadline 2023-11-27T15:30:27.258747+0000) 2023-11-27T15:30:32.152 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:31 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:31.730+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:32.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:31 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:31.724+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:32.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:31 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:31.724+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:32.337 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:32 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:32.055+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:32.838 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:32 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:32.441+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:51.957282+0000 front 2023-11-27T15:29:51.957264+0000 (oldest deadline 2023-11-27T15:30:16.057389+0000) 2023-11-27T15:30:32.838 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:32 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:32.441+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:01.958748+0000 front 2023-11-27T15:30:01.958859+0000 (oldest deadline 2023-11-27T15:30:27.258747+0000) 2023-11-27T15:30:32.838 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:32 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:32.619+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:33.027 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:32 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:32.526+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:00.979920+0000 front 2023-11-27T15:30:00.979999+0000 (oldest deadline 2023-11-27T15:30:29.180797+0000) 2023-11-27T15:30:33.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:32 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:32.710+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:33.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:32 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:32.710+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:33.152 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:32 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:32.761+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:33.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:32 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:32.746+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:33.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:32 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:32.746+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:33.429 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:33 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:33.105+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:33.777 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:33 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:33.478+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:00.979920+0000 front 2023-11-27T15:30:00.979999+0000 (oldest deadline 2023-11-27T15:30:29.180797+0000) 2023-11-27T15:30:33.777 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:33 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:33.478+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:09.682432+0000 front 2023-11-27T15:30:09.682248+0000 (oldest deadline 2023-11-27T15:30:32.582163+0000) 2023-11-27T15:30:33.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:33 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:33.699+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:33.778 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:33 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:33.699+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:33.838 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:33 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:33.425+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:51.957282+0000 front 2023-11-27T15:29:51.957264+0000 (oldest deadline 2023-11-27T15:30:16.057389+0000) 2023-11-27T15:30:33.838 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:33 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:33.425+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:01.958748+0000 front 2023-11-27T15:30:01.958859+0000 (oldest deadline 2023-11-27T15:30:27.258747+0000) 2023-11-27T15:30:33.838 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:33 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:33.425+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.259003+0000 front 2023-11-27T15:30:07.259035+0000 (oldest deadline 2023-11-27T15:30:32.459746+0000) 2023-11-27T15:30:33.838 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:33 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:33.657+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:33.839 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:33 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:33.657+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:34.152 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:33 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:33.722+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:34.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:33 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:33.722+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:34.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:33 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:33.722+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:34.420 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:34 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:34.127+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:34.527 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:34 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:34.470+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:00.979920+0000 front 2023-11-27T15:30:00.979999+0000 (oldest deadline 2023-11-27T15:30:29.180797+0000) 2023-11-27T15:30:34.527 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:34 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:34.470+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:09.682432+0000 front 2023-11-27T15:30:09.682248+0000 (oldest deadline 2023-11-27T15:30:32.582163+0000) 2023-11-27T15:30:34.837 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:34 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:34.417+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:51.957282+0000 front 2023-11-27T15:29:51.957264+0000 (oldest deadline 2023-11-27T15:30:16.057389+0000) 2023-11-27T15:30:34.838 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:34 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:34.417+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:01.958748+0000 front 2023-11-27T15:30:01.958859+0000 (oldest deadline 2023-11-27T15:30:27.258747+0000) 2023-11-27T15:30:34.838 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:34 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:34.417+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.259003+0000 front 2023-11-27T15:30:07.259035+0000 (oldest deadline 2023-11-27T15:30:32.459746+0000) 2023-11-27T15:30:34.838 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:34 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:34.657+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:34.838 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:34 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:34.657+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:35.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:34 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:34.686+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:35.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:34 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:34.686+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:35.152 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:34 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:34.700+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:35.152 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:34 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:34.718+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:35.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:34 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:34.718+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:35.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:34 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:34.718+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:35.448 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:35 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:35.169+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:35.448 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:35 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:35.169+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:35.777 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:35 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:35.421+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:00.979920+0000 front 2023-11-27T15:30:00.979999+0000 (oldest deadline 2023-11-27T15:30:29.180797+0000) 2023-11-27T15:30:35.777 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:35 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:35.421+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:09.682432+0000 front 2023-11-27T15:30:09.682248+0000 (oldest deadline 2023-11-27T15:30:32.582163+0000) 2023-11-27T15:30:35.778 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:35 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:35.655+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:35.778 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:35 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:35.655+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:35.778 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:35 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:35.655+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:35.838 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:35 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:35.618+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:35.838 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:35 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:35.618+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:35.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:35 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:35.444+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:51.957282+0000 front 2023-11-27T15:29:51.957264+0000 (oldest deadline 2023-11-27T15:30:16.057389+0000) 2023-11-27T15:30:35.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:35 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:35.444+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:01.958748+0000 front 2023-11-27T15:30:01.958859+0000 (oldest deadline 2023-11-27T15:30:27.258747+0000) 2023-11-27T15:30:35.839 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:35 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5[142659]: 2023-11-27T15:30:35.444+0000 7f853f2b6700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.259003+0000 front 2023-11-27T15:30:07.259035+0000 (oldest deadline 2023-11-27T15:30:32.459746+0000) 2023-11-27T15:30:36.152 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:35 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:35.673+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:36.153 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:35 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:35.673+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.598794+0000 front 2023-11-27T15:30:11.598756+0000 (oldest deadline 2023-11-27T15:30:35.098910+0000) 2023-11-27T15:30:36.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:35 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:35.723+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:36.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:35 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:35.723+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:36.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:35 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:35.723+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:36.338 INFO:journalctl@ceph.osd.5.smithi177.stdout:Nov 27 15:30:36 smithi177 bash[162707]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-5 2023-11-27T15:30:36.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:36 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:36.192+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:36.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:36 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:36.192+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:36.684 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:36 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:36.390+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:00.979920+0000 front 2023-11-27T15:30:00.979999+0000 (oldest deadline 2023-11-27T15:30:29.180797+0000) 2023-11-27T15:30:36.685 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:36 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:36.390+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:09.682432+0000 front 2023-11-27T15:30:09.682248+0000 (oldest deadline 2023-11-27T15:30:32.582163+0000) 2023-11-27T15:30:37.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:36 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:36.681+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:37.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:36 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:36.681+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:37.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:36 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:36.681+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:37.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:36 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:36.657+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:37.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:36 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:36.657+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:37.152 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:36 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:36.712+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:37.153 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:36 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:36.712+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.598794+0000 front 2023-11-27T15:30:11.598756+0000 (oldest deadline 2023-11-27T15:30:35.098910+0000) 2023-11-27T15:30:37.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:36 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:36.739+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:37.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:36 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:36.739+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:37.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:36 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:36.739+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:37.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:37 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:37.176+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:37.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:37 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:37.176+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:37.609 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.5.service' 2023-11-27T15:30:37.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:30:37.650 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-11-27T15:30:37.650 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-11-27T15:30:37.651 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.6 2023-11-27T15:30:37.715 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:37 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:37.437+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:00.979920+0000 front 2023-11-27T15:30:00.979999+0000 (oldest deadline 2023-11-27T15:30:29.180797+0000) 2023-11-27T15:30:37.715 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:37 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:37.437+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:09.682432+0000 front 2023-11-27T15:30:09.682248+0000 (oldest deadline 2023-11-27T15:30:32.582163+0000) 2023-11-27T15:30:37.715 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:37 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:37.634+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:37.716 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:37 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:37.634+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:37.716 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:37 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:37.634+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:38.027 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:37 smithi133 systemd[1]: Stopping Ceph osd.6 for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:30:38.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:37 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:37.655+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:38.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:37 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:37.655+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:38.152 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:37 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:37.750+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:38.152 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:37 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:37.750+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.598794+0000 front 2023-11-27T15:30:11.598756+0000 (oldest deadline 2023-11-27T15:30:35.098910+0000) 2023-11-27T15:30:38.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:37 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:37.723+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:38.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:37 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:37.723+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:38.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:37 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:37.723+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:38.527 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:38 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:38.238+0000 7f3ae68fb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:30:38.527 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:38 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:38.238+0000 7f3ae68fb700 -1 osd.6 98 *** Got signal Terminated *** 2023-11-27T15:30:38.527 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:38 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:38.238+0000 7f3ae68fb700 -1 osd.6 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:30:38.527 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:38 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:38.433+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:00.979920+0000 front 2023-11-27T15:30:00.979999+0000 (oldest deadline 2023-11-27T15:30:29.180797+0000) 2023-11-27T15:30:38.528 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:38 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:38.433+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:09.682432+0000 front 2023-11-27T15:30:09.682248+0000 (oldest deadline 2023-11-27T15:30:32.582163+0000) 2023-11-27T15:30:38.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:38 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:38.152+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:38.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:38 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:38.152+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:39.005 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:38 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:38.626+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:39.005 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:38 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:38.626+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:39.006 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:38 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:38.626+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:39.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:38 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:38.660+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:39.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:38 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:38.660+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:39.152 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:38 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:38.701+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:39.153 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:38 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:38.701+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.598794+0000 front 2023-11-27T15:30:11.598756+0000 (oldest deadline 2023-11-27T15:30:35.098910+0000) 2023-11-27T15:30:39.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:38 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:38.752+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:39.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:38 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:38.752+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:39.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:38 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:38.752+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:39.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:39 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:39.106+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:39.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:39 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:39.106+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:39.777 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:39 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:39.458+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:00.979920+0000 front 2023-11-27T15:30:00.979999+0000 (oldest deadline 2023-11-27T15:30:29.180797+0000) 2023-11-27T15:30:39.777 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:39 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:39.458+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:09.682432+0000 front 2023-11-27T15:30:09.682248+0000 (oldest deadline 2023-11-27T15:30:32.582163+0000) 2023-11-27T15:30:39.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:39 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:39.659+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:39.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:39 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:39.659+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:39.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:39 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:39.659+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:40.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:39 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:39.660+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:40.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:39 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:39.660+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:40.152 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:39 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:39.684+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:40.153 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:39 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:39.684+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.598794+0000 front 2023-11-27T15:30:11.598756+0000 (oldest deadline 2023-11-27T15:30:35.098910+0000) 2023-11-27T15:30:40.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:39 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:39.707+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:40.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:39 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:39.707+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:40.154 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:39 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:39.707+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:40.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:40 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:40.140+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:40.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:40 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:40.140+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:40.777 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:40.472+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:00.979920+0000 front 2023-11-27T15:30:00.979999+0000 (oldest deadline 2023-11-27T15:30:29.180797+0000) 2023-11-27T15:30:40.777 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:40.472+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:09.682432+0000 front 2023-11-27T15:30:09.682248+0000 (oldest deadline 2023-11-27T15:30:32.582163+0000) 2023-11-27T15:30:40.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:40.622+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:40.778 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:40.622+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:40.778 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:40.622+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:40.778 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:40 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:40.622+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:30:41.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:40 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:40.662+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:41.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:40 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:40.662+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:41.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:40 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:40.662+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:18.096894+0000 front 2023-11-27T15:30:18.096760+0000 (oldest deadline 2023-11-27T15:30:40.396958+0000) 2023-11-27T15:30:41.152 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:40 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:40.668+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:41.152 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:40 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:40.668+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.598794+0000 front 2023-11-27T15:30:11.598756+0000 (oldest deadline 2023-11-27T15:30:35.098910+0000) 2023-11-27T15:30:41.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:40 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:40.711+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:41.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:40 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:40.711+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:41.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:40 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:40.711+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:41.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:41 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:41.161+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:41.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:41 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:41.161+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:41.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:41 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:41.161+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:30:41.656 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:41 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:41.441+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:00.979920+0000 front 2023-11-27T15:30:00.979999+0000 (oldest deadline 2023-11-27T15:30:29.180797+0000) 2023-11-27T15:30:41.656 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:41 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:41.441+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:09.682432+0000 front 2023-11-27T15:30:09.682248+0000 (oldest deadline 2023-11-27T15:30:32.582163+0000) 2023-11-27T15:30:41.902 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:41 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:41.639+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:41.902 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:41 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:41.639+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.598794+0000 front 2023-11-27T15:30:11.598756+0000 (oldest deadline 2023-11-27T15:30:35.098910+0000) 2023-11-27T15:30:41.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:41 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:41.672+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:41.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:41 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:41.672+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:41.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:41 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:41.672+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:42.026 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:41 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:41.653+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:42.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:41 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:41.653+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:42.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:41 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:41.653+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:42.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:41 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:41.653+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:30:42.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:41 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:41.690+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:42.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:41 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:41.690+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:42.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:41 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:41.690+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:18.096894+0000 front 2023-11-27T15:30:18.096760+0000 (oldest deadline 2023-11-27T15:30:40.396958+0000) 2023-11-27T15:30:42.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:42 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:42.210+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:42.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:42 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:42.210+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:42.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:42 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:42.210+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:30:42.777 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:42 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:42.393+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:00.979920+0000 front 2023-11-27T15:30:00.979999+0000 (oldest deadline 2023-11-27T15:30:29.180797+0000) 2023-11-27T15:30:42.777 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:42 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6[121810]: 2023-11-27T15:30:42.393+0000 7f3ae0eed700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:09.682432+0000 front 2023-11-27T15:30:09.682248+0000 (oldest deadline 2023-11-27T15:30:32.582163+0000) 2023-11-27T15:30:42.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:42 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:42.621+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:42.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:42 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:42.621+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:42.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:42 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:42.621+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:42.778 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:42 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:42.621+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:30:42.902 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:42 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:42.622+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:42.902 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:42 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:42.622+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.598794+0000 front 2023-11-27T15:30:11.598756+0000 (oldest deadline 2023-11-27T15:30:35.098910+0000) 2023-11-27T15:30:42.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:42 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:42.643+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:42.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:42 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:42.643+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:42.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:42 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:42.643+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:42.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:42 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:42.643+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:30:43.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:42 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:42.690+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:43.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:42 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:42.690+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:43.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:42 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:42.690+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:18.096894+0000 front 2023-11-27T15:30:18.096760+0000 (oldest deadline 2023-11-27T15:30:40.396958+0000) 2023-11-27T15:30:43.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:43 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:43.257+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:43.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:43 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:43.257+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:43.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:43 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:43.257+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:30:43.891 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:43 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:43.610+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:43.891 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:43 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:43.611+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:43.891 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:43 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:43.611+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:43.892 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:43 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:43.611+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:30:43.902 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:43 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:43.617+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:43.903 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:43 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:43.617+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.598794+0000 front 2023-11-27T15:30:11.598756+0000 (oldest deadline 2023-11-27T15:30:35.098910+0000) 2023-11-27T15:30:43.903 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:43 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:43.617+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.099584+0000 front 2023-11-27T15:30:15.099565+0000 (oldest deadline 2023-11-27T15:30:42.699974+0000) 2023-11-27T15:30:43.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:43 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:43.620+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:43.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:43 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:43.620+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:43.904 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:43 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:43.620+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:43.904 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:43 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:43.620+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:30:44.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:43 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:43.644+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:44.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:43 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:43.644+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:44.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:43 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:43.644+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:18.096894+0000 front 2023-11-27T15:30:18.096760+0000 (oldest deadline 2023-11-27T15:30:40.396958+0000) 2023-11-27T15:30:44.277 INFO:journalctl@ceph.osd.6.smithi133.stdout:Nov 27 15:30:43 smithi133 bash[259365]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-6 2023-11-27T15:30:44.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:44 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:44.269+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:44.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:44 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:44.269+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:44.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:44 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:44.269+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:30:44.898 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:44 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:44.641+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:44.899 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:44 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:44.641+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:44.899 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:44 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:44.641+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:44.899 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:44 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:44.641+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:30:44.902 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:44 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:44.596+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:44.903 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:44 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:44.596+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.598794+0000 front 2023-11-27T15:30:11.598756+0000 (oldest deadline 2023-11-27T15:30:35.098910+0000) 2023-11-27T15:30:44.903 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:44 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:44.596+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.099584+0000 front 2023-11-27T15:30:15.099565+0000 (oldest deadline 2023-11-27T15:30:42.699974+0000) 2023-11-27T15:30:44.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:44 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:44.581+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:44.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:44 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:44.581+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:44.904 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:44 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:44.581+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:44.904 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:44 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:44.581+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:30:45.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:44 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:44.671+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:45.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:44 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:44.671+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:45.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:44 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:44.671+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:18.096894+0000 front 2023-11-27T15:30:18.096760+0000 (oldest deadline 2023-11-27T15:30:40.396958+0000) 2023-11-27T15:30:45.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:45 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:45.311+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:45.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:45 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:45.311+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:45.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:45 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:45.311+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:30:45.812 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.6.service' 2023-11-27T15:30:45.902 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:30:45.903 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-11-27T15:30:45.903 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-11-27T15:30:45.903 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.7 2023-11-27T15:30:45.904 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:45 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:45.578+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:45.904 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:45 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:45.578+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.598794+0000 front 2023-11-27T15:30:11.598756+0000 (oldest deadline 2023-11-27T15:30:35.098910+0000) 2023-11-27T15:30:45.904 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:45 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:45.578+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.099584+0000 front 2023-11-27T15:30:15.099565+0000 (oldest deadline 2023-11-27T15:30:42.699974+0000) 2023-11-27T15:30:45.904 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:45 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:45.588+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:45.905 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:45 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:45.588+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:45.905 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:45 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:45.588+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:45.905 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:45 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:45.588+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:30:46.026 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:45 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:45.663+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:46.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:45 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:45.663+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:46.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:45 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:45.663+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:46.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:45 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:45.663+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:30:46.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:45 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:45.624+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:46.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:45 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:45.624+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:46.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:45 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:45.624+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:18.096894+0000 front 2023-11-27T15:30:18.096760+0000 (oldest deadline 2023-11-27T15:30:40.396958+0000) 2023-11-27T15:30:46.258 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:45 smithi174 systemd[1]: Stopping Ceph osd.7 for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:30:46.581 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:46 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:46.254+0000 7fc0d0935700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:30:46.581 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:46 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:46.254+0000 7fc0d0935700 -1 osd.7 98 *** Got signal Terminated *** 2023-11-27T15:30:46.581 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:46 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:46.254+0000 7fc0d0935700 -1 osd.7 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:30:46.581 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:46 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:46.577+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:46.626 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:46 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:46.336+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:46.626 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:46 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:46.336+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:46.626 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:46 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:46.336+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:30:46.902 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:46 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:46.577+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.598794+0000 front 2023-11-27T15:30:11.598756+0000 (oldest deadline 2023-11-27T15:30:35.098910+0000) 2023-11-27T15:30:46.902 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:46 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:46.577+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.099584+0000 front 2023-11-27T15:30:15.099565+0000 (oldest deadline 2023-11-27T15:30:42.699974+0000) 2023-11-27T15:30:46.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:46 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:46.591+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:46.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:46 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:46.591+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:46.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:46 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:46.591+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:46.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:46 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:46.591+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:30:47.026 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:46 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:46.614+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:47.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:46 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:46.614+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:47.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:46 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:46.614+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:47.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:46 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:46.614+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:30:47.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:46 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:46.622+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:47.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:46 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:46.622+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:47.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:46 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:46.622+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:18.096894+0000 front 2023-11-27T15:30:18.096760+0000 (oldest deadline 2023-11-27T15:30:40.396958+0000) 2023-11-27T15:30:47.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:47 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:47.302+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:47.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:47 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:47.302+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:47.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:47 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:47.302+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:30:47.902 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:47 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:47.550+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:47.902 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:47 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:47.550+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.598794+0000 front 2023-11-27T15:30:11.598756+0000 (oldest deadline 2023-11-27T15:30:35.098910+0000) 2023-11-27T15:30:47.903 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:47 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:47.550+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.099584+0000 front 2023-11-27T15:30:15.099565+0000 (oldest deadline 2023-11-27T15:30:42.699974+0000) 2023-11-27T15:30:47.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:47 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:47.627+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:47.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:47 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:47.627+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:47.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:47 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:47.627+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:47.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:47 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:47.628+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:30:48.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:47 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:47.595+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:48.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:47 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:47.595+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:48.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:47 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:47.595+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:48.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:47 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:47.595+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:30:48.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:47 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:47.663+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:48.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:47 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:47.663+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:48.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:47 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:47.663+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:18.096894+0000 front 2023-11-27T15:30:18.096760+0000 (oldest deadline 2023-11-27T15:30:40.396958+0000) 2023-11-27T15:30:48.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:48 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:48.280+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:48.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:48 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:48.280+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:48.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:48 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:48.280+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:30:48.902 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:48 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:48.504+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:48.903 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:48 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:48.504+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.598794+0000 front 2023-11-27T15:30:11.598756+0000 (oldest deadline 2023-11-27T15:30:35.098910+0000) 2023-11-27T15:30:48.903 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:48 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:48.504+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.099584+0000 front 2023-11-27T15:30:15.099565+0000 (oldest deadline 2023-11-27T15:30:42.699974+0000) 2023-11-27T15:30:48.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:48 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:48.670+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:48.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:48 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:48.670+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:48.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:48 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:48.670+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:48.904 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:48 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:48.670+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:30:49.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:48 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:48.621+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:49.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:48 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:48.621+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:49.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:48 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:48.621+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:49.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:48 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:48.621+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:30:49.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:48 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:48.708+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:49.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:48 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:48.708+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:49.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:48 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:48.708+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:18.096894+0000 front 2023-11-27T15:30:18.096760+0000 (oldest deadline 2023-11-27T15:30:40.396958+0000) 2023-11-27T15:30:49.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:48 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:48.708+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.298071+0000 front 2023-11-27T15:30:26.298087+0000 (oldest deadline 2023-11-27T15:30:48.598200+0000) 2023-11-27T15:30:49.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:49 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:49.262+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:49.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:49 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:49.262+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:49.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:49 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:49.262+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:30:49.902 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:49 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:49.550+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:49.903 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:49 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:49.550+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.598794+0000 front 2023-11-27T15:30:11.598756+0000 (oldest deadline 2023-11-27T15:30:35.098910+0000) 2023-11-27T15:30:49.903 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:49 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:49.550+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.099584+0000 front 2023-11-27T15:30:15.099565+0000 (oldest deadline 2023-11-27T15:30:42.699974+0000) 2023-11-27T15:30:49.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:49 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:49.674+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:49.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:49 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:49.674+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:49.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:49 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:49.674+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:49.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:49 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:49.674+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:30:50.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:49 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:49.654+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:50.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:49 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:49.654+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:50.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:49 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:49.654+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:50.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:49 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:49.654+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:30:50.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:49 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:49.654+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:25.522892+0000 front 2023-11-27T15:30:25.522949+0000 (oldest deadline 2023-11-27T15:30:49.523292+0000) 2023-11-27T15:30:50.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:49 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:49.752+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:50.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:49 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:49.752+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:50.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:49 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:49.752+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:18.096894+0000 front 2023-11-27T15:30:18.096760+0000 (oldest deadline 2023-11-27T15:30:40.396958+0000) 2023-11-27T15:30:50.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:49 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:49.752+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.298071+0000 front 2023-11-27T15:30:26.298087+0000 (oldest deadline 2023-11-27T15:30:48.598200+0000) 2023-11-27T15:30:50.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:50 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:50.297+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:50.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:50 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:50.297+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:50.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:50 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:50.297+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:30:50.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:50 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:50.655+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:50.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:50 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:50.655+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:50.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:50 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:50.655+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:50.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:50 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:50.655+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:30:50.904 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:50 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:50.576+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.296997+0000 front 2023-11-27T15:30:02.297073+0000 (oldest deadline 2023-11-27T15:30:25.197134+0000) 2023-11-27T15:30:50.904 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:50 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:50.576+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.598794+0000 front 2023-11-27T15:30:11.598756+0000 (oldest deadline 2023-11-27T15:30:35.098910+0000) 2023-11-27T15:30:50.904 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:50 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:50.576+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.099584+0000 front 2023-11-27T15:30:15.099565+0000 (oldest deadline 2023-11-27T15:30:42.699974+0000) 2023-11-27T15:30:50.905 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:50 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7[147158]: 2023-11-27T15:30:50.576+0000 7fc0cb728700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.801005+0000 front 2023-11-27T15:30:26.801074+0000 (oldest deadline 2023-11-27T15:30:50.301162+0000) 2023-11-27T15:30:50.917 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:50 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:50.635+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:50.917 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:50 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:50.635+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:50.917 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:50 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:50.635+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:50.917 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:50 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:50.635+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:30:50.917 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:50 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:50.635+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:25.522892+0000 front 2023-11-27T15:30:25.522949+0000 (oldest deadline 2023-11-27T15:30:49.523292+0000) 2023-11-27T15:30:51.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:50 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:50.751+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:51.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:50 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:50.751+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:51.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:50 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:50.751+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:18.096894+0000 front 2023-11-27T15:30:18.096760+0000 (oldest deadline 2023-11-27T15:30:40.396958+0000) 2023-11-27T15:30:51.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:50 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:50.751+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.298071+0000 front 2023-11-27T15:30:26.298087+0000 (oldest deadline 2023-11-27T15:30:48.598200+0000) 2023-11-27T15:30:51.743 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:51 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:51.338+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:51.743 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:51 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:51.338+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:51.743 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:51 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:51.338+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:30:51.743 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:51 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:51.338+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:30:51.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:51 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:51.628+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:51.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:51 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:51.628+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:51.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:51 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:51.628+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:51.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:51 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:51.628+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:30:52.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:51 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:51.679+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:52.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:51 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:51.679+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:52.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:51 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:51.679+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:52.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:51 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:51.679+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:30:52.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:51 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:51.679+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:25.522892+0000 front 2023-11-27T15:30:25.522949+0000 (oldest deadline 2023-11-27T15:30:49.523292+0000) 2023-11-27T15:30:52.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:51 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:51.739+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:52.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:51 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:51.739+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:52.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:51 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:51.739+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:18.096894+0000 front 2023-11-27T15:30:18.096760+0000 (oldest deadline 2023-11-27T15:30:40.396958+0000) 2023-11-27T15:30:52.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:51 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:51.739+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.298071+0000 front 2023-11-27T15:30:26.298087+0000 (oldest deadline 2023-11-27T15:30:48.598200+0000) 2023-11-27T15:30:52.402 INFO:journalctl@ceph.osd.7.smithi174.stdout:Nov 27 15:30:51 smithi174 bash[161267]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-7 2023-11-27T15:30:52.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:52 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:52.313+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:52.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:52 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:52.313+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:52.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:52 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:52.313+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:30:52.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:52 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:52.313+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:30:53.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:52 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:52.647+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:53.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:52 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:52.647+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:53.028 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:52 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:52.647+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:53.028 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:52 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:52.647+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:30:53.028 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:52 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:52.647+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:25.522892+0000 front 2023-11-27T15:30:25.522949+0000 (oldest deadline 2023-11-27T15:30:49.523292+0000) 2023-11-27T15:30:53.061 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:52 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:52.666+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:53.061 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:52 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:52.666+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:53.061 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:52 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:52.666+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:53.061 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:52 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:52.666+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:30:53.062 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:52 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:52.666+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:30:53.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:52 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:52.739+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:53.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:52 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:52.739+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:53.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:52 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:52.739+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:18.096894+0000 front 2023-11-27T15:30:18.096760+0000 (oldest deadline 2023-11-27T15:30:40.396958+0000) 2023-11-27T15:30:53.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:52 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:52.739+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.298071+0000 front 2023-11-27T15:30:26.298087+0000 (oldest deadline 2023-11-27T15:30:48.598200+0000) 2023-11-27T15:30:53.333 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.7.service' 2023-11-27T15:30:53.373 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:30:53.374 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-11-27T15:30:53.374 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-11-27T15:30:53.374 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.8 2023-11-27T15:30:53.588 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:53 smithi177 systemd[1]: Stopping Ceph osd.8 for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:30:53.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:53 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:53.273+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:53.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:53 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:53.273+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:53.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:53 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:53.273+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:30:53.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:53 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:53.274+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:30:54.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:53 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:53.634+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:54.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:53 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:53.634+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:54.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:53 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:53.634+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:54.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:53 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:53.634+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:30:54.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:53 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:53.634+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:25.522892+0000 front 2023-11-27T15:30:25.522949+0000 (oldest deadline 2023-11-27T15:30:49.523292+0000) 2023-11-27T15:30:54.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:53 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:53.717+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:54.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:53 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:53.717+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:54.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:53 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:53.717+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:18.096894+0000 front 2023-11-27T15:30:18.096760+0000 (oldest deadline 2023-11-27T15:30:40.396958+0000) 2023-11-27T15:30:54.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:53 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:53.717+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.298071+0000 front 2023-11-27T15:30:26.298087+0000 (oldest deadline 2023-11-27T15:30:48.598200+0000) 2023-11-27T15:30:54.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:53 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:53.732+0000 7f2ce87fb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:30:54.089 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:53 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:53.732+0000 7f2ce87fb700 -1 osd.8 98 *** Got signal Terminated *** 2023-11-27T15:30:54.089 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:53 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:53.732+0000 7f2ce87fb700 -1 osd.8 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:30:54.152 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:53 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:53.670+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:54.152 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:53 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:53.670+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:54.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:53 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:53.670+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:54.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:53 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:53.670+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:30:54.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:53 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:53.670+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:30:54.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:54 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:54.310+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:54.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:54 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:54.310+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:54.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:54 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:54.310+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:30:54.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:54 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:54.310+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:30:54.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:54 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:54.644+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:54.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:54 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:54.644+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:54.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:54 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:54.644+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:54.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:54 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:54.644+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:30:54.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:54 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:54.644+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:30:55.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:54 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:54.661+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:55.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:54 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:54.661+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:55.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:54 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:54.661+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:55.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:54 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:54.661+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:30:55.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:54 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:54.661+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:25.522892+0000 front 2023-11-27T15:30:25.522949+0000 (oldest deadline 2023-11-27T15:30:49.523292+0000) 2023-11-27T15:30:55.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:54 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:54.740+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:55.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:54 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:54.740+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:55.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:54 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:54.740+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:18.096894+0000 front 2023-11-27T15:30:18.096760+0000 (oldest deadline 2023-11-27T15:30:40.396958+0000) 2023-11-27T15:30:55.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:54 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:54.740+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.298071+0000 front 2023-11-27T15:30:26.298087+0000 (oldest deadline 2023-11-27T15:30:48.598200+0000) 2023-11-27T15:30:55.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:55 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:55.288+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:55.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:55 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:55.288+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:55.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:55 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:55.288+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:30:55.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:55 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:55.288+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:30:55.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:55 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:55.646+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:55.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:55 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:55.646+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:55.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:55 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:55.646+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:55.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:55 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:55.646+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:30:55.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:55 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:55.646+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:30:56.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:55 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:55.615+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:56.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:55 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:55.615+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:56.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:55 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:55.615+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:56.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:55 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:55.615+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:30:56.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:55 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:55.615+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:25.522892+0000 front 2023-11-27T15:30:25.522949+0000 (oldest deadline 2023-11-27T15:30:49.523292+0000) 2023-11-27T15:30:56.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:55 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:55.757+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:56.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:55 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:55.757+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:56.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:55 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:55.757+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:18.096894+0000 front 2023-11-27T15:30:18.096760+0000 (oldest deadline 2023-11-27T15:30:40.396958+0000) 2023-11-27T15:30:56.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:55 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:55.757+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.298071+0000 front 2023-11-27T15:30:26.298087+0000 (oldest deadline 2023-11-27T15:30:48.598200+0000) 2023-11-27T15:30:56.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:56 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:56.245+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:56.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:56 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:56.245+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:56.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:56 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:56.245+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:30:56.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:56 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:56.245+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:30:56.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:56 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:56.602+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:56.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:56 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:56.602+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:56.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:56 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:56.602+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:56.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:56 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:56.602+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:30:56.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:56 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:56.602+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:30:56.904 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:56 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:56.602+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:32.456188+0000 front 2023-11-27T15:30:32.456248+0000 (oldest deadline 2023-11-27T15:30:55.956203+0000) 2023-11-27T15:30:57.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:56 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:56.628+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:57.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:56 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:56.628+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:57.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:56 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:56.628+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:57.028 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:56 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:56.628+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:30:57.028 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:56 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:56.628+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:25.522892+0000 front 2023-11-27T15:30:25.522949+0000 (oldest deadline 2023-11-27T15:30:49.523292+0000) 2023-11-27T15:30:57.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:56 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:56.729+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:57.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:56 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:56.729+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:57.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:56 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:56.729+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:18.096894+0000 front 2023-11-27T15:30:18.096760+0000 (oldest deadline 2023-11-27T15:30:40.396958+0000) 2023-11-27T15:30:57.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:56 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:56.729+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.298071+0000 front 2023-11-27T15:30:26.298087+0000 (oldest deadline 2023-11-27T15:30:48.598200+0000) 2023-11-27T15:30:57.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:57 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:57.199+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:57.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:57 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:57.199+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:57.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:57 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:57.199+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:30:57.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:57 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:57.199+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:30:57.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:57 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:57.557+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:57.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:57 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:57.557+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:57.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:57 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:57.557+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:57.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:57 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:57.557+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:30:57.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:57 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:57.557+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:30:57.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:57 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:57.557+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:32.456188+0000 front 2023-11-27T15:30:32.456248+0000 (oldest deadline 2023-11-27T15:30:55.956203+0000) 2023-11-27T15:30:57.950 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:57 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:57.615+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:57.951 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:57 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:57.615+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:57.951 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:57 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:57.615+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:57.951 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:57 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:57.615+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:30:57.951 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:57 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:57.615+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:25.522892+0000 front 2023-11-27T15:30:25.522949+0000 (oldest deadline 2023-11-27T15:30:49.523292+0000) 2023-11-27T15:30:58.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:57 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:57.740+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:58.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:57 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:57.740+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:58.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:57 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:57.740+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:18.096894+0000 front 2023-11-27T15:30:18.096760+0000 (oldest deadline 2023-11-27T15:30:40.396958+0000) 2023-11-27T15:30:58.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:57 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:57.740+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.298071+0000 front 2023-11-27T15:30:26.298087+0000 (oldest deadline 2023-11-27T15:30:48.598200+0000) 2023-11-27T15:30:58.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:57 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:57.740+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.899248+0000 front 2023-11-27T15:30:33.899282+0000 (oldest deadline 2023-11-27T15:30:57.399453+0000) 2023-11-27T15:30:58.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:58 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:58.199+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:58.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:58 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:58.199+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:58.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:58 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:58.199+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:30:58.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:58 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:58.199+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:30:58.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:58 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:58.199+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:30:58.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:58 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:58.586+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:58.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:58 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:58.586+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:58.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:58 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:58.586+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:58.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:58 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:58.586+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:30:58.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:58 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:58.586+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:30:58.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:58 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:58.586+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:32.456188+0000 front 2023-11-27T15:30:32.456248+0000 (oldest deadline 2023-11-27T15:30:55.956203+0000) 2023-11-27T15:30:59.026 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:58 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:58.581+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:30:59.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:58 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:58.581+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:30:59.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:58 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:58.581+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:30:59.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:58 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:58.581+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:30:59.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:58 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:58.581+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:25.522892+0000 front 2023-11-27T15:30:25.522949+0000 (oldest deadline 2023-11-27T15:30:49.523292+0000) 2023-11-27T15:30:59.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:58 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:58.581+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:34.224390+0000 front 2023-11-27T15:30:34.224306+0000 (oldest deadline 2023-11-27T15:30:57.724032+0000) 2023-11-27T15:30:59.087 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:58 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:58.690+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.894805+0000 front 2023-11-27T15:30:02.894835+0000 (oldest deadline 2023-11-27T15:30:25.794876+0000) 2023-11-27T15:30:59.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:58 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:58.690+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:07.495773+0000 front 2023-11-27T15:30:07.495744+0000 (oldest deadline 2023-11-27T15:30:33.395886+0000) 2023-11-27T15:30:59.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:58 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:58.690+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:18.096894+0000 front 2023-11-27T15:30:18.096760+0000 (oldest deadline 2023-11-27T15:30:40.396958+0000) 2023-11-27T15:30:59.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:58 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:58.690+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.298071+0000 front 2023-11-27T15:30:26.298087+0000 (oldest deadline 2023-11-27T15:30:48.598200+0000) 2023-11-27T15:30:59.088 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:58 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8[148151]: 2023-11-27T15:30:58.690+0000 7f2ce4df1700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.899248+0000 front 2023-11-27T15:30:33.899282+0000 (oldest deadline 2023-11-27T15:30:57.399453+0000) 2023-11-27T15:30:59.391 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:59 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:59.216+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:30:59.392 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:59 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:59.216+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:30:59.392 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:59 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:59.216+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:30:59.392 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:59 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:59.216+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:30:59.392 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:30:59 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:30:59.216+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:30:59.838 INFO:journalctl@ceph.osd.8.smithi177.stdout:Nov 27 15:30:59 smithi177 bash[162976]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-8 2023-11-27T15:30:59.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:59 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:59.621+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:30:59.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:59 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:59.621+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:30:59.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:59 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:59.621+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:30:59.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:59 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:59.621+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:30:59.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:59 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:59.621+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:30:59.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:30:59 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:30:59.621+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:32.456188+0000 front 2023-11-27T15:30:32.456248+0000 (oldest deadline 2023-11-27T15:30:55.956203+0000) 2023-11-27T15:31:00.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:59 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:59.557+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:31:00.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:59 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:59.557+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:31:00.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:59 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:59.557+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:31:00.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:59 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:59.557+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:31:00.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:59 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:59.557+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:25.522892+0000 front 2023-11-27T15:30:25.522949+0000 (oldest deadline 2023-11-27T15:30:49.523292+0000) 2023-11-27T15:31:00.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:30:59 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:30:59.557+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:34.224390+0000 front 2023-11-27T15:30:34.224306+0000 (oldest deadline 2023-11-27T15:30:57.724032+0000) 2023-11-27T15:31:00.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:00 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:00.227+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:00.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:00 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:00.227+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:00.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:00 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:00.227+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:00.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:00 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:00.227+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:00.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:00 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:00.227+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:00.680 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.8.service' 2023-11-27T15:31:00.721 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:31:00.721 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-11-27T15:31:00.722 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-11-27T15:31:00.722 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.9 2023-11-27T15:31:00.729 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:00 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:00.521+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:31:00.729 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:00 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:00.521+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:31:00.729 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:00 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:00.521+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:31:00.729 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:00 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:00.521+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:31:00.729 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:00 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:00.521+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:25.522892+0000 front 2023-11-27T15:30:25.522949+0000 (oldest deadline 2023-11-27T15:30:49.523292+0000) 2023-11-27T15:31:00.730 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:00 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:00.521+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:34.224390+0000 front 2023-11-27T15:30:34.224306+0000 (oldest deadline 2023-11-27T15:30:57.724032+0000) 2023-11-27T15:31:01.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:00 smithi133 systemd[1]: Stopping Ceph osd.9 for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:31:01.152 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:00 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:00.661+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:31:01.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:00 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:00.661+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:31:01.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:00 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:00.661+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:31:01.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:00 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:00.661+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:31:01.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:00 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:00.661+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:31:01.154 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:00 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:00.661+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:32.456188+0000 front 2023-11-27T15:30:32.456248+0000 (oldest deadline 2023-11-27T15:30:55.956203+0000) 2023-11-27T15:31:01.507 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:01 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:01.230+0000 7f883a2ed700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:31:01.508 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:01 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:01.230+0000 7f883a2ed700 -1 osd.9 98 *** Got signal Terminated *** 2023-11-27T15:31:01.508 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:01 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:01.230+0000 7f883a2ed700 -1 osd.9 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:31:01.508 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:01 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:01.504+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:31:01.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:01 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:01.218+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:01.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:01 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:01.218+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:01.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:01 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:01.218+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:01.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:01 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:01.218+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:01.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:01 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:01.218+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:01.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:01 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:01.504+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:31:01.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:01 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:01.504+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:31:01.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:01 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:01.504+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:31:01.778 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:01 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:01.504+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:25.522892+0000 front 2023-11-27T15:30:25.522949+0000 (oldest deadline 2023-11-27T15:30:49.523292+0000) 2023-11-27T15:31:01.778 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:01 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:01.504+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:34.224390+0000 front 2023-11-27T15:30:34.224306+0000 (oldest deadline 2023-11-27T15:30:57.724032+0000) 2023-11-27T15:31:02.152 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:01 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:01.702+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:31:02.152 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:01 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:01.702+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:31:02.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:01 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:01.702+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:31:02.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:01 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:01.702+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:31:02.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:01 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:01.702+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:31:02.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:01 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:01.702+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:32.456188+0000 front 2023-11-27T15:30:32.456248+0000 (oldest deadline 2023-11-27T15:30:55.956203+0000) 2023-11-27T15:31:02.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:02 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:02.218+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:02.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:02 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:02.218+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:02.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:02 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:02.218+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:02.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:02 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:02.218+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:02.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:02 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:02.218+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:03.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:02 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:02.532+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:31:03.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:02 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:02.532+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:31:03.027 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:02 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:02.532+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:31:03.028 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:02 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:02.532+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:31:03.028 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:02 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:02.532+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:25.522892+0000 front 2023-11-27T15:30:25.522949+0000 (oldest deadline 2023-11-27T15:30:49.523292+0000) 2023-11-27T15:31:03.028 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:02 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:02.532+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:34.224390+0000 front 2023-11-27T15:30:34.224306+0000 (oldest deadline 2023-11-27T15:30:57.724032+0000) 2023-11-27T15:31:03.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:02 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:02.686+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:31:03.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:02 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:02.686+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:31:03.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:02 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:02.686+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:31:03.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:02 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:02.686+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:31:03.154 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:02 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:02.686+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:31:03.154 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:02 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:02.686+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:32.456188+0000 front 2023-11-27T15:30:32.456248+0000 (oldest deadline 2023-11-27T15:30:55.956203+0000) 2023-11-27T15:31:03.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:03 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:03.247+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:03.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:03 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:03.247+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:03.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:03 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:03.247+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:03.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:03 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:03.247+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:03.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:03 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:03.247+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:03.776 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:03 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:03.519+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:31:03.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:03 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:03.519+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:31:03.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:03 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:03.519+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:31:03.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:03 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:03.519+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:31:03.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:03 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:03.519+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:25.522892+0000 front 2023-11-27T15:30:25.522949+0000 (oldest deadline 2023-11-27T15:30:49.523292+0000) 2023-11-27T15:31:03.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:03 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:03.519+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:34.224390+0000 front 2023-11-27T15:30:34.224306+0000 (oldest deadline 2023-11-27T15:30:57.724032+0000) 2023-11-27T15:31:04.152 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:03 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:03.653+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:31:04.152 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:03 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:03.653+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:31:04.152 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:03 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:03.653+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:31:04.152 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:03 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:03.653+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:31:04.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:03 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:03.653+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:31:04.153 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:03 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:03.653+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:32.456188+0000 front 2023-11-27T15:30:32.456248+0000 (oldest deadline 2023-11-27T15:30:55.956203+0000) 2023-11-27T15:31:04.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:04 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:04.208+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:04.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:04 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:04.208+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:04.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:04 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:04.208+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:04.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:04 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:04.208+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:04.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:04 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:04.208+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:04.776 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:04 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:04.482+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:31:04.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:04 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:04.482+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:31:04.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:04 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:04.482+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:31:04.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:04 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:04.482+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:31:04.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:04 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:04.482+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:25.522892+0000 front 2023-11-27T15:30:25.522949+0000 (oldest deadline 2023-11-27T15:30:49.523292+0000) 2023-11-27T15:31:04.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:04 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:04.482+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:34.224390+0000 front 2023-11-27T15:30:34.224306+0000 (oldest deadline 2023-11-27T15:30:57.724032+0000) 2023-11-27T15:31:04.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:04 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:04.628+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:31:04.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:04 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:04.628+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:31:04.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:04 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:04.628+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:31:04.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:04 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:04.628+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:31:04.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:04 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:04.628+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:31:04.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:04 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:04.628+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:32.456188+0000 front 2023-11-27T15:30:32.456248+0000 (oldest deadline 2023-11-27T15:30:55.956203+0000) 2023-11-27T15:31:05.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:05 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:05.167+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:05.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:05 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:05.167+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:05.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:05 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:05.167+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:05.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:05 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:05.167+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:05.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:05 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:05.167+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:05.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:05 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:05.453+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.114820+0000 front 2023-11-27T15:29:53.114920+0000 (oldest deadline 2023-11-27T15:30:17.214993+0000) 2023-11-27T15:31:05.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:05 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:05.453+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:04.118134+0000 front 2023-11-27T15:30:04.117875+0000 (oldest deadline 2023-11-27T15:30:27.018052+0000) 2023-11-27T15:31:05.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:05 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:05.453+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.920880+0000 front 2023-11-27T15:30:10.920674+0000 (oldest deadline 2023-11-27T15:30:35.020576+0000) 2023-11-27T15:31:05.777 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:05 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:05.453+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.020962+0000 front 2023-11-27T15:30:15.021006+0000 (oldest deadline 2023-11-27T15:30:40.321152+0000) 2023-11-27T15:31:05.778 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:05 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:05.453+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:25.522892+0000 front 2023-11-27T15:30:25.522949+0000 (oldest deadline 2023-11-27T15:30:49.523292+0000) 2023-11-27T15:31:05.778 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:05 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9[127248]: 2023-11-27T15:31:05.453+0000 7f88348df700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:34.224390+0000 front 2023-11-27T15:30:34.224306+0000 (oldest deadline 2023-11-27T15:30:57.724032+0000) 2023-11-27T15:31:05.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:05 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:05.593+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:31:05.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:05 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:05.593+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:31:05.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:05 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:05.593+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:31:05.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:05 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:05.593+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:31:05.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:05 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:05.593+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:31:05.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:05 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:05.593+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:32.456188+0000 front 2023-11-27T15:30:32.456248+0000 (oldest deadline 2023-11-27T15:30:55.956203+0000) 2023-11-27T15:31:06.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:06 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:06.156+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:06.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:06 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:06.156+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:06.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:06 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:06.156+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:06.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:06 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:06.156+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:06.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:06 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:06.156+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:06.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:06 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:06.156+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:41.406477+0000 front 2023-11-27T15:30:41.406512+0000 (oldest deadline 2023-11-27T15:31:05.506531+0000) 2023-11-27T15:31:06.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:06 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:06.563+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:31:06.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:06 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:06.563+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:31:06.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:06 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:06.563+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:31:06.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:06 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:06.563+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:31:06.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:06 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:06.563+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:31:06.904 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:06 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:06.563+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:32.456188+0000 front 2023-11-27T15:30:32.456248+0000 (oldest deadline 2023-11-27T15:30:55.956203+0000) 2023-11-27T15:31:06.904 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:06 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:06.563+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:42.357791+0000 front 2023-11-27T15:30:42.357883+0000 (oldest deadline 2023-11-27T15:31:05.857759+0000) 2023-11-27T15:31:07.527 INFO:journalctl@ceph.osd.9.smithi133.stdout:Nov 27 15:31:07 smithi133 bash[259682]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-9 2023-11-27T15:31:07.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:07 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:07.112+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:07.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:07 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:07.112+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:07.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:07 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:07.112+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:07.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:07 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:07.112+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:07.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:07 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:07.112+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:07.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:07 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:07.112+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:41.406477+0000 front 2023-11-27T15:30:41.406512+0000 (oldest deadline 2023-11-27T15:31:05.506531+0000) 2023-11-27T15:31:07.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:07 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:07.556+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:31:07.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:07 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:07.556+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:31:07.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:07 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:07.556+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:31:07.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:07 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:07.556+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:31:07.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:07 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:07.556+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:31:07.904 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:07 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:07.556+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:32.456188+0000 front 2023-11-27T15:30:32.456248+0000 (oldest deadline 2023-11-27T15:30:55.956203+0000) 2023-11-27T15:31:07.904 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:07 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:07.556+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:42.357791+0000 front 2023-11-27T15:30:42.357883+0000 (oldest deadline 2023-11-27T15:31:05.857759+0000) 2023-11-27T15:31:08.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:08 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:08.088+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:08.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:08 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:08.088+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:08.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:08 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:08.088+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:08.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:08 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:08.088+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:08.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:08 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:08.088+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:08.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:08 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:08.088+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:41.406477+0000 front 2023-11-27T15:30:41.406512+0000 (oldest deadline 2023-11-27T15:31:05.506531+0000) 2023-11-27T15:31:08.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:08 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:08.526+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:31:08.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:08 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:08.526+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:31:08.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:08 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:08.526+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:31:08.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:08 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:08.526+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:31:08.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:08 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:08.526+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:31:08.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:08 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:08.526+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:32.456188+0000 front 2023-11-27T15:30:32.456248+0000 (oldest deadline 2023-11-27T15:30:55.956203+0000) 2023-11-27T15:31:08.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:08 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:08.526+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:42.357791+0000 front 2023-11-27T15:30:42.357883+0000 (oldest deadline 2023-11-27T15:31:05.857759+0000) 2023-11-27T15:31:09.036 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.9.service' 2023-11-27T15:31:09.126 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:31:09.126 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-11-27T15:31:09.127 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-11-27T15:31:09.127 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.10 2023-11-27T15:31:09.479 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:09 smithi174 systemd[1]: Stopping Ceph osd.10 for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:31:09.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:09 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:09.090+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:09.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:09 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:09.090+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:09.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:09 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:09.090+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:09.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:09 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:09.090+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:09.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:09 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:09.090+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:09.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:09 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:09.090+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:41.406477+0000 front 2023-11-27T15:30:41.406512+0000 (oldest deadline 2023-11-27T15:31:05.506531+0000) 2023-11-27T15:31:09.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:09 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:09.475+0000 7f263f3fd700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:31:09.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:09 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:09.475+0000 7f263f3fd700 -1 osd.10 98 *** Got signal Terminated *** 2023-11-27T15:31:09.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:09 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:09.475+0000 7f263f3fd700 -1 osd.10 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:31:09.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:09 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:09.495+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:31:09.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:09 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:09.495+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:31:09.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:09 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:09.495+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:31:09.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:09 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:09.495+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:31:09.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:09 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:09.495+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:31:09.904 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:09 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:09.495+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:32.456188+0000 front 2023-11-27T15:30:32.456248+0000 (oldest deadline 2023-11-27T15:30:55.956203+0000) 2023-11-27T15:31:09.904 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:09 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:09.495+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:42.357791+0000 front 2023-11-27T15:30:42.357883+0000 (oldest deadline 2023-11-27T15:31:05.857759+0000) 2023-11-27T15:31:10.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:10 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:10.119+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:10.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:10 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:10.119+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:10.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:10 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:10.119+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:10.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:10 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:10.119+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:10.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:10 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:10.119+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:10.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:10 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:10.119+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:41.406477+0000 front 2023-11-27T15:30:41.406512+0000 (oldest deadline 2023-11-27T15:31:05.506531+0000) 2023-11-27T15:31:10.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:10 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:10.467+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:31:10.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:10 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:10.467+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:31:10.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:10 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:10.467+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:31:10.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:10 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:10.467+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:31:10.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:10 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:10.467+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:31:10.904 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:10 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:10.467+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:32.456188+0000 front 2023-11-27T15:30:32.456248+0000 (oldest deadline 2023-11-27T15:30:55.956203+0000) 2023-11-27T15:31:10.904 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:10 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:10.467+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:42.357791+0000 front 2023-11-27T15:30:42.357883+0000 (oldest deadline 2023-11-27T15:31:05.857759+0000) 2023-11-27T15:31:11.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:11 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:11.115+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:11.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:11 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:11.115+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:11.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:11 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:11.115+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:11.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:11 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:11.115+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:11.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:11 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:11.115+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:11.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:11 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:11.115+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:41.406477+0000 front 2023-11-27T15:30:41.406512+0000 (oldest deadline 2023-11-27T15:31:05.506531+0000) 2023-11-27T15:31:11.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:11 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:11.430+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:31:11.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:11 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:11.430+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:31:11.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:11 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:11.430+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:31:11.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:11 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:11.430+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:31:11.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:11 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:11.430+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:31:11.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:11 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:11.430+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:32.456188+0000 front 2023-11-27T15:30:32.456248+0000 (oldest deadline 2023-11-27T15:30:55.956203+0000) 2023-11-27T15:31:11.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:11 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:11.430+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:42.357791+0000 front 2023-11-27T15:30:42.357883+0000 (oldest deadline 2023-11-27T15:31:05.857759+0000) 2023-11-27T15:31:12.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:12 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:12.160+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:12.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:12 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:12.160+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:12.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:12 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:12.160+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:12.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:12 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:12.160+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:12.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:12 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:12.160+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:12.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:12 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:12.160+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:41.406477+0000 front 2023-11-27T15:30:41.406512+0000 (oldest deadline 2023-11-27T15:31:05.506531+0000) 2023-11-27T15:31:12.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:12 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:12.399+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:31:12.902 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:12 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:12.399+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:31:12.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:12 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:12.399+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:31:12.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:12 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:12.399+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:31:12.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:12 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:12.399+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:31:12.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:12 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:12.399+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:32.456188+0000 front 2023-11-27T15:30:32.456248+0000 (oldest deadline 2023-11-27T15:30:55.956203+0000) 2023-11-27T15:31:12.903 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:12 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:12.399+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:42.357791+0000 front 2023-11-27T15:30:42.357883+0000 (oldest deadline 2023-11-27T15:31:05.857759+0000) 2023-11-27T15:31:13.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:13 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:13.117+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:13.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:13 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:13.117+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:13.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:13 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:13.117+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:13.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:13 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:13.117+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:13.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:13 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:13.117+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:13.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:13 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:13.117+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:41.406477+0000 front 2023-11-27T15:30:41.406512+0000 (oldest deadline 2023-11-27T15:31:05.506531+0000) 2023-11-27T15:31:13.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:13 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:13.117+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6820 osd.7 since back 2023-11-27T15:30:47.207661+0000 front 2023-11-27T15:30:47.207584+0000 (oldest deadline 2023-11-27T15:31:13.107486+0000) 2023-11-27T15:31:13.652 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:13 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:13.383+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:31:13.652 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:13 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:13.383+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:31:13.652 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:13 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:13.383+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:31:13.653 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:13 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:13.383+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:31:13.653 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:13 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:13.383+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:31:13.653 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:13 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:13.383+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:32.456188+0000 front 2023-11-27T15:30:32.456248+0000 (oldest deadline 2023-11-27T15:30:55.956203+0000) 2023-11-27T15:31:13.653 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:13 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:13.383+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:42.357791+0000 front 2023-11-27T15:30:42.357883+0000 (oldest deadline 2023-11-27T15:31:05.857759+0000) 2023-11-27T15:31:14.337 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:14 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:14.080+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:14.337 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:14 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:14.080+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:14.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:14 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:14.080+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:14.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:14 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:14.080+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:14.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:14 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:14.080+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:14.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:14 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:14.080+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:41.406477+0000 front 2023-11-27T15:30:41.406512+0000 (oldest deadline 2023-11-27T15:31:05.506531+0000) 2023-11-27T15:31:14.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:14 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:14.080+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6820 osd.7 since back 2023-11-27T15:30:47.207661+0000 front 2023-11-27T15:30:47.207584+0000 (oldest deadline 2023-11-27T15:31:13.107486+0000) 2023-11-27T15:31:14.652 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:14 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:14.394+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:53.949316+0000 front 2023-11-27T15:29:53.949247+0000 (oldest deadline 2023-11-27T15:30:18.049449+0000) 2023-11-27T15:31:14.652 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:14 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:14.394+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6804 osd.1 since back 2023-11-27T15:30:02.651125+0000 front 2023-11-27T15:30:02.651150+0000 (oldest deadline 2023-11-27T15:30:25.551359+0000) 2023-11-27T15:31:14.653 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:14 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:14.394+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:11.452347+0000 front 2023-11-27T15:30:11.452364+0000 (oldest deadline 2023-11-27T15:30:33.752506+0000) 2023-11-27T15:31:14.653 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:14 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:14.394+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:17.253894+0000 front 2023-11-27T15:30:17.253900+0000 (oldest deadline 2023-11-27T15:30:42.553715+0000) 2023-11-27T15:31:14.653 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:14 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:14.394+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:24.254726+0000 front 2023-11-27T15:30:24.254679+0000 (oldest deadline 2023-11-27T15:30:52.455514+0000) 2023-11-27T15:31:14.653 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:14 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:14.394+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:32.456188+0000 front 2023-11-27T15:30:32.456248+0000 (oldest deadline 2023-11-27T15:30:55.956203+0000) 2023-11-27T15:31:14.653 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:14 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10[152659]: 2023-11-27T15:31:14.394+0000 7f26399ef700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:42.357791+0000 front 2023-11-27T15:30:42.357883+0000 (oldest deadline 2023-11-27T15:31:05.857759+0000) 2023-11-27T15:31:15.402 INFO:journalctl@ceph.osd.10.smithi174.stdout:Nov 27 15:31:15 smithi174 bash[161544]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-10 2023-11-27T15:31:15.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:15 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:15.085+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:15.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:15 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:15.085+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:15.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:15 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:15.085+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:15.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:15 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:15.085+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:15.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:15 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:15.085+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:15.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:15 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:15.085+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:41.406477+0000 front 2023-11-27T15:30:41.406512+0000 (oldest deadline 2023-11-27T15:31:05.506531+0000) 2023-11-27T15:31:15.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:15 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:15.085+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6820 osd.7 since back 2023-11-27T15:30:47.207661+0000 front 2023-11-27T15:30:47.207584+0000 (oldest deadline 2023-11-27T15:31:13.107486+0000) 2023-11-27T15:31:16.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:16 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:16.135+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:16.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:16 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:16.135+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:16.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:16 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:16.135+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:16.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:16 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:16.135+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:16.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:16 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:16.135+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:16.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:16 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:16.135+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:41.406477+0000 front 2023-11-27T15:30:41.406512+0000 (oldest deadline 2023-11-27T15:31:05.506531+0000) 2023-11-27T15:31:16.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:16 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:16.135+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6820 osd.7 since back 2023-11-27T15:30:47.207661+0000 front 2023-11-27T15:30:47.207584+0000 (oldest deadline 2023-11-27T15:31:13.107486+0000) 2023-11-27T15:31:16.724 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.10.service' 2023-11-27T15:31:16.764 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:31:16.764 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-11-27T15:31:16.764 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-11-27T15:31:16.764 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.11 2023-11-27T15:31:17.087 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:16 smithi177 systemd[1]: Stopping Ceph osd.11 for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:31:17.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:17 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:17.111+0000 7f3623656700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:31:17.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:17 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:17.111+0000 7f3623656700 -1 osd.11 98 *** Got signal Terminated *** 2023-11-27T15:31:17.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:17 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:17.111+0000 7f3623656700 -1 osd.11 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-11-27T15:31:17.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:17 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:17.112+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:17.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:17 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:17.112+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:17.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:17 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:17.112+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:17.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:17 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:17.112+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:17.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:17 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:17.112+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:17.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:17 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:17.112+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:41.406477+0000 front 2023-11-27T15:30:41.406512+0000 (oldest deadline 2023-11-27T15:31:05.506531+0000) 2023-11-27T15:31:17.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:17 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:17.112+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6820 osd.7 since back 2023-11-27T15:30:47.207661+0000 front 2023-11-27T15:30:47.207584+0000 (oldest deadline 2023-11-27T15:31:13.107486+0000) 2023-11-27T15:31:18.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:18 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:18.072+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:18.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:18 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:18.073+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:18.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:18 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:18.073+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:18.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:18 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:18.073+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:18.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:18 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:18.073+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:18.339 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:18 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:18.073+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:41.406477+0000 front 2023-11-27T15:30:41.406512+0000 (oldest deadline 2023-11-27T15:31:05.506531+0000) 2023-11-27T15:31:18.339 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:18 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:18.073+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6820 osd.7 since back 2023-11-27T15:30:47.207661+0000 front 2023-11-27T15:30:47.207584+0000 (oldest deadline 2023-11-27T15:31:13.107486+0000) 2023-11-27T15:31:19.337 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:19 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:19.038+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:19.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:19 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:19.038+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:19.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:19 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:19.038+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:19.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:19 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:19.038+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:19.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:19 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:19.038+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:19.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:19 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:19.038+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:41.406477+0000 front 2023-11-27T15:30:41.406512+0000 (oldest deadline 2023-11-27T15:31:05.506531+0000) 2023-11-27T15:31:19.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:19 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:19.038+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6820 osd.7 since back 2023-11-27T15:30:47.207661+0000 front 2023-11-27T15:30:47.207584+0000 (oldest deadline 2023-11-27T15:31:13.107486+0000) 2023-11-27T15:31:20.337 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:20 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:20.083+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:20.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:20 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:20.083+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:20.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:20 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:20.083+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:20.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:20 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:20.083+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:20.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:20 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:20.083+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:20.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:20 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:20.083+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:41.406477+0000 front 2023-11-27T15:30:41.406512+0000 (oldest deadline 2023-11-27T15:31:05.506531+0000) 2023-11-27T15:31:20.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:20 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:20.083+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6820 osd.7 since back 2023-11-27T15:30:47.207661+0000 front 2023-11-27T15:30:47.207584+0000 (oldest deadline 2023-11-27T15:31:13.107486+0000) 2023-11-27T15:31:21.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:21 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:21.065+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:21.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:21 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:21.065+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:21.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:21 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:21.065+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:21.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:21 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:21.065+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:21.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:21 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:21.065+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:21.338 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:21 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:21.065+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:41.406477+0000 front 2023-11-27T15:30:41.406512+0000 (oldest deadline 2023-11-27T15:31:05.506531+0000) 2023-11-27T15:31:21.339 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:21 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:21.065+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6820 osd.7 since back 2023-11-27T15:30:47.207661+0000 front 2023-11-27T15:30:47.207584+0000 (oldest deadline 2023-11-27T15:31:13.107486+0000) 2023-11-27T15:31:22.587 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:22 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:22.085+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6806 osd.0 since back 2023-11-27T15:29:50.797037+0000 front 2023-11-27T15:29:50.797123+0000 (oldest deadline 2023-11-27T15:30:16.697037+0000) 2023-11-27T15:31:22.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:22 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:22.085+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6804 osd.2 since back 2023-11-27T15:30:10.100218+0000 front 2023-11-27T15:30:10.100375+0000 (oldest deadline 2023-11-27T15:30:34.800444+0000) 2023-11-27T15:31:22.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:22 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:22.085+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6814 osd.3 since back 2023-11-27T15:30:15.301742+0000 front 2023-11-27T15:30:15.301809+0000 (oldest deadline 2023-11-27T15:30:40.601609+0000) 2023-11-27T15:31:22.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:22 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:22.085+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6812 osd.4 since back 2023-11-27T15:30:26.403497+0000 front 2023-11-27T15:30:26.403487+0000 (oldest deadline 2023-11-27T15:30:51.103357+0000) 2023-11-27T15:31:22.588 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:22 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:22.085+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.177:6812 osd.5 since back 2023-11-27T15:30:33.904665+0000 front 2023-11-27T15:30:33.904535+0000 (oldest deadline 2023-11-27T15:30:57.905347+0000) 2023-11-27T15:31:22.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:22 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:22.085+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.133:6822 osd.6 since back 2023-11-27T15:30:41.406477+0000 front 2023-11-27T15:30:41.406512+0000 (oldest deadline 2023-11-27T15:31:05.506531+0000) 2023-11-27T15:31:22.589 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:22 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11[153658]: 2023-11-27T15:31:22.085+0000 7f361bc44700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.174:6820 osd.7 since back 2023-11-27T15:30:47.207661+0000 front 2023-11-27T15:30:47.207584+0000 (oldest deadline 2023-11-27T15:31:13.107486+0000) 2023-11-27T15:31:23.007 INFO:journalctl@ceph.osd.11.smithi177.stdout:Nov 27 15:31:22 smithi177 bash[163241]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-osd-11 2023-11-27T15:31:24.272 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@osd.11.service' 2023-11-27T15:31:24.312 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:31:24.313 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-11-27T15:31:24.313 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-11-27T15:31:24.313 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.a 2023-11-27T15:31:24.777 INFO:journalctl@ceph.mds.a.smithi133.stdout:Nov 27 15:31:24 smithi133 systemd[1]: Stopping Ceph mds.a for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:31:25.276 INFO:journalctl@ceph.mds.a.smithi133.stdout:Nov 27 15:31:24 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-a[132447]: 2023-11-27T15:31:24.825+0000 7f1f16107700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:31:25.277 INFO:journalctl@ceph.mds.a.smithi133.stdout:Nov 27 15:31:24 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-a[132447]: 2023-11-27T15:31:24.825+0000 7f1f16107700 -1 mds.a *** got signal Terminated *** 2023-11-27T15:31:28.810 INFO:journalctl@ceph.mds.a.smithi133.stdout:Nov 27 15:31:28 smithi133 bash[259949]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-a 2023-11-27T15:31:29.111 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.a.service' 2023-11-27T15:31:29.194 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:31:29.194 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-11-27T15:31:29.195 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-11-27T15:31:29.195 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.d 2023-11-27T15:31:29.527 INFO:journalctl@ceph.mds.d.smithi133.stdout:Nov 27 15:31:29 smithi133 systemd[1]: Stopping Ceph mds.d for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:31:30.027 INFO:journalctl@ceph.mds.d.smithi133.stdout:Nov 27 15:31:29 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-d[131447]: 2023-11-27T15:31:29.699+0000 7f680bb04700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:31:30.027 INFO:journalctl@ceph.mds.d.smithi133.stdout:Nov 27 15:31:29 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-d[131447]: 2023-11-27T15:31:29.699+0000 7f680bb04700 -1 mds.d *** got signal Terminated *** 2023-11-27T15:31:31.840 INFO:journalctl@ceph.mds.d.smithi133.stdout:Nov 27 15:31:31 smithi133 bash[260089]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-d 2023-11-27T15:31:31.881 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.d.service' 2023-11-27T15:31:31.963 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:31:31.964 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-11-27T15:31:31.964 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-11-27T15:31:31.964 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.g 2023-11-27T15:31:32.277 INFO:journalctl@ceph.mds.g.smithi133.stdout:Nov 27 15:31:32 smithi133 systemd[1]: Stopping Ceph mds.g for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:31:32.777 INFO:journalctl@ceph.mds.g.smithi133.stdout:Nov 27 15:31:32 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-g[132055]: 2023-11-27T15:31:32.448+0000 7f4b8c16b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:31:32.777 INFO:journalctl@ceph.mds.g.smithi133.stdout:Nov 27 15:31:32 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-g[132055]: 2023-11-27T15:31:32.448+0000 7f4b8c16b700 -1 mds.g *** got signal Terminated *** 2023-11-27T15:31:37.026 INFO:journalctl@ceph.mds.g.smithi133.stdout:Nov 27 15:31:36 smithi133 bash[260231]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-g 2023-11-27T15:31:37.131 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.g.service' 2023-11-27T15:31:37.213 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:31:37.214 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-11-27T15:31:37.214 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-11-27T15:31:37.214 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.j 2023-11-27T15:31:37.527 INFO:journalctl@ceph.mds.j.smithi133.stdout:Nov 27 15:31:37 smithi133 systemd[1]: Stopping Ceph mds.j for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:31:38.027 INFO:journalctl@ceph.mds.j.smithi133.stdout:Nov 27 15:31:37 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-j[131059]: 2023-11-27T15:31:37.723+0000 7fb6ded92700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:31:38.027 INFO:journalctl@ceph.mds.j.smithi133.stdout:Nov 27 15:31:37 smithi133 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-j[131059]: 2023-11-27T15:31:37.723+0000 7fb6ded92700 -1 mds.j *** got signal Terminated *** 2023-11-27T15:31:40.527 INFO:journalctl@ceph.mds.j.smithi133.stdout:Nov 27 15:31:40 smithi133 bash[260378]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-j 2023-11-27T15:31:40.745 DEBUG:teuthology.orchestra.run.smithi133:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.j.service' 2023-11-27T15:31:40.826 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:31:40.826 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-11-27T15:31:40.826 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-11-27T15:31:40.826 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.b 2023-11-27T15:31:41.152 INFO:journalctl@ceph.mds.b.smithi174.stdout:Nov 27 15:31:40 smithi174 systemd[1]: Stopping Ceph mds.b for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:31:41.652 INFO:journalctl@ceph.mds.b.smithi174.stdout:Nov 27 15:31:41 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-b[156420]: 2023-11-27T15:31:41.213+0000 7fc8f91ac700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:31:41.652 INFO:journalctl@ceph.mds.b.smithi174.stdout:Nov 27 15:31:41 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-b[156420]: 2023-11-27T15:31:41.213+0000 7fc8f91ac700 -1 mds.b *** got signal Terminated *** 2023-11-27T15:31:45.652 INFO:journalctl@ceph.mds.b.smithi174.stdout:Nov 27 15:31:45 smithi174 bash[161810]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-b 2023-11-27T15:31:45.787 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.b.service' 2023-11-27T15:31:45.823 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:31:45.824 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-11-27T15:31:45.824 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-11-27T15:31:45.824 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.e 2023-11-27T15:31:46.152 INFO:journalctl@ceph.mds.e.smithi174.stdout:Nov 27 15:31:45 smithi174 systemd[1]: Stopping Ceph mds.e for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:31:46.652 INFO:journalctl@ceph.mds.e.smithi174.stdout:Nov 27 15:31:46 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-e[155743]: 2023-11-27T15:31:46.203+0000 7fdbcec53700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:31:46.726 INFO:journalctl@ceph.mds.e.smithi174.stdout:Nov 27 15:31:46 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-e[155743]: 2023-11-27T15:31:46.203+0000 7fdbcec53700 -1 mds.e *** got signal Terminated *** 2023-11-27T15:31:50.377 INFO:journalctl@ceph.mds.e.smithi174.stdout:Nov 27 15:31:50 smithi174 bash[161957]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-e 2023-11-27T15:31:50.563 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.e.service' 2023-11-27T15:31:50.599 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:31:50.599 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-11-27T15:31:50.599 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-11-27T15:31:50.599 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.h 2023-11-27T15:31:50.902 INFO:journalctl@ceph.mds.h.smithi174.stdout:Nov 27 15:31:50 smithi174 systemd[1]: Stopping Ceph mds.h for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:31:51.402 INFO:journalctl@ceph.mds.h.smithi174.stdout:Nov 27 15:31:50 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-h[156080]: 2023-11-27T15:31:50.961+0000 7f7fa9ce5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:31:51.402 INFO:journalctl@ceph.mds.h.smithi174.stdout:Nov 27 15:31:50 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-h[156080]: 2023-11-27T15:31:50.961+0000 7f7fa9ce5700 -1 mds.h *** got signal Terminated *** 2023-11-27T15:31:54.609 INFO:journalctl@ceph.mds.h.smithi174.stdout:Nov 27 15:31:54 smithi174 bash[162106]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-h 2023-11-27T15:31:54.615 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.h.service' 2023-11-27T15:31:54.651 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:31:54.651 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-11-27T15:31:54.651 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-11-27T15:31:54.651 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.k 2023-11-27T15:31:54.902 INFO:journalctl@ceph.mds.k.smithi174.stdout:Nov 27 15:31:54 smithi174 systemd[1]: Stopping Ceph mds.k for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:31:55.402 INFO:journalctl@ceph.mds.k.smithi174.stdout:Nov 27 15:31:55 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-k[155405]: 2023-11-27T15:31:55.026+0000 7f8939285700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:31:55.403 INFO:journalctl@ceph.mds.k.smithi174.stdout:Nov 27 15:31:55 smithi174 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-k[155405]: 2023-11-27T15:31:55.026+0000 7f8939285700 -1 mds.k *** got signal Terminated *** 2023-11-27T15:31:57.402 INFO:journalctl@ceph.mds.k.smithi174.stdout:Nov 27 15:31:57 smithi174 bash[162253]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-k 2023-11-27T15:31:57.443 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.k.service' 2023-11-27T15:31:57.477 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:31:57.477 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-11-27T15:31:57.478 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-11-27T15:31:57.478 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.c 2023-11-27T15:31:57.837 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:31:57 smithi177 systemd[1]: Stopping Ceph mds.c for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:31:58.337 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:31:57 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-c[160952]: 2023-11-27T15:31:57.881+0000 7fa34521e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:31:58.337 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:31:57 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-c[160952]: 2023-11-27T15:31:57.881+0000 7fa34521e700 -1 mds.c *** got signal Terminated *** 2023-11-27T15:32:02.479 INFO:journalctl@ceph.mds.c.smithi177.stdout:Nov 27 15:32:02 smithi177 bash[163512]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-c 2023-11-27T15:32:02.485 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.c.service' 2023-11-27T15:32:02.523 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:32:02.523 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-11-27T15:32:02.523 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-11-27T15:32:02.523 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.f 2023-11-27T15:32:02.837 INFO:journalctl@ceph.mds.f.smithi177.stdout:Nov 27 15:32:02 smithi177 systemd[1]: Stopping Ceph mds.f for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:32:03.337 INFO:journalctl@ceph.mds.f.smithi177.stdout:Nov 27 15:32:02 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-f[156898]: 2023-11-27T15:32:02.894+0000 7f6b813ef700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:32:03.338 INFO:journalctl@ceph.mds.f.smithi177.stdout:Nov 27 15:32:02 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-f[156898]: 2023-11-27T15:32:02.894+0000 7f6b813ef700 -1 mds.f *** got signal Terminated *** 2023-11-27T15:32:06.337 INFO:journalctl@ceph.mds.f.smithi177.stdout:Nov 27 15:32:06 smithi177 bash[163652]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-f 2023-11-27T15:32:06.456 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.f.service' 2023-11-27T15:32:06.491 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:32:06.491 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-11-27T15:32:06.491 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-11-27T15:32:06.492 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.i 2023-11-27T15:32:06.837 INFO:journalctl@ceph.mds.i.smithi177.stdout:Nov 27 15:32:06 smithi177 systemd[1]: Stopping Ceph mds.i for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:32:07.337 INFO:journalctl@ceph.mds.i.smithi177.stdout:Nov 27 15:32:06 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-i[157581]: 2023-11-27T15:32:06.868+0000 7f700d491700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:32:07.338 INFO:journalctl@ceph.mds.i.smithi177.stdout:Nov 27 15:32:06 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-i[157581]: 2023-11-27T15:32:06.868+0000 7f700d491700 -1 mds.i *** got signal Terminated *** 2023-11-27T15:32:10.337 INFO:journalctl@ceph.mds.i.smithi177.stdout:Nov 27 15:32:10 smithi177 bash[163793]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-i 2023-11-27T15:32:10.528 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.i.service' 2023-11-27T15:32:10.563 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:32:10.563 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-11-27T15:32:10.563 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-11-27T15:32:10.564 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.l 2023-11-27T15:32:10.837 INFO:journalctl@ceph.mds.l.smithi177.stdout:Nov 27 15:32:10 smithi177 systemd[1]: Stopping Ceph mds.l for 5227da4c-8d37-11ee-95a2-87774f69a715... 2023-11-27T15:32:11.337 INFO:journalctl@ceph.mds.l.smithi177.stdout:Nov 27 15:32:10 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-l[156554]: 2023-11-27T15:32:10.959+0000 7fb812955700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-11-27T15:32:11.338 INFO:journalctl@ceph.mds.l.smithi177.stdout:Nov 27 15:32:10 smithi177 ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-l[156554]: 2023-11-27T15:32:10.959+0000 7fb812955700 -1 mds.l *** got signal Terminated *** 2023-11-27T15:32:15.116 INFO:journalctl@ceph.mds.l.smithi177.stdout:Nov 27 15:32:15 smithi177 bash[163932]: ceph-5227da4c-8d37-11ee-95a2-87774f69a715-mds-l 2023-11-27T15:32:15.312 DEBUG:teuthology.orchestra.run.smithi177:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5227da4c-8d37-11ee-95a2-87774f69a715@mds.l.service' 2023-11-27T15:32:15.346 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-11-27T15:32:15.346 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-11-27T15:32:15.346 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 --force --keep-logs 2023-11-27T15:32:19.720 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 --force --keep-logs 2023-11-27T15:32:21.663 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 --force --keep-logs 2023-11-27T15:32:23.625 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:32:30.617 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:32:30.653 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-11-27T15:32:30.689 INFO:tasks.cephadm:Archiving crash dumps... 2023-11-27T15:32:30.691 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/crash to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467805/remote/smithi133/crash 2023-11-27T15:32:30.692 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/crash -- . 2023-11-27T15:32:30.780 INFO:teuthology.orchestra.run.smithi133.stderr:tar: /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/crash: Cannot open: No such file or directory 2023-11-27T15:32:30.780 INFO:teuthology.orchestra.run.smithi133.stderr:tar: Error is not recoverable: exiting now 2023-11-27T15:32:30.783 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/crash to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467805/remote/smithi174/crash 2023-11-27T15:32:30.784 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/crash -- . 2023-11-27T15:32:30.841 INFO:teuthology.orchestra.run.smithi174.stderr:tar: /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/crash: Cannot open: No such file or directory 2023-11-27T15:32:30.841 INFO:teuthology.orchestra.run.smithi174.stderr:tar: Error is not recoverable: exiting now 2023-11-27T15:32:30.843 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/crash to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467805/remote/smithi177/crash 2023-11-27T15:32:30.844 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/crash -- . 2023-11-27T15:32:30.884 INFO:teuthology.orchestra.run.smithi177.stderr:tar: /var/lib/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/crash: Cannot open: No such file or directory 2023-11-27T15:32:30.884 INFO:teuthology.orchestra.run.smithi177.stderr:tar: Error is not recoverable: exiting now 2023-11-27T15:32:30.885 INFO:tasks.cephadm:Checking cluster log for badness... 2023-11-27T15:32:30.886 DEBUG:teuthology.orchestra.run.smithi133:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-11-27T15:32:30.962 INFO:teuthology.orchestra.run.smithi133.stderr:grep: /var/log/ceph/5227da4c-8d37-11ee-95a2-87774f69a715/ceph.log: No such file or directory 2023-11-27T15:32:30.967 INFO:tasks.cephadm:Compressing logs... 2023-11-27T15:32:30.967 DEBUG:teuthology.orchestra.run.smithi133:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-27T15:32:30.972 DEBUG:teuthology.orchestra.run.smithi174:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-27T15:32:30.974 DEBUG:teuthology.orchestra.run.smithi177:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-27T15:32:31.004 INFO:teuthology.orchestra.run.smithi174.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-11-27T15:32:31.005 INFO:teuthology.orchestra.run.smithi177.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-11-27T15:32:31.076 INFO:teuthology.orchestra.run.smithi133.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-11-27T15:36:48.353 INFO:tasks.cephadm:Archiving logs... 2023-11-27T15:36:48.354 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/log/ceph to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467805/remote/smithi133/log 2023-11-27T15:36:48.356 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-27T15:36:55.639 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/log/ceph to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467805/remote/smithi174/log 2023-11-27T15:36:55.640 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-27T15:37:19.282 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/log/ceph to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467805/remote/smithi177/log 2023-11-27T15:37:19.283 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-27T15:37:35.575 INFO:tasks.cephadm:Removing cluster... 2023-11-27T15:37:35.575 DEBUG:teuthology.orchestra.run.smithi133:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 --force 2023-11-27T15:37:37.003 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 --force 2023-11-27T15:37:38.071 DEBUG:teuthology.orchestra.run.smithi177:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5227da4c-8d37-11ee-95a2-87774f69a715 --force 2023-11-27T15:37:39.071 INFO:tasks.cephadm:Removing cephadm ... 2023-11-27T15:37:39.071 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf /home/ubuntu/cephtest/cephadm 2023-11-27T15:37:39.120 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf /home/ubuntu/cephtest/cephadm 2023-11-27T15:37:39.142 DEBUG:teuthology.orchestra.run.smithi177:> rm -rf /home/ubuntu/cephtest/cephadm 2023-11-27T15:37:39.160 INFO:tasks.cephadm:Teardown complete 2023-11-27T15:37:39.160 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-27T15:37:39.172 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-27T15:37:39.173 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-27T15:37:39.178 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-27T15:37:39.185 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-27T15:37:39.337 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-27T15:37:39.347 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-27T15:37:39.348 DEBUG:teuthology.orchestra.run.smithi133:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-27T15:37:39.352 DEBUG:teuthology.orchestra.run.smithi174:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-27T15:37:39.355 DEBUG:teuthology.orchestra.run.smithi177:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-27T15:37:39.371 INFO:teuthology.orchestra.run.smithi177.stderr:bash: ntpq: command not found 2023-11-27T15:37:39.372 INFO:teuthology.orchestra.run.smithi174.stderr:bash: ntpq: command not found 2023-11-27T15:37:39.376 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-27T15:37:39.376 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2023-11-27T15:37:39.376 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 43 -573us[ -565us] +/- 94ms 2023-11-27T15:37:39.376 INFO:teuthology.orchestra.run.smithi177.stdout:^* hv02.front.sepia.ceph.com 3 6 377 42 -2910us[-2902us] +/- 62ms 2023-11-27T15:37:39.376 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 +5354us[+5354us] +/- 72ms 2023-11-27T15:37:39.376 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:37:39.377 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-27T15:37:39.378 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2023-11-27T15:37:39.378 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 40 -959us[ -959us] +/- 94ms 2023-11-27T15:37:39.378 INFO:teuthology.orchestra.run.smithi174.stdout:^* hv02.front.sepia.ceph.com 3 6 377 40 -3290us[-2908us] +/- 62ms 2023-11-27T15:37:39.378 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +4952us[+5334us] +/- 72ms 2023-11-27T15:37:39.378 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:37:39.390 INFO:teuthology.orchestra.run.smithi133.stderr:bash: ntpq: command not found 2023-11-27T15:37:39.398 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-27T15:37:39.398 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2023-11-27T15:37:39.399 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 39 -574us[ -522us] +/- 94ms 2023-11-27T15:37:39.399 INFO:teuthology.orchestra.run.smithi133.stdout:^* hv02.front.sepia.ceph.com 3 6 377 39 -2919us[-2867us] +/- 62ms 2023-11-27T15:37:39.399 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 +5318us[+5318us] +/- 72ms 2023-11-27T15:37:39.399 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-11-27T15:37:39.400 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-27T15:37:39.410 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-27T15:37:39.411 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-27T15:37:39.421 DEBUG:teuthology.orchestra.run.smithi133:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-27T15:37:39.735 DEBUG:teuthology.orchestra.run.smithi174:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-27T15:37:39.838 DEBUG:teuthology.orchestra.run.smithi177:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-27T15:37:39.935 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-27T15:37:40.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:37:40.013 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-27T15:37:40.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:37:40.048 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-27T15:37:40.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:37:40.080 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-27T15:37:40.091 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-27T15:37:40.101 INFO:teuthology.task.internal:Duration was 2565.674108 seconds 2023-11-27T15:37:40.102 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-27T15:37:40.112 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-27T15:37:40.112 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-27T15:37:40.116 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-27T15:37:40.119 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-27T15:37:40.165 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-27T15:37:40.167 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-27T15:37:40.272 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-27T15:37:40.878 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-27T15:37:40.878 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi133.front.sepia.ceph.com 2023-11-27T15:37:40.878 DEBUG:teuthology.orchestra.run.smithi133:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-27T15:37:41.009 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi174.front.sepia.ceph.com 2023-11-27T15:37:41.009 DEBUG:teuthology.orchestra.run.smithi174:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-27T15:37:41.035 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi177.front.sepia.ceph.com 2023-11-27T15:37:41.036 DEBUG:teuthology.orchestra.run.smithi177:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-27T15:37:41.062 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-27T15:37:41.062 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-27T15:37:41.067 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-27T15:37:41.078 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-27T15:37:41.204 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-27T15:37:41.204 DEBUG:teuthology.orchestra.run.smithi133:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-27T15:37:41.209 DEBUG:teuthology.orchestra.run.smithi174:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-27T15:37:41.212 DEBUG:teuthology.orchestra.run.smithi177:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-27T15:37:41.383 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-27T15:37:41.395 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-27T15:37:41.395 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-27T15:37:41.477 DEBUG:teuthology.orchestra.run.smithi174:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-27T15:37:41.504 DEBUG:teuthology.orchestra.run.smithi177:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-27T15:37:41.532 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-27T15:37:41.544 DEBUG:teuthology.orchestra.run.smithi133:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-27T15:37:41.549 DEBUG:teuthology.orchestra.run.smithi174:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-27T15:37:41.552 DEBUG:teuthology.orchestra.run.smithi177:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-27T15:37:41.575 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = core 2023-11-27T15:37:41.598 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = core 2023-11-27T15:37:41.625 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = core 2023-11-27T15:37:41.675 DEBUG:teuthology.orchestra.run.smithi133:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-27T15:37:41.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:37:41.717 DEBUG:teuthology.orchestra.run.smithi174:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-27T15:37:41.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:37:41.734 DEBUG:teuthology.orchestra.run.smithi177:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-27T15:37:41.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-27T15:37:41.750 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-27T15:37:41.762 INFO:teuthology.task.internal:Transferring archived files... 2023-11-27T15:37:41.763 DEBUG:teuthology.misc:Transferring archived files from smithi133:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467805/remote/smithi133 2023-11-27T15:37:41.763 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-27T15:37:41.845 DEBUG:teuthology.misc:Transferring archived files from smithi174:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467805/remote/smithi174 2023-11-27T15:37:41.845 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-27T15:37:41.884 DEBUG:teuthology.misc:Transferring archived files from smithi177:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467805/remote/smithi177 2023-11-27T15:37:41.884 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-27T15:37:41.923 INFO:teuthology.task.internal:Removing archive directory... 2023-11-27T15:37:41.923 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-27T15:37:41.927 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-27T15:37:41.930 DEBUG:teuthology.orchestra.run.smithi177:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-27T15:37:41.973 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-27T15:37:41.985 INFO:teuthology.task.internal:Not uploading archives. 2023-11-27T15:37:41.986 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-27T15:37:41.996 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-27T15:37:41.996 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-27T15:37:42.011 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-27T15:37:42.013 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-27T15:37:42.027 INFO:teuthology.orchestra.run.smithi174.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 27 15:37 /home/ubuntu/cephtest 2023-11-27T15:37:42.029 INFO:teuthology.orchestra.run.smithi177.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 27 15:37 /home/ubuntu/cephtest 2023-11-27T15:37:42.050 INFO:teuthology.orchestra.run.smithi133.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 27 15:37 /home/ubuntu/cephtest 2023-11-27T15:37:42.053 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-27T15:37:42.064 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-27T15:37:42.086 INFO:teuthology.nuke:Checking targets against current locks 2023-11-27T15:37:42.112 DEBUG:teuthology.nuke:shortname: smithi133 2023-11-27T15:37:42.112 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-27T15:37:42.133 DEBUG:teuthology.nuke:shortname: smithi174 2023-11-27T15:37:42.134 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-27T15:37:42.138 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467805', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-27 14:46:36.840520', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-27T15:37:42.155 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467805', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-27 14:46:36.836700', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-27T15:37:42.158 DEBUG:teuthology.nuke:shortname: smithi177 2023-11-27T15:37:42.158 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-27T15:37:42.178 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-11-27_14:43:37-fs-wip-choffman-62793-distro-default-smithi/7467805', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-27 14:46:36.838604', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-27T15:37:42.188 INFO:teuthology.orchestra.console.smithi133:Power off 2023-11-27T15:37:42.189 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-27T15:37:42.220 INFO:teuthology.orchestra.console.smithi174:Power off 2023-11-27T15:37:42.220 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-27T15:37:42.232 INFO:teuthology.orchestra.console.smithi177:Power off 2023-11-27T15:37:42.232 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-27T15:37:42.278 DEBUG:teuthology.orchestra.console.smithi177:power off output: Chassis Power Control: Down/Off 2023-11-27T15:37:42.279 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:37:42.284 DEBUG:teuthology.orchestra.console.smithi133:power off output: Chassis Power Control: Down/Off 2023-11-27T15:37:42.285 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:37:42.292 DEBUG:teuthology.orchestra.console.smithi174:power off output: Chassis Power Control: Down/Off 2023-11-27T15:37:42.292 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:37:42.301 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2023-11-27T15:37:42.307 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2023-11-27T15:37:42.337 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2023-11-27T15:37:46.302 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:37:46.310 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:37:46.337 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:37:46.413 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2023-11-27T15:37:46.419 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2023-11-27T15:37:46.445 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2023-11-27T15:37:50.418 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:37:50.427 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:37:50.446 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:37:50.530 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is off 2023-11-27T15:37:50.540 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is on 2023-11-27T15:37:50.559 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off 2023-11-27T15:37:50.631 INFO:teuthology.orchestra.console.smithi177:Power off completed 2023-11-27T15:37:50.660 INFO:teuthology.orchestra.console.smithi174:Power off completed 2023-11-27T15:37:54.545 DEBUG:teuthology.orchestra.console.smithi133:pexpect command: ipmitool -H smithi133.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-27T15:37:54.658 DEBUG:teuthology.orchestra.console.smithi133:check power output: Chassis Power is off 2023-11-27T15:37:54.758 INFO:teuthology.orchestra.console.smithi133:Power off completed 2023-11-27T15:37:54.865 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-workunit/fs/misc}} duration: 2565.6741075515747 failure_reason: 'Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi133 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5e4a743ad7fd1422ebd6c4dc950a21a303279207 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh''' flavor: default owner: scheduled_choffman@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=6d08514d206b4b3bafe35b4f1ce9f14b status: fail success: false 2023-11-27T15:37:54.865 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-27T15:37:54.971 INFO:teuthology.run:FAIL